Email is still the single most used communications tool on the internet and will probably remain so for some time to come. We send in the region of 300 Billion emails per day and for the most part, everything just works. On the face of it, email is simple: compose, address and send. But behind the scenes, the infrastructure that makes email work is not so simple ( or reliable ) as we’d like to think. Here follows a quick ( and hopefully reasonably simple ) overview of how email gets from a sender to a recipient.
The quick overview
sender -> sender MTA -> recipient MTA -> recipient
MTA = mail transfer agent
MUA = email client ( like Outlook, Thunderbird, etc. )
In very simple terms, one composes an email with an email client ( MUA ), addresses it to one or more recipients and then sends it through the local email server ( MTA ). The sender’s local email server connects to the recipient’s email server ( MTA ), and transfers the email to the recipient’s mail store. The recipient’s email client ( MUA ) will then poll the mail store on a regular basis and retrieve any emails waiting there.
But there is actually a whole lot more that happens in between the sender and recipient.
The in-depth story – sender side
sender -> sender mta [ av check, content checks, relay check ] -> relay
There’s quite a few steps that occur before an email even leaves your email server. Here follows some of them:
- av check: your email will normally be anti-virus checked before leaving the network to make sure that viruses are not propogated
- content check: the email content ( including body and subject line ) will be checked against a static list for content that should be blocked
- relay check: your network location is checked to see if it is allowed to use the specified email server to send email; we don’t want just anyone using our email server to send email otherwise they could use it to send spam ( also called an open relay )
- relay: once an email is ready to be sent, it may be routed through one or more relays or intermediate email servers
Relays / Smarthosts
A relay is simply an intermediate email server whose sole job is to route your email to its destination. Most ISP’s will make a relay available to their clients to use as a sending email server ( also called smtp or outgoing server ). More specifically, users who are on 3G, dialup or dynamic IP addresses will need to use a relay. For users who have their own in-house email server, this will typically be configured on the email server itself so nothing further needs to be done by individual users.
However, even if you have your own in-house email server, when traveling, you could have issues sending email for a number of reasons:
- the ISP you are currently connected to may not allow you to use email servers other than their own
- your in-house email server may not allow you to connect from external networks or may require you to authenticate with a username and password before sending email
You can speak to your email administrator for more information on configuring your email client when out on the road.
The in-depth story – recipient side
relay -> recipient mta [ RBL checks, greylisting, address checks, anti-spam checks, av check, content checks ] -> recipient
- RBL checks: real-time blackhole lists are lists of spamming servers that are maintained on the internet by a number of commercial and non-commercial companies; when an email is received, the sending server’s IP address is checked against these lists – if it matches then the sending server is considered a spammer and the email is dropped or bounced
- greylisting: when the recipient server receives an email from the combination of a sender and a sending server that it has never seen before, it temporarily rejects the email; the sending server will then resend the email a short whole later; the reason this process stops spam is because spammers are typically not setup to receive the temporary reject notice and thus will never resend the original email
- address checks: the recipient server will check that the sending server has a valid DNS hostname ( eg. mail.server.com ) and also that the hostname maps back to the IP address that is presented by the sending server ( reverse DNS check )
- anti-spam checks: these include intelligent ( heuristic ) checking of email and scoring of a variety of parameters of the email; if the total score exceeds a preset threshold value, then the email is considered to be spam
- anti-virus check: self explanatory?
- content checks: the subject line and body of the email are checked against a preset list of content ( eg. words or phrases ) – if matched, then the email is rejected or dropped
All of the above checks are there to reduce the amount of spam ( non-solicited ) emails that are received. Considering the amount of spam that is generated and sent each day ( ~ 90% of all email ), any opportunity to stop spam email is a good thing. Unfortunately, sometimes even valid email can be blocked by recipient servers.
When an email is blocked, an error message is typically sent back to the sender so that they ( or their email administrator ) can take some action that may result in future emails to the recipient being successfully accepted.
An example of some block messages
You’ll notice that when you receive an error message for an email that was not successfully delivered, there is normally a code associated with that error message like 550 or 554. These are SMTP error codes and provide a standard way of understanding email routing and other errors.
Mailbox is full
Well this one is self-explanatory – the mail store of the user you sent email to is full, they need to delete some email before they can receive more
Message exceeds size limit
The email you sent is larger than the recipient server is willing to accept; the size that can be received varies from server to server but you should try to keep your email size below 7MB
User unknown/invalid recipient/mailbox unavailable
The email address is wrong or does not exist
Please turn on SMTP Authentication in your mail client, or login to the IMAP/POP3 server before sending your message. (mail.server.com) is not permitted to relay through this server without authentication
In this case, the relay or outbound server requires your username and login to send email
blocked using zen.spamhaus.org; http://www.spamhaus.org/query/bl?ip=71.230.109.142
A server trying to deliver an email to us was regarded as a spam server by an RBL and blocked; in this case, the recipient server administrator needs to request the de-listing of their domain/address from the specific RBL list
Your access to this mail system has been rejected due to the sending MTA’s poor reputation. If you believe that this failure is in error, please contact the intended recipient via alternate means.
Similar to the example above, our email to someone else was blocked because the recipient server regarded us as being a spammer
Recipient address rejected: Policy Rejection- Please try later
An end user will probably not see this message directly but it’s an example of greylisting
Could not send your message. Error 421
This one is very cryptic because it only gives an error code but no real explanation of what went wrong. In this case, error 421 means the service was not available; more specifically it could mean that the recipient server had an issue and could not accept the email.
Sender address rejected: relay access denied
Quite a common one, this means that either you need to authenticate or you are not allowed to use this relay at all
Reading an error message carefully ( ignoring any complex addressing or other information ) can give you a clue as to why your email was rejected/bounced. Make the necessary changes ( or ask your email administrator ) and resend your email.
RBLs ( realtime blackhole lists )
The address of incoming emails is checked against RBLs to see if they are listed as spammers. RBLs, along with greylisting, is the biggest defense we have against spam email. However, when they fail to work correctly they can cause many issues. An RBL went off-line recently and as a result, any email server checking against this list, would block all incoming email.
Conclusion
While many assume that email is a guaranteed service, this is not the case; email can be blocked or dropped for any number of reasons including network errors, email server misconfiguration and anti-spam reasons. Some anti-spam measures can have a negative impact on the receipt of emails even when these are expected to be delivered.
Email can be made to work reasonably efficient and reliably but it requires correct configuration of both sides of the link, and anything in between, to function well.
For more information on using email efficiently, please see my article Email woes and etiquette