Email Authentication – SPF, DKIM, DMARC

Email imageIntroduction

When email was first designed, security was not considered important and up until fairly recently it was still possible to send an email from any address and get away with it.

However, as spam, phishing and spoofing attacks by email have become increasingly common there have been various attempts to make email more secure.  In the last year or so the major providers (AOL, Google, Microsoft etc.) have all seriously tightened their security and authentication requirements for validating and receiving email.  The result of this is that a lot of legitimate email is now being classified as spam or rejected by those providers.  In order to ensure that your email continues to be marked as legitimate and received by these larger providers it is now almost essential that you implement SPF, DKIM and DMARC on your domains otherwise many of your recipients will never see the emails you are sending!

So, what on earth is SPF, DKIM and DMARC?

Sender Policy Framework (SPF)

Quoted from Wikipedia: “Sender Policy Framework (SPF) is a simple email-validation system designed to detect email spoofing by providing a mechanism to allow receiving mail exchangers to check that incoming mail from a domain comes from a host authorized by that domain’s administrators.”

Basically, this tells the receiving mail server which servers are allowed to send email for your domain.

When you implement this you specify a list of hosts/ip addresses which can send email for your domain.  This has a couple of things to watch out for:

  • Forwarders: If you use email forwarders you have to configure the forwarding server to rewrite email headers so it is not using your from address (this is called Sender Rewriting Scheme – SRS).
  • Sending via your Internet Service Provider (ISP) email server or another SMTP server that you control:  You can’t.  Well, not unless you specifically add it to the SPF record which means you need to have control over the DNS for the domain you are using.
  • Sending via a mailing service (eg. mailchimp) or large provider (eg google apps for business – gsuite): You need to find out what their process is for validating domains and they will need to provide a domain that maps to ALL the IP addresses/hosts that they can use to send email.  Most now do this but it can be difficult finding the information with some services.

Domain Keys Identified Mail (DKIM)

Quoted from Wikipedia: “DomainKeys Identified Mail (DKIM) is an email authentication method designed to detect email spoofing. It allows the receiver to check that an email claimed to come from a specific domain was indeed authorized by the owner of that domain.[1] It is intended to prevent forged sender addresses in emails, a technique often used in phishing and email spam.”

The sending mail server adds a signature to the email which can be verified by the receiving mail server using a public key that is accessed via a DNS record on your domain.  This means a receiving mail server can verify if the email has been changed in any way from when it was originally sent.

This is generally a good thing, but again has a couple of caveats:

  • An intermediate mail server can not add any headers to the email (eg. spam score, routing information) without breaking the signature.
  • Emails with a DKIM signature cannot be forwarded onto mailing lists and other similar things without the forwarding mail server rewriting the sender as it’s own and adding it’s own DKIM signature – so the email would now come from the mailing list rather than your own address.

Domain-based Message Authentication, Reporting and Conformance (DMARC)

Quoted from Wikipedia: “DMARC is built on top of two existing mechanisms, Sender Policy Framework (SPF) and DomainKeys Identified Mail (DKIM). It allows the administrative owner of a domain to publish a policy on which mechanism (DKIM, SPF or both) is employed when sending email from that domain and how the receiver should deal with failures. Additionally, it provides a reporting mechanism of actions performed under those policies. It thus coordinates the results of DKIM and SPF and specifies under which circumstances the From: header field, which is often visible to end users, should be considered legitimate.”

This is the final piece in the puzzle and was not required until recently.  Around November 2016 I started seeing rejections and bounces because of missing DMARC policy records from the large providers.  As a result I now recommend that you must implement all three.

DMARC specifies what the receiving mail server should do when it receives an email that fails SPF/DKIM policies and allows you to specify an email address to receive reject information.  If there is no DMARC policy defined for your domain it is likely that the receiving provider will now reject anything that fails SPF/DKIM but you may not know about it.

Implementation

To implement SPF/DKIM and DMARC you must have full control over the DNS records for your domain as these are all configured using DNS records.

SPF

SPF is fairly simple.  Find a list of IP addresses and domains that are authorised to send your email and specify them in a DNS TXT record.

Full details of the syntax can be found at openspf.org.

An example record might look like this:

v=spf1 +a +mx +ip4:10.20.30.40 +include:_spf.google.com -all

Here, this specifies that email for your domain may be sent by IP address 10.20.30.40 and any hosts/IP addresses listed in the DNS record _spf.google.com

It is configured as a TXT record for the domain name (so if your email is bob@example.org you create a TXT record for example.org).

If you use cpanel just go to Email->Authentication and follow the instructions there.  You can customise the record and it will install and validate it for you.

DKIM

Enabling DKIM requires generation and setup of signing keys on your mail server.  A useful guide can be found here.  I have only had a need to setup DKIM on WHM/cpanel servers where all the setup and configuration is handled for you.  All you need to do in cpanel is enable DKIM under Email->Authentication.  This will create a DNS TXT record with the name _domainkey.example.org:

v=DKIM1; k=rsa; p=publickeygoeshere

If you are sending email using other providers (eg. Google Apps) you need to use their admin interface to enable DKIM and generate the correct DNS TXT record and key.  For example:

  • Google requires you to create DNS TXT record with google._domainkey.example.org.
  • Mailchimp requires a DNS CNAME record with k1._domainkey.example.org with this value: dkim.mcsv.net.

Warning: Any web-services you are hosting that send email will need to ensure that they send email through your SMTP server rather than via any local mail functions otherwise those emails will not receive a DKIM signature and the receiving mail server will reject them.

DMARC

This is quite easy to implement.  It’s just another DNS TXT record with the prefix _dmarc.

An example might look like this:

v=DMARC1; p=reject; sp=reject; adkim=r; aspf=r; fo=1; rf=afrf; pct=100; ruf=mailto:postmaster@example.org; ri=86400

and should be configured as a DNS TXT record named: _dmarc.example.org

In this example, we are telling the receiving mailserver to reject 100% of failures, and send a “forensic” report to the email address postmaster@example.org.

Note: It is possible to setup “aggregate” reports by email using the “rua=” tag but this will result in large amounts of data being sent as it reports on every pass/fail for receiving mailservers that generate those reports, and then you need to parse them.  So, I don’t advise activating this unless you have a service to analyse those reports or are concerned that your domain is being used for spam.

Unfortunately DMARC is not yet supported by cpanel servers (Jan 2017, version 60) but hopefully will be soon, making the configuration even easier.  For now you just have to use the zone edit tools to create your DNS entry.

Note: Ensure that you have both SPF and DKIM implemented and working before putting the DMARC policy in place!

MailMan Lists

In the admin interface you need to set the following parameter to Munge From:

Replace the From: header address with the list’s posting address to mitigate issues stemming from the original From: domain’s DMARC or similar policies

If that setting is not available, you need to upgrade mailman or you will get multiple rejections and damage your domain reputation.

Also ensure that Automatic Bounce Processing is enabled and has sensible defaults – in particular, make sure that subscriptions are disabled after a few bounces.

(Note: Wrap Message is the ideal setting as it doesn’t change the original email but that causes problems with some mail clients which causes them to display the message as an attachment, so Munge From is the best compromise).

WHM / cPanel

Make sure you enable Sender Rewriting Scheme (SRS) if you have any email forwarders configured.  Also enable DKIM and sender verification on inbound email.

This is done via the Exim Configuration Manager in WHM

  • Allow DKIM verification for incoming messages: → ON
  • Reject DKIM failures:→ ON
  • Sender Verification: → ON
  • Enable Sender Rewriting Scheme (SRS) Support: → ON

Ideally enable this one too, but you need to make sure all your clients are able to use a secure connection first:

  • Require clients to connect with SSL or issue the STARTTLS command before they are allowed to authenticate with the server. → ON

Checking everything

Don’t forget DNS can take up to 48-72hours to propagate around the world so you may need to wait before checking everything is fine.

There are lot’s of services available online for checking that you have everything configured correctly:

  • mxtoolbox.com: Runs multiple checks against your domain.  Be warned, you don’t need to worry about some of the warnings that it reports (eg. reverse DNS will never match your domain if you are on a shared server, many servers are configured to delay responses for unknown mail servers which causes mxtoolbox to report a slow (15sec+) response time).
  • https://www.mail-tester.com/ Lets you know about spamlists, DNS records and all that jazz.
  • dmarcian.com: Has tools for checking DMARC, SPF etc.
  • https://dmarcguide.globalcyberalliance.org/#/ – Wizard to help you create a dmarc record.
  • AOL Postmaster: Useful for diagnosing and resolving issues with delivery to aol addresses.
  • Google Postmaster: Useful for diagnosing and resolving issues with delivery to google addresses.
  • Hotmail/Outlook/Live Postmaster: Useful for diagnosing and resolving issues with delivery to microsoft addresses.

9 thoughts on “Email Authentication – SPF, DKIM, DMARC”

  1. Useful! I’ve not see all this put together in one place – thanks. I use mail-tester.com but of course there are quite a few.

  2. Matt,

    Thanks for this; it is timely as we’re seeing problems from several mail services. You’ve given me the tools I need to go after them. Nicely organized. Any chance of putting this material in the new CiviCRM doc tool called MKDocs? That way others could contribute examples of successful resolution of various email issues (which I’d do when I’ve worked through the issues I’m seeing). Again, thanks.

    1. Yes I probably should update the civicrm docs… There’s a few other bits of the puzzle that ought to be clarified in the civicrm docs too. Now to see when I find the time 🙂

  3. Great Article Matt!

    We implement SPF and DKIM policies and DNS records but you’ve done a good job explaining this here. Look forwards to seeing you soon.

    Jon-man

  4. I have found email authentication to be a big deal these days and it is an essential subject to know about. This is a good article which succeeds in bringing together many of the underlying concepts.

  5. A comprehensive and very useful article, thanks for this!
    However, seems to me the aggregate and forensic reports are used the other way around, in the DMARC example and note underneath…by the way- have you activated the forensic reports? If so, receiving any reports?

    1. Thanks. Pretty sure they work as I describe. Aggregate reports generate responses every time an email is delivered (if the receiving server supports DMARC) and the advice is not to use those unless you can process them by machine! The forensic reports that I do enable are very useful and I do occasionally receive reports – however, with everything properly configured you’ll find very little abuse on your domain so you’ll get very few forensic reports!

Leave a Reply

Your e-mail address will not be published. Required fields are marked *