Email Authentication – SPF, DKIM, DMARC

Email imageIntroduction

When email was first designed, security was not considered important and up until fairly recently it was still possible to send an email from any address and get away with it.

However, as spam, phishing and spoofing attacks by email have become increasingly common there have been various attempts to make email more secure.  In the last year or so the major providers (AOL, Google, Microsoft etc.) have all seriously tightened their security and authentication requirements for validating and receiving email.  The result of this is that a lot of legitimate email is now being classified as spam or rejected by those providers.  In order to ensure that your email continues to be marked as legitimate and received by these larger providers it is now almost essential that you implement SPF, DKIM and DMARC on your domains otherwise many of your recipients will never see the emails you are sending!

So, what on earth is SPF, DKIM and DMARC?

Sender Policy Framework (SPF)

Quoted from Wikipedia: “Sender Policy Framework (SPF) is a simple email-validation system designed to detect email spoofing by providing a mechanism to allow receiving mail exchangers to check that incoming mail from a domain comes from a host authorized by that domain’s administrators.”

Basically, this tells the receiving mail server which servers are allowed to send email for your domain.

When you implement this you specify a list of hosts/ip addresses which can send email for your domain.  This has a couple of things to watch out for:

  • Forwarders: If you use email forwarders you have to configure the forwarding server to rewrite email headers so it is not using your from address (this is called Sender Rewriting Scheme – SRS).
  • Sending via your Internet Service Provider (ISP) email server or another SMTP server that you control:  You can’t.  Well, not unless you specifically add it to the SPF record which means you need to have control over the DNS for the domain you are using.
  • Sending via a mailing service (eg. mailchimp) or large provider (eg google apps for business – gsuite): You need to find out what their process is for validating domains and they will need to provide a domain that maps to ALL the IP addresses/hosts that they can use to send email.  Most now do this but it can be difficult finding the information with some services.

Domain Keys Identified Mail (DKIM)

Quoted from Wikipedia: “DomainKeys Identified Mail (DKIM) is an email authentication method designed to detect email spoofing. It allows the receiver to check that an email claimed to come from a specific domain was indeed authorized by the owner of that domain.[1] It is intended to prevent forged sender addresses in emails, a technique often used in phishing and email spam.”

The sending mail server adds a signature to the email which can be verified by the receiving mail server using a public key that is accessed via a DNS record on your domain.  This means a receiving mail server can verify if the email has been changed in any way from when it was originally sent.

This is generally a good thing, but again has a couple of caveats:

  • An intermediate mail server can not add any headers to the email (eg. spam score, routing information) without breaking the signature.
  • Emails with a DKIM signature cannot be forwarded onto mailing lists and other similar things without the forwarding mail server rewriting the sender as it’s own and adding it’s own DKIM signature – so the email would now come from the mailing list rather than your own address.

Domain-based Message Authentication, Reporting and Conformance (DMARC)

Quoted from Wikipedia: “DMARC is built on top of two existing mechanisms, Sender Policy Framework (SPF) and DomainKeys Identified Mail (DKIM). It allows the administrative owner of a domain to publish a policy on which mechanism (DKIM, SPF or both) is employed when sending email from that domain and how the receiver should deal with failures. Additionally, it provides a reporting mechanism of actions performed under those policies. It thus coordinates the results of DKIM and SPF and specifies under which circumstances the From: header field, which is often visible to end users, should be considered legitimate.”

This is the final piece in the puzzle and was not required until recently.  Around November 2016 I started seeing rejections and bounces because of missing DMARC policy records from the large providers.  As a result I now recommend that you must implement all three.

DMARC specifies what the receiving mail server should do when it receives an email that fails SPF/DKIM policies and allows you to specify an email address to receive reject information.  If there is no DMARC policy defined for your domain it is likely that the receiving provider will now reject anything that fails SPF/DKIM but you may not know about it.

Implementation

To implement SPF/DKIM and DMARC you must have full control over the DNS records for your domain as these are all configured using DNS records.

SPF

SPF is fairly simple.  Find a list of IP addresses and domains that are authorised to send your email and specify them in a DNS TXT record.

Full details of the syntax can be found at openspf.org.

An example record might look like this:

v=spf1 +a +mx +ip4:10.20.30.40 +include:_spf.google.com -all

Here, this specifies that email for your domain may be sent by IP address 10.20.30.40 and any hosts/IP addresses listed in the DNS record _spf.google.com

It is configured as a TXT record for the domain name (so if your email is bob@example.org you create a TXT record for example.org).

If you use cpanel just go to Email->Authentication and follow the instructions there.  You can customise the record and it will install and validate it for you.

DKIM

Enabling DKIM requires generation and setup of signing keys on your mail server.  A useful guide can be found here.  I have only had a need to setup DKIM on WHM/cpanel servers where all the setup and configuration is handled for you.  All you need to do in cpanel is enable DKIM under Email->Authentication.  This will create a DNS TXT record with the name _domainkey.example.org:

v=DKIM1; k=rsa; p=publickeygoeshere

If you are sending email using other providers (eg. Google Apps) you need to use their admin interface to enable DKIM and generate the correct DNS TXT record and key.  For example:

  • Google requires you to create DNS TXT record with google._domainkey.example.org.
  • Mailchimp requires a DNS CNAME record with k1._domainkey.example.org with this value: dkim.mcsv.net.

Warning: Any web-services you are hosting that send email will need to ensure that they send email through your SMTP server rather than via any local mail functions otherwise those emails will not receive a DKIM signature and the receiving mail server will reject them.

DMARC

This is quite easy to implement.  It’s just another DNS TXT record with the prefix _dmarc.

An example might look like this:

v=DMARC1; p=reject; sp=reject; adkim=r; aspf=r; fo=1; rf=afrf; pct=100; ruf=mailto:postmaster@example.org; ri=86400

and should be configured as a DNS TXT record named: _dmarc.example.org

In this example, we are telling the receiving mailserver to reject 100% of failures, and send a “forensic” report to the email address postmaster@example.org.

Note: It is possible to setup “aggregate” reports by email using the “rua=” tag but this will result in large amounts of data being sent as it reports on every pass/fail for receiving mailservers that generate those reports, and then you need to parse them.  So, I don’t advise activating this unless you have a service to analyse those reports or are concerned that your domain is being used for spam.

Unfortunately DMARC is not yet supported by cpanel servers (Jan 2017, version 60) but hopefully will be soon, making the configuration even easier.  For now you just have to use the zone edit tools to create your DNS entry.

Note: Ensure that you have both SPF and DKIM implemented and working before putting the DMARC policy in place!

MailMan Lists

In the admin interface you need to set the following parameter to Munge From:

Replace the From: header address with the list’s posting address to mitigate issues stemming from the original From: domain’s DMARC or similar policies

If that setting is not available, you need to upgrade mailman or you will get multiple rejections and damage your domain reputation.

Also ensure that Automatic Bounce Processing is enabled and has sensible defaults – in particular, make sure that subscriptions are disabled after a few bounces.

(Note: Wrap Message is the ideal setting as it doesn’t change the original email but that causes problems with some mail clients which causes them to display the message as an attachment, so Munge From is the best compromise).

WHM / cPanel

Make sure you enable Sender Rewriting Scheme (SRS) if you have any email forwarders configured.  Also enable DKIM and sender verification on inbound email.

This is done via the Exim Configuration Manager in WHM

  • Allow DKIM verification for incoming messages: → ON
  • Reject DKIM failures:→ ON
  • Sender Verification: → ON
  • Enable Sender Rewriting Scheme (SRS) Support: → ON

Ideally enable this one too, but you need to make sure all your clients are able to use a secure connection first:

  • Require clients to connect with SSL or issue the STARTTLS command before they are allowed to authenticate with the server. → ON

Checking everything

Don’t forget DNS can take up to 48-72hours to propagate around the world so you may need to wait before checking everything is fine.

There are lot’s of services available online for checking that you have everything configured correctly:

  • mxtoolbox.com: Runs multiple checks against your domain.  Be warned, you don’t need to worry about some of the warnings that it reports (eg. reverse DNS will never match your domain if you are on a shared server, many servers are configured to delay responses for unknown mail servers which causes mxtoolbox to report a slow (15sec+) response time).
  • https://www.mail-tester.com/ Lets you know about spamlists, DNS records and all that jazz.
  • dmarcian.com: Has tools for checking DMARC, SPF etc.
  • https://dmarcguide.globalcyberalliance.org/#/ – Wizard to help you create a dmarc record.
  • AOL Postmaster: Useful for diagnosing and resolving issues with delivery to aol addresses.
  • Google Postmaster: Useful for diagnosing and resolving issues with delivery to google addresses.
  • Hotmail/Outlook/Live Postmaster: Useful for diagnosing and resolving issues with delivery to microsoft addresses.

AddHandler php with easyapache 4

I’ve recently been upgrading WHM/Cpanel web servers to use easyapache 4.  Amongst many benefits are ease of updates (it now uses rpms) and multiple php versions selectable by clients using multi php manager.

One client reported an issue as they are using an htaccess directive to parse html files as php.  Previously they were using:

AddHandler application/x-httpd-php5 .html

But this was no longer working…

After some google searching it turns out that handlers now use an easyapache specific format:

AddType application/x-httpd-ea-php54 .html

In cpanel, multiphp manager will add the following for force a specific
version of php:

# php -- BEGIN cPanel-generated handler, do not edit
# Set the “ea-php54” package as the default “PHP” programming language.
<IfModule mime_module>
  AddType application/x-httpd-ea-php54 .php .php5 .phtml
</IfModule>
# php -- END cPanel-generated handler, do not edit

Make sure the AddType and the selected php version matches!  So there we go, simple when you know how.

Controlling Milight Lights with a Raspberry Pi and NRF24L01+

picture of bulbIntroduction

I have around 16 Milight RGBW LED lightbulbs and strips (sold under various brands, limitlessled, easybulb etc). These were being controlled using four Milight Wifi bridges. However, the reliability was not great – some transmissions were missed, lights did not always work and careful positioning was required to ensure the signal could reach outlying lights.

Following a google search, solutions were found based around the ESP8266 and the Raspberry Pi. As I’m already using a number of Raspberry Pi units around the house this seemed like the obvious solution and simply required the connection of an NRF24L01+ module and some software running on the Pi.

Most of the work had already been completed by Torsten Tränkner (http://torsten-traenkner.de/wissen/smarthome/openmilight.php).

Connecting the NRF24L01+ Module

nrf24l01+ connection diagramConnect the NRF24L01+ module as shown in the diagram on the left (See https://pinout.xyz/ for an easy to use pin reference for the Pi).

 

 

 

 

 

 

The Software

Taking the code provided by Torsten (https://github.com/bakkerr/openmilight_pi) I modified and enhanced it to support emulation of up to four bridges from a single Pi – this means I can reduce the number of hardware devices and also the number of radio collisions.

As the Milight protocol is a send-only protocol there is no guarantee that the bulb received the message and you have to send the message multiple times.  However, if multiple bridges are sending at the same time these messages are likely to be lost.  By emulating four virtual bridges in the same software we can track and send the messages one at a time.  This reduces the possibility of collisions and shows a noticeable increase in reliability.

Clone this github repository and follow instructions in the README.md to compile and install the software.

The repository also includes an init script and a monit script to keep it running.  If you wish to use these copy them to the relevant locations and the executable to /opt.  The init script comes pre-configured with sample remote IDs so change these to suit your environment.

cp ./openmilight /opt/
cp ./openmilight.init.d /etc/init.d/openmilight
cp ./openmilight.conf.monit /etc/monit/conf.d/openmilight.conf

 

WH1080 WeatherStation with arduino and WeeWX

WH1080 Weather station mounted on roof

The Fine Offset WH1080 Weather Station (available at Clas OhlsonMaplin Electronics and numerous other places) is a low-cost, easy to use wireless weather station that transmits data over
868Mhz and can easily be decoded using an RFM12B chip and an Arduino.

Continue reading “WH1080 WeatherStation with arduino and WeeWX”

How to enable VERP for incoming mail delivery on Cpanel/WHM

Running software which handles bulk mailing such as the CiviCRM CiviMail component requires bounce processing if you don’t want your server to become blacklisted.  The preferred way to do this is using VERP handling (see https://en.wikipedia.org/wiki/Variable_envelope_return_path)

In CiviMail setup a bounce processing mailbox with username “bounce” and localpart “bounce+”.  Note you could use any username that you have created a mailbox for but the localpart must match it or the bounced email will not be sent to that user. Continue reading “How to enable VERP for incoming mail delivery on Cpanel/WHM”

Monitoring an APC UPS with node-red and MQTT

apc cs500 upsIntroduction

APC sell a decent range of USB connected Uninterruptable Power Supplies (UPS) such as the CS500 and BE700G.  These are supported by the linux utility apcupsd for control and monitoring.

This post explains how to monitor these UPS devices using Node-Red to obtain and publish data using MQTT.  It assumes you already have a working Node-Red install.

Continue reading “Monitoring an APC UPS with node-red and MQTT”

Embedded Read-only Root on Raspberry Pi and Odroid

Introduction

My main home automation controller runs on an Odroid U3 (bought from pollin.de).  Additionally I have a number of Raspberry Pi devices performing various duties around the home.

One of the most important features of an embedded system is that it doesn’t fail or need maintenance because of a power failure or an update etc.  This article explains how to run debian wheezy on these
devices with a read-only root filesystem. Continue reading “Embedded Read-only Root on Raspberry Pi and Odroid”

Cross compile openzwave control panel for armhf (Raspberry Pi) using Docker

This post describes how to build openzwave control panel for the armhf architecture using a docker image and qemu on a amd64 linux desktop (debian stretch).  Once built the ozwcp.tar.gz can be extracted and run on an armhf based system (eg. Raspberry Pi).

Install Docker

apt-get install docker.io apparmor apparmor-utils qemu-user-static wget

Continue reading “Cross compile openzwave control panel for armhf (Raspberry Pi) using Docker”

Modifying CUL firmware to increase send limit for eq3 MAX! heating thermostats

MAX-HeizkoerperthermostatPlus-640040d2Updated November 2016 to completely remove credits handling so there is no restriction on transmission

I’ve recently purchased 10 of the eq3 MAX! central heating thermostats (TRV replacements) as well as 3 shutter contacts and 2 eco switches from conrad electronics. I also bought the MAX! cube but then decided to use a CUL device from busware.de to control them using FHEM because this allows for easier interaction with them and use of the eco switches directly. Continue reading “Modifying CUL firmware to increase send limit for eq3 MAX! heating thermostats”