toolhq.io

All posts
8 min readby Jameel Haider

MTA-STS and TLS-RPT: enforcing TLS on inbound mail

SMTP encryption is opportunistic by default. A sending server connects, asks whether the receiver supports STARTTLS, and upgrades if the answer is yes. If the answer is no, it sends the message in the clear rather than fail. That fallback is what keeps mail flowing across a network of servers with wildly different capabilities, and it is also the weakness: anyone able to modify the connection can remove the STARTTLS advertisement and watch the sender quietly downgrade.

MTA-STS closes that gap for inbound mail to your domain. TLS-RPT tells you whether it is working.

What MTA-STS actually does

MTA-STS (RFC 8461) lets a domain publish a policy saying: mail for this domain must be delivered over TLS, to these MX hosts, with a valid certificate. A sending server that supports MTA-STS fetches that policy, caches it, and refuses to deliver if the connection cannot meet it.

The policy is deliberately published over HTTPS rather than DNS. That means its authenticity rests on the web PKI, which is already deployed everywhere, instead of requiring DNSSEC. It is the pragmatic counterpart to DANE, which achieves a similar goal but depends on DNSSEC being in place.

Two pieces are required.

A DNS record at _mta-sts.example.com, type TXT:

v=STSv1; id=20260818120000

The id is an opaque version string. Sending servers use it to notice that the policy changed. It must change whenever you edit the policy file, or caches will keep serving the old one until they expire.

A policy file served at https://mta-sts.example.com/.well-known/mta-sts.txt:

version: STSv1
mode: testing
mx: mx1.example.com
mx: mx2.example.com
mx: *.mail.example.com
max_age: 604800

The host mta-sts.example.com needs its own valid certificate. It is a separate hostname from your MX records and from your main site, and it must be reachable over HTTPS with a certificate that validates normally.

The three modes

  • none disables the policy. Used to retire MTA-STS cleanly, since simply deleting the file leaves cached policies in force until max_age.
  • testing asks senders to evaluate the policy and report failures, but deliver anyway. Nothing breaks. This is where every deployment starts.
  • enforce asks senders to refuse delivery when the policy cannot be met. This is the mode that provides the protection, and the mode that bounces mail if your MX list is wrong.

max_age is how long a sender may cache the policy, in seconds. A week is common. Note the implication: in enforce mode, a mistake stays cached at sending servers for up to max_age, so start low, around 86400, and raise it once stable.

TLS-RPT, the part people skip

TLS-RPT (RFC 8460) is a separate TXT record at _smtp._tls.example.com:

v=TLSRPTv1; rua=mailto:tls-reports@example.com

Sending servers that support it send you a daily JSON summary of TLS negotiation outcomes with your domain: successful sessions, certificate mismatches, expired certificates, failed STARTTLS, policy fetch errors.

This is the feedback loop that makes enforce mode safe. Deploying MTA-STS without TLS-RPT means switching to enforce with no visibility into whether senders can actually satisfy the policy. The reports are the difference between an informed change and a guess.

TLS-RPT works independently of MTA-STS, so you can publish it first and collect a week of data before writing a policy at all.

Deploying it in the right order

  1. Publish TLS-RPT alone. One TXT record, no risk. Wait several days and read the reports. If they already show TLS failures against your MX hosts, fix those before going further.
  2. Inventory every MX host. Query your MX records with the MX lookup tool and list every hostname, including backup MX and anything a mail provider adds. A host missing from the policy will be refused in enforce mode.
  3. Check certificates on each MX host. Each one needs a certificate valid for the name that appears in the MX record. Run each through the SSL checker, or check them together with the bulk SSL checker. Certificate names that do not match the MX hostname are the most common blocker.
  4. Stand up the policy host. Serve the file at https://mta-sts.example.com/.well-known/mta-sts.txt with content type text/plain. Confirm the certificate for that hostname is valid too.
  5. Publish the DNS record in testing mode. Verify both TXT records resolve using the DNS Lookup tool.
  6. Read reports for two to four weeks. You are looking for zero policy failures across a representative set of senders.
  7. Switch to enforce. Edit the policy file, change the id in the DNS record, and keep watching reports.

Skipping step 6 is the usual cause of a bad outcome. The reports tell you whether real senders can comply, and nothing else does.

Common failures

  • The id did not change. The policy file was edited, senders kept the cached copy, and nothing appeared to happen.
  • The policy host certificate is invalid or expired. Senders cannot fetch the policy, so they cannot apply it. In testing mode this shows up in reports; in enforce mode senders fall back to their cached policy until it expires.
  • An MX host is missing from the list. Backup MX records and provider added hosts are the usual omissions.
  • Certificate name mismatch. The MX record says mail.example.com but the certificate covers only example.com. This is invisible in normal opportunistic delivery and fatal under enforce.
  • A wildcard pattern that does not match. *.mail.example.com matches mx1.mail.example.com but not mail.example.com. The wildcard covers one label only.

Where it sits alongside everything else

MTA-STS protects the transport of mail arriving at your domain. It does nothing about authentication, so it complements rather than replaces SPF, DKIM and DMARC, which are covered in SPF, DKIM and DMARC explained. If you are working through inbound and outbound mail configuration together, the email health check reads the DNS side of all of it in one pass, and the deliverability checklist covers the ordering.