MTA-STS and TLS-RPT: setup, testing, and failure diagnosis

SMTP defaults to opportunistic encryption — try TLS, fall back to plaintext if it doesn't work. That fallback is exactly what STARTTLS-stripping attackers exploit. MTA-STS lets you tell the world: never deliver to me without TLS, and verify the certificate. TLS-RPT gives you reports when something goes wrong.

Use the authentication hub for the adjacent SPF, DKIM, DMARC, and DNS work, then use the MTA-STS checker to validate the transport policy itself.

Why opportunistic TLS is not enough

By default, an SMTP server sends a STARTTLS upgrade. If the receiver advertises support, the connection is encrypted. If it doesn't — or an on-path attacker strips the upgrade — the message goes out in cleartext. The sender has no way to know it happened. MTA-STS closes this gap by publishing a verifiable policy that says: don't downgrade.

The two pieces

Before changing policy, run the free MTA-STS checker to confirm the DNS signal and HTTPS policy are both reachable from public networks.

Step 1 — Publish the policy file

Host a plain-text policy at https://mta-sts.<your-domain>/.well-known/mta-sts.txt. Must be served over HTTPS with a valid certificate.

version: STSv1
mode: testing
mx: mail.example.com
mx: mail-backup.example.com
max_age: 604800

• `mode: testing` — receivers report failures but still deliver. Always start here.
• `mode: enforce` — receivers refuse to deliver if TLS fails. Move here after testing.
• `mode: none` — turn off MTA-STS gracefully without removing the record.
• `mx:` — list every MX host that should accept mail. Wildcards (*.example.com) allowed.
• `max_age` — how long receivers can cache (in seconds). 1 week (604800) is typical.

Step 2 — Publish the DNS pointer

Two TXT records: one announces the policy and its version (the id should change every time you update the policy file), one for TLS-RPT.

Step 3 — Wait and read TLS-RPT

TLS-RPT reports arrive daily as JSON-formatted email attachments. They list every TLS connection attempt to your MX hosts and the outcome. Look for failures: cert mismatches, expired certs, STARTTLS not offered, and downgrade attempts. Keep the adjacent MX, TLS, and authentication records under review with the domain monitoring workflow.

Step 4 — Move to enforce

1. Run in testing mode for at least 4 weeks.
2. Confirm TLS-RPT reports show no failures from major receivers (Google, Microsoft, Yahoo).
3. Update mta-sts.txt to mode: enforce and bump the id in DNS.
4. Watch reports for the next 2 weeks. Any new failure now means delivery is blocked.

MTA-STS vs DANE

DANE/TLSA is the DNSSEC-based alternative. It's stronger (signed in DNS, not bootstrapped over HTTPS) but requires DNSSEC, which adds operational complexity. Most teams pick MTA-STS because it works without DNSSEC. Some run both for receivers that prefer one or the other.

Run the MTA-STS checker to verify DNS and HTTPS publication. Use TLS-RPT results to diagnose real transport failures before changing from testing to enforce mode.