Punycode and homograph domains: when apple.com is not apple.com

DNS carries a narrow alphabet. Letters, digits and hyphens, case insensitive, and nothing else. That restriction dates from a time when the internet was mostly English speaking, and it long outlived its usefulness.

Internationalised domain names solve it without changing DNS at all, by encoding Unicode into the allowed character set. The mechanism works well. It also creates a class of phishing domain that is genuinely indistinguishable from the real thing.

What xn-- means

Punycode is the encoding, defined in RFC 3492. It rewrites a Unicode label into ASCII, and the result is prefixed with xn-- so resolvers know to decode it.

The algorithm keeps the ASCII characters in place, then appends an encoded description of which non ASCII characters to insert and where. münchen becomes mnchen plus 3ya, which says "insert ü after position 1". It is reversible, compact, and produces only lowercase letters, digits and hyphens.

Conversion happens in the client. The browser or mail client converts what the user typed into the ASCII form before the DNS query goes out, so the resolver only ever sees xn--. That is why the encoding needed no changes anywhere in the DNS infrastructure.

The whole label is either ASCII or encoded; the prefix applies per label, not per domain. A domain can be shop.münchen.de, encoded as shop.xn--mnchen-3ya.de, with the first label untouched.

The attack

Unicode contains many characters that render identically or near identically to Latin letters. Cyrillic а (U+0430) and Latin a (U+0061) are different code points that most fonts draw the same way. So are Cyrillic е, о, р, с, х and Greek ο, and there are hundreds more in the Unicode confusables table.

Register a domain where every Latin letter is replaced by its Cyrillic twin, and the rendered name is pixel identical to the original while resolving to a completely different registration. The well known 2017 demonstration used Cyrillic letters to register a domain that Chrome and Firefox both displayed as apple.com, with a valid TLS certificate and the padlock, while the underlying registration was xn--80ak6aa92e.com.

A certificate authority will issue for that domain quite correctly. The attacker owns it, domain control validation passes, and nothing about the certificate is fraudulent. The padlock has never meant "this is who you think it is", only "this connection is encrypted to whoever holds this name". Domain control validation explained covers what a CA actually checks.

Simpler variants do not need Unicode at all. rn reads as m at small sizes, l and I and 1 are close in many fonts, and an added or dropped hyphen goes unnoticed. Those work on ASCII domains and are more common in practice because they are cheaper to register and less likely to be blocked.

What browsers do

Browsers apply heuristics to decide whether to display the Unicode form or fall back to the raw xn-- string, which is obviously suspicious to a user.

The dominant rule is script mixing. A label containing characters from more than one script, with limited exceptions for genuine mixed script languages such as Japanese, is displayed as punycode. This kills the pure Cyrillic substitution attack in its mixed form. Chrome adds checks for whole script confusables, where every character comes from one script but the whole label looks like Latin, and maintains a list of specific confusable sequences. Firefox additionally lets certain registries be trusted based on their own anti confusion policies.

Two caveats matter. The heuristics protect the URL bar, not the link text on a page, not an email client's sender display, not a chat application's preview, and not a QR code. And they are heuristics: new confusable pairs keep being found, and the rules change between browser versions.

What registries do

Most registries restrict which scripts a label may contain, and many require all characters in a label to come from a single script or an approved combination. Some maintain variant tables, so registering a name also reserves its confusable equivalents, or blocks anyone else from taking them.

Coverage is uneven. Policy varies by TLD, enforcement varies by registrar, and the newer generic TLDs are inconsistent. Treat registry policy as a partial mitigation, not a guarantee.

Finding impersonations of your own domain

This is the part you control. Two sources are worth watching.

Certificate transparency logs. Every publicly trusted certificate is logged, so anyone registering a lookalike and putting HTTPS on it publishes that fact. Searching the logs for names similar to yours finds impersonations shortly after they are set up and usually before any campaign starts. The CT log lookup searches the logs for a domain, and certificate transparency logs explained covers what the logs contain and how to monitor them continuously.

Registration data. Once you have a candidate, check it. A domain registered three days ago, with privacy protection, at a registrar you have never used, is not a coincidence. The domain lookup tool shows registration and expiry dates, registrar and nameservers, and WHOIS and domain lookups explained covers reading the output.

To generate the candidate list, take your domain and produce the permutations an attacker would: confusable character substitutions, character omissions and doubles, adjacent key typos, hyphen insertion and removal, and the same name under other TLDs. Feed those through both checks. Doing it monthly catches most of what matters.

Handling IDNs in your own code

If your application accepts domains or URLs from users, a few rules prevent it from becoming the weak link:

1. Normalise with UTS-46 before comparing. Convert to the ASCII xn-- form and compare that. Comparing display forms means two different domains can compare equal, or the same domain can compare unequal depending on normalization. Most platforms have this built in: new URL() in browsers performs the conversion, Python has idna, Go has golang.org/x/net/idna.
2. Store the ASCII form. Display the Unicode form if you like, but the stored key and the thing you match allowlists against should be xn--.
3. Show punycode in security sensitive contexts. If you display a domain in a warning, a confirmation dialog or an audit log, showing xn--80ak6aa92e.com is more useful than showing something that looks like apple.com.
4. Do not write your own IDNA implementation. The mapping tables are large, versioned and full of edge cases, and the older IDNA2003 and newer IDNA2008 standards disagree about several characters including ß and the final sigma.

The URL parser breaks a URL into its parts so you can see exactly which host a string resolves to, which is useful when a link's display text and its actual target differ.

Email is a separate problem

Internationalised email addresses need more than punycode, because the local part before the @ is not a domain and is not encoded that way. SMTPUTF8 allows UTF-8 throughout the address, but support is inconsistent across servers, libraries, validation regexes and the systems downstream of them. An address that one hop accepts and the next rejects produces a bounce nobody can diagnose.

If you validate email addresses, be aware that most regular expressions in circulation assume ASCII and will reject legitimate international addresses outright. Regex for email validation covers why the regex approach is limited in general.

Related reading: WHOIS and domain lookups explained, certificate transparency logs explained, and parse a URL into its parts for the structure around the hostname.