Ambiguous characters, and what excluding them costs

What actually gets confused

The confusion is not evenly spread. A handful of pairs and triples account for nearly all of it, and they group by the shape people mistake rather than by any property of the characters themselves.

Two of these are not character confusions at all. The pair rn read at small sizes looks like m, and vv looks like w. So does cl for d in some faces. These are digram confusions, and no exclusion policy fixes them, because both characters involved are individually unambiguous. Any tool claiming to remove all ambiguous characters is overclaiming, and this is why.

The Unicode confusables data, which is the authoritative source for this kind of question, contains exactly seven ASCII-to-ASCII confusable mappings, including m to rn and the vertical bar to lowercase l. Everything else in the list below is a practical observation about typefaces rather than a formal Unicode confusability claim, which is the honest way to present it.

• 0, O, o, Q, D - the round ones. The worst set by a distance, and the one every alphabet handles first.
• 1, l, I, i - the vertical ones. Capital I and lowercase l are identical in most sans-serif faces.
• 5 and S - close in many faces, worse in condensed ones.
• 2 and Z - mostly a handwriting problem, but real on low-resolution displays.
• 8 and B - the closed counters make these converge at small sizes.
• 6, G and lowercase b - all three read as a bowl with an ascender.
• 9, g and q - all three read as a bowl with a descender.
• rn as m, vv as w, cl as d - digram confusions. These cannot be fixed by removing characters.

Typeface decides most of it

The same string is unambiguous in one face and unreadable in another, which means the font is part of the security decision whenever a human is in the loop.

The single largest improvement is a disambiguated zero, either slashed or dotted. It removes the 0 and O collision entirely, which is the most frequent one. Most monospaced faces designed for code do this, along with adding a tail to the lowercase l and a serif to the capital I, which handles the second-worst set.

The failure case is a proportional sans-serif at a small size, which is exactly what most web interfaces and most printed receipts use. If you are rendering a code that someone will retype, set it in a monospaced face at a generous size, and group it with separators. Grouping is not decoration: it gives the reader a place to resume after losing their position, and it makes transposition errors visible.

None of this helps if the value is read over a phone, which is a different problem with a different answer.

When it is worth doing

The test is simple: does a human ever transcribe this value from one medium to another? If the answer is no, excluding characters is pure loss.

It is worth doing for anything printed on paper or card, where the reader has no ability to copy and no feedback if they get it wrong. Voucher codes, licence keys, activation codes and the Wi-Fi password on a hotel card are all in this category.

It is worth doing for anything read aloud. Over a phone, 5 and S are the same sound in many accents, and so are the letters that share a rhyme. This is where a digits-only or a heavily restricted alphabet earns its cost, and where the NATO alphabet is the other half of the answer.

It is worth doing for a value a user types into a device with a poor keyboard, which in practice means a television, a games console, a printer or a smart display.

It is not worth doing for an API key, a session token, a database identifier, or anything else that lives entirely in copy and paste. There the exclusion buys nothing and costs entropy, and it also narrows the alphabet an attacker has to search, which is the part that usually goes unmentioned.

What it costs, exactly

Dropping from a 62-character alphanumeric alphabet to the 49-character nolookalikes set costs 0.3395 bits per character. That is 5.70 percent of the 5.9542 bits each character was carrying.

The recovery is one extra character for every 16.5 you had. Concretely: a 16-character alphanumeric string carries 95.267 bits, and a 17-character nolookalikes string carries 95.450. One character, and you are ahead of where you started.

That is the whole argument, and it is why the tradeoff is usually worth taking when a human is involved. The cost is real and it is small, and it is fully recoverable at a price you can pay. What is not defensible is taking the cost without telling the user, or claiming the exclusion is free.

The other half of the accounting is less comfortable. A reduced alphabet is public: the attacker knows which characters you removed, because your tool says so. So the search space really is 49 to the power L rather than 62 to the power L. Excluding characters does not obscure anything, it just costs bits. Present it as an ergonomic trade, never as a security feature.

Use a published alphabet

Every alphabet in the table above is citable to a primary source, which matters more than it sounds. A bespoke house alphabet is not interoperable with anyone else's decoder, cannot be referenced in your documentation, and provides no way for a reader to check that you removed the right things.

Crockford base32 is the strongest choice for anything read by a human, and it documents its own reasoning: I and L are excluded because they can be confused with 1, O because it can be confused with 0, and U because of accidental obscenity. It is exactly five bits per character, so encoding is clean, and the specification also defines hyphen grouping that decoders ignore, plus an optional mod-37 check symbol that will catch a single-character transcription error before the user gets an unhelpful failure.

The nanoid nolookalikes sets are the right choice when you want a larger alphabet and are generating rather than encoding. base58 is the right choice when you need compatibility with an existing ecosystem that uses it.

One thing not to assume: nanoid's own default alphabet is not unambiguous. It is the 64-character URL-safe set, in a deliberately scrambled order chosen to compress well, and it contains every confusable pair in the list above.

Do it before sampling, never after

Remove the characters from the alphabet, then generate. The tempting alternative, generating from the full alphabet and then substituting or regenerating individual offending characters, destroys uniformity: the replacement characters end up over-represented by exactly the frequency of the ones you removed.

This is the same failure as forcing a required character into a fixed position, and it has the same detection: a chi-square test on character frequency shows it immediately.

It is also worth checking what a tool actually removes rather than trusting its label. Of the fifteen generators audited for this site, four offer an exclude-ambiguous option. One names its examples in a label that is factually wrong about which characters it removes. Three of the four leave capital I in the alphabet while removing lowercase l, which addresses half of the second-worst confusable set. One applies an undisclosed and inconsistent set, dropping l, o and s from its lowercase alphabet while keeping S and L in its uppercase one. Print the removed set on screen and let the user check it.

Questions