Cracking the Code: Letter Frequency and How to Solve Any Cipher

ZG WT GK FGZ ZG WT ZIQZ OL ZIT JXTLZOGF.

Looks like nonsense, doesn’t it? It’s actually one of the most famous lines in the English language, wearing a disguise. By the end of this article you’ll know how to take that disguise off, and you won’t need a computer, a codebook or a degree in maths. You need one idea that is more than a thousand years old: some letters turn up far more often than others.

That simple fact is called letter frequency. It has toppled queens, won wars and launched a whole branch of mathematics. Today it’s the secret weapon behind every daily cipher puzzle, so put the kettle on and let’s crack some codes.

What is a substitution cipher?

A substitution cipher swaps every letter of a message for a different letter, using the same swap every time. If E becomes T once, E becomes T everywhere. The spaces and punctuation usually stay put, so the shape of each word survives even when its letters don’t.

People have hidden messages this way for a very long time:

Julius Caesar shifted each letter three places along the alphabet, so A became D and B became E. It worked well in an age when most people couldn’t read at all.
Al-Kindi, a 9th-century scholar in Baghdad, wrote the first known description of frequency analysis. He saw that every language has favourite letters, and that a cipher which only swaps letters can’t hide those favourites. That one idea made the simple substitution cipher breakable for good.
Mary, Queen of Scots found this out the hard way. In 1586 her coded letters to the Babington plotters were intercepted. Elizabeth I’s codebreaker, Thomas Phelippes, cracked them with frequency analysis. The decoded letters were the main evidence at her trial, and she was executed in 1587.
Edgar Allan Poe (The Gold-Bug, 1843) and Arthur Conan Doyle (The Adventure of the Dancing Men, 1903) both built famous stories around a hero cracking a cipher using exactly the method below.

The cipher in your daily puzzle is the same kind that tripped up a queen. The good news is that nobody’s head is on the line this time.

Letter frequency: the codebreaker’s secret weapon

In ordinary English, E is the most common letter by a mile, making up about one letter in every eight. Then come T, A, O, I and N. Those six letters alone make up roughly half of everything you read.

[embed: node/a42a0301-dcb0]

Old typesetters knew this off by heart. The keys on a Linotype machine were laid out by frequency, and the first two columns spelled ETAOIN SHRDLU. The phrase was so familiar that it sometimes ended up in newspapers by mistake.

This is why frequency works against a cipher. Swapping E for T changes the letter, but it can’t change how often the letter turns up. If one symbol appears far more than any other, it’s probably standing in for E or T. At the other end, a symbol that appears once or not at all is probably one of the rare four: J, X, Q or Z.

How to solve any cipher, step by step

Frequency gets you started and word shapes finish the job. Work through these steps roughly in order, pencilling in guesses as you go.

Count the letters. Tally how often each cipher letter appears. The top one or two are very likely E and T, and the next few are probably A, O, I or N. In a short quote, treat this as a hint, not a rule.
Look for one-letter words. In English these are almost always A or I. That’s a vowel cracked before you’ve really started.
Hunt for THE. It’s the most common word in English. A three-letter word that keeps repeating, especially one starting with your most common letter, is a strong candidate. AND is the runner-up.
Use the apostrophes. A single letter after an apostrophe is nearly always S, T, D or M (it’s, don’t, I’d, I’m). Two letters after one is usually LL, RE or VE.
Spot double letters. The commonest pairs are LL, EE, SS, OO, TT and FF. Doubled vowels other than EE and OO are rare, so AA or II almost never appear.
Read the word shapes. Words with repeated letters have fingerprints. A four-letter word whose first and last letters match (1–1) is often THAT. The pattern 1-2-2-1 is often NOON, DEED or SEES.
Lean on common pairs. The most frequent two-letter combinations in English are TH, HE, IN, ER, AN, RE and ON. If you’ve found T, the letter that keeps following it is probably H.
Fill in, then read across. Every letter you crack appears all over the puzzle. Write it in everywhere, then look for half-finished words your brain can complete. This is where the puzzle suddenly falls apart in your favour.

One more trick: in most puzzle ciphers no letter stands for itself. If you’re stuck choosing between two options, rule out any that would map a letter to itself.

Worked example: cracking our mystery line

Let’s go back to the line at the top of this article and solve it with the steps above.

ZG WT GK FGZ ZG WT ZIQZ OL ZIT JXTLZOGF

Count. Z appears 7 times, G 5 times and T 4 times. Z is the clear favourite, so it’s probably E or T.
Shapes. ZIQZ starts and ends with the same letter (1–1). That’s the classic shape of THAT, which makes Z = T, I = H and Q = A.
Confirm. If Z = T and I = H, then ZIT reads TH?. That has to be THE, so T = E. Three cracked letters already agree with each other.
Fill in. ZG becomes T?. With O the most likely next vowel, that’s TO, so G = O.
The long word. JXTLZOGF now reads ? ? E ? T ? O ?. Eight letters with E in third place and TIO near the end point to QUESTION. That gives J = Q, X = U, L = S, O = I and F = N.
Read across. FGZ = NOT, OL = IS, and suddenly the whole thing is obvious.

TO BE OR NOT TO BE, THAT IS THE QUESTION.

Hamlet, unmasked in six steps. Notice that we never needed WT or GK. Once you’ve cracked enough letters, the rest of the line usually reads itself. Here’s the key we built along the way:

Cipher letter	Z	I	Q	T	G	J	X	L	O	F	W	K
Real letter	T	H	A	E	O	Q	U	S	I	N	B	R

Common mistakes (and how to avoid them)

Trusting the counts too much. Frequency is a law of averages, and a 40-letter quote is a small sample. Our Hamlet line has more T’s than E’s and more O’s than A’s. Use the counts to pick your first guess, then let word shapes confirm it.
Committing too early. Pencil in a guess, test it against two or three other words, and only then treat it as solved. One wrong letter written in everywhere can send you round in circles for ages.
Ignoring the short words. It’s tempting to stare at the longest word, but two- and three-letter words have only a handful of possible answers. They’re where most ciphers crack first.
Forgetting the rare letters. Once the common letters are in, the leftovers are usually the awkward ones: J, Q, X and Z. Q is almost always followed by U, which is a handy freebie.
Not using the context. Daily cipher puzzles are usually famous quotes. If half the words are showing and it sounds familiar, trust your memory and fill it in.

Pro tip: cracking a cipher uses the same instinct as a good Hangman guess. Both are really about asking “which letters are most likely here?” Get good at one and you’ll get better at the other.

Your turn

You now have the same toolkit as al-Kindi, Thomas Phelippes and Sherlock Holmes: count the letters, find the short words, read the shapes and let each cracked letter unlock the next. The first few puzzles might take a while. After a week, you’ll find yourself spotting THAT and THE before you’ve even started counting.

Ready to try it for real? Today’s quote is waiting at the Cipher Desk. New to the game? Start with our guide to how Cipher works.

If you enjoyed this, you might also like:

Happy codebreaking!

Letter frequencies are typical figures for English text, from Wikipedia: Letter frequency.