Using natural language based plaintext

yields natural redundancy and biases in the plaintext.

We often see that words are abbreviated

during texting and chatting to avoid such redundancy,

and still get the message across.

Such redundancy and bias in

the plaintext alphabets can be used for cryptanalysis by the attacker.

Next to me is the frequency diagram of English letters.

It captures the statistics of

each English alphabets or how often the letter alphabets are used in English.

From the statistics, we see that there are letters that occur more often than the others.

For example, the letter E occurs the most often,

followed by the letter T. And,

there are other alphabets that occur less frequently.

For example, the letter Z occurs the least

frequently and the letter Q is the second to least frequent.

Such alphabet frequency biases that are natural in plaintext use,

can produce vulnerability to the attacker who wishes to break the cipher.

For example, monoalphabetic cipher is vulnerable in

getting such frequency based cryptanalysis.

Because monoalphabetic cipher maps each of

the plaintext alphabet to the same ciphertext alphabet,

the ciphertext alphabet frequency or

the occurrence characteristics of

the ciphertext will be the same as that of the plaintext alphabet.

The attacker can guess that the ciphertext alphabet start occurring more frequently,

corresponds to the plaintext alphabet that occur more frequently.

Going back to the example used previously with the plaintext, meet me later,

that 11 letters ciphertext had four Fs,

two Cs, two Us and other letters that appeared only once.

The alphabet F and the alphabet U corresponds to the plaintext E and T,

respectively which are the top two frequently occurring letters in English.

On the other hand, there were no M or N in

the ciphertext which were mapped from Q and Z, respectively.

Such statistics based cryptanalysis will become more

accurate as there are more samples to support the statistics.

In other words, the attackers cryptanalysis will

become more powerful as the plaintext length increases.

Such frequency analysis based

cryptanalysis cannot only occur at the single alphabet level,

but also with sequence of multiple alphabets.

For example, using a pair of letters,

the letter H is more likely to follow the letter T than others,

while the letter U is likely to follow the letter Q.

Also, among the triplet of letters T, H,

E occur very often as well.

Therefore, such frequency based cryptanalysis technique

can also be used by analyzing the sequence of alphabets.

The ideal case for cryptography is when

the alphabets have uniform distribution and they have equal probabilities of occurrence.

As discussed in the cryptography and information theory course,

such uniform distribution maximizes the information entropy of the alphabets.

Some, say for algorithms,

such as polyalphabetic ciphers,

try to smooth out

the frequency distribution and make the distribution closer to a uniform distribution.

Natural Language Redundancy

Symmetric Cryptography

Meet the Instructors