Alice wants to send Bob a message. She will use the english alphabet. a) What is

Question

Alice wants to send Bob a message. She will use the english alphabet.

a) What is the information entropy for a letter?

b) Alice converts every letter to bits and sends the bits to Bob. How many bits are needed per letter?

c) Alice then sends a 12-letter message, converting the whole message to bits rather than individually. How many bits are needed per letter?

d) Entropy for a single letter gives the entropy of X bits. Compare X to answers from part a,b (greater, less than, equal).

e) With X from (d), Alice attaches 1 byte header for every 10 letters. What is the information entropy for sending the letters plus the header?

Explanation / Answer

Part a

Entropy per Word is 11.82 bits

Average length per Word is 4.5 letters

So the Entropy per letter is 2.62 bits

This is only for half the letters as the other half are redundant.

Part b

As there are 26 letters and 2^5(32) covers all 26 letters. Therefore, 5 bits would be sufficient for each letter. For example, z will be denoted as 11010. In a whole byte, 5 bits can be the message and other 3 be 000. For z the byte can be 00011010. The 5 Least Significant Bits would be considered.

Part c

12 letter message means 12 bytes.

1 Byte = 8 Bits

So 12 Bytes = 12*8 bits = 96 bits

When the whole message is decoded as one then the number of bits per letter would be 5 if only meaningful bits are seen else 8 bits. So it is a bit tricky that way.

Part d

The entropy of a letter calculated by the Shanon calculator is 4.6612 which near 5 but less than 5.

So the original entropy of a letter is 4.6612 bits as out of 32 bits only 26 are used and rest are left unused.

Related Questions

Navigate

• Browse (All)
• Browse A
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.