Skip to main content

TL;DR

Shannon Entropy and Optimal Binary Codes: A canonical quantitative trading interview question at intermediate difficulty. Commonly asked at Two Sigma, DE Shaw, Citadel.

By Valenke Exam Prep Team·Last updated 2026-06-01
intermediateInformation & Entropy

Shannon Entropy and Optimal Binary Codes

Asked at: Two Sigma, DE Shaw, Citadel

Problem
A source emits symbols {A,B,C,D}\{A, B, C, D\} with probabilities {1/2,1/4,1/8,1/8}\{12\frac{1}{2}, 14\frac{1}{4}, 18\frac{1}{8}, 18\frac{1}{8}\}. Compute the Shannon entropy. Construct an optimal prefix-free binary code and verify it achieves the entropy bound.
Related concepts
Information EntropyHuffman Coding

Ready to practice for the Valenke Finance Exam?

Adaptive practice powered by Item Response Theory targets your weak areas. Start with 3 free sessions.

Start free practice →
Practice similar problemsAll core problems