Comparing/contrasting Shannon vs Von Neumann entropy may also be useful. What I love about this is it’s all entropy but what you can do with it in information theory is much nicer than what you can do with the pre information theory entropy.
https://en.wikipedia.org/wiki/Von_Neumann_entropy
—