Information theory is a statistical probability course in the context of machine learning. Starting from the famous Shannon’s equation of information, the course runs through basic discrete information encoding program to first and second variational problems which cover presence in noise and penalty.

After the initial introduction, the course will pick up to stochastic processes in information, asymptotic properties, along with channel capacities.

The last part of the course will be generalization of Shannon’s theorem that deals with value of information, Bayesian system, along with some applications of information theory on real world data.