Information Theory and Coding

Indian Institute of Technology Bombay

This course covers lessons on information theory and coding, entrophy, block code and its properties, huffman coding,Shannon-Fano-Elias, arithmetic coding, information channels, Lloyd-Max Quantizer and vector quantization.

Course topics:

  1. Introduction to Information Theory and Coding
  2. Definition of Information Measure and Entropy
  3. Extention of An Information Source and Markov Source
  4. Adjoint of An Information Source, Joint and Conditional Information Measure
  5. Properties of Joint and Conditional Information Measures and A Morkov Source
  6. Asymptotic Properties of Entropy and Problem Solving in Entropy
  7. Block Code and its Properties
  8. Instantaneous Code and Its Properties
  9. Kraft-Mcmillan Equality and Compact Codes
  10. Shannon's First Theorem
  11. Coding Strategies and Introduction to Huffman Coding
  12. Huffman Coding and Proof of Its Optamality
  13. Competitive Optamality of The Shannon Code
  14. Non-Binary Huffman Code and Other Codes
  15. Adaptive Huffman Coding Part-I
  16. Adaptive Huffman Coding Part-II
  17. Shannon-Fano-Elias Coding and Introduction to Arithmetic Coding
  18. Arithmetic Coding Part-I
  19. Arithmetic Coding Part-II
  20. Introdution to Information Channels
  21. Equivocation and Mutual Information
  22. Properties of Different Information Channels
  23. Reduction of Information Channels
  24. Properties of Mutual Information and Introdution to Channel Capacity
  25. Calculation of Channel Capacity for Different Information Channels
  26. Shannon's Second Theorem
  27. Discussion On Error Free Communication Over Noisy Channel
  28. Error Free Communication Over A Binary Symmetric Channel and Introdution to Continous Sources and Channels
  29. Differential Entropy and Evaluation of Mutual Information for Continuous Sources and Channels
  30. Channel Capacity of A BandLimited Continuous Channel
  31. Introduction to Rate-Distortion Theory
  32. Definition and Properties of Rate-Distortion Functions
  33. Calculation of Rate-Distortion Functions
  34. Computational Approach for Calculation of Rate-Distortion Functions
  35. Introdution to Quantization
  36. Lloyd-Max Quantizer
  37. Companded Quantization
  38. Variable Length Coding and Problem Solving in Quantizer Design
  39. Vector Quantization
  40. Transform Coding Part-I
  41. Transform Coding Part-II
Course Lectures