# Information Theory and Coding

Indian Institute of Technology Bombay

This course covers lessons on information theory and coding, entrophy, block code and its properties, huffman coding,Shannon-Fano-Elias, arithmetic coding, information channels, Lloyd-Max Quantizer and vector quantization.

Course topics:

1. Introduction to Information Theory and Coding
2. Definition of Information Measure and Entropy
3. Extention of An Information Source and Markov Source
4. Adjoint of An Information Source, Joint and Conditional Information Measure
5. Properties of Joint and Conditional Information Measures and A Morkov Source
6. Asymptotic Properties of Entropy and Problem Solving in Entropy
7. Block Code and its Properties
8. Instantaneous Code and Its Properties
9. Kraft-Mcmillan Equality and Compact Codes
10. Shannon's First Theorem
11. Coding Strategies and Introduction to Huffman Coding
12. Huffman Coding and Proof of Its Optamality
13. Competitive Optamality of The Shannon Code
14. Non-Binary Huffman Code and Other Codes
17. Shannon-Fano-Elias Coding and Introduction to Arithmetic Coding
18. Arithmetic Coding Part-I
19. Arithmetic Coding Part-II
20. Introdution to Information Channels
21. Equivocation and Mutual Information
22. Properties of Different Information Channels
23. Reduction of Information Channels
24. Properties of Mutual Information and Introdution to Channel Capacity
25. Calculation of Channel Capacity for Different Information Channels
26. Shannon's Second Theorem
27. Discussion On Error Free Communication Over Noisy Channel
28. Error Free Communication Over A Binary Symmetric Channel and Introdution to Continous Sources and Channels
29. Differential Entropy and Evaluation of Mutual Information for Continuous Sources and Channels
30. Channel Capacity of A BandLimited Continuous Channel
31. Introduction to Rate-Distortion Theory
32. Definition and Properties of Rate-Distortion Functions
33. Calculation of Rate-Distortion Functions
34. Computational Approach for Calculation of Rate-Distortion Functions
35. Introdution to Quantization
36. Lloyd-Max Quantizer
37. Companded Quantization
38. Variable Length Coding and Problem Solving in Quantizer Design
39. Vector Quantization
40. Transform Coding Part-I
41. Transform Coding Part-II
• ##### Mod-01 Lec-01 Introduction to Information Theory and Coding
Prof. S.N. Merchant
• ##### Mod01 Lec-02 Definition of Information Measure and Entropy
Prof. S.N. Merchant
• ##### Mod-01 Lec-03 Extension of An Information Source and Markov Source
Prof. S.N. Merchant
• ##### Mod-01 Lec-04 Adjoint of An Information Source, Joint and Conditional Information Measures
Prof. S.N. Merchant
• ##### Mod-01 Lec-05 Properties of Joint and Conditional Information Measures and a Markov Source
Prof. S.N. Merchant
• ##### Mod-01 Lec-06 Asymptotic Properties of Entropy and Problem Solving in Entropy
Prof. S.N. Merchant
• ##### Mod-01 Lec-07 Block Code and Its Properties
Prof. S.N. Merchant
• ##### Mod-01 Lec-08 Instantaneous Code and Its Properties
Prof. S.N. Merchant
• ##### Mod-01 Lec-09 Kraft-Mcmillan Equality and Compact Codes
Prof. S.N. Merchant
• ##### Mod-01 Lec-10 Shannon`s First Theorem
Prof. S.N. Merchant
• ##### Mod-01 Lec-11 Coding Strategies and Introduction to Huffman Coding
Prof. S.N. Merchant
• ##### Mod-01 Lec-12 Huffman Coding and Proof of Its Optimality
Prof. S.N. Merchant
• ##### Mod-01 Lec-13 Competitive Optimality of The Shannon Code
Prof. S.N. Merchant
• ##### Mod-01 Lec-14 Non-Binary Huffman Code and Other Codes
Prof. S.N. Merchant
• ##### Mod-01 Lec-15 Adaptive Huffman Coding part-1
Prof. S.N. Merchant
• ##### Mod-01 lec-16 Adaptive Huffman Coding Part-2
Prof. S.N. Merchant
• ##### Mod-01 Lec-17 Shannon-Fano-Elias Coding and Introduction to Arithmetic Coding
Prof. S.N. Merchant
• ##### Mod-01 Lec-18 Arithmetic Coding Part-1
Prof. S.N. Merchant
• ##### Mod-01 Lec-19 Arithmetic Coding Part-2
Prof. S.N. Merchant
• ##### Mod-01 Lec-20 Introduction to Information Channel
Prof. S.N. Merchant
• ##### Mod-01 Lec-21 Equivocation and Mutual Information
Prof. S.N. Merchant
• ##### Mod-01 Lec22 Properties of Different Information Channels
Prof. S.N. Merchant
• ##### Mod-01 Lec-23 Reduction of Information Channels
Prof. S.N. Merchant
• ##### Mod-01 Lec-24 Properties of Mutual Information and Introduction to Channel Capacity
Prof. S.N. Merchant
• ##### Mod-01 Lec-25 Calculation of Channel Capacity for Different Information Channel
Prof. S.N. Merchant
• ##### Mod-01 Lec-26 Shannon`s Second Theorem
Prof. S.N. Merchant
• ##### Mod-01 Lec-27 Discussion on Error Free Communication Over Noisy Channel
Prof. S.N. Merchant
• ##### Mod-01 Lec-28 Error Free Communication Over a Binary Symmetric Channel
Prof. S.N. Merchant
• ##### Mod-01 Lec-29 Differential Entropy and Evaluation of Mutual Information
Prof. S.N. Merchant
• ##### Mod-01 Lec-30 Channel Capacity of a Bandlimited Continuous Channel
Prof. S.N. Merchant
• ##### Mod-01 Lec-31 Introduction to Rate-Distortion Theory
Prof. S.N. Merchant
• ##### Mod-01 Lec-32 Definition and Properties of Rate-Distortion Functions
Prof. S.N. Merchant
• ##### Mod-01 Lec-33 Calculation of Rate-Distortion Functions
Prof. S.N. Merchant
• ##### Mod-01 Lec-34 Computational Approach For Calculation of Rate-Distortion Functions
Prof. S.N. Merchant
• ##### Mod-01 Lec-35 Introduction to Quantization
Prof. S.N. Merchant
• ##### Mod-01 Lec-36 Lloyd-Max Quantizer
Prof. S.N. Merchant
• ##### Mod-01 Lec-37 Companded Quantization
Prof. S.N. Merchant
• ##### Mod-01 Lec-38 Variable Length Coding and Problem Solving In Quantizer Design
Prof. S.N. Merchant
• ##### Mod-01 Lec-39 Vector Quantization
Prof. S.N. Merchant
• ##### Mod01 Lec-40 Transform Part-1
Prof. S.N. Merchant
• ##### Mod-01 Lec-41 Transform Coding Part-2
Prof. S.N. Merchant