EE 7V84: Source Coding and Compression

Spring 2001

Instructor	Aria Nosratinia, EC 2.514, Tel: 972-883-2894
Time	Tue-Thu. 5:30-6:45pm
Place	TBA
Textbook	Gersho & Gray, Vector Quantization and Signal Compression, Kluwer
Grading	Midterm-1 (15%), Midterm-2 (30%), Final Exam (40%), Homeworks (10%), class participation (5%)
Prerequisite	One semester of advanced random processes required. One-semester background in Information theory recommended (discuss with instructor).
Office Hours	Tue-Thu. 4:30-5:20pm

Efficient communication and storage of information is critical to all aspects of science and technology, especially engineering. Signals, in their original form, are analog and their description in general requires an infinite amount of information. To reduce the requirements on communication bandwidths (or alternatively, the length of storage media), signals are represented in approximate (quantized) form.

This course explores the theory and practice of quantization and compression of signals. Source coding is part of the general theory of communication, and is closely related to proability, random processes, and information theory, as well as signal processing.

Overview
- Information and entropy
- Important inequalities in information theory
- Entropy coding
  - Prefix codes
  - Huffman coding
  - Arithmetic coding
  - Ziv-Lempel codes
- Introduction to rate-distortion theory
- Elementary Estimation Theory
- Linear Prediction, the Levinson-Durbin Algorithm
Scalar Quantization
- Encoder-decoder decomposition of quantizers
- Performance of scalar quantizers
- Asymptotic theory, Bennet Formulae
- Optimality conditions
- Properties of optimal quantizers
- The Lloyd-Max algorithm
Vector Quantization
- Structural properties
- Performance
- High resolution theory
- Optimality conditions
- Implications of optimality
- The generalized Lloyd algorithm
- Entropy-constrained VQ
- Drawbacks: delay, complexity, and curse of dimensionality
Predictive Coding
- Difference Quantization
- Fundamental theorem of predictive quantization
- Performance bounds
Transform Coding
- Motivation
- The bit allocation problem
- Optimality and the Karhunen-Loeve transform
- Performance gains
- Other transforms
Extensions of Vector Quantization
- Tree-structured VQ
- Classified VQ
- Other constrained VQ
- Scalar Vector Quantization (SVQ)
- Trellis Coded Quantization (TCQ)

Aria Nosratinia

Last modified: January 2001

EE 7V84: Source Coding and Compression

Spring 2001

Contents: