EE 6341: Information Theory
Fall 2008
|
Instructor
|
Aria Nosratinia,
ECSN 4.208, Tel: 972-883-2894
|
|
Time
|
Mon-Wed. 4:00-5:15pm
Cancelled classes: October 29, December 1, 3. Makeup classes
will be announced.
|
|
Place
| ECSS 2.312
|
|
Textbooks
|
Cover and Thomas, Elements of Information Theory, 2nd Edition,
John Wiley (required)
Gallager, Information Theory and Reliable Communication, John
Wiley (recommended)
|
|
Grading
|
Midterms (20%+20%), Class Project (40%), Homeworks (10%),
class participation (10%)
|
|
Important Dates:
|
Midterm 1: Sept. 24 Midterm 2: October 27, project presentations
Dec 5,8.
|
|
Office Hours
|
Mon-Wed 3-3:50pm, otherwise by appointment
|
|
Course Notes
|
Will be made available via WebCT
|
|
Teaching Assistant
|
Balkan Kecicioglu
|
Information theory has been responsible for the key ideas behind cell
phones, music CD's, portable music palyers like ipod, HDTV cable and
sattelite television, as well as high-capacity hard disks on computers
and laptops. The data compression, communication, and storage that is
key to the operation of all these devices started from ideas developed
in infomration theory.
This course is designed for the beginning graduate student in the areas
of communications or signal processing. It will illuminate the
underlying reasons for many of the topics learned in other courses, and
prepare the student for high-quality research.
The background needed for this course is limited to basic
calculus and the ability to calculate simple probabilities.
The grading is based on homeworks, midterms, and a course
project. For the course project, the students will choose, with the
instructor's assistance, paper or papers to read, understand, and
analyze, which they will eventually present to the entire class.
Suggested Projects
An interesting movie
about Shannon, the father of information theory.
Contents:
- Information, Entropy, and AEP
- Entropy and mutual information
- Chain rules and information inequalities
- Markov Chains and entropy rate
- Typical sets and Asymptotic Equipartition Property (AEP)
- Data Compression
- Compression and entropy
- Kraft inequality
- Huffman Codes
- Arithmetic coding
- Channel Capacity
- Jointly typical sequences
- Discrete channel capacity and Shannon's 2nd theorem
- Source-channel separation
- Differential Entropy
- Gaussian channel capacity
- Continuous (waveform) channels
- Capacity and Feedback
- Error exponent analysis
- The Wireless Channel
- Implications of the Gaussian Capacity Formula
- Fading and Outage Capacity
- Diversity and Multiplexing
- The MIMO channel
- Introduction to Network Information Theory
- The multiple-access channel
- The broadcast channel
- The relay channel
- The Slepian-Wolf problem
- The max-flow min-cut bound
- Information Theory and Statistics (optional seminar series)
- Method of types
- Large deviations
- Chernoff Stein Lemma
- Fischer information and Cramer-Rao inequality
Aria Nosratinia
Last modified: August 2008