EE 8371 - Information Theory
Spring 2014

---

Course Information

• Class Time : Mondays, Wednesday 12:30-1:50 pm
• Class Location : Caruth 183
• Instructor : Dinesh Rajan
• Prerequisite : Basic undergraduate course on probability. A graduate course on stochastic process might be helpful, but is not required. A high level of mathematical maturity is required.
• Textbooks: Elements of Information Theory, T. Cover and J. Thomas. Either 1st or 2nd edition will work
• Other good reference books:
1. A first course in information theory, Raymond Yeung
2. Information theory, Inference, and learning algorithms, David Mackay
• Office hours: TBD

Grading

• Home works (10%)
• Mid term Exams 2 (25% each)
• Final Exam (40%)
  The exams will be open book. However, you are not allowed to look for solutions to problems on the Internet. You are also prohibited from looking at previous years exams/homeworks and their solutions. On the homeworks, you are encouraged to work in teams; however, each one of you should turn in your solutions independently. There will be between 6-8 homeworks.

Topics

• Basic information theory
• Measures of information - Entropy, relative entropy, mutual information
• Asymptotic equipartition property
• Entropy rates of stochastic processes
• Data compression
• Channel capacity
• Differential entropy and the Gaussian channel
• Source channel separation
• Applications of Information theory
• Physics, Finance and Business, Biology
• Rate distortion theory

The depth of coverage of the last topic depends on the availability of time.

---

· Disability Accommodations: Students needing academic accommodations for a disability must first be registered with Disability Accommodations & Success Strategies (DASS) to verify the disability and to establish eligibility for accommodations. Students may call 214-768-1470 or visit http://www.smu.edu/alec/dass to begin the process. Once registered, students should then schedule an appointment with the professor to make appropriate arrangements.

· Religious Observance: Religiously observant students wishing to be absent on holidays that require missing class should notify their professors in writing at the beginning of the semester, and should discuss with them, in advance, acceptable ways of making up any work missed because of the absence. (See University Policy No. 1.9.)

· Excused Absences for University Extracurricular Activities: Students participating in an officially sanctioned, scheduled University extracurricular activity should be given the opportunity to make up class assignments or other graded assignments missed as a result of their participation. It is the responsibility of the student to make arrangements with the instructor prior to any missed scheduled examination or other missed assignment for making up the work. (University Undergraduate Catalogue)

· Student Learning Outcomes (SACS)

· Outcome (I): An ability to use the techniques, skills, and modern engineering tools necessary for engineering practice

· Outcome (II): Ability to apply knowledge of mathematics, science, and engineering

· Outcome (III): Ability to identify, formulate, and solve engineering problems

· Outcome (IV): Ability to communicate effectively

· Final Exams:  Final course examinations shall be given in all courses where they are appropriate, must be administered as specified in the official examination schedule, and shall not be administered during the last week of classes or during the Reading Period. 

---

back