Information Theory

Subject ELEN90030 (2014)

Note: This is an archived Handbook entry from 2014.

Credit Points: 12.50
Level: 9 (Graduate/Postgraduate)
Dates & Locations:

This subject has the following teaching availabilities in 2014:

Semester 2, Parkville - Taught on campus.
Pre-teaching Period Start not applicable
Teaching Period not applicable
Assessment Period End not applicable
Last date to Self-Enrol not applicable
Census Date not applicable
Last date to Withdraw without fail not applicable


Timetable can be viewed here. For information about these dates, click here.
Time Commitment: Contact Hours: Up to 36 hours of lectures
Total Time Commitment:

200 hours

Prerequisites:

Enrolment in a research higher degree(Masters or PhD) in Engineering

Corequisites:

None

Recommended Background Knowledge:

None

Non Allowed Subjects:

None

Core Participation Requirements:

For the purposes of considering request for Reasonable Adjustments under the Disability Standards for Education (Cwth 2005), and Students Experiencing Academic Disadvantage Policy, academic requirements for this subject are articulated in the Subject Description, Subject Objectives, Generic Skills and Assessment Requirements of this entry. The University is dedicated to provide support to those with special requirements. Further details on the disability support scheme can be found at the Disability Liaison Unit website: http://www.services.unimelb.edu.au/disability/

Contact

Dr. Marcus Nathan Brazil

Email: brazil@unimelb.edu.au

Subject Overview:

AIMS

Information Theory provides the fundamental backbone of reliable communications, reliable data storage, and data compression. This subject provides the rigorous basis of `information', showing it to have deep links to randomness, the ability to reduce data to its essence, and to the ultimate limits to communication.

INDICATIVE CONTENT

This subject is aimed at postgraduate (research) students. The subject material covers the core topics of Information Theory including: Shannon entropy, Mutual Information, lossless and lossy source coding, Shannon's celebrated channel capacity and channel coding theorem, differential entropy and the Gaussian channel. In addition other topics are selected from rate distortion theory, network information theory, distributed source coding, Kolmogorov Complexity, and possibly other related applications in communications theory and statistical inference. Technically the subject combines probabilistic models of idealised communication and the mathematics of applied probability.

Learning Outcomes:

Intended Learning Outcomes (ILO)

Having completed this subject it is expected that the student be able to:

  1. understand and apply the Shannon notion of entropy to model data coding and communication situations,
  2. manipulate Information Theoretic identities and to understand their intuitive meanings,
  3. analyse and understand the classic channel models and their capacity analysis in particular Discrete Memoryless Channels, the Gaussian channel, and the critical ideas of random coding and joint typicality,
  4. understand the limits to data compression and be able to design codes which can approach the ultimate Shannon limit,
  5. analyse the nature of information flow in other systems covered in the course such as rate distortion theory and network information theory.



Assessment:
  • Continuous assessment of homework assignments, not exceeding 30 pages in total over the semester, worth 40%;
  • One written examination not exceeding three hours at the end of semester, worth 60%.

Hurdle requirement: Students must pass the written exam to pass the subject.

Intended Learning Outcomes (ILOs) 1and 2 are assessed in the final exam and submitted assignments.

Prescribed Texts:

None

Breadth Options:

This subject is not available as a breadth subject.

Fees Information: Subject EFTSL, Level, Discipline & Census Date
Generic Skills:
  • Ability to apply knowledge of basic science and engineering fundamentals;
  • In-depth technical competence in at least one engineering discipline;
  • Ability to undertake problem identification, formulation and solution;
  • Ability to utilise a systems approach to design and operational performance;
  • Expectation of the need to undertake lifelong learning, capacity to do so;
  • Capacity for independent critical thought, rational inquiry and self-directed learning;
  • Intellectual curiosity and creativity, including understanding of the philosophical and methodological bases of research activity;
  • Openness to new ideas and unconventional critiques of received wisdom;
  • Profound respect for truth and intellectual integrity, and for the ethics of scholarship
Notes:

LEARNING AND TEACHING METHODS

The subject is delivered through lectures and homework assignments

INDICATIVE KEY LEARNING RESOURCES

Students are provided with lecture notes, including worked examples, assignment problems, and recommended reading lists comprising textbooks and journal articles.

CAREERS / INDUSTRY LINKS

Exposure to research literature and the rigour expected at the level of postgraduate study.

Related Course(s): Master of Philosophy - Engineering
Ph.D.- Engineering

Download PDF version.