# Lectures

## Please Note

Since Winter 2021-22, we are back to normal lectures, and since Summer 2022, we are not required to provide the lecture videos anymore. The record of last semesters are available on FAUtv; however, the contents might be subject to some minor changes in each semester. If you are following the lectures based on the videos on FAUtv, make sure to inform yourself about possible minor changes

If you want to discuss lecture-related questions, please write me in advance an email. We could then arrange a time slot and discuss your questions

# List of Lectures per Semester

## Winter 2022-23

Information Theory and Coding

Checkout the lecture at StudOn | UnivIS

The lectures are consistent with the textbook "Information Theory, Inference, and Learning Algorithms". The PDF version of the book can be found here

A tutorial manuscript has been also developed for this lecture. The last version can be downloaded here

The final exam is written and open-book. The date is announced by the examination office

The lectures given in Winter 2021-22 were recorded. The videos are available on FAUtv. Please note that there might be some minor changes compared to the last semester; thus, make sure to inform yourself about the possible changes

Tutorials on Random Matrix Theory

## Summer 2022

Compressed Sensing

Tutorials on Mobile Communications

## Winter 2021-22

We are back to in-person sessions

Information Theory and Coding

Checkout the lecture at StudOn | UnivIS

The lectures are consistent with the textbook "Information Theory, Inference, and Learning Algorithms". The PDF version of the book can be found here

A tutorial manuscript has been also developed for this lecture. The last version can be downloaded here

The final exam is written and open-book. The date is announced by the examination office

The lectures are recorded and available on FAUtv

Tutorials on Random Matrix Theory

## Summer 2021

The lectures in this semester are given per Zoom

Compressed Sensing

Tutorials on Mobile Communications

## Winter 2020-21

The lectures in this semester are given per Zoom

Tutorials on Information Theory and Coding

Tutorials on Random Matrix Theory

# Course Descriptions

## Information Theory and Coding

This is a basic lecture on information theory. In principle, attending the lecture requires taking no prerequisites; however, a quick review of basic concepts from probability theory and stochastic processes is strongly suggested. The lecture is consistent with the textbook *Information Theory, Inference and Learning Algorithms* by David J.C. MacKay in addition to few selected topics. The textbook can be accessed freely online here.

The main contents covered in the lecture are

Fundamentals of Information Theory: Definition of the basic concepts like entropy and mutual information

Basics of Bayesian Inference: Introduction to Bayesian inference, its connections to information theory and the applications

Source Coding: Introduction to block coding and typicality, proof of the first theorem of Shannon, well-known source coding algorithms such as Huffman coding, Lempel-Ziv coding and Arithmetic codes, as well as the Burrows-Wheeler transform

Channel Coding: The mathematical model for a communication channel, proof of Shannon's channel coding theorem, basics of channel coding, a quick introduction on low-density parity check (LDPC) codes

Basics of Message Passing: Fundamentals of message passing, the sum-product algorithm, applications of message massing in Bayesian inference

For this lecture, I have further prepared a tutorial manuscript. You can download the last version of the manuscript here.

## Compressive Sensing

This is a primary lecture on compressive sensing. Prerequisites for this lecture are lectures on information theory and stochastic signal processing. The lecture consists of two major parts: The first part covers classical concepts in compressive sensing and is closely consistent with the textbook A Mathematical Introduction to Compressive Sensing by Holger Rauhut and Simon Foucart. The second part focuses in information-theoretic viewpoint and is prepared by collecting materials from the literature. The lecture-notes are available on GitHub.

The list of contents is as follows:

Basics of Sparse Recovery: The problem of recovering a sparse vector from an under-determined system of equations, the optimal noise-free approach, i.e., zero-norm minimization, the fundamental necessary and sufficient conditions for recovery

Well-known Sparse Recovery Algorithms: The well-known convex relaxations such as LASSO and basis pursuit (BP), iterative matching pursuit algorithms, such as orthogonal matching pursuit (OMP), compressive sampling matching pursuit (CoSaMP) and subspace pursuit, thresholding algorithms, such as iterative hard and soft thresholding

Recovery from Noisy Observations: The method of regularized least-squares (RLS), the Dantzig approach for sparse recovery, extension of iterative algorithms to recovery from noisy samples

Properties of Sensing Matrices: General approach for deriving a recovery guarantee, the null space property and related guarantees, coherence of a matrix and related recovery guarantees, restricted isometry property (RIP) and sufficient constraints for successful sparse recovery

Random Matrices in Compressive Sensing: Probabilistic formulation of RIP for random matrices, RIP of random Gaussian matrices

Information-Theoretic Compressive Sensing: Optimal approach for compressive sensing from information-theoretic viewpoint, the optimal compression bound for noise-less compressive sensing

Bayesian Framework for Compressive Sensing: A Bayesian approach for sparse recovery in noisy compressive sensing, maximum-a-posterior techniques, risk minimization techniques, an introduction to approximate message passing (AMP)

## Tutorials on Random Matrix Theory

This tutorial is to cover the concepts of the Random Matrix Theory lecture given by Prof. Ralf R. Müller in winter semesters. We address three major parts; namely,

Classical random matrix theory, where we practice concepts like

Circle laws

Classical definitions, such as Stieltjes transform, R-transform and S-transform

Free probability theory, where we practice

Concept of freeness

Free central limit theorem

Determining polynomial expansions of free random matrices

Large-system analysis via the replica method

How to use the replica method for asymptotic analysis in communications and signal processing

Sample large-system analysis: The problems of vector precoding, CDMA detection and sparse recovery

For better understanding of the third part, we have some introductory lectures in the tutorials going through the following topics:

Introduction to statistical mechanics

The statistical mechanical interpretation of information-theoretic metrics

Introduction to the spin glass theory and the analysis via the replica method

The random energy model (REM) and its replica analysis

For the third parts of tutorials, we use some contents from the textbook Statistical Physics and Information Theory by Neri Merhav.

## Tutorials on Mobile Communications

This is a tutorial course on mobile communications being consisted with the lecture Mobile Communications given by Prof. Dr.-Ing. Ralf R. Müller in summer semesters. For this tutorial, I have prepared a tutorial manuscript whose last version can be downloaded here. In the tutorial sessions we go through the following concepts:

Basic concepts of mobile communication systems and their classifications according to different criteria

Principles of antenna and array-antennas

Mathematical description of time-variant radio channels

Concept of diversity and various diversity techniques

Multiplexing and duplexing schemes for multiuser communications

Modulation schemes used in wireless systems

Basics of channel coding techniques with the focus on their adaptation for communication over wireless channels

Fundamentals of the GSM system