Deep Learning for Multimedia
since 2019 at Technische Universität München (Lecturer)
The course Deep Learning for Multimedia (Master level) covers the methods, algorithms and underlying machine learning concepts for extracting information from audio, visual, and textual unstructured content using state-of-the art algorithms, especially deep learning based algorithms and architectures e.g. CNN, Autoencoder, LTSM. In addition, existing frameworks and libraries (e.g. Keras, Scikit-learn) and how to use them with audio, visual, and textual content countered in (multi-) media applications and services will be discussed.
The course is part of the module Applied Machine Learning, in which the information extracted using deep learning algorithms discussed in this course will be used as an input to create models for examining underlying (business) questions. The complementary course Practical Concepts of Machine Learning Data Analysis covers the algorithmic concepts of analysing empirical data to determine an abstract data model for such data. This is closely connected to Big Data and Data Mining Applications. All other practical aspects of data analysis shall also be addressed, such as cleaning data, outlier removal, handling missing data, noise removal, dimension reduction, visualization, cross-validation etc.
Deep Learning for Multimedia adresses the issue that content generated for human consumption in the form of video, text, or audio, is unstructured from a machine perspective since the contained information is not readily available for processing. Information extraction from unstructured data describes therefore how one can extract the salient information from generic content in order to generate a descriptive structured representation. The thus created meta-data can then be further processed automatically, in particular for creating models explaining or predicting samples e.g. in recommendation systems.
Integral part of the course is a project, in which the students learn how to apply deep learning algorithms to extract information from video clips and the practical issues that need to be considered.
Past courses
Quality of Experience – Concepts and Applications
2017-2018 at Technische Universität München (Lecturer)
The course Quality of Experience – Concepts and Applications (Master level) covers the concepts and applications of Quality of Experience (QoE) and the corresponding algorithms and assessment methodologies, but also how QoE complements in particular multimedia signal processing. The focus will be on audio-visual stimuli in the context of telecommunication- and multimedia-related applications, systems and services.
The course is part of the module Data Analysis for Quality of Experience Assessments, in which QoE prediction models will be created based on the concepts, methods and algortihms introduced in this course. Data Analysis covers the algorithmic concepts of analysing empirical data to determine an abstract data model for such data. This is closely connected to Big Data Applications. The course will focus on a subset of algorithms, relevant for the task of QoE assessments. All other practical aspects of data analysis shall also be addressed, such as cleaning data, outlier removal, handling missing data, noise removal, dimension reduction, visualization, cross-validation etc.
QoE described the quality or “goodness” of (audio-visual) stimuli as perceived by human observers in a specific context. It allows us to assess the influence of processing steps not only with respect to the physical properties of the signal itself e.g. SNR, but also how any degradations influence the observers’ experience by taking into account the psychophysical properties of human perception.
Integral part of the course is a project, in which the students implement QoE algorithms and conduct QoE assessment task (Evaluation report 2018).
Digital Broadcast Engineering
2015-2016 at Technische Universität München (Lecturer)
The course Digital Broadcast Engineering (Master level) covers both traditional linear digital broadcasting technology especially DVB, but also non-linear broadcasting via (adaptive) streaming, in particular DASH, and interactive TV. The main goal of the course is to provide the students with an understanding of today’s broadcasting landscape and the components in digital broadcasting systems. The students learn how concepts from channel coding, source coding, communications engineering and communication networks are used to design robust broadcasting systems, but also that broadcasting can be Internet-based via IPTV or OTT.
Integral part of the course is a project, in which the students implement a prototypical interactive TV broadcasting chain, from a DVB-based play-out using a table generator, multiplexer and modulator, to interactive TV applications using the HTML-based HbbTV. This allows the students to apply the theoretical principles to a practical use-case that represents the ongoing convergence of the broadcast and broadband worlds.
The results of the course’s evaluation by the participating students was above the average of all lectures at the Department of Electrical Engineering and Information Technology (Evaluation report 2015,Evaluation report 2016).
Digital Video
2008-2013 at Technische Universität München (Teaching Assistant)
The course Digital Video (Bachelor level) consists of a lecture integrated with tutorials and Matlab programming problems, a poster presentation, and a hands-on film team project. Together with Klaus Diepold and Martin Rothbucher, I developed this new version based on the previous lecture-only format of Digital Video. The main goal we achieved was the increased practical application and transfer of theoretical knowledge to practical problems, in this case by applying and transferring basic concepts of digital video processing to the creation of a short 10-15 minute film. Moreover, the students gained first project management and extended team work experience during the course.
For the support and supervision, Martin Rothbucher and I received the TUM student’s association best teaching award for the course in 2011. Also the results of the evaluation by the students of the course were above average in each semester.
We extended this concept in 2011 to the interdisciplinary “Zwei in einem Boot” (Two in one boat) by including students studying towards a master degree in education from TUM’s School of Education. In this extended concept, the education students were supervising the engineering students as part of their course work: on the one hand supporting them in the soft-skills aspects of the project, but on the other hand also practising their own skills and competence. Hence both student groups benefited from this joint approach.
“Zwei in einem Boot” was subsequently awarded a prize (50000 EUR) for innovative teaching concepts in 2012 by the Stifterverband der Deutschen Wirtschaft, one among only 15 selected concepts out of 210 applications Germany-wide.