Applied Machine Intelligence / Deep Learning for Multimedia

since 2019 at Technische Universität München (Lecturer)

The course Deep Learning for Multimedia (Master level) covers the methods, algorithms and underlying machine learning concepts for extracting information from audio, visual, and textual unstructured content using state-of-the art algorithms, especially deep learning based algorithms and architectures e.g. CNN, Autoencoder, LTSM. In addition, existing frameworks and libraries (e.g. Keras, Scikit-learn) and how to use them with audio, visual, and textual content countered in (multi-) media applications and services will be discussed.

The course is part of the module Applied Machine Learning, in which the information extracted using deep learning algorithms discussed in this course will be used as an input to create models for examining underlying (business) questions. The complementary course Practical Concepts of Machine Learning Data Analysis covers the algorithmic concepts of analysing empirical data to determine an abstract data model for such data. This is closely connected to Big Data and Data Mining Applications. All other practical aspects of data analysis shall also be addressed, such as cleaning data, outlier removal, handling missing data, noise removal, dimension reduction, visualization, cross-validation etc.

Deep Learning for Multimedia adresses the issue that  content generated for human consumption in the form of video, text, or audio, is unstructured from a machine perspective since the contained information is not readily available for processing. Information extraction from unstructured data describes therefore how one can extract the salient information from generic content in order to generate a descriptive structured representation. The thus created meta-data can then be further processed automatically, in particular for creating models explaining or predicting samples e.g. in recommendation systems.

Integral part of the course is a project, in which the students learn how to apply deep learning algorithms to extract information from video clips and the practical issues that need to be considered.