Sie sind hier: Startseite › Studium und Lehre › Sommersemester 2020 › Seminar Mathematical Foundations …

Informationen zum Seminar: Mathematical Foundations of Statistical Learning (SS 2020)

Dozent: Prof. Dr. Angelika Rohde

Assistent: Dario Kieffer

Zeit, Ort: Di 10-12, SR 127, Ernst-Zermelo-Straße 1, online

Vorbesprechung: Do 06.02.2020, 14:00 Uhr, Raum 232, Ernst-Zermelo-Str. 1

 

Aktuelles

 
Achtung: Aus aktuellem Anlass wird das Seminar online stattfinden.
Die Vorträge werden mit Hilfe von Folien per Bildschirmübertragung durchgeführt.

Das Seminar kann wahlweise in deutscher oder englischer Sprache abgehalten werden.
  • Donnerstag, 06. Februar 2020, 14:00 Uhr: Vorbesprechung und Themenvergabe.

 

Vorträge

  • Dienstag, 12. Mai 2020, 10:00 Uhr: Einführung in Statistical Learning, basierend auf Vapnik, V. (1999).

Folien: pdf.     Handout: pdf.

  • Dienstag, 26. Mai 2020, 10:00 Uhr: Support Vector Machines, basierend auf Blanchard, G. & Bousquet, O. & Massart, P. (2008).

Folien: pdf.     Programmcode: zip.

  • Dienstag, 02. Juni 2020, 10:00 Uhr: Vergleich von Koltchinskii, V. & Panchenko, D. (2002) und Steinwart, I. & Scovel, C. (2007), Teil 1.

Folien: pdf.     Ausarbeitung: pdf.

  • Dienstag, 09. Juni 2020, 10:00 Uhr: Vergleich von Koltchinskii, V. & Panchenko, D. (2002) und Steinwart, I. & Scovel, C. (2007), Teil 2.

Folien: pdf.     Ausarbeitung: pdf.

  • Dienstag, 23. Juni 2020, 10:00 Uhr: Nichtparametrische Regression und neuronale Netze basierend auf Schmidt-Hieber, J. (2020).

Folien: pdf.     Beweis Theorem 2: pdf.

  • Dienstag, 21. Juli 2020, 10:00 Uhr: Bachelorvortrag über Risk Bounds for Statistical Learning.

Folien: pdf.

  • Dienstag, 28. Juli 2020, 10:00 Uhr: Bachelorvortrag über M-Schätzung und MLE-Theorie in logistischer Regression.

Folien: pdf.

 

Inhalt

 

Statistical Learning Theory has demonstrated its usefulness by providing the ground for developing successful and well-founded learning algorithms. The usual framework is as follows. We consider a space X of possible inputs (instance space) and a space Y of possible outputs (label sets). The product space X×Y is assumed to be measurable and is endowed with an unknown probability measure. Based on n independent input-output pairs (X1,Y1), …, (Xn,Yn) sampled according to this probability measure, the goal of a learning algorithm is to pick a function g in a space G of functions from X to Y in such a way that this function should capture as much as possible the relationship (which may not be of a functional nature) between the random variables X and Y.

In this seminar, we study this problem in a mathematically rigorous way, particularly focusing on recent learning algorithms. The theory of empirical processes will be shown to play a fundamental role in their analysis.

 

Vorkenntnisse

 

  • Notwendige Vorkenntnisse: Analysis und Grundlagen der Stochastik
  • Nützliche Vorkenntnisse: Wahrscheinlichkeitstheorie

 

Literatur


Einführung in Statistical Learning

  • Vapnik, V. (1999). An Overview of Statistical Learning Theory. IEEE Transactions on Neural Networks 10, 988-999. pdf
  • Bousquet, O. & Boucheron, S. & Lugosi, G. (2004). Introduction to Statistical Learning Theory. Advanced Lectures on Machine Learning, 169-207. pdf
  • Hastie, T. & Tibshirani, R. & Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. pdf

 

Weiterführende Literatur

  • Koltchinskii, V. & Panchenko, D. (2002). Empirical Margin Distributions and Bounding the Generalization Error of Combined Classifiers. The Annals of Statistics 30, 1-50. pdf
  • Tsybakov, A. (2004). Optimal Aggregation of Classifiers in Statistical Learning. The Annals of Statistics 32, 135-166. pdf
  • Massart, P. & Nédélec, É. (2006). Risk Bounds for Statistical Learning. The Annals of Statistics 34, 2326-2366. pdf
  • Steinwart, I. & Scovel, C. (2007). Fast Rates for Support Vector Machines using Gaussian Kernels. The Annals of Statistics 35, 575-607. pdf 
  • Blanchard, G. & Bousquet, O. & Massart, P. (2008). Statistical Performance of Support Vector Machines. The Annals of Statistics 36, 489-531. pdf
  • Wang, Z. & Liu, H. & Zhang, T. (2014). Optimal Computational and Statistical Rates of Convergence for Sparse Nonconvex Learning Problems. The Annals of Statistics 42, 2164-2201. pdf
  • van Erven, T. & Grünwald, P. & Mehta, N. & Reid, M. & Williamson, R. (2015). Fast Rates in Statistical and Online Learning. Journal of Machine Learning Research 16, 1793-1861. pdf
  • Bauer, B. & Kohler, M. (2019). On Deep Learning as a Remedy for the Curse of Dimensionality in Nonparametric Regression. The Annals of Statistics 47, 2261-2285. pdf
  • Cai, T. & Wei, H. (2020). Transfer Learning for Nonparametric Classification: Minimax Rate and Adaptive Classifier. The Annals of Statistics, to appear. pdf
  • Schmidt-Hieber, J. (2020). Nonparametric Regression using Deep Neural Networks with ReLu Activation Function. The Annals of Statistics (discussion paper), to appear. pdf

 

Ergänzende Literatur

  • Dümbgen, L. (2017). Empirische Prozesse. Vorlesungsskript, Universität Bern. pdf