Machine Learning Fundamentals

Master of Science in Informatics in Grenoble

Master of Science in Industrial and Applied Mathematics

Université Grenoble Alpes - Institut National Polytechnique de Grenoble

Part I | Supervised LearningThis part gives an overview of foundations of supervised learning. We will see that learning is an inductive process where a general rule is to be found from a finite set of labeled observations by minimizing the empirical risk of the rule over that set. The study of consistency gives conditions that, in the limit of infinite sample sizes, the minimizer of the empirical risk will lead to a value of the risk that is as good as the best attainable risk. The direct minimization of the empirical risk is not tractable as the latter is not derivative, hence learning algorithms find the parameters of the learning rule by minimizing a convex upper-bound (or surrogate) of the empirical risk. We present, classical strategies for unconstrained convex optimization: gradient descente, Quasi-Newton approach, and conjugate gradient descente. We present classical learning algorithms for binary classification: the perceptron, logistic regression and boosting by linking the development of these models to the Empirical Risk Minimization framework as well as the Multi-class classification paradigm. Particularly, we present Multi-Layer Perceptron as well as the back-propagation algorithm that is in use in deep learning. |
||

Part II | Unsupervised and semi-supervised LearningWe will present generative models for clustering as well as two powerful tools for parameter estimation namely Expectation-Maximization (EM) and Classification Expectation-Maximization (CEM) algorithms. In the context of Big Data, labeling observations for learning is a tedious task. Semiu-supervised paradigm aims at learining with few labeled and a huge amount of unlabeled data. In this part we review the three families of techniques proposed in semi-supervised learning, that is Graphical, Generative and Discriminant models. |

**HW1:**Part I of the exam 2013-2014. Due before October the 12th, 14:00 (hard deadline)**HW2:**Perceptron, Adaline and Logistic Regression Programs. Reports are due before November the 2nd, 14:00 (hard deadline). Sendcorresponding to your homework in the notebook. Before sending add you name to the beginning of the file :**just one single file****YOUR_NAME_LinearModels.ipynb**

**HW3:****Question 4**of the exam 2019-2020. Due before November the 16th, 14:00 (hard deadline)