Machine Learning Fundamentals
Master of Science in Informatics in Grenoble
Artificial Intelligence and the Web option
Université Grenoble Alpes
Part I  Supervised Learning This part gives an overview of foundations of supervised learning. We will see that learning is an inductive process where a general rule is to be found from a finite set of labeled observations by minimizing the empirical risk of the rule over that set. The study of consistency gives conditions that, in the limit of infinite sample sizes, the minimizer of the empirical risk will lead to a value of the risk that is as good as the best attainable risk. The direct minimization of the empirical risk is not tractable as the latter is not derivative, hence learning algorithms find the parameters of the learning rule by minimizing a convex upperbound (or surrogate) of the empirical risk. We present, classical strategies for unconstrained convex optimization: gradient descente, QuasiNewton approach, and conjugate gradient descente. We present classical learning algorithms for binary classification: the perceptron, logistic regression and boosting by linking the development of these models to the Empirical Risk Minimization framework as well as the Multiclass classification paradigm. Particularly, we present MultiLayer Perceptron as well as the backpropagation algorithm that is in use in deep learning. 

Part II  Unsupervised and semisupervised Learning We will present generative models for clustering as well as two powerful tools for parameter estimation namely ExpectationMaximization (EM) and Classification ExpectationMaximization (CEM) algorithms. In the context of Big Data, labeling observations for learning is a tedious task. Semiusupervised paradigm aims at learining with few labeled and a huge amount of unlabeled data. In this part we review the three families of techniques proposed in semisupervised learning, that is Graphical, Generative and Discriminant models. 