Statistical and Machine Learning

The course has two parts: statistical learning and machine learning. The first part focuses on applied aspects of statistical learning while also including the underlying algorithms. Supervised methods are addressed with particular emphasis on classification, including logistic regression, classification trees, linear and quadratic discriminant analysis, K-nearest neighbour, and support vector machines. Regression methods, such as linear regression, splines, generalised additive models, and regression trees, are also included. In addition, unsupervised methods, such as principal component analysis, k-means and hierarchical clustering, are covered. Model validation is addressed through cross-validation and bootstrap methods. The course also discusses regularisation in model selection, analysis of high-dimensional data, and the improvement of predictive performance through methods such as model averaging, bagging, and boosting.
The second part introduces machine learning with a focus on neural networks and deep learning. The perceptron is introduced as a basic element for linear separability, and its limitations in classification problems are discussed. After this, various activation functions and the use of the sigmoid perceptron to handle non-linear problems are covered. The course also includes different types of learning paradigms, such as supervised, unsupervised, and reinforcement learning. Feedforward neural networks and algorithms for backpropagation are covered, as well as recurrent neural networks (RNNs). The course concludes with an overview of deep learning, where fundamental principles and various types of architectures are discussed in relation to their applicability to practical problems.