Lectures for Statistical Theory and Modelling, 7.5 hp

This page contains a short description of the contents, reading instructions and additional material for each lecture.

Schedule

The course schedule can be found on TimeEdit.

Literature

The MSA listed below are section numbers from the course book Wackerley, Mendenhall and Scheaffer (2021). Mathematical Statistics with Applications, 7th edition, Cengage.

The BLprequel listed below are section numbers from a book Bayesian Learning - the prequel that I have started writing for this course.

Lecture contents

Preparatory - Basic maths (math check/self-study)
This is not a lecture, but check that you remember this prerequisite high school maths, or otherwise freshen it up at the start of the course.
Read: BLprequel 1.1-1.7.

Lecture 1 - Functions. Differentiation.
Read: BLprequel 1.8-1.14 | Slides
Notebooks and widgets: Exponential function | Logarithms | Derivatives

Lecture 2 - Optimization. Integration.
Read: BLprequel 1.15-1.16 | Notebook on function optimization | Slides
Widgets: Integrals | Common functions and their derivatives
Code: Newton-Raphson algorithm in R | Gradient Ascent algorithm in R

Lecture 5 - Joint and conditional distributions. Covariance and correlation. Bayes theorem.
Read: MSA 5.1-5.8, 5.11 | BLprequel 1.16 (double integrals) | BLprequel 7.1-7.7 (alternative to MSA) | Slides

Lecture 6 - Transformation of random variables. Monte Carlo simulation. Law of large numbers. Central limit theorem.
Read: MSA 6.1-6.4, 7.3 | Blprequel 6.1, 5.2-5.4 (alternative to MSA) | Law of large numbers notebook | central limit theorem notebook | Slides
Widgets: Law of large numbers | central limit theorem

Lecture 7 - Point estimation. Maximum likelihood. Sampling distributions.
Read: MSA 9.1-9.2, 9.3 (pages 448-451), 9.4, 9.7 | Sections 1-4 of tutorial on maximum likelihood | Slides
Widgets: Sampling distribution and Likelihood | ML - Bernoulli data | ML - Poisson data

Lecture 8 - Vectors and matrices. Linear Regression. Multivariate normal distribution.
Read: MSA A1.1-A1.7, 5.10, 11.10-11.11 | Slides
Widgets: Bivariate normal distribution

Lecture 9 - Observed and Fisher information. Maximum likelihood in large samples.
Read: BLprequel 8.4-8.6 | Slides
Widgets: Second derivative as function curvature | Likelihood and Information

Lecture 10 - Numerical optimization for maximum likelihood estimation
Read: tutorial on maximum likelihood | Slides
Code: Optim for Poisson model | Optim for Gamma model | Optim for Exponential Regression | Sampling distribution for the MLE

Lecture 11 - Logistic, Poisson regression and beyond.
Read: Sections 5-7 of tutorial on maximum likelihood | Slides
Widgets: Poisson regression

Lecture 12 - Nonlinear regression. Regularization. Time series. Autocorrelation function regression.
Read: Slides
Widgets: Simulate AR and estimate autocorrelation function

Lecture 13 - Example exam 1
Read: Some selected old exam.