Machine learning

Exam - cheat sheet available for formulas!

• Introduction
  • What is ML?
  • Supervised ML
    • Classification
    • Regression
  • Unsupervised ML
  • What isn’t ML?
• Methodology
  • Performing an experiment
    • What if you need to test many models?
    • The modern recipe
    • Cross-validation
  • What to report
    • Classification
      • What’s a good error (5%)?
      • Performance metrics
        • Confusion matrix (contingency table)
        • Precision and recall
    • Regression
    • Errors & confidence intervals
  • The no-free-lunch theorem and principle
  • Cleaning your data
    • Missing data
    • Outliers
    • Class imbalance
  • Choosing features
  • Normalisation & standardisation
    • Normalisation
    • Standardisation
    • Whitening
  • Dimensionality reduction
• Linear models
  • Defining a model
  • But which model fits best?
    • Mean squared error loss
    • Optimization & searching
      • Black box optimisation
        • Random search
        • Simulated annealing
        • Parallel search
        • Branching search
      • Gradient descent
      • Classification losses
        • Least-squares loss
  • Neural networks (feedforward)
    • Overview
    • Classification
    • Dealing with loss - gradient descent & backpropagation
  • Support vector machines (SVMs)
  • Summary of classification loss functions
• Probability
  • Probability basics
    • Probability theory
  • (Naive) Bayesian classifiers
  • Logistic “regression” (classifier)
  • Information theory
    • Maximum likelihood
    • Normal distributions (Gaussians)
      • 1D normal distribution (Gaussian)
      • Regression with Gaussian errors
      • n-D normal distribution (multivariate Gaussian)
      • Gaussian mixture model
    • Expectation-maximisation
• Deep learning
  • Deep learning systems (autodiff engines)
    • Tensors
    • Functions on tensors
  • Backpropagation revisited
    • Multivariate chain rule
    • Backpropagation with tensors - matrix calculus
  • Making deep neural nets work
    • Overcoming vanishing gradients
    • Minibatch gradient descent
    • Optimizers
      • Momentum
      • Nesterov momentum
      • Adam
    • Regularizers
      • L2 regularizer
      • L1 regulariser
      • Dropout regularisation
  • Convolutional neural networks
  • Deep learning vs machine learning
  • Generators
  • Generative adversarial networks
    • Vanilla GANs
    • Conditional GANs
    • CycleGAN
    • StyleGAN
    • What can we do with a generator?
  • Autoencoders
    • Turning an autoencoder into a generator
  • Variational autoencoders
• Tree models and ensembles
  • Tree models
    • Decision trees (categorical)
    • Regression trees (numeric)
    • Generalization hierarchy
  • Ensembling methods
    • Bagging
    • Boosting
      • AdaBoost (binary classifiers)
      • Gradient boosting
    • Stacking
• Sequences, models for sequential data
  • Sequences
  • Markov models
  • Embedding models
  • Recurrent neural networks
  • LSTMs
• Matrix models
  • Recommender systems
  • Matrix factorization
    • Bias control
    • The ‘cold start’ problem
  • Graph models
  • Validating embedding models
• Reinforcement learning
  • What is reinforcement learning?
  • Approaches
    • Random search
    • Policy gradient
    • Q-learning
  • Alpha-stuff
    • AlphaGo
    • AlphaZero
    • AlphaStar

Programming reference