Title: ML4QS

Q: Five-factor framework of motivations
A: self-healing, self-discipline, sef-design, self-association, self-entertainment

Q: For a variable X:
* subscript - [...]
* superscript - [...]
* instance is [...]
A: single instance; observed variable; a vector of observations of variables

Q: model is [...] when it [...]
A: shattered; can represent every possible labelling

Q: VC Dimension
A: the max number of input vectors (points) that can be shattered

Q: all hypothesis sets with [...] are [...]
A: finite VC-dimension; PAC learnable

Q: In-sample error
A: error rate for data in the training set

Q: Out of sample error
A: error rate on the new data set ("generalization error")

Q: Learning is possible if the [...] is a good estimator of the [...]
A: in-sample error; out-of-sample error

Chauvenet's criterion:
1. Take mean and std for attribute
2. Use as parameters for normal distribution
3. For each instance, compute probability of observation
4. Outlier if probability does not match

Mixture models:
1. Assume data can be described with K normal distributions
2. Maximize the product of probabilities of observing the attribute values
3. Outliers are points with lowest probabilities

Q: Simple distance-based approach: outlier if [...] is [...] from that point
A: more than a certain fraction of other points in the dataset; more than a certain distance

Q: Local outlier factor takes [...] into account (only looks at [...])
A: density; set of neighbors

Q: Distribution-based outlier detection
A: Chauvenet's criterion, mixture models

Q: Distance-based outlier detection
A: Simple, local outlier factor

Q: Imputation methods
A: mean, median, categorical mode, linear regression, Kalman filter

Kalman filter
- needs temporal data
- replaces outliers with new values
- uses historical data for state, adds white noise

Principal component analysis:
- find new features explaining most of variability in data
- select number of components based on explained variance

Q: Categorical pattern types
A: co-occurring, in succession

Q: Support level
A: how often the pattern occurs compared to the number of instances

Q: Generating categorical patterns:
- [...] first instance for support ([...] history)
- do not consider [...]
A: ignore; insufficient; how often

Q: For a window size of x, we need [...] frequencies
A: x+1

Q: [...] means [...] due to [...]
A: Larger window size; more points removed; overlap

Q: [...] sillhouette: some points in cluster are [...]
A: Negative; closer to a different cluster

Q: Predictive models without notion of time
A: feedforward NNs, support vector machines, k-nearest neighbor, decision trees, naive bayes, ensembles

Q: Predictive models with notion of time
A: time series analysis, recurrent NNs, echo state NNs, dynamical systems models

Q: Feedforward NN calculates output from [...] and [...] through an [...]
A: inputs; weights; activation function

Q: Signum function
A: 1 if positive, 0 if 0, -1 if negative

Q: Identity function
A: \(\psi(v) = v\)

Q: Convolutional NNs
A: Multi-layer, contain layers that automatically identify features in input space

Q: Support vector machines
A: Binary classification, try to find the hyperplane with maximum distance between two classes

Q: K-nearest neighbor
A: Look at k closest points from training set, assign class value based on target values of the neighbors

Decision trees: nodes are decision points, branches are attribute values, leaves are assigned result

Front: Naive Bayes
Back: Uses conditional probability for classification

Ensembles:
- bagging: take m samples of size n with replacement, generate model for each sample, combine output using majority voting (e.g. random forest)
- boosting: iteratively creates models focusing on areas where previous models failed

Q: Regularisation
A: Punish the complexity of the model

Q: Lagged autocorrelation
A: Measures corelation between a time series and a shifted variant of itself

Q: ARIMA
A: Autoregressive integrated moving average

Q: Reinforcement learning
A: Tries to find actions to maximize a reward at a later point in time

Q: SARSA
A: State-action-reward-state-action. Pick actions same way at each step -- on-policy

Q: [...] solve the problem of [...]
A: LSTM networks; long term memory

Q: Echo state network
A: Has a reservoir of randomly collected neurons, only learn output weights

Q: Dynamical systems models
A: Express differential equations between attributes and targets, based on domain knowledge

Q: Q-learning
A: Like SARSA, but don't do next action before updating values, assume highest value (off-policy)

Q: U-tree algorithm
A: For converting continuous space to discrete -- create splits