**Examiner:** Prof. Nöth

**Atmosphere:** Friendly.

**Questions:**

K Nearest Neighbor (from exercises)
  * Q: He drew a coordinate system with some points. What step is necessary before applying KNN itself?
  * A: Normalize the data, e.g. to [-1, 1].
  * Q: How does it work?
  * Q: How did we implement it?
  * Q: We learned of something that is related to KNN ...
  * A: Bayes Classifier.

Bayes
  * Q: Formulate the Bayes rule and name all parts.
  * Q: Formulate Decision Function.
  * Q: Relation to KNN:
  * A: Error is at most twice of Bayes (not the classifier) Error Rate (see http://cseweb.ucsd.edu/~elkan/151/nearestn.pdf for more info).
  * A: Bayes Classifier with the identity covariance matrix is the same as Nearest Neighbour.
  * Q: How can you calculate the class conditional pdf?
  * A: Assume gaussian distribution and perform Maximum Likelihood.
  * Q: What do you do if the distribution is not known?
  * A: GMM.

Gaussian Mixture Models
  * Q: What are the parameters?
  * A: Mean, covariance matrix and relative portion of samples for every distribution.
  * Q: How can you initialize the parameters?
  * A: Take random samples and calculate parameters with their help.
  * Q: To what can this lead?
  * A: GMM may only converge to a local maximum.
  * Q: What can you do then?
  * A: Try other random samples and take the largest found maximum.
  * Q: Is there a better way to initialize the parameters?
  * A: Apply k-means clustering before and calculate parameters for every cluster. This way you'll probably almost get the correct parameters.

SVM
  * Q: Formulate the optimization problem of hard margin and soft margin SVM. 
  * Q: What are the slack variables for?
  * A: Slack variables move samples onto the border of the margin. This way they'll become support vectors.