Machine Learning Trivia Questions
- What loss function is specifically used by Support Vector Machines to maximize the margin between classes?
- What mathematical rule is the fundamental foundation of the backpropagation algorithm used to train neural networks?
- What non-symmetric measure from information theory is used to quantify the difference between a target probability distribution P and an approximation Q?
- What parameter-efficient fine-tuning method adapts a large model by training small low-rank update matrices while keeping the original pretrained weights frozen?
- What property does a classifier have when its predicted confidence levels match the empirical frequency of correctness, so that predictions made with 80% confidence are right about 80% of the time?
- What regularization technique penalizes the sum of the absolute values of the weights, often leading to sparse weight vectors?
- What specific divergence measure is used in Variational Autoencoders (VAEs) to ensure the learned latent distribution stays close to a prior distribution?
- What specific type of Autoencoder is designed to generate new data by learning the parameters of a probability distribution?
- What term describes an agent achieving high reward by exploiting flaws or loopholes in a misspecified reward function instead of accomplishing the task its designer intended?
- What term describes the phenomenon where a model performs well on training data but poorly on unseen test data?
- What term describes the tendency of a neural network to abruptly lose previously learned capabilities when it is trained on a new task or data distribution?
- What theoretical measure quantifies the capacity (complexity) of a statistical classification algorithm, defined as the size of the largest set of points it can shatter?
- What three-letter abbreviation names the technique of retrieving relevant documents from an external store and conditioning a language model's generation on them, extending its knowledge beyond its trained weights?
- What training strategy, named by analogy to education, presents examples in a meaningful order from easy to hard rather than randomly, often improving convergence and final performance?
- What type of tree-like diagram is used in hierarchical clustering to illustrate the arrangement of the clusters produced by the relevant algorithm?
- What variant of gradient descent updates model parameters using only a single training instance at each step?
- When training Variational Autoencoders (VAEs), which technique is used to allow backpropagation through the stochastic sampling layer?
- Which 2012 CNN architecture won the ImageNet competition by a large margin, signaling the start of the 'Deep Learning' revolution?
- Which 2013 neural word embedding model introduced the Skip-gram and Continuous Bag-of-Words (CBOW) architectures?
- Which 2018 Turing Award winner is often associated with the development of the Boltzmann machine and the popularization of backpropagation?
- Which 2022 DeepMind result showed that large language models were badly undertrained, recommending that model size and the number of training tokens be scaled in roughly equal proportion for a given compute budget?
- Which 2023 method aligns language models directly from preference data with a simple classification-style loss, removing the separate reward model and reinforcement-learning loop used by RLHF?
- Which activation function is defined by the formula f(x) = max(0, x)?
- Which activation function is most commonly used in the final layer of a multi-class classification neural network to produce a probability distribution?
- Which adversarial architecture consists of two networks, a Generator and a Discriminator, competing in a zero-sum game?
- Which 1996 clustering algorithm is density-based and can identify outliers as 'noise' points?
- Which clustering algorithm iteratively assigns points to the nearest cluster center and then updates the center based on the mean of assigned points?
- Which dimensionality reduction technique minimizes the Kullback-Leibler divergence between similarity distributions in high and low-dimensional spaces?
- Which ensemble method builds multiple decision trees independently using a random subset of features and merges their results by voting or averaging?
- Which ensemble method trains models sequentially, where each new model attempts to correct the errors made by previous ones?
- Which generative model uses a 'reparameterization trick' to allow backpropagation through a stochastic latent variable?
- Which imitation-learning algorithm overcomes behavioral cloning's compounding errors by iteratively rolling out the current policy, having an expert label the states it visits, and aggregating those into the training set?
- Which large-scale image database supplied the 1,000-class dataset used in the 2012 ILSVRC competition?
- Which loss function is the standard choice for training a neural network for multi-class classification with a Softmax output layer?
- Which neural network activation function can lead to 'dead' neurons because its gradient is zero for all negative inputs?
- Which optimization algorithm combines the benefits of both AdaGrad and RMSProp by using both momentum and adaptive learning rates?
- Which regularization technique prevents overfitting by randomly setting a fraction of input neurons to zero during each training step?
- Which researcher, who co-authored the 2012 AlexNet paper, later became the Chief Scientist at OpenAI?
- Which researcher won the 2018 Turing Award alongside Geoffrey Hinton and Yoshua Bengio for their work on Deep Learning?
- Which RNN variant simplifies the LSTM by combining the forget and input gates into a single 'update gate'?
- Which specific type of RNN was designed to mitigate the vanishing gradient problem using a 'forget gate'?
- Which technique improves neural network training by normalizing layer inputs to have a mean of zero and a variance of one for each mini-batch?
- Which theorem states that there is no single machine learning algorithm that performs better than all others across every possible problem?
- Which three researchers are often credited with the 1986 seminal paper on the backpropagation algorithm?
- Who developed the neocognitron, a hierarchical multilayered neural network that inspired modern Convolutional Neural Networks?
- Who is credited with proving the Universal Approximation Theorem for sigmoid activation functions in 1989?
- Who is often credited with developing the first artificial neural network called the 'Perceptron' in 1958?
- Who is the primary author of the 2014 paper that first introduced Generative Adversarial Networks (GANs)?