Which 2018 Google model revolutionized NLP by using a 'masked language model' objective to learn bidirectional representations?

Show answer

BERT

BERT (Bidirectional Encoder Representations from Transformers) allows the model to consider the context of a word from both the left and the right. This solved the limitation of previous models that were traditionally left-to-right or right-to-left.

Also asked as:

Which Google-developed NLP model, released in 2018, uses bidirectional representations to consider word context from both the left and right?

Would you remember this in 6 months?

Trivia Forge uses spaced repetition so facts stick. Free, no signup needed.

Practice Computer Science questions →

More Computer Science trivia

Which fundamental theorem of probability is used to calculate the posterior probability of a hypothesis given observed evidence?
Which probability inequality provides an upper bound on the probability that the sum of bounded independent random variables deviates from its expected value by more than a certain amount?
Which randomized hashing technique ensures that for any two distinct keys, the probability of a collision is at most 1/m, where m is the number of slots?
Which search strategy for hyperparameter tuning samples from a distribution of possible values rather than checking every point on a fixed grid?
Which second-order optimization method uses the inverse of the Hessian matrix to determine the update direction?
Which technique involves training a model on a large dataset and then fine-tuning it on a smaller, task-specific dataset?

See all 19 →