Terminator not coming yet (sorry AGI maniacs)
New paper by Google provides evidence that transformers (GPT, etc) cannot generalize beyond their training data.
https://arxiv.org/abs/2311.00871
“Our empirical results show transformers demonstrate near-optimal unsupervised model selection capabilities, in their ability to first in-context identify different task families and in-context learn within them when the task families are well-represented in their pretraining data.
However when presented with tasks or functions which are out-of-domain of their pretraining data, we demonstrate various failure modes of transformers and degradation of their generalization for even simple extrapolation tasks. Together our results highlight that the impressive ICL abilities of high-capacity sequence models may be more closely tied to the coverage of their pretraining data mixtures than inductive biases that create fundamental generalization capabilities”.
What does this mean? Nothing new for those already working on this. It is certainly a good thing for safety, meaning a model not trained to do X cannot do X. It also means you should use models for what they were trained to do.
Once again reality proves wrong all those claiming that Terminator is coming.
