From Deep Learning to AGI: The Decade That Changed AI

In a recent conversation with No Priors‘ Elad Gil and Sarah Guo, Google DeepMind’s Vice President of Research, Oriol Vinyals, offered a compelling perspective on the evolution of artificial intelligence (AI) over the past decade and its trajectory towards Artificial General Intelligence (AGI).

Vinyals characterizes the period from 2010 to 2020 as the “deep learning era.” During this time, a set of general algorithms – including stochastic gradient descent, deep learning neural networks, and reinforcement learning – were applied across various domains with remarkable success.

These algorithms, while remaining fundamentally the same, were adaptable enough to achieve breakthroughs in diverse fields.

“You could just take the same algorithm and then change the dataset and of course tweak a couple of things and voila, you get protein folding, really advanced, from the traditional methods,” Vinyals explained.

This approach led to significant achievements such as mastering the game of Go, surpassing human performance on the ImageNet Challenge, and advancing state-of-the-art speech recognition and image generation.

The next pivotal insight came from recognizing the power of language modeling as an abstraction for generality. Vinyals points to the GPT-2 paper as a turning point, which proposed that language modeling could be used to solve a wide array of tasks, not just specific ones.

This realization led to the development of models that were not only reusable in their algorithms but also general in their applications.

“So that’s why I think AGI is getting closer,” Vinyals stated. “We have powerful models, first in 2010-2020, now we have general models.”

He also highlighted the recent breakthrough in multimodality as another significant step towards AGI. These techniques now extend beyond language to encompass vision, sound, and video, creating powerful general models that are beginning to “tick many boxes” in the definition of AGI.

Vinyals’ insights offer a clear narrative of AI’s progression: from the era of specialized deep learning models to the current landscape of increasingly general and multimodal AI systems. This evolution suggests that the field is inching closer to the long-sought goal of Artificial General Intelligence.

As AI continues to advance at a rapid pace, Vinyals’ perspective provides valuable context for understanding where we’ve been and where we might be headed in the quest for more capable and generalized artificial intelligence systems.