Ten observations and predictions for AI in 2025
This is something I wrote in December - feels a bit dated already!
As we 2024 concludes, I've been thinking about what's next for AI. Here are ten observations and predictions for 2025 - would love to hear your perspectives and what I might have missed!
1. Given the huge R&D investment going into frontier AI, we’ll continue to see tremendous amounts of innovation and progress along multiple fronts.
2. Unlike much of traditional computer science where algorithms can be analyzed on paper, deep learning remains highly empirical - ideas must be tested at scale to confirm their viability. With more known ideas than can be developed and tested in 2025, expect progress to continue at pace for several years before any slowdown.
3. Still, progress will continue to appear gradual to those directly involved, and will appear to take place in occasional big leaps to the general public.
4. LLM Maximalism (the idea that foundation models such as ChatGPT will lead to AGI with just more data and compute - say with version 6!) is dead. But transformer based LLMs are the closest thing to AGI we still have.
5. AI/ML techniques such as Reinforcement learning, search/planning, bayesian networks and logic will claim their rightful place next to pure deep learning based architectures. The path to AGI is pursued along one of these paths:
A. A reasoning system with LLM at its core complemented by RL and Search (e.g., OpenAI’s o3). Here the LLM sits sort of separately and search happens in token space.
B. An enhanced LLM with reasoning taking place within the models’ latent space(e.g., Meta’s recent COCONUT paper).
C. Some other technique that doesn’t involve an LLM at all (e.g., JEPA).
6. AGI benchmarking remains challenging, as existing metrics either can be gamed or measure only necessary but insufficient conditions. While François Chollet is developing ARC-AGI-2, more effort is needed in this area.
7. Evaluators/verifiers will be developing rapidly as they are needed for reasoning and agentic systems.
8. Ever-smaller models will achieve capabilities currently limited to much larger models, bringing costs down and pushing more AI capabilities to edge devices.
9. Research efforts (such as mechanistic interpretation) towards understanding and explaining foundational models will progress slowly given the relatively low investment. But this is an essential ingredient in addressing the alignment problem.
10. I agree with the idea that Embodied AI needs its own foundational model, similar to how LLMs unified different natural language processing capabilities (such as translation, sentiment classification, conversation, etc). Such a model, incorporating LLM-like knowledge and reasoning, would be crucial even for specific applications like self-driving. However, this breakthrough is unlikely in 2025.