A Path to AGI: Predicted by Visionary Rob May
The journey toward Artificial General Intelligence (AGI) is one of the most ambitious pursuits in the tech world. While many discussions focus on predictions for the coming years, Rob May’s article, An AGI Prediction Beyond 2025, takes a deeper dive into the fundamental building blocks required to achieve AGI. Let’s simplify and explore his key insights.
The Central Question: What Does It Take to Achieve AGI?
AGI refers to machines capable of performing any intellectual task that humans can, learning, reasoning, and adapting to entirely new domains without specific training. Rob May challenges the prevalent belief that scaling existing AI models (adding more data and compute) is the sole path to AGI.
“Human intelligence evolved before text data… That leads me to believe that intelligence is a computational issue more than a data one.”
Instead of just scaling up, May argues for addressing computational and architectural challenges in novel ways. He outlines five core ideas that, when combined, may bring us closer to AGI.
1. Mixture of Experts (MoE): Specialization Over Generalization
The concept of Mixture of Experts (MoE) has been gaining traction in AI research. Instead of relying on a single massive model to perform all tasks, MoE involves smaller, specialized models that collaborate.
“At inference time, an input is either routed to the most appropriate model or… routed to multiple models who ‘vote’ on the best output.”
May envisions extending this idea to Mixture of Architectures, where different types of models (e.g., neural networks, symbolic logic systems) run on specialized hardware, working together seamlessly. This approach leverages the strengths of each model type while reducing computational overhead.
2. Positional Embeddings: Adding Context to Data
In transformer models like GPT, positional embeddings encode the position of each word in a sequence, enabling models to understand context over long passages.
“The important thing for this discussion isn’t how this works; it’s that it’s an added level of meta information about the underlying data set.”
May suggests that leveraging more meta information—like hierarchical or contextual embeddings—will be crucial for AGI. This could involve combining multiple types of embeddings to create richer representations of data.
3. Explosion of AI Hardware: Tailoring Compute to AI
Most AI systems today rely on GPUs, but they were originally designed for graphics, not AI.
“That’s why the last decade has seen an explosion in AI hardware chips… Companies like Cerebras, Groq, Sambanova, Tenstorrent, and others are leading this space.”
To achieve AGI, custom hardware designed specifically for AI tasks will be essential. These chips will allow more efficient and powerful computations, enabling new AI architectures like MoE and neural-symbolic hybrids to thrive.
4. Neural Oscillations: Synchronizing Intelligence
In the human brain, neural oscillations—waves of electrical activity—coordinate information processing across different regions. AI currently lacks an equivalent mechanism for synchronizing its models.
“I believe that making AGI will require us to come up with something that mirrors neural oscillations.”
May suggests that systems-level embeddings, which provide real-time contextual snapshots of an AI system’s state, could serve as a computational equivalent. This would allow different AI components to work together more effectively, much like the brain’s synchronized neural activity.
5. Symbolic Recursion: Revisiting Old Ideas
In his discussion of symbolic recursion, May draws from the ideas in Douglas Hofstadter’s book, Gödel, Escher, Bach: An Eternal Golden Braid.
“Intelligence is a symbolic logic system that, within the system, contains a symbol that also represents the entire system itself.”
Neural networks alone have limitations, particularly in areas like logical reasoning and self-representation. May sees value in combining symbolic logic with neural approaches—a neuro-symbolic AI—to create systems that can reason and adapt more holistically.
Putting It All Together: A Path to AGI
May believes the path to AGI involves combining these five ideas into a cohesive system:
Mixture of Architectures: Specialized models working together across tasks.
Meta-Embeddings: Richer contextual understanding of inputs.
Custom AI Hardware: Efficient, domain-specific computational tools.
System Synchronization: Unified state representations akin to neural oscillations.
Neuro-Symbolic Integration: Blending neural and symbolic reasoning.
He envisions a world where AGI systems don’t operate as isolated models but as integrated ecosystems. For example, imagine an AI system where each model (labeled A, B, C, D, and E) performs a specific task. Model A processes human voice while also receiving inputs from other models (e.g., B for context, C for visual cues) and a system-wide state representation. This holistic integration adds depth and adaptability to AI processing.
“This approach provides more context because a model now doesn’t just take its own normal input, but takes a bunch of systems-level contextual input as well.”
Why This Matters
May’s vision challenges the status quo in AI, emphasizing that intelligence is not just about processing vast amounts of data but also about coordination, contextual understanding, and specialization. Achieving AGI will require collaboration between researchers, engineers, and hardware designers to address these multifaceted challenges.
For those watching the race to AGI, May’s article serves as both a roadmap and a call to action. As he concludes:
“In my head it all makes sense and seems like a logical flow… If you have questions, or areas of this idea need more clarification, please reach out.”
The path to AGI is complex, but by integrating diverse approaches, we may be closer than we think to unlocking true machine intelligence.