Sakana AI Signals Strategic Shift Beyond Transformers, Emphasizing High-Risk AI Research

Sakana AI has shared an update.

Co-founder and CTO Llion Jones, one of the original authors of the “Attention Is All You Need” paper that introduced the Transformer architecture, has highlighted concerns about the current direction of AI research. In a post adapted from a recent TED AI talk, Jones argues that while the industry now has more capital and talent, it operates under greater investor pressure and fear of being “scooped,” which pushes teams toward incremental improvements on existing Transformer models rather than high-risk, exploratory research. He compares the present focus on refinements to Transformers with the pre-Transformer era’s incremental work on RNNs, much of which became obsolete after the Transformer breakthrough.

Jones states that Sakana AI is deliberately shifting away from heavy reliance on Transformers and is prioritizing research that would not occur without the company’s involvement, emphasizing exploration of new paradigms rather than optimization of the current dominant architecture. For investors, this signals that Sakana AI is positioning itself as a high-risk, high-upside research-driven company, seeking to discover the “next” foundational model rather than compete on incremental performance gains. If successful, such breakthroughs could materially enhance its long-term value and defensibility in the AI ecosystem; however, this strategy may lead to less predictable development timelines and near-term commercial outcomes compared with peers focused on optimizing existing Transformer-based products. The post also implicitly appeals to investors who are willing to fund speculative, long-horizon AI research, suggesting that Sakana AI’s capital needs and partner profile will align with backers comfortable with research risk in pursuit of outsized technological and competitive advantages.