OpenAI, Google, and Anthropic hit roadblocks in advanced AI development
NEW YORK, UNITED STATES — Leading artificial intelligence companies OpenAI, Google, and Anthropic are struggling in their pursuit of more advanced AI models. This marks a potential slowdown in the rapid progress the industry has witnessed over the past few years.
AI training hurdles and data limitations
Bloomberg reported that OpenAI’s latest model, Orion, has particularly struggled with coding questions it hadn’t been trained on. The model’s performance improvement over existing systems like GPT-4 has been less impressive than anticipated.
Similarly, Google‘s upcoming Gemini software isn’t meeting internal benchmarks, while Anthropic has delayed the release of its anticipated Claude 3.5 Opus model.
The tech giants’ collective difficulties underscore the waning returns in building ever-larger models using massive amounts of data and computing power.
Scarcity of data and rising costs
The AI industry’s quest for groundbreaking models faces another key challenge: data scarcity. Historically, AI models have been trained using freely available online content, but the pool of new, high-quality data is drying up. According to insiders, Orion’s coding struggles, for example, stem from a limited supply of coding data.
“It is less about quantity and more about quality and diversity of data,” explains Lila Tretikov, head of AI strategy at New Enterprise Associates.
“We can generate quantity synthetically, yet we struggle to get unique, high-quality datasets without human guidance, especially when it comes to language.”
As Margaret Mitchell of AI startup Hugging Face noted, progress in AI may require different training approaches to achieve performance breakthroughs.
Tech companies are exploring synthetic data generation and other novel methods, but such avenues have their limitations. The result is slower, costlier development cycles. Dario Amodei, CEO of Anthropic, has indicated that while companies currently spend around $100 million to train cutting-edge models, this figure could reach $100 billion in the coming years.
Industry implications and future direction
Despite these challenges, companies are adapting their strategies. Google’s Gemini received functionality updates, such as generating images of people, while OpenAI has refined existing features, including a voice assistant for ChatGPT. Both companies are shifting focus from mere model size to practical applications and improved user experiences.
As CEO Sam Altman noted on Reddit, “We will have better and better models, but I think the thing that will feel like the next giant breakthrough will be agents.”
The company remains optimistic about future releases, albeit without a major model like GPT-5 on the immediate horizon.
