AI accuracy issues persist despite tech advances, experts warn
CALIFORNIA, UNITED STATES — Despite the rapid advancement of generative artificial intelligence, there are still reliability issues that continue to challenge the technology’s practical applications.
Generative AI systems, including popular chatbots, frequently make erroneous suggestions, a problem known as hallucinations. For instance, users have been wrongly advised to add glue to pizza recipes or consume rocks for health benefits, showcasing the bizarre mistakes AI can make.
Expert consensus on AI limitations
The Association for the Advancement of Artificial Intelligence (AAAI) recently released a report highlighting these concerns. A panel of experts and a survey of over 400 members underscored the need for increased reliability in AI systems. “We tend to be a little bit cautious and not believe something until it actually works,” noted Vincent Conitzer, a professor at Carnegie Mellon University.
AAAI said that the envisioned improvements in AI accuracy are not expected to materialize soon.
Challenges in enhancing AI reliability
The report underscores the need for AI tools to become substantially more reliable to approach or surpass human intelligence levels—a milestone known as artificial general intelligence (AGI).
Francesca Rossi, AAAI president, emphasized the importance of responsible AI development to align technological progress with human values. “We all need to work together to advance AI in a responsible way,” she wrote.
Potential solutions and expert skepticism
The AAAI report outlines key methodologies aimed at improving AI accuracy, including fine-tuning with human feedback, retrieval-augmented generation, and chain-of-thought processing.
Yet, about 60% of surveyed experts expressed skepticism that issues of factuality and trustworthiness in AI would be resolved in the near future.
John Thickstun, assistant professor at Cornell University, advises caution: “Scaling up existing models might seem promising, but it hasn’t yet provided the breakthrough in accuracy many had hoped for.”
The long road to artificial general intelligence
The pursuit of AGI, which aims to create models capable of human-level thought, is another significant challenge. Most researchers believe current techniques are unlikely to achieve AGI soon. Thickstun noted, “I think we’re very far away from this,” highlighting the lack of a built-in concept of truth in current models.
Despite these challenges, researchers continue to explore AI’s potential, recognizing its rapid advancements, especially since the launch of OpenAI‘s ChatGPT in 2022. While AI has shown remarkable progress, its reliability and accuracy issues must be addressed to fulfill its promise.
