AI outsmarts human experts in neuroscience predictions: UCL Research

LONDON, UNITED KINGDOM — A study by University College London (UCL) researchers has revealed that artificial intelligence can predict neuroscience study outcomes more accurately than human experts.
The research, published in Nature Human Behaviour, demonstrates a significant leap forward in AI’s capabilities beyond mere information retrieval.
“Our work investigates whether LLMs can identify patterns across vast scientific texts and forecast outcomes of experiments,” said Dr. Ken Luo, lead author of the study.
He emphasized the inefficiency of traditional trial-and-error approaches in research and highlighted AI’s potential to streamline discovery processes.
The BrainBench challenge
Using a specially developed tool called BrainBench, researchers pitted 15 large language models (LLMs) against 171 qualified neuroscience experts.
The challenge involved distinguishing between genuine and modified research abstracts. The results were striking – LLMs achieved an impressive 81% accuracy, while human experts managed only 63%.
Even the most specialized neuroscientists only reached 66% accuracy, falling short of the AI’s performance.
BrainGPT: A specialized solution
The research team took their investigation further by developing BrainGPT, a specialized version of the Mistral language model trained specifically on neuroscience literature. This tailored approach proved even more effective, achieving an 86% accuracy rate in predicting study results.
This international collaboration, involving institutions from multiple countries including the United Kingdom, United States, Germany, and Australia, marks a milestone in the integration of AI tools in scientific research.
Future of research collaboration
Professor Bradley Love from UCL Psychology & Language Sciences offers a thought-provoking perspective: “We suspect it won’t be long before scientists are using AI tools to design the most effective experiment for their question. While our study focused on neuroscience, our approach was universal and should successfully apply across all of science.”
"Large language models surpass human experts in predicting neuroscience results" w @ken_lxl and https://t.co/YOhCmQlJsu. LLMs integrate a noisy yet interrelated scientific literature to forecast outcomes. https://t.co/49WYirBdBv 1/8 pic.twitter.com/ysk3GsKsUN
— Bradley Love (@ProfData) November 27, 2024
Looking ahead, Dr. Luo envisions a transformative future for scientific research: “We envision a future where researchers can input their proposed experiment designs and anticipated findings, with AI offering predictions on the likelihood of various outcomes. This would enable faster iteration and more informed decision-making in experiment design.”
GPT-4 beats human analysts in financial forecasting
Meanwhile, AI can also outperform human experts in analyzing corporate financial statements and predicting future earnings, according to new research from the University of Chicago.
The findings are published in the working paper, “Financial Statement Analysis with Large Language Models,” which focused on the capabilities of GPT-4, a powerful language model developed by OpenAI.
The study highlights a novel approach where GPT-4 uses “chain-of-thought” prompts to mimic the cognitive process of a financial analyst. This method enables the AI to dissect balance sheets and income statements—absent of any descriptive text—to forecast company performance.
This technique has proven effective, with GPT-4 achieving a 60% accuracy rate in predicting earnings growth direction, surpassing the 53-57% accuracy typical among human analysts.