Illinois bans AI-only mental health therapy to protect patient safety
ILLINOIS, UNITED STATES — Illinois has signed a law prohibiting the use of artificial intelligence for delivering therapy without a licensed clinician’s supervision.
The legislation, signed by Governor J.B. Pritzker, aims to protect patients from potentially harmful, unregulated AI products while preserving the role of qualified human providers.
Why Illinois prohibits AI-led mental health therapy
Lawmakers gave the example of plain anecdotes of how AI chatbots responded instead of giving potentially harmful advice, including prescribing a little dose of meth to a fictional drug addict.
This highlighted the urgent threat that unscrutinized algorithms, which rely on potentially incorrect data on the internet, will pose to disadvantaged groups.
In turn, the act draws a sharp line between what artificial intelligence may and may not perform: it can help to make administrative and supplementary decisions but is not allowed to be involved in therapeutic decision-making.
This will limit the possibility of critical care being provided by unqualified personnel, as well as prevent it from becoming a low-cost but dangerous alternative to professional medical help. The state will enforce this with investigations and fines of up to $10,000 for violations.
“The people of Illinois deserve quality healthcare from real, qualified professionals and not computer programs that pull information from all corners of the internet to generate responses that harm patients,” said Mario Treto Jr., Secretary of the Illinois Department of Financial and Professional Regulation (IDFPR).
Illinois sets precedent for regulating AI in healthcare
The act represents one of the nation’s first concerted efforts to proactively regulate AI’s application in a sensitive healthcare sector, rather than reacting to crises after they occur.
It is directly in response to the high-velocity, unregulated spread of AI-based mental health apps, which legislators were concerned may exploit at-risk people in need of actual clinical help.
The state has placed itself in a position where it is able to curb these threats by providing special powers to the Illinois Department of Financial and Professional Regulation (IDFPR) to review and punish violations.
This action reflects a growing trend towards a more cautious approach to the use of AI in healthcare. It mirrors conclusions of organizations such as the Congressional Budget Office that state the practical application of technology is not consistent and its overall effects are extremely difficult to determine.
The requirement to exercise human control over AI in patient care puts Illinois in a precedent-setting position where the scientific demonstration, not merely technological feasibility, of safety and efficacy should be the guide in integrating AI into patient care.
“This legislation stands as our commitment to safeguarding the well-being of our residents by ensuring that mental health services are delivered by trained experts who prioritize patient care above all else,” Treto Jr. notes.
