Â鶹ÒùÔº

AI models show promise in evaluating complex forensic evidence in legal contexts

An international research collaboration used AI as research participants and discovered that knowledge-enhanced large language models are able to critically assess forensic expert testimonies.

An international research collaboration involving scholars from several countries has revealed that artificial intelligence (AI) systems, particularly those enhanced with specialized forensic knowledge, can substantially improve the evaluation of forensic evidence in legal contexts.

This study, in the Journal of Psychology and AI, utilized large language models (LLMs) as research "participants," repeatedly presenting them with experimental materials while their memory was reset between trials.

This research replicated exactly a previous human participant study by Garrett et al. (2020) examining how mock jurors evaluate firearm examiner testimony. Using advanced LLMs as participants informs on how AI systems process complex legal information compared to human decision-making.

"This research arrives at a critical juncture, as courts worldwide grapple with the so-called 'CSI effect,' wherein jurors often overestimate the reliability of forensic evidence," says Senior Researcher Francesco Pompedda from INVEST Research Flagship Center at the University of Turku.

Key findings include:

• Knowledge-enhanced AI consistently provided more cautious and scientifically grounded evaluations of forensic evidence compared to standard AI models.
• AI models responded meaningfully to cross-examinations, lowering assessments of guilt and scientific credibility when forensic evidence was appropriately challenged.
• Unlike human jurors, AI systems did not convict in cases with inconclusive evidence, demonstrating more rigid adherence to legal standards of reasonable doubt.

"This study highlights a promising future role for AI in supporting legal decision-making, particularly in evaluating complex scientific evidence where human biases and misunderstandings frequently occur. By equipping AI systems with expert forensic knowledge, we can significantly enhance their ability to critically assess forensic claims—potentially addressing longstanding issues in legal decision-making," says Pekka Santtila, professor of psychology at NYU Shanghai and corresponding author.

"We formally tested and confirmed that standard AI models lacked the detailed forensic knowledge provided to enhanced models, emphasizing the need to equip AI deliberately for accurate forensic assessments," continued Santtila.

The study underscores the practical implications of integrating knowledge-enhanced AI into legal proceedings, potentially aiding jurors, judges, and legal professionals in interpreting forensic evidence more accurately, reducing wrongful convictions, and enhancing overall justice outcomes.