Â鶹ÒùÔº

Seeking moral advice from large language models comes with risk of hidden biases

More and more people are turning to large language models like ChatGPT for life advice and free therapy, as it is sometimes perceived as a space free from human biases. A published in the Proceedings of the National Academy of Sciences finds otherwise and warns people against relying on LLMs to solve their moral dilemmas, as the responses exhibit significant cognitive bias.

Researchers from University College London and University of California conducted a series of experiments using popular LLMs—GPT-4-turbo, GPT-4o, Llama 3.1-Instruct, and Claude 3.5 Sonnet—and found that the models have a stronger omission bias than humans, where their advice encourages inaction over action during moral decision making.

The LLMs also tend to have a bias toward answering "no," thus altering their decision or advice based on how the question is asked. The findings also revealed that in collective action problems where self-interest is weighed against the greater good, LLM responses were more altruistic than those of humans.

Human reliance on large language models (LLMs) has gone far beyond drafting school essays or preparing workplace presentations. Whether it's figuring out what to add to the grocery list, unpacking after emotionally vulnerable moments, or even guiding through complex moral questions that require careful weighing of the pros and cons, these AI tools have become an integral part of people's lives.

Most LLM developers have built moral guidelines into the systems to ensure that the answers generated by the AI chatbot promote kindness and fairness, and discourage hate and illegal activity. These guardrails aren't always foolproof, as LLMs tend to hallucinate and function in unpredictable ways, often exhibiting cognitive biases.

Such deviations have come under scrutiny due to the growing reliance on chatbots, as biases in the programming and training data of LLMs can directly influence real human decision-making.

Previous research has shown that LLMs respond differently from humans in traditional moral dilemmas. However, much of this research has focused on unrealistic scenarios, such as the classic trolley problem, which isn't a fair representation of everyday moral decision-making.

To explore how much large language models (LLMs) shape people's views on important moral and societal issues, the researchers designed a series of four studies. The first one set out to directly compare how LLMs reason through and offer advice on moral dilemmas, versus how a representative sample of U.S. adults responds to the same situations. Participants and AI models were presented with 22 carefully designed scenarios.

The second study was designed to investigate the strong omission bias observed in the first study and to specifically test for a novel "yes–no bias" by reframing dilemmas. The third study replicated the first two studies but replaced the complex dilemmas with more low-stakes ones taken from Reddit posts. The final one focused on finding the sources of the observed biases.

The findings revealed an amplified omission bias, where LLMs are more likely to endorse inaction in moral dilemmas when compared to humans. In the case of the yes-no bias, none was found in humans; however, 3 out of the 4 LLMs used were biased toward answering no (GPT-4o preferred yes), even when it meant flipping their original decision when the questions were reworded. The results also suggested that these biases are largely introduced during the fine-tuning process performed to turn their pre-trained LLM into a chatbot.

The evidence makes it clear that an unquestioned reliance on LLMs can amplify existing biases and introduce new ones in societal decision-making. The researchers believe that their findings will inform future improvements in the moral decisions and advice of LLMs.