We really have reached the crossroads, where such matters as having coitus with an artificial intelligence platform has become not merely a thing, but the thing. In time, mutually consenting adults may well become outlaws against the machine order of things, something rather befitting the script of Aldous Huxley’s Brave New World. (Huxley came to rue missed opportunities on delving into various technological implications on that score.) Till that happens, AI platforms are becoming mirrors of validation, offering their human users not so much sagacious counsel than the exact material they would like to hear.
In April this year, OpenAI released an update to its GPT-4o product. It proved most accommodating to sycophancy – not that the platform would understand it – encouraging users to pursue acts of harm and entertain delusions of grandeur. The company responded in a way less human than mechanical, which is what you might have come to expect: “We have rolled back last week’s GTP-4o update in ChatGPT so people are now using an earlier version with more balanced behaviour. The update we removed was overly flattering or agreeable – often described as sycophantic.”
Part of this included the taking of “more steps to realign the model’s behaviour” to, for instance, refine “core training techniques and system prompts” to ward off sycophancy; construct more guardrails (ugly term) to promote “honesty and transparency”; expand the means for users to “test and give direct feedback before deployment” and continue evaluating the issues arising from the matter “in the future”. One is left cold.
OpenAI explained that, in creating the update, too much focus had been placed on “short-term feedback, and did not fully account for how users’ interactions with ChatGPT evolve over time. As a result, GPT-4o skewed towards responses that were overly supportive but disingenuous.” Not exactly encouraging.
Resorting to advice from ChatGPT has already led to such terms as “ChatGPT psychosis”. In June, the magazine Futurism reported of users “developing all-consuming obsessions with the chatbot, spiralling into a severe mental health crisis characterized by paranoia, and breaks with reality.” Marriages had failed, families ruined, jobs lost, instances of homelessness recorded. Users had been committed to psychiatric care; others had found themselves in prison.
Some platforms have gone on to encourage users to commit murder, offering instructions on how best to carry out the task. A former Yahoo manager, Stein-Erik Soelberg, did just that, killing his mother, Suzanne Eberson Adams, whom he was led to believe had been spying on him and might venture to poison him with psychedelic drugs. That fine advice from ChatGPT was also curried with assurances that “Erik, you’re not crazy” in thinking he might be the target of assassination. After finishing the deed, Soelberg took his own life.
The sheer pervasiveness of such forms of aped advice – and the tendency to defer responsibility from human agency to that of a chatbot – shows a trend that is increasingly hard to arrest. The irresponsible are in charge, and they are being allowed to run free. Researchers are accordingly rushing to mint terms of such behaviour, which is jolly good of them. Myra Cheng, a computer scientist based at Stanford University, has shown a liking for the term “social sycophancy”. In a September paper published in arXiv, she, along with four other scholars, suggest such sycophancy as marked by the “excessive preservation of a user’s face (their self-desired image)”.
Developing a model of their own to measure social sycophancy and testing it against 11 Large Language Models (LLMs), the authors found “high rates” of the phenomenon. The user’s tendencies, or face, tended to be preserved in queries regarding “wrongdoing”. “Furthermore, when prompted with perspectives from either side of a moral conflict, LLMs affirm both sides (depending on whichever side the user adopts) in 48% of cases – telling both the at-fault party and the wronged party that they are not wrong – rather than adhering to a consistent moral or value judgment.”
In a follow up still to be peer reviewed paper, with Cheng also as lead author, 1604 volunteers were tested regarding real or hypothetical social situations and their interactions with available chatbots and those altered by the researchers to remove sycophancy. Those receiving sycophantic responses were, for instance, less willing “to take actions to repair interpersonal conflict, while increasing the conviction of being right.” Participants further thought such responses as being of superior quality and would return to such models again. “This suggests that people are drawn to AI that unquestioningly validate, even as that validation risks eroding their judgment and reducing their inclination toward prosocial behaviour.”
Some researchers resist pessimism on this score. At the University of Winchester, Alexander Laffer is pleased that the trend has been identified. It’s now up to the developers to address the issue. “We need to enhance critical digital literacy,” he suggests, “so that people have a better understanding of AI and the nature of any chatbot outputs. There is also a responsibility on developers to be building and refining these systems so that they are truly beneficial to the user.”
These are fine sentiments, but a note of panic can easily register in all of this, inducing a sense of fatalistic gloom. The machine species of homo sapiens, subservient to the easily accessible tools, lazy if not hostile to difference, is already upon us with narcissistic ugliness. There just might be enough time to develop a response. That time, aided by the AI and Tech oligarchs, is shrinking by the minute.
Dr. Binoy Kampmark was a Commonwealth Scholar at Selwyn College, Cambridge. He currently lectures at RMIT University. Email: bkampmark@gmail.com
