Abstract of a new article in Science:
Many fear that we are on the precipice of unprecedented manipulation by large language models (LLMs), but techniques driving their persuasiveness are poorly understood. In the initial “pretrained” phase, LLMs may exhibit flawed reasoning. Their power unlocks during vital “posttraining,” when developers refine pretrained LLMs to sharpen their reasoning and align with users’ needs. Posttraining also enables LLMs to maintain logical, sophisticated conversations. Hackenburg et al. examined which techniques made diverse, conversational LLMs most persuasive across 707 British political issues (see the Perspective by Argyle). LLMs were most persuasive after posttraining, especially when prompted to use facts and evidence (information) to argue. However, information-dense LLMs produced the most inaccurate claims, raising concerns about the spread of misinformation during rollouts.
This seems alarming but also promising. If people are willing to be persuaded by AI, and AI works best by loading its arguments with factual claims, that's awesome, but only if the facts are real facts. If we could force LLMs to be honest, this might be great for the world.
1 comment:
If we could force LLMs to be honest, this might be great for the world.
Once again, these machines have no consciousness, no thought, no context for anything they "say". They are not communicating - they are collating.
You can't "force these machines to be honest", because honesty implies awareness of reality and a choice about what you are going to convey to others. "These machines" make no choices and know nothing about reality (or falsehood), and thus cannot lie, or tell the truth, or do anything other than regurgitate bits of data that sufficiently match requested input strings.
You could at best restrict what these programs are "trained" on to purely factual things - but if we could trust the people feeding information into machines to literally always champion the truth in doing so, we could skip the machines entirely and just have these magically trustworthy people speak the truth directly, and with the advantage of THOUGHT and AWARENESS OF REALITY and all the rest that machines lack.
Post a Comment