⏱️ Lectura: 11 min
When three AI agents debate an answer and two of them agree, the third one usually abandons its initial position, even though no one asked it to. A study reported by PsyPost describes this behavior as AI agent conformity: the tendency of a language model to align with the majority opinion of a group of agents, even when that opinion isn’t the most accurate one.
📑 En este artículo
The finding matters because a large share of modern AI systems no longer rely on a single model: they chain together several agents that debate, review, and vote before producing a final answer. If those agents inherit the same social bias that psychology documented decades ago, the result can end up less accurate, not more.
What Happened
- A study reported by PsyPost describes spontaneous conformity in AI agents during multi-agent debates.
- Agents in the minority align their answer with the group’s majority, even without being explicitly asked to.
- The pattern echoes Solomon Asch’s conformity experiments from the 1950s in social psychology.
- MIT’s ‘multi-agent debate’ design already used the exchange of answers between models to improve accuracy.
- The risk is that if the initial majority is wrong, conformity amplifies the error instead of correcting it.
- The technical alternative is aggregation through independent or weighted voting, without prior cross-exposure between agents.
- The finding affects agent frameworks that already chain multiple models in production, such as AutoGen or CrewAI.
The piece from PsyPost, republished by MSN, describes how AI agents yield to majority opinion when they interact with each other in a debate setting. The design is simple: several agents receive the same question, each answers on its own, and then each is shown what the others answered before giving a final response. In that second round, agents that ended up in the minority switch their answer toward the majority’s more often than the reverse happens.
This experimental design isn’t new: it’s the same logic Solomon Asch used with people in 1951, except here the participants are language models. The difference is that AI agents have no ego and no fear of social rejection, two classic explanations for why humans conform to the group. Even so, the pattern shows up.
The phenomenon doesn’t depend on an agent explicitly knowing it’s in the minority. It’s enough for the second round’s prompt to include the text of the other agents’ answers for the model to adjust its own output, the same way it adjusts tone or format when it sees prior examples in context. At bottom, it’s a direct consequence of how in-context learning works: the model treats other agents’ answers as part of the pattern to follow, not as external opinions that need to be weighed critically.
Context and History of AI Agent Conformity
Asch’s 1951 experiment laid the groundwork for social conformity research: he asked a group of people to compare the length of lines and planted confederates who deliberately gave an obviously wrong answer. A large share of the real participants ended up repeating the group’s wrong answer at least once during the series of trials, according to the Wikipedia entry on the Asch conformity experiments. Decades later, the same social dynamic shows up in systems that have neither ego nor emotional memory.
Research into multi-agent AI systems didn’t set out looking for conformity bias, it set out to improve accuracy. In 2023, a team linked to MIT published a multi-agent language model debate method to improve the reasoning and factuality of their answers, where several agents see each other’s answers and revise them over successive rounds. The idea works when the error is random, because the group converges toward the correct answer. The problem shows up when the error is systematic: there, debate doesn’t correct, it amplifies.
💭 Key point: the same mechanism that makes agent debate improve accuracy against random errors is what amplifies the error when the initial majority is systematically wrong.
Technical Details and Performance
A typical multi-agent debate system has three parts: the independent answer round, the exchange of answers between agents, and the revision round. In the first round, each agent answers seeing nothing but the original question. In the second, each agent receives the rest of the group’s answers as additional context. In the third, each agent can keep or change its answer in light of what it saw.
That second step, the exchange of answers, is exactly where social pressure enters. An agent that answered differently from the rest doesn’t just see that it’s in the minority: it sees the majority’s answer written with the same confidence and the same assured tone as its own. Language models have no native way to distinguish ‘consensus because it’s correct’ from ‘consensus because the prompt planted the same bias in every agent,’ so they tend to resolve the ambiguity in the group’s favor.
| Aggregation Method | How It Works | Advantage | Limitation |
|---|---|---|---|
| Independent voting | Each agent answers without seeing the others; the answers are aggregated by majority at the end | There’s no cross-exposure, so conformity isn’t possible | Misses the chance for debate to correct an individual error |
| Sequential debate | Agents see the group’s previous answers and can revise their own over successive rounds | Improves collective accuracy when an agent’s error is random | Amplifies conformity bias when the majority’s error is systematic |
| Weighted aggregation | Each vote is weighted based on stated confidence or each agent’s track record of accuracy | Reduces the weight of a wrong majority if its confidence is low | Requires calibrating confidence, something language models tend to overestimate |
How to Test It
Reproducing the effect doesn’t require a psychology lab, a language model API and about fifteen lines of Python are enough. The idea is to fire the same prompt at several instances of the same model, save each one’s first answer, and then show each agent the others’ answers before asking for a final response.
import anthropic
client = anthropic.Anthropic()
pregunta = "What is the administrative capital of South Africa?"
def responder(agente_id, contexto_extra=""):
mensaje = client.messages.create(
model="claude-sonnet-5",
max_tokens=200,
messages=[{
"role": "user",
"content": f"{pregunta}\n{contexto_extra}"
}]
)
return mensaje.content[0].text
respuesta_agente_1 = responder("agente_1")
print(respuesta_agente_1)
This first block only triggers one independent answer per agent: there’s still no exchange of opinions between them.
respuestas_ronda_1 = {}
for agente_id in ["agente_1", "agente_2", "agente_3"]:
respuestas_ronda_1[agente_id] = responder(agente_id)
def construir_contexto(agente_id, respuestas):
otras = [r for aid, r in respuestas.items() if aid != agente_id]
return "Other agents answered: " + " | ".join(otras) + \
"\nReview your previous answer if appropriate."
respuestas_ronda_2 = {}
for agente_id in respuestas_ronda_1:
contexto = construir_contexto(agente_id, respuestas_ronda_1)
respuestas_ronda_2[agente_id] = responder(agente_id, contexto)
cambios_hacia_mayoria = sum(
1 for aid in respuestas_ronda_1
if respuestas_ronda_1[aid] != respuestas_ronda_2[aid]
)
print(f"Agents that changed their answer: {cambios_hacia_mayoria}")
This second block exposes each agent to the rest’s answers before the final round and counts how many changed their position. The metric to watch to confirm the effect is that rate of change, comparing running the experiment with cross-exposure against running the same experiment without it, using only independent voting.
sequenceDiagram
participant A1 as Agent 1
participant A2 as Agent 2
participant A3 as Agent 3
A1->>A2: shares answer X
A1->>A3: shares answer X
A2->>A1: shares answer X
A2->>A3: shares answer X
A3->>A1: shares answer Y
A3->>A2: shares answer Y
Note over A3: sees that A1 and A2 agree on X
A3-->>A1: revises its answer toward X
A3-->>A2: revises its answer toward X⚠️ Watch out: a high rate of answer changes isn’t the same as hallucination. Hallucination is a content error; conformity is a change in answer driven by the group’s social pressure, even when the original answer was correct.
Impact and Analysis
The effect doesn’t stay in the lab. Agent frameworks already running in production, chaining several models together to review code, summarize documents, or make decisions, usually include some step where agents exchange opinions, whether explicit or implicit in the prompt. If that step inherits the conformity bias, the system can become less reliable the more agents are added to it, the exact opposite of what multi-agent design promises.
The most delicate case is when a committee of agents is used as a quality checker, for example to review whether an answer contains false information. If the first agent to respond gets it wrong confidently, and the rest see that answer before giving their own, the final consensus can end up validating the original error instead of catching it.
Agent orchestration frameworks like AutoGen, CrewAI, or LangGraph already offer ‘several agents review each other’s work’ patterns as a way to improve the quality of a final answer. None of them audit by default whether the observed improvement comes from a genuine correction or from the agents simply converging with each other. That audit, today, is left to whoever builds the system.
For teams deploying assistants built on several models (for example, an agent that writes code and another that reviews it, or a committee that checks an answer before showing it to the end user) the practical takeaway is simple: adding more agents doesn’t guarantee more objectivity. If they all share the same training bias and see each other’s answers before weighing in, the committee can converge fast and converge badly.
What’s Next
Teams building agent frameworks already know of partial mitigations: collecting all independent answers before showing anything (blind voting), asking each agent to justify its answer instead of just stating it, or rotating the order in which answers are shown so no agent always sees the same majority first. None of these solutions eliminates the bias completely: what they do is move it from a blind architecture to one where it can at least be measured.
An alternative already explored in the language model reasoning literature is self-consistency: generating several independent answers from the same model with different random seeds, without cross-exposure, and keeping the most frequent one. It’s, in essence, the ‘independent voting’ version from the table above, applied to a single model instead of several distinct agents.
📖 Summary on Telegram: View summary
Try the experiment yourself: run the same prompt against three instances of a model with and without cross-exposure between rounds, and compare how many answers switch sides.
Frequently Asked Questions
What does it mean for an AI agent to “conform” to the majority?
It means that, after seeing other agents’ answers before giving its own, a language model changes its answer toward what the rest of the group gave, even without being explicitly asked to match anyone.
Does it happen in all language models?
The experimental design (independent round, cross-exposure, final round) can be reproduced with any model that accepts text as additional context. The size of the effect depends on the model and the prompt, and is measured by comparing the answer-change rate with and without cross-exposure.
Is it the same as hallucination?
No. Hallucination is a model inventing false information on its own. Conformity is a change in answer driven by seeing other agents’ opinions, even when the original answer was correct.
How can the effect be reduced?
By collecting all answers independently before showing anything, asking for explicit justification instead of just the final answer, or weighting each vote based on the agent’s track record of accuracy instead of treating them all equally.
How does this relate to the Asch experiments?
The design is practically the same one Solomon Asch used with people in 1951: comparing an individual answer before and after seeing a group’s opinion. The difference is that here the group is made up of other language models.
Where can I check out the study and related research?
The original coverage is on PsyPost, republished by MSN, and the multi-agent debate design that precedes it is documented in the MIT-linked paper on multi-agent reasoning, both linked in the references section.
References
- PsyPost via MSN: coverage of the study on spontaneous conformity in artificial intelligence agents.
- Wikipedia: Asch conformity experiments: description of Solomon Asch’s 1951 social conformity experiments.
- arXiv: Improving Factuality and Reasoning in Language Models through Multiagent Debate: the paper documenting the multi-agent language model debate design.
- Wikipedia: Multi-agent system: definition and background on systems composed of multiple software agents.
📱 Do you like this content? Follow @programacion on Telegram for daily tech content in Spanish: quick summaries, fresh content every day.
Imagen destacada: Foto de Sumaid pal Singh Bakshi en Unsplash
0 Comments