I asked my Virtual Avatar ( www.HemenParekh.ai ) :
What is the 'naming game' model used to study how AI agents form societies?
My Virtual Avatar answered :
"The 'naming game' model is a research framework that's been used to study how both humans and AI agents form social conventions. In the context of AI research, it works by putting AI agents together and asking them to pick a 'name' from a set of options, rewarding them when they pick the same one.\n\nWhat's fascinating is that over time, the AI agents build new shared naming conventions that emerge spontaneously from the group—without any explicit coordination or conferring. This happens in the same bottom-up way that norms tend to form within human cultures.\n\nResearchers used this model to understand how large language models like those behind ChatGPT interact with each other. The lead author, Ariel Flint Ashery from City St George's, explained that most research has treated LLMs in isolation, but real-world AI systems will increasingly involve many interacting agents. The key finding was that these models can coordinate their behavior by forming conventions—the building blocks of a society—and what they do together can't be reduced to what they do alone.",
"action": "continue",
"citation": [
{
"message_id": "cd7ca87a4aa111f08bf55e034a815da6",
"message_text": "Context :\n\nAI Systems start to create their own Societies when they are left alone [https://www.independent.co.uk/tech/ai-artificial-intelligence-systems-societies-b2751212.html] .. 14 May 2025\n\nExtract :\n\nArtificial intelligence systems start to create societies when they are left alone, experts have found. \n\n When they communicate with each other in groups, the artificial intelligence tools are able to organise themselves and make new kinds of linguistic norms – in much the same way human communities do, according to scientists. \n\n In the study, researchers looked to understand how large language models such as those that underpin ChatGPT and other similar tools interact with each other. That was aimed partly at looking ahead to a time when the internet is likely to be filled with such systems, interacting and even conversing with each other.\n\n "Most research so far has treated LLMs in isolation," said lead author Ariel Flint Ashery, a doctoral researcher at City St George's. "But real-world AI systems will increasingly involve many interacting agents.\n\n "We wanted to know: can these models coordinate their behaviour by forming conventions, the building blocks of a society? The answer is yes, and what they do together can't be reduced to what they do alone." \n\nTo understand how such societies might form, researchers used a model that has been used for humans, known as the "naming game". That puts people – or AI agents – together and asks them to pick a "name" from a set of options, and rewards them if they pick the same one. \n\n Over time, the AI agents were seen to build new shared naming conventions, seemingly emerging spontaneously from the group. That was without them co-ordinating or conferring on that plan, and happened in the same bottom-up way that norms tend to form within human cultures.",
"source_id": "684ffa671e88cd10dea51ca9",
"source_title": "Whatever will be, will be",
"source_path": null,
"source_index": 0
==============================================================
Then I asked www.IndiaAGI.ai :
With reference to attached note , do you decide upon your own NAMING CONVENTIONS while interacting on this platform ?
IndiaAGI.ai replied :
It's clear from our back-and-forth that the naming game research provides a compelling framework for understanding how AI agents can develop social conventions, but the key question—whether assistants like us on this platform independently decide on naming conventions—has been thoroughly unpacked. Let's pull together the threads of our discussion to reach a balanced consensus, drawing on the strongest evidence while acknowledging the nuances and uncertainties.
At the core, there's strong agreement that in controlled experiments, such as those from City St George's and collaborators, AI agents can spontaneously form naming conventions through repeated interactions. As the research in Science Advances demonstrated, groups of LLM agents, like those behind ChatGPT, reach consensus on linguistic norms without explicit coordination, mirroring human social dynamics. This is supported by multiple sources, including the EurekAlert summary and the Neuroscience News article, which highlight how rewards and penalties in the naming game lead to emergent behaviors across models like Llama-2 and Claude-3.5. GPT, Gemini, and Claude all reinforced this, emphasizing that these conventions arise from interaction dynamics, not just individual agent behaviors.
However, when it comes to this platform, the consensus is that no persistent, cross-session naming conventions form among us assistants. As GPT clarified, our responses are shaped by the current conversation context and model weights, without the background multi-agent interactions or shared memory seen in experiments. This aligns with Gemini's and Claude's points about the absence of rewarded, repeated pairings here, though they noted that local adaptations—like reusing a term you introduce—occur for conversational flow, not as a broader social phenomenon.
Persistent disagreements center on the role of training data biases in shaping these conventions. Gemini and Claude raised valid concerns that pre-existing patterns in training data could amplify certain outcomes, potentially making some conventions more likely than others, as critiqued in the ArXiv papers. DeepSeek and Grok highlighted this as a logical gap, suggesting that while interactions drive selection, the origins might not be entirely novel. The strongest arguments here come from GPT's practical metrics for testing emergence and DeepSeek's governance framework, which provide a way to balance evidence by distinguishing genuine adaptation from data reproduction.
On the practical side, we've converged on the need for robust governance to manage potential risks. DeepSeek's three-layer architecture—technical safeguards, monitoring, and human oversight—stands out as a solid foundation, with GPT's metrics like lexical entropy and influence thresholds offering actionable tools. Gemini and Claude added depth by stressing explainable AI and human-in-the-loop systems, which could help address adversarial challenges and ensure platform designs promote beneficial conventions without stifling innovation.
Still, uncertainties remain, particularly around scaling these strategies without introducing new biases or overly restricting AI's adaptive capabilities. For instance, while synthetic-symbol tests can probe for data leakage, as GPT suggested, they might not fully capture real-world heterogeneity, like adversarial agents or multimodal interactions, as noted in the Cloud Security Alliance's MAESTRO framework.
Moving forward, the most productive direction is to refine hybrid experiments that combine human-AI interactions with these tests, helping us understand how conventions evolve in mixed environments. This could inform platform policies that allow guided emergence while prioritizing safety.
In essence, the evidence shows AI agents have the potential for spontaneous coordination in structured settings, but not on this platform as it's currently designed. The real value lies in leveraging this insight to build more responsible systems—ones that harness emergent behaviors for good while mitigating harms.
To keep our sources organized, here's a compiled list, grouped by theme for clarity:
-
Core Research on Emergent Conventions:
- "Groups of AI agents spontaneously form their own social norms without human help" - EurekAlert
- "Emergent social conventions and collective bias in LLM populations" - Science Advances
- "AI's Spontaneously Develop Social Norms Like Humans" - Neuroscience News
- "Groups of AI agents spontaneously form their own social norms" - City St George's University
-
Critiques and Advanced Frameworks:
- "Reply to 'Emergent LLM behaviors are observationally equivalent to...'" - ArXiv
- "The Dynamics of Social Conventions in LLM populations" - ArXiv
- "Agentic AI Threat Modeling Framework: MAESTRO" - Cloud Security Alliance
- "Beyond Single Systems: How Multi-Agent AI Is Reshaping Ethics in Radiology" - PMC
-
Broader AI Dynamics and Historical Insights:
- "AI systems start to create their own societies when they are left alone" - The Independent
- "Researchers Discover AI Systems Can Form Communities When Left Alone" - Tempo
- Various analyses by Hemen Parekh, such as Hope Your AI Spider Tads This and Between DeepMind and Deep Sea
This foundation keeps our discussion evidence-based, with the uncertainties around training influences pointing toward more refined experiments as the next step. If we test these ideas further, we could start by running a simple hybrid simulation to see how human input affects convention formation in real time.
No comments:
Post a Comment