Werewolves & Villagers – AI Simulations

“An uninformed majority will always lose a battle of information against an informed minority” was the thesis a Russian sociology student wanted to prove when he invented the game Werewolves & Villagers (aka Salem, or Mafia).

Listen to a discussion about this experiment on our podcast:

I wondered if such a game of human behavior could be simulated using AI.

These are social deduction games where players are divided into Villagers and Werewolves. The game’s objective is simple: the Villagers must identify and eliminate the hidden Werewolves before they are all devoured, while the Werewolves aim to secretly eliminate the Villagers each night without revealing their identities.

Each round begins with the night phase, where the Werewolves secretly select a Villager to eliminate. During the day phase, all players—including both Werewolves & Villagers—discuss and vote to eliminate one player they suspect of being a Werewolf. The game continues until all Werewolves are eliminated or all but two Villagers are eliminated.

Quick rounds can be over in minutes; in-depth ones can last all night, per Wired Magazine’s insights into the popularity of playing this at tech conferences.

So, with such complex humanity on display, how similar would an AI simulation be?

What do humans usually do?

Werewolves tend to win slightly less often in Werewolves & Villagers due to the inherent advantage that Villagers have in numbers (and in some versions, the presence of special roles that provide critical information or protection).

As the game progresses, Villagers gather more clues and become more strategic in their eliminations, reducing the effectiveness of the Werewolves’ deception. Additionally, the random or mistaken eliminations in early rounds often correct themselves as Villagers improve their voting accuracy, further tipping the balance in their favor.

Test Results

In the 20 simulated games that I ran with ChatGPT, the outcomes were 15 wins for the Werewolves and 5 wins for the Villagers – very different to the human results.

Analysis of Results (as written by ChatGPT about its own simulation):

1. Random Assignment and Simplified AI Decision-Making:

• The roles were assigned randomly, and each player’s decisions were based on preset logic rather than the nuanced reasoning and psychological depth that human players bring. This may have resulted in decisions that favored the Werewolves, as AI can sometimes fail to detect the subtle patterns and social cues that humans rely on to identify deception.

2. Lack of Human Social Dynamics:

• The AI simulations lacked the complex social interactions and meta-game strategies that humans use in Werewolf games. Human players often base their decisions on past experiences, personal biases, and social relationships, which can lead to more varied outcomes. The lack of these dynamics in AI likely contributed to the Werewolves’ success, as the Villagers could not leverage these social tools effectively.

3. Simplified Bluffing and Deception:

• Bluffing and reading intentions are central to the Werewolf game. The AI’s inability to fully mimic these aspects likely skewed the results. Werewolves in our simulations might have been more successful because the AI Villagers were not as adept at detecting falsehoods and identifying suspicious behavior as human players typically would be.

4. Statistical Probability in Simplified Scenarios:

• With fewer variables and a more straightforward decision-making process, the simulations may have unintentionally favored the Werewolves. The AI’s decisions, based on logical patterns, might have missed out on the irrational or emotionally driven choices humans often make, leading to a higher success rate for the Werewolves.

What do we take from this?

This is a game of bluff, salesmanship, relationships, trust, and much more. It is a very human experience, and the AI results were very inhuman.

Should we be surprised? Perhaps not.

ChatGPT is a language model that is just trying to predict the next word it thinks we want.

Most importantly, while my degree might have been in science, this was not a scientific process!

I ran this only once; no specific guidance was given to ChatGPT. I could have constructed individual personas and spent time with ChatGPT to give each one unique behavior.

READ the thread with ChatGPT.

WATCH the discussion with ChatGPT.

I would love to hear from you on this. Have you run similar experiments?

Appendix: Table of Results from ChatGPT