Trying to moderate a large community on the Internet is not an easy job. Anyone who tries to enforce basic rules of civility and respect in an open Internet community of any decent size is fighting a never-ending battle against John Gabriel’s Greater Internet Fuckwad Theory, which turns usually normal people into complete, well, you know.
Those problems are only compounded on a service like Xbox Live. Anyone who has joined a public match in a first-person shooter on the Xbox 360 knows the platform is infamous as a haven for antisocial, antagonistic, hate-spewing preteen boys pumped up with adrenaline from shooting virtual people in the virtual face.
But Stephen Toulouse, who served as Xbox Live’s head of enforcement for five years before departing earlier this year, said managing Xbox Live’s tide of miscreant behavior was just a matter of bringing consistent punishment to those who break the rules.
“The reason bad behavior on the Internet occurs is because of a lack of consequences,” he told Ars Technica in an interview about his tenure at Microsoft. “I grew up in the world of arcades, and playing sports in a local park, where if I spouted off about some kid to his mother, there could be immediate and physical retribution. There were consequences [for] that behavior.”
The beat cop
Toulouse’s job at Microsoft was to bring consequences to the virtual world, using a team of hundreds of “enforcers” around the world to make sure Xbox Live’s tens of millions of users were sticking to the Xbox Live Code of Conduct. That meant stopping game-breaking problems like unauthorized cheating and console modifications, but also handling interpersonal issues like harassment, abuse, threats, obscenity, defamation, spam, and “racial, ethnic, or religious slurs.”
While many users are under the impression that the entire enforcement process is automated, Toulouse said every complaint sent in by an Xbox Live user is actually reviewed by a human, usually in much less than the targeted 24-hour window. Toulouse said he couldn’t discuss the forensic “secret sauce” his team used to confirm that those complaints were genuine, but he assured me the enforcement tools lead to an error rate that was “ridiculously low.”
They also allowed Toulouse and his team to confront people with the evidence of their misdeeds before bringing down punishments like temporary suspensions or permanent system bans. “It really is one of those scenarios where, when you have to answer questions, it was like ‘Yeah, we really did catch you,’” he said. “They just want to know they were caught.”
Enforcement wasn’t just a matter of reacting to complaints, though. Sometimes, Toulouse’s enforcers would serve as “beat cops” on the Xbox Live scene by jumping in to do spot checks in games that tended to contain the most problem players. More than actually catching rulebreakers red-handed, Toulouse said this kind of active Internet policing provided a valuable visible reminder for players that they needed to stay on their best behavior.
“I can’t tell you how many times I joined games and people would quiet up or go away. Those are the people who were misbehaving… I never had a problem with the entire game going silent, but I’d join a game and then hear one of the players go, ‘Oh, now you shut up’ to the other player.”
But dealing with problem players was nothing compared to dealing with certain parents, who “might have a certain set of expectations and perceptions and they may not match reality,” Toulouse said. Even parents that were very conscientious about limiting a child’s access to the Internet on the family computer often didn’t realize that today’s game consoles were similar general purpose devices. “So they put a game console in their kids room, without quite realizing that if they don’t somehow lock down the router or lock down the Internet connection at night, I guarantee you the child is doing something at 2am that the parent would be shocked by,” Toulouse said.
“One of the biggest problems I ran into is what we called the ‘Christmas morning problem,’” he continued. “We had these great parental controls on the Xbox, and other consoles have them too, but the parent hands the box to the child and says, ‘OK, fire it up,’ and the kid creates an adult account. Six months later, that kid does something horrible and gets punished. The parent comes to us and says, ‘What did my child do?’ and they are shocked and appalled. We tell them about the parental controls and they’re very excited, but they never knew… I think the industry has a long way to go still in educating parents about what has become a general purpose device and how much it interacts with the Internet.”
When Toulouse started in 2007, there was a rule in place banning all discussion of sexual orientation, gay or straight, from the service entirely. The rule’s main purpose was to provide some protection against the extremely common, extremely pejorative use of terms like “gay” as a slur against an opponent in an Xbox Live match, Toulouse said, partly on the theory that such terms just “didn’t belong in a gaming context.”
This kind of automatic punishment started to become untenable, though, when players began complaining about facing punishment for referring to their own sexual orientation as a matter of pride in Gamertags like “theGAYERgamer” and in other areas of the service. So, in early 2010, Toulouse and his team worked with GLAAD to modify the policy to allow for certain uses of sexual orientation terms, while still limiting harassment.
While the blanket rule against sexual orientation discussion might have made sense when Xbox Live was primarily an online gaming service, Toulouse said, things weren’t quite as clear-cut once the service started incorporating more general social networking features like Twitter and Facebook. “Suddenly it became sort of archaic and silly to [punish people who would] just state ‘I love my boyfriend,’ and [have] that person be male,” he said. “The vast majority of [the use of gay terms] was still pejorative, but it was one of those things where you take a step back and say, ‘You know, the population has changed, the function of the service has changed slightly, so we need to go ahead and take on the effort to divine whether something is pejorative or not.’”
And as the Xbox Live player base grew over the years to include more casual players, Toulouse said he found his team had to scale up its capabilities faster than normal to handle an increased rate of complaints. “[The hardcore players] might not complain, and it leads you into a false sense of where people are having safe experiences,” he said. “You find in the hardcore world people are more tolerant of miscreant behavior,” he said. “They either are trash talkers themselves or don’t view that as necessarily against the rules, even though it is. But you start to throw in the casual gamer that plays the occasional Modern Warfare 3 match, and that population expects that there be repercussions for breaking the rules.”
As the public face of Xbox Live enforcement, Toulouse said he occasionally ran into disgruntled players begging to get their accounts unbanned. For the most part, though, he said the feedback he and his team got at public events was overwhelmingly positive, despite the costs it imposed on them.
“If you think about it, it’s sort of a tax,” he said of the enforcement role. “Resources that went into my team are resources that don’t go into doing new things in the avatar marketplace, for instance… People at the end of the day want to have a safe and enjoyable experience, and I don’t see how you can increase those players’ experience without some form of enforcement or contravention.”