"The time I tried to ruin Halo 2" (Destiny)

by Kermit , Raleigh, NC, Wednesday, July 03, 2019, 09:24 (1759 days ago) @ CyberKN

John Hopson, A Lead User Researcher for many Microsoft and Bungie games, wrote a pretty good article about the time he tried to tell Bungie that Halo 2's matchmaking system was going to fail, and why he was wrong. It's a great insight into how you can draw the wrong conclusions by looking at data incorrectly.

I loved this article (and at the time I also loved the Halo 3 article in Wired that was referenced). I work at a software company and have been pretty heavily involved with design and user testing. We're all biased, and that shows up from every participant in the process. The designer is biased and notices the feedback that justifies the design. Often, users are not reliable witnesses to their own preferences--they may say that they consistently use some aspect of the interface that you rarely observe them using, for instance. A year or so ago, we implemented a pretty radical change in our design, and got feedback from a few users very late that they didn't like it, but it was just a few, and that's another thing: sometimes the most articulate and strongest feedback gets the most credence even though it's not representative. We recognize now that our new design is worlds better than what we had.

As with most things in life, the map is not the territory. Data is subject to misinterpretation.