Matchmaking - they admit it. (Destiny)

by Kahzgul, Thursday, January 21, 2016, 22:54 (3323 days ago) @ Ragashingo

Yup. "Skill based" matchmaking is confirmed. Not discussed is the "closed loops" of skill players only being matched against the same pool of teams and players repeatedly, regardless of match outcomes. Briefly mentioned is that they are working on addressing how the current matchmaking is causing latency by not prioritizing connection quality. Also not discussed is how their netcode gives an advantage to lagging players.

Matchmaking aside, they also ignored the current "thorn" meta in crucible, as well as the state of snipers, shotguns, and then all of the other guns that aren't The Last Word or Thorn.

Ok, I'm very confused:

1. What is "Skill based" matchmaking and how does it differ from your ideal form of matchmaking? It seems obvious that the matchmaking system uses multiple inputs (skill, latency, etc) to form matches so I really don't know what you are going on about here.

I'll try to answer this as clearly as possible, but you're right that it's complicated. My basic issue is that it seems like that game generates a pool of about 20 players whenever I log on, and those are the only 20 players I'm going to see all night long. Even after a massive blowout and the "we're breaking up the teams" message, I'll be connected to the same players for the next game. The "skill based" system Bungie uses does not seem to look at whether or not you're in a fireteam, nor does it look at success or failures from that particular evening. Rather it seems to use a lifetime of data, and as such it's found that I'm skill 7 (I'm making this number up) and only will ever match me against other skill 7 players. The problem with that is that I'm never matched against skill 8 or skill 6 players, so there's never an opportunity for me to demonstrate that skill 7 is an accurate value. Instead, I'm only matched against other skill 7 players, and they with me, and because there's about a 50% win rate we will never, ever be matched against anyone outside of our specific pool of 20-ish players. Let's examine that win rate, because on the surface a 50% win rate seems ideal, right? But it's not. For some reason, there are two guys who are insanely good in this pool of players, and they always play solo. When they get matched together, their team wins by mercy rule every single time. When they are on opposite teams, they split about 50/50. But the rest of us lay our entire fortune and the feet of these two amazing players. For whatever reason, Bungie's "skill based" matchmaking (this really is a misnomer) keeps those two guys in this bracket where they absolutely dominate the rest of us, and we're considered just as good because our win rates generally match that of these two guys. Maybe a little lower as we're sometimes on the losing side of the uneven teams.

The matchmaking Bungie implemented has resulted in closed loops of player pools based on how little variance there is in the skill section game to game. It's giving too much weight to the perceived skill of the players.

Now, I don't pretend to know what Bungie is doing to make their matchmaking happen, but I know that in December it got really shitty. Suddenly every game I play against at least one guy in South America with a shitty connection, and it's the same overall group of players every time. This didn't matter during SRL because of how that gametype worked, but it sure as hell matters in crucible. Just last week my buddies and I decided to run wild in Rift, and not only did we get matched against the exact same 6 solo players three mercy rule games in a row, but we saw the same guy leave and rejoin 3 times in one match (I assume because he recognized us and didn't want to get stomped for a fourth game). That kind of matchmaking should not happen. It's not fun if you steamroll every opponent, and it's not fun if you lose by 10,000 points every game. When the game says it's breaking up the teams, it needs to not rematch those players for the next round. Before December, it didn't. Now it does, and Bungie - until today - has been silent on the issue.

When I worked on Call of Duty, matchmaking was a big deal. People still complain, certainly. What we landed on, which I think is probably the best, is that we have a separate ranked ladder with ELO matchmaking where you will never play the same team back to back. Then, for unranked play, we look for hosts with the best connection, and then fill in around that host with similarly ranked players. BUT we put in a randomness factor. Every few games you are matched completely blind without any attention to your actual rating. This means sometimes a godly player comes into the lobby, and sometimes a terrible one does. It also means you have a chance to dramatically increase your rating when you get matched in an expert lobby, or dramatically lower it by stinking in a poor lobby. This outlier data is immensely helpful for finding real skill values, and it also gives a much more varied experience to the players. If they don't like their lobby, they can always back out and re-join later. It *should* match them to a new lobby (though in practice it won't always if they haven't at least played 1 game in that lobby first).

Anyway, Bungie's matchmaking changed dramatically in December and the result is many many many re-matches and frequently being connected to people with simply awful connections, which - because of the weird Destiny netcode - lags the whole game for everyone.

2. Seems to me that any individual game, especially with players of similar skill level, can result in a close call or a blowout based just on a few mistakes or clutch plays. Doesn't getting matched against the same group of players make sense? Should one win or loss really be enough to kick me into an entire different skill group?

No, and that's not what I'm advocating. It's more that if you just won or lost a blowout game (mercy rule game), and the lobby gets broken up, you should never be re-matched with the exact same people for the next game. There are have got to be 11 other people who weren't in the last game that you could be matched with for the next one.

3. What kind of discussion of lag do you want? It kinda sounds like you want them to admit they are horrible and apologize. I haven't played a multiplayer game yet, from any developer, where lag isn't sometimes an issue. I can't help feel you're blowing this out of proportion and focusing too much on it...

It's just a constant frustration. Destiny, without lag, is one of (if not THE) most nuanced and exciting PvP games I've ever played. There are loads of variables and it favors teamwork over individual skill, but not so much that a highly skilled player can't wreck a medicore team. With lag, all of that goes out the window as your precious resources are wasted on someone glitching through the map who can then kill you from the other side of a wall. Most pvp games have a single host managing connections so that if one player lags, that player is a sitting duck who can't hurt you back. In destiny that player is immune to all damage and can still shoot. It is definitely more generalized lag since the December patch, but it is still pretty bad. I keep focusing on it because it's kind of mind-blowing how long it's been seemingly ignored by Bungie. For any competitive PvP scenario where you want to implement a skill based matchmaking system, you have to do everything in your power to eliminate lag and other non-skill elements that can dramatically affect gameplay. Bungie isn't doing that. You're got to quarantine the bad connection players in order to preserve the game for everyone else. Giving a higher priority to connection quality will address this issue (this update seems to imply they will make that change, but then again nothing ever "implied" in the updates has ever come true, so I'm loathe to believe it until they outright say it).

4. Nobody is ignoring the Crucible meta. Did they discuss it in this specific weekly update? No. Are they ignoring it? Absolutely not. Simple common sense and the history of Destiny receiving several weapon balance updates should tell you that. Furthermore, there was a great two part Planet Destiny podcast around the end of the year where Jon Weisnewski and Sage Merrill talked at length about each and every issue you just mentioned. And more. Search out Crucible Radio episodes 27 and 28 for that.

Maybe ignoring is too strong a word, but Destiny's weapon balance history is atrocious. Once every 6 months really isn't balancing weapons at all, it's just changing the game to be different. If you're not fine tuning or staying on top of trends, you're not balancing the game. Mass Effect 3's weekly updates is the gold standard. Bungie is getting a pretty crappy "also ran" or "participant" trophy in my book.

I'll look into the crucible radio podcasts. For some reason I thought that these were more fan-based and didn't include actual devs? I've never listened to them so it seems that I'm missing a huge font of information here if that's truly the case.

Sorry to go all numbered list on you, but it's just not fun to read your repetitive posts on things like matchmaking and balance. Especially when the point beneath the surface always seems to be Bungie is incompetent and not paying attention. :/

No worries! I have a definite opinion on what works and doesn't work for PvP multiplayer and I really really really want Bungie to make the changes that will bring Destiny up to being what I consider the pinnacle of PvP FPS games. The potential has been there since day 1, and I know Bungie has the talent to do it, but at the same time I see them making mistakes that other FPS games solved a decade ago and that frustrates me. Map voting in lobbies would be nice. Custom games please? Just an "exploration" mode for PvP maps so you can check out the geometry and practice movement without getting sniped out of every jump would be a huge help for noobs and pros alike. If I didn't care so much about this game and Bungie, I wouldn't get as worked up about it as I do.