Not quite how it works... (Destiny)

by RC , UK, Monday, March 23, 2015, 19:44 (3781 days ago) @ BeardFade

TrueSkill is not a terribly accurate measure of skill because it heavily weights winning,

In fact winning or losing is the only gamestat Trueskill takes into account.

which is not a skill/talent that an individual controls.

Each individual of a team will have a partial contribution to a game's outcome. Trueskill recognises this and that is why the larger the team size is (and the fewer teams), the more games it takes to converge an individual player's skill estimate.

Or in another way, the amount of information from each game has to be split up between the members of a team. In small teams (1v1) all of that information is about one player directly relative to another player, but in larger teams (8v8) the information has to be split many ways.

Winning requires all team members to play well (or at least enough to make up for the worst).

Nope, they don't need to play well, they just have to be better than the other team.

This is why you still get terribly unbalanced matches.

Trueskill makes no attempt (and can't really be expected to) account for performance variations or run-away wins. The example given on the trueskill page is of 2 people who are racing cars. B makes a bold move at a corner to try to overtake Player A, they fail, and crash. B drives slowly for the rest of the race - knowing they've lost - and end up crossing the finishing line 30 seconds after A. B isn't really 30 seconds worse than A, and if they hadn't tried to overtake would have come within a few seconds of A. But the point of Trueskill isn't to find 'how much better' they are in score-terms but rather to find a relative ranking and theoretical draw probabilities. B will do other races against C, D and others and win those instead, placing their estimate between A and the rest of the field.

A great player who plays with bad teams will still have a lower TrueSkill than is accurate for his or her abilities.

If a player chooses to play with those players, then that is their choice to be ranked the same as them. Conversely, the people they play with will have slightly higher skill estimates than they really are.

A totally valid criticism of Trueskill is that it can't separate, identify and quantify the 'teamwork effect' of a familiar group of players since it treat's the team skill as a sum of the individual players. Arguably too, the skill estimates are only valid in too narrow of an application. In the real world of online games with various gametypes, teamsizes and maps there is no inter-prediction of skill and no model of how to do so.

However, in the narrow scope of a single playlist, a player is expected to be matched into different teams, play against different teams and their individual skill is slowly teased out from the results of many games. Everyone does this, so the incidence of 'bad teammates' vs. 'good teamates' and 'I had a good game' vs. 'I had a bad game' should even out.

Personally, I think creating an algorithm that does not weight winning so heavily and K/D and KA/D more would be a better one. Positive k/d's almost guarantee you took more points from the other team than they took from you, which essentially is the key to winning. Get enough positive K/Ds and you win. That way, individuals with high K/D's would get ranked higher in TrueSkill and you would face people as adept at killing as you are.

Problem is that any metric that is not about winning and losing directly is potentially open to abuse. It also immediately favours certain playstyles. i.e. go around killing, hanging back, sniping, rather than actually contributing to winning the game by making captures.

If you remember, Halo Reach had the 'Arena Rating' system that attempted something along these lines. It favoured a playstyle around hoarding power weapons for yourself, rather than what was strictly best for the team's liklihood of winning.