In my idle time, I have been playing various computer games. Some of these games (such as World of Warcraft) feature randomly assembled teams that compete against each other. After playing these team player vs. player (PVP) games for a while, I became frustrated at the total randomness in the quality and skill of the players that were on a team. Elite players dominated and bad players could profit by doing nothing and still be on a winning team.
This frustration brought back memories of other team sports that I have participated in which had similar issues. Years ago I dreamed that such issues might be solved by creating a rating system similar to the one used for chess and applying it to members of team competitions. It would be great to go to a pick up game of touch football or Ultimate Frisbee and know that your teammates were of similar skill and aptitude. But I knew such dreams were fantasies because even if a rating system were possible, there was no way it would ever be applied to real sports. The overhead of maintaining information on all the players (and making players use it) would be quite cumbersome and most players would not play enough games so that a proper rating could be given to them.
However, online computer games do not have these problems. It is easy to track the results of players and it is even possible to track performance data so that it is possible to determine which player contributed more to the team effort. Also dedicated players of computer games tend to play a lot more team matches, usually 1000s of team matches a year. In this case, a real multiplayer rating system (using the Elo system that is used for chess as the base) might be quite possible. How could I demonstrate this? One way would be to create a simulation of players and team matches and then show that a rating system would correctly determine the relative skill of the players to some degree of accuracy. This turned out to be large effort and the results produced many interesting facts, many of them unrelated to the number of players on a team. Because of this, I decided to write an extensive document about what I had done.