Of course, with a free game, there are (at least) two ways to
One system designed to measure relative rank is the chess Elo ranking system, but it was also designed for very different constraints: although it's possible to convince someone to throw a match, it's not possible to manufacture an array of opponents, all of whom are willing to throw a match. More tellingly for the easy grouping of Dungeon Runners PvP, Elo only handles two opponents. Guild Wars guild combat and World of Warcraft arena rankings use Elo, but only to rank the formed groups - that doesn't work at all with pick-up groups, or tracking your rating when playing with a common group and playing alone.
Glicko and Glicko-2 are expanded models moderately based on Elo ranking (with a secondary rating that amounts to the confidence of the system in the rating). It has the same drawbacks regarding stooge accounts and grouping that Elo has, but has been further expanded by Microsoft in the form of TrueSkill™ to manage the additional information (and uncertainty!) of ranking individuals in a team vs. team match.
One problem none of these rankings systems have dealt with, however, is lopsided groups. TrueSkill™, for instance, relies at least somewhat (I haven't yet checked how deep this assumption goes in the calculations) on teams of even size for the way it calculates the relative skill of each team. It would take two very good players to have the same cumulative score as three decent players, yet in Dungeon Runners PvP the two players would have a sizable advantage due to the way balance between lopsided teams.
A simple solution is to compare score averages instead of score sums, but it seems like that loses a lot of the additional information you keep in score confidence values. Maybe it's good enough, maybe not... I'm investigating it. :-)
Another great analysis of these problems was written back in 2006 by Christopher Allen & Shannon Appelcline: Collective Choice: Competitive Ranking Systems. It's an interesting read and has definitely given me more food for thought on the subject.
