Average Opponent Rating Has No Meaning

People love statistics but they can be misleading.

http://en.wikipedia.org/wiki/Lies,_damned_lies,_and_statistics

"Lies, damned lies, and statistics" is a phrase describing the persuasive power of numbers, particularly the use of statistics to bolster weak arguments, and the tendency of people to disparage statistics that do not support their positions. It is also sometimes colloquially used to doubt statistics used to prove an opponent's point.

There was quite a hue and cry when the Average Opponent Rating (AOR) was temporarily dropped recently. Now that it has returned evaluate these two players.

fcpanginen: Rating 1927 and AOR 1832. Plays down by 95 points

olopitan: Rating 1899 and AOR 1880. Plays down by 19 points

Note: Playing down, even by a huge margin, is not a problem. The rating system takes care of this. A player gains very little with a win and loses much with a loss or draw. It all works out in the end.

Based on rating alone, as it should, fcpanginen is better. Others might argue olopitan's opponents are stronger and this has an impact on their evaluation.

If these two players were to play a match, of however many games you feel necessary to prove the better player, who do you pick?

First choose without looking at any other statistics except rating and AOR.

Second choose after you look at any other statistics available. If your choice is different what changed your mind? 

An interesting post.

 First of all, your two examples have "statistics" (rating, AOR) that are a little too close for me to swing any way to begin with.  

As a rule, I would always go with the guy who picks fights with more stronger players, but these are a bit close ( 50 point diff. in AOR)

Like you said the ratings formula takes care of it.  If 1900 guy scores 90% against opponents rated 358-374 points below him, then that's his correct rating.  You can't "trick" the rating system by playing up or down.

If a separate 1900 scores 50% against equal opposition then his rating is equally valid.

One problem though is you start at 1200 and your rating is very volatile.  So depending on the order of your wins and losses someone may have a 50% score vs 1800 opposition and be rated higher or lower than 1800 (so long as they have less than, say, 200 games completed).

And I have to admit when the avg opponent is way off it does make me question the rating... When I had a quick rating of 2100 but my win percent was over 90% and my avg opponent was 600 points lower (I sent completely open seeks and never declined a game) I had to wonder.

Not that 1900 is correct either ;)  I'll start playing again sometime (against players near my rating only) and see where it ends up.

Let us try something else. Maybe formulas would be more concrete. The simplest is something like this.

fcpanginen is (1927 - 1899) rating points minus (some factor times 48) AOR points better than olopitan.

Let us say my formula is 28 - (0 times 48) = 28 points better. In a 100 game match, according to this blended rating, fcpanigen is the favourite due to his 28 point advantage.

With a formula it does not matter how close the statistics of the two players are. You can pretend the numbers are further apart and still suggest a formula that you would apply to other players.

Even then,  what if player A played tons of open tournaments (not Swiss, but chess.com's round-robin flavor) where he was regularly paired with atleast 5 players rated 400+ points lower than him each round) ? (This would drag down his AOR considerably)

What if player B selectively cherry-picked his opponents to be ">1800" at all times? (ensuring a higher AOR)

To the OP's first post's point, you really need to dig deeper to see how the two would fare against each other.

The rating system alleviates this by giving very small rating adjustments, but it does not eliminate it.  With a high enough volume of games, playing down (or up) does make a difference.  Just look at ChessNetwork.

Let us break it down once more. Does the AOR add value to the site or not?

Yes.

The ChessNetwork problem was fixed. The minimum gain for a win is now zero points rather than one. He no longer plays rated games.

Everybody missed the (accidental?) pun... average opponent rating has no meaning.

It does from  an "lower bound filter" standpoint. If have no interest in playing somebody with a AOR that is considerably lower than what I consider competitive, I find use for it. For e.g. a 1900 with an AOR of near 1500 is probably not going to be somebody I challenge/want to play (outside of a tournament where I am pretty well damned forced to)

But if I am setting an upper-bound, I will state that it is less accurate for my needs. If I was looking to play a 2100+ and really get a tough fight, I'd be silly to assume that he must have a 2000+ AOR at all costs. I'd invest time in looking at other aspects of his  stats/game history.

Yes, there was a bug in the calculation that contributed to the extremity of the example, but the fact remains that someone can inflate their rating by cherry picking their opponents.  The AOR is currently the only way to idenfity when this is the case.

It is quite possible for two 1800 rated players to be of two completely different strengths:

Player 1 has a win/loss record hovering right around .500, an average opponent rating of 1800 and a rating history graph that's plateaued.

Player 2 has a win/loss record of around 80% wins, an average opponent rating of 1500 and a rating history graph that shows a very slow but steady incline.

Who would you expect to be the stronger player?  How would you tell if AOR were removed?

I suppose if you artificially change the pool of players then the rating will be altered as well.  Even though Chessnetwork's specific exploitation has been taken care of, it's a good example of how manipulating the pool can affect the rating.