FIx the rating system

The rating system on Chess.com is a total waste of time. It bears no resemblance to reality. Its a nice idea, but really ruins the site for me. I hope they can fix it. (If the managers of this site reads this post - I'd be very happy to help them out - if they care to contact me.)

I kinda wanna agree and disagree. I’m pretty sure it’s a bit unrated. Everyone’s rating should get a 300+ edit. Could make things clearer. 

Honestly this is a bit of a tough one for me.

What elements of the rating system do you believe to be unreasonable?

A common  complaint is that the number in one system doesn't match the number in another.  That ignores that ratings are only relevant within a system and do not have any particular reason to match another system.  The 300-point comment would be a way of getting close to another system that starts newbies at 1500 while Chess.com used to start newbies at 1200 and still has a lot starting at that number.  A one-time adjustment of 300 points while starting newbies kind of centered around 1200 would merely result in people complaining about the under-rated newbies.

The site uses the Glicko rating system and it's a well established and sound system. Pretty sure other sites use the same system. 

As has been said, the starting rating can have an impact on the overall pool, but you can't really compare ratings across pools, especially if another site starts everyone at a higher rating. 

The whole purpose of a rating system is to show past performance in a pool and aid in pairing players of similar rating. 

Replying to why I think the rating system is not working - a few reasons in priority order:

1. I play 15 min games rapid and 5 min games in in blitz. My rapid rating is 1200 and stable; my blitz rating is 800 and stable. I don't believe this difference can be explained because of the different nature of the game.

2. If the rating system works but somehow settles to different absolute values, the relative rating should be accurate so I looked at my percentile: in blitz I am 50% in rapid around 90%. Again, I don't believe this difference can be explained by the different style of game.

3. I think the basis of Glicko system is a clever idea - ratings to move more rapidly for unestablished player with an uncertain rating compared to stabilised players. I suspect the system only works if most established players have an accurate rating but we never get to that point because the mass of established platers are playing other established players with inaccurate (near random) ratings in the middles rating bands.

4. I wonder if starting unestablished players with a rating of 1200 (which is around the 90%-ile might be too high & causes the on-going inaccurate ratings for the mass of players. Above ratings of 1200 perhaps the rating system works, but below it appears not. Perhaps unestablished players should start at around the 50%-ile  - i.e. 700 to 800 or lower.

5. I wonder if the system is not generating accurate ratings for mass players because although the ratings of unestablished players varies rapidly initially (because of low confidence) the rating of the established player moves the same amount even when they play an unestablished player (when there is doubt that the rating of their opponent is accurate). It would seem better that both ratings are impacted by lack of confidence in one player's rating. - possible the rating of established players doesn't change after playing an unestablished player. (It would be interesting to test in background to see how stable the ranks of established players differs from the two methods).

5. Anecdotally, I seem to have a different win percentage at different times of the day. Could this be true generally & is it an indicator of the relative number of established players vs unestablished players playing at those times?

6 Checking the histograms of player ratings in global stats - the number of players in blitz and rapid are much higher than the sum of the number of players in each 100 point rating band. What is going on here?  Perhaps just some minor error but is it an indicator of some wider problem in the rating system.

7. I play the chess computer on the site occasionally and find I can reliably beat the computer with a rating of 1500. Have the chess computers been calibrated against people with reliable ratings - if so, is my rating 800, 1200 or 1500. Not a big issue this - but it adds to lack of confidence in the overall system.

8. Has the whole rating system been calibrated to people with reliable ratings (e.g. from club tournament play playing volunteers from chess.com across the rating bands) It might be a good method to establish if the rating system is working and lead to fixes. The ELO system and Glicko system are effectively the same barring this method for adjusting rating of unestablished players. (However some bias will exist - ELO system wonl ever lose of gain points since winners and losers add/deduct the same number of points; Glicko does not guarantee this so some bias will exist especially if unestablished players are given an initial rating different from the average of all players).

9. It would be interesting to experiment with other dimensions of confidence a players rating beyond just the number of games they have played - e.g. subscribers to the site rather than non-paying users; more use of the AI cheat engine etc.

Overall, I have lost confidence in the the Glicko system - just because it sounds a good idea & is an 'industry standard' doesn't mean it works. Chess.com will have vast amounts of data to prove/disprove it works & experiment with variations - testing alternatives before implementing. Glicko has parameters T and C which need to be determined / not pre-defined - have alternative values of these been tested? This would show how sensitive the rank ratings are to different values of these parameters. Is the overall system inherently stable or inherently chaotic - I suspect the latter as things stand.

The main reason for this difference are the different pools you are competing in.

In blitz pool, you have all the titled players "stealing" away rating points from the pool. In rapid pool, there are much less titled players, so on lower levels, the difference is pretty big.

As you rise up in ratings, rating difference between rapid and blitz becomes smaller and smaller until players rated 2 200 or so, start being higher rated in blitz.

In short, this difference exists because people playing blitz and rapid are not completely the same.

Almost the whole community plays blitz. Almost no one (relatively speaking) plays rapid. Among other things, once you get past a certain rating, you start running into some very suspicious opponents in the rapid pool. You still get them plenty in blitz but not at the same frequency. That's certainly been a major deterrent for me and most players I've talked with about this.

And I think #8 is basically correct; most people I know have vastly higher *blitz* ratings than rapid, but that might not be the case at other rating ranges. 

I agree this is possible - but I don't believe the pools are so different, nor that the game is so different to change my percentile from 50% to 90%

Unfortunately, its just an opinion - I can't prove it - but probably chess.com have the data to prove or disprove this.

@BKPete, regarding points 1 and 2, isn't absolutely impossible to have wildly different ratings and percentiles in blitz and rapid. They are different games where time and how you handle time trouble can cause a big difference in game outcomes. Some people are just not as good at faster time controls.

Points 3,4, and 5 (first one) are part of the rating system. Players have a rating deviation (RD) value and that impacts the magnitude of change and how results are calculated. Playing a player with a high RD and losing isn't going to have the same result as playing a player with the same rating and a low RD. The site also starts players at their chosen level (from 400-2000), but with the way RD works, that doesn't have a huge impact on the overall system.

For 5 (part two) time of day can have different impacts. One, the overall pool of players at certain times is different. It's not terribly different  from OTB rated chess where players in one region can have the same ratings as another region but be at different strengths relative to each other. Then there is the personal impact on time of day in playing. Someone after a long day at work will have differences from their same play earlier in the day when fresh. 

On 6, I've never added up all the bands but I'm going to guess they aren't really that far off.

On 7, bot ratings are estimates and are notoriously off, especially at lower rating bands. It's hard to make a bot play at a certain rating level. That's not what they were designed to do and the settings have to be tweaked to try and get some semblance of reality, which doesn't work very well.

For 8, Mark Glickman is a mathematician and knows what he is doing. US Chess uses one of his designed systems, which I believe is tweaked specifically for their needs, and what the site uses is a later version of that original one. Pretty sure it's considered a sound and accurate system within a pool of players.

Just try to find some users with different ratings. You will see that (on average) as you climb, the difference gets smaller and smaller, and on some 2 000+ rating (I am not sure where exactly) people start being higher rated in blitz.


Now, there are exceptions of course, and you might be the case where the difference is just a bit bigger than usual, but on average, I think it is the most plausible explanation.

Here you can find an average difference for people who have completed this survey, and who have OTB rating as well:

https://chessgoals.com/rating-comparison/

I hope it will be helpful.

But if you look in the global stats - there are more rapid players than blitz players.  Are those figures correct - as per my question on point 6

Hm what about number of games played? Maybe people who just join the site and play a few games might be more likely to click ten minute (or whatever the cutoff is, I have no idea) than three minute. 

Pools aren't some zero sum game where there is a finite pool where high rated players steal from others. Large rating differences result in either no rating loss to the losing player, or a minimal amount, and the higher rated player either gains no rating or a minimal amount. The vast majority of players are going to be getting most of their games with players closer to their rating, contingent on their seek settings and potentially how many Swiss and club events they play (where the pools can be less random)

Yeah, but games played are not fair assessment either. I mean, it is much easier to play 100 3|2 games than 100 15|10 games.


It is not that whole community doesn't play rapid, it is just that higher rated people, mostly plays blitz, and on your level it is as you've said - most comunity plays blitz.


Which is exactly why there is that rating gap on lower levels.

I am not saying stealing in a bad way. It is a colorful way to describe that as there are more extremely better players than the rest of the pool, rating gaps will be bigger. And of course, it is not a direct transfer of rating between 2 500+ rated player and 800 rated player. 2 500 takes rating points from 2 400, 2 400 from 2 300 and so on.

In rapid pool it is pretty rare to see people rated over 2 000. There are some, and I've played some, but there are not many people like that.


In blitz that is not the case. And if there are let's say 50 "God tier" players in blitz, it is obvious that they will have much higher rating than top 50 in rapid where there are not that many  "God tier" players. And there are a lot more "God tier -1" level in blitz as well.

So the conclusion is, in blitz you have many more 2 900+ players, in rapid not that many, so the rest of the pool will have to compensate this difference, hence the rating will be shared more evenly in rapid rating pool. For me this is logical. And that is why the lower you are in the food chain, you will probably have a bigger difference between rapid and blitz.

I think that OP is a bit of an extreme case, based on the link I've shared, but the difference is there for sure, especially on lower levels.

10|0 which used to be blitz, and still is for OTB play, has been one of the top three time controls on site. In 2017 it had the highest number of games played as a single time control. Now that the time control is Rapid, I have no doubt rapid can have more players than blitz, especially now that the league system is in place.

The numbers from two months ago were probably different.

That's really interesting. I have never played 10/0 on here (or if I did it's been a million years) so I had no idea what category it was. 

I felt like such a loser when I looked at ur profile.  But yeah, probably right about the rating complaint.