Is there a database with all finished games in chess.com history?

Sort:
VIDYUTGANESH

?

justbefair
VIDYUTGANESH wrote:

?

Yes.  

Do you have access?

Yes.

You can bring up any game that you or any member has played using the search function in the archive. (On the website.)

 

VIDYUTGANESH
justbefair wrote:
VIDYUTGANESH wrote:

?

Yes.  

Do you have access?

Yes.

You can bring up any game that you or any member has played using the search function in the archive. (On the website.)

 

I know that, but I want to search for a particular move order, in general without searching it from a specific person's archive 

Martin_Stahl
VIDYUTGANESH wrote:
justbefair wrote:
VIDYUTGANESH wrote:

?

Yes.  

Do you have access?

Yes.

You can bring up any game that you or any member has played using the search function in the archive. (On the website.)

 

I know that, but I want to search for a particular move order, in general without searching it from a specific person's archive 

 

No, there isn't a single available database of all games played on site that can be accessed like that.

VIDYUTGANESH
Martin_Stahl wrote:
VIDYUTGANESH wrote:
justbefair wrote:
VIDYUTGANESH wrote:

?

Yes.  

Do you have access?

Yes.

You can bring up any game that you or any member has played using the search function in the archive. (On the website.)

 

I know that, but I want to search for a particular move order, in general without searching it from a specific person's archive 

 

No, there isn't a single available database of all games played on site that can be accessed like that.

ah yes, that's what I was asking about, thanks....any chance one such database may be built soon?

Martin_Stahl
VIDYUTGANESH wrote:

ah yes, that's what I was asking about, thanks....any chance one such database may be built soon?

 

I'm pretty sure there won't be. The chess.com database is huge with millions of games a day being played.  I don't know the size of the database, but has to be in the ten's of billions by now, if not larger.

 

In 2014, there were already over a billion: https://www.chess.com/article/view/chesscom-1-billion-games-served

Commando_Droid

TWIC from Week In Chess has fresh games every week 

OrphanGenerator
Martin_Stahl wrote:
VIDYUTGANESH wrote:

ah yes, that's what I was asking about, thanks....any chance one such database may be built soon?

 

I'm pretty sure there won't be. The chess.com database is huge with millions of games a day being played.  I don't know the size of the database, but has to be in the ten's of billions by now, if not larger.

 

In 2014, there were already over a billion: https://www.chess.com/article/view/chesscom-1-billion-games-served

Didn't the site stop working for a period of time because 2^31 games were played once? Yeah, it'll be way too huge.

Martin_Stahl
OrphanGenerator wrote:
Martin_Stahl wrote:
VIDYUTGANESH wrote:

ah yes, that's what I was asking about, thanks....any chance one such database may be built soon?

 

I'm pretty sure there won't be. The chess.com database is huge with millions of games a day being played.  I don't know the size of the database, but has to be in the ten's of billions by now, if not larger.

 

In 2014, there were already over a billion: https://www.chess.com/article/view/chesscom-1-billion-games-served

Didn't the site stop working for a period of time because 2^31 games were played once? Yeah, it'll be way too huge.

 

It was an iOS issue: https://slate.com/technology/2017/06/chess-com-app-crashes-on-older-apple-devices-after-people-played-one-game-too-many.html

OrphanGenerator
Martin_Stahl wrote:
OrphanGenerator wrote:
Martin_Stahl wrote:
VIDYUTGANESH wrote:

ah yes, that's what I was asking about, thanks....any chance one such database may be built soon?

 

I'm pretty sure there won't be. The chess.com database is huge with millions of games a day being played.  I don't know the size of the database, but has to be in the ten's of billions by now, if not larger.

 

In 2014, there were already over a billion: https://www.chess.com/article/view/chesscom-1-billion-games-served

Didn't the site stop working for a period of time because 2^31 games were played once? Yeah, it'll be way too huge.

 

It was an iOS issue: https://slate.com/technology/2017/06/chess-com-app-crashes-on-older-apple-devices-after-people-played-one-game-too-many.html

Oh. But still, my point remains.

Closed_username1234
VIDYUTGANESH wrote:

?

Well, lichess did it. Maybe that can help. 

VIDYUTGANESH
Christianf859 wrote:
VIDYUTGANESH wrote:

?

Well, lichess did it. Maybe that can help. 

how to access it?

Masik5

so I can not find a database, to show me how often a particular position was played on chess.com, right? I could only find, how often a certain user reached a certain position, correct?

Ilampozhil25

you can find how often a particular position was reached

just go to their analysis board 

Masik5

"just go to their analysis board" I can find settings in the analysis board view to search in a standard database, but I would be interested to know, how often a certain position was reached in particular in blitz games on chess.com, not connected to a certain user. Does exist, what I have in mind? looks like it doesn't, but maybe I am missing something.

T_R_U_T_H

Very strange to see comments above that it's a difficult task to store all the games. How much data do you really think it takes to record a game of chess?
Way back in 2018 Lichess could compress games to as little as 4.4 bits per move. Not bytes, BITS: https://lichess.org/blog/Wqa7GiAAAOIpBLoY/developer-update-275-improved-game-compressionEven if chess.com had a trillion games played, and you included a little bit of efficiently formatted metadata (user index, date, etc.), I would be surprised if it doesn't all on a USB drive.
The server capacity it takes to manage many users simultaneously has nothing to do with how hard it is to store the games.

Masik5

I am just used to the lichess games database, looks like it is possible to store lichess games there and to search for a position, independent of who played the game, what is a really likable feature, but I do not know what is the main difference to chess.com. I would not know what is "strange" about the question, if another website is already able to offer a search function for positions of games played on the server.

RoyalHunter33

Here's what a full chess.com game database would look like (hint: not going to be available to you for a while).

Size to Store Single game
Using these assumptions, we can estimate a game costs roughly 0.125kb space to retain.

  • 50 BYTES per game (5 bits per move 40 turns per game, so 80 moves is 400 BITS)
  • 64 BYTES for players' data/rank at that time.
  • 8 Bytes for Game data, date, type match, result

So roughly 0.122 KB per game

Total Games

We know:

  • in 2014 there were a billion total
  • In 2021 4.8 billion games were played in that year alone, so let's assume since 2014 the site has had 4 billion games/year

So at the end of 2023 this would by 37 billion total games played. 

Potential Database

So the total SIZE in space would be 4.5 terabytes, made up of 37 billion records (rows) and over a hundred columns to evaluate. Yes, this could be placed on a USB drive, and certainly on powerful servers. However, the likelihood of having users' leverage this dataset for any meaningful analysis is pretty absurd. The CPU that would need to properly filter and calculate the number of wins/losses for a Sicilian defense would be huge.

Looking at Microsoft's recommendations for SQL databases, our potential database would be unheard of with having far more columns than recommended, string size too long, and too many objects.

For simplified home users who may be familiar with Excel (not the best tool for sure for data analysis), consider that the size would need to be reduced by over 1000 times to fit in a table, and then the computer would need to be filtering/calculating a data set of a over a million rows with hundreds of column. 
End result, it's too big to be evaluated as a whole.

Masik5

That doesn't explain why another website is able to provide exactly the mentioned feature since years.

Martin_Stahl
Masik5 wrote:

That doesn't explain why another website is able to provide exactly the mentioned feature since years.

They have a lot fewer games, think at least one or two orders if magnitude smaller. I think they are also using a completely different type of database engine as well.