Please devote more resources to bandwidth

There are a few indications I've found that Chesscom is trying to conserve bandwidth at the cost of giving everyone what they want.

• There is no Chesscom Open Database
• Every list of games has to be downloaded one page at a time
• Libraries can't be downloaded en masse, but one game at a time
• The ability to download an entire tournament as one file disappears (at least temporarily) about one hour after the tournament is over -- so there's a very limited window there
• The Published Data API is exactly what you'd do if you wanted to open up access to your database very selectively and in a way that almost no one would be able to use
• Videos seem to be compressed as much as possible without compromising quality, except they go too far anyway and create tinny, slightly-pixelated results
• Getting rid of Live may have something to do with this, too
• Your streams are hosted elsewhere

Of course that's all normal, best practice. Except:

• An open database, or at least an open elite database, would make a ton of people happy
• Being able to download every list of games in one go would also have that effect
• Being able to do that with libraries would move people away from the broken blogging section and over to the quite-functioning library section
• Giving us tournaments at any time would lower the number of people tuning in, unfortunately, But it would still give users what they keep asking for
• Creating actual high-def video, a la YouTube, would revitalize a dying section of the site that nonetheless keeps being used to sell the diamond membership

The answer seems pretty obvious. Devote more of your resources to bandwidth. You hosted a legendary tournament just now, largely on our dime, and to someone like me all that ever came of it was an amazing collection of PGNs. It's good for your brand, but it should also be pretty obvious from reading these forums that most users couldn't care less about your brand. They make decisions based on how much value they get for their money. If you devote some of your resources that way, that increases the value of the site, instead of alienating those of us who may be fairly annoyed that the recent price increase seems to have gone not to any UI improvement, but to giving half a million dollars to one person for adding a handful of top-quality games to everyone's database.

Just a very biased notion, but I really think, in the end, it would shore up your user base instead of eroding it.

Why do you spend all that effort downloading games? 

It can't be for your own use.  Do you resell/repurpose the data?

Who did they give a half a million dollars to?  For a few games.

Like a lot of people, I try to build my own massive databases. This is why there is such a thing as the Lichess Open Database.

You're right that this is not just for my own use, however. Databases like this are great for redistributing freely. I don't know if my current version of Chess Nerd will stay the way it is, but for now I'm putting up databases from freely-available resources, including what I've been able to glean via others from the Published Data API.

As far as half a million dollars goes, didn't they award that to Wesley So yesterday? He seems like a great person, but that was site fees. Obviously referring to the Finals of the CGC as a few games is being rude, but to me that's what it is.

Not everyone is enamored with big data, but I absolutely am. Part of this is the simple exigency of the moment. I don't have the capacity to learn or study, so I'll probably never break 800 in rapid. I put my energies into busy work, and that's turned into a love for (and obsession with) using ChessBase on an almost constant basis. It has tons of big data uses that don't include grandmastery uses. :-)

As per selling your data, not a chance. Someone else may want to, but I don't think that should be an issue. It isn't for Lichess. I believe they say on their open database page that the games can even be sold if one wanted to, however I doubt anyone has bothered to try and charge for a free resource.

If you aren't familiar with why people love working with databases in CB or Scid, that shouldn't reflect on these ideas. The point of denigrating the CGC (which wasn't a good idea, in retrospect) was to point out what the tournament will mean to posterity.

The size of the chess.com game database is huge. There are over 10 million games a day being played. The site has been around since 2007 and while the numbers weren't that high in the past, there's likely well over 100 billion games. 

Just querying that database to get a subset of the data takes resources and being able to download it would take a lot more, including bandwidth.

Regarding the ability to download tournament games, that's possible in the completed tournaments page and doesn't expire.

Live is being retired to allow for upgrades/changes that can't be done without it, as I understand it. The site isn't going to maintain two interfaces long term, requiring updates to both in order to upgrade other things those interfaces rely on.

As to video, I can't speak to the visual quality, but all premium levels, purchased on or after September 1st have access to the full video library.

It's true that 100B games is way more than anyone would need. I found a bunch of test games the other day between a chess engine and itself. I was amazed at all the free data, but after downloading just a fraction of it I realized that it was way too massive. However, there are ways to determine the rating, and lists to add in the appropriate Elo for that engine, so that's a project. Someone could mine them for novelties, for instance.

But that 100B is the perspective I wasn't really considering. It also just hadn't occurred to me that the big data aspect was so niche. I understand now that it could be a touchy subject (especially if I start saying snarky things, as I did). So my bad. But my intentions are certainly good. And data scientists would be overjoyed -- although I guess you have those covered with the API.

I'm sure there are a lot of interesting things that could be done with the full games DB and I would guess some staff are working on projects using that data.