Databases

So, for Christmas some overseas relatives bought me an activation key for ChessBase 2009 Premium. :) I've never worked with databases before and I reckon it's time to do so, in order to look at more master games, annotating them myself and improving my analysis / opening repertoire.

But I have some questions about the software:

1) I've got access to the online database which I believe has about 3.8 million games. Is anybody able to confirm how many games this has? Also, is it possible for me to download all of or a significant portion of the games in the online database?

2) When searching for games in the online database (e.g. Searching for games played by Kramnik) a maximum of 1000 games comes up even though the system says there are 2.7 thousand. How can I access the other 1.7 thousand games?

3) I downloaded a free database called "Million Database" which has 1.8 million games (WOW!) but they are most recently from 1998. I would really like a database with recent games so that I can see modern opening variations, whether for use in my games or just for interest's sake. Is there a free database I can download which has games from the last 1-10 years?

4) I see TWIC is supported by ChessBase and I understand the games of the week come out every Tuesday. I can easily download these into chessbase and import them into my personal database which I'll update on Tuesdays. However, there's obviously a lot more issues of TWIC than the three that ChessBase downloads when I click the button. How can I download the entire history of TWIC? (I think this would be a valuable resource for GM games.)

5) I would also like some annotated games that I can go through. I have books with annotated games (Zurich '53 in particular) but I like to analyse those thoroughly on a board. Nipping quickly through some GM games on ChessBase seems like a great way to get some ideas into my games, so are there any sources for annotated games?

Thanks your your patient help and expertise, in advance.

-Hypocrism

http://chess.co.uk/twic/twic

TWIC archive going back to 2002

Chessbase Mega Database 2011 has close to 5 million games, so I would guess the online database has around the same number.  As you note, I think you can download 1000 games at a time.  To access different batches of games, you can vary your search criteria, e.g. by player name or opening.  Icofybase is free, and the games run up to 2006.  TWICs from 1998 to 2002 can be downloaded here.  The commercial products Chessbase Magazine, Mega Database and Sahovski Informant include annotated games.

you will be able to tell if corruption happens, IF the program stalls while doing an integrity check on a large database [ whenever you look for doubles ] and you have to force a shutdown of the program while its running - - - thats the only time when the db gets corrupted i think. but i've rarely had this problem, and only with large dbs - - - greater than half a million games at least, and even dbs of about 5 million, the integrity checks go smoothly though they take a lot longer with really large dbs.

re: final question , it depends on the first few moves - i suppose you did a reference search

you can sort games by move column, etc. - i generally delete all games less than 10 moves, sort my dbs by eco, maintain 1 large database of quality games for reference, others for specific repertoire lines and otb games, and also a database for annotated games

Hi there - thanks for your response. I've done a lot of poking around today with TWIC, icofy bases, and done a lot of copy-pasta work into my reference database. How can I check if it's corrupted - if something is wrong with it I'd like to fix that now and not risk losing games. I can access all games I've tried to so far and the opening reference function seems to work well.

Is it normal for the opening reference to search through and find only 50,001 games in the first few moves? Is that a maximum for efficiency in the first few moves?

another place for games is iccf, some high quality games games there [correspondence advanced chess [engine+ human play].

you can find some fairly good annotated games even in the chess.com articles and blogs.

have fun with your databasing!!

edit: somehow i made one of my posts disappear,  probably editing my first one , interesting, now a response to another post, cool . :-)

The Search Booster for your reference database probably gets corrupted more often than just about anything else.  If you are using the Reference pane, and it returns a result of 'no games' for a position where you know there must be games in the database, right-click on your reference database, choose Tools - Delete Search Booster, and then when it's done right-click again, and choose Tools - Create Search Booster.

If a particular database stalls when trying to open up one of the keys, it may be corrupted.  In this post, Gonnosuke recommends deleting some of the non-essential files to see if that helps.  If you really get into serious trouble, you can create an empty PGN database (File - New - Database then choose PGN from the Save as Type dropdown menu at bottom of the dialogue box), drag the games into it, search through for text files or games with empty headers, and delete those games or files, and then convert the new database back to cbh format.

You should probably backup your reference database before adding games to it (Right-click - Tools - Backup).  Also when adding games from TWIC, it probably wouldn't hurt to delete the text files at the beginning of each TWIC archive before attempting to add the games to another database.