Chess database that gives option to download all the positions of all the games in EPD or Fen format

Sort:
Trialanderror111

Hello Chess Community,

I was wondering if there is any Chess database / paid unpaid that provides this feature to be able to download an entire collection of games... having for each position within each game the Fen string? (probably the output of all the FEN strings for all those game positions in a .txt file?

I could find that there are two tools out there which help to convert PNG  strings into FEN positions. Great stuff but the tools I could find worked only for 1 game at the time. I am looking to extract this information for multiple games at the same time.

Do you have there any potential solutions for me? I thought I found 1 before but it looks like Windows Defender saw a security risk with that program so it's not really something I would want on my computer.

Many thanks in advance as I was driving myself nuts now for almost an entire week trying to find ways to make it happen.

Trialanderror111

Hi @llama_l Thank you! and I really appreciate your reply but the problem is that I don't know how I could write any kind of script for it as I am not an IT person. Would you be able to walk me through this in a little bit more detail (best would be step by step)? So I could get this setup for myself? I think once I've got the ball rolling it's gonna work but .... well this is like rocket science to me right now...

Trialanderror111

@llama_l no it's not pointless at all. Yes you are right if I would have 100 games it would give me depending on the amount of moves played in those games a lot of FEN strings to work with. 100 games would be not enough though. I am thinking about a couple of 1000 of games to get the results I expect. Yes I am compiling all this data. I have started doing this research already but I it's impossible to get all the FEN strings out of all the PNG files I have by simply going game by game. I need to be able to either extract all those FEN strings from a Database directly or to get them extracted for the entire PNG file at once which might contain about 5000 games. I am not the first using this approach and I will certainly not be the last person to do this. The issue here is that it's really difficult to find a tool that is able to do this for me...

Trialanderror111
llama_l wrote:
Trialanderror111 wrote:

Would you be able to walk me through this in a little bit more detail (best would be step by step)?

If you're starting from zero it might take around one month. Better to find some youtube series that does this.

I googled a bit. Some series assume you know stuff and start way too fast. The link below starts assuming you know nothing, so it actually explains things in detail, and you'd get enough skill to do basic stuff quickly (and actually understand what you're doing).

https://www.youtube.com/watch?v=UjeNA_JtXME&list=PLlRFEj9H3Oj7Bp8-DfGpfAfDBiblRfl5p&index=1&ab_channel=ChuckSeverance

My god, that's a link to learn coding in Python. I see why you sent this to me now but this is not really what I am looking for right now. Thanks anyway happy.png An application where I simply have to click a button and it executes all the data I am looking for is something like I might be interested in.

Trialanderror111
BaphometsChess wrote:
Data

Yes data! I am trying to compile all this data and transform it into information wink. I could really use some help or useful advise here!

Trialanderror111
MoveBlundaMoveBlunda wrote:

On a side note, and I can make another thread but I think we might want to talk more about EPD. I am all for adding things to FEN capabilities, but I see in the description on chessprogrammingorg it states, "Side to move is one lowercase letter for either White ('w') or Black ('b')."

Doesn't this make things confusing when looking at it? Does anyone have an example EPD file of a game, or is it something people will always still use a .pgn for a game but the static position of one point in a game is "painted" differently?

Hi, your question is indeed a little bit different to the one I am asking to be honest. I am not interested in any possible/ potential side lines. I am merely interested in having all the Fen strings for each and every game. For your example I believe you could theoretically use both EPD but also easily PGN files as those are great for this purpose. The side lines will usually be painted in either a different color, font, font size something like this. Potential commentary will also be in a different color again. So nothing you could mix up that easily when running it in a Chess GUI.

My purpose is a little different and PNG files do not help here. Unfortunately. tear

Trialanderror111
llama_l wrote:
Trialanderror111 wrote:

An application where I simply have to click a button and it executes all the data I am looking for is something like I might be interested in.

If you knew python you could do this in a day... and since this isn't the last time in your life you'll be interested in data, it might be worth investing a few months into learning how to do it.

Appreciate it! I do know how to use SQL and VBA programming for writing Excel Macros (haven't used this for a while now but the knowledge is there) Python is a completely different computer language again. I do have a general interest in learning it. For many different reasons actually but I simply don't have the time and mental capacity right now to do this properly.

I do have to ask though.... Do YOU know how to do this and if so would you please be willing to share it with me? I mean if it's really that easy and not that extremely time consuming to create...... that is. I don't want to be taking all of your time...

Trialanderror111
llama_l wrote:
Trialanderror111 wrote:

Do YOU know how to do this and if so would you please be willing to share it with me?

If you gave me a collection of games (I assume thousands of games) I could give you the FEN of every position in a day, but it would be very tedious.

Writing something with a GUI (buttons you can click on) would take me longer and be even more tedious.

There might be some kind of program commission thing online... surely somewhere out there you could pay someone a little money and have them do this for you.

Hi llama_l, I gave your comment some thought and I'd love to take you up on your offer to kindly provide me those FEN strings to those games by providing you with those games if you don't mind?! meh

Trialanderror111
MoveBlundaMoveBlunda wrote:
Trialanderror111 wrote:
llama_l wrote:
Trialanderror111 wrote:

Do YOU know how to do this and if so would you please be willing to share it with me?

If you gave me a collection of games (I assume thousands of games) I could give you the FEN of every position in a day, but it would be very tedious.

Writing something with a GUI (buttons you can click on) would take me longer and be even more tedious.

There might be some kind of program commission thing online... surely somewhere out there you could pay someone a little money and have them do this for you.

Hi llama_l, I gave your comment some thought and I'd love to take you up on your offer to kindly provide me those FEN strings to those games by providing you with those games if you don't mind?!

You didn't read the bottom part?

"surely somewhere out there you could pay someone a little money and have them do this for you"

I am pretty sure that was a subtle/polite way to reject your offer, not accept it.

I read it but why do we not let Lama talk for himself, shall we? wink He can still always decline...

Jaybird127
I also have a need to get FENs for all positions a PGN file. I’m building my repertoire and I think there might exist duplicate positions. Having the FENs would be definitive, as I could just search for duplicates.