Game explorer does not show transpositions?!?!!

Sort:
Waitaka

I was using game explorer on an opening, and went 1.e4 Nc6 2.Nf3 e5, and saw that at this point the most played white move is 3.d4 (34,803 games).

BUT when we go 1.e4 e5 2.Nf3 Nc6 we reach exactly the same position, but game explorer shows 3.Bb5 as the most white played move (128,105 games).

So, Game explorer uses the move sequence as an index to search the opening database. This limits the database so much, and also leads to incorrect position analysis.

Is there a way to use the FEN, not the move sequence, as an index to the Game explorer database?

Nytik

Interesting. I think that's a bug. The Game Explorer must include transpositions, as sometimes I click a move which has had, say, 20 people play it, but on the next move there are 150 games with one particular move played! So, it certainly DOES include such a thing. Perhaps this is just a single, isolated case.

EDIT: I think the database IS done by FEN. That's why it's at the bottom of the list.

RyanMK

However, once you play 3. Bb5 after the first sequence, you see there are 97,000+ games with that position. Obviously it does typically search by FEN, but is bugged on that one sequence.

Waitaka

Any words from chess.com staff?

Scarblac

Technically the FENs for both positions are different - the move count since the last pawn move (for the 50 moves rule) is different. Don't know if they use that in the database, would be a bit silly in an opening database :-)

Waitaka
Scarblac wrote:

Technically the FENs for both positions are different - the move count since the last pawn move (for the 50 moves rule) is different. Don't know if they use that in the database, would be a bit silly in an opening database :-)


 Scarbac, you are right:

1.e4 Nc6 2.Nf3 e5

r1bqkbnr/pppp1ppp/2n5/4p3/4P3/5N2/PPPP1PPP/RNBQKB1R w KQkq e6 1 3

1.e4 e5 2.Nf3 Nc6

r1bqkbnr/pppp1ppp/2n5/4p3/4P3/5N2/PPPP1PPP/RNBQKB1R w KQkq - 3 3

Maybe you just found the problem.

Waitaka

Nothing yet. Not fixed, and no word from chess.com staff...

TheGrobe

I raised this issue ages ago in one of the Feature Request and Wish-list threads to little fanfare.  I'd hope it gets fixed at some point because when looking at the Win/Draw/Loss percentages for each candidate move, the total number of games from the resulting position is the ideal data-set -- not just the games that arrived in the resulting position from the current one.

Nytik

I do think Scarblac has touched upon the issue. If the Opening Explorer displays pages based on FEN (which I assume it does) then there is a problem concerning openings with pawn moves. (All of them! Wink) They are likely to merge later down the line, however it is obviously an issue in certain positions.

However, in current conditions, presumably an entire rewrite of how the Explorer works would be required to fix this. So, we either learn to live with it, or wait until chess.com has answered all of its current requests- in 2050.

TheGrobe

It may just be a simple matter if ignoring the pawn count when determining whether two positions equate.  The database is virtually useless past the early middle-game, so I don't really see it ever being relevant, but if it were a concern you could arbitrarily use some indicator like the number of remaining pawns to determine whether you do or do not take it into account.

VLaurenT

Yes it had already been spotted and mentioned a while ago to the chess.com staff, but they haven't solved it yet.

The problem is that Game Explorer considers the positions as different, because in the 1st one, a pawn has just moved 2 squares, thus adding an en passant square to the FEN string (that's the "e6" just before "1 3" in the first string).

If you play 3.Bb5 in each position, then you should arrive at the same node. Smile

TheGrobe

But the ability to capture via en passant actually does make them different positions so if this is the difference they are not actually transpositions.  For all practical purposes, however, the move count in the opening phase is irrrelevant and different counts should not be considered different positions until very late in the game.

Scarblac
TheGrobe wrote:

But the ability to capture via en passant actually does make them different positions so if this is the difference they are not actually transpositions.  For all practical purposes, however, the move count in the opening phase is irrrelevant and different counts should not be considered different positions until very late in the game.


But in this case, there is no white pawn on d5 or f5 that could capture pawn e5 en passant, so it does not matter.

The FEN standard took the option of recording the en passant square regardless of whether a capture there is actually possible.

Fixing it so that the en passant square is ignored only when it is irrelevant is probably not so easy to implement, so that's probably why it's been on the to-do list for a while (as hicetnunc says).

atomichicken

It's more than a bit strange in my opinion that d4 was played so much more in the 1st transposition than the 2nd.. Is the mind-set of White usually different reaching it from that move order or something?!

atomichicken

Also I noticed c3 was played exactly the same amount of times in both, co-incidence?

EDIT: No, it's the same for most of the others aswell..

TheGrobe
Scarblac wrote:
TheGrobe wrote:

But the ability to capture via en passant actually does make them different positions so if this is the difference they are not actually transpositions.  For all practical purposes, however, the move count in the opening phase is irrrelevant and different counts should not be considered different positions until very late in the game.


But in this case, there is no white pawn on d5 or f5 that could capture pawn e5 en passant, so it does not matter.

The FEN standard took the option of recording the en passant square regardless of whether a capture there is actually possible.

Fixing it so that the en passant square is ignored only when it is irrelevant is probably not so easy to implement, so that's probably why it's been on the to-do list for a while (as hicetnunc says).


Gotcha -- fair point, i didn't really look at the specific position and I didn't realize that FEN indicated ep regardless of whether there is a pawn in a position to make the capture.  In this case it also needs to be addressed in fixing this issue.

maartenderie

I know this thread is old but this might intrest some of you;

http://web.archive.org/web/20070822204038/http://www.seanet.com/~brucemo/topics/zobrist.htm

I'm having problems with this line:

1.e4 c5 2.Nf3 e6 3.d4 cxd4 4.Nxd4 a6 5.Bd3 Nf6 6.O-O g6 

and after

7.c4 

It goes from 3 to >200 games. From the above, I understand why this is the case. But it doesn't help me study my openings :( I hope chess.com will implement an improvement of the game explorer.

P.S. If you think ep is important (I do), you can always add it to the zobrist-keys.

helltank

The problem is that there are so many transpositions possible in chess that it would be highly unlikely that someone would take the time to root them out and program them in, even if we limit our search to widely-used openings.

For example, did you know that Alekhine's can transpose into Vienna Game? 1.e4 Nf6 2.Nc3 e5

And did you know that the Scandinavian Defence can transpose into the Blackmar-Diemer Gambit? 1.e4 d5 2.d4 

maartenderie

I expect the code to do this to be quite simple. And it doesn't have to be fast since you only have to calculate once (or every update). Just program it and press execute right before you go to bed. (If anyone cares, I'll write up some pseudocode for you, just yell).

Like helltank said, transpositions happen often and not everybody knows about them. I care to know because it causes me to memorize less and get a better grip of the opening.