Syzygy not choosing shortest path for mate ?

Hello all,

I recently installed Syzygy 3-4-5 pieces on various chess apps (DroidFish, Shredder, Scid)

and entered the longest mate win endgame position for Queen vs Rook

(35 moves, FEN : 8/8/8/3k4/Q7/8/K6r/8 w - - 0 1).

While with Nalimov tables the shortest path is always found (35 moves),

with Syzygy path is almost always longer (40, 42 moves, ...).

I checked the files with md5, all are ok.

Even stranger sometimes a same position is repeted twice !

Any idea ?

Thanks !

Syzygy tablebases aren't designed to find the shortest path to mate, just to find mate. Nalimov tablebases have that goal and are a lot larger due to the extra needed data to accomplish that.

Ok thank you for the information

Too bad Stockfish engines only accept Syzygy

as Nalimov tables are really fast and accurate

I heard Gaviota are also like Nalimov too

Do you know if other engines use these ones ?

It's not a matter of accuracy. Tablebases are absolutely accurate but they make different choices on a conflict of interest. Nalimov takes you to the nearest mate on the assumption that speed matters. Syzygy takes you to the nearest winning ZERO-point (capture or pawn move) on the assumption that risk reduction matters. This minimizes the risk of a 50-move draw in combination with moves played before the tablebase was invoked. Note that some chess rules are not fixed. In games there is a 50-move draw rule but not in most compositions.

Ok I understand that Syzygy would prefer an intermediate capture on the way to mate to avoid the draw risk for 50 moves rule.
But for the position I mentionned (Queen vs Rook) why not choose the direct 35 moves mate since it is below 50 moves ?
I also, when testing computer vs computer with same position, got a mate with 31 moves (with even a position repetition at beginning), which means a sub-optimal defense : 

1.Kb3 Rh3+ 2.Ka2 Rh2+ 3.Kb3 Rh3+ 4.Kc2 Rh2+ 5.Kd1 Rh1+ 6.Ke2 Rh2+ 7.Kf3 Rh3+ 8.Kf2 Rc3
9.Ke2 Kc5 10.Qe4 Kb5 11.Qe7 Rc5 12.Kd3 Rd5+ 13.Kc3 Rc5+ 14.Kd4 Rc4+ 15.Kd5 Rc2 16.Qb7+ Ka4 17.Qb6 Rc1 18.Kd4 Rg1 19.Qa6+ Kb3 20.Qd3+ Ka4 21.Kc5 Rc1+ 22.Kd5 Rc7 23.Qa6+ Kb3 24.Qb6+ Kc2 25.Qxc7+ Kd3 26.Qe5 Kc2 27.Qe3 Kb1 28.Qd2 Ka1 29.Kc4 Kb1 30.Kc3 Ka1 31.Qb2#

Too bad there is not tablebases combining the advantages of Nalimov (speed, shortest mate path) and 50 move draw risk reduction of Syzygy.
This shouldn't exceed much Nalimov's tables size (8 GB) which isn't a concern any more
for modern computers, and even smartphones.

I cannot precisely answer your question as I don't know how your "app" works (if that is what you use). I only use the full web version https://syzygy-tables.info/ . The web version allows you to pick your own path as it lists all legal moves and checkmate- and/or zero-durations. Your app might reduce your options by automatic generation of "syzygy moves" or by giving you limited information. You should not confuse the core of the syzygy tablebase with the behaviour of certain computer interfaces mining the tablebase in one way or another.

The reason to seek the fastest DTZ (distance to zero-position) rather than DTM (distance to mate position) is that the position you enter in syzygy may not be the 1st one in the 50-M series. You might have been fooling around with the same material for 20 moves before you ask syzygy what to do. Whether or not that is a realistic scenario for your app I don't know for the reason given in my opening line.

If you believe there is a tablebase that combines the advantages of DTM and DTZ then you don't understand the issue. The choice for the one or the other depends on the needs of the user at particular times in particular situations. In the same way that travelling from A to B is sometimes better by train and sometimes better by car. The advantages of both are not realizable in one single vehicle that is best for all occasions.

Summarized: Tablebases are not a simple subject. There are complications which imply that the user may have to decide where the tablebase can't. You can't leave it all to the machine.

Hi.

As you pointed out, Syzygy tries to minimize the zeroing move (capture or pawn advance).

In KQ vs KR, it will prefer to capture the rook in 10 moves over delivering mate in 11 moves (I am making these number up, but I am sure you get the idea).

Also, when defending it will prefer to get mated in 11, rather than losing the rook in 10 moves.

About the repetitions, I do not know.

Greetings.

That's a good example! The tablebases have no knowledge of the past since they do not manage complete games. syzygy's strategy is superior in cases where you played 40 moves around with Q and R before invoking syzygy. Had it gone for the fast mate, your opponent had drawn by claiming 50 moves without capture or pawn move and your next move mate would never happen!

Also note there are endgames with multiple collisions with the 50M line. You might have to capture a pawn to forestall the first collision and then again negotiate another one before a checkmate comes in sight. Syzygy will help you through these phases if there are timely zero-points but Nalimov will betray you! However, syzygy's help is limited. Though it will guide you along zero-points it has no actual knowledge of the 50M rule. So it will not advise you to delay a capture such that the phase thereafter can be completed within 50 moves! 

Ok, thank you for the explanations and also for the syzygy site, that I didn't know.
I entered my QR position a few times, with different browsers and always got the same result, 
which is fine and what I expect from endgame tablebases.
But what is strange is that I got DTZ=59 and DTM=69.
Why proposing a path that zeroes in 59 moves when a mate is available in 35 ?
If it was something like, say, dtz 32, dtm 42, I would say ok, it takes more time
to mate because this path allows him to zero in 32 moves, which is less than 35.
But this is not the case, he's even falling in 50 moves rules since he's chosing
zeroing in 59 !
I think we all agree that : 
-White strategy should be to capture the rook as soon as possible to avoid
falling in the 50 move rule, since, after that, it only takes few moves more to mate with the queen
-In other hand, black strategy should be to delay this capture as late as possible, 
trying to reach 50 moves rule
A few words about my configuration : I have a computer with windows 10 running Shredder, 
and another one with Linux running Scid vs. PC, in another place.
I have in both the same engine, Stockfish 15, and the same tablebases, 
Nalimov for the GUI, and Syzygy for the engines (as they only accept the latter).
Strangely, Shredder uses Nalimov tables for moves whereas Scid only uses it 
for information, in Tablebase window, and Syzygy is used for the moves
It would be great if Scid could behave like my Shredder configuration but it seems
it is not possible.
I like to practice endgame positions but I don't want to bother much about 50 move rule, 
best would be an option (like a check box) which allows to toggle between goals
(shortest mate/zero path).
But even If I had no choice and I had to deal only with 50 move goal , ok, I could accept it.
What's annoying me the most is that game is so slower with syzygy (3, 4, 5, 10 secondes for moves)
whereas immediate reply with Nalimov.
At the beginning of QR position, for exemple, there is lot of variation calculating, 
without dtm information, and only evaluation (+152 most of time).
It is only after some moves that a dtm is proposed.
But what I expect from 3-4-5 tablebases is to give a dtz/dtm, and to play according to it, 
as soon as we get only 5 pieces left in the board.
I don't know why it's not the case, and even worse, path is always changing (in mover numbers), 
between 2 trials (sometimes resolving the position in 32 moves, other 42, 52 etc...)
Shouldn't it be more deterministic ?
Is it the same for you ?
Do I have a way to improve my scid configuraton ?
I see there are some parameters avilable like SyzygyProbeDepth=1, SyzygyProbeLimit=7, etc ..., 
but they don't seem wrong to me ...

Hi.

It is not failing the 50-move rule.

59 here means 59 plies, 59 half-moves. Or 29 and a half moves (30 moves for one side, 29 moves for the other side). Less than 50 moves.

Greetings.

Just this one; too tired with this stuff to answer everything. Syzygy does not know the 50-move rule and therefore can't decide that 69 is satisfactory. It hops to nearby zero-points as a matter of principle such that you run minimum risk with any active abort rule. If FIDE changes 50 to 43 in 25 years, then syzygy will still work in the same way! Also that will help you to overcome the blind section of your game preceding the syzygy analysis. Suppose you already played 16 full non-capture/non-pawn moves before handing the position to syzygy then going for 69 DTM half-moves path will kill your win while 59 DTZ will save you! 

Note: also be careful not to confuse full moves and half moves (plies) as peepchuy wrote as well. Looks like your 35 are full moves while DTMs and DTZs are in half-moves!!

" So it will not advise you to delay a capture such that the phase thereafter can be completed within 50 moves! "

Are you sure that's correct? I thought it did. Can you give an example?

No tablebase will help you out under rules that include the 50 move if you footle around for 40 moves first, because no existing or planned tablebase takes account of the triple repetition rule which always comes in tow.

E.g., a mate in 16 is possible after move 34 here (shown). Both Nalimov and Syzygy put Ka2 at the top of the list which draws in 1.

SF15 manages to avoid that but, surprisingly, screws up with 20 minutes on the clocks! In KRvK!!!

@Zagazik

Quickest mate is not an objective for Syzygy, it will happily take over a hundred moves in a mate in two position, but it gets there if it says it will so long as the ply count for the first position is less than four. There is also no such thing as a distance to mate in Syzygy. How long it takes depends on which top moves you take and there can be large variations. 

But if you get repeats of positions (regarded as the same under FIDE art. 9.2.2) while Syzygy moves are made there is something rotten in the state of Denmark. It may repeat positions that occurred before you started using it, as in the above example. 

Hi MARattigan,

Glad you joined us since you are (possibly the one person here) better on this tablebase stuff than me! I am not suggesting that syzygy will predictably save you when you have been fiddling around in the game before the tablebase analysis. It gives you however a better chance to avoid the 50M disaster than the exclusive DTM approach. Especially, it will save you if you were not so stupid as to repeat any position before invoking the tablebase. Going for the DTZ is a philosophy for risk reduction and getting the best expected payout in the end.

I am not certain about the yes/no of delaying captures or pawn moves to avoid frustrated wins but I concluded that after analyzing quite a number of 2N vs 1/2/3 pawns endings some time ago. As usually, I keep no records and can't reproduce examples from them. The answer may depend on the tools one has available for mining the tablebase and the interpretation of the scores. For instance, the syzygy web interface sorts the results by DTZ which may be taken as an advisory order by unsuspecting users.

The scope of my tablebase comments was to assess the various factors affecting the choice of tablebases and the consequencies in the process. It was an answer to a complaint about syzygy to show that tablebase decisions are not as simple as they look on first sight!

@Arisktotle

Hi.

Would certainly be interested if you can find an example.

@Zagazik

Clarification of, "but it gets there if it says it will so long as the ply count for the first position is less than four". 

If you pass the position to the syzygy-tables.info site it can literally say "White wins" when he doesn't. You have to add the ply count you've got to the <100 DTZ and see if it's still <100. (But read the docs - you may also need to adjust the value given by one when the DTZ shown after the top move doesn't drop by one from the DTZ shown for the position.) What I mean by "Syzygy says it will" is really you deduce it will after you've done the arithmetic.

The ply count <4 just ensures you haven't repeated any positions that are regarded as the same under FIDE art. 9.2.2.   

Hello all,

ok, didn't know for half moves, and, yes the 35 moves are full moves,

so I understand better the engine choice of dtz 59.

I tried to enter another famous endgame, KBN-K, which is, as everyone knows,

a (not-so-easy) white win, without pawn nor capture.

FEN : 4k3/8/8/8/8/8/8/3NKB2 w - - 0 1

So, without surprise, Syzygy gives same DTM/DTZ (57).

And trying with Shredder/Nalimov, I got mating with 29th move,

which is correct : (28X2)+1=57

But trying with my SCID/Syzygy configuration, I never got mate with 29 moves,

the most accurate being 33 moves, which also, were very slow (about 10 seconds/move).

On Droidfish/Syzygy, it was quite different : results ranging from 32 moves to 50 moves,

but with very fast moves.

So I don't understand much the differences between :

-My Syzygy configuration and Syzygy site

-My SCID/Syzygy results and my droidfish/Syzygy results,

especially given the fact that my computer is much more powerful than my smartphone.

Do Syzygy files come with different versions ?

Maybe a tuning/configuration issue ?

Any tips anyone ?

@MARattigan :

No, repeat positions occur entirely while computer vs computer game, using Syzygy,

so indeed, and as you say, something is rotten in kingdom of Denmark !

@Zagazik

Obviously something wrong, but I think "state of Denmark" was correct.

StockFish - at least as implemented on chess.com - has the habit of favoring repeat moves to see if you are willing to make a mistake that allows a faster(?) or safer(?) win. As SF is in control of the game we don't know how and when it interfaces to syzygy. It might have retained some of its human "I know better" character

Thank you pfren, so you mean it is possible to have a local syzygy dtz/dtm evaluation

as accurate as on server side ?

And yes i realize DTM69=(34X2)+1=35 full moves, , so not even a difference between Syzygy/Nalimov, so no dillemma here for the QR position, between shortest mate and closest zero point.

@Zagazik

I believe the DTM figures from that server are simply taken from a DTM tablebase, they're not in fact Syzygy. The 7 man DTM figures disappeared from the reports when some sac of smegma infected the Lomonosov site with ransomware and if you try a 6 or 7 man position with a lone king they're also absent because I don't think Nalimov et al bothered to generate them.

DTM50 tablebases are under construction but don't hold your breath. The figures from those won't replicate the DTM figures though and still won't take the triple repetition rule into account.