Determination of default moves in a given opening

Sort:
Sqod

Here is a small progress report on on an idea I proposed about a year ago on this site (http://www.chess.com/forum/view/general/idea-for-opening-books-desired-next-move). I got a little ridicule for the idea at the time, but as I suspected, it seems the idea does have some merit.

My idea is that in the absence of any moves from your opponent that immediately interact with your opening development, there exist default moves that you want to play and should play when you have the opportunity I mentioned. I believe this idea also extends to the middlegame and possibly also to the endgame. This idea was also implied by a passage in John Emm's book on openings, which I quoted below:

----------

(p. 44)
The 'Ideal' Development

Just for a bit of fun, imagine if White were allowed ten unanswered moves at the
beginning of the game: what would he do? Okay, 1 e4 2 Bc4, 3 Qh5 and 4 Qxf7
mate would be a good answer (!), but say White could only move each piece once
and wasn't allowed to make any direct threats. If you asked this question to some
strong chess players, their answers might different very slightly, but most would put
forward something like this:

1 e4 2 d4 3 Nf3 4 Nc3 5 Bc4 6 O-O 7 Bg5 8 Qe2 9 Rad1 10 Rfe1

Emms, John. 2006. Discovering Chess Openings: Building Opening Skills from Basic Principles. Guildford, CT: The Globe Pequot Press.

----------

By using an online opening database I found a fairly objective way to find such default moves that is fairly obvious, so this is no major discovery. I need to stress that default moves are valid and usable *only* when there is no immediate interaction with the opponent's units, which means that any threats that require immediate attention have obvious importance that override the use of a mindless default move. "Interaction" also includes pawns meeting head-to-head, as in e4 ...e5, since that will often preclude a default pawn advance such as d4.

The method of determining default moves is fairly mechanical: you just look one move ahead for what would be your next move, for several lines that stem from the opponent's most common replies, then do a little statistical summary to see which move occurs the most frequently. For example, the database shows that the ten most popular responses to 1. e4 are listed below, with my judgment as to whether that response is interacting or non-interacting.

1...c5 {Interacting since it affects the intent to play d4.}
1...e5 {Interacting since it affects the intent to play d4.}
1...e6 {Non-interacting.}
1...c6 {Non-interacting.}
1...d6 {Non-interacting.}
1...d5 {Interacting since each pawn threatens the other.}
1...g6 {Non-interacting.}
1...Nf6 {Interacting since the pawn is threatened.}
1...Nc6 {Non-interacting.}
1...b6 {Non-interacting.}

If we retain only the non-interacting moves and list say the top three responses by White (assuming that many exist), we get:

 

 

The pattern is overwhelmingly obvious: By far the most common next move by White when White has enough freedom is 2. d4. This is exactly White's goal and ideal setup, per John Emms' excerpt above. More specifically, if we look at only the most common response, we get the statistics:

2. d4 {Chosen 5/6 times => default move 83% of the time.}
2. Nf3 {Chosen 1/6 time => default move 17% of the time.}

Per my recent repertoire convention, whenever I go to the trouble to determine the default move, I put a note to that effect in with my move tree, which when combined with the opponent's options, looks like this:

 

This pattern can be repeated until there is insufficient historical information in the database. (You have to come up with your own convention for how much is considered "insufficient.") For example, if we choose the first non-interacting move encountered in the list of responses in the database, which in the above case is 1...e6 or the French Defense, and continue that process for a few steps, this is the result: (* means wildcard--any move)

 

 

I've found this method to work far into the middlegame, though the tendency then is for there to be fewer options at each move, therefore there is less data to definitively determine a default move. However, in my experience there usually exist non-interacting move options well into the middlegame.

All told, this technique has these main benefits to me:

(1) It suggests the best move when my opponent plays a non-book move, at least if my opponent plays a non-interacting move. (If it's *not* a non-interacting move, and it's non-book, then it's probably a mistake!)

(2) It often allows me to extend my repertoire one move beyond the end of book, especially if my opponent played a non-interacting move.

(3) It begins to suggest an overall plan. Since opening books badly neglect this consideration, it's a least a start in the right direction of providing more general information about the opening.

(4) It allows me to reduce the size of my repertoire since a default move covers a number of rarely seen non-interacting responses.

(5) It sometimes tells me proper piece placement, especially when a piece requires two moves to place. For example, after White plays Na3, is he aiming for Nc4 or rather Nc2? The default next move sometimes tells me this.

I'm interested in any intelligent feedback on this idea and/or method, whether positive or negative. There are a number of nuances and implications I won't get into, unless I get some feedback.

sibi_90

Hi Sqod, this is a very good idea, this will be very helpful if more stronger players give their DREAM middle game positions (along with further plans) for/against various openings they play in their games. 

I would like to give my dream position while playing against the Sicilian dragon. The strongest attacks against the Sicilian dragon tells us to castle queenside but there is a general rule that " only stronger players should castle queenside whereas beginners should castle king side ". The following position of mine involves Kingside castling.

 

 In the above position i have played many moves for white and few key moves of the sicilian dragon (assuming the remaining moves of black may be non-interactive ones).

Few points i consider from this position are

1. Exchange the f4 pawn (by playing f5 or e5 or both)

2. b3 Bishop and the f1 Rook are aiming at the weak f7 pawn

3. Place my queen on d2 and exchange the dark square bishops

4. Will use the fact of the pinned f7 pawn while making a kingside attack

5. If possible(ie if more non-interacting moves are played) i will place both the rooks on g3 and h3 aiding the kingside attack

 The above position may not be the best one yet i'll improve my dream position as and when possible.

najdorf96

Indeed. I too have a similar albeit a more simplistic style. I call it, Triple-Option offence. On defense, it is known as, Ninja D.

najdorf96

I would say your system has merit. But being as experienced as I am, it wouldn't work well for me. Guess I'm just an old dog. I can see how new players could n should try it out. Seems very viable.

Sqod

Thanks for all the feedback. I've been thinking about this proposed algorithmic style approach of mine over the past several hours, and I did realize that it's possibly a little too simplistic because the specific move that lost the tempo could be important for these reasons:

(1) The opponent's move could have been an error that has an immediate refutation that has a refuting move that is different from the default move. If we regard my approach as positional and heuristic in nature rather than tactical in nature, however, the problem goes away.

(2) After exploiting a tempo-losing move, your piece or pawn may not be where you really want it.

I'm specifically thinking of this recently posted game where White was losing a lot of tempi in the opening:

https://www.chess.com/forum/view/game-analysis/infuriatingly-frustrated---how-do-i-improve

After 5. Qb5+?! c6, Black has gained a tempo, but it could be that Black didn't want his c-pawn at c6, but rather at c5, which could alter his plans. I believe I can write up a very general algorithm that takes any such situation into consideration, but I'll have to think about it a little, first.

(3) More typically there exist multiple default moves that depend partly on the player's philosophy. This is not much of a problem because in an opening book, for example, the underlying assumptions or philosophies could be stated at the beginning, as applying to all moves, which would leave only one default move, or different default moves could be shown but marked with the conforming philosophy. In either case, the problem goes away.

(4)  There often exist multiple default moves with different popularity percentages, due to considerations even outside of the philosophies/assumptions mentioned before, therefore the best response would have to take the specifics into consideration again, which is the same underlying issue mentioned in (3). This isn't much of a problem, because you can put popularity percentages after each default move in a move list, especially in descending order of preference percentage, although if you want an algorithm that produces the single best move as mentioned in (2) and (3), the algorithm will have to be modified somewhat.

(5) The opponent's move may not be an *immediate* threat, but it could be a tactical threat severe enough that 2-move lookahead is required in order to prepare for the threat. This probably isn't much of a problem since:  (a) we already know that tactics take priority over positional issues and such a move clearly falls into the tactical category; (b) in practice, especially in the early opening, threats tend to require only 1-move lookahead, so my proposed method tends to cover most cases encountered in practice.

 

Still, although admittedly this approach isn't full developed yet, I definitely believe it's on the right track. I'll keep people updated if and when I make improvements to this approach. I can see already it's going to tie very nicely into plans and philosophies, which are more general topics of high importance and high interest.

 I'll respond to specific comments next, especially with regard to the Dragon Sicilian position shown.

Sqod
sibi_90 wrote:

 In the above position i have played many moves for white and few key moves of the sicilian dragon (assuming the remaining moves of black may be non-interactive ones).

 

 

Thanks, I'm glad you like the idea.

Your ideal setup idea is slightly different than my default move idea, but not by much. Since you're essentially assuming White had *multiple* default/free moves to get to that position, that raises the practical question as to whether your ideal setup can be derived or objectively justified by the data in online databases.

I believe the database predictions can come close to producing ideal positions by various algorithms, such as by selecting the top n default moves, or by looking farther ahead in the database, but the latter would take a *lot* of work to do by hand: even *one* default move requires lengthy move tree exploration and documentation when done by hand! I'll be thinking about that problem, too. Another complication is that such ideal positions rely on using Emms' criteria mentioned above in the posted quote from his book: that no unit be moved a second time and that no direct threats be made. This is a very artificial constraint for my taste, and has the obvious limitation that it can apply only to the opening, not the middlegame. Also, numerous openings require two or more moves to
"fully" develop a piece to its intended location, such as ...Nbd7-f6, or Na3-c2.

By the way, this development issue is related to another interesting topic that somebody posted on this site several months ago (in 2015): how long does it take to completely develop, for each opening? I believe the realistic answer is that many openings involve attacks before one or both sides are fully developed in a general way. I didn't save the URL for that thread (if somebody finds it, please let me/us know what it is), but I've been thinking about it ever since. It's another question to which players of a given opening should know the answer, along with what the default moves are, what the plans are, how drawish a position is, how many tempi can be safely lost in that position, proper unit placement, typical winning moves, denudation rate, and more. It's sobering to realize that virtually no opening books have even *one* of these general pieces of information mentioned for that opening, which is partly why I believe chess theory is still in a very primitive state compared to what it could be.

Sqod
Fiveofswords wrote:

Theres a bunch of moves you want to do, and your move order is determined by the opponents mvoe order, as well as what mvoes they have which are still legal.

Yes, I've been thinking about all of this in the past day, and now I believe I would call my default move algorithm presented above a "weak heuristic method." It's definitely not designed for tactical positions or even for finding the best move, but I still definitely find it useful enough that I will continue to invest my time in using it to find default moves for my own repertoire.

My need for some such method arose from my high frustration over extreme lack of needed information on openings and positions: there is such an extreme lack of useful information about ideas, plans, and piece placement in any format whatsoever (books, articles, videos, etc.) that I had to come up with *something* to help me out, and this method has definitely been helping me. One of my worst problems is making the transition from opening (book) to middlegame: I follow either book or my own memory of a known sequence of moves when playing against a computer, then when the computer deviates from book, I'm often completely at a loss. The same is true when I play against human opponents, although of course I use only my memory from books in that case. I'm like a computer in that way: the moment I'm out of book, I don't know what I'm supposed to do next. I typically don't know anything about the variation where the book dropped me off, so I don't even know where to place my pieces, which is quite dangerous because that's a common way to lose in the opening. If I knew the plans, that would help of course, but opening books of don't have that material, especially for late in the opening or middlegame. Nor do they tell of common motifs, common winning moves, or anything else that would even give a player a hint as to what is going on or what to do. Along with the default move line of text in use in my repertorie file, I also use a line of text to describe common unit placement, one line for each unit for each player, and an arbitrary number of lines of text to describe common motifs for that variation. I'll post some pictures of my repertoire, later today, where it shows how I keep such information, since I think that would be useful for a lot of people.

I'm still thinking about how to best logically generate ideal postions based on database moves, and how to represent changes in plans as you've been talking about. I'm very aware of that issue in my own play, so I know exactly what you're talking about. I suspect this can be best represented using a directed graph that describes strengths, weaknesses, and associated plans with each move, but that's a whole additional project, and I'm already hurting badly on time lately as a result of getting a new full-time job.

Since we're on the topic, and since you're a Petrov player, this might be a good time to give a specific example of what I mean... Last night I lost to Computer on this site when playing Black in the Nimzovich Attack of Petrov's Defense, namely:

1. e4 e5 2. Nf3 Nf6 {Petrov's Defense.} 3. Nxe5 d6 4. Nf3 Nxe4 5. Nc3 {Nimzovich Attack.} Nxc3 6. dxc3 Be7

I believe I understand reasonably well the motivations/ideas behind all these moves, but Black's last response 6...Be7 leaves me almost clueless. There is no name for that variation, as far as I know, and no explanation: nothing except statistics mentioning the typical outcome (it draws 44% of the time) and the move's popularity, which is #1 in preference ranking, and that it is *over 10 times* as popular as the #2 preference: 6...Nc6. Why why why? I understand that knights generally should be developed before bishops, so why is this an exception? You mentioned that Black needs to play ...Be7 when White plays Re1, but here White hasn't played Re1, so why is this an exception? I also don't see that White has any tactical threats that motivate 6...Be7 over 6...Nc6, so why why why? This is extremely frustrating. You talk about thinking in more general terms than the specific moves, and I fully agree, but where the heck does anybody *get* that knowledge? Even if you say that each player has to learn that knowledge on his own, surely players all over must be making notes to themselves in some form as to what the ideas and plans are, but where does any substantial collection of such notes exist? The best I've been able to find are comments in opening books about plans related to *early* opening moves, maybe out to move 5 if we're lucky, and occasional good tidbits of such knowledge deeper in the positions mentioned in YouTube videos, but where is such needed information *in general*? That's the practical problem I've been facing when trying to develop as a player, and why I've been forced to resort to looking a typical single moves as a weak substitute.


Sqod
Fiveofswords wrote:
But pretty much every conceivable logical development involves be7...so you play that first. You play the most clear moves first.
 
Aha, now that makes sense, and it conforms with other things I've read and heard, like disguising your plans by using transpositions (as in aiming for the QGD as Black), and like Seirawan's comment about not letting White know where you intend to castle in the Pirc, and like the hypermodern strategy about being noncommittal. Thanks.
 
 
sibi_90

Hi Sqod, you said that you want to create an opening repertoire and wanted to create a format for smooth transition from opening to middlegame even if our opponents play the non-book moves, 

I would like to suggest that the theory can be learnt for the book moves and remember the positions after move-5 or move-10 instead of the moves or the move order so that if our opponent plays out of book moves (probably non-interactive ones) then we can find the least possible moves to reach the position we have in our mind and continue to execute the theory we learnt in that opening.

For example i shall post the suggested position for white in french defense-advanced variation.

This is one of the positions white desires to get. (even if black plays out of book moves, white can get this position and continues its plans)

White's plans are exchanging one pair of knights and bring in the queen to g4/h4 for the kingside attack after black castles kingside.

 

 

However I usually go out of these book moves for white and try to reach the following two positions with pawn sacrifice so pretty much black will play my prepared lines.  

 

 From this position the entire middle game will be the control of the centre and maintaining the occupied larger space.

 

 

 

 

 

 In this position any single mistake by black will give an immediate equality to the game. white will be attacking d4 pawn and defending e5 pawn meanwhile black will be attacking the e5 pawn while defending the d4 pawn.

While all the play will be focussed on the centre i'll be looking for an opportunity to sacrifice either of the bishops to open the opponent's king since my knights are ready to jump into the kingside attack along with the queen.

 

In both these positions, if my opponent plays interacting moves then it would take more number of moves for me to reach the desired position, if opponent's moves are non-interactive ones then i would reach these positions quickly and then continue the plan.

Sqod

As promised, here is a cleaned-up and pared-down excerpt from my repertoire, in this case for when I play Black against the Nimzovich Attack of Petrov's Defense. This excerpt shows how I represent default moves, motifs, unit placement, and plans in my repertoire. The plans are very sparse in this excerpt since that is a huge topic I am still developing. Anyway, this type of representation shows where I'm headed, and might be useful to other people making notes to themselves without the aid of apps.

 

MY CURRENT NOTATION CONVENTION

"..." = my unofficial name (usually for an opening)

oob = out of book (move)

pt = my pro tempore response (i.e., for the time being)

 

symm = Plan: To draw by keeping the positions as symmetrical as possible.

att(X @ S) = Attacks unit X at square S.

da X = Drives away piece X.

 

b x% = Black wins x% of the time

bw x% = Black and White each win x% of the time

n x% = nobody wins x% of the time (i.e., it's a draw)

nb x% = nobody and Black each win x% of the time

nw x% = nobody and White each win x% of the time

w x% = White wins x% of the time

 
ThrillerFan

A better way to look at positions, rather than giving White 10 free moves, and to evaluate a plan, is to determine the number of "Free Moves" your opponent gets.  The fewer the free moves to achieve your plan, the more likely your plan will succeed.

Here's how it works.  Take your current position, and assume you are White.  You can move whatever you want, however many times you want, but Black gets to respond to all checks and threats.  Count the number of moves you had to take to achieve that goal, NOT COUNTING moves that Black responds to.  If this number is 1, you can achieve the goal by force if Black has no counter-threat (i.e. He cannot stop you via defensive measures).  The larger that number is above 1, the less likely it is going to succeed.

Take your number and subtract 1 - That's the number of "free moves" Black gets.  If you see that your attack is going to give Black 4 or 5 Free Moves, you probably aren't ready to attack and should be building up your forces, completing your development, etc.

 

Take the following as an Example:

Here, White wins material by force.  He wants to get the Queen to g7 to mate the Black King.  There are many ways to get the Queen to the g-file or long diagonal, but all but one way involve 2 moves to get there followed by the mate on the 3rd move, and giving Black at least one free move before the "threat" occurs.  3 moves, 1 threat, minus 1 is 1 Free move for Black.  However, there is a line where the number of "Free Moves" for Black is zero, and White wins material.

After 1.Qe1!, the Rook cannot move anywhere to guard g3 and e5 at the same time, and the Rook is under attack.  If the Rook goes to b3, covering g3, then 2.Qe5! threatens the Rook on b8 and mate on g7.  If the Rook goes to b5, covering e5, then Qg3 threatens the Rook on b8 and mate on g7.

Note that Black can't even get an exchange with 1...Ra4, thinking he can take the Bishop for a Rook, because after 1.Qe1 Ra4 2.Qg3 (or 2.Qe5), if Black takes the Bishop on a1, which isn't check because of the Knight on g1, then 3.Qxb8+ is mate in one because of the back rank (The knight can block on c8, but then the capture of it is checkmate on move 4).

So because Black never got a "Free Move", the attack was sure to work.  Again, the higher that number is (Number of Moves Needed to Achieve the goal, whether it be mate or win of material MINUS Number of Moves that the Opponent must respond to a direct threat MINUS one), the worse your odds are in the attacking plan succeeding.  If that number is 0, it succeeds by force barring a counter attack, meaning it can't be stopped by defensive measures.

Sqod

sibi's post was great in that it pushed me into extending my idea for more moves, which is a great test to see how well it works. I spent some time on this, and here are my summarized conclusions:

(1) I needed to much more carefully define what an "interacting move" was, which I did.

(2) I found that very few moves existed in the database that were truly non-interacting, therefore this method cannot be extended past 4 moves from the opening position unless I relax the definition I am using.

(3) The first 4 default moves I found were exactly what I predicted, and conformed exactly to Emms' quote, so the method worked well for as deep as it was able to reach.

Here are some details...

The definitions I used were:

"interacting move" =

A move that, for the opponent's next move, does any of the following:

(1) limits the motion (namely by blocking) of any of the opponent's units, or

(2) newly threatens the safety of any of the opponent's units (including threats that would be an unsafe capture for the current side to move), or

(3) adds additional capture coverage on a square to which the opponent's unit might move,

if any of the opponent's units (including any that might move away from the protection of a currently threatened unit) were to move on the next move.

"non-interacting move" = Any move that is not an interacting move.

EXAMPLES

1. e4 e5 {Interacting, because it prevents 2. e5 and prevents a completely safe 2. d4.}

1. e4 d6 {Interacting, because it prevents a safe 2. e5.}

1. e4 a6 {Interacting, because it prevents a safe 2. Bb5.}

1. e4 Nh6 {Interacting, because it prevents a completely safe 2. g4.}

1. e4 Nf6 {Interacting, because it threatens 2...Nxe4.}

1. e4 b6 {Non-interacting, because it does not prevent Ba6, does not prevent any immediate pawn advance, and does not threaten anything.}

The fact that so few non-interacting moves were found in the database is not surprising, since almost by definition a non-interacting move is one that virtually doesn't do anything useful at all, which means those moves tend to be weak. In fact, White won very high percentages of the games, sometimes even 100%, where Black played certain non-interacting moves.

Here are the final statistics, using my highly constrained definition. Only the following opening variations extended past 3 moves:

----------

SUMMARY OF ALL FIRST 2 CHOICES FOR WHITE WHEN ALL OF BLACK'S REPONSES ARE NON-INTERACTING

1. e4 e6 2. d4

1. e4 c6 2. d4

1. e4 g6 2. d4

1. e4 b6 2. d4

1. e4 h6 2. d4

DEFAULT 2ND MOVES FOR WHITE: 2. d4 {5/5 = 100%.}

----------

SUMMARY OF ALL FIRST 3 CHOICES FOR WHITE WHEN ALL OF BLACK'S REPONSES ARE NON-INTERACTING

1. e4 e6 2. d4 b6 3. Nf3

1. e4 e6 2. d4 g6 3. Nf3

1. e4 c6 2. d4 b6 3. c4

1. e4 g6 2. d4 b6 3. Nc3

1. e4 b6 2. d4 g6 3. Nc3

1. e4 h6 2. d4 {No non-interacting responses exist in database.}

DEFAULT 3RD MOVES FOR WHITE: 3. Nf3 {2/5 = 40%.}, 3. Nc3 {2/5 = 40%.}, 3. c4 {1/5 = 20%.}

----------

SUMMARY OF ALL FIRST 4 CHOICES FOR WHITE WHEN ALL OF BLACK'S REPONSES ARE NON-INTERACTING

1. e4 e6 2. d4 g6 3. Nf3 b6 4. Nc3 {No non-interacting responses exist in database.}

 

DEFAULT 4TH MOVES FOR WHITE: 4. Nc3 {1/1 = 100%.}

----------

Using a trick to make it look like Black is not responding at all, here are White's statistically ideal opening moves, in order, in the absence of no response from Black:


P.S.--Note that I showed 3. Nf3 as White's default 3rd move instead of 3. Nc3, even though the latter move tied numerically (both 40%). This preference can be justified in a number of ways, especially by weighted average or by decreasing prevalence of the listed moves, though I didn't bother to explain the details here.


Bulacano

I still don't see why you consider Nh6 interacting and b6 not interacting. Owen's defense is not as bad as the Adams defense since Bb7 will interact with e4 and should thus be considered interacting. 2. g4 is not totally necessary and the knight on h6 cannot go to g4 or f5 on the next move. 

Sqod

FOS & chesscoach,

Remember that this is a *heuristic* approach, which is why it is looking ahead only one move to determine threats and so on: one move alone is trying to approximate roughly everything that is going on in every opening in chess at once: a huge task, to put it mildly. Similarly, I'm using dumb, blind statistics to approximate intelligent behavior: another huge task. My idea is that, although there are many exceptions such as useful fianchettoes and more required lookahead, when averaged out over many moves the exceptions go away and we're left with the general intent of the players as it evolves in time.

That's why ...Nh6 is considered interacting: statistics don't understand what a center is, what center control is, or relative unit values, so the variation 1. e4 Nh6 2. g4 Nxg4 3. Qxg4 looks just as good as any other variation. I have an idea I call "refocusing" to impart more intelligence into the statistics, but it's prohibitively lengthy and my simpler approach is working remarkably well already.
 
I have new ideas and definitions that I'll post soon. My goal is to relax the definition of "interacting" so that I can pull in more moves, which should allow me to extend this method to more moves, and I want to try it out on the opening position and siri's Dragon Sicilian position.
 
Sqod

OK, I put in a lot of work on this in the last few days, where my main goal was to push beyond 4 default moves from the opening position. I wanted to see at least the first 5 default moves. Most of my effort went into developing an unambiguous and fairly thorough vocabulary so that I could describe the effects of each move. To summarize the end result: I failed.

The main success on this week's work was to develop a pretty decent vocabulary that will extend to any type of move in any part of the game. My idea was that once I had the vocabulary in place, especially as to what an "immediately interacting" move meant by listing several criteria to define "immediately interacting," I could then eliminate the least important criteria, one by one, until I was pulling in enough "non-immediately interacting" moves that I could pull in enough data from the database to see what White usually does, on the average.

The main problem was that although I did pull in more moves after the first move, there still wasn't enough applicable data in the database, probably because so few people played so many quasi-weak moves in a row. That's what I was afraid of, and I don't see any good way around that problem, unless there exists a database with a lot of games where weak beginners are playing against strong players.

Therefore I will have to rethink this problem. I haven't yet truly tried my original intent of removing the least important criteria from the list; what I did was use my new vocabulary to subdivide move effects to such a fine degree that I wasn't excluding as many moves for my statistical analysis. In my next session I'll look at trying to do that, and I'll think about some other methods of tackling the problem, such as pulling in games where the default move is used but not as the #1 preferred response.

So much work went into this that I'll just throw out a few items of vocabulary, criteria, and results without explanation since explanation would take too long. In short, I still got 4 default moves but because my new criteria ruled out all openings beyond the 3rd move except for the French Defense, the first 4 default moves across all openings turned out to be what I posted before as the default moves for the French: 1. e4 2. d4 3. Nf3 4. Bd3. By the time I got to the 5th move for White, the database had only 2 applicable games in it, with White's 5th moves as 5. O-O and 5. c3, with no way to determine which might have been preferred, if either.

----------

MY DEFINITIONS

 

()

the capturing force = conceptually, the unseen force emanating from any given unit that allows that unit to forcelight or to capture another unit on any square along its line of force

REFERENCE

https://www.chess.com/forum/view/general/how-to-analyze-position-scientifically-part-1

 

()

to forcelight = analogous to "to spotlight", except instead of casting a focused light upon an object, 

a given unit on a given square casts a capturing force upon another sou.

This terminology forces the colloquial terminology of "to attack a square" to change to "to forcelight a square".

 

()

immediately = within the next full move; considering only the opponent's reply, not any follow-up move

 

()

line of force = one path of emanation of a unit's capturing force. Most units emanate multiple lines of force. 

For example, a rook emanates four lines of force, a queen emanates eight lines of force, and a pawn emanates two lines of force.

 

()

oot = of the time (e.g., 75% oot)

 

()

sac = sacrifice

 

()

sou = square or unit. Squares and units are related in that either one can be forcelit.

 

()

to uttack = to attack a (given) unit

This terminology forces the colloquial terminology of "to attack a unit" to change to "to uttack a unit".

Therefore the phrase "to uttack a square" would not be allowed, and would have no meaning.

 

()

 

wrt = with respect to (This is a standard math abbreviation.)

----------

SYMBOLS AND ACRONYMS

 

SINGLE SYMBOLS

 

@ {at}

 

BOARD OBJECTS (squares and units)

 

S ::= <file name><rank number>

U ::= <P | N | B | R | Q | K}

 

<file name> ::= <a | b | c | d | e | f | g | h>

<rank number> ::= <1 | 2 | 3 | 4 | 5 | 6 | 7 | 8>

 

(IMMEDIATE) OUTCOME CODES

 

<outcome code> ::= <eaim | eeim | ebim | esim>

 

   The "outcome" means what would happen if the uttack continued without an opponent's response that defended the uttacked unit.

   Note: An ebim uttack is either a mistake, a gambit, or a combinatorial sacrifice:

there is no simple way to know which one, based only on the codes used in this system.

   Note: All gambits are ebim uttacks.

 

eaim {ends ahead in material unless the opponent alters the conditions, such as by defensive measures}

ebim {ends behind in material unless the opponent alters the conditions}

eeim {ends equal in material unless the opponent alters the conditions}

esim = eaim or eeim {ends satisfactorily in material unless the opponent alters the conditions}

 

EXAMPLES

 

1. e4 e6 2. d4 Nc6 {ebim-uttacks P@d4.}

1. e4 e6 2. d4 d5 {eeim-uttacks P@e4.}

1. e4 e5 2. f4 {ebim-uttacks P@e5.}

1. d4 e5 {ebim-uttacks P@d4.}

 

SAFETY

 

<safety code> ::= {sac | safe}

 

sac = loses material

safe = does not lose material (includes exchanges that keep the material balance equal)

 

TYPES OF FORCELIGHTING

 

   Syntax:

 

ooi-forcelight S

(or

[retain-][<outcome code>-]-oop-<safety code>-forcelight S [wrt U]

 

   Note: There exist two forms of this expression,

since if no unit can move to the square in question then it is pointless to consider the possible outcomes of a capture there.

   Note: In cases where there exist multiple units that can move to the square in question,

my convention is to write a separate expression for each such possible unit,

since the outcome depends on which unit moves there,

and to write the type of unit after the "wrt" for clarity as to which unit was intended by the expression.

 

<occupation code> ::= <oop | ooi>

 

oop {opponent's (immediate) occupation possible}

ooi {opponent's (immediate) occupation impossible}

 

   "retain-" means this move makes no change to the previous outcome code for this square.

 

TYPES OF UTTACKS

 

   Note: Any uttack on any unit also implies that that unit is forcelit.

 

   Syntax:

 

[[<outcome code>-]<safety code>-]uttack U@S

 

----------

NEW CRITERIA

 

"immediately interacting move" = A move that immediately does any of the following:

(1) captures of the opponent's units, or

(2) loses one's own unit, or

(3) blockades an opponent's unit, or

(4) uttacks an opponent's unit, or

and where the capture would not leave the uttacker behind in material if the opponent's unit captured it, or

(5) oop-forcelights a square (to which an opponent's unit can legally move) 

and where the capture would not leave the uttacker behind in material if the opponent's unit moved there, or

(6) adds additional capture coverage on a square to which the opponent's unit might move.

----------
ALGORITHM (not complete yet)
 
[deleted 2-10-16]
----------
IMMEDIATE EFFECTS OF EACH DEFENSE, THROUGH MOVE #1
 
ii = immediately interacting
nii = non-immediately interacting
 
1. e4
 
1... c5 {ii}
eaim-oop-safe-forcelights b4
eeim-oop-safe-forcelights d4
1... e5 {ii}
blockades P@e5
eeim-oop-safe-forcelights d4
eaim-oop-safe-forcelights f4
1... e6 {nii}
ooi-safe-forcelights d4
ooi-safe-forcelights f4
1... c6 {ii}
eaim-oop-safe-forcelights b5
ooi-safe-forcelights d5
1... d6 {ii}
ooi-safe-forcelights c5
eaim-oop-safe-forcelights e5
1... d5 {ii}
eaim-safe-uttacks P@e4
eeim-oop-safe-forcelights c4
eaim-oop-safe-forcelights c4
1... g6 {ii}
ooi-safe-forcelights f5
eaim-safe-oop-forcelights h5
1... Nf6 {ii}
eaim-safe-uttacks P@e4
ebim-oop-safe-forcelights g4 wrt P
eaim-oop-safe-forcelights g4 wrt Q
1... Nc6 {ii}
ooi-safe-forcelights a5
eaim-oop-safe-forcelights b4
ebim-oop-safe-forcelights d4
eaim-oop-safe-forcelights e5
1... b6 {nii}
ooi-safe-forcelights a5
ooi-safe-forcelights c5
1... a6 {ii}
eaim-oop-safe-forcelights b5
1... g5 {ii}
ooi-safe-forcelights f4
eeim-oop-safe-forcelights h4
1... f5 {ii}
eaim-sac-uttacks P@e4
eeim-oop-sac-forcelights g4 wrt P
eaim-oop-sac-forcelights g4 wrt Q
1... a5 {ii}
eaim-oop-safe-forcelights b4
1... Nh6 {ii}
ooi-safe-forcelights f5
eeim-oop-safe-forcelights g4 wrt P
eaim-oop-safe-forcelights g4 wrt Q
1... f6 {ii}
eaim-oop-safe-forcelights e5
ooi-safe-forcelights g5
1... h6 {nii}
ooi-safe-forcelights g5
1... h5 {ii}
eeim-oop-safe-forcelights g4 wrt P
eaim-oop-safe-forcelights g4 wrt Q
1... Na6 {ii}
asim-oop-safe-forcelights b4 wrt P
eaim-oop-safe-forcelights b4 wrt Q
ooi-forcelights c5
1... b5 {ii}
eeim-oop-sac-forcelights a4
eeim-oop-sac-forcelights c4 wrt P
eaim-oop-sac-forcelights c4 wrt B
----------
 
aman_makhija

Absolutely correct. 

aman_makhija
Fiveofswords wrote:

like i start e4 and the only things that will prevent me from playing 2d4 are e5 c5 d5 nf6 f5 and b5. other moves dont give me any reason to change my mind about d4.

B5? You mean you accept the Orangutan gambit? I play 1.e4 b5 2.d4

aman_makhija
Fiveofswords wrote:
aman_makhija wrote:
Fiveofswords wrote:

like i start e4 and the only things that will prevent me from playing 2d4 are e5 c5 d5 nf6 f5 and b5. other moves dont give me any reason to change my mind about d4.

B5? You mean you accept the Orangutan gambit? I play 1.e4 b5 2.d4

 

oh no i meant to say g4. i wouldnt play d4 cause id rather jsut win the pawn

You don't just win the pawn. Black gets c6 and d5 and often wins.

Sqod

I made many new updates to my algorithm and terminology, and spent about three days going through many hundreds (thousands?) of responses in an online database to see if my new algorithm worked. The good news: It does seem to work exactly as I wanted. The bad news: When done by hand it takes prohibitively long, so this updated algorithm will either have to be done by computer for a fully accurate answer, or will have to involve some shortcuts if done by hand.

I'm calling my old algorithm the Non-Predicting Non-Refocusing Algorithm, and my new algorithm the Predicting Refocusing Algorithm. I didn't get a chance to write up my new algorithm, but it's quite similar. Here's the old algorithm in full (with probably a few minor typos regarding the append operations):

----------
NON-PREDICTING NON-REFOCUSING ALGORITHM

Obtain access to an online database, especially one that includes beginners' games, ideally against advanced players.
Played Lines List = The line played up to the position that is to be analyzed. [2]
Repeat until either the desired number of moves is traversed, or until the database lacks sufficient data:
   (1) All Responses = a list all immediate responses to the last move in Played Lines List that are listed in the database [3]
   (2) for each move in All Responses:
      (2a) if the move's ii status is not immediately clear then list all the effects of this move [5]
      (2b) determine if the move is ii or nii, and write the result next to that move. [4]
   (3) All Responses With Status = All Responses, with each move marked as ii or nii
   (4) Nii List = a list of only the nii moves that were just found in the previous step
   (5) Nii List With Responses = Nii List with a set of responses for each last move in the previous list
   (6) Default Move = the most popular move, overall, for each move in the Nii List, based on the responses for that move in Nii List With Responses [1]
   (7) (new) Played Lines List = Played Lines List + Nii List (the lists are appended)
   (8) (new) Played Lines List = Played Lines List + Default Move appended to the end of each line of the Nii List

TERMINOLOGY

ii = immediately interacting
nii = not immediately interacting
uttack = attack on a unit

FOOTNOTES

[1] Default Move
   There exist different methods to determining the most common response, such as:
by count of each move across all responses and selecting the move with the max count
by count across the top N responses and selecting the move with the max count
by weighted average across all responses and selecting the move with the highest weight
by weighted average across the top N responses and selecting the move with the highest weight

[2] Played Lines List
   This list starts with a single line. Other lines get added later.
This list may start with only a single half-move (e.g., 1. e4).

[3] All Responses
   When performing this algorithm by hand, it is much faster to select only the top N responses,
such as where N = 3, or N = 5, though the resulting answers will not be as accuate.

[4] effects of the move
   Use the listed effects of this move if that list helps.

[5] how to determine status

   Status is derived only from the functions used here: blockades, forcelights, uttacks.
If one effect is so overwhelmingly important that the others are irrelevant to consider,
then there is no need to generate every possible effect.

EXAMPLE

Played Lines List =
1. e4

(1) All Responses =
1... c5
1... e5
1... e6
1... c6
1... d6
1... d5
1... g6
1... Nf6
1... Nc6
1... b6
1... a6
1... g5
1... f5
1... a5
1... Nh6
1... f6
1... h6
1... h5
1... Na6
1... b5

(3) All Responses With Status =
1... c5 {ii}
1... e5 {ii}
1... e6 {nii}
1... c6 {nii}
1... d6 {nii}
1... d5 {ii}
1... g6 {nii}
1... Nf6 {ii}
1... Nc6 {nii}
1... b6 {nii}
1... a6 {nii}
1... g5 {nii}
1... f5 {ii}
1... a5 {nii}
1... Nh6 {nii}
1... f6 {nii}
1... h6 {nii}
1... h5 {nii}
1... Na6 {nii}
1... b5 {nii}

(4) Nii List =
1... e6 {nii}
1... c6 {nii}
1... d6 {nii}
1... g6 {nii}
1... Nc6 {nii}
1... b6 {nii}
1... a6 {nii}
1... g5 {nii}
1... a5 {nii}
1... Nh6 {nii}
1... f6 {nii}
1... h6 {nii}
1... h5 {nii}
1... Na6 {nii}
1... b5 {nii}

(5) Nii List With Responses =

1... e6 {nii}

2. d4 - 1st appearance of this move
2. d3
2. Nf3
2. Qe2
2. Nc3
2. b3
2. c4
2. f4
2. g3
2. e5
2. Bc4
2. Bb5
2. c3
2. Ne2
2. a3
2. Be2
2. f3
2. Qf3
2. Bd3
2. g4
2. h3
2. h4
2. Nh3
2. b4
2. Qh5

1... c6 {nii}

2. d4 - 2nd appearance of this move
2. Nc3
2. c4
2. d3
2. Nf3
2. f4
2. Ne2
2. Bc4
2. b3
2. g3
2. c3
2. e5
2. Qe2
2. f3
2. Qf3
2. a3
2. Be2
2. b4
2. h4
2. Qh5
2. Bd3
2. Bb5
2. Qg4

...

(6) Default Move = 2. d4, since d4 appears the most often at the top of Nii List With Responses

(7) Played Lines List =
1. e4
+
1... e6 {nii}
1... c6 {nii}
1... d6 {nii}
1... g6 {nii}
1... Nc6 {nii}
1... b6 {nii}
1... a6 {nii}
1... g5 {nii}
1... a5 {nii}
1... Nh6 {nii}
1... f6 {nii}
1... h6 {nii}
1... h5 {nii}
1... Na6 {nii}
1... b5 {nii}

(8) Played Lines List =
1. e4
1... e6 {nii} + 2. d4
1... c6 {nii} + 2. d4
1... d6 {nii} + 2. d4
1... g6 {nii} + 2. d4
1... Nc6 {nii} + 2. d4
1... b6 {nii} + 2. d4
1... a6 {nii} + 2. d4
1... g5 {nii} + 2. d4
1... a5 {nii} + 2. d4
1... Nh6 {nii} + 2. d4
1... f6 {nii} + 2. d4
1... h6 {nii} + 2. d4
1... h5 {nii} + 2. d4
1... Na6 {nii} + 2. d4
1... b5 {nii} + 2. d4

(1) All Responses =

1. e4
1... e6 {nii} 2. d4
2... d5
2... c5
2... b6
2... a6
2... d6
2... Nf6
2... g6
2... c6
2... Ne7
2... Nc6
2... Be7
2... Bb4
2... b5
2... f5
2... h6
2... Qh4
2... Qe7
2... a5
2... g5
2... e5
2... Qf6
2... f6
2... Bd6
2... Nh6
2... Bc5

...
----------
I worked out some mildly clever enhancements to produce my new algorithm. The first and naive algorithm, as I mentioned earler, quickly ran out of database entries because I was putting so many constraints on what was allowed to be considered an "immediately interacting" move. (That's a new terminology update, too, motivated by the comments people posted, since my earlier term "interacting move" wasn't clear enough about it looking ahead only until the next full move.) In order to put more smarts into my algorithm without having to remove logical criteria about "ii" status, I made the algorithm look ahead by giving it the upcoming default move. (That's why I call the new algorithm "predicting.") How do I get the upcoming default move if that's what the algorithm is trying to figure out? Instead of producing only one default move at each step, I produce an entire list of default moves at each step, and the second move on that list is the predicted default move. Then, to "focus" the meaning of "interacting" on only the moves played so far + the upcoming move, my new criteria considers only moves that immediately interact with those useful moves, not with worthless moves like ...Nh6 that I was considering before. Since at each iteration of the algorithm the new default moves list is updated, the algorithm always has access to the predicted upcoming default move, so at each iteration the algorithm "refocuses" on just those relatively few moves so as to have more accurate heuristics and to pull in more possibilities than before.

For example, after the 1st default move is determined (here 1. e4), here is a partial list of the next most likely default moves:

By weighted average:

2. d4   82%
2. Nf3  62%
2. Nc3  51%
2. Bc4  50%
2. c4   31%
2. f4   ?
2. g3   ?

Default Move = 2. d4

Even with the help of a spreadsheet, finding and pasting those moves into a spreadsheet was very time-consuming.

After the 2nd default move is determined, here is a partial list of the next most likely default moves:

By count with the shown move as 1st choice:

3. Nf3   46
3. Nc3   26
3. c4    7
3. Bc4   4
3. Bd3   0

At that point I was using count as a shortcut since the weighted average done in a spreadsheet was far too time-consuming.

The upshot? I was able to accurately predict the 5th default move, which was my goal in this period of study: 5. Bc4. To my delight, this conforms exactly to Emms' quote, which again suggests I'm on the right track.

Next study on my agenda: To apply this algorithm to a position several moves into the opening, namely siri's Dragon Sicilian. That is sure to take a few more days, especially since I'm already discovering new situations to consider in my ii/nii criteria due to new types of situations occurring on the board that I didn't have to consider before. I'll try to post my Predicting Refocusing Algorithm next time, too.

sibi_90

Sqod, I like the idea of using spreadsheet to note all the moves and using the number of games played as a filter to determine the default moves.

Inorder to get the database of games played between average players vs strong players we can probably download the pgn files of recent non-thematic turn based tournament games from chess.com but that would take quite a long time.

Meanwhile, I just thought of what the chess engine would suggest if the opponent doesn't make an interacting move and came up with the following,