Chess, Starcraft II, and the Learning Process

As a programmer and a chessplayer, I've followed the exploits of the chess AI AlphaZero with great interest:  I even gave a talk on its algorithm at work.  So when my son told me that the team which made AlphaZero had also created an AI to play Starcraft II, I was curious, though skeptical that I'd be able to follow the games.  He assured me that good commentary would illuminate them, and I watched the first game of the match between AlphaStar and the very strong human player Serral.  (For the curious, it can be found at https://www.youtube.com/watch?v=nbiVbd_CEIA with commentary by Artosis.)

I was intrigued, and went on to watch the other four matches.  AlphaStar's play, if the commentary was to be believed, was quite erratic--it had a weird tic of starting buildings, canceling them, and immediately restarting them, for no humanly comprehensible reason--yet it won the match 4-1.  Its great advantage seemed to be its ability to pay attention to every part of the game map at once.  It was throttled to play no faster than its human opponent (a staggering 400-500 actions per minute) but it didn't mind splitting them among five different locations with six kinds of units involved.  Humans find this difficult, and in all but one game Serral suffered the death of a thousand cuts from constant attacks in every direction.  AlphaStar wasn't a tactical genius like AlphaZero; it simply had inhuman resources of attention.

But by this point I had unexpectedly developed a huge fangirl crush on Serral's play.  So elegant!  So fluid!  I watched a bunch more of his games online, but I didn't really understand what I was seeing, even with commentary.   So I resolved to learn to play SC2, despite knowing that I would surely be terrible at it--it's a real-time game, and as a 58-year-old lightning reflexes are not my forte.

I learned to play chess fifty years ago, and I don't remember the process all that well.  My young students do recapitulate it for me, but that's not quite the same.  So it was fascinating to observe the process of learning a difficult new skill, and it shed light on some topics that also arise in teaching and learning chess.

The Same Game a Thousand Times

On a web site dedicated to SC2 there was a post from someone of about my ability level.  He said, I've played 3000 games or so, but I don't quite get what the top players are doing with their queens and their hatcheries--how does that work?  The chess equivalent would be, I've played 3000 games or so, but I'm baffled how my opponent sometimes moves his king and his rook at the same time--how does that work?

It's possible this post was satire, but I think the pattern it describes, in less extreme form, is quite real.  Especially if you play blitz, it's very easy to just keep rushing on to the next game and never looking back.  But there is a good chance that you are playing the same game over and over, making the same mistakes over and over, and not learning much of anything. 

You might think that sheer repetition would eventually teach you something.  But one of my martial arts instructors said, it's a mistake to think that practice makes perfect.  Only perfect practice makes perfect.  Practice just makes permanent.

When I started to play Starcraft 2 I grabbed an opening variation off the net (there are tons of them, as in chess, and for much the same reasons), committed it to memory, and played it a lot.  I won some, I lost some.  I looked at master games, but they were doing everything better than I was, and copying them was beyond my abilities.  I developed some crude heuristics--things like "if you build roaches you ought to use them right away; they aren't any good later on."  But I got stuck quite quickly.  (I would guess I was briefly in Class E, in chess terms, and then got to D and got stuck there.)  I was playing the same thing over and over, with more or less success depending on my opponent.  It wasn't getting any better.

I think this is particularly common in chess when people learn to play by playing online.  It is so easy just to hit "next game" and never look back.  I feel lucky that when I started playing in tournaments (in the late 1970's) I resolved to keep a notebook of every tournament game.  Painstakingly copying them out into the notebook (and trying to make sense of my error-prone scoresheets) forced me to confront what had gone wrong.  I still do this, though I use a computer app rather than a notebook:  and I go further, posting my annotations on my blog, in large part to make sure I annotate even the games I would much rather forget.  (You, dear readers, are a motivational device!)

The SC2 version of this is watching replays of your games, which the system makes easy to do immediately--if you save it for later, you may have trouble figuring out which game it was.  It's no trouble at all, I've found, to review games you won.  But it's particularly useful, if painful, to review games you lost.  (Having your husband and son hang over your shoulder and say "Why did you do that?  Did you panic?" is optional, though sometimes helpful.)  Currently I am adding to the pain by writing down, minute by minute, how far I was able to advance my economy.  It does make a surprising difference.

Optimizing on Immediate Gains

The technical description of AlphaStar brought up some interesting points.  Algorithms of this sort normally learn by playing many, many games against themselves.  When AlphaStar was left to do this, it would find a strategy that was easy to learn and refine it obsessively, bolstered by early victories.  But the strategies that are easiest to learn aren't necessarily the best ones, and AlphaStar kept getting stuck. Its designers had to give it "helpers", inspired by the use of training partners in professional Starcraft 2, who were designed not to play optimally but to probe AlphaStar for weaknesses.  This "motivated" it to abandon its easy but inferior strategies rather than polishing them forever.

(You can read the technical details at https://deepmind.com/blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii).

Yasser Seirawan talks about this in Winning Chess Openings.  He learned a lot of traps as a beginner, and would gleefully deploy them and win games.  He showed such a game to a mentor, and the mentor asked, "What did you learn from this game?"  He had to admit, since he'd played that exact trap several times, he'd learned nothing.  Furthermore, while quick opening traps are successful against weaker players, they are difficult to scale up to the top levels.  And even when they work, they're denying you practice in the middlegame and endgame. 

My chess strategy throughout my youth was to pray I could win before the endgame arrived, and feebly lose if my prayers weren't answered.  I earned quite a high rating doing this, but obviously I was giving myself a hard ceiling--a top player has to know endgames.  At the time the gain from polishing my sacrificial middlegame play (which was my best skill) seemed more impressive than the slow grind of learning endgames, especially since I was terrible at them.

I finally broke down and started studying endgames seriously about three years ago.  It was a slog.  But I won my first Senior Championship by winning an equal-material rook and pawn endgame against an Expert, which was astonishing to me.  I still have a lot to learn, but there's progress.  And studying your worst topic generally gives quicker gains than studying your best, even though, like AlphaStar, I find it difficult to motivate myself to do it!

It was interesting, and a little embarrassing, to realize that I was recapitulating my poor chess-learning strategy in SC2.  I had an opening which (in my hands anyway) was only good for quick wins against inexperienced or careless opponents.  I was polishing it, but with little success.  I wasn't doing the SC2 equivalent of developing pieces or controlling the center or countering the enemy plan:  I was making a whole lot of roaches and trying to overwhelm the opponent.  My little chess students like to do this too:  they grab a powerful piece, generally the queen, and just try to bash the opponent with it.  There's a hard limit to how far that will take you, in chess or in SC2, no matter how well you do it.

Two steps back, three steps forward

I found a series of teaching videos by ViBE, a professional SC2 player, and tried following his advice.  It was quite frustrating, as he promptly forbade almost all of the tactical tricks I had painfully taught myself.  Even as an adult learner with plenty of experience, I found it very difficult to give up my slightly successful techniques and take the rating hit while I learned new ones.  I played against AI a good deal because I didn't want to constantly lose to other humans.  It was tough going.

But after a couple months stuck in low Silver (feels like chess class D) I was abruptly high Gold (maybe class C?) after two weeks of diligently eschewing tactics and early attacks and just building my economy.  (Let's not talk too much about how many hours of Starcraft that was.)

I had a chance to study with a similarly gifted teacher, IM Nicolai Minev, in 1987 when I was preparing for the US Women's Championship.  I hired him to help me get ready...and we accomplished nothing.  With the pressure of the big tournament ahead, I was not willing to take the two steps back.  I wouldn't let go of the strategies that had gotten me to low Expert (the competition was far less fierce than it is today!) and without some willingness to let go, I couldn't learn what he doubtless had to teach.  Even if I had been more teachable I don't know if we could have made this work--perhaps the time was just too short--but the fact that I was self-taught and stiff-necked sure didn't help.

Here's the first video in ViBE's SC2 teaching series, for the curious.  Warning:  ViBE's language is not safe for work, and some of it is casually homophobic.  But he's a genuinely gifted teacher.  https://www.youtube.com/watch?v=_F2CNMDV6AM.  It was interesting watching him try to teach this wickedly hard set of skills, and this led to some thoughts about the chess teaching process:  but this blog is already a wall of text, so I'll put those in part 2.

A Disclaimer

Blizzard, the company which owns StarCraft 2, has a record of abusive behavior.  I don't buy from them.  I salve my conscience by noting that SC2 is free to play, but I'm still uneasily aware that my participation promotes the game, if only in a small way.  I'd like it on record that I really wish this excellent game belonged to an excellent company that I could wholeheartedly support, but that's just not the case.