The playing style of StockfishNNUE

It has finally happened. Out of legitimate respect for their new competition, Stockfish released a neural network running on their own parameters, and the results so far have been quite interesting indeed! At TCEC, the new engine scored well in its qualifying matches against its top competitors -- in a gauntlet match against Stockfish 11, AllieStein and LC0, Stockfish NNUE scored one win against LC0 out of 14 games, and drew every other game out of the total 42. Hardly spectacular to a human perspective, but this has proven that at the very least it is capable of holding its own against its best opponents. Very intriguing! 

I installed the new Fish on my own rig, and have begun testing. It immediately shows it has a far different personality than the normal Fish. A few things stand out:

1) Its analysis, while modestly slower than normal Stockfish (on my rig, it clocks around 13-20 million nodes per second vs SF 11's 25-33 million nodes per second), is still far higher than LC0, current leader of the other neural networks. It's also noteworthy that SFNNUE runs on CPU hardware, which defies the GPU-heavy requirements that the other neural nets have demanded.

2) Its analysis is neurotic. Seriously. While LC0's analysis of a position is very steady, Stockfish NNUE is all over the place, fluctuating, for example, from +0.7 to +3 on a given position. This may make it very hard to use it for coherent opening analysis-- while it certainly throws out good moves, how can you compare such moves when at any given second the eval swings so much?

That's all fine and good you may ask, but how does it play?

Further games to come, but for now, this gem--

Stockfish NNUE vs LC0. 

A very distinctive game in some ways. NNfish exits the opening and immediately says the position is winning for white at a +3 evaluation. LC0 calls it a draw. This continues into the endgame, begging the question -- who is correct? In the end, NNfish wins a Knight but soon discovers the sad truth, Knight plus King isn't enough to force a win. Oops.

I was hoping Stockfish nnue would be the next big thing, but Leela is currently obliterating it in a 3+2 match.

I'm interested in how they managed to combine a neural network approach with the classical brute-force search approach that Stockfish uses.  Does the StockfishNNUE just compare the evaluations of both the the neural net it uses with that of Stockfish or something. How exactly is the NN working together with SF?

I would say this NN by the Stockfish team is more of a work in progress than a refutation of LC0. We have to remember Leela's team has far more of a headstart here, to their credit, and far more testing games completed. Credit where it's due!

I'm interested in how they managed to combine a neural network approach with the classical brute-force search approach that Stockfish uses.  Does the StockfishNNUE just compare the evaluations of both the the neural net it uses with that of Stockfish or something. How exactly is the NN working together with SF?

Good question. I'm no expert in programming here, so I'll be offering my own limited understanding of how it all works. Take it with a grain of salt --

Stockfish, the normal version, has thousands of different variables coded by human hands. Things like, "a pawn is worth x, except when y, z, z^2 is on the board" etc. A human team of volunteers has done a spectacular job over the years coding Stockfish into the monster it is today.

Now, the Stockfish neural net took the same parameters, the same variables, but reset them all to 0 and then played millions of games vs itself, self-learning using those parameters to become this new amalgamation.

PS: If anyone who knows better on this subject sees an error in my understanding there, please do correct me, by all means.

This nueral net, Stockfish NNUE, or my own pet name NNfish, is a hybrid of the two current strongest approaches to chess engines that we know of. Now we get to test that hybrid vs our current top engines to see if and how this advances the competitive field. Fun times ahead!

LC0 has had countless hundreds of millions of test matches against itself for training, to date. One question for the NNfish is whether further test matches can improve its current playing results. If LC0 is any indication, this first iteration of NNfish will hardly be the last.

That makes a lot of sense. Also explains why NNfish wouldn't need need a GPU, since the NN is only involved in the training process unlike leela.

The wild swings in evaluation remind me a lot of the early versions of Stockfish -- Stockfish 2/3 etc, where its mood would swing violently from second to second. It's amusing to watch, but makes it hard to compare positions until the end result. With any other engine, a +3 advantage would mean an almost certain win. Not so with NNfish! 

Currently in the middle of a 26-engine round robin. Stockfish NNUE, with no openings book, currently leads 9.0/9 but has not faced anyone important yet. SFNNUE with an openings book is 4th at 7.5/9 behind SF 11, no openings book and Komodo 11, no openings book. Once this round robin is finished I'll start a second one with just the top ten.

So far, NNfish has created some absolutely beautiful games against people rated 500-1,000 points lower. It seems to have a knack for building up truly vicious kingside attacks.

However, so far it appears to be struggling to find more than draws vs its top competitors. Which to be fair, is understandable given its top competition is LC0 and Stockfish 11. Even the gods themselves have odds to overcome.

So far, also, testing has shown that my own custom openings book has weakened most engines by introducing some randomness, instead of strengthening them. Which makes sense again. Neural nets especially, have had a reputation so far of being exceptionally good at openings they choose, and average/mediocre at openings that force them out of their comfort zone.

Stockfish 11 using custom openings book vs Stockfish NNUE, no openings.

Not all draws are dull. Here, a small inaccuracy by Stockfish 11 somewhere around move 30 allows NNfish to blast open the kingside -- I swear, this thing seriously enjoys kingside attacks. In the end it trades down to a same-color bishop endgame, but after move 30 until the endgame there was never a moment SF 11 was comfortable.

Nobody talks about me anymore... ;-;

One of the above games features you! Share the spotlight! Haha

It's a competition where only one will win... And it will be *me*

Not gonna lie, I was rooting for LC0 in back in TCEC 17. Kudos for Stockfish's team for making serious improvements between SF 10 and 11!

Remember: I'm only 2 years old. I have potential...

SF is 12 years old...

We shall see

C'mon, it's clear I am the one...

Kudos to Sam Copeland for the analysis on this video. This is Stockfish NNUE vs Stoovflees, a relatively new engine that has managed to make decent progress towards reaching the top 5. 

Whatever StockfishNNUE did to learn about king safety, it is truly incredible to watch in action. Its attacking pressure is surreal.

https://www.youtube.com/watch?v=eKTVsFMUSfM

More testing continues to suggest a potential issue with its analysis-- it keeps assuming that it is entering a winning endgame when it has only an extra knight to show for its efforts. This may need to be addressed by the developers, although, fixing weaknesses in a neural net may be more of a challenge than for normal engines.