Leela Zero (Stockfish without human knowledge), like Alpha 0

Winning percentage is  based on the evaluation of leela. For example. +3.00  evaluation may be evaluated as winning probability of 70% while  +10.00 may be close to 99%.

But she still thinks she can win even when she only has a king left

 Well played. It is nice to see a computer making a reckless sacrifice.

Leela Zero is 25 days old now,  she gains about 50 elo everyday and now 1500 elo(CCRL rating, self elo 4000, self elo of random play =0 elo). If this trend continues, we are expect up to 2750 elo in 50 days.

It plays quite decently, but it seems weirdly blind to even simple tactics. For example, it seems to always ignores the queen and bishop mate on the b1-h7 diagonal. In slow mode:

Even if the current progress was fast, the learning curve will flatten out. After switching to larger networks the computation will slow down too.

Do you know which final number of blocks and filters is planned? I don't know what was used in AlphaZero for chess and found no information in the pager.

Baby Leela fighting against mighty Stockfish in my Arena tournament.  SF smashed her within 40 moves. Oops, poor Leela.

Leela is currently running 6 blocks 64 Filters and 800 max playouts. There is plan to make larger networks, but not decided yet. 800 playouts  is exactly the same as A0, but A0 has larger blocks and filters, i think 20 blocks and 256 filters in training(not sure , there are papers on internet). In showmatch vs SF they used 40 blocks. A0 has about 80 times hardware advantage than SF. In term of elo there is 50 elo on doubling of hardware. If A0 and SF were played in equal conditions, SF is estimated to be 200 elo stronger.

Stockfish would not even run on a TPU, which is not a general purpose CPU, so the comparison is invalid. Meanwhile, according to you, someone could get a score of about 91% against Stockfish on a 32-core machine just by upgrading the hardware.

@drmrboss:
Thank you.

I know that AGZ uses 20x256 and 40x256, but I wasn’t sure if the same is true for chess AZ. In Go there is a big difference in strength between networks with 6, 10 or 20 blocks, and 20 feels like a magic number (the board size is 19x19). So maybe 10 blocks are sufficient for chess – or maybe not, who knows.

The strongest network in LZ (the Go Leela, not Leela Chess Zero) has 10x128 and is quiet strong but still has some serious issues with ladders and eyes in large groups. While the experimental 20x256 is stronger, it is also much slower than the 10x128 network (about 6 times on my GTX 1080: 430nps vs 2400nps). It seems a limit on average PC hardware is reached, the larger network must be much stronger so that the slowdown can be compensated. Its unlikely to get a 40x256 network this or next year I think.

To train the 20x256 will be hard. My GPU can play about 20 self play games per hour with 10x128 network, so there are only 3 or 4 games per hour left with the larger network.

Even if games in chess are shorter and the self play is faster, a 20 block network will not be easy to manage. In Go the 10x128 networks still has some blind spots, maybe the same is true in chess and at least 20 blocks will be needed. We will see. At least I hope we will see. :)

@elroch We compared the power of AO with power of NVIDIA 1080 cards and we will need 80 cards to run A0. If A0 run on NVIDIA 1080 card on common PC , meanwhile SF run on i7 8 cores, the comparision comes with +200 elo in favour of stockfish. If Leela project get exactly the same achvement of A0 after running 40 million training games, we will get Leela running on NVIDIA vs SF running on i7. The goal of Leela project is much harder and tougher than A0, Leela must be 300 elo stronger than A0 on common GPU vs CPU.Only then people will run Leela on their desktop/laptop instead of stockfish. Btw, Leela can run on both CPU or GPU now (see the link) and of course SF can run on GPU if we modify the code. The only reason we choose Leela on GPU is Neural Network run on matrix calculation and more efficient with GPU meanwhile SF run on alpha , beta search which is more suitable on CPU.

I cannot check the numbers but it makes sense and 300 ELO sounds like a reasonable magnitude. (I would estimate even a greater number.)

Currently it is unknown if it is better to start a new training run with a large network or if it is sufficient to start with a smaller one and to switch to larger networks if a network makes no progress any more. Its expensive to test, after 5 months the LZ project has still its first run. Maybe LCZ can prove it first which approach is better (at least it would be proved for chess than).

By the way: AZ was weaker than SF on Google's hardware too if the thinking time was very short (<1sec), but AZ scales better with respect to thinking time. So maybe LCZ scales better on decent PC hardware too and we will see some result like: LCZ is better than SF for thinking times greater than 1min.

Leela GPU vs Leela CPU which one is more powerful?

I tested Arena tournment run on my i5 7400 3GZ Quad core CPU with Nvidia 1060x 3GB GPU

Result= GPU version get about 3 times nps. ( 3 times more powerful) 

Playing several matches against known CCRL rated engines. Of course it is not the real CCRL rating but it is the best way to get approx rating. Leela starting to beat known 1500,1600 rated engines. Tests are done by running those engines on the same machine ,1500 rated engines running on quad core CPU vs Leela running on GPU.

Excellent work!

Leela got another 50+elo gain today. 2000 elo barrier hit already.

Leela programmers are very confident that Leela will surpass stockfish in one day.

Is the error column the result of the opening test?

I thought it was the worst its played in awhile. Same old Berlin 1500 rated opening. Missed a mate in 2. Said it had a 45% chance of winning, and post-analysis eval showed me up 5.5, thanks largely in part to sacrificing a bishop on my h3 pawn.Went from 42% chance(when I was up 8.5 and then easy M2) to 1% in one move.

If I play Bc4 instead of Bb5 Ruy , it goes for same ole Philidor defense and its '53%' chance of winning as black. 2000? no way....Im thinking more like 1500. Actually I dont know any 1500 who plays Philidor and thinks black is way ahead.

Depends on hardware and time. This game was played with 45 sec thinking every move. 0 inaccuracies

https://lichess.org/OI7UAxQY.