Python Chess
I’m delighted to give you this guest post by Niklas Fiekas, the creator of Python Chess. You may think Python Chess is just another chess engine. It isn’t. It’s a library of routines which can manipulate and analyze chess data using Python. After I learnt about Python Chess I immediately went to Code Academy and took their course on Python. I really see this set of tools becoming a key part in any testing or development frame-work.
Over to Niklas…
Python-chess by example: Automating the Bratko-Kopec Test
Decades ago (1982) Kopec and Bratko published a systematic test for assessing the strength chess-playing programs (pdf). Despite its age it can still be fairly challenging, even for modern chess engines. Of course 24 test positions are not going to be enough for a definite result, especially given that most positions are of a specific type: fairly closed and the solution often involves some kind of sacrifice.
For each of the 24 positions a unique best move is known. The engine is given 120 seconds to solve each position. 1 point is awared for the correct best move.
The positions are given as EPDs. In this article we are going to automate the process of testing an UCI engine, introducing python-chess along the way.
1k1r4/pp1b1R2/3q2pp/4p3/2B5/4Q3/PPP2B2/2K5 b - - bm Qd1+; id "BK.01"; 3r1k2/4npp1/1ppr3p/p6P/P2PPPP1/1NR5/5K2/2R5 w - - bm d5; id "BK.02"; 2q1rr1k/3bbnnp/p2p1pp1/2pPp3/PpP1P1P1/1P2BNNP/2BQ1PRK/7R b - - bm f5; id "BK.03"; ... r2qnrnk/p2b2b1/1p1p2pp/2pPpp2/1PP1P3/PRNBB3/3QNPPP/5RK1 w - - bm f4; id "BK.24";
Why Python (and python-chess)?
When it comes to chess programming, C++ often is the language of choice. Performance is critical. Not nescessarily so, in this case. All intensive calculations will be done by the engine. We just want to convert the given EPD lines, send them to the engine and handle its response. Python is a very good tool for high-level stuff like this (or making a mini chess computer, or making a website to probe tablebases or creating a cross platform chess GUI). We will use python-chess to deal with the chess rules and the involved formats: EPDs, FENs and the UCI protocol. python-chess can also read and write PGNs, read Polyglot opening books and probe Syzygy tablebases.
Chess positions
FEN: 1k1r4/pp1b1R2/3q2pp/4p3/2B5/4Q3/PPP2B2/2K5 b – – 0 1
BK.01 is one of the easier positions: Black to move and mate in 3
In python-chess a position is represented by a chess.Bitboard(). To create a position from a FEN:
position = chess.Bitboard("1k1r4/pp1b1R2/3q2pp/4p3/2B5/4Q3/PPP2B2/2K5 b - - 0 1")
You can then check if a specific move is legal:
move = chess.Move.from_uci("d6d1")
assert move in position.legal_moves
Or to get the shorter SAN notation of the move:
assert position.san(move) == "Qf1+"
Or to make a move and see if it is a check (and much more):
position.push(move) assert position.is_check()
Here we are just going to parse an EPD line to setup a position and get the related info:
position = chess.Bitboard()
epd_info = position.set_epd('1k1r4/pp1b1R2/3q2pp/4p3/2B5/4Q3/PPP2B2/2K5 b - - bm Qd1+; id "BK.01";')
assert epd_info["bm"] == chess.Move.from_uci("d6d1") # Qd1+
assert epd_info["id"] == "BK.01"
Communicating with an UCI engine
Next we will communicate via UCI with a chess engine like Maverick. First start the engine process:
engine = chess.uci.popen_engine("Maverick64.exe")
The engine expects an initial uci command. Then it will send information about its identity and options.
engine.uci() assert engine.name == "Maverick 0.51 x64" assert engine.author == "Steve Maughan" assert engine.options["Show Search Statistics"].default == True
Now setup the position:
engine.ucinewgame() engine.position(position)
And calculate for two minutes. The result is the best move (according to the engine) and an optional ponder move (the expected reply the engine wants to ponder on).
bestmove, pondermove = engine.go(movetime=120000)
assert bestmove == Move.from_uci("d6d1")
Or in fact comparing it with the expected best move from the EPD:
if bestmove == epd_info["bm"]:
score += 1.0
else:
score += 0.0
Putting it all together
#!/usr/bin/python
import chess
import chess.uci
import sys
def test_epd(engine, epd):
position = chess.Bitboard()
epd_info = position.set_epd(epd)
engine.ucinewgame()
engine.position(position)
enginemove, pondermove = engine.go(movetime=120000)
if enginemove == epd_info["bm"]:
print "%s (expecting %s): +1" % (
epd_info["id"],
position.san(epd_info["bm"]))
return 1.0
else:
print "%s (expecting %s): +0 (got %s)" % (
epd_info["id"],
position.san(epd_info["bm"]),
position.san(enginemove))
return 0.0
if __name__ == "__main__":
engine = chess.uci.popen_engine(sys.argv[1])
engine.uci()
epds = """\
1k1r4/pp1b1R2/3q2pp/4p3/2B5/4Q3/PPP2B2/2K5 b - - bm Qd1+; id "BK.01";
3r1k2/4npp1/1ppr3p/p6P/P2PPPP1/1NR5/5K2/2R5 w - - bm d5; id "BK.02";
2q1rr1k/3bbnnp/p2p1pp1/2pPp3/PpP1P1P1/1P2BNNP/2BQ1PRK/7R b - - bm f5; id "BK.03";
rnbqkb1r/p3pppp/1p6/2ppP3/3N4/2P5/PPP1QPPP/R1B1KB1R w KQkq - bm e6; id "BK.04";
r1b2rk1/2q1b1pp/p2ppn2/1p6/3QP3/1BN1B3/PPP3PP/R4RK1 w - - bm Nd5; id "BK.05";
2r3k1/pppR1pp1/4p3/4P1P1/5P2/1P4K1/P1P5/8 w - - bm g6; id "BK.06";
1nk1r1r1/pp2n1pp/4p3/q2pPp1N/b1pP1P2/B1P2R2/2P1B1PP/R2Q2K1 w - - bm Nf6; id "BK.07";
4b3/p3kp2/6p1/3pP2p/2pP1P2/4K1P1/P3N2P/8 w - - bm f5; id "BK.08";
2kr1bnr/pbpq4/2n1pp2/3p3p/3P1P1B/2N2N1Q/PPP3PP/2KR1B1R w - - bm f5; id "BK.09";
3rr1k1/pp3pp1/1qn2np1/8/3p4/PP1R1P2/2P1NQPP/R1B3K1 b - - bm Ne5; id "BK.10";
2r1nrk1/p2q1ppp/bp1p4/n1pPp3/P1P1P3/2PBB1N1/4QPPP/R4RK1 w - - bm f4; id "BK.11";
r3r1k1/ppqb1ppp/8/4p1NQ/8/2P5/PP3PPP/R3R1K1 b - - bm Bf5; id "BK.12";
r2q1rk1/4bppp/p2p4/2pP4/3pP3/3Q4/PP1B1PPP/R3R1K1 w - - bm b4; id "BK.13";
rnb2r1k/pp2p2p/2pp2p1/q2P1p2/8/1Pb2NP1/PB2PPBP/R2Q1RK1 w - - bm Qd2; id "BK.14";
2r3k1/1p2q1pp/2b1pr2/p1pp4/6Q1/1P1PP1R1/P1PN2PP/5RK1 w - - bm Qxg7+; id "BK.15";
r1bqkb1r/4npp1/p1p4p/1p1pP1B1/8/1B6/PPPN1PPP/R2Q1RK1 w kq - bm Ne4; id "BK.16";
r2q1rk1/1ppnbppp/p2p1nb1/3Pp3/2P1P1P1/2N2N1P/PPB1QP2/R1B2RK1 b - - bm h5; id "BK.17";
r1bq1rk1/pp2ppbp/2np2p1/2n5/P3PP2/N1P2N2/1PB3PP/R1B1QRK1 b - - bm Nb3; id "BK.18";
3rr3/2pq2pk/p2p1pnp/8/2QBPP2/1P6/P5PP/4RRK1 b - - bm Rxe4; id "BK.19";
r4k2/pb2bp1r/1p1qp2p/3pNp2/3P1P2/2N3P1/PPP1Q2P/2KRR3 w - - bm g4; id "BK.20";
3rn2k/ppb2rpp/2ppqp2/5N2/2P1P3/1P5Q/PB3PPP/3RR1K1 w - - bm Nh6; id "BK.21";
2r2rk1/1bqnbpp1/1p1ppn1p/pP6/N1P1P3/P2B1N1P/1B2QPP1/R2R2K1 b - - bm Bxe4; id "BK.22";
r1bqk2r/pp2bppp/2p5/3pP3/P2Q1P2/2N1B3/1PP3PP/R4RK1 b kq - bm f6; id "BK.23";
r2qnrnk/p2b2b1/1p1p2pp/2pPpp2/1PP1P3/PRNBB3/3QNPPP/5RK1 w - - bm f4; id "BK.24";"""
score = 0.0
for epd in epds.split("\n"):
score += test_epd(engine, epd)
engine.quit()
print "-------------------------------"
print "%g / 24" % score
Congratulations Maverick 0.51 x64! On my machine you score 18/24, which is almost on a par with Stockfish 6 64’s 19/24.
python bratko-kopec.py Maverick64.exe BK.01 (expecting Qd1+): +1 BK.02 (expecting d5): +1 BK.03 (expecting f5): +0 (got Ng5) BK.04 (expecting e6): +1 BK.05 (expecting Nd5): +1 BK.06 (expecting g6): +1 BK.07 (expecting Nf6): +1 BK.08 (expecting f5): +1 BK.09 (expecting f5): +0 (got Bd3) BK.10 (expecting Ne5): +1 BK.11 (expecting f4): +1 BK.12 (expecting Bf5): +1 BK.13 (expecting b4): +1 BK.14 (expecting Qd2): +1 BK.15 (expecting Qxg7+): +1 BK.16 (expecting Ne4): +1 BK.17 (expecting h5): +0 (got h6) BK.18 (expecting Nb3): +0 (got f5) BK.19 (expecting Rxe4): +1 BK.20 (expecting g4): +1 BK.21 (expecting Nh6): +1 BK.22 (expecting Bxe4): +0 (got Ne5) BK.23 (expecting f6): +0 (got Bf5) BK.24 (expecting f4): +1 ------------------------------- 18 / 24
> python bratko-kopec.py stockfish-6-64.exe BK.01 (expecting Qd1+): +1 BK.02 (expecting d5): +1 BK.03 (expecting f5): +0 (got Rg8) BK.04 (expecting e6): +1 BK.05 (expecting Nd5): +1 BK.06 (expecting g6): +1 BK.07 (expecting Nf6): +0 (got Bb4) BK.08 (expecting f5): +1 BK.09 (expecting f5): +0 (got Re1) BK.10 (expecting Ne5): +1 BK.11 (expecting f4): +1 BK.12 (expecting Bf5): +1 BK.13 (expecting b4): +1 BK.14 (expecting Qd2): +1 BK.15 (expecting Qxg7+): +1 BK.16 (expecting Ne4): +1 BK.17 (expecting h5): +0 (got Rb8) BK.18 (expecting Nb3): +1 BK.19 (expecting Rxe4): +1 BK.20 (expecting g4): +1 BK.21 (expecting Nh6): +1 BK.22 (expecting Bxe4): +1 BK.23 (expecting f6): +0 (got Bf5) BK.24 (expecting f4): +1 ------------------------------- 19 / 24
Bonus
The original Brato-Kopec test has one more rule: Sometimes the engine changes its mind. If it had the solution at the 30, 60 or 90 second mark but then discarded it, 1/4, 1/3 or 1/2 points are awarded. In order to do this we need to look at the engines principal variations while it is thinking. This is included in the full code of this example.
You can find out more about Python Chess at https://github.com/niklasf/python-chess.
Since Maverick is now playing chess I really need a testing framework. Testing has always been important but I think Fabian Letouzey really “turned some heads” when he started making rapid progress with Fruit back in 2004. His secret turned out to be making decisions about engine parameters and potential improvements based on cold hard facts i.e. testing. Since then we’ve seen much more discussion about testing. Then came Rybka and Houdini who championed, what I’d call, turbo-testing. This is where the games are super short – something like 60 moves in 20 seconds. Ten years ago I think this would have been frowned upon as too fast and the results will likely not scale to longer time controls. As it turns out this fear seems to be incorrect. Turbo-testing seems to be the best way to quickly test potential improvements.