Python Chess

I’m delighted to give you this guest post by Niklas Fiekas, the creator of Python Chess. You may think Python Chess is just another chess engine. It isn’t. It’s a library of routines which can manipulate and analyze chess data using Python.  After I learnt about Python Chess I immediately went to Code Academy and took their course on Python. I really see this set of tools becoming a key part in any testing or development frame-work.

Over to Niklas…

Python-chess by example: Automating the Bratko-Kopec Test

Decades ago (1982) Kopec and Bratko published a systematic test for assessing the strength chess-playing programs (pdf). Despite its age it can still be fairly challenging, even for modern chess engines. Of course 24 test positions are not going to be enough for a definite result, especially given that most positions are of a specific type: fairly closed and the solution often involves some kind of sacrifice.

For each of the 24 positions a unique best move is known. The engine is given 120 seconds to solve each position. 1 point is awared for the correct best move.

The positions are given as EPDs. In this article we are going to automate the process of testing an UCI engine, introducing python-chess along the way.

1k1r4/pp1b1R2/3q2pp/4p3/2B5/4Q3/PPP2B2/2K5 b - - bm Qd1+; id "BK.01";
3r1k2/4npp1/1ppr3p/p6P/P2PPPP1/1NR5/5K2/2R5 w - - bm d5; id "BK.02";
2q1rr1k/3bbnnp/p2p1pp1/2pPp3/PpP1P1P1/1P2BNNP/2BQ1PRK/7R b - - bm f5; id "BK.03";
...
r2qnrnk/p2b2b1/1p1p2pp/2pPpp2/1PP1P3/PRNBB3/3QNPPP/5RK1 w - - bm f4; id "BK.24";

 

Why Python (and python-chess)?

When it comes to chess programming, C++ often is the language of choice. Performance is critical. Not nescessarily so, in this case. All intensive calculations will be done by the engine. We just want to convert the given EPD lines, send them to the engine and handle its response. Python is a very good tool for high-level stuff like this (or making a mini chess computer, or making a website to probe tablebases or creating a cross platform chess GUI). We will use python-chess to deal with the chess rules and the involved formats: EPDs, FENs and the UCI protocol. python-chess can also read and write PGNs, read Polyglot opening books and probe Syzygy tablebases.

Chess positions

FEN: 1k1r4/pp1b1R2/3q2pp/4p3/2B5/4Q3/PPP2B2/2K5 b – – 0 1

BK.01 is one of the easier positions: Black to move and mate in 3

In python-chess a position is represented by a chess.Bitboard(). To create a position from a FEN:

position = chess.Bitboard("1k1r4/pp1b1R2/3q2pp/4p3/2B5/4Q3/PPP2B2/2K5 b - - 0 1")

You can then check if a specific move is legal:

move = chess.Move.from_uci("d6d1")
assert move in position.legal_moves

Or to get the shorter SAN notation of the move:

assert position.san(move) == "Qf1+"

Or to make a move and see if it is a check (and much more):

position.push(move)
assert position.is_check()

Here we are just going to parse an EPD line to setup a position and get the related info:

position = chess.Bitboard()
epd_info = position.set_epd('1k1r4/pp1b1R2/3q2pp/4p3/2B5/4Q3/PPP2B2/2K5 b - - bm Qd1+; id "BK.01";')
assert epd_info["bm"] == chess.Move.from_uci("d6d1") # Qd1+
assert epd_info["id"] == "BK.01"

Communicating with an UCI engine

Next we will communicate via UCI with a chess engine like Maverick. First start the engine process:

engine = chess.uci.popen_engine("Maverick64.exe")

The engine expects an initial uci command. Then it will send information about its identity and options.

engine.uci()
assert engine.name == "Maverick 0.51 x64"
assert engine.author == "Steve Maughan"
assert engine.options["Show Search Statistics"].default == True

Now setup the position:

engine.ucinewgame()
engine.position(position)

And calculate for two minutes. The result is the best move (according to the engine) and an optional ponder move (the expected reply the engine wants to ponder on).

bestmove, pondermove = engine.go(movetime=120000)
assert bestmove == Move.from_uci("d6d1")

Or in fact comparing it with the expected best move from the EPD:

if bestmove == epd_info["bm"]:
    score += 1.0
else:
    score += 0.0

 

Putting it all together

#!/usr/bin/python

import chess
import chess.uci
import sys


def test_epd(engine, epd):
    position = chess.Bitboard()
    epd_info = position.set_epd(epd)

    engine.ucinewgame()
    engine.position(position)

    enginemove, pondermove = engine.go(movetime=120000)

    if enginemove == epd_info["bm"]:
        print "%s (expecting %s): +1" % (
            epd_info["id"],
            position.san(epd_info["bm"]))
        return 1.0
    else:
        print "%s (expecting %s): +0 (got %s)" % (
            epd_info["id"],
            position.san(epd_info["bm"]),
            position.san(enginemove))
        return 0.0


if __name__ == "__main__":
    engine = chess.uci.popen_engine(sys.argv[1])
    engine.uci()

    epds = """\
1k1r4/pp1b1R2/3q2pp/4p3/2B5/4Q3/PPP2B2/2K5 b - - bm Qd1+; id "BK.01";
3r1k2/4npp1/1ppr3p/p6P/P2PPPP1/1NR5/5K2/2R5 w - - bm d5; id "BK.02";
2q1rr1k/3bbnnp/p2p1pp1/2pPp3/PpP1P1P1/1P2BNNP/2BQ1PRK/7R b - - bm f5; id "BK.03";
rnbqkb1r/p3pppp/1p6/2ppP3/3N4/2P5/PPP1QPPP/R1B1KB1R w KQkq - bm e6; id "BK.04";
r1b2rk1/2q1b1pp/p2ppn2/1p6/3QP3/1BN1B3/PPP3PP/R4RK1 w - - bm Nd5; id "BK.05";
2r3k1/pppR1pp1/4p3/4P1P1/5P2/1P4K1/P1P5/8 w - - bm g6; id "BK.06";
1nk1r1r1/pp2n1pp/4p3/q2pPp1N/b1pP1P2/B1P2R2/2P1B1PP/R2Q2K1 w - - bm Nf6; id "BK.07";
4b3/p3kp2/6p1/3pP2p/2pP1P2/4K1P1/P3N2P/8 w - - bm f5; id "BK.08";
2kr1bnr/pbpq4/2n1pp2/3p3p/3P1P1B/2N2N1Q/PPP3PP/2KR1B1R w - - bm f5; id "BK.09";
3rr1k1/pp3pp1/1qn2np1/8/3p4/PP1R1P2/2P1NQPP/R1B3K1 b - - bm Ne5; id "BK.10";
2r1nrk1/p2q1ppp/bp1p4/n1pPp3/P1P1P3/2PBB1N1/4QPPP/R4RK1 w - - bm f4; id "BK.11";
r3r1k1/ppqb1ppp/8/4p1NQ/8/2P5/PP3PPP/R3R1K1 b - - bm Bf5; id "BK.12";
r2q1rk1/4bppp/p2p4/2pP4/3pP3/3Q4/PP1B1PPP/R3R1K1 w - - bm b4; id "BK.13";
rnb2r1k/pp2p2p/2pp2p1/q2P1p2/8/1Pb2NP1/PB2PPBP/R2Q1RK1 w - - bm Qd2; id "BK.14";
2r3k1/1p2q1pp/2b1pr2/p1pp4/6Q1/1P1PP1R1/P1PN2PP/5RK1 w - - bm Qxg7+; id "BK.15";
r1bqkb1r/4npp1/p1p4p/1p1pP1B1/8/1B6/PPPN1PPP/R2Q1RK1 w kq - bm Ne4; id "BK.16";
r2q1rk1/1ppnbppp/p2p1nb1/3Pp3/2P1P1P1/2N2N1P/PPB1QP2/R1B2RK1 b - - bm h5; id "BK.17";
r1bq1rk1/pp2ppbp/2np2p1/2n5/P3PP2/N1P2N2/1PB3PP/R1B1QRK1 b - - bm Nb3; id "BK.18";
3rr3/2pq2pk/p2p1pnp/8/2QBPP2/1P6/P5PP/4RRK1 b - - bm Rxe4; id "BK.19";
r4k2/pb2bp1r/1p1qp2p/3pNp2/3P1P2/2N3P1/PPP1Q2P/2KRR3 w - - bm g4; id "BK.20";
3rn2k/ppb2rpp/2ppqp2/5N2/2P1P3/1P5Q/PB3PPP/3RR1K1 w - - bm Nh6; id "BK.21";
2r2rk1/1bqnbpp1/1p1ppn1p/pP6/N1P1P3/P2B1N1P/1B2QPP1/R2R2K1 b - - bm Bxe4; id "BK.22";
r1bqk2r/pp2bppp/2p5/3pP3/P2Q1P2/2N1B3/1PP3PP/R4RK1 b kq - bm f6; id "BK.23";
r2qnrnk/p2b2b1/1p1p2pp/2pPpp2/1PP1P3/PRNBB3/3QNPPP/5RK1 w - - bm f4; id "BK.24";"""

    score = 0.0

    for epd in epds.split("\n"):
        score += test_epd(engine, epd)

    engine.quit()

    print "-------------------------------"
    print "%g / 24" % score

Congratulations Maverick 0.51 x64! On my machine you score 18/24, which is almost on a par with Stockfish 6 64’s 19/24.

python bratko-kopec.py Maverick64.exe
BK.01 (expecting Qd1+): +1
BK.02 (expecting d5): +1
BK.03 (expecting f5): +0 (got Ng5)
BK.04 (expecting e6): +1
BK.05 (expecting Nd5): +1
BK.06 (expecting g6): +1
BK.07 (expecting Nf6): +1
BK.08 (expecting f5): +1
BK.09 (expecting f5): +0 (got Bd3)
BK.10 (expecting Ne5): +1
BK.11 (expecting f4): +1
BK.12 (expecting Bf5): +1
BK.13 (expecting b4): +1
BK.14 (expecting Qd2): +1
BK.15 (expecting Qxg7+): +1
BK.16 (expecting Ne4): +1
BK.17 (expecting h5): +0 (got h6)
BK.18 (expecting Nb3): +0 (got f5)
BK.19 (expecting Rxe4): +1
BK.20 (expecting g4): +1
BK.21 (expecting Nh6): +1
BK.22 (expecting Bxe4): +0 (got Ne5)
BK.23 (expecting f6): +0 (got Bf5)
BK.24 (expecting f4): +1
-------------------------------
18 / 24

 

> python bratko-kopec.py stockfish-6-64.exe
BK.01 (expecting Qd1+): +1
BK.02 (expecting d5): +1
BK.03 (expecting f5): +0 (got Rg8)
BK.04 (expecting e6): +1
BK.05 (expecting Nd5): +1
BK.06 (expecting g6): +1
BK.07 (expecting Nf6): +0 (got Bb4)
BK.08 (expecting f5): +1
BK.09 (expecting f5): +0 (got Re1)
BK.10 (expecting Ne5): +1
BK.11 (expecting f4): +1
BK.12 (expecting Bf5): +1
BK.13 (expecting b4): +1
BK.14 (expecting Qd2): +1
BK.15 (expecting Qxg7+): +1
BK.16 (expecting Ne4): +1
BK.17 (expecting h5): +0 (got Rb8)
BK.18 (expecting Nb3): +1
BK.19 (expecting Rxe4): +1
BK.20 (expecting g4): +1
BK.21 (expecting Nh6): +1
BK.22 (expecting Bxe4): +1
BK.23 (expecting f6): +0 (got Bf5)
BK.24 (expecting f4): +1
-------------------------------
19 / 24

Bonus

The original Brato-Kopec test has one more rule: Sometimes the engine changes its mind. If it had the solution at the 30, 60 or 90 second mark but then discarded it, 1/4, 1/3 or 1/2 points are awarded. In order to do this we need to look at the engines principal variations while it is thinking. This is included in the full code of this example.

You can find out more about Python Chess at https://github.com/niklasf/python-chess.

Free Version of Visual Studio Professional 2013

Last week Microsoft release a Community Edition of Visual Studio 2013.  This is a free version of the Professional edition of Visual Studio 2013. Previously Microsoft’s free edition was Visual Studio Express. This only compiled to 32 bit and didn’t include a Profiler or Profiler-Guided-Optimization.  The new Community Edition includes all of these goodies and can generate 64 bit executables.  This is a big deal for chess programmers. It means you can easily develop and test your engine from within the same high quality development environment (I know we’ve always had GCC but this is more integrated than the mishmash of open source tools).

There are some constraints on who can use the Community Edition. You can only use it if your engine isn’t commercial, or if it brings in less than $1 million per year – I’m sure that covers every chess engine developer!!

You can find out more and download here.

The Holiday season is coming. I’m hoping to have more time to dedicate to chess programming. For the last six months I’ve been swamped with work – but I think there is a light at the end of the tunnel. More updates to come.

Chess Engine Testing Framework

Chess-Engine-TestingSince Maverick is now playing chess I really need a testing framework.  Testing has always been important but I think Fabian Letouzey really “turned some heads” when he started making rapid progress with Fruit back in 2004.  His secret turned out to be making decisions about engine parameters and potential improvements based on cold hard facts i.e. testing.  Since then we’ve seen much more discussion about testing.  Then came Rybka and Houdini who championed, what I’d call, turbo-testing.  This is where the games are super short – something like 60 moves in 20 seconds.  Ten years ago I think this would have been frowned upon as too fast and the results will likely not scale to longer time controls.  As it turns out this fear seems to be incorrect.  Turbo-testing seems to be the best way to quickly test potential improvements.

When developing Monarch my testing really wasn’t systematic.  I’d run games of varying lengths.  Maybe 50 games or at most 100 games.  In hindsight Monarch really wasn’t tested well.  I’m determined not to do that with Maverick.  Thankfully there are also some great tools to help with the testing.  The tool which seems to be regarded as the best is CuteChess-Cli.  Another one is LittleBlitzer.  Both enable you to run multiple games in parallel and with minimal screen updates.  I decided to plump for CuteChess-Cli as my tool of choice.  Here’s how I went about setting up the testing framework:

  1. First you need to download CuteChess-Cli
  2. I decided to create a folder for the testing framework as a sub-folder of my root directory.  This way any paths will be quite short.
  3. I then selected a number of engines which will be Maverick’s sparing partners.  The criteria I set for selecting the engine is as follows:
    • Stable – each engine must be rock solid stable
    • A playing strength of between 1800 ELO and 2200 ELO.  This will provide a range of opponents, all of whom will initially be better players.
  4. The engines I selected are as follows:
    • Monarch 
    • Clueless
    • MadChess
    • Predateur
  5. I created a few “good-old-fashioned” batch files to execute the actual tests.  Here’s an example of one of the test batch files:

@ECHO OFF

SET varYYYY=%DATE:~10,4%
SET varMM=%DATE:~4,2%
SET varDD=%DATE:~7,2%
SET varTodaysDate=%varYYYY%%varMM%%varDD%

SET PGN_DATABASE=Games%varTodaysDate%.pgn

c:CuteChesscutechess-cli -engine name="Maverick 0.10 Beta" cmd=Maverick dir=c:CuteChessMaverickNew1 -engine name="Predateur 2.2.1 x32" cmd=Predateur dir=C:CuteChessPredateur -each proto=uci tc=60/180 book=c:CuteChessobook.bin -pgnout c:CuteChessGames%PGN_DATABASE% -games 500 -concurrency 4 -wait 5

pause 10

As you can see this, runs one match of 500 games between Maverick and Predateur.  Four games are played concurrently on the same machine.  All the games are stored in a PGN which contains today’s date (you may need to adjust the code at the top of the bat file for different date formats – mine is US).   For editing these types of files I’d suggest NotePad++.  When installed you simply right click on a .bat file and you have the option to edit it using NotePad++.  It also color code the formatting – very useful.

Since it may be useful to some of you, here’s my whole testing framework zipped up and ready for download.

50% Improvement in Perft Speed

I decided to see what Mavericks performance is like through the lens of a profiler.  I’m using CodeBlocks as my GCC IDE.  It was remarkably simple to get the profiler working.  The first thing I notices was the large amount of time taken to see if a move resulted in discovered check, and therefore illegal.  The time consumed by this one (small) procedure was almost as much as the move-generation code.  I thought something must be wrong.

After some prodding and poking of the code I realized there was a much better way to accomplish the same task.  In my original code I was doing a looking from the king through the “from_square” and seeing if there was a rook, bishop or queen on the relevant path.  This is what I did in Monarch but it’s really letter-box style thinking.  It’s much faster to calculate all of pins at the start of the move generation process and store them in a bitboard.  Then when you make the move you only need to check if there is a discovered check if the piece is pinned (a simple “AND”) – which is rare.  You still need to perform the check if the piece is pinned, it’s a king move or an en-passant move.  I also “inlined” the “is_in_check_after_move” procedure.

The result of these changes is a boost to the perft speed of about 50%.  Maverick now crunches 76 million nps on my humble 2.2 GHz i7 notebook. 

As an aside, my notebook is two years old.  On my wife’s newer 2.4 GHz i7 Maverick’s speed is 99 million nps. Somehow Maverick’s architecture is more suited to newer machines.  I assume it’s a bigger cache.

Debugging a Chess Move Generator

Perft is a fantastic way to debug your chess engine’s move generator (as well as the make and un-make routines).  Here are some tips which you may find helpful:

  • Find a set of test positions with known perft scores and which cover all the weird and obscure chess moves.  See my previous post Perfect Perft for some good examples.
  • Use an engine such as Sharper to split out the perft of each sub move for a position (for Sharper this is the divide command).  This way you can see which move gives you a problem.
  • Write a “flip_board” routine which (as the name suggests), flips the board.  This means you can test each perft position from white and black’s perspective.  The ensures you’re writing Color Blind Code!  Here’s my perft code which automatically flips the board:

//--Position 1
for(i = 0; i <= 1; i++){ set_fen(position, "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1"); ok &= (perft(position, 6) == 119060324); flip_board(position); }

  • Use "asserts" liberally in conjunction with a "board_integrity" routine.  The "board_integrity" routine is key.  It simply ensures that everything makes sense in the chess board object / record - returning "true" if everything is OK and "false" if it find something wrong.  For example does the "all_pieces' bitboard agree with each individual color's occupancy bitboards?  Does the pawn has which has been created incrementally agree with a hash value created from scratch?  I add asserts at the beginning and end of almost every routine.  Here's my board integrity code:

BOOL integrity(struct t_board *board){
int i;
t_chess_color c;
t_bitboard b;

//-- Link between squares and bitboards
for(i = 0; i < 64; i++){ if (board->square[i] != BLANK)
if ((board->piecelist[board->square[i]] & SQUARE64(i)) == 0)
return FALSE;
}

//-- Color bitboards match up
for(c = WHITE; c <= BLACK; c++){ b = 0; for(i = KNIGHT; i <= KING; i++){ b |= board->pieces[c][i];
}
if (b != board->occupied[c])
return FALSE;
if (board->square[board->king_square[c]] != PIECEINDEX(c, KING))
return FALSE;
if (SQUARE64(board->king_square[c]) != board->pieces[c][KING])
return FALSE;
if (popcount(board->pieces[c][KING]) != 1)
return FALSE;
}

//-- All pieces match up with occupancy
if ((board->occupied[WHITE] | board->occupied[BLACK]) != board->all_pieces)
return FALSE;

//-- Castling rights
for (i = 0; i < 4; i++){ if (board->castling & ((uchar)1 << i)){ if (board->square[castle[i].king_from] != PIECEINDEX((i >> 1), KING))
return FALSE;
if (board->square[castle[i].rook_from] != PIECEINDEX((i >> 1), ROOK))
return FALSE;
}
}

//-- Hashing
if (board->pawn_hash != calc_pawn_hash(board))
return FALSE;
if (board->hash != calc_board_hash(board))
return FALSE;

return TRUE;
}

If you follow these tips and your engine agrees with the perft scores of the suggested positions (in debug mode), then I'm 99% sure you'll have a bug free move generator and make / unmake move routine.  Writing test routines never sounds appealing, but having code which you know is (almost) bug free is a GREAT feeling!!

Free Chess Bitboard Viewer

If your chess engine is bitboard based you will need to be able to visualize what the bitboard looks like as an actual chess board.  I can’t image developing a bitboard engine without this type of utility.  So I created a simple chess bitboard viewer. 

You download it here: Chess Bitboard Viewer

Chess Bitboard Viewer

It runs on Windows 32 / 64 bit systems.  It’s super-easy to use (i.e. if you cannot use it think about finding a different hobby).  I’ve zipped it up since I assume downloading a raw (and rare) exe file will make some virus detector scream like a banshee!  BTW it has been checked for viruses and it’s clean – (but use it at your own risk).

You can click on a square and the decimal number will update – or you change the decimal number and the board will update.  You can also copy and paste from the clipboard. 

I hope it’s useful – feedback and comments welcome!