In a recent Computerphile video, Professor Nick Hawes of the Oxford Robotics Institute breaks down Monte Carlo Tree Search, or MCTS, a groundbreaking approach to automating sequential decisions. As Professor Hawes explains, “The fundamental idea of MCTS is to use random simulations to estimate the value of each move.” The algorithm unfolds in four steps: selection, expansion, simulation, and backpropagation.

During selection, MCTS navigates the existing search tree, choosing moves that balance known rewards and exploration—guided by the famous UCT, the Upper Confidence Bound for Trees. In the words of Professor Hawes, “we pick the move that maximizes the UCT score.” When an unexplored move is reached, expansion adds a new node to the tree.

Next, simulation plays a complete game at random, providing an unbiased sample of outcomes. Finally, backpropagation updates every node along the path with the simulation result, refining the tree’s estimates. By repeating these steps thousands or millions of times, MCTS gradually converges on optimal strategies. The beauty of Monte Carlo Tree Search lies in its simplicity and adaptability—it needs no handcrafted evaluation functions, only time and computation to sharpen its decisions.
Link to Article