Simulation / Optimization / Computer vision / Game systems / Simulated annealing · 7 min read
Three cats. Fifteen turns. One useful model.
Building a screenshot-to-plan tool taught me that a search algorithm is only as useful as the world it is searching. Try the planner, then follow the decisions behind it.
Muhammad Musa
A small board with coupled decisions
I wanted a practical loop: give the tool a starting screenshot and the money available, get back exact command placements, then inspect what those commands would do. The map is only seven columns by five rows. The hard part is that three cats share it. A building another cat destroys is no longer the same building when the next cat arrives.
Adam Kulikowski’s KaijuGo article gave me the starting point: simulate the game, then search its command arrangements with simulated annealing. I built my own screenshot-to-plan workflow around that idea. My emphasis became the boundaries between reading an image, modeling the rules, and recommending a plan. I have not established a head-to-head improvement over his implementation.
Pixels are evidence, not a board specification
The screenshot reader uses Pillow and NumPy to compare tile and command appearances against reference templates. It is calibrated to the supplied camera view. Proportional resizing works; a different crop can need four-point alignment. It does not understand arbitrary screenshots the way a general vision model might.
I put an editable board beside the image because detection errors have consequences. Confusing a silver building with gold changes the reward. Missing a bottom-floor command changes both the route and the refund. The user reviews the inferred cells, cats, beds and commands before a search can start.
The budget deserves its own boundary. Available money is entered manually. Existing commands are refundable, so the search budget is available money plus those refunds, counted once. For the included example, $100 available plus $100 in placed commands gives a $200 plan budget. Entering $200 as the available amount would quietly give the solver money the player does not have.
The simulator was the first thing to debug
I checked the implementation against the supplied instruction panels and direct clarifications. A direction command changes facing after destroying a floor. Power Up adds 1,000; a power plant doubles the current power. When the two happen together, the doubling comes before the addition. Small sequencing differences produce very different totals.
All three cats act in the same world. I resolve movement and collisions before applying the surviving cat’s destination-floor effects. Swapping adjacent positions is safe. Reaching another cat’s bed causes a reversal rather than a finish. Simultaneous arrivals are ordered by current power; the first arrival gets a fixed bonus, while the next two receive multipliers.
Those cases became regression tests. They matter more than making the search impressive on one familiar board. A faster optimizer will exploit an incorrect rule just as enthusiastically as a correct one. I also keep uncertain fractional spike rounding explicit; the conservative setting avoids plans that depend on it.
Search for progress without changing the score
The optimizer uses independently seeded restarts of simulated annealing. It can change several commands together, focus mutations on floors the cats actually visit, and remove purchases that no longer fit the budget. Coordinated changes matter because a useful route can require several turns to be placed together.
Distance to beds helps the search compare incomplete routes. That guidance belongs to the search objective, not the result presented to the user. The displayed score is always the simulated combined power. Otherwise a helpful heuristic could quietly become a number that the game itself never awards.
I keep the best valid plan throughout the run. A deadline or an early stop returns that incumbent, rather than requiring the current search state to be the best one. The interface stays usable while the search works, and the output names coordinates and exact floors instead of asking the user to decipher an abstract command array.
A result worth being precise about
On the included example, a recorded 60-second run found a predicted total of 213,625 with $200 spent. Red reached its bed on turn eight. Blue and Green arrived on turn fifteen, with Blue taking the second-arrival bonus and Green the third. All three survived in the simulator.
A later 180.05-second run examined 99,230,720 candidates and returned the same best score. That is a useful experiment record, not proof that the score is globally optimal. It is also not an observed Roblox score. The next validation step is to compare the predicted replay with the game, starting at the first turn where they differ.
The second saved layout exposed a different problem: a Stomp icon split into several small image components. Improving that detection and testing a different alignment was more useful than simply running the optimizer longer on an incorrectly read board.
An answer should come with a way to inspect it
The planner returns the purchases, their total cost, the refund instructions and every predicted turn. You can change a cell, rerun the search, and see why the result changed. That makes the tool useful even when a screenshot needs correction or an assumption needs revisiting.
The playable version is separate from the public portfolio code. Recognition and search run behind a private service; the website provides the interface and the returned plan. A loading state, a stopped run and an unavailable service each need to leave the board understandable.
The lesson I would carry into another project is to keep the stages inspectable. Reading the input, defining the world and searching for a decision are different jobs. A large final number should never make those intermediate decisions disappear.