botting my fest app's mini-game

the oasis 2026 app (bits pilani's fest app) had a small driving game: three lanes, a red car at the bottom, coloured blocks falling towards you. you swiped left or right to dodge. it sped up until you crashed, and your score went on a leaderboard. my best by hand was 789.

i wanted to see how far a bot could get. it hit #2 on the leaderboard with 21,971 points in one 8 min 40 s run, after about 2.5 hours and 36 live runs. by the end of the fest that was #5.

ground rules

before writing any code i set some rules, mostly so dvm (the department that built the app) doesn't get mad:

• ·only play on my own phone, like a player would: read the screen, send swipes.
• ·no patching, repackaging or re-signing the apk.
• ·no touching the app's network traffic: no forged scores, no replayed api calls.

reading the apk to understand the game is fine. changing how it runs is not. (there are no prizes, for the record.)

what the game is made of

i pulled the apk with adb and unzipped it. no unity, no cocos, no webview game. it's a react native app (expo, hermes bytecode), drawn with skia and animated with reanimated.

that was lucky. reanimated keeps the source of its ui-thread "worklets" as plain strings inside the bundle, so grepping for lane turned up the game loop:

function changeLane_loopTs3(s, L, dir) {
  if (!s.running.value || s.paused.value) return false;
  const next = Math.max(0, Math.min(CFG.lanes - 1, s.lane.value + dir));
  if (next === s.lane.value) return false;
  s.lane.value = next;
  s.carX.value = withSpring(L.laneCenters[next], CFG.laneSpring);
  return true;
}

from the worklet strings:

• ·3 lanes, and the only move is changing lane. no jump, no duck.
• ·one horizontal swipe moves exactly one lane.
• ·each row blocks one lane, or two lanes 30% of the time, never all three.
• ·the obstacle colours are cosmetic.
• ·score is distance plus 50 per near miss.

seeing the game

the loop: the phone streams its screen to my mac, the mac finds the car and the obstacles, picks a lane and sends swipes back.

adb exec-out screencap takes about 0.7 s per frame, and an obstacle crosses the whole screen in about 1.1 s. useless. streaming with screenrecord --output-format=h264 and decoding in python with pyav gives about 120 fps, with frames about 0.1 s old. piping the same stream through ffmpeg buffered over a second first, so decoding in-process was the trick.

detection is plain opencv:

• ·mark every pixel that's colourful or bright.
• ·mask out the hud (score, best, pause button).
• ·find connected blobs.
• ·keep the ones shaped like obstacles (132 px squares) or like the car.

lane dashes are only 18 px wide, so they fall out of the size filter for free. the car's dark windshield split it into two "obstacles" at first, so blobs stacked in the car's rows get merged.

first runs, first bugs

run 1 lasted 12.6 s. the countdown traffic light and the big "3" got boxed as obstacles, and the bot swiped before the game accepted input, so its idea of which lane it was in was wrong from the start. fix: only act while the pause button is on screen, and only accept solid, square blobs.

then it moved into a lane that was blocked right then. the scoring ignored obstacles that would have "passed" by the time the car arrived. fix: score every option by seconds until it first collides, and only move if that beats staying put.

after that, roughly one new failure per run:

• ·run 2 (27.7 s): too scared to cross the middle lane. i'd assumed a lane change took 0.18 s; the debug frames said 0.05 to 0.1.
• ·run 4 (29.8 s): one bad frame match made the speed estimate jump from 1,600 to 2,293 px/s, so the bot thought an obstacle beside it would clear in time. one update can now move the estimate by at most 15%.
• ·run 5 (38.5 s): its escape lane was clear by 0.004 s according to the model. added a 0.1 s safety buffer.
• ·runs 6 and 7: stuck in an edge lane while the middle kept refilling, then a two-lane row arrived. when idle it now drifts back to the middle, where both escapes are one swipe away.

faster swipes

every swipe went through adb shell input swipe. on android 16 input is a two-line shell script that calls cmd input, and a swipe took about 150 ms.

• ·writing raw touch events to /dev/input/event3 was blocked. the adb shell user is in the input group, but selinux said no anyway.
• ·the shortest swipe the game accepts is about 15 ms. it's a pan gesture and needs real movement; 1, 5, 8 and 10 ms were all ignored.
• ·cmd input swipe … 16, plus starting the second swipe of a two-lane move 30 ms after the first, took a two-lane move from about 300 ms to 110.
• ·the car tilts while changing lanes, which makes its box wider than my car filter allowed. mid-move it vanished and its pieces turned into fake obstacles. widened the filter.

median survival went from 32 s to 54 s. a faster usb cable would not have helped: a round trip during play was 36 ms. the delay was all on the phone.

the bug i wrote three times

to save time, the bot assumes a swipe worked and updates its lane straight away, then checks against the camera later. but the camera is about 0.1 s behind. if it checks too early it still sees the old lane, "corrects" itself back, and plans from the wrong lane.

fix: only trust the camera in frames captured after the move has finished, plus 0.2 s, and only if the car sits in another lane for 3 frames in a row. after that the log showed zero lane corrections across five runs, and the median went to 72 s. i then wrote the same bug twice more in the simulator.

one rare failure stayed: about 1 swipe in 200 sent the car the wrong way. i never found out why.

reading the exact numbers

the worklet strings had code but no constants. those live in the bytecode. hermes-dec disassembled the bundle into an 85mb text file and the config was just sitting there:

CFG = { lanes: 3, baseSpeed: 380, maxSpeed: 900, accel: 9, pxPerMeter: 40,
        spawnGapStart: 500, spawnGapMin: 320, doubleSpawnChance: 0.3,
        carW: 54, carH: 96, obsW: 50, obsH: 50, nearMissGap: 30, nearMissBonus: 50 }
laneSpring = { damping: 16, stiffness: 240, mass: 0.6 }

units are dp (×2.625 on this phone). two things followed.

a lane change is fully predictable. the car leaves its old lane 76 ms after the bot decides and is in the new one at 104 ms.

and the game can be survived forever with perfect timing. at top speed a row arrives every 0.356 s and blocks at most two lanes for 0.144 s. between rows every lane is free for 0.211 s. no pattern is a real trap, so every death is a timing or planning error.

logging the car's position after every move and fitting that spring to 94 clean moves gave the real swipe delay: 20 ms median. my config had guessed 90. that one number had shaped half my decisions.

a simulator and a planner

with the exact rules i wrote sim.py. it runs the game's own loop at 120 hz and calls the bot's real decision code, with 0.1 s old frames, obstacles hidden behind the hud, some detection noise and swipe delays sampled from real measurements.

then a row planner. it searches lane plans over the next second on a 20 ms grid, with the exact spring and hitbox, and picks the plan that survives longest with the most slack.

with perfect input the planner reached 10,000 points in 48 of 48 simulated runs. with 1% of swipes lost, neither bot ever did. planning was solved; input was the bottleneck. live, the planner lasted 175 s, 187 s, and one run my test script stopped at 237 s.

the injector

the last piece was an 87 line java program, compiled to dex and run on the phone with app_process, the same trick scrcpy uses. it stays running, reads swipe commands from stdin and injects the touch events itself, so there's no process to start per swipe:

MotionEvent e = MotionEvent.obtain(down, when, action, x, Y, 1.0f, 1.0f, 0, 1.0f, 1.0f, deviceId, 0);
e.setSource(InputDevice.SOURCE_TOUCHSCREEN);
inject.invoke(im, e, 2); // INJECT_INPUT_EVENT_MODE_WAIT_FOR_FINISH

its first two live tests failed badly: swipes 0.3 to 0.4 s late, or lost. i almost threw it out. the real cause was a simulation job i'd forgotten in the background, which had kept my mac at a load average of 105 for 40 minutes. every live run in that window was junk.

on an idle mac the gain is real but small: about 7 ms at the median, with fewer slow swipes.

the first proper run with it lasted 520 seconds: 856 lane changes, 21,971 points.

that's one run, not a success rate, and some of it is luck. it also used the old greedy decision code, not the planner. i haven't refitted the planner's timing for the injector yet.

what i'd tell myself at the start

• ·stop guessing numbers. i guessed swipes took 90 ms, they took 20, and half the bot was built around the wrong one.
• ·the camera is always 0.1 s behind. you will forget this. i forgot it three times.
• ·open the apk first. the exact numbers were sitting in the bundle the whole time.
• ·write the simulator earlier. a hundred fake runs take the same minute as one real one.
• ·before blaming the new thing, check what's still running in the background.