Test keyboard input in a three.js game, frame by frame
How a coding agent presses keys in a three.js game on exact frames of a paused clock and reads the game's state back as JSON, so the same inputs give the same result on every run, using the paprika3d plugin's game_act and game_observe.
The problem
A three.js game reads the keyboard, steps its state once per animation frame, and draws the result. To test that holding ArrowRight moves the player, or that a jump cannot be repeated in mid-air, a test has to send key events on known frames and then read the state. Two things get in the way. Animation frames follow the display's refresh rate and the machine's load, so a test that presses a key for "half a second" moves the player a different distance on each run. And the state lives in the page's JavaScript, behind a canvas, so a screenshot shows where the player is drawn without saying what its position or velocity is.
This guide uses paprika3d, a Claude Code plugin that runs the game in headless Chrome on a paused clock. Its game_act tool sends keys, mouse, touch or game actions for a given number of frames, each frame 1/60 s of game time, and replies with the game's state after them.
Install
paprika3D needs Google Chrome and Node 20.11 or later.
claude plugin marketplace add paprika3d/paprika3d
claude plugin install paprika3d@paprika3d
npm install three
The game
Keys work on any page, since paprika3d sends them as real browser events. To get the state back as JSON, the game exposes window.__game with describe(), observe(), act(input) and setState(partial), as described on the convention page. Three habits make the game testable:
- State lives in plain variables, apart from the meshes that draw it.
- Real key events and
act({ key, down })go through the same function. - Each frame advances the game by its elapsed time, capped at 0.1 s, so the paused clock decides how far the game moves.
This page moves a ball with the arrow keys and jumps on Space:
<!-- verify -->
<!doctype html>
<html><head><meta charset="utf-8"><title>Jumper</title>
<style>html,body{margin:0;overflow:hidden}canvas{display:block}</style></head>
<body>
<script type="importmap">{"imports":{"three":"./node_modules/three/build/three.module.js"}}</script>
<script type="module">
import * as THREE from 'three';
// Game state lives in plain variables, apart from the meshes that draw it.
const SPEED = 3, JUMP = 6, GRAVITY = 18; // units per second, units per second, units per second squared
const held = new Set();
let x = 0, y = 0, vy = 0, jumps = 0;
function step(dt) {
x += ((held.has('ArrowRight') ? 1 : 0) - (held.has('ArrowLeft') ? 1 : 0)) * SPEED * dt;
vy -= GRAVITY * dt;
y = Math.max(0, y + vy * dt);
if (y === 0) vy = Math.max(0, vy);
}
// One input path for real keys and for the test harness.
function act({ key, down = true }) {
if (key === ' ' && down && y === 0) { vy = JUMP; jumps++; } // one-shot: acts on keydown
if (down) held.add(key); else held.delete(key);
}
addEventListener('keydown', e => act({ key: e.key }));
addEventListener('keyup', e => act({ key: e.key, down: false }));
const renderer = new THREE.WebGLRenderer({ antialias: true });
renderer.setSize(innerWidth, innerHeight);
document.body.appendChild(renderer.domElement);
const scene = new THREE.Scene();
scene.background = new THREE.Color(0x182030);
const camera = new THREE.PerspectiveCamera(50, innerWidth / innerHeight, 0.1, 100);
camera.position.set(0, 2, 9);
scene.add(new THREE.HemisphereLight(0xffffff, 0x334455, 2));
const ground = new THREE.Mesh(new THREE.BoxGeometry(12, 0.2, 2), new THREE.MeshStandardMaterial({ color: 0x4a6a4a }));
ground.name = 'ground';
ground.position.y = -0.6;
scene.add(ground);
const player = new THREE.Mesh(new THREE.SphereGeometry(0.5, 32, 16), new THREE.MeshStandardMaterial({ color: 0xff8030 }));
player.name = 'player';
scene.add(player);
let last = null;
renderer.setAnimationLoop((now) => {
step(last === null ? 1 / 60 : Math.min((now - last) / 1000, 0.1)); // never more than 0.1 s a frame
last = now;
player.position.set(x, y, 0);
renderer.render(scene, camera);
});
const round = (n) => +n.toFixed(3);
window.__game = {
describe: () => ({ name: 'jumper', units: 'world units, y up', keys: { ArrowRight: 'hold to move right', ArrowLeft: 'hold to move left', ' ': 'jump' } }),
observe: () => ({ x: round(x), y: round(y), vy: round(vy), jumps }),
act,
setState: (s) => { ({ x = x, y = y, vy = vy, jumps = jumps } = s); },
};
</script>
</body></html>
The tool calls, in order
Each call is followed by the reply paprika3d gave for this page on 2026-10-14. Run folder paths are shortened to <project>.
1. Open the game. With window.__game present, the reply carries the game's own describe().
game_open {"url": "index.html", "cwd": "<project>"}
Opened http://127.0.0.1:61887/index.html, paused at frame 0.
{"name":"jumper","units":"world units, y up","keys":{"ArrowRight":"hold to move right","ArrowLeft":"hold to move left"," ":"jump"},"hasConvention":true}
Procedural geometry: import { geo, scatter } from "./geo.js" (see the skill's geometry reference).
Next: game_look to see it, then game_act for each interaction the request names.
Run folder: <project>/paprika3d/runs/2026-10-14T07-52-41-459Z
2. Read the starting state. game_observe returns observe() plus a harness block: the frame, renderer counts, console errors since the last observe, and what changed.
game_observe {}
{"x":0,"y":0,"vy":0,"jumps":0,"harness":{"frame":0,"renderer":{"calls":0,"triangles":0,"textures":0,"geometries":0,"programs":0},"scene":{"objects":3,"meshes":2,"lights":1,"cameras":0},"errors":[],"changed":null}}
3. Hold a key for a set number of frames. Each step advances frames frames (default 1). Holding ArrowRight for 30 frames is half a second of game time, at 3 units per second:
game_act {"steps": [{"key": "ArrowRight", "frames": 30}, {"key": "ArrowRight", "down": false}]}
{"after":{"x":1.5,"y":0,"vy":0,"jumps":0},"changed":{"x":1.5}}
The player moved 1.5 units, as the speed says it should.
4. Jump, and stop at the top. A key goes down on one frame and up on the next; wait steps frames with no input. With a jump speed of 6 and gravity of 18, the ball stops rising a third of a second after the jump, which is where these steps end:
game_act {"steps": [{"key": " "}, {"key": " ", "down": false}, {"wait": 18}]}
{"after":{"x":1.5,"y":0.95,"vy":0,"jumps":1},"changed":{"y":0.95,"jumps":1}}
vy is 0 at the top, and jumps counted one jump.
5. Test a rule: no jump in mid-air. Press Space again at the top of the jump and let the ball fall:
game_act {"steps": [{"key": " "}, {"key": " ", "down": false}, {"wait": 40}]}
{"after":{"x":1.5,"y":0,"vy":0,"jumps":1},"changed":{"y":0}}
jumps is still 1: the second press was ignored, and the ball landed.
6. Look at it. game_look confirms the player is drawn where the state says.
game_look {}
Picture: <project>/paprika3d/runs/2026-10-14T07-52-41-459Z/look-1.png
ID image: <project>/paprika3d/runs/2026-10-14T07-52-41-459Z/look-1-ids.png
On screen (2):
ground 8.7% [0, 614, 1279, 689]
player 0.7% [735, 542, 831, 639]
Never visible: none
7. Run it again. Reopening the page and sending the same steps gives the same state on the same frame:
game_open {"url": "index.html", "cwd": "<project>"}
game_act {"steps": [{"key": "ArrowRight", "frames": 30}, {"key": "ArrowRight", "down": false}]}
{"after":{"x":1.5,"y":0,"vy":0,"jumps":0},"changed":{"x":1.5}}
game_act {"steps": [{"key": " "}, {"key": " ", "down": false}, {"wait": 18}]}
{"after":{"x":1.5,"y":0.95,"vy":0,"jumps":1},"changed":{"y":0.95,"jumps":1}}
game_observe {}
{"x":1.5,"y":0.95,"vy":0,"jumps":1,"harness":{"frame":51,"renderer":{"calls":2,"triangles":972,"textures":1,"geometries":2,"programs":1},"scene":{"objects":3,"meshes":2,"lights":1,"cameras":0},"errors":[],"changed":null}}
What to read in the reply
after: the state when the last step's frames are done. Compare it with what the request says the input should do.changed: the keys whose values moved. A key press that changes nothing is a dead control.harness.errorsingame_observe: console errors since the last observe. A game that throws on an input shows it here.harness.frame: the frame count, to confirm how much game time the steps covered.
Pitfalls
- One-shot controls act on keydown. A key can go down and up within one frame, so a jump or a shot that waits for a held key across frames may never fire.
- Release held keys. A key sent with no
down: falsestep stays held for the following calls. - Never read real time. A game that moves by
Date.now()deltas read outside the animation frame, or by a fixed amount per frame with no cap, gives different results at a different frame rate. The clock paprika3d installs fakesrequestAnimationFrame,performance.now,Dateand timers. - Randomness. Use
Math.random; paprika3d seeds it, so the same steps see the same game. - For exact rules, lock a contract. For a game whose rules can be written down (scoring, collisions, turn order),
game_contractturns the user's prompt into keypoints andgame_checkruns them after each change.
Without paprika3d
A browser test can drive the same game. With Playwright or Puppeteer, page.keyboard.down('ArrowRight') and page.keyboard.up('ArrowRight') send real key events, and page.evaluate(() => window.__game.observe()) reads the state. To make the result repeatable, the test also needs control of time: Playwright's page.clock API can install a fake clock and advance it, or the game can expose a step(dt) function the test calls directly with a fixed 1/60 s. For logic alone, keeping the rules in a module with no three.js imports lets a unit test runner such as Vitest call step and act with no browser at all.