My phone's camera watches my hand, and my hand drives this PC: the pointer, the clicks, and the keys for a game. It starts in a dry run where the real mouse never moves, and goes live only on a deliberate click.
Game mode steering a racing game. The wheel on screen follows the angle of my wrists, the keys go only to the game's own window, and the panel on the left shows the hand rate, the steering angle and the measured delay from camera frame to key press.
The phone is only a camera. Hand tracking runs in the browser on the phone, and the numbers it produces travel to a helper on my PC. AirControl reads that stream and nothing else: it does not open the camera, does not see the picture, and does not need a model of its own to move the mouse.
Setup is a one off calibration per monitor. I point at each corner and hold still until a ring fills, which centres the active area where my hand naturally points rather than assuming it.
It opens in a dry run where the real mouse never moves and no key is ever sent, and what it would have done is shown on screen instead. Leaving the dry run asks a question that only a mouse click can answer. Buttons and tick boxes respond to clicks alone, so no key press can turn control on. The choice is remembered for next time, except after a session that did not close cleanly, which always comes back in the dry run.
Two key combinations stop it immediately: control passes off, any held button is released, and a gesture cannot turn it back on until the box in the window is ticked again. If the hand disappears for a second while control is on, the cursor stops where it is and any held button is released, then picks up again when the hand returns. Moving the real mouse makes the hand stand back for a second and then continue from wherever the cursor now is, which it works out by comparing the cursor against where it last put it rather than installing a system wide hook.
For a two dimensional game the hand stays still in one place and only its tilt and shape matter. Rolling it like a steering wheel past a set angle holds the arrow key; coming back towards upright releases it, with the release threshold set lower than the press so the key cannot flicker. A closed fist jumps, held for as long as the fist stays closed. The tilt is measured from the shape of the hand itself, so where it sits in the picture does not matter.
The keys are real key down and key up events with scan codes, sent to whichever game window is in front. The app never takes the focus, so it cannot steal a click from the game. Everything is released if the hand is lost, if control or game mode is switched off, or if the emergency stop is used. Each press is logged with the delay from camera frame to key: the app's own share of that is a fraction of a millisecond, and the rest is the trip from the phone, measured between 60 and 130 milliseconds.
For the racing profile the window becomes a cockpit: a steering wheel bolted in place that turns around its hub by the angle of my wrists, with my hands drawn on its grips, a glow that follows the steering, and large action text for each input. Separate recordings of the wheel and the panel can be captured for editing.