Duckends
Feedback collection with no form and no button: real-time custom object detection reads a thumbs-up or thumbs-down from a camera feed.
Built at Tezeract, where I was Lead Engineer. I architected and developed this product personally: the AI pipeline, the backend and the system design.
Architecture Overview
A single-stage detector trained on hand gestures, running continuously on modest edge hardware against a live camera feed. Detections are not events on their own. They pass through a temporal confirmation layer that requires a gesture to be held and stable across frames before it counts, and a debounce that prevents one person's held gesture from registering repeatedly.
The Challenges
Problem
People wave, scratch their heads, gesture while talking and hold objects. A detector firing on any frame that contains a thumbs-up shape produces a feedback stream that is mostly noise, and noise in a feedback product is worse than no product, because the client acts on it.
Solution
We made intent the thing being detected, not the gesture. A response requires the gesture to be held stable, roughly stationary, for a sustained window of frames, deliberately longer than an accidental hand position persists but short enough not to feel like a wait. A debounce window then suppresses repeat firings from the same held gesture. Both thresholds were tuned against footage of real deployment conditions rather than against the validation set, because the failure mode we cared about does not appear in clean training data.
Problem
General hand-pose models are heavy, and most of their capability was irrelevant. We needed two classes, reliably, on hardware that costs very little and sits in the enclosure of a kiosk.
Solution
We trained a small custom detector on exactly the two gestures, with the training data collected under the lighting and camera geometry of the actual deployment rather than from a public dataset. The narrow scope is what made the accuracy achievable at that model size: the model does not need to understand hands, only to distinguish two specific shapes from everything else, in a fixed setting. Scoping the model down to the problem, rather than adopting a capable general one and fighting its cost, was the decision the product rested on.
Problem
A camera-based interface is exclusionary in ways a button is not. Someone whose hands are full, who uses a wheelchair positioned outside the camera's framing, or who has limited hand mobility cannot respond at all, and unlike a broken button, the system gives them no indication that it failed.
Solution
The honest answer is that this is a supplement, not a replacement, and we were clear with clients about it: gesture capture raises response volume in settings where nobody was filling in the form anyway, but removing the conventional input path alongside it converts a low response rate into a systematically biased one. The correct deployment keeps a physical fallback and treats the camera as the path of least resistance rather than the only path, which is a product constraint, not a limitation we expected to engineer away later.