Formole
A virtual soccer coaching app that analyses a phone-recorded kick with pose estimation and returns the kind of correction a coach would give.
Built at Tezeract, where I was Lead Engineer. I architected and developed this product personally: the AI pipeline, the backend and the system design.
Architecture Overview
A mobile capture client feeding a Python inference service. The app guides recording so footage lands inside a known framing envelope, uploads the clip, and a server-side pipeline runs pose estimation per frame, reconstructs the kick as a time series of joint positions, and converts that series into biomechanical measurements before any feedback text is generated. The rule layer that turns measurements into coaching sits above the model and is editable without retraining.
The Challenges
Problem
Consumer phone footage is close to a worst case for pose estimation. The camera is propped against a water bottle at an unknown height, the sun is behind the player, the framing cuts off a foot at the moment of contact. Training a model robust to all of that is a research programme, not an MVP.
Solution
We moved the problem out of the model and into the capture experience. The recording screen overlays a framing guide and refuses to submit clips where the required joints are not visible for the whole strike, which narrows the input distribution enough that a standard pose estimator performs reliably inside it. The tradeoff is explicit and worth defending: we made the user do slightly more work at capture time in exchange for feedback that is actually trustworthy, rather than accepting any video and silently degrading to guesses.
Problem
A pose model returns joint coordinates. A player wants to know why the ball sliced. Nothing in the model output bridges that gap, and the naive move of showing the skeleton overlay and calling it analysis looks impressive and helps nobody.
Solution
We inserted a measurement layer between the model and the user. Joint positions across frames are reduced to the quantities a coach actually reads: plant-foot placement relative to the ball, hip and shoulder rotation through the strike, the frame of contact, follow-through direction. Those measurements are then compared against thresholds derived with domain input, so the app says the plant foot is too far behind the ball rather than reporting an angle. Keeping the thresholds in a rule layer rather than learning them end to end meant coaching logic could be corrected in an afternoon when a domain expert disagreed with it.
Problem
Running the model on-device avoids upload latency and video bandwidth, which matters for users on constrained mobile data. Running it server-side gives consistent results and lets the model improve without shipping an app update. Both were defensible and the decision was not reversible cheaply.
Solution
We put inference on the server and spent the effort on making the upload tolerable instead, clipping to the relevant seconds on-device before upload, and treating analysis as asynchronous with a notification rather than blocking the user on a progress bar. The deciding factor was iteration speed: at that stage the coaching logic was changing weekly, and an on-device model would have tied every improvement to an app store review cycle.