Evaluate human motion with precision and deep networks.
A pose-based machine learning pipeline that automatically detects unsafe exercise form using standard camera input, utilizing MediaPipe and Temporal Convolutional Networks.
4,979
Annotated repetitions
222
Features per frame
33
Body landmarks tracked
147k
Model parameters
The Silent Epidemic
Poor exercise form is a major cause of injuries, yet the vast majority of athletes train without real-time supervision. They repeat the exact same errors over and over until they inevitably lead to injury.
Current solutions are completely unscalable. Our objective is to automatically evaluate posture and classify exercises to provide corrective feedback using a software-based alternative on standard RGB video.
Watching lifters blow out their lumbar spine because they can't accurately gauge their torso angle mid-lift is the most frustrating part of physiotherapy. We need automated biomechanical feedback systems yesterday. 🤦♀️
Without proper form, these lifts can cause more harm than good.
Good morning. Deadlifts didn't wreck your back. Your shitty form did. Deadlifts done properly are an elite exercise for posterior chain strength. Stop blaming the exercise and fix your angles.
Incorrect Joint Angles
Knee valgus, elbow flare, shoulder impingement — small angular errors that compound into serious injuries over time.
Poor Spinal Alignment
Lumbar rounding under heavy load creates dangerous shear forces on the spine. A problem invisible to the lifter themselves.
Asymmetric Movement
Left-right imbalances and uneven loading patterns lead to chronic overuse injuries that develop gradually.
Invisible Errors
The Anatomy of Injury
Knee valgus, elbow flare, shoulder impingement — small angular errors that compound into serious injuries over time, entirely invisible to the lifter without real-time tracking.
Rules Can't
Keep Up
The same joint angle can mean “perfect form” in one exercise and “injury risk” in another. Fixed rules break down when the real world is this complex.
Context-Dependent
Same knee angle is correct for squats but dangerous for lunges. ML learns the context.
High Variability
Different body types, camera angles, and movement speeds make hard-coded rules impractical.
Pattern Recognition
ML learns from massive datasets (like Fitness-AQA) and generalizes across users and environments.
“If knee angle < 90°, flag as unsafe”
Breaks with different exercises, body types, camera angles
Learns from 4,979 annotated repetitions across the Fitness-AQA dataset
Generalizes to new users, angles, and speeds automatically
End-to-End Architecture
A multi-stage machine learning pipeline performing pose extraction, feature normalization, and temporal action quality assessment.
Input Data
Unprocessed video clips representing single repetitions of compound barbell exercises, capturing high variance in tempo and camera angles.
Pose Estimation
MediaPipe extracts 33 body landmarks per frame. Missing frames are linearly interpolated and coordinates are normalized using shoulder width and hip centering.
Temporal Processing
A Temporal Convolutional Network (TCN) processes 222 biomechanical features (coordinates, 12 joint angles, velocities) over a fixed sequence of 100 frames.
Multi-Task Output
Dual heads perform classification to predict specific form errors and regression to generate a composite score of exercise quality.
222 features per frame
Each video frame is transformed into a rich feature vector combining spatial positions, joint mechanics, and movement dynamics.
88%
Reduction in variance via hip-centred normalization
Trained on Real Data
The Fitness-AQA dataset provides 4,979 annotated repetitions across compound exercises, with temporal error annotations and subject-level splits to guarantee robust modeling.
Back Squat
Error Prevalence
Under the Hood
A Temporal Convolutional Network with dual-task heads — lightweight enough for real-time deployment, powerful enough to catch subtle form errors.
Why TCN over LSTM?
All timesteps processed simultaneously — parallelizable on GPU, no sequential bottleneck.
Exponential Dilation
Dilations of 1, 2, 4, 8, 16, 32 — the final block sees the entire 100-frame sequence.
Dual Output Heads
Quality score regression via Huber Loss + error classification via BCE with pos_weight up to 12x for rare errors.
Lightweight Model
Only ~147k parameters. Small enough for mobile deployment, large enough to capture complex biomechanics.
Regularization Stack
Dropout 0.2, weight decay 1e-4, gradient clipping at 1.0, early stopping with patience of 10 epochs.
Temporal Convolutional Network
147k
Parameters
6
TCN Blocks
100+
Receptive Field