A few months ago, Zac and I won the "Future of Health" track at UK Hack — a16z's London hackathon with Mistral AI.
The idea was: point your phone camera at your face, we tell you your biological age, heart rate, sleep quality, and whether you look like you've been making good decisions lately.
Software eats hardware
Most health tracking requires hardware. Wearables, blood tests, clinic visits. We wanted to see how far we could get with just a phone camera and some good models.
Turns out, pretty far.
With just a 10-second video of your face from an iPhone and good lighting, we can extract a surprising amount of signal. Skin texture, micro-expressions, eye movement patterns. Feed an LLM with the right prompts and it starts outputting things that actually correlate with clinical biomarkers.
We built a Next.js app that captures video, ships it to a Python backend, runs it through Pixtral, and returns a full health dashboard. Functional age, cardiovascular indicators, cognitive engagement score.
Two brains, one face
Mixture of models. Pixtral handles the visual-aging markers — skin texture, facial structure, signs of aging. But for the real biomarkers, we layered in traditional computer vision techniques that don't need an LLM.
Pupillary movements give you cognitive load and neurological signals. Blood flow detected through subtle skin color changes (remote photoplethysmography) gets you heart rate and variability. Eye movement patterns reveal sleep quality and fatigue.
The LLM is the storyteller — it takes the raw signals and turns them into 'you've aged 2 years in the last 6 months, probably the sleep deprivation.'
20 hours later
- Frontend: Next.js on Vercel
- Backend: Python/Flask on Heroku
- Database: Supabase
- Model: Mistral Pixtral-8b
- Build time: ~20 hours
Brilliant or insane?
LLMs are weirdly good at reading faces. Not in a creepy way — in a "this is probably useful for healthcare" way. The outputs held up — we cross-checked against published functional age estimation methods and the correlation was stronger than we expected.
The judges seemed to agree. We took home the health track, which felt good given we'd spent most of the night arguing about whether this was brilliant or insane.
Probably both.
Bonus: Parliament
A few weeks later, we got invited to drinks at the Houses of Parliament with MPs and winners from the other tracks. Not bad for a weekend hack.
