Recovering timefrom pixels
The benchmark data had no usable timing — so I read it back off the images themselves.
To model how an exam evolves over time, the model needs to know when each frame was taken. The public Aptos benchmark has no structured per-frame timing — the elapsed time is only printed as text, burned into the corner of each image.
So the timing had to be read back with OCR. The first approach was not reliable enough to trust as a supervision signal — so instead of guessing, the methods were benchmarked explicitly against 150 manually-checked frames.
Three OCR methods, 150 hand-checked frames.
86 / 150 frames
76 / 150 frames
144 / 150 frames
The six remaining “failures” were not failures at all — they were Indocyanine Green Angiography frames, a different scan type entirely. On genuine FA frames, Gemini reached 144 / 144 — a perfect score, and the discovery doubled as a free, automatic way to discard non-FA frames.
Not just a fix for this project — a genuine, reusable enrichment of the Aptos dataset for the whole MedAI group.
With timing recovered, the data could finally be read as a timeline. It showed striking variability — some examinations run up to 90 minutes, most at most 14 — and frames cluster near the beginning and end of an exam, sparse in the relative middle. That shape directly informed the next finding.