Beyond theLast-Frame
Teaching a retinal-scan AI to read time — not just the last snapshot.
A graduation project on temporal deep learning for the automated grading of retinal inflammation in fluorescein angiography.
A film,not a photo.
A fluorescein angiography exam is closer to a short film than a photograph. A dye is injected and tracked through the retinal vasculature over ten to fifteen minutes — and clinicians diagnose by how the brightness moves, not how it looks at one instant.
The deep-learning pipeline at Idiap’s MedAI group treated each exam as a single static image — the last frame. For several disease patterns, that discards the entire basis of the diagnosis. This project set out to fix that.
Leakage
Pooling
Staining
Window defect
Current FA classification pipelines insufficiently utilise the temporal information present in the examination — limiting diagnostic performance, robustness and interpretability.
What thisproject found.
Each is a self-contained piece of the work — open any one to go as deep as you like. Together they trace the project from raw data to a working temporal pipeline.
Recovering time from pixels
The benchmark data had no usable timing — so I read it back off the images themselves.
57% → 100% accuracy · 32,844 timestamps recoveredCalibrating phases to the data
The textbook says fluorescein angiography has three fixed phases. I tested that against the data — and it did not hold.
Textbook 47.5 / 197.5 s → data-driven 103 / 518 sA cheap test before an expensive one
GPU time on a shared cluster is scarce, so I built a way to rank ideas before training any of them.
Rank configurations with an LDA probe — pre-trainingThe temporal pipeline
A drop-in temporal block — so any gain is provably from modelling time, not from swapping the backbone.
Frozen backbone · 12 embeddings · GRUWhat the model gets wrong
I built a viewer for the model's mistakes one by one — and it pointed at the preprocessing, not the model.
Root cause traced to 334 → 224 px downscalingAnd then?Where it stands — for the next researcherRead on ↓For whoevercontinues it.
ROC-AUC — last frame only (prior work)
Test AUC — state at time of writing
Honest framing: this is a working method and pipeline, not a finished clinical product. The numbers are real but provisional. The value for whoever picks this up is the groundwork — a re-runnable benchmark, a calibrated method, and 0 validated frame timestamps added to the dataset.
- A working, MedNet-integrated temporal evaluation pipeline.
- A reproducible LDA phase-calibration probe and separability benchmark.
- 32,844 validated frame timestamps, reusable by the whole group.
- A two-layer automated unit-test suite.
- The explicit time-embedding track (the recommended next direction).
- HOJG-scale external validation — the key open research question.
- Integration with production clinical workflows.
Thank you fortaking the time.
A real research contribution — built carefully, reviewed honestly, and made reusable for the team that continues it.