Temporal sequence · miss analysis

Misclassified sequences: exact dataloader frames (≤12), attention α, softmax at the selected timestep, and softmax vs time.

test · accuracy=0.651 · macro ROC-AUC=0.846

Train dir: /remote/idiap.svm/temp.medai/tvanrijn/Graduation_Project/beyond-the-last-frame/results/temporal-hyperftype-phase-strat-attn-max/Train_8
Checkpoint: /remote/idiap.svm/temp.medai/tvanrijn/Graduation_Project/beyond-the-last-frame/results/temporal-hyperftype-phase-strat-attn-max/Train_8/model-at-highest-validation-auc-epoch=16.ckpt
Temporal JSON: /remote/idiap.svm/temp.medai/tvanrijn/Graduation_Project/beyond-the-last-frame/config/temporal_hyperftype.json
Datadir: /remote/idiap.svm/resource.dataset13/3rd_APTOS_Angiographic_Report

settings.json
{
  "model": "temporal-vitgru",
  "datamodule": "angioreport-hyperftype-seq",
  "epochs": 20,
  "batch_size": 16,
  "seed": 42,
  "device": "cuda",
  "parallel": 2,
  "task_type": "multiclass",
  "pretrained_weights": "retfoundgreen_statedict.pth",
  "cache_samples": false,
  "backbone_unfreeze_last_n": 4,
  "class_weight_crossentropy": true,
  "frame_span": "phase_stratified",
  "bound_early_mid": 103.0,
  "bound_mid_late": 518.0,
  "temporal_pooling": "attention",
  "global_pool": "max",
  "lr_backbone": 9.68e-06,
  "lr_head": 9.68e-05
}
#1

Train/Train/1522_R · test

12 model frames (max 12) · frame_span=phase_stratified · P(true)=0.008 · prefix argmax flips=1 · min prefix margin=0.318

Ground truth 4 — window defect
Prediction (exam-level) 0 — leakage

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.002
early
1.jpg
#1 α=0.002
early
3.jpg
#2 α=0.002
early
4.jpg
#3 α=0.003
early
5.jpg
#4 α=0.007
mid
9.jpg
#5 α=0.076
mid
14.jpg
#6 α=0.122
mid
18.jpg
#7 α=0.149
mid
19.jpg
#8 α=0.141
late
20.jpg
#9 α=0.151
late
21.jpg
#10 α=0.157
late
22.jpg
#11 α=0.187
late
FA frame

Softmax vs time

Softmax vs frame · T=12 0 1 2 3 4 5 6 7 8 9 10 11 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
04window defectFalse
14window defectFalse
24window defectFalse
34window defectFalse
40leakageTrue
50leakageTrue
60leakageTrue
70leakageTrue
80leakageTrue
90leakageTrue
100leakageTrue
110leakageTrue
#2

Train/Train/724_L · test

3 model frames (max 12) · frame_span=phase_stratified · P(true)=0.008 · prefix argmax flips=1 · min prefix margin=0.270

Ground truth 2 — no hyperfluorescence
Prediction (exam-level) 0 — leakage

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.016
mid
1.jpg
#1 α=0.170
late
2.jpg
#2 α=0.813
late
FA frame

Softmax vs time

Softmax vs frame · T=3 0 1 2 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
02no hyperfluorescenceFalse
10leakageTrue
20leakageTrue
#3

Train/Train/0_R · test

12 model frames (max 12) · frame_span=phase_stratified · P(true)=0.011 · prefix argmax flips=2 · min prefix margin=0.011

Ground truth 1 — staining
Prediction (exam-level) 0 — leakage

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.002
early
2.jpg
#1 α=0.002
early
3.jpg
#2 α=0.002
early
5.jpg
#3 α=0.008
early
6.jpg
#4 α=0.050
mid
7.jpg
#5 α=0.100
mid
8.jpg
#6 α=0.142
mid
9.jpg
#7 α=0.146
mid
10.jpg
#8 α=0.160
late
11.jpg
#9 α=0.171
late
12.jpg
#10 α=0.115
late
13.jpg
#11 α=0.100
late
FA frame

Softmax vs time

Softmax vs frame · T=12 0 1 2 3 4 5 6 7 8 9 10 11 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
01stainingFalse
14window defectFalse
24window defectFalse
30leakageTrue
40leakageTrue
50leakageTrue
60leakageTrue
70leakageTrue
80leakageTrue
90leakageTrue
100leakageTrue
110leakageTrue
#4

Train/Train/335_L · test

3 model frames (max 12) · frame_span=phase_stratified · P(true)=0.015 · prefix argmax flips=0 · min prefix margin=0.173

Ground truth 2 — no hyperfluorescence
Prediction (exam-level) 4 — window defect

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.344
mid
1.jpg
#1 α=0.280
late
2.jpg
#2 α=0.376
late
FA frame

Softmax vs time

Softmax vs frame · T=3 0 1 2 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
04window defectTrue
14window defectTrue
24window defectTrue
#5

Train/Train/486_L · test

7 model frames (max 12) · frame_span=phase_stratified · P(true)=0.017 · prefix argmax flips=1 · min prefix margin=0.017

Ground truth 4 — window defect
Prediction (exam-level) 0 — leakage

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.003
early
1.jpg
#1 α=0.010
mid
2.jpg
#2 α=0.025
mid
3.jpg
#3 α=0.099
late
4.jpg
#4 α=0.204
late
5.jpg
#5 α=0.373
late
6.jpg
#6 α=0.285
late
FA frame

Softmax vs time

Softmax vs frame · T=7 0 1 2 3 4 5 6 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
01stainingFalse
10leakageTrue
20leakageTrue
30leakageTrue
40leakageTrue
50leakageTrue
60leakageTrue
#6

Train/Train/971_L · test

2 model frames (max 12) · frame_span=phase_stratified · P(true)=0.024 · prefix argmax flips=0 · min prefix margin=0.902

Ground truth 1 — staining
Prediction (exam-level) 2 — no hyperfluorescence

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.527
mid
1.jpg
#1 α=0.473
late
FA frame

Softmax vs time

Softmax vs frame · T=2 0 1 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
02no hyperfluorescenceTrue
12no hyperfluorescenceTrue
#7

Train/Train/335_R · test

12 model frames (max 12) · frame_span=phase_stratified · P(true)=0.028 · prefix argmax flips=2 · min prefix margin=0.011

Ground truth 4 — window defect
Prediction (exam-level) 3 — pooling

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.000
early
1.jpg
#1 α=0.000
early
2.jpg
#2 α=0.008
early
3.jpg
#3 α=0.010
mid
4.jpg
#4 α=0.042
mid
5.jpg
#5 α=0.067
late
6.jpg
#6 α=0.123
late
7.jpg
#7 α=0.120
late
8.jpg
#8 α=0.194
late
9.jpg
#9 α=0.149
late
10.jpg
#10 α=0.138
late
11.jpg
#11 α=0.148
late
FA frame

Softmax vs time

Softmax vs frame · T=12 0 1 2 3 4 5 6 7 8 9 10 11 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
02no hyperfluorescenceFalse
12no hyperfluorescenceFalse
24window defectFalse
34window defectFalse
44window defectFalse
53poolingTrue
63poolingTrue
73poolingTrue
83poolingTrue
93poolingTrue
103poolingTrue
113poolingTrue
#8

Train/Train/1373_L · test

7 model frames (max 12) · frame_span=phase_stratified · P(true)=0.029 · prefix argmax flips=2 · min prefix margin=0.054

Ground truth 2 — no hyperfluorescence
Prediction (exam-level) 4 — window defect

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.084
early
1.jpg
#1 α=0.077
early
2.jpg
#2 α=0.007
early
3.jpg
#3 α=0.084
mid
4.jpg
#4 α=0.090
mid
5.jpg
#5 α=0.233
mid
6.jpg
#6 α=0.423
late
FA frame

Softmax vs time

Softmax vs frame · T=7 0 1 2 3 4 5 6 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
04window defectTrue
14window defectTrue
22no hyperfluorescenceFalse
31stainingFalse
41stainingFalse
51stainingFalse
61stainingFalse
#9

Train/Train/28_L · test

12 model frames (max 12) · frame_span=phase_stratified · P(true)=0.032 · prefix argmax flips=0 · min prefix margin=0.830

Ground truth 1 — staining
Prediction (exam-level) 2 — no hyperfluorescence

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.068
early
2.jpg
#1 α=0.079
early
5.jpg
#2 α=0.046
early
7.jpg
#3 α=0.051
early
8.jpg
#4 α=0.056
mid
9.jpg
#5 α=0.070
mid
10.jpg
#6 α=0.073
mid
11.jpg
#7 α=0.104
mid
11.jpg
#8 α=0.102
mid
12.jpg
#9 α=0.096
late
13.jpg
#10 α=0.109
late
14.jpg
#11 α=0.145
late
FA frame

Softmax vs time

Softmax vs frame · T=12 0 1 2 3 4 5 6 7 8 9 10 11 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
02no hyperfluorescenceTrue
12no hyperfluorescenceTrue
22no hyperfluorescenceTrue
32no hyperfluorescenceTrue
42no hyperfluorescenceTrue
52no hyperfluorescenceTrue
62no hyperfluorescenceTrue
72no hyperfluorescenceTrue
82no hyperfluorescenceTrue
92no hyperfluorescenceTrue
102no hyperfluorescenceTrue
112no hyperfluorescenceTrue
#10

Train/Train/1318_R · test

6 model frames (max 12) · frame_span=phase_stratified · P(true)=0.033 · prefix argmax flips=2 · min prefix margin=0.048

Ground truth 1 — staining
Prediction (exam-level) 3 — pooling

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.025
mid
1.jpg
#1 α=0.044
mid
2.jpg
#2 α=0.137
mid
3.jpg
#3 α=0.278
mid
4.jpg
#4 α=0.251
late
5.jpg
#5 α=0.264
late
FA frame

Softmax vs time

Softmax vs frame · T=6 0 1 2 3 4 5 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
04window defectFalse
14window defectFalse
23poolingTrue
33poolingTrue
40leakageFalse
50leakageFalse
#11

Train/Train/1223_L · test

12 model frames (max 12) · frame_span=phase_stratified · P(true)=0.033 · prefix argmax flips=1 · min prefix margin=0.029

Ground truth 1 — staining
Prediction (exam-level) 4 — window defect

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.015
early
3.jpg
#1 α=0.009
early
4.jpg
#2 α=0.014
mid
14.jpg
#3 α=0.024
mid
25.jpg
#4 α=0.045
mid
44.jpg
#5 α=0.092
early
47.jpg
#6 α=0.102
early
55.jpg
#7 α=0.140
mid
55.jpg
#8 α=0.146
mid
55.jpg
#9 α=0.142
mid
55.jpg
#10 α=0.138
mid
55.jpg
#11 α=0.134
mid
FA frame

Softmax vs time

Softmax vs frame · T=12 0 1 2 3 4 5 6 7 8 9 10 11 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
01stainingFalse
14window defectTrue
24window defectTrue
34window defectTrue
44window defectTrue
54window defectTrue
64window defectTrue
74window defectTrue
84window defectTrue
94window defectTrue
104window defectTrue
114window defectTrue
#12

Train/Train/185_R · test

3 model frames (max 12) · frame_span=phase_stratified · P(true)=0.035 · prefix argmax flips=1 · min prefix margin=0.061

Ground truth 2 — no hyperfluorescence
Prediction (exam-level) 4 — window defect

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.108
mid
1.jpg
#1 α=0.228
mid
2.jpg
#2 α=0.664
late
FA frame

Softmax vs time

Softmax vs frame · T=3 0 1 2 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
01stainingFalse
14window defectTrue
24window defectTrue
#13

Train/Train/817_L · test

6 model frames (max 12) · frame_span=phase_stratified · P(true)=0.036 · prefix argmax flips=1 · min prefix margin=0.013

Ground truth 2 — no hyperfluorescence
Prediction (exam-level) 4 — window defect

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.170
early
1.jpg
#1 α=0.144
early
2.jpg
#2 α=0.145
early
3.jpg
#3 α=0.149
mid
4.jpg
#4 α=0.182
late
5.jpg
#5 α=0.210
late
FA frame

Softmax vs time

Softmax vs frame · T=6 0 1 2 3 4 5 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
01stainingFalse
11stainingFalse
24window defectTrue
34window defectTrue
44window defectTrue
54window defectTrue
#14

Train/Train/156_R · test

10 model frames (max 12) · frame_span=phase_stratified · P(true)=0.037 · prefix argmax flips=0 · min prefix margin=0.099

Ground truth 1 — staining
Prediction (exam-level) 3 — pooling

Softmax at selected frame

Model frames + attention α

12.jpg
#0 α=0.008
late
13.jpg
#1 α=0.014
late
14.jpg
#2 α=0.045
late
15.jpg
#3 α=0.070
late
16.jpg
#4 α=0.123
late
17.jpg
#5 α=0.181
late
18.jpg
#6 α=0.120
late
19.jpg
#7 α=0.141
late
20.jpg
#8 α=0.125
late
21.jpg
#9 α=0.170
late
FA frame

Softmax vs time

Softmax vs frame · T=10 0 1 2 3 4 5 6 7 8 9 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
03poolingTrue
13poolingTrue
23poolingTrue
33poolingTrue
43poolingTrue
53poolingTrue
63poolingTrue
73poolingTrue
83poolingTrue
93poolingTrue
#15

Train/Train/688_R · test

5 model frames (max 12) · frame_span=phase_stratified · P(true)=0.037 · prefix argmax flips=1 · min prefix margin=0.103

Ground truth 0 — leakage
Prediction (exam-level) 4 — window defect

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.216
early
1.jpg
#1 α=0.314
early
2.jpg
#2 α=0.283
early
56.jpg
#3 α=0.100
late
57.jpg
#4 α=0.087
late
FA frame

Softmax vs time

Softmax vs frame · T=5 0 1 2 3 4 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
04window defectTrue
14window defectTrue
24window defectTrue
34window defectTrue
42no hyperfluorescenceFalse
#16

Train/Train/481_L · test

4 model frames (max 12) · frame_span=phase_stratified · P(true)=0.045 · prefix argmax flips=2 · min prefix margin=0.001

Ground truth 2 — no hyperfluorescence
Prediction (exam-level) 1 — staining

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.151
early
1.jpg
#1 α=0.094
early
2.jpg
#2 α=0.143
mid
3.jpg
#3 α=0.612
late
FA frame

Softmax vs time

Softmax vs frame · T=4 0 1 2 3 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
01stainingTrue
12no hyperfluorescenceFalse
21stainingTrue
31stainingTrue
#17

Train/Train/927_L · test

11 model frames (max 12) · frame_span=phase_stratified · P(true)=0.052 · prefix argmax flips=1 · min prefix margin=0.245

Ground truth 0 — leakage
Prediction (exam-level) 4 — window defect

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.003
early
1.jpg
#1 α=0.002
early
2.jpg
#2 α=0.002
early
3.jpg
#3 α=0.008
early
4.jpg
#4 α=0.018
early
5.jpg
#5 α=0.029
mid
6.jpg
#6 α=0.065
mid
7.jpg
#7 α=0.133
late
8.jpg
#8 α=0.208
late
9.jpg
#9 α=0.265
late
10.jpg
#10 α=0.267
late
FA frame

Softmax vs time

Softmax vs frame · T=11 0 1 2 3 4 5 6 7 8 9 10 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
04window defectTrue
14window defectTrue
24window defectTrue
34window defectTrue
44window defectTrue
54window defectTrue
64window defectTrue
74window defectTrue
83poolingFalse
93poolingFalse
103poolingFalse
#18

Train/Train/497_R · test

12 model frames (max 12) · frame_span=phase_stratified · P(true)=0.052 · prefix argmax flips=1 · min prefix margin=0.058

Ground truth 3 — pooling
Prediction (exam-level) 0 — leakage

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.003
early
2.jpg
#1 α=0.003
early
4.jpg
#2 α=0.013
early
6.jpg
#3 α=0.015
early
7.jpg
#4 α=0.038
mid
11.jpg
#5 α=0.053
mid
16.jpg
#6 α=0.094
mid
20.jpg
#7 α=0.124
mid
20.jpg
#8 α=0.118
mid
21.jpg
#9 α=0.158
late
22.jpg
#10 α=0.201
late
23.jpg
#11 α=0.179
late
FA frame

Softmax vs time

Softmax vs frame · T=12 0 1 2 3 4 5 6 7 8 9 10 11 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
01stainingFalse
11stainingFalse
21stainingFalse
31stainingFalse
41stainingFalse
50leakageTrue
60leakageTrue
70leakageTrue
80leakageTrue
90leakageTrue
100leakageTrue
110leakageTrue
#19

Train/Train/655_L · test

7 model frames (max 12) · frame_span=phase_stratified · P(true)=0.053 · prefix argmax flips=1 · min prefix margin=0.064

Ground truth 3 — pooling
Prediction (exam-level) 1 — staining

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.008
early
1.jpg
#1 α=0.004
early
2.jpg
#2 α=0.057
mid
3.jpg
#3 α=0.235
mid
4.jpg
#4 α=0.182
mid
5.jpg
#5 α=0.271
late
6.jpg
#6 α=0.243
late
FA frame

Softmax vs time

Softmax vs frame · T=7 0 1 2 3 4 5 6 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
02no hyperfluorescenceFalse
12no hyperfluorescenceFalse
21stainingTrue
31stainingTrue
41stainingTrue
51stainingTrue
61stainingTrue
#20

Train/Train/303_R · test

3 model frames (max 12) · frame_span=phase_stratified · P(true)=0.053 · prefix argmax flips=1 · min prefix margin=0.291

Ground truth 1 — staining
Prediction (exam-level) 4 — window defect

Softmax at selected frame

Model frames + attention α

0.jpg
#0 α=0.078
mid
1.jpg
#1 α=0.271
late
2.jpg
#2 α=0.650
late
FA frame

Softmax vs time

Softmax vs frame · T=3 0 1 2 ● leakage ● staining ● no hyperfluoresc ● pooling ● window defect

Per-frame prefix argmax

targmaxlabelmatches exam-level pred
04window defectTrue
14window defectTrue
20leakageFalse