MedicalBenchmark
AllenAI: Olmo 3 7B Think provider

Olmo 3 7B Think

283

#283 of 319 models in the general ranking

Cumulative performance across 3 MIR exams

Net score

244.00 pts

Accuracy

55.2%

Correct / Incorrect

331 / 261

Total Cost

$0.41

Overall Performance

(vs. average)
Accuracy

55.2%

avg: 80.6%

Net score

244.00 pts

avg: 453.30 pts

Correct

331

avg: 483

Incorrect

261

avg: 90

Total Cost

$0.41

avg: $9.58

Average response time

15.5s

avg: 17.9s

Output Tokens

1.9M

avg: 1.3M

Reasoning Tokens

1.6M

avg: 898K

Average confidence

97.6%

avg: 95.4%

Breakdown by Exam

MIR 2024
282
Correct
112
Incorrect
84
Accuracy
56.0%
Net score
84.00
MIR 2025
287
Correct
99
Incorrect
98
Accuracy
49.5%
Net score
66.33
MIR 2026
283
Correct
120
Incorrect
79
Accuracy
60.0%
Net score
93.66