MedicalBenchmark
AllenAI: Olmo 3 7B Instruct provider

Olmo 3 7B Instruct

309

#309 of 319 models in the general ranking

Cumulative performance across 3 MIR exams

Net score

105.66 pts

Accuracy

36.7%

Correct / Incorrect

220 / 343

Total Cost

$0.14

Overall Performance

(vs. average)
Accuracy

36.7%

avg: 80.6%

Net score

105.66 pts

avg: 453.30 pts

Correct

220

avg: 483

Incorrect

343

avg: 90

Total Cost

$0.14

avg: $9.58

Average response time

18.2s

avg: 17.9s

Output Tokens

536K

avg: 1.3M

Reasoning Tokens

0

avg: 898K

Average confidence

85.6%

avg: 95.4%

Breakdown by Exam

MIR 2024
313
Correct
67
Incorrect
124
Accuracy
33.5%
Net score
25.66
MIR 2025
308
Correct
65
Incorrect
124
Accuracy
32.5%
Net score
23.66
MIR 2026
304
Correct
88
Incorrect
95
Accuracy
44.0%
Net score
56.33