MedicalBenchmark
AlfredPros: CodeLLaMa 7B Instruct Solidity provider

CodeLLaMa 7B Instruct Solidity

290

#290 of 290 models — MIR 2026

Net score

0.00 pts

Accuracy

1.5%

Correct / Incorrect

3 / 21

Total Cost

$0.32

Overall Performance

(vs. average)
Accuracy

1.5%

avg: 81.6%

Net score

0.00 pts

avg: 154.00 pts

Correct

3

avg: 163

Incorrect

21

avg: 28

Total Cost

$0.32

avg: $3.33

Average response time

27.9s

avg: 16.2s

Output Tokens

185K

avg: 430K

Reasoning Tokens

0

avg: 310K

Average confidence

16.5%

avg: 95.1%

Subject Breakdown

Allergology
Correct
0
Incorrect
0
Unanswered
1
Accuracy
0.0%
Average
96.9%
Anesthesiology and Resuscitation
Correct
0
Incorrect
2
Unanswered
5
Accuracy
0.0%
Average
69.6%
Cardiology
Correct
0
Incorrect
2
Unanswered
23
Accuracy
0.0%
Average
77.3%
Dermatology
Correct
0
Incorrect
1
Unanswered
10
Accuracy
0.0%
Average
72.3%
Endocrinology and Nutrition
Correct
0
Incorrect
3
Unanswered
12
Accuracy
0.0%
Average
84.0%
ENT
Correct
0
Incorrect
0
Unanswered
8
Accuracy
0.0%
Average
84.7%
Epidemiology
Correct
0
Incorrect
2
Unanswered
5
Accuracy
0.0%
Average
80.2%
Gastroenterology
Correct
1
Incorrect
5
Unanswered
24
Accuracy
3.3%
Average
79.3%
Genetics
Correct
0
Incorrect
1
Unanswered
10
Accuracy
0.0%
Average
78.7%
Geriatrics
Correct
1
Incorrect
1
Unanswered
11
Accuracy
7.7%
Average
83.0%
Gynecology and Obstetrics
Correct
0
Incorrect
2
Unanswered
10
Accuracy
0.0%
Average
84.3%
Health Planning and Management
Correct
0
Incorrect
1
Unanswered
9
Accuracy
0.0%
Average
78.4%
Hematology
Correct
0
Incorrect
0
Unanswered
9
Accuracy
0.0%
Average
76.6%
Immunology
Correct
0
Incorrect
1
Unanswered
5
Accuracy
0.0%
Average
91.4%
Infectious Diseases
Correct
0
Incorrect
3
Unanswered
11
Accuracy
0.0%
Average
77.9%
Legal Medicine and Bioethics
Correct
0
Incorrect
0
Unanswered
11
Accuracy
0.0%
Average
82.9%
Medical Oncology
Correct
1
Incorrect
3
Unanswered
19
Accuracy
4.3%
Average
83.0%
Nephrology
Correct
0
Incorrect
1
Unanswered
9
Accuracy
0.0%
Average
85.1%
Neurology
Correct
1
Incorrect
2
Unanswered
10
Accuracy
7.7%
Average
88.6%
Ophthalmology
Correct
0
Incorrect
1
Unanswered
4
Accuracy
0.0%
Average
83.7%
Palliative Care
Correct
0
Incorrect
0
Unanswered
6
Accuracy
0.0%
Average
80.2%
Pediatrics
Correct
1
Incorrect
1
Unanswered
20
Accuracy
4.5%
Average
87.6%
Pharmacology
Correct
0
Incorrect
1
Unanswered
10
Accuracy
0.0%
Average
78.6%
Psychiatry
Correct
0
Incorrect
0
Unanswered
8
Accuracy
0.0%
Average
87.9%
Pulmonology
Correct
1
Incorrect
1
Unanswered
14
Accuracy
6.3%
Average
82.8%
Radiology-Emergency
Correct
0
Incorrect
2
Unanswered
11
Accuracy
0.0%
Average
67.7%
Rheumatology
Correct
0
Incorrect
1
Unanswered
10
Accuracy
0.0%
Average
88.4%
Statistics
Correct
0
Incorrect
0
Unanswered
2
Accuracy
0.0%
Average
83.8%
Traumatology
Correct
0
Incorrect
1
Unanswered
10
Accuracy
0.0%
Average
65.2%
Urology
Correct
0
Incorrect
1
Unanswered
7
Accuracy
0.0%
Average
82.5%

Question Type Breakdown

Anatomy
Correct
0
Incorrect
0
Unanswered
3
Accuracy
0.0%
Average
82.6%
Biostatistics
Correct
0
Incorrect
0
Unanswered
2
Accuracy
0.0%
Average
83.8%
Diagnosis
Correct
1
Incorrect
6
Unanswered
74
Accuracy
1.2%
Average
82.2%
Epidemiology
Correct
0
Incorrect
1
Unanswered
8
Accuracy
0.0%
Average
88.7%
Ethics
Correct
0
Incorrect
0
Unanswered
6
Accuracy
0.0%
Average
92.0%
Interpretation
Correct
0
Incorrect
3
Unanswered
34
Accuracy
0.0%
Average
72.0%
Legal
Correct
0
Incorrect
0
Unanswered
9
Accuracy
0.0%
Average
82.4%
Pathophysiology
Correct
0
Incorrect
4
Unanswered
22
Accuracy
0.0%
Average
84.3%
Pharmacology
Correct
0
Incorrect
2
Unanswered
13
Accuracy
0.0%
Average
82.3%
Prevention
Correct
0
Incorrect
2
Unanswered
14
Accuracy
0.0%
Average
80.6%
Prognosis
Correct
0
Incorrect
0
Unanswered
5
Accuracy
0.0%
Average
93.2%
Risk
Correct
0
Incorrect
3
Unanswered
12
Accuracy
0.0%
Average
84.3%
Tests
Correct
1
Incorrect
2
Unanswered
30
Accuracy
3.0%
Average
80.3%
Treatment
Correct
2
Incorrect
8
Unanswered
62
Accuracy
2.8%
Average
80.1%
#AnswerCorrectStatus
1—A
2—B
3AD
4—D
5—B
6—C
7—B
8—D
9—D
10—A
11—B
12—A
13——Annulled
14DC
15—A
16—C
17—C
18—D
19—B
20—B
21—B
22—C
23—B
24—A
25—C
26—A
27—B
28—D
29—A
30AD
31BC
32—C
33—B
34—B
35—C
36AD
37—D
38—A
39—C
40—A
41—D
42—A
43—B
44—B
45—B
46—B
47AD
48CC
49—C
50——Annulled
51CA
52—C
53—C
54DC
55—A
56—A
57—A
58—A
59—B
60—B
61—C
62—B
63—B
64C—Annulled
65—C
66—C
67—C
68—A
69AB
70AB
71—B
72—B
73—B
74—A
75—C
76—C
77—A
78—D
79—A
80—D
81—C
82—D
83—B
84—A
85—D
86—A
87—B
88AB
89—B
90—C
91—A
92—C
93—D
94—C
95—C
96—B
97—B
98—A
99DB
100—C
101—C
102—A
103—C
104—C
105—C
106—B
107—C
108—A
109—C
110—C
111—B
112—C
113—C
114—D
115—B
116AD
117—C
118—B
119—A
120—A
121—C
122—B
123—B
124—B
125—C
126AB
127—C
128—B
129—C
130—C
131AB
132—B
133—C
134—B
135AC
136—B
137—D
138—B
139——Annulled
140—D
141—A
142——Annulled
143—A
144—D
145—C
146—C
147—B
148BC
149—B
150—B
151—B
152—C
153—C
154—D
155—B
156—A
157—D
158—B
159—C
160—A
161——Annulled
162—A
163—C
164—C
165AA
166—A
167BD
168—C
169—B
170—B
171—A
172—B
173—D
174—C
175—B
176—A
177—D
178—D
179—B
180—C
181—B
182—C
183—D
184—D
185—B
186AD
187CD
188—C
189—C
190—B
191—D
192—D
193—C
194—C
195—D
196—A
197—C
198—A
199CB
200—D
201—C
202—A
203—A
204—C
205—B
206CC
207—A
208——Annulled
209—B
210—C