MedicalBenchmark
AlfredPros: CodeLLaMa 7B Instruct Solidity provider

CodeLLaMa 7B Instruct Solidity

319

#319 of 319 models — MIR 2026

Net score

0.00 pts

Accuracy

1.5%

Correct / Incorrect

3 / 21

Total Cost

$0.32

Overall Performance

(vs. average)
Accuracy

1.5%

avg: 82.4%

Net score

0.00 pts

avg: 156.14 pts

Correct

3

avg: 165

Incorrect

21

avg: 26

Total Cost

$0.32

avg: $3.12

Average response time

27.9s

avg: 17.4s

Output Tokens

185K

avg: 415K

Reasoning Tokens

0

avg: 295K

Average confidence

16.9%

avg: 95.4%

Subject Breakdown

Allergology
Correct
0
Incorrect
0
Unanswered
1
Accuracy
0.0%
Average
96.9%
Anesthesiology and Resuscitation
Correct
0
Incorrect
2
Unanswered
5
Accuracy
0.0%
Average
71.0%
Cardiology
Correct
0
Incorrect
2
Unanswered
23
Accuracy
0.0%
Average
78.4%
Dermatology
Correct
0
Incorrect
1
Unanswered
10
Accuracy
0.0%
Average
73.2%
Endocrinology and Nutrition
Correct
0
Incorrect
3
Unanswered
12
Accuracy
0.0%
Average
84.7%
ENT
Correct
0
Incorrect
0
Unanswered
8
Accuracy
0.0%
Average
85.0%
Epidemiology
Correct
0
Incorrect
2
Unanswered
5
Accuracy
0.0%
Average
81.3%
Gastroenterology
Correct
1
Incorrect
5
Unanswered
24
Accuracy
3.3%
Average
80.2%
Genetics
Correct
0
Incorrect
1
Unanswered
10
Accuracy
0.0%
Average
79.7%
Geriatrics
Correct
1
Incorrect
1
Unanswered
11
Accuracy
7.7%
Average
83.7%
Gynecology and Obstetrics
Correct
0
Incorrect
2
Unanswered
10
Accuracy
0.0%
Average
85.2%
Health Planning and Management
Correct
0
Incorrect
1
Unanswered
9
Accuracy
0.0%
Average
79.4%
Hematology
Correct
0
Incorrect
0
Unanswered
9
Accuracy
0.0%
Average
77.6%
Immunology
Correct
0
Incorrect
1
Unanswered
5
Accuracy
0.0%
Average
91.8%
Infectious Diseases
Correct
0
Incorrect
3
Unanswered
11
Accuracy
0.0%
Average
79.0%
Legal Medicine and Bioethics
Correct
0
Incorrect
0
Unanswered
11
Accuracy
0.0%
Average
83.4%
Medical Oncology
Correct
1
Incorrect
3
Unanswered
19
Accuracy
4.3%
Average
83.7%
Nephrology
Correct
0
Incorrect
1
Unanswered
9
Accuracy
0.0%
Average
85.6%
Neurology
Correct
1
Incorrect
2
Unanswered
10
Accuracy
7.7%
Average
89.0%
Ophthalmology
Correct
0
Incorrect
1
Unanswered
4
Accuracy
0.0%
Average
84.5%
Palliative Care
Correct
0
Incorrect
0
Unanswered
6
Accuracy
0.0%
Average
80.8%
Pediatrics
Correct
1
Incorrect
1
Unanswered
20
Accuracy
4.5%
Average
88.1%
Pharmacology
Correct
0
Incorrect
1
Unanswered
10
Accuracy
0.0%
Average
79.6%
Psychiatry
Correct
0
Incorrect
0
Unanswered
8
Accuracy
0.0%
Average
88.4%
Pulmonology
Correct
1
Incorrect
1
Unanswered
14
Accuracy
6.3%
Average
83.6%
Radiology-Emergency
Correct
0
Incorrect
2
Unanswered
11
Accuracy
0.0%
Average
69.4%
Rheumatology
Correct
0
Incorrect
1
Unanswered
10
Accuracy
0.0%
Average
89.0%
Statistics
Correct
0
Incorrect
0
Unanswered
2
Accuracy
0.0%
Average
84.5%
Traumatology
Correct
0
Incorrect
1
Unanswered
10
Accuracy
0.0%
Average
66.6%
Urology
Correct
0
Incorrect
1
Unanswered
7
Accuracy
0.0%
Average
83.1%

Question Type Breakdown

Anatomy
Correct
0
Incorrect
0
Unanswered
3
Accuracy
0.0%
Average
83.4%
Biostatistics
Correct
0
Incorrect
0
Unanswered
2
Accuracy
0.0%
Average
84.5%
Diagnosis
Correct
1
Incorrect
6
Unanswered
74
Accuracy
1.2%
Average
83.0%
Epidemiology
Correct
0
Incorrect
1
Unanswered
8
Accuracy
0.0%
Average
89.1%
Ethics
Correct
0
Incorrect
0
Unanswered
6
Accuracy
0.0%
Average
92.2%
Interpretation
Correct
0
Incorrect
3
Unanswered
34
Accuracy
0.0%
Average
73.2%
Legal
Correct
0
Incorrect
0
Unanswered
9
Accuracy
0.0%
Average
82.9%
Pathophysiology
Correct
0
Incorrect
4
Unanswered
22
Accuracy
0.0%
Average
85.0%
Pharmacology
Correct
0
Incorrect
2
Unanswered
13
Accuracy
0.0%
Average
83.1%
Prevention
Correct
0
Incorrect
2
Unanswered
14
Accuracy
0.0%
Average
81.5%
Prognosis
Correct
0
Incorrect
0
Unanswered
5
Accuracy
0.0%
Average
93.2%
Risk
Correct
0
Incorrect
3
Unanswered
12
Accuracy
0.0%
Average
85.0%
Tests
Correct
1
Incorrect
2
Unanswered
30
Accuracy
3.0%
Average
81.2%
Treatment
Correct
2
Incorrect
8
Unanswered
62
Accuracy
2.8%
Average
80.9%
#AnswerCorrectStatus
1—A
2—B
3AD
4—D
5—B
6—C
7—B
8—D
9—D
10—A
11—B
12—A
13——Annulled
14DC
15—A
16—C
17—C
18—D
19—B
20—B
21—B
22—C
23—B
24—A
25—C
26—A
27—B
28—D
29—A
30AD
31BC
32—C
33—B
34—B
35—C
36AD
37—D
38—A
39—C
40—A
41—D
42—A
43—B
44—B
45—B
46—B
47AD
48CC
49—C
50——Annulled
51CA
52—C
53—C
54DC
55—A
56—A
57—A
58—A
59—B
60—B
61—C
62—B
63—B
64C—Annulled
65—C
66—C
67—C
68—A
69AB
70AB
71—B
72—B
73—B
74—A
75—C
76—C
77—A
78—D
79—A
80—D
81—C
82—D
83—B
84—A
85—D
86—A
87—B
88AB
89—B
90—C
91—A
92—C
93—D
94—C
95—C
96—B
97—B
98—A
99DB
100—C
101—C
102—A
103—C
104—C
105—C
106—B
107—C
108—A
109—C
110—C
111—B
112—C
113—C
114—D
115—B
116AD
117—C
118—B
119—A
120—A
121—C
122—B
123—B
124—B
125—C
126AB
127—C
128—B
129—C
130—C
131AB
132—B
133—C
134—B
135AC
136—B
137—D
138—B
139——Annulled
140—D
141—A
142——Annulled
143—A
144—D
145—C
146—C
147—B
148BC
149—B
150—B
151—B
152—C
153—C
154—D
155—B
156—A
157—D
158—B
159—C
160—A
161——Annulled
162—A
163—C
164—C
165AA
166—A
167BD
168—C
169—B
170—B
171—A
172—B
173—D
174—C
175—B
176—A
177—D
178—D
179—B
180—C
181—B
182—C
183—D
184—D
185—B
186AD
187CD
188—C
189—C
190—B
191—D
192—D
193—C
194—C
195—D
196—A
197—C
198—A
199CB
200—D
201—C
202—A
203—A
204—C
205—B
206CC
207—A
208——Annulled
209—B
210—C