MedicalBenchmark
AlfredPros: CodeLLaMa 7B Instruct Solidity provider

CodeLLaMa 7B Instruct Solidity

319

#319 of 319 models — MIR 2025

Net score

0.00 pts

Accuracy

3.5%

Correct / Incorrect

7 / 22

Total Cost

$0.37

Overall Performance

(vs. average)
Accuracy

3.5%

avg: 77.9%

Net score

0.00 pts

avg: 143.96 pts

Correct

7

avg: 156

Incorrect

22

avg: 35

Total Cost

$0.37

avg: $3.36

Average response time

33.4s

avg: 19.0s

Output Tokens

226K

avg: 430K

Reasoning Tokens

0

avg: 306K

Average confidence

16.4%

avg: 95.2%

Subject Breakdown

Allergology
Correct
0
Incorrect
0
Unanswered
4
Accuracy
0.0%
Average
87.9%
Anesthesiology and Resuscitation
Correct
0
Incorrect
0
Unanswered
6
Accuracy
0.0%
Average
82.3%
Cardiology
Correct
0
Incorrect
4
Unanswered
18
Accuracy
0.0%
Average
78.6%
Dermatology
Correct
0
Incorrect
0
Unanswered
12
Accuracy
0.0%
Average
69.4%
Endocrinology and Nutrition
Correct
1
Incorrect
3
Unanswered
12
Accuracy
6.3%
Average
83.5%
ENT
Correct
0
Incorrect
1
Unanswered
7
Accuracy
0.0%
Average
74.8%
Epidemiology
Correct
1
Incorrect
0
Unanswered
6
Accuracy
14.3%
Average
69.1%
Gastroenterology
Correct
0
Incorrect
3
Unanswered
18
Accuracy
0.0%
Average
74.1%
Genetics
Correct
0
Incorrect
0
Unanswered
6
Accuracy
0.0%
Average
69.5%
Geriatrics
Correct
1
Incorrect
0
Unanswered
10
Accuracy
9.1%
Average
77.5%
Gynecology and Obstetrics
Correct
1
Incorrect
3
Unanswered
15
Accuracy
5.3%
Average
86.7%
Health Planning and Management
Correct
0
Incorrect
0
Unanswered
2
Accuracy
0.0%
Average
82.6%
Hematology
Correct
0
Incorrect
1
Unanswered
10
Accuracy
0.0%
Average
82.7%
Immunology
Correct
1
Incorrect
0
Unanswered
8
Accuracy
11.1%
Average
83.3%
Infectious Diseases
Correct
0
Incorrect
4
Unanswered
23
Accuracy
0.0%
Average
74.9%
Legal Medicine and Bioethics
Correct
1
Incorrect
0
Unanswered
4
Accuracy
20.0%
Average
68.4%
Medical Oncology
Correct
2
Incorrect
2
Unanswered
21
Accuracy
8.0%
Average
87.2%
Nephrology
Correct
1
Incorrect
4
Unanswered
9
Accuracy
7.1%
Average
84.8%
Neurology
Correct
0
Incorrect
2
Unanswered
18
Accuracy
0.0%
Average
77.3%
Ophthalmology
Correct
0
Incorrect
0
Unanswered
5
Accuracy
0.0%
Average
74.2%
Palliative Care
Correct
2
Incorrect
0
Unanswered
2
Accuracy
50.0%
Average
78.6%
Pediatrics
Correct
1
Incorrect
6
Unanswered
19
Accuracy
3.8%
Average
71.9%
Pharmacology
Correct
3
Incorrect
1
Unanswered
13
Accuracy
17.6%
Average
74.1%
Psychiatry
Correct
0
Incorrect
0
Unanswered
8
Accuracy
0.0%
Average
83.0%
Pulmonology
Correct
0
Incorrect
0
Unanswered
14
Accuracy
0.0%
Average
80.4%
Radiology-Emergency
Correct
0
Incorrect
2
Unanswered
12
Accuracy
0.0%
Average
69.4%
Rheumatology
Correct
0
Incorrect
2
Unanswered
13
Accuracy
0.0%
Average
76.6%
Statistics
Correct
1
Incorrect
0
Unanswered
2
Accuracy
33.3%
Average
76.6%
Traumatology
Correct
1
Incorrect
1
Unanswered
16
Accuracy
5.6%
Average
79.3%
Urology
Correct
0
Incorrect
0
Unanswered
7
Accuracy
0.0%
Average
80.7%

Question Type Breakdown

Anatomy
Correct
0
Incorrect
0
Unanswered
7
Accuracy
0.0%
Average
78.6%
Biostatistics
Correct
1
Incorrect
0
Unanswered
3
Accuracy
25.0%
Average
79.8%
Diagnosis
Correct
3
Incorrect
12
Unanswered
73
Accuracy
3.4%
Average
79.9%
Epidemiology
Correct
0
Incorrect
1
Unanswered
4
Accuracy
0.0%
Average
76.7%
Ethics
Correct
1
Incorrect
0
Unanswered
2
Accuracy
33.3%
Average
74.1%
Interpretation
Correct
2
Incorrect
8
Unanswered
32
Accuracy
4.8%
Average
70.7%
Legal
Correct
0
Incorrect
0
Unanswered
4
Accuracy
0.0%
Average
64.6%
Pathophysiology
Correct
0
Incorrect
4
Unanswered
23
Accuracy
0.0%
Average
76.1%
Pharmacology
Correct
2
Incorrect
1
Unanswered
10
Accuracy
15.4%
Average
83.3%
Prevention
Correct
0
Incorrect
0
Unanswered
12
Accuracy
0.0%
Average
75.6%
Prognosis
Correct
0
Incorrect
0
Unanswered
7
Accuracy
0.0%
Average
80.8%
Risk
Correct
0
Incorrect
1
Unanswered
4
Accuracy
0.0%
Average
85.2%
Tests
Correct
0
Incorrect
2
Unanswered
25
Accuracy
0.0%
Average
77.9%
Treatment
Correct
3
Incorrect
8
Unanswered
70
Accuracy
3.7%
Average
77.3%
#AnswerCorrectStatus
1—B
2—A
3—C
4AB
5DA
6AC
7—C
8AA
9—A
10—D
11—D
12—D
13DB
14—D
15——Annulled
16—B
17—B
18—A
19—C
20CA
21—B
22—D
23—C
24—D
25—C
26——Annulled
27DC
28——Annulled
29—D
30—B
31—D
32—A
33—D
34—D
35—B
36—D
37—C
38—C
39—D
40—A
41—D
42—C
43—B
44—D
45DD
46—A
47—A
48—A
49—D
50—B
51—C
52—B
53—D
54—B
55—A
56——Annulled
57—C
58—B
59—D
60AA
61—A
62BD
63—B
64—D
65—A
66—A
67AB
68AB
69—B
70BA
71—D
72—A
73—D
74—C
75—A
76—B
77—B
78—B
79—C
80—C
81—C
82—D
83—B
84—D
85—C
86—C
87—A
88BD
89—B
90—A
91—B
92—C
93—B
94AC
95—A
96—C
97—D
98AC
99—A
100—C
101DB
102AD
103—A
104—C
105—A
106—C
107—B
108—D
109—B
110—C
111—A
112—C
113—B
114—D
115—D
116—C
117—A
118—D
119DC
120—B
121—D
122—C
123—C
124—C
125AD
126—D
127—B
128—D
129—A
130—D
131—D
132DA
133—B
134AC
135—B
136—C
137—A
138—D
139—D
140—B
141AA
142—A
143—B
144—B
145—D
146—C
147—B
148—A
149—A
150—D
151—A
152—A
153—B
154DB
155—B
156—C
157AA
158—C
159—C
160—A
161—A
162——Annulled
163—D
164—C
165—A
166AB
167—C
168—D
169—B
170—B
171—C
172—A
173—A
174—B
175—B
176—C
177—C
178—A
179—D
180—A
181—B
182CC
183—B
184—B
185—B
186——Annulled
187—C
188—D
189DD
190—A
191—B
192—A
193—C
194—A
195—A
196—A
197—B
198—C
199—D
200—C
201DB
202—A
203—D
204—C
205—B
206—D
207—A
208—C
209—C
210—B