MedicalBenchmark
NeverSleep: Lumimaid v0.2 8B provider

Lumimaid v0.2 8B

280

#280 of 290 modelsMIR 2026

Net score

50.33 pts

Accuracy

43.0%

Correct / Incorrect

86 / 107

Total Cost

$0.05

Overall Performance

(vs. average)
Accuracy

43.0%

avg: 81.6%

Net score

50.33 pts

avg: 154.00 pts

Correct

86

avg: 163

Incorrect

107

avg: 28

Total Cost

$0.05

avg: $3.33

Average response time

4.1s

avg: 16.2s

Output Tokens

61K

avg: 430K

Reasoning Tokens

0

avg: 310K

Average confidence

95.6%

avg: 95.1%

Subject Breakdown

Allergology
Correct
0
Incorrect
1
Unanswered
0
Accuracy
0.0%
Average
96.9%
Anesthesiology and Resuscitation
Correct
3
Incorrect
4
Unanswered
0
Accuracy
42.9%
Average
69.6%
Cardiology
Correct
12
Incorrect
13
Unanswered
0
Accuracy
48.0%
Average
77.3%
Dermatology
Correct
3
Incorrect
7
Unanswered
1
Accuracy
27.3%
Average
72.3%
Endocrinology and Nutrition
Correct
8
Incorrect
7
Unanswered
0
Accuracy
53.3%
Average
84.0%
ENT
Correct
4
Incorrect
4
Unanswered
0
Accuracy
50.0%
Average
84.7%
Epidemiology
Correct
2
Incorrect
5
Unanswered
0
Accuracy
28.6%
Average
80.2%
Gastroenterology
Correct
16
Incorrect
13
Unanswered
1
Accuracy
53.3%
Average
79.3%
Genetics
Correct
5
Incorrect
5
Unanswered
1
Accuracy
45.5%
Average
78.7%
Geriatrics
Correct
2
Incorrect
11
Unanswered
0
Accuracy
15.4%
Average
83.0%
Gynecology and Obstetrics
Correct
7
Incorrect
5
Unanswered
0
Accuracy
58.3%
Average
84.3%
Health Planning and Management
Correct
3
Incorrect
6
Unanswered
1
Accuracy
30.0%
Average
78.4%
Hematology
Correct
3
Incorrect
6
Unanswered
0
Accuracy
33.3%
Average
76.6%
Immunology
Correct
4
Incorrect
2
Unanswered
0
Accuracy
66.7%
Average
91.4%
Infectious Diseases
Correct
4
Incorrect
10
Unanswered
0
Accuracy
28.6%
Average
77.9%
Legal Medicine and Bioethics
Correct
7
Incorrect
3
Unanswered
1
Accuracy
63.6%
Average
82.9%
Medical Oncology
Correct
11
Incorrect
11
Unanswered
1
Accuracy
47.8%
Average
83.0%
Nephrology
Correct
5
Incorrect
5
Unanswered
0
Accuracy
50.0%
Average
85.1%
Neurology
Correct
6
Incorrect
6
Unanswered
1
Accuracy
46.2%
Average
88.6%
Ophthalmology
Correct
2
Incorrect
2
Unanswered
1
Accuracy
40.0%
Average
83.7%
Palliative Care
Correct
4
Incorrect
2
Unanswered
0
Accuracy
66.7%
Average
80.2%
Pediatrics
Correct
8
Incorrect
13
Unanswered
1
Accuracy
36.4%
Average
87.6%
Pharmacology
Correct
4
Incorrect
7
Unanswered
0
Accuracy
36.4%
Average
78.6%
Psychiatry
Correct
3
Incorrect
5
Unanswered
0
Accuracy
37.5%
Average
87.9%
Pulmonology
Correct
5
Incorrect
10
Unanswered
1
Accuracy
31.3%
Average
82.8%
Radiology-Emergency
Correct
8
Incorrect
5
Unanswered
0
Accuracy
61.5%
Average
67.7%
Rheumatology
Correct
6
Incorrect
5
Unanswered
0
Accuracy
54.5%
Average
88.4%
Statistics
Correct
1
Incorrect
1
Unanswered
0
Accuracy
50.0%
Average
83.8%
Traumatology
Correct
1
Incorrect
8
Unanswered
2
Accuracy
9.1%
Average
65.2%
Urology
Correct
3
Incorrect
5
Unanswered
0
Accuracy
37.5%
Average
82.5%

Question Type Breakdown

Anatomy
Correct
1
Incorrect
1
Unanswered
1
Accuracy
33.3%
Average
82.6%
Biostatistics
Correct
1
Incorrect
1
Unanswered
0
Accuracy
50.0%
Average
83.8%
Diagnosis
Correct
37
Incorrect
41
Unanswered
3
Accuracy
45.7%
Average
82.2%
Epidemiology
Correct
4
Incorrect
5
Unanswered
0
Accuracy
44.4%
Average
88.7%
Ethics
Correct
5
Incorrect
1
Unanswered
0
Accuracy
83.3%
Average
92.0%
Interpretation
Correct
16
Incorrect
20
Unanswered
1
Accuracy
43.2%
Average
72.0%
Legal
Correct
6
Incorrect
2
Unanswered
1
Accuracy
66.7%
Average
82.4%
Pathophysiology
Correct
13
Incorrect
13
Unanswered
0
Accuracy
50.0%
Average
84.3%
Pharmacology
Correct
5
Incorrect
9
Unanswered
1
Accuracy
33.3%
Average
82.3%
Prevention
Correct
3
Incorrect
13
Unanswered
0
Accuracy
18.8%
Average
80.6%
Prognosis
Correct
2
Incorrect
3
Unanswered
0
Accuracy
40.0%
Average
93.2%
Risk
Correct
5
Incorrect
10
Unanswered
0
Accuracy
33.3%
Average
84.3%
Tests
Correct
10
Incorrect
22
Unanswered
1
Accuracy
30.3%
Average
80.3%
Treatment
Correct
28
Incorrect
41
Unanswered
3
Accuracy
38.9%
Average
80.1%
#AnswerCorrectStatus
1BA
2AB
3AD
4CD
5BB
6CC
7B
8AD
9BD
10AA
11AB
12CA
13CAnnulled
14CC
15AA
16CC
17AC
18CD
19DB
20BB
21CB
22CC
23BB
24DA
25CC
26BA
27AB
28DD
29AA
30CD
31AC
32BC
33DB
34CB
35BC
36CD
37DD
38DA
39BC
40DA
41CD
42AA
43DB
44BB
45CB
46BB
47DD
48DC
49DC
50CAnnulled
51AA
52CC
53CC
54BC
55CA
56CA
57AA
58AA
59BB
60CB
61DC
62BB
63DB
64AAnnulled
65CC
66DC
67CC
68BA
69CB
70BB
71CB
72BB
73BB
74AA
75CC
76CC
77CA
78DD
79DA
80AD
81CC
82DD
83DB
84DA
85DD
86BA
87AB
88DB
89AB
90AC
91A
92C
93DD
94BC
95AC
96AB
97CB
98AA
99BB
100CC
101DC
102AA
103CC
104AC
105CC
106CB
107CC
108CA
109AC
110DC
111AB
112BC
113BC
114DD
115BB
116CD
117BC
118DB
119AA
120DA
121AC
122BB
123DB
124BB
125AC
126AB
127CC
128BB
129CC
130CC
131BB
132CB
133C
134CB
135CC
136CB
137DD
138CB
139BAnnulled
140BD
141BA
142BAnnulled
143AA
144CD
145C
146BC
147DB
148CC
149DB
150B
151BB
152CC
153BC
154DD
155BB
156DA
157D
158AB
159BC
160AA
161CAnnulled
162AA
163AC
164AC
165BA
166BA
167DD
168CC
169BB
170AB
171AA
172BB
173DD
174AC
175CB
176BA
177AD
178BD
179CB
180CC
181BB
182CC
183AD
184DD
185DB
186DD
187DD
188BC
189AC
190BB
191DD
192BD
193CC
194BC
195DD
196AA
197DC
198CA
199BB
200DD
201AC
202AA
203AA
204BC
205BB
206AC
207AA
208AAnnulled
209CB
210DC