Medical annotation: labeling de-identified clinical texts
12,000 clinical text fragments labeled for a medical documentation tool, with a 500-fragment pilot, dual annotation on 1,200 fragments and 96.4% label agreement before resolution.
- Client
- A European team building a medical documentation processing tool
- Period
- Q2 2026
- Volume
- 12,000 text fragments
67-year-old female presents with persistent cough and dyspnea for five days. History of type 2 diabetes and hypertension. Started metformin 500 mg twice daily and amoxicillin 500 mg three times daily for suspected pneumonia. Reports mild chest pain; no fever at triage. Referred to pulmonology.
1,200 double-annotated fragments
Client secure environment
Challenge
The client needed a labeled text set to train a model that extracts diagnoses, symptoms and medication orders. The original guideline interpreted abbreviations, negations and mentions of past conditions ambiguously. The handling of these cases had to be agreed before main annotation began.
Solution
HUBBLE GROUP ran a pilot annotation of 500 fragments and refined the guideline together with the client. Once the rules were agreed, the team annotated the main set.
For quality control, 1,200 fragments were independently processed by two annotators. A QA specialist reviewed disagreements; questions requiring medical interpretation were passed to the client's designated expert. All work was done on texts de-identified by the client, inside the client's secure environment.
Result
All 12,000 fragments were delivered within three weeks in the agreed JSONL format.
On the independently double-annotated sample, 96.4% of fragments had fully matching label sets before disagreements were resolved. The remaining 3.6% went through additional review before inclusion in the final set.
The client accepted the delivery after reviewing an agreed sample of 600 fragments.
Basis of results
Dual-annotation report, discrepancy resolution log and acceptance confirmation. Annotator agreement is not an assessment of the model's clinical accuracy.
About confidentiality
The client's name and the source medical texts are not published. The process, volume and quality figures were agreed with the client.
Ready to get started?
Tell us about your workload and receive a tailored proposal. We reply within one business day.
Request a Quote