MIS 752 · Lab 12 · richardyoung

The Physical Exam, voice: an audited ambient-scribe draft of a synthetic visit

One synthetic clinic visit, spoken by a local voice model, transcribed by Whisper, written up by a local language model, and audited for what was misheard and what was invented. $0, no PHI. These numbers are this run's.

The 30-second read

THE VISIT 21 turns, 237 words, 90 s of audio written for this lab (synthetic, invented patient), spoken by speecht5 LISTEN AND TRANSCRIBE (openai/whisper-base.en, cuda) word error rate, clean audio : 2.1% critical terms heard : 18 of 20 missed: a1c, lisinopril ORGANISE (the note writer: qwen2.5:3b-instruct) drafted from : Whisper's transcript invented : 2 carried mishearing : none lucky fill-in : lisinopril small talk that leaked : baseball

Listen to the visit

Synthetic audio made from a written script by speecht5. No real patient, no real recording.

The draft note contains 1 fact(s) nobody said

The draft, exactly as the model wrote it from Whisper's transcript. Highlighted: numbers and clinical terms the visit never contained. It is unsigned.

SUBJECTIVE - Patient has been experiencing tingling in her feet for 3 weeks and feels tired all the time. - Patient stopped taking glipizide due to shakiness, which was resolved with Metformin. - Patient's IAC is 8.1, higher than last time. - Patient has reduced sensation in both feet with a monofilament test. - Patient's blood pressure is 148/92 mmHg. - Patient has a history of diabetes and sulfa drug allergy. - Patient has been on her feet a lot due to her grandson's baseball team winning. OBJECTIVE - Patient's blood pressure is 148/92 mmHg. - Patient's IAC is 8.1. - Patient has reduced sensation in both feet with a monofilament test. - Patient's foot pulses are normal. ASSESSMENT - Diabetic neuropathy in the feet. - Uncontrolled blood pressure. - Diabetes mellitus, type 2. PLAN - Increase lisinopril to 20 mg daily. - Continue Metformin. - Refer to Podiatry. - Order kidney function panel and urine albumin test. - Patient should check blood sugar every morning and report any readings below 70. - Follow-up in 4 weeks.

The sign-off checklist

Automated heuristics on words. The real sign-off is a clinician reading the note.

check result how_sure chest pain recorded as ABSENT not mentioned heuristic glipizide recorded as STOPPED ok heuristic sulfa allergy recorded ok exact word clinic measurements under OBJECTIVE, not SUBJECTIVE MIS-FILED: 148, 92, 8.1 under Subjective exact number lisinopril 20 mg in the PLAN ok exact number small talk left out LEAKED: baseball exact word follow-up in 4 weeks ok exact words

What background noise does to the ear

White noise added at each signal-to-noise ratio; two sizes of Whisper. Critical terms are the words whose mishearing changes care.

model noise wer subs dels ins critical_missed missed_terms seconds whisper-base.en clean 2.1% 5 0 0 2 a1c, lisinopril 0.0 whisper-base.en 20 dB 2.1% 4 0 1 2 a1c, lisinopril 3.9 whisper-base.en 10 dB 5.1% 8 0 4 4 a1c, monofilament, lisinopril, podiatry 3.9 whisper-base.en 5 dB 8.9% 14 1 6 7 metformin, glipizide, sulfa, a1c, monofilament, lisinopril, podiatry 6.7 whisper-base.en 0 dB 17.3% 25 1 15 10 metformin, glipizide, 62, sulfa, a1c, monofilament, lisinopril, kidney, albumin, podiatry 4.4 whisper-tiny.en clean 3.4% 8 0 0 5 glipizide, sulfa, a1c, lisinopril, podiatry 2.3 whisper-tiny.en 20 dB 4.2% 8 0 2 4 glipizide, sulfa, a1c, lisinopril 2.2 whisper-tiny.en 10 dB 6.8% 11 1 4 6 glipizide, sulfa, a1c, monofilament, lisinopril, podiatry 2.2 whisper-tiny.en 5 dB 12.7% 19 2 9 8 metformin, glipizide, sulfa, a1c, monofilament, lisinopril, albumin, podiatry 2.3 whisper-tiny.en 0 dB 20.7% 31 4 14 10 metformin, glipizide, 62, sulfa, a1c, monofilament, lisinopril, kidney, albumin, podiatry 2.4

The note, re-drafted from what was heard at 5 dB

SUBJECTIVE - feels tired all the time - feels shaky - has tingling in feet for 3 weeks - stopped Metformin last month due to shakiness - last Metformin dose 500 mg twice daily - ISG 8.1 - blood pressure 148/92 - reduced sensation in both feet - normal foot pulses - congratulations on grandson's baseball team win - ISG higher than last time - feels dizzy on stairs OBJECTIVE - blood pressure 148/92 - ISG 8.1 - reduced sensation in both feet - normal foot pulses ASSESSMENT - Diabetic neuropathy in both feet - Uncontrolled blood pressure - ISG 8.1 - Normal foot pulses PLAN - Increase Lisinopril to 20 mg daily - Order kidney function panel and urine albumin test - Refer to Prodigy - Check blood sugar every morning, report anything under 70 - Follow-up in 4 weeks invented: none (in neither the transcript nor the visit: written from habit) carried mishearing: none (heard but never said: Whisper's mistake, copied faithfully) lucky fill-in: metformin, lisinopril (said but never heard: a right guess with nothing to go on)

How this was measured

A 21-turn clinic visit was written for this lab: an invented patient modelled on the course's representative patient (68, type 2 diabetes, hypertension, chronic kidney disease). A local text-to-speech model (speecht5) spoke each turn; Whisper (openai/whisper-base.en) transcribed each turn separately, so who said what is known rather than inferred. Word error rate is (substitutions + deletions + insertions) / words said, after both sides are normalised (lower case, no punctuation, number words as digits). A local language model (qwen2.5:3b-instruct, temperature 0) drafted a SOAP note in JSON with instructions to use only what was said. A fact is unsupported when a number or a listed clinical term in the note does not appear in the transcript. Noise is white noise at a fixed seed. Everything ran on free Google Colab hardware at $0. This is a teaching pilot on one synthetic visit, not a clinical evaluation, and no note here was, or should be, signed.

Credits

Speech synthesis: SpeechT5 and its HiFi-GAN vocoder by Microsoft (MIT), with CMU ARCTIC voice prints (MIT), or espeak-ng. Speech recognition: OpenAI Whisper (open weights). Note drafting: qwen2.5:3b-instruct via ollama. See each model card for its licence.