Somewhere tonight, while you sleep, software is reading medical records. Not metaphorically. Millions of clinical notes, the intimate shorthand of examinations and diagnoses and bad news delivered gently, are being parsed by algorithms that never tire, looking for patterns of illness that translate into numbers that translate into money.
This is not a dystopian premise. It is the routine machinery of modern American healthcare, where artificial intelligence now helps determine how sick tens of millions of people officially are, and therefore how billions of dollars flow. The interesting question is not whether machines should read our charts. That ship has sailed, docked, and been repainted. The question is what we are owed by the people who run the machines.
What the machines are doing in there
The system works like this. Insurers covering older Americans are paid according to risk scores built from members’ documented diagnoses. Sicker populations, larger payments, which is fair when the documentation is true. AI entered because the documentation lives in unstructured clinical notes, years of them per patient, and machines read faster than humans by orders of magnitude.
So the machines read. They surface conditions doctors documented but no one coded. They flag, at least the well-built ones do, conditions that were coded but never properly documented. Their conclusions feed the scores; the scores feed the payments; the payments, in aggregate, are large enough that federal auditors now employ two thousand coders to check the outputs, and settlements over bad records run to nine figures. Reviews published this spring found 81 to 91 percent of certain sampled diagnosis codes unsupported at three audited plans. The machines, and the humans directing them, have not always been careful.
The ethics, without the hand-wringing
Debates about medical AI tend to oscillate between techno-optimism and vague dread. More useful is the specific work being done on the ethical dimensions of AI in risk adjustment, which cuts the problem into questions concrete enough to answer.
Consent and dignity: patients consented to care, and to the record-keeping care requires. Did they consent to algorithmic re-reading of those records for financial optimisation? The honest answer is that the law says yes and intuition says it never asked. Dignity does not demand the machines stop; it demands the reading serve the patient’s interests, accurate records, funded care, and not merely the reader’s.
Truthfulness: an AI that hunts only for diagnoses that raise payments, ignoring the recorded conditions that evidence does not support, is not neutral technology. It is an instrument tuned to a purpose, and the purpose shows in its outputs. The recent enforcement wave punished exactly this asymmetry. The ethical machine, it turns out, is the one that corrects in both directions, including against its owner’s revenue.
Explanation: when an algorithm concludes something about your body, someone should be able to say why, pointing to the sentence in the note, the clinical rule, the human who confirmed it. Opaque conclusions about identifiable people are a species of disrespect, and, conveniently for the cause of ethics, they now also fail audits.
Responsibility: the machine reads; a person answers. Every serious framework, regulatory and moral, converges on keeping a human meaningfully in the loop, not as a rubber stamp but as the accountable author of the final claim. The 3 a.m. reading is tolerable precisely because a daylight human must eventually sign it.
Why this obscure corner matters to everyone
You may never interact with American risk adjustment. But the arrangement it is negotiating, algorithms reading intimate records, producing consequential conclusions, under audit, with humans accountable, is the template being drafted for all of us. Credit files, employment records, insurance histories, the sprawling digital shadow each of us casts: machinery is being pointed at all of it, and the norms are being set wherever the money and the scrutiny arrive first.
Healthcare arrived first. Its emerging settlement is worth stating plainly, because it is better than we might have feared: the reading may happen, but it must be truthful in both directions, explainable to the person it concerns, attributable to an accountable human, and reconstructable years later by a hostile examiner. Not perfect. Enforceable, which in ethics-adjacent matters is usually worth more.
The chart, revisited
There is an old intimacy to the medical record. It is where our frailties are written down by people sworn, more or less, to use them for our good. The machine at 3 a.m. joins a long line of readers: the covering physician, the night nurse, the specialist reviewing before a referral. What we ask of it is what we quietly asked of them. Read carefully. Conclude honestly. Be prepared to say why. And remember whose story it is.
The technology is new. The obligation is not. The audits, for once, are on the obligation’s side.
READ ALSO: Why Glass Inspection Automation is Revolutionizing Manufacturing in California