Skip to content
Yu Akagi
English日本語
All work

Watcher

A simulator for patient trajectories. Given a patient's medical history, it generates possible futures. Trained on more than 200 million real clinical records.

Watcher's architecture — a patient timeline of coded events, times, and lab values, encoded and continued by a decoder-only transformer that samples the next event.
Role
First author & lead engineer
Built with
PythonPyTorch
Links
PaperDocumentationSource code

Why a patient is hard to simulate

Computer simulation is a powerful way to deal with uncertainty. Engineers test a bridge in software long before anyone builds it. Weather services simulate how the atmosphere moves to work out how likely rain is tomorrow.

Medicine does not work this way. A patient's course depends on treatment, lifestyle, environment, social circumstances, and chance, and those interact in ways nobody can write down as a set of equations. So most decisions in medicine are made without any simulation beforehand, and without a precise estimate of the odds.

Learning from real-world patient data

A generative model can find the patterns in what has already happened, and use them to produce a patient's future course as data.

Hospitals already hold that history. Diagnoses, drugs, test results, admissions and discharges accumulate in the electronic health record in enormous numbers. We built Watcher, a generative model, and trained it on more than 200 million real clinical records from the University of Tokyo Hospital. Give it a patient's course so far, and it simulates what comes next.

How the model generates a patient timeline, one event at a time.

Give it the same history twice and you get two different simulations. The model is probabilistic, and that matters here, because a patient's future is not fixed. They might fall on the ward and break a hip. They might turn out to be allergic to a drug. Treatment might go well and send them home, or a rare complication might keep them in. Many outcomes stay possible. Watcher learns those outcomes and how often each one happens, and is built to produce them at the rates it learned.

How it works

Watcher works the same way as a large language model. The only difference is what it reads. Where an LLM treats characters and words as tokens, Watcher treats the events in a patient's record as tokens: a diagnosis, a timestamp, a test result. What it writes is not text, then, but patient data itself.

Watcher's architecture — categorical, numeric, and temporal entries are each encoded, summed, and passed through decoder-only transformer layers to a prediction head that samples the next event.
Codes, numbers, and times are encoded separately, then read together by a decoder-only transformer.

A record is not simply a different kind of text, though. It holds three kinds of data, and each has to be encoded differently before the model can read them as one sequence:

Categorical
Diagnoses, drugs, and orders, handled as a vocabulary of codes.
Numeric
Lab values, converted to percentiles so that units and scales do not matter.
Temporal
Years, months, days, hours, and minutes, encoded as values rather than positions.

Encoding time explicitly is what lets the model answer when, not only what next. Two records can contain exactly the same events and still describe very different situations, if one unfolds over hours and the other over months.

How accurate is it?

A simulator is only useful if what it generates resembles reality. We compared generated timelines against what actually happened to real patients: how often events occurred, what values came back from the lab, and how those events were spaced in time. The generated data matched closely.

The most important property is calibration. If the model says something happens one time in ten, does it happen one time in ten? Across a range of outcomes and time horizons, the observed rates stayed close to the predicted ones.

Integrating it into an electronic health record

Building a model and getting one into a hospital are different problems. We built TwinEHR, a proof-of-concept electronic health record, to work on the second. It runs the simulator behind the screens a clinician already reads, and puts the results back into the chart. The paper presents it as a proposed route to deployment, and as evidence that a simulator like this can be fitted to an EHR that is already running.

The digital-twin EHR system: data flows from the primary EHR through standardisation and extraction to a digital-twin server, where a GPU simulator answers simulation requests and returns results to the user's chart view.
Real records are standardised and copied into a separate digital-twin server. Simulation requests are handled by the model on GPUs, and the results are returned to the chart.

To keep training and deployment realistic, the pipeline is built on HL7, the standard hospitals already use to exchange clinical data. Because hospitals speak it already, records from different sites can be brought into the same shape, and in principle the same model can serve more than one hospital.

What this could make possible

A generative model like Watcher has only just been built, and it needs far more validation and testing before anyone relies on it. In time, though, it could support work like this.

  • Personalised care. Treatment plans worked out for the individual patient, based on what the simulation says is likely for them.
  • Virtual trials. Testing a new drug on a computer before testing it on people.
  • Counterfactuals. Simulating different treatments and comparing how each one is likely to turn out.

The work is described in the paper. The model and the system around it are both open: see the documentation and the repository.

Thank you for reading.