The model shares information across patients who have similar health problems which leads to better predictions

Statistical researchers from the University of Washington, after analyzing medical records from thousand of patients, have developed a statistical model for predicting what other medical problems that a patient might encounter. This predictive algorithm has been used in a medical setting for the first time. It is based on social media algorithms like say Facebook which suggests ‘friend’ or Amazon which suggests ‘products’ based on the person’s interaction with these platforms. 

How does it work? For instance, if a patient has already had dyspepsia and epigastric pain, there are good chances that he might suffer from heartburn. Tyler McCormick, an assistant professor of statistics and sociology at the University of Washington, says that, “This provides physicians with insights on what might be coming next for a patient, based on experiences of other patients. It also gives a predication that is interpretable by patients.”

The algorithm will be published in the upcoming issue of the Journal Annals of Applied Statistics. It will be co-authored by Cynthia Rudin of Massachusetts Institute of Technology, and David Madigan of Columbia University.

Explaining the difference of his model from others, Mr McCormick said that “it shares information across patients who have similar health problems. This allows for better predictions when details of a patient’s medical history are sparse.”

Many a times, new patients might not have detailed records such as files listing ailments and drug prescriptions from the previous doctor. Here “the algorithm”, Mr McCormick explains, “can compare the patient’s current health complaints with other patients who have a more extensive medical record that includes similar symptoms and the timing of when they arise. Then the algorithm can point to what medical conditions might come next for the new patient. We’re looking at each sequence of symptoms to try to predict the rest of the sequence for a different patient.”

Interestingly, the algorithm will also be helpful in situations where it is statistically difficult to predict a less common condition. “For instance, most patients do not experience strokes, and accordingly most models could not predict one because they only factor in an individual patient’s medical history with a stroke. But McCormick's model mines medical histories of patients who went on to have a stroke and uses that analysis to make a stroke prediction,” says the press release.

The statisticians used medical records obtained from a multi-year clinical drug trial involving tens of thousands of patients aged 40 and older. The records also included demographic details, such as gender and ethnicity, as well histories of medical complaints and prescription medications of the patient.

According to the press release, it was found that of the 1,800 medical conditions in the dataset, most of them—1,400— occurred fewer than 10 times. McCormick and his co-authors had to come up with a statistical way to not overlook those 1,400 conditions, while alerting patients who might actually experience those rarer conditions. They came up with a statistical modelling technique that is grounded in Bayesian methods, the backbone of many predictive algorithms. McCormick and his co-authors call their approach the Hierarchical Association Rule Model and are working toward making it available to patients and doctors.

"We hope that this model will provide a more patient-centred approach to medical care and to improve patient experiences," Mr McCormick said.