The Negative Aspect of Machine Learning in Healthcare

Assistant Professor Marzyeh Ghassemi analyzes how disguised biases in medical data can jeopardize artificial intelligence approaches.

While completing her dissertation in computer science at MIT, Marzyeh Ghassemi wrote numerous papers on how machine learning strategies from artificial intelligence could be applied to medical data to anticipate patient outcomes. “It was not till the end of my Ph.D. work that one of my committee members asked: ‘Did you ever review to see how well your model worked across various groups of individuals?'”.

That question was enlightening for Ghassemi, who had previously examined the performance of models in aggregate across all patients. After a closer look, she saw that models usually operated differently– specifically even worse– for populations involving Black women, a revelation that took her by surprise. “I had not linked earlier that health disproportions would translate straight to model disproportions,” she claims. “And given that I am a noticeable minority woman-identifying computer scientist at MIT, I am fairly certain that several others were not familiar with this either.”.

In a paper released Jan. 14 in the journal Patterns, Ghassemi– that earned her doctorate in 2017 and is currently an assistant professor in the Department of Electrical Engineering and Computer Science and the MIT Institute for Medical Engineering and Science (IMES)– and her coauthor, Elaine Okanyene Nsoesie of Boston University, provide a cautionary note concerning the prospects for AI in medicine. “If used very carefully, this technology can boost performance in health care and possibly decrease inequities,” Ghassemi claims. “But if we are not really mindful, technology could aggravate care.”.

The flaws of machine learning

All of it boils down to data, given that the AI tools in question train themselves by processing and examining vast amounts of data. However, the data they receive is produced by people. Humans are imperfect and whose judgments may be clouded by the that they interact in different ways with patients based on their race, gender, and age, without even knowing it.

Furthermore, there is still significant uncertainty around medical conditions themselves. “Doctors trained at the same medical school for a decade can, and often do, disagree about a patient’s diagnosis,” Ghassemi states. That is different from the applications where existing machine-learning algorithms succeed– like object-recognition tasks– because almost every person in the world will certainly agree that a dog is, actually, a dog.

Machine-learning algorithms have also prospered in mastering games like chess and Go, where both the rules and the “win conditions” are clearly specified. Physicians, nonetheless, do not constantly agree on the rules for treating patients, and even the win condition of being “healthy” is not extensively agreed upon. “Doctors understand what it means to be ill,” Ghassemi explains, “and we have the most data for individuals when they are sickest. However, we do not get much data from people when they are healthy because they’re much less most likely to see doctors at that point.”.

Future of machine learning in healthcare

Even mechanical devices can support flawed data and disparities in treatment. For example, pulse oximeters, which have been calibrated predominately on light-skinned people, do not precisely measure blood oxygen levels for darker-skinned people. Furthermore, these inadequacies are most poignant when oxygen levels are low– exactly when accurate readings are most urgent. Likewise, women face increased risks during “metal-on-metal” hip replacements, Ghassemi and Nsoesie write, “due partly to anatomic differences that are not taken into consideration in implant design.” Details like these could be buried within the data fed to computer models whose output will certainly be impaired as a result.

Originating from computers, the product of machine-learning algorithms provides “the sheen of objectivity,” according to Ghassemi. However, that can be misleading and dangerous because it is more challenging to search out the defective data provided en masse to a computer than it is to discount the recommendations of a single, possibly inept (and maybe even racist) physician. “The problem is not machine learning itself,” she firmly insists. “It is individuals. Human caregivers generate bad data occasionally because they are not perfect.”.

Nonetheless, she still believes that machine learning can provide advantages in health care in terms of much more efficient and fairer practices and recommendations. One key to realizing the promise of machine learning in health care is to boost the quality of data, which is not an easy task. “Imagine if we can take data from doctors that have the best performance and share that with other physicians that have much less training and experience,” Ghassemi states. “We really need to gather this data and audit it.”.

Further investigations

The hurdle here is that the collection of data is not incentivized or rewarded, she notes. “It is hard to get a grant for that or ask students to allot time on it. Furthermore, data providers might claim, ‘Why should I provide my data out for free when I can sell it to a company for millions?’ However, researchers should have the ability to access data without needing to deal with questionings like: ‘What paper will I get my name on for giving you access to data that remains at my institution?’.

Read the original article on MIT News.

Related “Uncovering the Secrets of the Big Bang With Machine Learning”.