by Perelman School of Medicine at the University of Pennsylvania

Sample quality control image sent to PACS from the AI Server. Transparent overlays show segmented liver, spleen, visceral fat, and subcutaneous fat from an AI segmentation algorithm used to screen for hepatic steatosis. This allows the radiologist to quickly verify accurate liver and spleen segmentations before including AI spleen and liver measurements in the clinical report. Credit: Journal of Imaging Informatics in Medicine (2024). DOI: 10.1007/s10278-024-01200-z

Our imagination for artificial intelligence is expansive and ambitious. While there are plenty of dystopian tropes, pop culture is full of hopeful examples of what we believe artificial intelligence could bring to us, ranging from operating systems that cure loneliness to assistants that push the limits of humans' physiological capabilities.

Maybe the most famous fictional AI is Roy Batty—an impossibly strong soldier android (or "replicant" in the "Blade Runner" films). What has made the character enduring is less about his superhuman capabilities, and more about what hewed him closer to our most core desire: to live, and to live longer.

At present, many consumer AI tools fall short of the potential imagined in science fiction. In the medical realm, though, an AI program now widely used at Penn Medicine can give us some of the life-sustaining help that Batty wanted most.

Recently, Penn AInSights, an AI-guided imaging system that helps create a more-precise, three-dimensional view of internal organs, was named a CIO 100 winner for its work in the field of radiology.

The program, at its root, is a clinical support tool for physicians, allowing them to look at images of people's livers, spleens, kidneys, and more to determine with some exactitude if the organs are showing any abnormal traits that could shorten lives.

With this precise knowledge of whether a patient has developed something like fatty liver disease, or is showing warning signs of diabetes, or that their kidneys may fail in time, Penn Medicine's clinical staff can take steps to help patients sooner and more effectively than ever before, potentially adding years to their lives.

Normal or abnormal?

"When you look at the liver you say, 'Okay, is this normal?'" said Charles Kahn, MD, MS, a professor of Radiology in the Perelman School of Medicine at the University of Pennsylvania. "You eyeball it and use some measurements to say whether it's big or small. It's kind of like when you look at someone and think whether they could play basketball or be a jockey at the Preakness. But, sometimes, it isn't as easy as that."

From the very first X-rays, radiologic imaging has necessarily involved reconciling two-dimensional images with the space a body part takes up in physical reality. Measurements of length and width may not be perfect indicators of "big" or "small" on three-dimensional objects.

"If your spleen is longer than 13 centimeters, that's considered big," said Kahn. "So if you've got a spleen the shape of a hot dog that's 15 centimeters long, that would be called enlarged. But the actual volume would make it small."

That's where AInSights shows its value.

Trained on thousands of images, the program can quickly analyze a huge amount of imaging and effectively build a digital 3D model of organs. From there, it can flag potential issues, right within the technological tools that clinicians already use every day.

Built and thoroughly tested at Penn Medicine before it was brought to clinicians for fine-tuning to serve patient populations, AInSights was first developed with images gathered through researchers, and built in partnership with Penn Medicine IS teams well-versed in how doctors use technology to diagnose and treat patients.

"The value of this, really, is about having an end-to-end pipeline of development, one of the successful times that Information Services, research, and clinical teams came together," said Ameena Elahi, MPA, RT(R), CIIP, an application manager in Penn Medicine Information Services (IS).

AInSights has been particularly effective because of the holes in other products.

"You look around and there are countless vendors selling AI solutions, with the vast majority in Radiology," said Walter Witschey, Ph.D., an associate professor of Radiology, who helped build the AInSights program and has done research with it. "But despite that number and the huge interest in AI in medicine, they really haven't been adopted by hospitals in a huge way because there were simple problems of integration that were being overlooked by the vendors."

The regular clinical pathway for interpreting radiological images has been for the images to be taken, manually reviewed by radiologists, then put into a report. AInSights was built as a supportive tool to sit, invisibly, atop that infrastructure and seamlessly integrate itself into the process—while improving it.

"The model looks at the images, generates AI annotations and quantifies the traits of what it's looking at—that's given to the radiologist, all automatically," Witschey said.

The process to build this has taken years, but the program has gotten exponentially better and is now used at Penn Medicine to analyze roughly 2,000 scans of the abdomen or chest a month.

"We started off testing it and it was literally an hour to get the final product," Elahi remembered. "Then, quickly, we got it down to about 10 minutes. And, since, we've gotten it down a lot further so it is really clinically convenient."

A paper published in the Journal of Imaging Informatics in Medicine in July (co-authored by Elahi, Kahn, Witschey, and others) showed that the "turnaround time" for CT scans of the abdomen was just 2.8 minutes.

Expanding opportunities

With this type of technology, clinicians are able to do some "opportunistic screening," said Kahn. Someone doing a CT scan to monitor a kidney condition can then also have their liver, spleen, pancreas, and the bottoms of their lungs screened for any extra issues. Human radiologists would obviously focus mainly on the kidney, but the AI could flag anything of note elsewhere.

"There's a lot of information in the 400 to 500 images you end up looking at," Kahn said. "Some of these things are not detectable to the unaided eye, so having these tools really plays into that."

This is allowing clinicians to get on top of conditions that could progress nearly invisibly until they become a serious problem.

For instance, Witschey said that the program, by scanning pixel-level image data, can find patterns among imaging features such as the fattiness of the liver and create a predictive measure for whether someone has diabetes without needing the help of a typical diabetic panel of testing. This makes it easier to recommend follow-up diagnostic testing for those patients.

Additionally, a program taking AInSights out of the abdomen and to the brain is working to assist radiologists in searching for dementia, such as in Alzheimer's disease, which is challenging because of how subtle the changes in imaging can be.

"We can measure the size of the various parts of the brain and compare them to a large database of people who have normal brain imaging to see what parts of the brain have changed and how severe brain volume loss might be," said Ilya Nasrallah, MD, Ph.D., an associate professor of Radiology, who leads implementation of AI tools in the department. "We anticipate will add confidence to our assessment process in dementia screening and inform management of the condition."

Of special importance to AInSights is the Penn Medicine BioBank. More than 40,000 people have had whole genome sequencing stored in the biobank, and tens of thousands have imaging attached to it, Kahn said.

"That really helps us tease out what we call 'imaging phenotypes,' which we can work to connect with information about a person's genetics," Kahn said.

All of this is toward creating a system that can quickly and easily help clinicians decide "what's normal, and what's not" and then decide the most effective course of action.

"We want to develop simple things like a 'nomogram' for spleen volume, which would allow us to look at our patient population and say, 'This spleen is normal for a 33-year-old woman, but for a 70-year-old male patient, that's not right.'"

The future

Eventually, the hope is that AInSights can be used for the whole suite of imaging done at Penn Medicine, including for cancer, neuromuscular degeneration, and cardiovascular conditions.

As it stands right now, the AInSights is extremely cost-effective to run: Less than a dollar per patient, and just about $700 per month at this large health system that sees a high volume of patients.

That makes it appealing even beyond Penn Medicine.

"We have had conversations about getting this into developing countries where AI support would be incredibly valuable," Witschey said.

Having such a powerful system also allows for greater public health applications. Kahn said they're planning on looking at the distribution of "unrecognized kidney disease by zip codes" to better map out underdiagnoses by the social determinants of health. It's a step further in rooting out what was once unseen.

At one point in "Blade Runner," Roy Batty encounters the man who made his artificial eyes. Batty's character is deeply motivated by the scope and wonder of what he has seen, far exceeding what most, if not all, natural, human eyes could see.

He says to the manufacturer, "If only you could see what I've seen with your eyes."

With AInSights, the team at Penn Medicine now can.

More information: Neil Chatterjee et al, A Cloud-Based System for Automated AI Image Analysis and Reporting, Journal of Imaging Informatics in Medicine (2024). DOI: 10.1007/s10278-024-01200-z

Provided by Perelman School of Medicine at the University of Pennsylvania