by Medical University of Vienna

Scenario of high-risk patients with multiple nevi. a: Graphical abstract of scenario of monitoring of high-risk individuals with multiple nevi. Due to the large number of lesions per patient, this scenario requires a more patient-centered and less lesion-centered approach. Most melanomas detected during monitoring are noninvasive, slow-growing lesions. Short-term monitoring of these melanomas, while not optimal, is considered acceptable. b: Malignancy probability predictions of the baseline SL model according to management predictions of the RL model for benign lesions (n = 7320) and melanomas (n = 55). The red dashed horizontal line indicates the median value of the melanoma probability selected by 10 experts as threshold for excision. The black dashed horizontal line indicates the minimum value. Black line = median, boxes = 25th–75th percentiles, whiskers = values within 1.5 times the interquartile range. Credit: Nature Medicine (2023). DOI: 10.1038/s41591-023-02475-5

Artificial intelligence (AI) is already being used to diagnose skin cancer, but it cannot (yet) keep pace with the complex decision-making of doctors in practice. An international research team led by Harald Kittler of MedUni Vienna has now explored a learning method in which greater accuracy in AI results can be achieved by incorporating human decision-making criteria. In this way, the rate of correct skin cancer diagnoses made by dermatologists was improved by 12%. The study was published in Nature Medicine.

The researchers based their study on the reinforcement learning (RL) model and integrated (human) criteria in the form of "reward tables" into the AI system. Reward tables are tools that incorporate the positive and negative consequences of clinical assessments into the decision-making process from both the physician's and the patient's perspective.

On this basis, AI diagnosis results were not only rated as right or wrong, but were "rewarded" or "penalized" with a certain number of plus or minus points depending on the impact of the diagnosis or the resulting decisions.

Learning from human assessments

"In this way, the AI learned to take into account not only image-based features, but also consequences of misdiagnosis in the assessment of benign and malignant skin manifestations," says study leader Harald Kittler from the Department of Dermatology at MedUni Vienna.

As a result, as the study shows, the accuracy of the diagnosis of skin cancer could be significantly improved: The sensitivity for melanoma, for example, was increased from 61.4 to 79.5% and for basal cell carcinoma from 79.4 to 87.1%.

Overall, the use of RL increased the rate of correct diagnoses made by dermatologists by 12%, while the rate of optimal decisions for management and therapy of the disease increased from 57.4 to 65.3%.

Such improved performance of AI-based skin cancer diagnosis is also because RL reduces the AI's overconfidence in its own predictions and makes more nuanced and human-compatible suggestions. "This, in turn, helps physicians make more accurate decisions tailored to individual patients in complex medical scenarios," Harald Kittler said. Although the current work focused mainly on skin cancer diagnosis, the basic ideas could also be used in other areas of medical decision-making.

More information: Catarina Barata et al, A reinforcement learning model for AI-based decision support in skin cancer, Nature Medicine (2023). DOI: 10.1038/s41591-023-02475-5

Journal information: Nature Medicine 

Provided by Medical University of Vienna