Today we present a paper from the team of Marinka Zitnik from Harvard Medical School. Drug repurposing, the search for new therapeutic uses for approved drugs, is often a serendipitous and opportunistic effort to expand the use of drugs in new diseases. The clinical utility of drug repurposing AI models remains limited because they are too focused on diseases for which drugs are already available. Here, the authors propose TxGNN, a graph-based model for zero-sample drug repurposing that identifies therapeutic candidates for diseases with limited or no existing drug treatment options.Trained on the medical knowledge graph, TxGNN utilizes a graph neural network and a metrics learning module to rank potential indications and contraindications for drugs for 17,080 diseases. In a benchmark test against eight methods, TxGNN improved the prediction accuracy of indications by 49.2% and contraindications by 35.1% under a rigorous zero-sample evaluation. To facilitate model interpretation, the interpreter module of TxGNN provides transparent multi-hop medical knowledge pathways as an explanation of the basis of TxGNN predictions. Manual evaluation of the TxGNN interpreter showed that TxGNN predictions and interpretations performed encouragingly on multiple performance dimensions, especially on dimensions beyond accuracy. Many of TxGNN's new predictions are highly consistent with indicative out-of-care prescriptions previously made by physicians in large healthcare systems.TxGNN's drug repurposing predictions are accurate, consistent, and can be investigated by human experts through multiple-hop interpretable reasoning pathways.

Currently, only 5-7% of the world's more than 7,000 rare diseases have FDA-approved drugs. The global burden of disease can be reduced through drug repurposing, i.e., expanding the use of existing drugs through the discovery of new indications. Drug repurposing utilizes safety and efficacy data from approved drugs and can accelerate clinical adoption and reduce development costs. About 30% of FDA-approved drugs gain at least one new indication, with some drugs accumulating more than a dozen indications. However, most drug repurposing is discovered serendipitously, such as through out-of-indication prescribing by physicians or patient experience.

Predicting the efficacy of all drugs for all diseases can help select drugs with fewer side effects, design more effective multi-targeted therapies, and systematically repurpose existing drugs. Technological advances have allowed us to prospectively match drugs to new indications by analyzing the medical knowledge graph. Machine learning has been used to analyze molecular interaction networks, reveal the structure of perturbed genes in diseases, and design treatments accordingly.

Although computational approaches have identified promising repurposed drug candidates for complex diseases, there are two major challenges: first, most diseases do not yet have approved drugs, especially rare diseases, and second, the indications for repurposed drugs may be unrelated to their originally developed uses. To address these issues, this paper proposes the TxGNN model, a graph-based model designed for multi-disease zero-sample drug repurposing, covering 17,080 diseases, including diseases with no treatment options.The TxGNN is trained based on the Medical Knowledge Graph, which utilizes a graph neural network to embed drugs and diseases into an optimized representation space to perform zero-sample prediction.The TxGNN also contains an interpreter module that provides transparent multi-hop paths to explain the basis of prediction, and its predictions are highly consistent with out-of-indication clinical prescriptions.

Link to model predictions and explanations: http://txgnn.org

Model Part

As shown in Figure 1b, zero-sample drug repurposing refers to the prediction of therapeutic candidates for diseases for which there are no or only a few therapeutic options. Mathematically, the model inputs drug-disease pairs and outputs the likelihood that a drug will work for that disease. The gold standard label for evaluating models comes from the previously compiled Medical Knowledge Graph (KG), which contains 9,388 indications and 30,675 contraindications covering 17,080 diseases, 92% of which lack FDA-approved drugs, and primarily involves rare and complex diseases.The KG also includes 7,957 potential drug repurposing candidates, ranging from FDA-approved drugs to experimental drugs in clinical trials.

Figure 1

The core idea of the TxGNN model is that drugs either directly target disease perturbed networks or indirectly propagate therapeutic effects through disease related networks. In Fig. 1c, the TxGNN model consists of two modules: a prediction module and an interpretation module. The prediction module utilizes a graph neural network to generate meaningful representations on the medical KG through self-supervised pre-training. Subsequently, through fine-tuning, the model is able to handle different therapeutic tasks, predict indications and contraindications of drugs on a wide range of diseases, and especially make zero-sample predictions on diseases with no or only a few treatment options. The interpretation module uses the GraphMask method to generate a sparse but sufficient subgraph that extracts multi-hop paths between drugs and diseases to help experts understand the basis of the model's predictions.

To achieve zero-sample prediction, TxGNN uses a metric learning component to transfer knowledge from labeled diseases to unlabeled diseases by exploiting the genetic and genomic network information shared between diseases. The model generates a prediction by generating signature vectors for each disease and integrating information about related diseases based on similarity. In this process, TxGNN can effectively summarize the knowledge of similar diseases and apply it to the prediction of the target disease. The model not only generates prediction scores for diseases, but also provides interpretable multi-hop pathways through the interpretation module, which explain the association logic between drugs and diseases.

Treatment Matching and Zero Sample Drug Reuse

The authors compare TxGNN with eight methods that include statistical techniques for network medicine such as Kullback-Leibler (KL) and Jensen-Shannon (JS) scatter, graph theoretic network proximity methods, diffusion state distance (DSD), and state-of-the-art graph neural network methods such as Relational Graph Convolutional Networks (RGCNs), Heterogeneous Graph Transformers (HGTs), and Heterogeneous Attention Networks (HANs), as well as the natural language processing model BioBERT.

First, the authors used a standard benchmark strategy for evaluating drug repurposing AI models by randomly disrupting the drug-disease treatment pairs and using a portion of them as the retention set (Figure 2c). Under this strategy, the diseases in the retained set have a number of drug indications and contraindications in the training dataset. Thus, the goal of model generalization is to identify treatment candidates for diseases for which some drugs are already available.

Figure 2

The authors use the area under the exact recall curve (AUPRC) as an evaluation metric, as it measures the trade-off between recall and precision of the model at different thresholds. Experimental results in this setting showed that three of the eight existing methods had an AUPRC of more than 0.80, with HAN performing the best with an AUPRC of 0.873. TxGNN also performed similarly to these existing methods. In indication prediction, TxGNN had an AUPRC of 0.913, a 4.3% improvement over HAN.

Considering that standard assessment strategies are not appropriate for evaluating diseases for which no FDA-approved drugs are available, the authors evaluated the model's performance in zero-sample drug repurposing. The authors first randomly retained a set of diseases and then moved all their associated drugs to the retained set (Figure 2d). In the absence of any drug similarity data, TxGNN performed significantly better than all existing methods. TxGNN improved the AUPRC by 19.0% in indication prediction and 23.9% in contraindication prediction over the suboptimally performing method.TxGNN was the only method to maintain consistent performance across all scenarios.

Evaluation of zero sample drug repurposing in different disease areas

To evaluate the performance of drug repurposing models on these challenging diseases, the authors curated a rigorously reserved dataset containing a set of biologically relevant diseases called “disease domains”. For each disease domain, the authors removed all drug indications and contraindications from the training dataset, as well as a portion of the relationships between drugs and other medical concepts. The dataset evaluated the performance of the model in diseases with limited molecular data and no existing drugs (Figure 3a).

Figure 3

The authors benchmarked TxGNN on a rigorously retained dataset (Figure 3b-f) and found that it outperformed existing methods in all disease areas. Across the nine disease areas, TxGNN showed relative AUPRC gains of 0.5-59.3% (mean gain 25.72%) in indication prediction and 11.8-35.6% (mean gain 18.67%) in contraindication prediction.

Human-centered assessment of TxGNN drug candidates

To assess the utility of the TxGNN Multi-hop Interpretable Pathway for expert assessment, the authors conducted a pilot study involving clinicians and scientists. Participants included five clinicians, five clinical researchers, and two pharmacists (Figure 4c). When assessing drug-disease indication predictions, participants were asked to evaluate 16 predictions of TxGNN, 12 of which were accurate. The authors recorded participants' assessment accuracy, thinking time, and confidence ratings for each prediction for a total of 192 trials. The user study took an average of approximately 65 minutes and consisted of assessing TxGNN's drug-disease predictions, completing a usability questionnaire, and conducting a semi-structured interview.

Figure 4

In post-task questionnaires and interviews, participants reported higher levels of satisfaction when using the TxGNN interpreter (Figure 4e), with 11 out of 12 (91.6%) agreeing or strongly agreeing that the predictions and interpretations provided by the TxGNN were valuable. In contrast, when there was no explanation, 8 out of 12 (75.0%) indicated that they disagreed or strongly disagreed with relying only on TxGNN's predictions. Participants' confidence in correct predictions was significantly higher when the TxGNN interpreter was included. Some participants noted that the multi-hop interpretable explanation was very helpful when examining molecular target interactions recognized by the TxGNN interpreter and could guide the assessment of potential adverse drug events.

Consistency of TxGNN prediction basis with medical evidence

The authors examined the consistency of TxGNN predictions for drugs and their multi-hop explanations with medical reasoning for three rare diseases. This assessment process was divided into three stages (Figure 5a). First, experts identified drugs that could potentially be used for repurposing for a particular disease by querying the TxGNN predictor, which provides candidate drugs with a prediction confidence level and their ranking relative to other candidates. Next, the TxGNN interpreter is used to explain why the drug was selected for repurposing, and the model demonstrates the association between the drug and the disease through a multi-hop interpretable pathway. Finally, independent medical evidence is collected and analyzed to validate the model's predictions and its explanations.

Figure 5

First, the authors examined TxGNN's prediction of Klefteira syndrome. This is a rare disorder caused by mutations in the EHMT1 gene that results in delayed speech development, autism spectrum disorders, and hypotonia in children, often accompanied by a poorly developed brain and inactive neural pathways. When querying the TxGNN predictor, zolpidem was recommended as the most promising drug repurposing candidate (Figure 5b).The TxGNN interpreter indicated that zolpidem's effect on GABRG2 may reduce autism susceptibility and improve prefrontal cortex function.

Next, the authors examined TxGNN predictions for Ehlers-Danlos syndrome, a rare connective tissue disorder that affects 1-9 people per 100,000 people. The disease is caused by mutations in the genes encoding collagen (COL1A1 and COL1A2) and manifests as poor wound healing and abnormal scarring.The TxGNN predictor listed retinoic acid as a preferred candidate for drug repurposing. Retinoic acid is transported via albumin (ALB) and targets ALDH1A2, helping to alleviate collagen loss and inflammation (Figure 5c).

Finally, the authors analyzed the prediction of TxGNN for a rare disease, nephrogenic syndrome of inappropriate antidiuresis (NSIAD). This disease is characterized by dysregulation of water and sodium balance and is caused by mutations in the AVPR2 gene. Similar to patients with congestive heart failure, NSIAD patients face water retention problems, and the disorder is strongly associated with the AVPR2 and NPR1 genes.The TxGNN predictor ranked amyl nitrite as one of the top five drugs (Fig. 5d).The TxGNN interpreter suggests that the association between NSIAD and amyl nitrite is linked through the AVPR2, congestive heart failure, and NPR1 genes .

Evaluating TxGNN using electronic medical records

The robustness of TxGNN suggests that its novel predictions, i.e., drugs that have not yet received clinical approval for a disease but rank high, may be of potential clinical value. Since these therapies are not yet approved for therapeutic use, there is no readily available gold standard to validate them. Given the long-standing clinical practice of out-of-indication drug prescribing, the authors used the co-occurrence enrichment of disease-drug pairs in health system electronic medical records (EMRs) as a proxy indicator of potential indications.

The authors compiled a cohort of 1,272,085 adults with at least one drug prescription and one diagnosis per patient from Mount Sinai Health System medical records (Figure 6a-d). In this cohort, 40.1% were male and the mean age was 48.6 years. The authors selected diseases for which at least one patient was diagnosed and medications that were prescribed to at least ten patients, resulting in a dataset containing 478 diseases and 1,290 medications.

Figure 6

Within these medical records, the authors calculated the co-occurrence enrichment of disease-drug pairs by measuring the ratio of the odds of a particular drug being used for a particular disease to its odds of being used for other diseases, yielding 619,200 log odds ratio (log(OR)) values with the necessary statistical corrections. The authors found that FDA-approved disease-drug pairs had significantly higher log(OR) values than other pairs (Figure 6e).

For 478 diseases based on EMR phenotypes, TxGNN generated a ranked list of therapeutic candidates. The authors excluded drugs already associated with the disease and categorized the remaining new candidates into: rank 1, top 5, top 5%, and bottom 50%, and calculated their respective mean log(OR) values (Figure 6f). The authors further analyzed the prediction of TxGNN for Wilson's disease, a rare disease that causes copper accumulation in the liver and triggers cirrhosis in children (Figure 6g). The authors observed that TxGNN's prediction of the likelihood of indication for most drugs was close to zero, with only a few drugs showing a high likelihood of indication.

Discussion

Drug repurposing, as an approach to drug discovery, aims to address the high costs, long lead times, and risks associated with developing new drugs. The traditional “single-disease-single-prediction-model” strategy has improved success rates, but most successes are due to accidental discoveries. The authors propose a multi-disease prediction strategy to achieve comprehensive drug repurposing, especially for complex, neglected or rare diseases that lack known treatment options. To this end, the authors developed the TxGNN graph-based model, specifically designed to address diseases with limited data and treatment options.TxGNN is capable of performing zero-sample extrapolation, predicting therapeutic candidates for never-before-seen diseases, and generating interpretable multihop pathways to help experts analyze the potential biological response to a drug. Despite its excellent performance, TxGNN's capabilities are still dependent on the quality of the medical knowledge graph, and data bias and updating issues will need to be addressed in the future.

Bibliography

Huang K, Chandak P, Wang Q, et al. A foundation model for clinician-centered drug repurposing[J]. Nature Medicine, 2024: 1-13.