The application of machine learning methods in modern cancer therapy
DOI:
https://doi.org/10.17721/1812-5409.2025/2.34Keywords:
mathematical modeling, supervised learning, unsupervised learning, reinforcement learning, precision medicine, personalized therapyAbstract
The purpose of this study is to analyze modern approaches to assessing the efficacy and safety of drugs used in anti-cancer therapy using machine learning methods. Particular attention is paid to the prospects of implementing such methods in mathematical oncology, a field that actively uses mathematical modeling and computer simulations in oncological research.
As part of the study, we searched for and studied relevant scientific sources on the application of machine learning in oncology. As a result, a systematic review of the literature covering the use of machine learning in this area was conducted.
The main machine learning approaches, such as Supervised Learning (SL), Unsupervised Learning (UL), and Reinforcement Learning (RL), are analyzed in the context of modern oncology. Specific examples of the use of various machine learning algorithms in research related to cancer treatment, as well as more general oncological problems, are considered. The advantages and limitations of these approaches are assessed depending on the goals set, for example, in such tasks as predicting treatment response, biomarker identification, and automated medical image analysis.
The study found that machine learning is already being actively implemented in oncology research and demonstrates high efficiency in solving various problems, in particular, identifying hidden patterns that are not available in traditional analysis. At the same time, it is noted that there are numerous other areas where the use of machine learning can significantly enhance scientific research and clinical practice. In particular, the use of reinforcement learning algorithms in the field of personalized precision medicine, which plays a key role in creating individualized approaches to cancer treatment, looks promising.
Pages of the article in the issue: 217 - 226
Language of the article: Ukrainian
References
Alsaadi, F. E., Yasami, A., Volos, C., Bekiros, S., & Jahanshahi, H. (2023). A new fuzzy reinforcement learning method for effective chemotherapy. Mathematics, 11(2), 477. https://doi.org/10.3390/math11020477
Ammad-Ud-Din, M., Khan, S. A., Wennerberg, K., & Aittokallio, T. (2017). Systematic identification of feature combinations for predicting drug response with Bayesian multi-view multi-task linear regression. Bioinformatics, 33(14), 359–368.
Badwan, B. A., Liaropoulos, G., Kyrodimos, E., Skaltsas, D., Tsirigos, A., & Gorgoulis, V. G. (2023). Machine learning approaches to predict drug efficacy and toxicity in oncology. Reports Methods, 3(2), 100413. https://doi.org/10.1016/j.crmeth.2023.100413
Bertsimas, D., & Wiberg, H. (2020). Machine learning in oncology: Methods, applications, and challenges. JCO Clinical Cancer Informatics, 4, 885–894. https://doi.org/10.1200/CCI.20.00072
Bhandari, A., Gu, B., Kashkooli, F. M., & Zhan, W. (2024). Image-based predictive modelling frameworks for personalised drug delivery in cancer therapy. Journal of Controlled Release, 370, 721–746. https://doi.org/10.1016/j.jconrel.2024.05.004
Bray, F., Laversanne, M., Sung, H., Ferlay, J., Siegel, R. L., Soerjomataram, I., & Jemal, A. (2024). Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: A Cancer Journal for Clinicians, 74, 229–263. https://doi.org/10.3322/caac.21834
Chang, Y., Park, H., Yang, H. J., Lee, S., Lee, K. Y., & Kim, T. S. (2018). Cancer Drug Response Profile scan (CDRscan): A deep learning model that predicts drug effectiveness from cancer genomic signature. Scientific Reports, 8(1), 8857. https://doi.org/10.1038/s41598-018-27214-6
Cortes, C., & Vapnik, V. (2009). Support-vector networks. Chemical Biology & Drug Design, 297(3), 273–297. https://doi.org/10.1007/BF00994018
De los Rios de la Serna, C. D., Boers-Doets, C. B., Wiseman, T., Radia, B., & Hammond, R. (2024). Early recognition and management of side effects related to systemic anticancer therapy for advanced breast cancer. Seminars in Oncology Nursing, 40, 151553. https://doi.org/10.1016/j.soncn.2023.151553
Eckardt, J.-N., Wendt, K., Bornhäuser, M., & Middeke, J. M. (2021). Reinforcement learning for precision oncology. Cancers, 13(18), 4624. https://doi.org/10.3390/cancers13184624
Eguale, T., Buckeridge, D. L., Verma, A., Winslade, N. E., Benedetti, A., & Hanley, J. A. (2016). Association of off-label drug use and adverse drug events in an adult population. JAMA Internal Medicine, 176(1), 55–63. https://doi.org/10.1001/jamainternmed.2015.6058
Fan, K., Cheng, L., & Li, L. (2021). Artificial intelligence and machine learning methods in predicting anti-cancer drug combination effects. Briefings in Bioinformatics, 22(6), 1–12. https://doi.org/10.1093/bib/bbab271
Firoozbakht, F., Yousefi, B., & Schwikowski, B. (2021). An overview of machine learning methods for monotherapy drug response prediction. Briefings in Bioinformatics, 23(1), bbab408. https://doi.org/10.1093/bib/bbab408
Geeleher, P., Cox, N. J., & Huang, R. (2014). Clinical drug response can be predicted using baseline gene expression levels and in vitro drug sensitivity in cell lines. Genome Biology, 15(3), R47. https://doi.org/10.1186/gb-2014-15-3-r47
Greene, C. S., Tan, J., Ung, M., Moore, J. H., & Cheng, C. (2014). Big data bioinformatics. Journal of Cellular Physiology, 229(12), 1896–1900. https://doi.org/10.1002/jcp.24662
Guan, S., & Wang, G. (2024). Drug discovery and development in the era of artificial intelligence: From machine learning to large language models. Artificial Intelligence Chemistry, 2, 100070. https://doi.org/10.1016/j.aichem.2024.100070
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning: Data mining, inference, and prediction. Springer Science & Business Media.
Kim, Y., Kim, D., Cao, B., Carvajal, R., & Kim, M. (2020). PDXGEM: Patient-derived tumor xenograft-based gene expression model for predicting clinical response to anticancer therapy in cancer patients. BMC Bioinformatics, 21(1), 288. https://doi.org/10.1186/s12859-020-03633-z
Kong, J. H., Lee, H., Kim, D., Han, S. K., Ha, D., & Shin, K. (2020). Network-based machine learning in colorectal and bladder organoid models predicts anti-cancer drug efficacy in patients. Nature Communications, 11(1), 5485. https://doi.org/10.1038/s41467-020-19313-8
Kurilov, R., Haibe-Kains, B., & Brors, B. (2020). Assessment of modelling strategies for drug response prediction in cell lines and xenografts. Scientific Reports, 10, 2849. https://doi.org/10.1038/s41598-020-59656-2
Li, C., Guo, Y., Lin, X., Feng, X., Xu, D., & Yang, R. (2024). Deep reinforcement learning in radiation therapy planning optimization: A comprehensive review. Physica Medica, 125, 104498. https://doi.org/10.1016/j.ejmp.2024.104498
Li, Y., Umbach, D. M., & Krahn, J. M. (2021). Predicting tumor response to drugs based on gene-expression biomarkers of sensitivity learned from cancer cell lines. BMC Genomics, 22, 272. https://doi.org/10.1186/s12864-021-07581-7
Liu, J., Li, M., & Chen, X. (2022). AntiMF: A deep learning framework for predicting anticancer peptides based on multi-view feature extraction. Methods, 207, 38–43. https://doi.org/10.1016/j.ymeth.2022.07.017
Liu, M., Shen, X., Pan, W. (2022). Deep reinforcement learning for personalized treatment recommendation. Statistics in Medicine, 41(20), 4034–4056. https://doi.org/10.1002/sim.9491
Lopez, R. G. (2024). Reinforcement learning in oncology: A comprehensive review. Utrecht University, 36073 https://studenttheses.uu.nl/handle/20.500.12932/47144
Malyutina, A., Majumder, M. M., Wang, W., Pessia, A., Heckman, C. A., & Tang, J. (2019). Drug combination sensitivity scoring facilitates the discovery of synergistic and efficacious drug combinations in cancer. PLoS Computational Biology, 15(5), e1006752. https://doi.org/10.1371/journal.pcbi.1006752
Menden, M. P., Iorio, F., Garnett, M., McDermott, U., Benes, C. H., Ballester, P. J., & Saez-Rodriguez, J. (2013). Machine learning prediction of cancer cell sensitivity to drugs based on genomic and chemical properties. PLoS One, 8(4), e61318. https://doi.org/10.1371/journal.pone.0061318
Mnih, V., Kavukcuoglu, K., & Silver, D. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533. https://doi.org/10.1038/nature14236
Nilashi, M., Ahmadi, H., Abumalloh, R. A., Alrizq, M., Alghamdi, A., & Alyami, S. (2024). Knowledge discovery of patients reviews on breast cancer drugs:Segmentation of side effects using machine learning techniques. Heliyon, 10(19), 38563. https://doi.org/10.1016/j.heliyon.2024.e38563
Parker, J. S., Mullins, M., Cheang, M. C. U., Leung, S., Voduc, D., & Vickery, T. (2009). Supervised risk predictor of breast cancer based on intrinsic subtypes. Journal of Clinical Oncology, 27(8), 1160–1167. https://doi.org/10.1200/JCO.2008.18.1370
Partin, A., Brettin, T. S., Zhu, Y., Narykov, O., Clyde, A., Overbeek, J., & Stevens, R. L. (2023). Deep learning methods for drug response prediction in cancer: Predominant and emerging trends. Frontiers in Medicine, 10, 1086097. https://doi.org/10.3389/fmed.2023.1086097
Phan, L. T., Park, H. W., Pitti, T., Madhavan, T., & Jeon, Y.-J. (2022). MLACP 2.0: An updated machine learning tool for anticancer peptide prediction. Computational and Structural Biotechnology Journal, 20, 4473–4480. https://doi.org/10.1016/j.csbj.2022.07.043
Podgorelec, V., Kokol, P., Stiglic, B., & Rozman, I. (2002). Decision trees: An overview and their use in medicine. Journal of Medical Systems, 26, 445–463. https://doi.org/10.1023/a:1016409317640
Pratap, D. (2017). Statistics for machine learning: Build supervised, unsupervised, and reinforcement learning models using both Python and R. Packt Publishing.
Rafique, R., Islam, S. M. R., & Kazi, J. U. (2021). Machine learning in the prediction of cancer therapy. Computational and Structural Biotechnology Journal, 19, 4003–4017. https://doi.org/10.1016/j.csbj.2021.07.003
Riechelmann, R. P., Zimmermann, C., Chin, S. N., Wang, L., O'Carroll, A., & Zarinehbaf, S. (2008). Potential drug interactions in cancer patients receiving supportive care exclusively. Journal of Pain and Symptom Management, 35(5), 535–543. https://doi.org/10.1016/j.jpainsymman.2007.06.009
Rockne, R. C., & Scott, J. G. (2019). Introduction to Mathematical Oncology. JCO Clinical Cancer Informatics, 3, 1–4. https://doi.org/10.1200/CCI.19.00010
Sakellaropoulos, T., Vougas, K., Narang, S., Koinis, F., Kotsinas, A., & Polyzos, A. (2019). A deep learning framework for predicting response to therapy in cancer. Cell Reports, 29(11), 3367–3373. https://doi.org/10.1016/j.celrep.2019.11.017
Shan, W., Shen, C., Luo, L., & Ding, P. (2023). Multi-task learning for predicting synergistic drug combinations based on auto-encoding multirelational graphs. iScience, 26, 108020. https://doi.org/10.1016/j.isci.2023.108020
She, S., Chen, H., Ji, W., Sun, M., Cheng, J., Rui, M., & Feng, C. (2022). Deep learning based multi-drug synergy prediction model for individually tailored anticancer therapies. Frontiers in Pharmacology, 13, 1032875. https://doi.org/10.3389/fphar.2022.1032875
Shen, C., Nguyen, D., Chen, L., Gonzalez, Y., McBeth, R., Qin, N., Jiang, S. B., & Jia, X. (2020). Operating a treatment planning system using a deep-reinforcement learning-based virtual treatment planner for prostate cancer intensity-modulated radiation therapy treatment planning. Medical Physics, 47(6), 2329–2336. https://doi.org/10.1002/mp.14114
Siddharth, N., Korot, E., Fu, D. J., Zhang, G., Mishra, K., & Lee, A. Y. (2022). Reinforcement learning in ophthalmology: Potential applications and challenges to implementation. The Lancet Digital Health, 4, 692–697. https://doi.org/10.1016/S2589-7500(22)00128-5
Sutton, R. S., & Barto, A. (2018). Reinforcement learning: An introduction (2nd ed.). MIT Press.
Teplytska, O., Ernst, M., Koltermann, L. M., Valderrama, D., Trunz, E., Vaisband, M., Hasenauer, J., Fröhlich, H., & Jaehde, H. (2024).
Learning methods for precision dosing in anticancer drug therapy: A scoping review. Clinical Pharmacokinetics, 63, 1221–1237. https://doi.org/10.1007/s40262-024-01409-9
Tibshirani, R. (2011). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73(3), 273–282. https://doi.org/10.2307/41262671
Timilsina, M., Tandan, M., & Nováček, V. (2022). Machine learning approaches for predicting the onset time of the adverse drug events in oncology. Machine Learning with Applications, 9, 100367. https://doi.org/10.1016/j.mlwa.2022.100367
Tothill, R. W., Tinker, A. V., George, J., Brown, R., Fox, S. B., Lade, S., & Bowtell, D. D. (2008). Novel molecular subtypes of serous and endometrioid ovarian cancer linked to clinical outcome. Clinical Cancer Research, 14, 5198–5208. https://doi.org/10.1158/1078-0432.CCR-08-0196
Triguero, I., García, S., & Herrera, F. (2015). Self-labeled techniques for semi-supervised learning: Taxonomy, software and empirical study. Knowledge and Information Systems, 42(2), 245–284. https://doi.org/10.1007/s10115-013-0706-y
Van Buuren, S. (2018). Flexible imputation of missing data. CRC Press.
Wang, J., Miao, J., Yang, X., Li, R., Zhou, G., Huang, Y., Lin, Z., Xue, W., Jia, X., Zhou, J., Huang, R., Ni, D. (2020). Auto-weighting for breast cancer classification in multimodal ultrasound. ArXiv. https://doi.org/10.48550/arXiv.2008.03435
Xu, C., Song, Y., Zhang, D., Bittencourt, L. K., Tirumani, S. H., & Li, S. (2023). Spatiotemporal knowledge teacher–student reinforcement learning to detect liver tumors without contrast agents. Medical Image Analysis, 90, 102980. https://doi.org/10.1016/j.media.2023.102980
Yang, S., & Kar, S. (2023). Application of artificial intelligence and machine learning in early detection of adverse drug reactions (ADRs) and drug-induced toxicity. Artificial Intelligence in Chemistry, 2, 100011. https://doi.org/10.1016/j.aichem.2023.100011
Yu, K.-H., Beam, A. L., & Kohane, I. S. (2018). Artificial intelligence in healthcare. Nature Biomedical Engineering, 2(10), 719–731. https://doi.org/10.1038/s41551-018-0305-z
Zade, A.E., Haghighi, S. S., & Soltani, M. (2020). Reinforcement learning for optimal scheduling of glioblastoma treatment with temozolomide. Computer Methods and Programs in Biomedicine, 193, 105443. https://doi.org/10.1016/j.cmpb.2020.105443
Zhang, T., Zhang, L., Payne, P. R. O., & Li, F. (2021). Synergistic drug combination prediction by integrating multiomics data in deep learning models. In Methods in Molecular Biology, 2194, 223–238. https://doi.org/10.1007/978-1-0716-0849-4_12
Zhou, J.-B., Tang, D., He, L., Lin, S., Lei, J. H., Sun, H., Xu, X., & Deng, C.-X. (2023). Machine learning model for anti-cancer drug combinations: Analysis, prediction, and validation. Pharmacological Research, 194, 106830. https://doi.org/10.1016/j.phrs.2023.106830
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Ivan Tiurdo, Natalya Kizilova
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
