Keywords
Medicine
Risks
regulation
systematic literature review
How to Cite
Abstract
This study examined the key risks associated with the use of artificial intelligence (AI) in medical practice, emphasizing the profound transformations that technology is driving in the healthcare sector. Based on a systematic literature review and consultation of other bibliographic sources, ten major risks were identified: biases, discrimination, social implications, bias denial, black-box problems, reinforcement of prejudices, explainability, transparency, intelligibility, and privacy. The development of AI in healthcare has led to systems that are far more autonomous and complex than initially expected, posing significant challenges due to their direct impact on human health. This situation highlights the need for regulation and oversight by the relevant public authorities. Although the regulation of new technologies requires careful consideration regarding when and how to regulate, the risks associated with AI in medicine are already well-recognized. Failing to intervene could be seen as governmental inaction in fulfilling the responsibility to ensure health and equity. AI should function as a support tool, not a replacement for physicians, ensuring that specialists validate the accuracy and effectiveness of algorithmic recommendations.
References
Aquino, Yves Saint James, et al. "Practical, Epistemic and Normative Implications of Algorithmic Bias in Healthcare Artificial Intelligence: A Qualitative Study of Multidisciplinary Expert Perspectives." Journal of Medical Ethics, February 2023. Accessed June 2, 2024. https://pubmed.ncbi.nlm.nih.gov/36823101/.
Arrieta, Alejandro Barredo, et al. "Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges Toward Responsible AI." Cornell University, 2019. Accessed March 3, 2024. https://arxiv.org/pdf/1910.10045.
Benanti, Paolo. Oráculos: Entre Ética e Governança dos Algoritmos. São Leopoldo: Unisinos, 2020.
Biller-Andorno, Nikola, et al. "AI Support for Ethical Decision-Making Around Resuscitation: Proceed with Care." Journal of Medical Ethics 48 (March 2021): 175–83. Accessed December 12, 2023. https://www.medrxiv.org/content/10.1101/2020.08.17.20171769v1.
Capasso, Marianna, and Steven Umbrello. "Responsible Nudging for Social Good: New Healthcare Skills for AI-Driven Digital Personal Assistants." Medicine, Health Care and Philosophy 25 (2021): 11–22. Accessed April 3, 2024. https://link.springer.com/article/10.1007/s11019-021-10062-z.
Corrêa, Nicholas Kluge, Camila Galvão, and James William Santos, et al. "Worldwide AI Ethics: A Review of 200 Guidelines and Recommendations for AI Governance." Patterns 4 (October 13, 2023): 100857. https://doi.org/10.1016/j.patter.2023.100857.
Dourado, Daniel de Araujo, and Fernando Mussa Abujamra Aith. "A Regulação da Inteligência Artificial na Saúde no Brasil Começa com a Lei Geral de Proteção de Dados Pessoais." Revista Saúde Pública 56, no. 80 (2022). Accessed June 8, 2024. https://www.scielosp.org/pdf/rsp/2022.v56/80/pt.
Facchini Neto, Eugenio, and Roberta Scalzilli. "Pode a Ética Controlar o Desenvolvimento Tecnológico? O Caso da Inteligência Artificial, à Luz do Direito Comparado." In Tutela Jurídica do Corpo Eletrônico, edited by Cristiano Colombo, Wilson Engelmann, and José Luiz de Moura Faleiros Junior. Indaiatuba: Foco, 2022.
Ferrario, Andrea, Sophie Gloecker, and Nikola Biller-Andorno. "Ethics of the Algorithmic Prediction of Goal of Care Preferences: From Theory to Practice." Journal of Medical Ethics 49 (November 2023): 165–74. Accessed May 15, 2024. https://jme.bmj.com/content/medethics/49/3/165.full.pdf.
Galvão, Maria Cristiane Barbosa, and Ivan Luiz Marques Ricarte. "Revisão Sistemática da Literatura: Conceituação, Produção e Publicação." Logeion: Filosofia da Informação 6, no. 1 (September 2019): 57–73. Accessed December 21, 2023. https://sites.usp.br/dms/wp-content/uploads/sites/575/2019/12/Revis%C3%A3o-Sistem%C3%A1tica-de-Literatura.pdf.
Goodman, Katherine E., et al. "Preparing Physicians for the Clinical Algorithm Era." The New England Journal of Medicine 389, no. 6 (August 2023): 483–87. Accessed May 10, 2024. https://pubmed.ncbi.nlm.nih.gov/37548320/. https://doi.org/10.1056/NEJMp2304839.
Gunning, David, and David W. Aha. "DARPA’s Explainable Artificial Intelligence Program." AI Magazine, Association for the Advancement of Artificial Intelligence, 2019, 44–59.
Markus, Aniek F., Jan A. Kors, and Peter R. Riknbeek. "The Role of Explainability in Creating Trustworthy Artificial Intelligence for Health Care: A Comprehensive Survey of the Terminology, Design Choices, and Evaluation Strategies." Journal of Biomedical Informatics, 2020. Accessed May 15, 2024. https://pubmed.ncbi.nlm.nih.gov/33309898/.
Moreno-Sanchez, Pedro A. "An Automated Feature Selection and Classification Pipeline to Improve Explainability of Clinical Prediction Models." In IEEE 9th International Conference on Healthcare Informatics (ICHI), Finland, 2021.
Nogaroli, Rafaella. "Implicações da IA na Medicina: ChatGPT Já Faz Diagnósticos e É Aprovado para Residência Médica." Gazeta do Povo, April 13, 2023. Accessed May 15, 2024. https://www.gazetadopovo.com.br/opiniao/artigos/implicacoes-da-ia-na-medicina-chatgpt-ja-faz-diagnosticos-e-e-aprovado-para-residencia-medica/?fbclid=PAAab1Qw_lqKO0fGnZ1OYMioIOmw-ZC_m2lGpWJyyi5kYZra8IYpCIpgKnPCM.
Ossa, Laura Arbalaez, et al. "A Smarter Perspective: Learning with and from AI-Cases." Artificial Intelligence in Medicine 135 (January 2023). Accessed May 10, 2024. https://www.sciencedirect.com/science/article/pii/S093336572200210X.
Paranjape, Ketan, et al. "Short Keynote Paper: Mainstreaming Personalized Healthcare–Transforming Healthcare Through New Era of Artificial Intelligence." IEEE Journal of Biomedical and Health Informatics 24, no. 7 (July 2020). Accessed April 10, 2024. https://ieeexplore.ieee.org/document/8988253.
Rajkumar, Radhika. "AI Risks Are Everywhere – and Now MIT Is Adding Them All to One Database." ZDNet, August 14, 2024. Accessed August 15, 2024. https://www.zdnet.com/article/ai-risks-are-everywhere-and-now-mit-is-adding-them-all-to-one-database/.
Rogers, Wendy, Heather Draper, and Stacy Carter. "Evaluation of Artificial Intelligence Clinical Applications: Detailed Case Analyses Show Value of Healthcare Ethics Approach in Identifying Patient Care Issues." Bioethics 35, no. 7 (2021): 623–33. Accessed April 10, 2024. https://onlinelibrary.wiley.com/doi/10.1111/bioe.12885.
Rueda, Jon, et al. "Just Accuracy? Procedural Fairness Demands Explainability in AI-Based Medical Resource Allocations." Open Forum, December 21, 2022. Accessed April 10, 2024. https://link.springer.com/article/10.1007/s00146-022-01614-9.
Sánchez-Caro, Javier, and Fernando Abellán. Telemedicina y Protección de Datos Sanitarios. Granada: Comares, 2002. Accessed May 15, 2024. https://protecciondata.es/wp-content/uploads/2021/10/Sanchez-Caro-Javier-y-Abellan-Fernando-Telemedicina-y-Proteccion-de-Datos-Sanitarios-Edit.-El-Partal-Granada-2002.pdf.
Schwab, Klaus. A Quarta Revolução Industrial. Translated by Daniel Moreira Miranda. São Paulo: Edipro, 2016.
Selbst, Andrew, and Julia Powles. "Meaningful Information and the Right to Explanation." International Data Privacy Law 7, no. 2 (November 2017): 233–42. Accessed May 15, 2024. https://academic.oup.com/idpl/article/7/4/233/4762325.
Skinovsky, James, Maurício Chibata, and Daniel Emílio Dalledone Siqueira. "Realidade Virtual e Robótica em Cirurgia – Aonde Chegamos e para Onde Vamos?" Revista do Colégio Brasileiro de Cirurgiões 35, no. 5 (2008): 334–37. Accessed November 15, 2023. https://www.scielo.br/j/rcbc/a/rFD6rx7BbL5y37gPKJMPDNK/.
Smallman, Melanie. "Multi-Scale Ethics – Why We Need to Consider the Ethics of AI in Healthcare at Different Scales." Science and Engineering Ethics 28, no. 63 (2022). Accessed November 10, 2023. https://link.springer.com/article/10.1007/s11948-022-00396-z.
Stahl, Bernd Carsten, et al. "A Systematic Review of Artificial Intelligence Impact Assessments." Artificial Intelligence Review 24 (2023): 1–33. Accessed February 3, 2024. https://pubmed.ncbi.nlm.nih.gov/37362899/.
Ulmenstein, Ulrich Von, et al. "Limiting Medical Certainties? Funding Challenges for German and Comparable Public Healthcare Systems Due to AI Prediction and How to Address Them." Frontiers in Artificial Intelligence 5 (2022). Accessed June 10, 2024. https://www.semanticscholar.org/paper/Limiting-medical-certainties-Funding-challenges-for-Ulmenstein-Tretter/099516d7d9f7dbe6f3625a03e9b66ee5c05bab46.
Wachter, Sandra, Brent Mittelstadt, and Luciano Floridi. "Why a Right to Explanation of Automated Decision-Making Does Not Exist in the General Data Protection Regulation." International Data Privacy Law 7, no. 2 (2017). Accessed June 12, 2024. https://academic.oup.com/idpl/article/7/2/76/3860948.
Yigitcanlar, Tan, Juan M. Corchado, and Rashid Mehmood, et al. "Responsible Urban Innovation with Local Government Artificial Intelligence (AI): A Conceptual Framework and Research Agenda." Journal of Open Innovation: Technology, Market, and Complexity 7, no. 71 (2021). https://doi.org/10.3390/joitmc7010071.
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright (c) 2024 Vanessa Schmidt Bortolini, Wilson Engelmann, Alexandre de Souza Garcia