Lost In Translation? Evaluating AI Solutions For Language Inclusivity In Global Surgery Literature

Lost In Translation? Evaluating AI Solutions For Language Inclusivity In Global Surgery Literature

Habib O Alagbo, Katayoun S Madani , Chandler Hinson, Brenda Feres, Sofia Wagemaker, Shen Hong Law, Fatima Qamar, Neharaj Pitla, Ramitha Eshan Ruwanpathirana, Christopher Dodgion
Hospital center of University of Coimbra, Coimbra, Portugal; Association of Academic Global Surgery, Delaware, United States; Frederick P. Whiddon College of Medicine, University of South Alabama, Mobile, United States; International Student Surgical Network, Sint-Truiden, Belgium; State Medical University, Kursk, Russia; Faculty of Medicine, MAHSA University, Kuala Lumpur, Malaysia; Houston Methodist Hospital, Houston, United States; Xavier University School of Medicine, Oranjestad, Aruba; Medical College of Wisconsin, Milwaukee, United States

Background:
Language barriers significantly exacerbate global surgical disparities by restricting access to essential knowledge. AI translation tools offer scalable solutions to address these barriers, enhancing equity through improved accessibility to global surgery literature. Effective AI translation must achieve accuracy while preserving original formatting and readability.

Methods:
We evaluated three leading AI translation platforms—DeepL, ChatGPT, and Google Translate—by translating four peer-reviewed global surgery articles from English to Spanish. Translation quality was assessed quantitatively using validated computational metrics ordered by their correlation with human judgment: COMET, BLEURT, BERTScore, METEOR, and BLEU. Qualitative evaluations considered readability, terminology accuracy, and preservation of academic formatting.

Results:
DeepL demonstrated superior performance in translation quality, achieving the highest scores in COMET (92.0), BLEURT (89.8), and BERTScore (93.3), which represent strong indicators of human-like translation accuracy. ChatGPT closely followed, showing competitive scores in COMET (91.5) and equal BLEURT performance (89.8). Google Translate ranked lowest among the evaluated platforms across most metrics, with scores notably lower in COMET (90.8), BLEURT (89.0), and BERTScore (92.5). While all three platforms provided high functional capacity, DeepL most consistently retained original document formatting, enhancing readability and preserving academic presentation integrity.

Conclusion:
AI-powered translation, particularly via DeepL, demonstrates significant potential for reducing language-based inequities in global surgery. Effective translations must not only achieve semantic accuracy but also retain document functionality and formatting fidelity. Prioritizing robust, human-aligned translation metrics can help global surgery initiatives choose the most suitable AI tools, thus advancing equity through enhanced language inclusion.

Keywords: Artificial Intelligence, Global Surgery, Equity, Language, Translation