Skip to main content

News and Events


 

Chair’s Message

pth-use-this-oneWe are moving toward our vision with a number of activities across our various programs. We have updated our strategic plan in response to the 10-year academic program review that we recently completed. For our research-oriented MS and PhD programs, we have recently added a specialization in Data Science. We are completing a curriculum revision for our on line applied clinical informatics MS which will be effective Fall 2020. The work of our fellows in the clinical informatics fellowship program has received plaudits from clinical administrators and faculty, and we are currently recruiting a new faculty member in our department to assist with this program (view position description).  We are also recruiting a faculty member in medical education to start Summer 2020 (view position description). This is the beginning of a new cycle of admissions to our graduate programs, and we look forward to another productive year, and new growth in our department.

Cordially,

Peter Tarczy-Hornoch, MD
Chair and Professor, Department of Biomedical Informatics and Medical Education

Biomedical Informatics and Medical Education Newsletter

August 4, 2025 – August 8, 2025

UPCOMING LECTURES AND SEMINARS
BIME 590 – See you in autumn quarter – 9/25/2025!

UPCOMING EXAMS
Title: Cultural Adaptation and Evaluation of LLM-Driven Mental Health Conversational Agents
Student: Serena Xie
Date/Time: Monday, August 18th, 2025, 11am PT
In-person location: Health Science Building (HST) T473
Zoom: https://washington.zoom.us/my/cohenta

Abstract: Mental health disparities disproportionately affect underserved caregiver populations, in part due to limited availability of culturally responsive interventions. While large language model (LLM)-driven conversational agents offer promise for scalable mental health support, their outputs often suffer from cultural misalignment, limiting engagement among diverse populations. This dissertation develops and evaluates a generalizable, low-resource workflow for dynamic cultural adaptation of LLM-based mental health agents. Grounded in formative qualitative work with Chinese American and Latino American family caregivers and community stakeholders, I developed a cultural context database capturing salient caregiving challenges. Two adaptation strategies were implemented: (1) prompt-based “cultural prompting” and (2) a retrieval-augmented generation (RAG) workflow that dynamically integrates relevant cultural context during real-time interactions. Controlled evaluations showed the RAG-based approach outperformed both prompt-based and non-adapted agents on cultural responsiveness, perceived empathy, and therapeutic alliance. A randomized pilot study with Chinese American caregivers demonstrated improvements in short-term emotional well-being. This work advances scalable, culturally responsive AI agents for equitable mental health support and offers a generalizable workflow for adaptation across diverse populations and care contexts.

July 28, 2025 – August 1, 2025

UPCOMING LECTURES AND SEMINARS
BIME 590 – See you in autumn quarter – 9/25/2025!

ANNOUNCEMENTS
Internship – Real World Evidence (RWE) Team
Location: Remote or Hybrid (based on applicant location)
Duration: 3–6 months
Compensation: Paid Internship
Start Date: Flexible (Summer/Fall 2025 preferred)

Position Summary:

The Natera RWE team is seeking a highly motivated and analytical intern to contribute to real-world data initiatives involving large-scale clinical and genomic datasets. The ideal candidate will have hands-on experience with clinical coding systems (ICD-10, CPT, NDC), a keen interest in advanced analytics using AI/LLMs, and exposure to digital pathology and multimodal biomedical data.

This internship provides a unique opportunity to work at the intersection of clinical informatics, genomics, and machine learning, supporting projects that drive impactful healthcare insights.

For Full Details and How to Apply contact rigreen@natera.com

PAPERS, PUBLICATIONS & PRESENTATIONS

  • S. Zeng, L. Liu, J. Wen, M. Yetisgen, R. Etzioni, and G. Luo. TrajSurv: Learning Continuous Latent Trajectories from Electronic Health Records for Trustworthy Survival Prediction. Proc. 2025 Machine Learning for Healthcare Conference, Rochester, MN, Aug. 2025.

July 21, 2025 – July 25, 2025

UPCOMING LECTURES AND SEMINARS
BIME 590 – See you in autumn quarter – 9/25/2025

ANNOUNCEMENTS
Please join us in congratulating Kevin Chen who successfully passed his PhD General Exam!
Title: Precision Prognostication in Post-Cardiac Arrest Brain Injury using Deep Learning

Abstract: Neurological injury is a primary determinant of mortality and long-term disability in cardiac arrest survivors, yet current prognostic tools fail to capture the dynamic complexity of recovery. This proposal aims to develop a data-driven framework that combines deep learning with structural neuroscience to decode hypoxic-ischemic brain injury. At its core, the proposal has three transformative aims:  the novel application of autoencoder neural networks to uncover latent injury patterns from neuroimaging that may elude conventional radiological assessment, the identification of clinically distinct patient subgroups through unsupervised clustering of these injury patterns potentially revealing recovery trajectories, and the integration of multimodal data identifying neural hubs that can govern the recovery of consciousness. By moving beyond traditional one-dimensional prognostication, this approach characterizes the spatial heterogeneity of brain injury and also generates clinically actionable insights through patient stratification. The framework’s ability to pinpoint vulnerable brain regions enables targeted monitoring and early intervention that may lead to improved neurological outcomes.

PAPERS, PUBLICATIONS & PRESENTATIONS

  • Wesley Surento had an abstracted accepted by conference workshop ISMRM Workshop on Breast MRI: Technological Advances & Clinical Applications (https://www.ismrm.org/workshops/2025/BreastMRI/). It will be on September 13-15, 2025 in Vegas.
  • Zhaoyi Sun, Wen-Wai Yim, Özlem Uzuner, Fei Xia, Meliha Yetisgen, A scoping review of natural language processing in addressing medically inaccurate information: Errors, misinformation, and hallucination, Journal of Biomedical Informatics, 2025, 104866, https://doi.org/10.1016/j.jbi.2025.104866.

July 14, 2025 – July 18, 2025

UPCOMING LECTURES AND SEMINARS
BIME 590 – See you in autumn quarter – 9/25/2025!

ANNOUNCEMENTS
Recent BIME graduate Yue Guo was selected as a finalist for the Edward H. Shortliffe doctoral dissertation award: https://amia.org/about-amia/amia-awards/signature-awards/edward-h-shortliffe-doctoral-dissertation-award

PAPERS, PUBLICATIONS & PRESENTATIONS

  • Fecho K, Glusman G, Baranzini SE, Bizon C, Brush M, Byrd W, Chung L, Crouse A, Deutsch E, Dumontier M, Foksinska A, Hadlock J, He K, Huang S, Hubal R, Hyde GM, Israni S, Kenmogne K, Koslicki D, Marcette JD, Mathe EA, Mesbah A, Moxon SAT, Mungall CJ, Osborne J, Pasfield C, Qin G, Ramsey SA, Reese J, Roach JC, Rose R, Soman K, Su AI, Ta C, Vaidya G, Weber R, Wei Q, Williams M, Wu C, Xu C, Yakaboski C; Biomedical Data Translator Consortium. Announcing the Biomedical Data Translator: Initial Public Release. Clin Transl Sci. 2025.

UPCOMING EXAMS
Title: Precision Prognostication in Post-Cardiac Arrest Brain Injury using Deep Learning
Student: Kevin Chen
Date/Time: Monday, July 21, 2025, 9am – 11am PT
In-person location: 850 Republican Street, Building C, SLU C123A
Zoom: https://washington.zoom.us/j/91423524935

Abstract: Neurological injury is a primary determinant of mortality and long-term disability in cardiac arrest survivors, yet current prognostic tools fail to capture the dynamic complexity of recovery. This proposal aims to develop a data-driven framework that combines deep learning with structural neuroscience to decode hypoxic-ischemic brain injury. At its core, the proposal has three transformative aims:  the novel application of autoencoder neural networks to uncover latent injury patterns from neuroimaging that may elude conventional radiological assessment, the identification of clinically distinct patient subgroups through unsupervised clustering of these injury patterns potentially revealing recovery trajectories, and the integration of multimodal data identifying neural hubs that can govern the recovery of consciousness. By moving beyond traditional one-dimensional prognostication, this approach characterizes the spatial heterogeneity of brain injury and also generates clinically actionable insights through patient stratification. The framework’s ability to pinpoint vulnerable brain regions enables targeted monitoring and early intervention that may lead to improved neurological outcomes.

June 30, 2025 – July 4, 2025

UPCOMING LECTURES AND SEMINARS
BIME 590 – See you in autumn quarter – 9/25/2025!

ANNOUNCEMENTS
On July 1st, Michelle Stoffel, MD, PhD (alumnae of the UW clinical informatics fellowship program) became Program Director of the Clinical Informatics Fellowship for the ACGME-accredited University of Minnesota-Hennepin HealthCare Clinical Informatics Fellowship Program. This multisite program combines the strengths of academic and community medical practice with research and didactic opportunities to prepare physicians for excellence in clinical informatics leadership roles.

PAPERS, PUBLICATIONS & PRESENTATIONS

  • Patricia S. Groves, Yelena Perkhounkova, Amany Farag, Maria Hein, Janice A. Sabin, Matthew J. Witry and Brad Wright, 2025, Concern and credibility: a factorial survey experiment on nurse judgments and intent to report patient-expressed safety events. 2025.  BMC Nursing, 24:798
  • LM-Merger: A workflow for merging logical models with an application to gene regulatory network models” by Luna Xingyu Li,  Boris Aguilar, John H Gennari, and Guangrong Qin

June 23, 2025 – June 27, 2025

UPCOMING LECTURES AND SEMINARS
BIME 590 – See you in autumn quarter – 9/25/2025!

PAPERS, PUBLICATIONS & PRESENTATIONS

  • Deepthi Mohanraj, Raina Langevin, Libby Shah, Janice Sabin, Brian R. Wood, Wanda Pratt, Nadir Weibel, Andrea L. Hartzler. Leveraging Provocative Design Methods to Address Implicit Bias in Clinical Interactions through Technology. AMIA Annual Symposium 2025. Full paper
  • Julia C. Dunbar, Wanda Pratt, Lily Jeffs, Chelsea Ng, Sanaa Sayed, Jodi Smith, Ari H. Pollack. My Kidney T.R.E.K. – Thinking, Reflecting, and Empowering Kidney Transplant Patients, through technology. AMIA Annual Symposium 2025. Full paper
  • Hyeyoung Ryu, Sungha Kang, Wanda Pratt. Mitigating Stigma and Fostering Support: Improving AI-Generated Counterspeech for Microaggressions. AMIA Annual Symposium 2025. Full paper
  • Emma J. McDonnell, Wanda Pratt. From Chronic Health Condition to Disability Identity: Opportunities for Health Informatics Engagement. AMIA Annual Symposium 2025. Full paper
  • Lisa Dirks, Victoria BearBow, Wanda Pratt. Enhancing Health Research Results Dissemination for American Indian and Alaska Native Communities through Indigenous Community-Centered Design. AMIA Annual Symposium 2025. Full paper
  • Portuguese, Andrew J., Emily C. Liang, Jennifer J. Huang, Yein Jeon, Danai Dima, Rahul Banerjee, Mary Kwok, Kara I. Cicero, Alexandre V. Hirayama, Ryan Basom, Christy Khouderchah, Mazyar Shadman, Lawrence Fong, Andrew J. Cowan, and Jordan Gauthier. 2025. “Extramedullary Disease Is Associated with Severe Toxicities Following B‑Cell Maturation Antigen CAR T‑Cell Therapy in Multiple Myeloma.” Haematologica, June 19.
  • Ding X, Sheng Z, Hur B, Tauscher J, Ben-Zeev D, Yetişgen M, Pakhomov S, Cohen T. Tailoring task arithmetic to address bias in models trained on multi-institutional datasets. Journal of Biomedical Informatics. 2025 Jun 8:104858.
  • Sheng, Z., Ding, X., Hur, B., Li, C., Cohen, T., & Pakhomov, S. (2025). Mitigating confounding in speech-based dementia detection through weight masking. To appear in: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), Vienna, Austria, 2025.
  • Saha, Aparajita;Molani, Sevda; Mease, Philip; Hadlock, Jennifer. Outcome-wide analysis of electronic health records data for identifying sequelae in Behçet’s disease. Accepted for podium abstract presentation at AMIA Annual Symposium 2025.
  • Oliver Bear Don’t Walk; Lauren W Yowelunh McLester-Davis; Susan Brown Trinidad.
    Rectifying Genocidal Data Stewardship: A Commentary on Ethical and Legal Obligations for Sharing Data With Tribal Entities
    https://www.jmir.org/2025/1/e77946
  • Xie SJ, Zhai S, Liang Y, Li J, Fan X, Cohen T,Yuwen W. Cultural Prompting Improves the Empathy and Cultural Responsiveness of GPT-Generated Therapy Responses. Accepted to AMIA Annu Symp Proc. 2025. Full paper.
  • Wang L, Carrington D, Filienko D, Jazmi C., Xie SJ, De Cock M, Iribarren S, Yuwen, W. (2025). Large language model-powered conversational agent delivering Problem-Solving Therapy (PST) for family caregivers: Enhancing empathy and therapeutic alliance using in-context learning. Accepted to AMIA Annu Symp Proc. 2025. Full paper. Also, In arXiv [cs.AI]. arXiv. http://arxiv.org/abs/2506.11376

June 16, 2025 – June 20, 2025

UPCOMING LECTURES AND SEMINARS
BIME 590 – See you in autumn quarter – 9/25/2025!

ANNOUNCEMENTS
Dr. Neil Abernethy, who supported the Fred Hutchinson Cancer Research Center (FHCRC), CDC, HHS, and WA-DOH during the pandemic, is lead author of this Nature Digital Medicine paper describing the Coronavirus Prevention Network (CoVPN) Volunteer Screening Registry (VSR), which he co-designed with partners from FHCRC and Oracle Inc. The VSR collected over 650,000 volunteers and distributed them through hundreds of network sites to facilitate recruitment of diverse participants into multiple Phase 3 clinical trials. This effort helped accelerate COVID-19 vaccine development, underscoring the critical role informatics can play in rapidly addressing national health crises.

https://www.nature.com/articles/s41746-025-01666-3?utm_source=rct_congratemailt&utm_medium=email&utm_campaign=oa_20250506&utm_content=10.1038/s41746-025-01666-3

 

PAPERS, PUBLICATIONS & PRESENTATIONS

  • Feng Chen had poster presentation accepted by AMIA annual symposium: “Detecting PTSD in Clinical Interviews: A Comparative Analysis of NLP Methods and Large Language Models”: Feng Chen, Dror Ben-Zeev, Gillian Sparks, Arya Kadakia, Trevor Cohen. arXiv:2504.01216.
  • Good news from Anne Turner.  “Our abstract entitled, Digital Ageism and Informatics Research Involving Older Adults with Cognitive Impairment, has been accepted for panel presentation at the AMIA 2025 Annual Symposium this fall.  Two BIME-associated faculty,  Dr George Demiris (UPenn) and Dr. Amanda Lazar (Univ. of Maryland), along with Dr. Greg Alexander (Columbia) will be participating in the panel.”
  • Kuan-Ching Wu, Basia Belza, Donna Berry, Frances Lewis, Oleg Zaslavsky, and Andrea Hartzler. UTI risk factors in older people living with dementia: A conceptual framework and a scoping review. Dementia. https://journals.sagepub.com/eprint/E4MC5YDWAP3DDZFWIABI/full
  • Young D, Bartlett LE, Reid N, Hartzler AL, Bradford MC, Goss CH, Pilewski JM, Dunitz JM, Saavedra M, Berry DL, Kapnadak SG, Ramos KJ. Personal Narratives to Support Learning about Lung Transplant for People with Cystic Fibrosis. Patient Education and Counseling. 2025 May 5:108822.
  • Ji Z, Wedgeworth P, Mertens K, Jackson SL, Akinsoto NO, Klein J, Isaac M, Hartzler AL. EHR-Based Social Needs Screening and Referral in Primary Care: Clinician and Staff Perspectives on Practices, Barriers, and Benefits. AMIA Annual Symposium 2025, Full paper
  • Sarrieddine A, Lai C, Bear Don’t Walk O, Reid NFH, Sawicki G, Berlinski A, Rosenfeld M, Hartzler AL. Technology and Human Support Systems in Decentralized Clinical Trials: A Participant-Centered Case Study in Cystic Fibrosis. AMIA Annual Symposium 2025, Full paper
  • Lai C, Casanova-Perez R, Langevin R, Tsendenbal A, Saxena S, Pratt W, Sabin J, Wood BR, Weibel N, Hartzler AL. Automated Assessments of Clinical Encounters: Provider Perspectives. AMIA 2025 Annual Symposium, Poster.
  • Hartzler AL, Chung RY, Sarrieddine A, McNamara S, Lee M, Daines C, Davis J, Joseph L, Brown RF, Drake G, Revay D, Rushing S, Vanderbilt N, Rosenfeld M, Ong T. Are we ready for home spirometry? Patient and family perspectives on standardized implementation in pediatric clinical care NACFC 2025
  • Vasbinder, Knapp, Grassy, Harris, Basile, Young, Milinic, Bartlett, Hartzler, Kapnadak, Smith, Riekert, Berry, Ramos. Evaluating shared decision making in lung transplant discussions between clinicians and people with severe to advanced cystic fibrosis lung disease. NACFC 2025
  • Langevin R, Lai C, Mohanraj D, Shah L, Casanova-Perez R, Tsedenbal A, Saxena S, Sabin J, Wood BR, Pratt W, Weibel N, Hartzler AL. UnBIASED: Developing communication feedback technology to address implicit bias in patient-provider interactions . WPRN Annual meeting 2025.
  • Langevin R, Martin L, Bolivar KN, Keen S, Wedgeworth P, Lui PP, Hartzler AL. Using Large Language Models to Evaluate Patient-Centered Note Writing in Clinical Documentation. AMIA Clinical Informatics Conference, Anaheim CA, May 2025.

June 9, 2025 – June 13, 2025

UPCOMING LECTURES AND SEMINARS
BIME 590 – See you in autumn quarter – 9/25/2025!

ANNOUNCEMENTS
You’re invited to help us celebrate 2024-25 BHI graduates, new emeritus faculty, and student/faculty awardees at the BHI End-of-Year Graduation Celebration! Here are the details:

When: Ceremony this Friday, June 13th from 2:00-3:00pm with reception immediately following from 3:00-5:00pm

Where:

  • SLU Orin Smith Auditorium for the 2:00-3:00pm ceremony or remotely (Zoom link below);
  • SLU 123 A & B for the 3:00-5:00pm reception (in person only)

Zoom link: https://washington.zoom.us/j/95887407122?pwd=8BbgfuyqBR2ISMEuthvMUV9P88qU5J.1

_______________________
Authors: Jimmy Phuong, MSPH, PhD; Adam B Wilcox, PhD, FAMIA; Shruti Sehgal, MD(Hom), MS; Anthony Solomonides, PhD, MSc, FAMIA, FACMI; Juan Espinoza, MD

Title: Maturity Models 101 – What does Roadmapping Institutional Growth in Informatics look like?. [Tutorial session]
Date: 11/15/2025
Time: 8:30 AM – 12:00 PM EST
Session Code: W04

_______________________
Please join us in congratulating Ehsan Alipour who successfully passed his PhD Defense!
Title: Evaluating Multi-Modal Data Fusion Approaches for Predictive Clinical Models Using Multiple Medical Data Domains

Abstract: Multimodal deep learning models have emerged as powerful tools in biomedical research, offering the ability to integrate diverse data sources such as clinical records, multi-omics data, imaging, survey responses, and wearable data to enhance predictive accuracy and deepen understanding of complex medical phenomena. Central to multimodal modeling is the process of data fusion, where information from different modalities is integrated in a unified model. Three primary fusion strategies exist in deep learning: early fusion (feature-level), intermediate fusion and late fusion (decision-level). While widely adopted in other domains, their comparative performance and implementation considerations remain underexplored in biomedical applications, where data heterogeneity, missingness, and varying dimensionality present additional challenges.

This dissertation aims to evaluate the implications of data fusion strategies in developing multimodal predictive models in medicine. Across three distinct aims, we assess the impact of early, intermediate, and late fusion techniques on predictive performance, implementation complexity, and generalizability using diverse combinations of data types, outcomes, and modeling strategies. These studies span multiple datasets and outcome types (binary vs continuous) providing a broad view of fusion strategy utility in real-world biomedical settings.

In Aim 1, we evaluated and compared early, intermediate, and late fusion strategies for integrating longitudinal EHR, genomic, and survey data to predict chronic kidney disease (CKD) progression in patients with type 2 diabetes using a novel transformer-based multimodal architecture. Using data from the NIH’s All of Us initiative, we trained models on a cohort of approximately 40,000 patients. While unimodal models—particularly those based on EHR data—achieved strong baseline performance with an AUROC of 0.73 (0.71 – 0.75), the inclusion of multimodal data offered only marginal improvement with an AUROC of 0.74 (0.72 – 0.76), with the benefit limited to the early fusion approach and lacking statistical significance. This aim highlighted the challenges of overfitting in complex fusion architectures and emphasized the role of modality-specific predictive strength.

In Aim 2, we extended the fusion analysis to imaging data by combining a convolutional neural network (CNN) trained on longitudinal cross-sectional imaging with a shallow neural network trained on clinical and pathology variables to predict post-surgical margin status in patients with soft tissue sarcoma (n=202). Here, the intermediate fusion strategy significantly outperformed other approaches, achieving an AUROC of 0.80 (0.66–0.95), suggesting that cross-modal interactions between histologic features and imaging embeddings may be best captured through intermediate fusion. This result demonstrated the potential value of intermediate fusion when complementary signals exist across modalities.

In Aim 3, we explored fusion strategies for estimating continuous CT-derived body composition metrics (e.g., visceral, and subcutaneous fat volumes) using only chest radiographs and clinical variables in a dataset of 1,088 patients. A multitask multimodal model was developed and evaluated across early, intermediate, and late fusion strategies. Late fusion consistently delivered the best performance across most body composition metrics, closely followed by intermediate fusion. These results suggest that when individual modalities offer high independent predictive power, decision-level integration may be optimal for regression tasks.

Collectively, this work provides a comprehensive evaluation of data fusion strategies in multimodal biomedical modeling, highlighting their strengths, limitations, and practical considerations. Findings suggest that no single fusion strategy universally outperforms the others; rather, optimal fusion depends on data characteristics, model architecture, and task-specific objectives. This dissertation lays the groundwork for future research aimed at developing adaptive fusion strategies tailored to the complexities of real-world biomedical data.

PAPERS & PUBLICATIONS

 

June 2, 2025 – June 6, 2025

UPCOMING LECTURES AND SEMINARS
BIME 590 – See you in autumn quarter – 9/25/2025!

ANNOUNCEMENTS
Andrea Hartzler was elected serve on the SOM Council on Appointments and Promotions for a 3-year term!

_______________________

Please join us in congratulating Velvin Fu who successfully passed her PhD Defense!

Title: Evaluating and Enhancing Large Language Models (LLMs) in the Clinical Domain

Abstract: Recent advancements in large language models (LLMs) have demonstrated human-level performance on many specialized medical tasks, even without annotated training data. However, three main challenges remain: (1) due to the sensitive and highly specialized nature of clinical narratives, as well as the high cost of human expert annotation, there is a lack of high-quality, well-structured, and clinically meaningful datasets for LLM training and evaluation; (2) current medical LLMs show limited generalization ability to interpret and extract complex clinical information on certain unseen natural language understanding (NLU) tasks; and (3) as LLMs are typically trained on vast amounts of data, there is a substantial risk of data contamination, where evaluation benchmarks unintentionally overlap with training data, leading to inflated test performance and potentially reduced performance on truly novel tasks.

In this work, we address these limitations through three core aims: (1) develop benchmark datasets for clinical information extraction (IE), a key NLU subtask, across two critical medical domains, and evaluate the performance of multiple state-of-the-art (SOTA) transformer-based language models (LMs), under both fine-tuning and in-context learning settings; (2) develop a more generalizable medical NLU model via instruction tuning, demonstrating enhanced performance on previously unseen clinical NLU datasets; and (3) systematically review existing detection approaches for data contamination and evaluate those approaches on datasets used during pre-training and fine-tuning LLMs, with our own and three other widely used open-source LLMs.

In summary, our work contributes to the development of both clinical benchmarks and robust LLMs, as well as highlighting the ongoing challenges in benchmarking LLMs’ generalizability.

UPCOMING EXAMS
Title: Evaluating Multi-Modal Data Fusion Approaches for Predictive Clinical Models Using Multiple Medical Data Domains
Student: Ehsan Alipour
Date/Time: Wednesday, June 11, 2025, 11am – 1 pm PT
In-person location: 850 Republican Street, Building C, SLU C259
Zoom: https://washington.zoom.us/my/peter.th

Abstract: Multimodal deep learning models have emerged as powerful tools in biomedical research, offering the ability to integrate diverse data sources such as clinical records, multi-omics data, imaging, survey responses, and wearable data to enhance predictive accuracy and deepen understanding of complex medical phenomena. Central to multimodal modeling is the process of data fusion, where information from different modalities is integrated in a unified model. Three primary fusion strategies exist in deep learning: early fusion (feature-level), intermediate fusion and late fusion (decision-level). While widely adopted in other domains, their comparative performance and implementation considerations remain underexplored in biomedical applications, where data heterogeneity, missingness, and varying dimensionality present additional challenges.

This dissertation aims to evaluate the implications of data fusion strategies in developing multimodal predictive models in medicine. Across three distinct aims, we assess the impact of early, intermediate, and late fusion techniques on predictive performance, implementation complexity, and generalizability using diverse combinations of data types, outcomes, and modeling strategies. These studies span multiple datasets and outcome types (binary vs continuous) providing a broad view of fusion strategy utility in real-world biomedical settings.

In Aim 1, we evaluated and compared early, intermediate, and late fusion strategies for integrating longitudinal EHR, genomic, and survey data to predict chronic kidney disease (CKD) progression in patients with type 2 diabetes using a novel transformer-based multimodal architecture. Using data from the NIH’s All of Us initiative, we trained models on a cohort of approximately 40,000 patients. While unimodal models—particularly those based on EHR data—achieved strong baseline performance with an AUROC of 0.73 (0.71 – 0.75), the inclusion of multimodal data offered only marginal improvement with an AUROC of 0.74 (0.72 – 0.76), with the benefit limited to the early fusion approach and lacking statistical significance. This aim highlighted the challenges of overfitting in complex fusion architectures and emphasized the role of modality-specific predictive strength.

In Aim 2, we extended the fusion analysis to imaging data by combining a convolutional neural network (CNN) trained on longitudinal cross-sectional imaging with a shallow neural network trained on clinical and pathology variables to predict post-surgical margin status in patients with soft tissue sarcoma (n=202). Here, the intermediate fusion strategy significantly outperformed other approaches, achieving an AUROC of 0.80 (0.66–0.95), suggesting that cross-modal interactions between histologic features and imaging embeddings may be best captured through intermediate fusion. This result demonstrated the potential value of intermediate fusion when complementary signals exist across modalities.

In Aim 3, we explored fusion strategies for estimating continuous CT-derived body composition metrics (e.g., visceral, and subcutaneous fat volumes) using only chest radiographs and clinical variables in a dataset of 1,088 patients. A multitask multimodal model was developed and evaluated across early, intermediate, and late fusion strategies. Late fusion consistently delivered the best performance across most body composition metrics, closely followed by intermediate fusion. These results suggest that when individual modalities offer high independent predictive power, decision-level integration may be optimal for regression tasks.

Collectively, this work provides a comprehensive evaluation of data fusion strategies in multimodal biomedical modeling, highlighting their strengths, limitations, and practical considerations. Findings suggest that no single fusion strategy universally outperforms the others; rather, optimal fusion depends on data characteristics, model architecture, and task-specific objectives. This dissertation lays the groundwork for future research aimed at developing adaptive fusion strategies tailored to the complexities of real-world biomedical data.