Implementation and comparison of large language models for ICU notes transcription and summarisation

This project focuses on the comparison and potential implementation of Large Language Models (LLMs) aiming to assist the clinical handovers process in an Intensive Care Unit (ICU) environment. This process is commonly identified as “the transfer of professional responsibility and accountability for some or all aspects of care for a patient and generally occurs between the shifts of supervising nursing staff.

Project Aims

Data Acquisition and Literary Analysis: Discover and download datasets, ideally large in scale, relating to ICU handovers, such as transcripts, considering data validity and accuracy. Identify, from a literary analysis, up to date methodologies for LLM comparison, pertaining to a medical domain.

Comparison and Benchmarking: Create a rigorous system for benchmarking and Deploy LLMs against ICU datasets to evaluate information retention and clinical accuracy. Quantify performance using automated metrics alongside to benchmark model outputs, runtimes, and system requirements.

Checking and Validation: Qualitatively analyse results and outputs from comparison, identifying errors in model outputs, such as hallucinations, as well as model tendencies such as missing information or difficulty summarising medical terminology.

Reflection and Evaluation: Produce a final research paper explaining methodology, results, technical contributions, and outcomes of this project.

Implementation of Results: Use the results and findings of this project to continue to develop an end-to-end pipeline for a practical application of an LLM in an ICU environment using Even Realities G1 smart glasses.

Smart Glasses used in this project