MENLO PARK — TUD Dresden University of Technology is at the forefront of innovation in healthcare through its Clinical AI research group at the Else Kröner Fresenius Center for Digital Health. The group is transforming cancer care by applying cutting-edge computational techniques, particularly in precision oncology. Their work focuses on enhancing clinical decision-making by using advanced deep learning methods to process vast amounts of data—from radiographic images and genomic data to clinical reports—that often remain underutilized.
“Our goal is to develop tools that directly improve daily clinical tasks,” says Dr. Isabella Wiest, Junior Research Group Leader for Clinical AI and Large Language Models (LLMs) in Medicine. The group has made significant strides using Llama 3.1, developing tools to structure and extract actionable insights from medical data while streamlining clinical workflows. By leveraging these innovations, they bridge the gap between the complex field of data science and the practical demands of medical professionals, ensuring that critical patient data is accessible when needed.
The research group’s focus on structured data is crucial for facilitating large-scale analysis and supporting secondary applications. Initially working with Llama 2, the group selected this model for its ability to handle sensitive patient data securely and locally, as opposed to relying on cloud-based solutions. Dr. Wiest notes, “Locally deployable LLMs, such as Llama, offer flexibility and efficiency, making it easier to process complex medical data directly within healthcare institutions.”
The group’s transition to quantized Llama models using Hugging Face and Llama.cpp has reduced memory requirements, enabling large models to run on standard hospital hardware. This breakthrough is critical for enabling AI deployment in clinical settings. Today, TU Dresden’s Clinical AI group uses Llama 3.1 models, including 8B and 70B, for various clinical applications, such as extracting insights from medical documents, automating patient data anonymization, supporting clinical decision-making, and improving medical coding accuracy.
As the research group continues to develop the next generation of LLM-based solutions, they are exploring agent-based approaches where models can take actions under the supervision of medical professionals. The team also faces challenges in ensuring that AI models are intuitive for clinicians and that variable outputs from information extraction tasks are reliably structured. By combining features from llama.cpp, such as grammar restrictions, with their workflows, they can ensure outputs like JSON are consistent and structured.
Privacy and security are paramount in healthcare, and TU Dresden’s approach ensures that sensitive patient data is processed entirely within the institution, enhancing trust and compliance with strict data protection regulations. Prof. Jakob N. Kather, leader of the Clinical AI research group, emphasizes, “By keeping data processing in-house, we not only protect patient data but also ensure compliance with legal and ethical standards, increasing trust in AI tools used in clinical environments.”
Looking to the future, TU Dresden plans to expand the use of Llama models in more complex areas, such as real-time clinical data analysis and personalized patient care. As the Llama ecosystem evolves, the group intends to enhance multilingual capabilities and refine its handling of medical jargon, making these tools more accessible in diverse healthcare environments. The goal is to continue pushing the boundaries of AI in healthcare, improving patient outcomes and broadening access to advanced medical care.
