Designing a Healthcare LLM for Efficient Medical Documentation
Since the advent of LLM’s like GPT4 everyone in various industries has been trying to harness their power. Healthcare is an industry where this is a specifically challenging problem due to the high accuracy requirements. Prompt Engineering is a common technique used to design instructions for model responses, however, its challenges lie in the fact that the generic models may not be trained to accurately execute these specific tasks.
I will present our journey of developing a cost-effective medical LLM surpassing GPT4 in medical note-writing tasks. I will touch upon our trials with medical prompt engineering, GPT4’s limitations, and training an optimized LLM for specific medical tasks. I will also showcase multiple comparisons and experiments done on model sizes, training data, and pipeline designs – including RAG and RL – that enabled me (and my team) to outperform GPT4 with smaller models maintaining precision, reducing biases, preventing hallucinations and enhancing note-writing style.
The talk will also consist of techniques used by us to de-identify sensitive data to responsibly develop these models and also LLMOps and practices which allow quick iterations and metric supported experiments. I will also talk about the role of reinforcement learning in latest LLM development as it has become more crucial with time to align the models to humans for wide adoption in industry.
About the speakers
Sagar Goyal
Senior NLP Engineer at DeepScribe Inc.
In his professional journey, Sagar has held positions such as Researcher, Applied Scientist, and Senior Machine Learning (and NLP) Engineer across various organizations, including Microsoft, Snap, and Deep Scribe, as well as research labs like the Max Planck Institute. He expanded State-of-the-Art (SOTA) research in graphs at MPI and, at Microsoft, began his career with exposure to both industry and academia, developing and deploying (at scale) a unique patented technology – CASPR – based on transformers architecture. With a robust background in Machine Learning, Deep Learning, Engineering, and NLP, Sagar’s diverse skill set enables him in his current role as a Senior NLP Engineer at DeepScribe. Leading a team, he is involved in developing a medical LLM that serves as an essential everyday tool for doctors, saving them time and enabling them to attend to more patients. The LLM-based technology assists doctors in SOAP note-writing, automating the process and outperforming GPT-4 based solutions, ultimately saving doctors 2 hours every day. Sagar has an accepted talk in the WSDM industry track, where he strives to share his experience working at the intersection of Healthcare and AI, discussing potential use-cases, challenges, and the role of AI in healthcare with various industry experts. With over 40 citations across 5 research papers and a US patent, he constantly strives to contribute to making the world a better place.
Eti Rastogi
Senior NLP Engineer at DeepScribe Inc.
Eti Rastogi has 2 years of experience working in the industry as an SAP-ABAP Analyst where she was actively involved in understanding and implementing client’s functional requirements, suggesting practical solutions for smooth functioning of business processes. She worked as one of the lead ABAPer in an SAP ERP implementation project at Schneider Electric where she also got the opportunity to get directly involved with the clients at their office in Paris, France. She looks forward to taking advantage of this experience in all my future endeavors.
When
Sessions: April 2nd – 3rd 2024
Trainings: April 15th – 19th 2024
Contact
Presented by
