Using Healthcare-Specific LLM’s for Data Discovery from Patient Notes & Stories

Electronic medical records (EHR’s) contain a lot of structured data and unstructured text about patients. It is most often messy, incomplete, inconsistent, duplicative, and as a result requires a lot of time from doctors and data professionals to get answers from. Large language model (LLM) generative AI interfaces can potentially improve the efficiency and completeness of these workflows by enabling providers or analysts to just ask natural language questions and get short, straight answers – but how accurate and reliable are they?

This session describes benchmarks and lessons learned from building such a pilot system on data from the US Department of Veterans Affairs, a health system which serves over 9 million veterans and their families. This collaboration with VA National Artificial Intelligence Institute (NAII), VA Innovations Unit (VAIU) and Office of Information Technology (OI&T) show that while out-of-the-box accuracy of current LLM’s on clinical notes is unacceptable, it can be significantly improved with pre-processing, for example by using John Snow Labs’ clinical text summarization models prior to feeding that as content to the LLM generative AI output. We will also review responsible and trustworthy AI practices that are critical to delivering these technology in a safe and secure manner.


About the speaker

R. Spencer Schaefer

Chief Health Informatics Officer at Kansas City VAMC – Department of Veteran Affairs

R. Spencer Schaefer, PharmD currently serves as the Chief Health Informatics Officer for the Kansas City VAMC, co-lead for the Medical Informatics Data Science and Emerging Technologies (MInDSET) team and is the AI Solution Architect for the VA National Artificial Intelligence Institute (NAII). 
He has over 20 year experience in medical informatics, software and cloud service provider development and big data analytics. 
His focus for the Veteran’s Health Administration is design, development, and implementation of advanced technologies to improve patient care and reduce staff burden.  He is currently collaborating with VHA, Office of Research and Development, VA Innovations, academia, industry partners and VA Office of Information Technology Division of Data and Analytics to engineer world class cloud computing architecture to accelerate AI research and operational deployment for the Department of Veterans Affairs. 




Online Event: October 3-5, 2023



Presented by