Evaluating Large Language Models on Clinical & Biomedical NLP Benchmarks

The field of natural language processing (NLP) is rapidly advancing, and accurate clinical and biomedical NLP models are becoming increasingly important. In this keynote speech, Veysel will present a detailed comparison of the state-of-the-art large language models (LLMs)  (namely ChatGPT, GPT-3, GPT-3.5 and BioGPT) and the pre-trained  deep learning (DL) models from Spark NLP for Healthcare on various NLP tasks, including NER, relation extraction, assertion status, de-identification, and entity resolution.

Through his talk, Veysel will demonstrate the strengths and weaknesses of both types of models and aim to show that pre-trained DL models could still be more accurate choice for these types of tasks in real world clinical settings. He will also discuss important factors such as privacy, cost, tunability, freshness, and pre-processing gaps associated with the use of large language models. His presentation will include the benchmarking results, providing attendees with an opportunity to review the methods and evaluate the findings for themselves.

The overall goal of this talk is to provide a fair and unbiased analysis of the performance of these models in clinical and biomedical NLP tasks, to help attendees make informed decisions about the best choice for their use case.

About the speaker

Veysel Kocaman

Lead Data Scientist at John Snow Labs

Veysel is a Lead Data Scientist and ML Engineer at John Snow Labs, improving the Spark NLP for the Healthcare library and delivering hands-on projects in Healthcare and Life Science.

He is a seasoned data scientist with a strong background in every aspect of data science including machine learning, artificial intelligence, and big data with over ten years of experience. He’s also pursuing his Ph.D. in ML at Leiden University, Netherlands, and delivers graduate-level lectures in ML and Distributed Data Processing.

Veysel has broad consulting experience in Statistics, Data Science, Software Architecture, DevOps, Machine Learning, and AI to several start-ups, boot camps, and companies around the globe. He also speaks at Data Science & AI events, conferences and workshops, and has delivered more than a hundred talks at international as well as national conferences and meetups.



Online Event: April 4-5, 2023




Presented by