Organizations that develop or deploy Generative AI solutions in healthcare are subject to more than 70 national and state laws, regulatory rules, and industry standards. Once an organization establishes an AI Governance framework, its policies will include dozens of controls that must be implemented for each AI project. This session describes a subset of these controls that can be automated with current tools:
- Automated execution of medical LLM benchmarks during system testing and when monitoring in production, including coverage of medical ethics, medical errors, fairness and equity, safety and reliability – using Pacific AI
- Automating generation and executing of LLM test suites for custom solutions, including testing for robustness, bias, fairness, representation, and accuracy – using LangTest
- Automated generation of model cards, complying with transparency laws and including explained benchmark results – based on the CHAI draft model card standard
Ben Webster
at NLPLogix
Ben Webster is the VP of AI Solutions at NLP Logix, where he was the company’s first employee. With over 11 years at the company, Ben has led hundreds of machine learning projects across various industries, driving innovation and delivering impactful AI solutions. Notably, he has played a key role in automating audio and language processes for the largest healthcare patient survey provider in the U.S. and delivering AI-driven solutions for multiple organizations within the human resources sector. Ben’s groundbreaking work in language modeling earned him a patent, further cementing his position as a leader in the AI field.
Originally from Baltimore, MD, Ben has lived in Jacksonville since 2024, where he has become an active community advocate, particularly in educational initiatives. He volunteers for programs like UNF’s Data Science for Social Good, NLP Logix’s Analytics Boot Camp, and collaborates with Edward Waters University to help shape the growth of its technology department.
Ben holds a Master’s degree in Mathematics from the University of North Florida. In 2023, he was named one of the Jacksonville Business Journal’s Ultimate Tech Leaders.
David Talby
CEO at Pacific AI
David Talby is the CEO at Pacific AI and John Snow Labs, helping companies apply artificial intelligence to solve real-world problems in healthcare and life science.
David is the creator of Spark NLP – the world’s most widely used natural language processing library in the enterprise. He has extensive experience building and running web-scale software platforms and teams – in startups, for Microsoft’s Bing in the US and Europe, and to scale Amazon’s financial systems in Seattle and the UK.
David holds a Ph.D. in Computer Science and Master’s degrees in both Computer Science and Business Administration. He was named USA CTO of the Year by the Global 100 Awards in 2022 and Game Changers Awards in 2023.
