BioGPT: Generative Language Models for Healthcare and Beyond
Pre-trained language models have attracted increasing attention in the biomedical domain, inspired by their great success in the general natural language domain. Among the two main branches of pre-trained language models in the general language domain, i.e. BERT (and its variants) and GPT (and its variants), the first one has been extensively studied in the biomedical domain, such as BioBERT and PubMedBERT, while the second one is under explored. In this talk, we will show how a generative langauge model, BioGPT, together with fine-tuning, can be used for biomedical relation extraction, quesion answering, document classification and generating descriptions for biomedical terms. In particular, BioGPT is the first to achieve human-level performance on the PubMedQA task.
Renqian Luo
Researcher at Microsoft Research
Renqian Luo is a Researcher at Microsoft Research AI4Science. Previously, he was a Researcher of Deep and Reinforcement Learning Group, Machine Learning Group at Microsoft Research Asia (MSRA). He got the Ph.D. degree from University of Science and Technology of China (USTC), School of Computer Science and Technology in 2021, in a joint Ph.D. program with USTC and MSRA, advised by Dr. Tao Qin and Prof. Enhong Chen. Prior to that, he obtained the Bachelor’s degree from the same university in 2016.
His research focus on deep learning, automated machine learning and their applications to natural language processing, computer vision, speech processing. His representative works include : NAO, LightSpeech, NAS-BERT, BioGPT.
Yingce Xia
Senior Researcher at Microsoft Research
Yingce Xia is a principle researcher at Microsoft Research AI4Science. He received his BS and Ph.D degrees from University of Science and Technology of China in 2013 and 2018, supervised by Dr. Tie-Yan Liu and Prof. Nenghai Yu. His research areas include machine learning, drug discovery and natural language processing. His works, dual learning and deliberation learning, have been widely used in natural language processing and computer vision. He published around 50 papers in top conferences like ICML, NeurIPS, ICLR, CVPR, AAAI, etc. In 2019, his team won four champions on WMT’19 translation competition (English-to-German, German-to-English, German-to-French and French-to-German), and the techniques have been transferred to Microsoft translation system.
