In this talk we will discuss the recent rise in popularity of sparse expert models (e.g. mixture of experts) and their implications for the NLP field. Spasity allows for networks to have inputs to get different subsets of the model weights, which allows for vastly larger models that have smaller computational footprints. These networks have recently achieved state-of-the-art results on many famous NLP benchmarks while showing tremendous computational savings. We discuss the recent research advances and the future of what these models achieve and why it has large implications for the way machine learning models are used.