Modern Machine Learning Language Models
Date:
In this invited talk for the Selenium Day I presented an overview of the technical innovations that have led to modern machine learning successes with language processing. This involved discussing what is special about processing text, the fundamentals of recurrent processing, the development of attention and self-attention models, and finally how this led to the Transformer architecture.