Building a Transformer from Scratch Workshop

23.5. Building a Transformer from Scratch Workshop#

23.5.1. Workshop Summary#

This workshop provides a practical, interactive way to learn about transformers by building a simple language model. Participants dive into model components, training pipelines, and the ingredients of a typical end-to-end language model set-up.

23.5.1.1. Prerequisites#

  • Familiarity with PyTorch framework

  • Familiarity with foundational machine learning topics, such as feedforward networks and gradient descent

23.5.2. Workshop Slides#

To download the “Building a Transformer from Scratch” workshop slides, click the link below.

KempnerLLM Distributed Training Workshop

Contents