(s9_workshops_and_trainings:llm_distributed_training)=
# Large Language Model Distributed Training Workshop


## Workshop Summary

This workshop highlights parallelization techniques for training large language models.  The workshop covers techniques such as Distributed Data Parallelism (DDP), Model Parallelism (MP), Tensor Parallelism (TP), Pipeline Parallelism (PP), and Fully Sharded Data Parallelism (FSDP). In addition to reviewing the advantages of each technique and their use cases, this workshop provides hands-on examples to help with understanding LLM distributed training approaches.

### Prerequisites
- Familiarity with PyTorch framework and Python programming
- Familiarity with LLMs
- Familiarity with High Performance Computing (HPC) cluster

## Workshop Slides 

To download the "Large Language Model Distributed Training" workshop slides, click the link below.

{download}`Kempner LLM Distributed Training Workshop </_static/workshop/Kempner_LLM_Distributed_Training_Workshop.pdf>`

<div style="text-align: center;">
 <iframe src="/_static/workshop/Kempner_LLM_Distributed_Training_Workshop.pdf" width="90%" height="460px" style="border: none;"></iframe>
</div>