(s9_workshops_and_trainings:intro_to_distributed_computing)=
# Introduction to Distributed Computing Workshop

## Workshop Summary

This workshop covers the types of computational problems that distributed computing can address and key forms of communication between computing servers. The focus is on two widely-used forms of distributed computing: embarrassingly parallel processes, useful for tasks like hyperparameter sweeps, and Distributed Data Parallel (DDP) processes, which facilitate training machine learning models across multiple GPUs.

As part of the Workshops @ Kempner series, this interactive workshop provides a practical introduction to distributed computing in research settings. Topics include:

- What distributed computing is and when to use it
- Forms of communication between distributed systems
- Embarrassingly parallel processes using SLURM array jobs (e.g., for hyperparameter sweeps)
- Distributed Data Parallel (DDP) in PyTorch for training multi-layer perceptrons across multiple GPUs

### Prerequisites

- Basic knowledge of SLURM
- Familiarity with multi-layer perceptrons
- PyTorch and backpropagation knowledge is helpful but not required

## Workshop Slides 

To view the "Introduction to Distributed Computing" workshop slides, click the following link:

{download}`Introduction to Distributed Computing Workshop</_static/workshop/Kempner_Intro_to_Distributed_Computing_Workshop.pdf>`

<div style="text-align: center;">
  <iframe src="/_static/workshop/Kempner_Intro_to_Distributed_Computing_Workshop.pdf" width="90%" height="460px" style="border: none;">
  </iframe>
</div>