uv Environment

3.5. uv Environment#

uv is a fast Python package and project manager. It can create a virtual environment, install Python packages, record direct dependencies in a pyproject.toml file, and lock exact dependency versions in a uv.lock file. This makes it useful for maintaining a separate, reproducible environment for each research project.

uv manages only Python packages. It does not replace the cluster’s software modules for system software such as CUDA, compilers, or MPI. Load those modules separately before you create and run the environment.

3.5.1. Installing `uv`#

First check whether uv is already available:

uv --version

If the command is not found, install uv with its official standalone installer:

curl -LsSf https://astral.sh/uv/install.sh | sh

Start a new shell and check the installation:

uv --version

If the command is still not found, add the standard user binary directory to your PATH and add the same line to ~/.bashrc:

export PATH="$HOME/.local/bin:$PATH"

Note

The standalone installation of uv does not require Conda or an existing Python installation. See the uv installation documentation for alternative installation and upgrade methods.

3.5.2. Creating an Environment for a Project#

The recommended uv workflow keeps the dependency declarations and lockfile in the project directory. The virtual environment is stored in a .venv directory by default.

Step 1: Enter an interactive compute session if required by cluster policy, then inspect the currently loaded modules:
```
module list
```
Step 2: If you want to begin with a clean module environment, purge the loaded modules and load any modules required by the project. For example:
```
module purge
module load python
```
Step 3: Create a new project and enter its directory:
```
uv init myproject
cd myproject
```
If you are working in an existing project that already contains pyproject.toml, enter that directory and do not run uv init.
Step 4: Select the Python version for the project:
```
uv python pin 3.12
```
uv records this choice in .python-version. If no suitable Python is found, uv downloads its own managed build by default. To use the cluster’s Python instead, create the environment with the loaded interpreter:
```
uv venv --python "$(which python)" --no-managed-python
```
Step 5: Add the packages required by the project:
```
uv add numpy pandas matplotlib
```
This command updates pyproject.toml and uv.lock, creates .venv if necessary, and installs the resolved packages.
Step 6: Run code in the project environment:
```
uv run python analysis.py
```
uv run checks that the environment agrees with the project files before running the command. You do not need to activate the environment.

To use the environment in the same way as a conventional Python virtual environment, activate it with:

source .venv/bin/activate

When finished, deactivate it with:

deactivate

Tip

Prefer uv add <package> for project dependencies. Unlike uv pip install <package>, it records the dependency in pyproject.toml and updates uv.lock, allowing collaborators and batch jobs to reproduce the environment.

3.5.2.1. Managing Packages#

Add a runtime dependency, optionally with a version constraint:

uv add scipy
uv add "numpy>=2.0,<3"

Add a development-only dependency:

uv add --dev pytest ruff

Remove a dependency:

uv remove scipy

Inspect the resolved dependency tree:

uv tree

Upgrade all dependencies within the constraints in pyproject.toml, or upgrade one package:

uv lock --upgrade
uv lock --upgrade-package numpy
uv sync

uv also provides a pip-compatible interface for an existing virtual environment. This can be useful for a project that only supplies a requirements.txt file:

uv venv
uv pip install -r requirements.txt

This pip-compatible workflow does not add dependencies to pyproject.toml. For a project you maintain, import the requirements into the project metadata instead:

uv add --requirements requirements.txt

3.5.4. Using the Environment in a Slurm Job#

Create and test the environment before submitting a long-running job. In the job script, load the same non-Python modules used when the environment was created, enter the project directory, and use uv run:

#!/bin/bash
#SBATCH --job-name=uv-example
#SBATCH --time=00:30:00
#SBATCH --mem=4G

module purge
module load python
export PATH="$HOME/.local/bin:$PATH"

cd /path/to/myproject
uv run --locked python analysis.py

If the environment has already been synchronized and compute nodes cannot access the package index, prevent uv from accessing the network:

uv run --locked --offline python analysis.py

Warning

The offline command succeeds only when the required Python installation, environment, and packages are already available. Run uv sync --locked in the project directory before submitting the job.

3.5.5. Storing Environments and the Cache Outside the Home Directory#

By default, the project environment lives at <project>/.venv and the shared cache under ~/.cache/uv. Both can consume substantial storage and many inodes. For large projects, keep the project and its .venv in an appropriate lab directory and move the cache to that filesystem as well:

mkdir -p /n/holylabs/LABS/<lab_name>/Users/<username>/.cache/uv
export UV_CACHE_DIR=/n/holylabs/LABS/<lab_name>/Users/<username>/.cache/uv

Add the UV_CACHE_DIR setting to ~/.bashrc if you want it to apply in future shells, and verify it with:

uv cache dir

Keeping the cache and virtual environment on the same filesystem gives uv the best chance to use links instead of copying package files. You can remove cache data that is no longer needed with:

uv cache prune

To check the cache size, clear it entirely, or clear the entries for a single package:

du -sh $(uv cache dir)   # show how much space the cache is using
uv cache clean           # remove all cache entries
uv cache clean ruff      # remove cache entries for one package

Warning

Scratch storage is periodically purged. It can be suitable for a disposable cache, but do not rely on it as the only location for project source code or lockfiles. If .venv is purged, recreate it with uv sync --locked.

3.5.6. Troubleshooting#

3.5.6.1. Confirm Which Python and Environment Are Being Used#

uv run which python
uv run python --version
uv run python -c "import sys; print(sys.executable)"

The executable should normally be <project>/.venv/bin/python.

3.5.6.2. Recreate a Broken Environment#

Because .venv is generated from the project files, it can be removed and recreated. From the project directory, move the existing environment out of the way and synchronize again:

mv .venv .venv.old
uv sync --locked

After confirming that the new environment works, delete .venv.old.

3.5.6.3. Packages Requiring CUDA, MPI, or Compilers#

Load the required cluster modules before running uv add, uv sync, or the project. Some Python packages have no pre-built wheel and must be compiled locally, which may require compiler and system-library modules. If a package depends heavily on non-Python libraries, a Conda environment, a container, or the cluster’s software modules may be more appropriate than a uv-only environment.