11. Coding Agent Skills#
This chapter turns the other Software Engineering for Research chapters into skills for coding agents such as Claude Code and Codex. A skill is a SKILL.md file of instructions that an agent loads when a task matches the skill’s description. With these skills installed, your agent follows the practices in this handbook when it works on your code. To learn how skills work, see Building Custom Tools and MCP Servers.
Skills are instructions, not limits. For limits that must hold, use the permission rules and hooks in Guardrails.
11.1. All skills#
Skill |
Chapter |
Use it to |
|---|---|---|
Branch, commit, review, and merge with Git and GitHub, and handle secrets and licenses |
||
Structure, refactor, and review research code |
||
Write docstrings, comments, READMEs, and citation files |
||
Write tests, measure coverage, and set up CI |
||
Package, version, and release Python code |
||
Pin environments, seed runs, version data, and archive results |
Each skill file is at https://handbook.eng.kempnerinstitute.harvard.edu/skills/<skill-name>/SKILL.md. If you are not sure which to pick, install all six. An agent reads only each skill’s name and description until a task needs the rest.
11.2. Install a skill#
Claude Code and Codex read the same SKILL.md format, the open Agent Skills standard, from different folders. Each skill goes in its own folder there, as <skill-name>/SKILL.md.
Agent |
For all your projects |
For one project |
|---|---|---|
Claude Code |
|
|
Codex |
|
|
Install the skills on the machine where the agent runs, such as your computer or the cluster. On the cluster, your home directory is the same on every node, so skills you install for all your projects work in every job. To share a project’s skills with everyone who clones the repository, commit its skills folder.
Note
An agent follows a skill’s instructions, so read a skill before you install it, as you would any agent configuration; see Untrusted repositories. The skills on this page contain instructions only, with no scripts.
11.2.1. Ask your agent#
Point the agent at this page. Name the skills you want, or ask for all of them. Ask the agent to download each file with curl. A web-fetch tool can return a summary instead of the file.
Install the testing-and-continuous-integration and reproducible-research skills
for all my projects. Download each SKILL.md with curl, using the links on this page:
https://handbook.eng.kempnerinstitute.harvard.edu/s2_swe_for_research/coding_agent_skills.html
To install the skills for one project only, write “for this project” instead of “for all my projects”. The agent may ask you to approve the downloads. When it finishes, load the skills as described in Use a skill.
If your agent runs in a sandbox, such as the one in Agent sandboxing, it may not be able to download the files or write to its skills folder. In that case, install the skills by hand.
11.2.2. Install by hand#
Set dest to a folder from the table. The list holds all six skills; delete the lines for skills you do not want. For one project, run the commands from the repository’s top folder.
dest=~/.claude/skills # for one project: dest=.claude/skills
base=https://handbook.eng.kempnerinstitute.harvard.edu/skills
skills=(
collaborative-code-development
software-design-principles
documentation-and-readability
testing-and-continuous-integration
package-development
reproducible-research
)
for skill in "${skills[@]}"; do
mkdir -p $dest/$skill
curl -fsSL $base/$skill/SKILL.md -o $dest/$skill/SKILL.md || echo "Failed: $skill"
done
ls $dest/*/SKILL.md
dest=~/.agents/skills # for one project: dest=.agents/skills
base=https://handbook.eng.kempnerinstitute.harvard.edu/skills
skills=(
collaborative-code-development
software-design-principles
documentation-and-readability
testing-and-continuous-integration
package-development
reproducible-research
)
for skill in "${skills[@]}"; do
mkdir -p $dest/$skill
curl -fsSL $base/$skill/SKILL.md -o $dest/$skill/SKILL.md || echo "Failed: $skill"
done
ls $dest/*/SKILL.md
The loop prints nothing for a skill that installs, and ls then lists each installed SKILL.md file. If every skill fails, check your connection. If one skill fails, check the spelling of its name. Then run the commands again. A failed first install leaves an empty folder, which you can delete. To update the skills when the chapters change, run the loop again. It replaces the files, including any edits you made. To remove a skill, delete its folder.
11.3. Use a skill#
To |
Claude Code |
Codex |
|---|---|---|
List installed skills |
Run |
Run |
Use a skill automatically |
Ask for a task that matches its description |
Ask for a task that matches its description |
Call a skill by name |
Type |
Type |
Load new skills |
Start a new session, or run |
Picked up automatically; if a skill does not appear, restart Codex |
Each skill tells the agent to follow your project’s own conventions first, such as a CONTRIBUTING file or agent instructions. The skills for Git, packaging, and reproducibility also list actions that the agent takes only when you ask, such as pushing, publishing, deleting, or starting long runs. Project conventions do not override that list, but your own requests count as asking, whether you type them or write them in your agent instructions.
11.4. Skills by chapter#
Open a skill to read it. Each one links to its chapter for the full explanation.
Collaborative Code Development
The collaborative-code-development skill, from Collaborative Code Development. Download its SKILL.md.
---
name: collaborative-code-development
description: Use when committing, branching, resolving a merge conflict, preparing or reviewing a pull request, or setting up a shared research repository with Git and GitHub. Covers branches, commit messages, pull requests, code review, merge conflicts, CI, shared environments, formatting and linting, secrets, licenses, and integrations such as Zenodo. From the Kempner Institute Computing Handbook.
---
# Collaborative Code Development
Use these practices when you work with Git and GitHub on research code. They summarize the Collaborative Code Development chapter of the Kempner Institute Computing Handbook: https://handbook.eng.kempnerinstitute.harvard.edu/s2_swe_for_research/collaborative_code_development.html
If the project has its own conventions, for example in its README, a CONTRIBUTING file, or agent instructions, follow those first. Project conventions do not override the Ask first rules. A request from the user, typed or written in the user's own agent instructions, counts as asking.
## Ask first
- Commit, push, open issues or pull requests, merge, create tags or releases, change a repository's visibility, or rewrite history only when the user asks.
- Never rebase or force-push a branch that others have already pulled.
- Ask the user which license to use. Do not choose one.
## Branches
- Start from an up-to-date `main`: `git checkout main && git pull origin main`.
- Work on a short-lived branch named after its issue and the author's initials, so `main` stays stable. For example, `iss42_add_data_loader_js` is for issue 42 by an author with the initials js.
- Default to GitHub Flow: one branch per issue, a pull request, review, then merge. External contributors without write access use fork and pull.
- Merge on shared branches. Rebase only to tidy a local branch before you merge it.
## Commits
- Write the subject in the imperative mood, so it completes "If applied, this commit will ...". Capitalize it and leave off the trailing period, for example `Fix data loading error with empty CSV files`.
- Add a body that explains what changed and why, and reference the issue, for example `Closes #23`.
- Keep one logical change per commit.
- When a commit includes work by other people, add a `Co-authored-by: Name <email>` trailer for each person, after a blank line at the end of the message.
## Pull requests and review
- Send every change, even a one-line fix, through a pull request.
- When the user asks for a pull request, keep it focused, under about 400 lines where possible, and link the issue it closes.
- In review, check correctness and edge cases, readability and naming, and reproducibility: pinned dependencies, seeds set, and no hard-coded local paths. Suggest improvements rather than only flagging problems.
## Merge conflicts
- Fetch or pull the latest `main` before you start and before you push, so branches stay close.
- Keep features in separate files or modules with clear interfaces, so two people rarely edit the same lines.
- Commit or stash work before a merge or rebase. A clean working tree lets `git merge --abort` or `git rebase --abort` restore the previous state.
To resolve a conflict:
1. Edit the file to the intended result.
2. Delete the `<<<<<<<`, `=======`, and `>>>>>>>` markers.
3. Run `git add` on the file.
4. Finish with `git commit` or `git rebase --continue`.
## Environment and style
- Use one isolated environment per project, and one environment tool for the team: pip with `venv`, conda or mamba, uv, or Poetry.
- Pin exact versions in a committed spec or lock file (`requirements.txt`, `environment.yml`, `uv.lock`, or `poetry.lock`), and rebuild the environment from it.
- Use one formatter and one linter with a committed config, for example `ruff format .` and `ruff check .` with a `[tool.ruff]` table in `pyproject.toml`. Run the same checks in pre-commit and in CI.
- Reformat an existing codebase once, in its own commit, to keep that noise out of later reviews.
- Use one test framework for the project; pytest is a common default.
## CI and feature flags
- Add a GitHub Actions workflow under `.github/workflows/` that installs the project and runs the tests on every push and pull request. Add linting, type checks, and coverage as the project grows.
- Merge unfinished work behind a feature flag that is off by default, for example one read from an environment variable. Test with the flag both off and on, and delete the flag once the feature is permanent.
## Repository basics
- Include a README (purpose, setup, usage), a dependency spec, a `.gitignore`, a `CONTRIBUTING.md`, and a `LICENSE`.
- Keep unpublished research in a private repository. Suggest making it public once the work is published or disseminated.
## Security and licensing
- Never commit secrets such as API keys, tokens, or passwords. List secret files in `.gitignore`, and load secrets at runtime from environment variables or a secrets manager.
- Scan for leaked secrets with GitHub secret scanning or `gitleaks git .`, and scan dependencies with Dependabot or `pip-audit`.
- Treat a secret that was ever committed as compromised, because it stays in Git history. Tell the user to rotate or revoke it.
- Check that the licenses of dependencies and reused code are compatible with the project's license, and credit code you adapt.
- Follow institutional and funder policies for sensitive, restricted, or human-subjects data.
## Live collaboration
- Do not host a VS Code Live Share or JupyterLab collaboration session on the Kempner AI cluster, because guests could run code under the host's cluster account. Host it on a personal computer, or share the code through Git.
- Commit work from a live session and send it through the normal pull request workflow.
## Integrations
- For a citable DOI, connect a public, licensed repository to Zenodo. Zenodo archives each GitHub release and mints a DOI for it.
- Read the Docs builds and hosts documentation, and Codecov or Coveralls reports coverage from CI. Status badges in the README show build, coverage, and DOI status.
Software Design Principles
The software-design-principles skill, from Software Design Principles. Download its SKILL.md.
---
name: software-design-principles
description: Use when designing, writing, refactoring, or reviewing the structure of research code, or when turning a script into reusable modules. Covers KISS, DRY, YAGNI, separation of concerns, modules with small interfaces, pure functions, dependency injection, the strategy and factory patterns, adapters for external libraries, deterministic design, APIs and CLIs, and layered architecture. From the Kempner Institute Computing Handbook.
---
# Software Design Principles
Use these principles when you design, write, refactor, or review research code. They summarize the Software Design Principles chapter of the Kempner Institute Computing Handbook: https://handbook.eng.kempnerinstitute.harvard.edu/s2_swe_for_research/software_design_principles.html
If the project has its own conventions, for example in its README, a CONTRIBUTING file, or agent instructions, follow those first. Treat these principles as defaults, not rigid rules.
## Core principles
- Keep it simple (KISS). Prefer the simplest design that solves the current problem. Clear code that is a little longer beats clever code.
- Do not repeat yourself (DRY). When the same logic appears in several places, extract one function and reuse it.
- Build only what the current analysis needs (YAGNI). Add an extension point when a second real use case appears, not before.
- Separate concerns, such as data loading, computation, and plotting, and give each function, class, or module one job.
## Modules and interfaces
- Group related logic into cohesive modules that talk to each other only through small, stable interfaces.
- Hide implementation details, so callers depend on what a function does, not on how it does it.
- Draw module boundaries around the decisions most likely to change, such as a file format, a storage backend, or a numerical method.
- Check each boundary: you should be able to say what the module does in one sentence without saying how it works.
## Testable code
- Put the core logic in pure, deterministic functions: same input, same output, no side effects.
- Keep file access, network calls, randomness, and printing in thin wrappers at the edges.
- Pass collaborators in, such as a client, a model, or a clock, instead of creating them inside a function, so a test can pass a stand-in.
- Avoid hidden global mutable state. Pass state in as arguments.
- Keep functions short, with one clear job.
## Reuse and extension
- Pass paths, thresholds, and constants in as parameters instead of hard-coding them.
- Add behavior by passing in a function (the strategy pattern) instead of growing an `if`/`elif` chain.
- Use a small factory when a config value or CLI argument picks the object to build. Map each name to a class in a dictionary, and raise a clear `ValueError` for an unknown name.
- Use a design pattern only when it removes real, present pain. In Python, a plain function is often enough.
- Move code that several projects reuse into an installable package.
## External dependencies
- Make the core logic depend on an interface you define, for example a `typing.Protocol`, not on a specific library client or file format.
- Wrap each external library or service in one thin adapter, so swapping a backend changes one file.
- In tests, pass a simple fake that implements the same interface.
## Reproducible design
- Seed randomness explicitly with a local generator, `numpy.random.default_rng(seed)`, not the legacy global `numpy.random.seed`.
- Pass the seed and other settings in as parameters or read them from a config file. Do not bury magic numbers in function bodies.
- Avoid logic whose output depends on iteration order, wall-clock time, or accumulated state.
- Save the parameters, seed, and code version with each result.
## Documentation as design
- Write the signature and docstring before the body. Per PEP 257, the docstring is the contract: behavior, arguments, return values, side effects, exceptions raised, and restrictions on when to call it.
- If a unit is hard to document without many caveats, simplify or split it.
- Record significant design decisions in a short Architecture Decision Record: context, decision, and consequences.
## APIs and CLIs
- Use clear, consistent names, and keep the public surface small, with sensible defaults.
- Prefix internal names with one underscore, and declare the public API with `__all__`.
- Validate inputs, and raise specific exceptions with messages that say what was wrong and how to fix it.
- Build CLIs with `argparse`, or Click or Typer for larger tools. Give every argument help text, and give options defaults so a basic run is short.
- Keep CLI parsing thin: parse the arguments, log them, then call a normal function that does the work.
- Exit with 0 on success and a non-zero code on failure.
## Code style and layers
- Default to functions plus simple data containers such as a `dataclass`. Add a class when several operations share and update the same state. Avoid deep inheritance.
- For larger tools, use four layers:
- Interface: a CLI, notebook, or API call.
- Orchestration: the steps of a run.
- Core science: models and numerical methods.
- Infrastructure: files, storage, and services.
- Keep the core science free of I/O and UI: no file paths, network calls, argument parsing, or printing. The core imports neither the interface nor the infrastructure layer.
- Do not add layers to a short script.
## Refactoring
- Refactor in small steps that keep behavior unchanged, and run the tests after each step.
- Keep refactoring commits separate from commits that change behavior.
- Treat code smells, such as duplicated code, long functions, and unclear names, as prompts to look for an improvement.
Documentation and Readability
The documentation-and-readability skill, from Documentation and Readability. Download its SKILL.md.
---
name: documentation-and-readability
description: Use when writing or reviewing docstrings, comments, a README, or project documentation, or when making research code easier to read. Covers READMEs, PEP 257 docstrings in Google or NumPy style, comments that explain why, PEP 8 naming, small functions and guard clauses, type hints, doctest, Sphinx and MkDocs, data dictionaries, and CITATION.cff files. From the Kempner Institute Computing Handbook.
---
# Documentation and Readability
Use these practices when you write or review code, comments, or documentation in a research project. They summarize the Documentation and Readability chapter of the Kempner Institute Computing Handbook: https://handbook.eng.kempnerinstitute.harvard.edu/s2_swe_for_research/documentation_and_readibility.html
If the project has its own conventions, for example in its README, a CONTRIBUTING file, or agent instructions, follow those first. Keep the docstring style the project already uses.
## Readable code
- Use descriptive names that reveal intent. Avoid ambiguous abbreviations and single-letter names, except for short loop counters.
- Follow PEP 8: `snake_case` for functions and variables, `UPPER_CASE` for constants, and lines of at most 79 characters, or the limit the team agreed on, such as 99.
- Keep functions small and focused on one thing. Reconsider a function once it grows past about 40 lines.
- Return early on invalid input and edge cases (guard clauses), so the main logic is not deeply nested.
- Replace magic numbers with named constants, for example `DISCOUNT_RATE = 0.9`.
- Make behavior match the name. A function named like a lookup, such as `get_user`, must not write files or change its input.
- Let formatters and linters enforce the style guide.
## Comments and docstrings
- Write comments that explain why: intent, assumptions, and non-obvious reasoning. Do not repeat what the code says.
- Give each public module, function, class, and method a docstring as its first statement (PEP 257).
- Use one structured docstring style across the project, Google style or NumPy style, and document parameters, return values, and raised exceptions.
- Add type hints (PEP 484) to public interfaces. They document types for static checkers, but Python does not enforce them at runtime.
- Add a short `>>>` example where it helps, and check it with `python -m doctest example.py`.
## README and project docs
- Make the README the front door: a one-sentence description, installation, a minimal runnable example, and how to reuse and cite the work. Keep it short and link out for depth.
- Start with what the code is for and why, then the common case. Put options and edge cases later.
- Write for a reader who does not share your context, and state your assumptions.
- For larger docs, separate the four Diátaxis types: tutorials, how-to guides, reference, and explanation.
- Build API reference from docstrings with Sphinx (autodoc, plus napoleon for Google or NumPy style), or with MkDocs and mkdocstrings. Read the Docs can build and host the docs on each push.
## Research documentation
- Record the method, parameter values and ranges, random seeds, software versions, and assumptions behind each result.
- Document each dataset with a README or data dictionary that lists every variable with its meaning, units, allowed values, and provenance. Aim for FAIR data: Findable, Accessible, Interoperable, and Reusable.
- Add a `CITATION.cff` file at the repository root, and update its `version` and `doi` for each archived release. Add a `doi` only after Zenodo or another archive mints one. Never invent a DOI.
```yaml
cff-version: 1.2.0
message: "If you use this software, please cite it as below."
title: "Example Analysis Toolkit"
authors:
- family-names: Smith
given-names: Jane
version: 1.0.0
doi: 10.5281/zenodo.1234567 # the DOI the archive minted for this release
```
## Before you finish
- New and changed public functions and classes have docstrings.
- The README still matches any setup or usage you changed.
- New dependencies are documented.
- Suggest clearing notebook outputs before a commit, but do not clear them without asking.
Testing and Continuous Integration
The testing-and-continuous-integration skill, from Testing and Continuous Integration. Download its SKILL.md.
---
name: testing-and-continuous-integration
description: Use when writing, running, or reviewing tests, measuring coverage, or setting up CI for a research repository. Covers pytest tests, fixtures, parametrize, mocking and monkeypatch, Hypothesis, coverage with pytest-cov, GitHub Actions workflows, and research checks such as float tolerances, seeded randomness, and regression tests against saved references. From the Kempner Institute Computing Handbook.
---
# Testing and Continuous Integration
Use these practices when you write, run, or review tests, or set up CI, for research code. They summarize the Testing and Continuous Integration chapter of the Kempner Institute Computing Handbook: https://handbook.eng.kempnerinstitute.harvard.edu/s2_swe_for_research/testing_and_continuous_integration.html
If the project has its own conventions, for example in its README, a CONTRIBUTING file, or agent instructions, follow those first. Use the project's existing test framework; for a new project, use pytest.
## What to test
- Write many fast unit tests, fewer integration tests, and a few end-to-end tests (the test pyramid). Test each behavior at the lowest level that can check it.
- Unit-test the building blocks, such as a normalization function, a distance metric, or a data parser.
- When you fix a bug, add a regression test that fails on the old behavior and passes on the new one.
- Test data loading and preprocessing on small fixtures, including missing values and malformed rows.
- Cover edge cases and failure modes: empty input, boundary values, and invalid arguments. Use `pytest.raises` to check that an error is raised.
## How to write tests
- Structure each test as Arrange, Act, Assert, and check one behavior per test.
- Name each test for the behavior it checks, for example `test_normalize_scales_values_to_sum_to_one`.
- Keep tests independent and deterministic: no shared mutable state, no dependence on run order, seeded randomness, and fixed timestamps instead of the wall clock.
- Replace network calls, databases, and slow models with stand-ins, using `unittest.mock` (`Mock`, `patch`) or the pytest `monkeypatch` fixture.
- Use `@pytest.fixture` for reusable setup, and `@pytest.mark.parametrize` instead of copying a test to change its inputs.
- Use Hypothesis (`@given` with strategies) for invariants that must hold for all inputs, such as an output shape, a valid range, or a conserved quantity.
- Test behavior through the public interface, not implementation details.
## Research-specific checks
- Compare floats with a tolerance, never `==`. Use `pytest.approx` for single numbers (defaults: relative 1e-6, absolute 1e-12) and `numpy.testing.assert_allclose` for arrays (defaults: `rtol=1e-7`, `atol=0`). Set `atol` explicitly when expected values can be zero.
- Seed generators, for example `numpy.random.default_rng(seed)`, and assert on properties that hold for any draw, such as shape, range, or a mean within bounds, rather than on exact random values.
- Prefer small inputs whose answers you can work out by hand or analytically.
- For computations too large to check by hand, save a known-good output once and compare later runs against it with a tolerance.
```python
import numpy as np
from numpy.testing import assert_allclose
def softmax(x):
e = np.exp(x - np.max(x)) # subtract max for numerical stability
return e / e.sum()
def test_softmax_matches_known_values():
result = softmax(np.array([0.0, 0.0]))
assert_allclose(result, [0.5, 0.5]) # tolerance-based, not ==
assert_allclose(result.sum(), 1.0) # probabilities sum to one
```
## Coverage
- Measure coverage with pytest-cov: `pytest --cov=mypackage --cov-report=term-missing`. Add `--cov-branch` for branch coverage.
- Use the report to find untested code that matters, such as error handlers and rarely hit branches. Do not chase 100%: coverage measures what ran, not whether the results were checked.
## Continuous integration
- Run the tests on every push and pull request with a GitHub Actions workflow in `.github/workflows/ci.yml`.
- Use a matrix for the Python versions the project supports, and cache pip downloads.
- Run quick checks, such as linting and unit tests, before slower ones, and keep CI fast.
- Suggest branch protection that makes the CI check required before a merge.
```yaml
# .github/workflows/ci.yml
name: CI
on: [push, pull_request]
jobs:
test:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest]
python-version: ["3.11", "3.12", "3.13"]
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
with:
python-version: ${{ matrix.python-version }}
cache: pip
- run: pip install -e ".[test]"
- run: pytest
```
The workflow installs a `test` extra. Declare it in `pyproject.toml`, for example `test = ["pytest", "pytest-cov"]` under `[project.optional-dependencies]`, or change the install step to match the project.
## Before you finish
- Run the test suite and report the result.
- On the Kempner AI cluster, run tests on a compute node in a SLURM job, never on a login node. Ask before you start a long or GPU test run.
- New or changed behavior has tests, including edge cases.
- Tests are deterministic and pass when run alone or in any order.
Package Development
The package-development skill, from Package Development. Download its SKILL.md.
---
name: package-development
description: Use when turning research code into an installable Python package, editing pyproject.toml, or preparing a release. Covers the src layout, pyproject.toml metadata, build backends, python -m build, editable installs, testing the installed package with tox or nox, publishing with twine through TestPyPI, optional dependency extras, package data with importlib.resources, semantic versioning, Git tags, changelogs, and SPDX license fields. From the Kempner Institute Computing Handbook.
---
# Package Development
Use these practices when you turn research code into an installable Python package, change its packaging, or prepare a release. They summarize the Package Development chapter of the Kempner Institute Computing Handbook: https://handbook.eng.kempnerinstitute.harvard.edu/s2_swe_for_research/package_development.html
If the project has its own conventions, for example in its README, a CONTRIBUTING file, or agent instructions, follow those first. Project conventions do not override the Ask first rules. A request from the user, typed or written in the user's own agent instructions, counts as asking. Keep the project's build backend and layout unless the user asks you to change them.
## Ask first
- Upload to TestPyPI or PyPI, push tags, or create a GitHub release only when the user asks.
- Ask the user to run each upload in their own terminal, where twine prompts for the API token. Do not ask for, print, or write the token into a file.
- Ask the user which license to use. Do not choose one.
## When to package
- Package code once it is reused across projects or shared with a paper. Keep the package small and focused; not every analysis needs to become a library.
## Layout and metadata
- Use the src layout: importable code in `src/<package_name>/` with an `__init__.py`, tests in a top-level `tests/` directory, and `pyproject.toml`, `README.md`, and `LICENSE` at the root. The import name uses underscores, for example `my_package` for the project `my-package`.
- In `pyproject.toml`, declare the build backend in `[build-system]` (for example setuptools, hatchling, flit-core, or pdm-backend) and the metadata in `[project]`: `name` (required, and never dynamic), `version`, and `dependencies`.
- Set `readme` to the README file, which becomes the PyPI project page, and add `[project.urls]` links such as Documentation and Source.
- Put heavy or optional dependencies in `[project.optional-dependencies]` extras, such as `gpu` or `viz`, so the default install stays light.
- Ship small data files inside the package, and read them with `importlib.resources` instead of hard-coded paths.
```toml
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[project]
name = "my-package"
version = "0.1.0"
readme = "README.md"
license = "MIT" # the license the user chose
license-files = ["LICENSE"]
dependencies = ["numpy", "requests>=2.0"]
[project.optional-dependencies]
viz = ["matplotlib"]
[project.urls]
Documentation = "https://my-package.readthedocs.io/"
Source = "https://github.com/owner/my-package"
```
```python
from importlib.resources import files
content = files("my_package").joinpath("data.csv").read_text(encoding="utf-8")
```
## Build and test
- For development, install the project in editable mode with `pip install -e .`, and run `pytest` from the project root.
- Build the sdist and wheel with `python -m build` only when the package is ready to share.
- Test the installed package in clean environments, across the Python versions you target, with tox or nox. Run the same check in CI.
## Publish
When the user asks you to publish:
1. Build the sdist and wheel with `python -m build`.
2. Ask the user to upload to TestPyPI first: `python -m twine upload --repository testpypi dist/*`.
3. In a fresh virtual environment, check that the package installs from TestPyPI and imports: `pip install --index-url https://test.pypi.org/simple/ --no-deps my-package`, then `python -c "import my_package"`. An environment with the editable install already has the package, so it proves nothing.
4. Ask the user to upload to PyPI: `python -m twine upload dist/*`.
- Uploads use an API token, not a password: the username is `__token__`, and the password is the token, including its `pypi-` prefix. TestPyPI needs its own account and token.
- A released version cannot be reused on PyPI. To ship a fix, bump the version and build fresh artifacts.
- For a package that provides a command-line tool, users can install it with `pipx install`.
## Versions and releases
- Use semantic versioning, `MAJOR.MINOR.PATCH`: MAJOR for breaking API changes, MINOR for backward-compatible features, and PATCH for backward-compatible fixes.
- Keep the version in one place: the `version` field in `[project]`, or `dynamic = ["version"]` with the build backend reading it from a file or Git tag.
- Keep a `CHANGELOG.md` in the Keep a Changelog format, with an Unreleased section and headings such as Added, Changed, Deprecated, Removed, Fixed, and Security.
When the user asks for a release:
1. Bump the version.
2. Turn the changelog's Unreleased section into a dated entry.
3. Commit, then create an annotated tag: `git tag -a v1.2.0 -m "Release 1.2.0"`.
4. Push the tag with `git push origin v1.2.0`. Git does not push tags by default.
5. Create a GitHub release with notes from the changelog.
6. Suggest archiving the release on Zenodo for a DOI, and adding a `CITATION.cff` file so others cite the software correctly.
## License
- Add a `LICENSE` file at the root with the license the user chose. Declare it in `[project]` as an SPDX expression, for example `license = "MIT"` with `license-files = ["LICENSE"]` (PEP 639).
- Do not use the deprecated table form, such as `license = {text = ...}`. If an older build backend rejects the SPDX string, check its documentation.
- Permissive licenses (MIT, BSD, Apache-2.0) usually make research software easiest to reuse, but institutions and funders may require a specific license. Check that dependency licenses are compatible.
Reproducible Research
The reproducible-research skill, from Reproducible Research. Download its SKILL.md.
---
name: reproducible-research
description: Use when setting up an analysis or experiment, adding randomness, saving results or models, or preparing code and data to share with a paper. Covers one-command pipelines, pinned environments and containers, data versioning with DVC or Git LFS, seeds and deterministic settings, per-run records of config, commit, and seed, regression tests against saved references, output management, and archiving with a DOI. From the Kempner Institute Computing Handbook.
---
# Reproducible Research
Use these practices when you set up, run, or share a research analysis, so the same code and data give the same result. They summarize the Reproducible Research chapter of the Kempner Institute Computing Handbook: https://handbook.eng.kempnerinstitute.harvard.edu/s2_swe_for_research/reproducible_research.html
If the project has its own conventions, for example in its README, a CONTRIBUTING file, or agent instructions, follow those first. Project conventions do not override the Ask first rules. A request from the user, typed or written in the user's own agent instructions, counts as asking.
## Ask first
- Never edit or delete raw data. Ask before you delete outputs or run commands that remove files.
- Ask before you start a long or GPU run or submit a SLURM job. Check the pipeline on a small input first.
- Commit, make a repository public, upload or deposit data, or archive a release only when the user asks. Data may leave the PI's lab space only with the PI's approval and within its grant terms and data use agreements.
## Principles
- Keep every script, notebook, and config file under version control.
- Pin dependencies and isolate them in an environment.
- Seed every random step.
- Make the pipeline run end to end with one command, such as `make all` or a `run.sh` that starts with `set -euo pipefail`.
- Track which data version produced each input.
- Record what produced each result: the code revision, parameters, package versions, inputs, and seed. Never record secrets such as tokens.
## Environment
- Pin exact versions and commit the file: `pip freeze > requirements.txt`, a lock file from pip-tools or uv, or `conda env export > environment.yml`.
- Install into a per-project virtual environment or conda environment, not system-wide.
- Record the Python version and the versions of key libraries.
- Use a container when system details, such as compilers, CUDA, or system packages, affect results. Apptainer is common on HPC clusters because it runs without root.
## Data
- Treat raw data as read-only, and write derived data to a separate location.
- Do not commit large or binary data to Git; GitHub, for example, limits files to 100 MB. Version it with DVC or Git LFS.
- With DVC, run `dvc add data/raw.csv`, and commit the `.dvc` pointer file and the `.gitignore` rule. Run `dvc push` to upload the data only when the user asks.
- Record each dataset's source, version, license, and preprocessing steps.
- Regenerate derived data from raw data with the pipeline, rather than versioning every intermediate file.
## Randomness
- Seed every random number generator in use: Python's `random`, NumPy, and the framework, for example `torch.manual_seed(seed)`.
- Prefer local generators, such as `numpy.random.default_rng(seed)` or a `torch.Generator`, over global state.
- When the iteration order of sets matters, set `PYTHONHASHSEED` to a fixed integer before Python starts, for example in `run.sh` or the job script. Setting it inside a running script has no effect.
- When exact reproducibility matters in PyTorch, call `torch.use_deterministic_algorithms(True)` and set `torch.backends.cudnn.deterministic = True` and `torch.backends.cudnn.benchmark = False`. This can be slower, and results can still differ across releases, platforms, and CPU versus GPU.
- Log the seed with the results.
## Experiment records
- Put parameters in a config file, such as YAML or JSON, rather than editing the source for each run.
- For each run, save the resolved config, the Git commit hash, and the seed in the run's own output folder. You can also log them with MLflow or Weights & Biases (W&B).
- Keep a project README or lab notebook of what was tried, what worked, and why.
```python
import subprocess, shutil, json, yaml
from datetime import datetime
from pathlib import Path
cfg = yaml.safe_load(open("config.yaml")) # all parameters come from the config
commit = subprocess.check_output( # the exact code revision
["git", "rev-parse", "HEAD"], text=True).strip()
stamp = datetime.now().strftime("%Y-%m-%d_%H%M%S")
out = Path("runs") / f"{stamp}_{commit[:8]}" # one folder per run
out.mkdir(parents=True)
shutil.copy("config.yaml", out / "config.yaml") # save the resolved config
(out / "run_meta.json").write_text( # plus commit and seed
json.dumps({"commit": commit, "seed": cfg["seed"]}, indent=2))
```
## Outputs
- Treat models, figures, and tables as outputs you regenerate from code, data, and config. Do not edit them by hand.
- Keep generated outputs out of Git, for example with `outputs/` in `.gitignore`. Version large artifacts with DVC or object storage.
- Use one output folder per run, with machine-readable names that include a version or an ISO 8601 date (`YYYY-MM-DD`).
- Save a metadata file next to each artifact with the commit, data version, and seed.
## Check reproducibility
- Run the pipeline twice on a small input on the same machine, and compare the outputs. Then run it again in a clean environment rebuilt from the lock file.
- Add a regression test that runs the pipeline on a small input and compares the result with a saved reference, using `numpy.testing.assert_allclose` or `pytest.approx` rather than exact equality.
- When you change behavior on purpose, inspect the difference and confirm the new output is correct. Then regenerate the reference, and commit it as a deliberate update when the user asks.
- Run the pipeline end to end on a small input in CI.
## Share and archive
- Publish the code in a public repository, with a `LICENSE` file for the license the user chose, once the work is ready to share.
- Archive a tagged GitHub release on Zenodo to get a DOI, and cite that release's DOI in the paper.
- Deposit data in a repository that gives it a DOI, such as Zenodo, Figshare, Dryad, Dataverse, or a domain repository found through re3data. Follow the FAIR principles.
- Bundle a research compendium: `README.md`, `LICENSE`, `CITATION.cff`, `environment.yml`, `data/`, `analysis/`, and `outputs/`.