Writing Reproducible Python

  • PEP 723 allows inline metadata for Python dependencies.
  • Use uv to run single-file scripts with pure Python requirements (numpy, pandas).
  • Use Pixi when your script depends on system libraries or compiled binaries (eonclient, ffmpeg).
  • Combine them with a Pixibang (pixi exec ... -- uv run) for fully reproducible, complex scientific workflows.

Modules, Packages, and The Search Path

  • sys.path is the list of directories Python searches for imports.
  • The order of search is: Current Directory -> Standard Library -> Installed Packages.
  • A Package is a directory containing an __init__.py file.
  • The __init__.py file is the first file loaded when import PKG is run.
  • Code that works locally because of the current directory will fail when shared unless properly installed.

The Project Standard: pyproject.toml

  • pyproject.toml is the standard recipe for Python projects (PEP 621).
  • uv add manages dependencies and ensures reproducibility via uv.lock.
  • uv run executes code in an isolated, editable environment without manual activation.
  • Isolation: uv enforces a clean environment, preventing accidental usage of unlisted packages.
  • Manifest vs. Lock: pyproject.toml declares what we need; uv.lock records exactly what we installed.

Quality Assurance: Testing and Linting

  • Development Dependencies (uv add --dev) keep tools like linters separate from library requirements.
  • Ruff is the modern standard for fast Python linting and formatting.
  • Pytest verifies code correctness; Src Layout makes test discovery reliable.
  • Pre-commit hooks ensure no bad code ever enters your version control history.

The Changelog ProblemBuilding ArtifactsPublishing to TestPyPIAutomating Release (GitHub Actions)

  • Towncrier prevents changelog conflicts by using “News Fragments”.
  • uv build creates standard sdist and wheel artifacts.
  • uvx twine allows one-off publishing without polluting your environment.
  • TestPyPI is the sandbox for practicing release engineering.

If It Isn't Documented, It Doesn't ExistThe Documentation GapWriting Good DocstringsSetting Up SphinxBuilding the DocumentationDeploying to the Web

  • Docstrings are the raw material for documentation. Use the NumPy style (Parameters, Returns, Examples) for scientific code.
  • Sphinx with autoapi generates a complete API reference by scanning your source code, requiring no manual .rst files per module.
  • intersphinx enables cross-project links (e.g., to NumPy, Python), making your documentation part of the broader ecosystem.
  • Documentation builds can be automated with GitHub Actions and deployed to GitHub Pages on every push to main.
  • PR Previews let reviewers see documentation changes visually before merging, catching formatting issues that a green checkmark cannot.

The Gatekeeper: Continuous IntegrationThe Limits of Local HooksAnatomy of a Workflow FileThe Test MatrixConnecting CI to Releases

  • Continuous Integration runs your test suite on a neutral server on every push, catching problems that local hooks miss.
  • astral-sh/setup-uv provides a cached, high-performance uv environment in GitHub Actions.
  • A Matrix Strategy tests across multiple operating systems and Python versions in parallel.
  • CI can gate releases: the release job uses needs: to ensure tests pass before publishing.

Compiled Extensions: The Shared Object Pipeline

  • Shared Objects: Scientific Python packages function primarily as delivery mechanisms for compiled binaries (.so files).
  • Installation: “Installing” a package physically amounts to copying these binaries into site-packages.
  • Cibuildwheel: Automates the creation of binary wheels for all platforms, removing the need for users to possess compilers.