This page lists tools and resources relevant to reproducibility of computational research, organized by category. Inclusion here does not constitute endorsement, this is a curated reference list, not an exhaustive catalog. Resources are provided for reference only and are maintained by their respective developers and content owners.
Provenance capture tools
ReproZip
Traces system calls to bundle all files, libraries, and environment details needed to reproduce a computation into a portable archive.
reprozip.org ↗Sciunit
Versioned execution history system that allows individual runs to be inspected, compared, and shared across branches.
GitHub ↗noWorkflow
Captures fine-grained provenance from Python scripts without requiring a workflow definition, recording function calls and file accesses.
GitHub ↗Workflow engines
Nextflow
Defines pipelines as DAGs of named processes with declared inputs and outputs, with strong incremental execution and container integration.
nextflow.io ↗Snakemake
Python-based workflow management system that creates reproducible and scalable data analyses using a readable, domain-specific language.
snakemake.github.io ↗DVC
Data version control system providing incremental execution, data versioning, and experiment comparison for explicitly defined workflows.
dvc.org ↗CWL / cwltool
An open standard for writing portable, reproducible workflows. Tools that support CWL can run the same pipeline across different systems without modification.
commonwl.org ↗Pegasus WMS
Runs large scientific workflows across HPC, cloud, and grid systems. Handles job scheduling, data movement, and failure recovery, and logs provenance automatically as jobs run.
pegasus.isi.edu ↗Galaxy
A web-based platform where you can build, run, and share analysis workflows without writing code. Keeps a full history of every step, making results easy to reproduce and share.
galaxyproject.org ↗WDL / Cromwell
A workflow language paired with an execution engine from the Broad Institute. Widely used in genomics for writing pipelines that run the same way locally, on a cluster, or in the cloud.
openwdl.org ↗Notebook environments
Jupyter Lab
Next-generation web-based interface for Project Jupyter, providing a flexible environment for interactive and reproducible computing.
jupyterlab.readthedocs.io ↗Jupyter Notebook
Original web application for creating and sharing computational documents with live code, equations, visualizations, and narrative text.
jupyter.org ↗Papermill
Tool for parameterizing, executing, and analyzing Jupyter notebooks, enabling reproducible notebook workflows at scale.
papermill.readthedocs.io ↗Binder / repo2docker
Point it at a GitHub repo and it spins up a live, reproducible environment in the browser. No setup required, anyone can run your notebooks exactly as you did.
mybinder.org ↗Data versioning and management
DataLad
Git for data. Tracks versions of large datasets alongside code, records what commands produced what outputs, and lets you re-run any prior analysis from a single command.
datalad.org ↗Experiment tracking
MLflow
Tracks what you ran, with what parameters, and what it produced. Useful for keeping experiments organized and being able to go back and reproduce any past result.
mlflow.org ↗Weights & Biases
Logs every detail of a training or compute run (hyperparameters, metrics, code version, system info) so you can compare runs and reproduce any result later.
wandb.ai ↗Sacred
A lightweight Python library that captures the configuration, code state, and outputs of each experiment run. Low overhead, easy to add to existing scripts.
GitHub ↗Sumatra
Lightweight experiment tracking built for simulation workflows. Records parameters, code version, and outputs for each run with minimal changes to your existing scripts.
neuralensemble.org ↗OpenML
A shared platform for reproducible ML experiments. Upload datasets, tasks, and results so others can benchmark against them or build on your work directly.
openml.org ↗Publishing & packaging formats
RO-Crate
Machine-readable format for describing research objects and their provenance using JSON-LD, designed for interoperability at publication.
researchobject.org ↗Whole Tale
Platform that packages code, data, and environments into shareable, executable units for frictionless reproducibility at publication.
wholetale.org ↗Code Ocean
Helps computational teams provision compute, set up environments, scale pipelines, and get a reproducible, traceable record of their work.
codeocean.com ↗Renku
An open platform from EPFL that ties together GitLab, Jupyter, and workflow tracking in one place. Designed for collaborative research where you want code, data, and results to stay connected.
renkulab.io ↗Artifact repositories
Zenodo
Free, open repository from CERN for archiving code, data, and software. Assigns a DOI to everything you deposit, making it citable and permanently accessible.
zenodo.org ↗Figshare
A repository for sharing research outputs, datasets, figures, code, and preprints with DOIs. Commonly required as the artifact submission platform at CS venues including SC and DSN.
figshare.com ↗OSF
A free platform for managing research projects from start to finish. Supports preregistration, file storage, versioning, and sharing, all in one place and built around open science norms.
osf.io ↗Containers & environments
Apptainer / Singularity
Rootless container platform designed for HPC environments, enabling portable and reproducible execution of software across systems.
apptainer.org ↗Docker
Platform for developing, shipping, and running applications in containers, ensuring consistent environments across development and production.
docker.com ↗Conda
Open-source package and environment management system that allows reproducible installation of software dependencies across platforms.
docs.conda.io ↗Computational testbeds
Chameleon Cloud
An NSF-funded bare-metal cloud testbed for CS research, run by UChicago and TACC. Used as the default platform for SC artifact evaluation, with tools for packaging and sharing reproducible experiments.
chameleoncloud.org ↗MINCER
A Chameleon Cloud appliance for standardized hardware performance measurement. Wraps PAPI in Docker containers so you can collect CPU, GPU, and energy metrics consistently across architectures.
REPETO webinar ↗Standards, reports & guidelines
NASEM 2019 Report
The 2019 National Academies consensus report that set the definitions for reproducibility and replicability used across science today. The starting point for understanding why this matters and what to do about it.
nationalacademies.org ↗FAIR Principles
Four principles: Findable, Accessible, Interoperable, Reusable, for how research data should be shared. Now referenced by most journals, funders, and artifact evaluation processes.
go-fair.org ↗W3C PROV-DM
The W3C standard for representing provenance: who did what, when, and with what inputs. The common vocabulary that most provenance tools and formats are built on top of.
w3.org ↗ProvONE
An extension of W3C PROV-DM designed specifically for scientific workflows. Adds concepts like programs, ports, and channels to better capture how data moves through a pipeline.
provone.org ↗ACM Artifact Badging
ACM's official policy for recognizing reproducible research. Defines what "Artifacts Available," "Artifacts Evaluated," and "Results Reproduced" mean and how badges are awarded across ACM venues.
acm.org ↗SC Reproducibility Initiative
SC's program requiring an Artifact Description appendix with every submitted paper, with optional Artifact Evaluation for accepted work. The main reproducibility process for the HPC and systems community.
sc25.supercomputing.org ↗Community & initiatives
REPETO
NSF-funded reproducibility initiative based at UChicago. Runs the annual Summer of Reproducibility student program and hosts webinars and community events focused on practical reproducibility in HPC.
repeto.cs.uchicago.edu ↗cTuning / CK
A framework for packaging and sharing ML and systems experiments so others can reproduce and build on them. Used by MLCommons and several conference artifact evaluation processes.
GitHub ↗Lifecycle-Spanning Tool
Developed at RCC UChicago
Tracey
A lightweight, end-to-end reproducibility tool that captures computational workflows automatically from real executions covering the full arc from exploration to publication. No pipeline pre-specification required.
View on GitHub ↗View Website ↗- ✓ Automatic background capture, no per-command annotation
- ✓ Derives workflow DAGs from execution history, not declarations
- ✓ Supports HPC environments with native Slurm integration
- ✓ Packages workflows into portable Apptainer/Singularity containers
- ✓ Minimal measurable overhead for polling-based capture mode
We are interested in suggestions of additional content for this section. Please share additional resources and tools that are missing on this list by emailing reproducibility@rcc.uchicago.edu.