OPERATE / PRODUCTION

Make every transformation accountable.

Production readiness is more than a successful transform. Keep the artifact, configuration, schema fingerprint, evaluation report, and test evidence together so another developer can reproduce the decision.

artifact + lineageCI matrixPyPI 0.5.0

Ship checklist →

1. Define the data contract

Contract	Check	Failure action
Input schema	Required columns, types, and target are present.	Stop and inspect the artifact schema.
Role policy	Identifiers, protected columns, and time/group columns are deliberate.	Update configuration; do not silently drop data.
Split policy	Classification is stratified; time/group data uses matching split strategy.	Rerun evaluation with explicit columns.
Output schema	Feature names and order stay stable for inference.	Reject the artifact and investigate drift.

2. Ship the artifact, not a notebook cell

Terminalrecommended run bundle
datadoc run data.csv \
  --target churn \
  --task classification \
  --output-dir runs/churn-v1 \
  --evaluate

# inspect before deployment
Get-ChildItem runs/churn-v1  # PowerShell
ls runs/churn-v1            # Unix

Keep at minimum: pipeline.json, manifest.json, the approved plan.json, the evaluation report, and the package version. The JSON artifact is the source of truth for transform behavior.

Do not upload secrets or raw production data.

Artifacts can contain category values and fitted statistics. Treat them as data assets; review privacy and retention requirements before publishing them.

3. CI gates for contributors

Terminallocal equivalent of CI
python -m pytest -q
python -m ruff check datadoc tests
python -m ruff format --check datadoc tests
python -m compileall -q datadoc
uv lock --check
uv build

cd web
npm ci
npm run build

The repository matrix covers Python 3.10–3.12, multiple operating systems, package building, and frontend compilation.

Open-source rule

Optional dependencies remain optional. A user who only wants local profiling should not need an API key, scikit-learn, or a web server.

4. Release to PyPI

Version 0.5.0 is the current release. Before upload, commit the source and docs, push to GitHub, wait for CI, tag the same version, and validate the artifacts.

Terminalmaintainer flow
python -m pip install --upgrade twine
python -m twine check dist/*
python -m twine upload dist/datadoc_cli-0.5.0*

Never put a PyPI token in source control.

Use trusted publishing or a short-lived token stored in the environment. See RELEASE_CHECKLIST.md in the repository for the complete sequence.

5. What to monitor after release

Input schema drift and unseen categories.
Null and missingness rates compared with the profile baseline.
Output feature count and names.
Model performance on a separately governed evaluation set.
Artifact version and package version used in every job.