DATADOC / DOCUMENTATION

Prepare data once.
Reuse it safely.

DATADOC profiles tabular data, proposes explainable transformations, fits them on training data only, and gives you a reusable artifact for validation, test, and inference.

v0.6.1Python 3.10–3.12CSV + ParquetOffline core

Get started →Run quickstart

FIRST COMMAND

Inspect before you change.

Start with a read-only profile. Every later decision can be reviewed, overridden, and reproduced.

datadoc profile train.csv --target churn

profile → plan → fitfrozen artifact → transform → evaluate

Quickstart

Use a training file with an optional target. DATADOC writes JSON and Parquet artifacts without requiring a notebook.

Terminaltrain → transform → evaluate
pip install datadoc-cli
datadoc profile train.csv --target churn --output profile.json
datadoc plan train.csv --target churn --task classification --output plan.json
datadoc fit train.csv --target churn --output artifacts/churn-pipeline.json
datadoc transform validation.csv \
  --pipeline artifacts/churn-pipeline.json \
  --output validation-features.parquet

Pythonsame artifact, any process
import polars as pl
from datadoc import DataDocPipeline, PipelineConfig

train = pl.read_csv("train.csv")
validation = pl.read_csv("validation.csv")
pipeline = DataDocPipeline(
    PipelineConfig(target="churn", task="classification")
).fit(train)

features = pipeline.transform(validation)
pipeline.save("artifacts/churn-pipeline.json")

PowerShellWindows-friendly
python -m pip install datadoc-cli
datadoc fit .\train.csv --target churn `
  --output .\artifacts\churn-pipeline.json
datadoc transform .\validation.csv `
  --pipeline .\artifacts\churn-pipeline.json `
  --output .\validation-features.parquet

What happens here?

fit learns medians, category vocabularies, datetime rules, and optional scaling statistics from train.csv. transform reuses those frozen values; it never learns from validation rows.

One mental model

Every DATADOC interface maps to the same six-step workflow. If you understand this flow, you understand the CLI, SDK, and web UI.

01ProfileFind nulls, duplicates, constants, IDs, roles, and schema risks. 02PlanReview proposed operations before any data is modified. 03FitLearn train-only statistics and save a versioned artifact. 04TransformApply the same output schema to validation, test, or new data. 05EvaluateCompare a candidate against a baseline for a declared task. 06ShipExport code, lineage, and artifacts that another person can inspect.

What DATADOC handles

01 / PROFILE

Data-quality findings

Missingness, duplicates, constants, cardinality, invalid values, datetime confidence, and suspected leakage.

02 / TRANSFORM

Safe preparation

Train-fitted imputation, missing indicators, category encoding, datetime features, optional clipping, and scaling.

03 / ARTIFACT

Portable state

Save the input contract, output schema, configuration, plan, fitted values, and artifact version as JSON.

04 / EVALUATE

Measured outcomes

Use classification or regression baselines with stratified, ordered, or group-aware validation.

05 / UI

Local workspace

Review findings, approve a plan, preview output, inspect lineage, and download artifacts from the local dashboard (Ctrl+K palette).

06 / AI ADVISORY

Semantic Hypotheses

Multi-provider AI explainer (OpenAI, Gemini, Anthropic, Ollama) infers hidden sentinels, missingness mechanisms, and domain feature formulas.

Choose your path

Best for repeatable batch jobs, automated reports, and CI.

Read the CLI reference →

Best for notebooks, services, and custom orchestration.

Read the Python SDK →

Best for exploring a dataset locally with visual feedback.

Read the UI guide →

What you get back

Output	Purpose	When it appears
`profile.json`	Facts, roles, findings, and schema fingerprint.	`profile` or `run`
`plan.json`	Explainable operations and rationales.	`plan` or `run`
`pipeline.json`	Frozen train-only transformation state.	`fit` or `run`
`report.html`	Standalone, shareable interactive HTML health & audit report (with optional AI summary).	`report` or `report --ai`
`comparison.html`	Visual side-by-side dataset diff (null resolution, feature lifecycle, shifts).	`compare --html`
`evaluation.json`	Baseline, candidate, metric, split, and warnings.	`evaluate` or `run --evaluate`
`manifest.json`	Lineage: input fingerprint, config, output schema, and artifact links.	`run`

When something goes wrong

Target errors

If the target is missing, null, or accidentally transformed, stop and fix the declaration. Use --target column_name; DATADOC never guesses a supervised target.

Schema errors

If transform reports missing columns or incompatible types, compare the new file with the artifact’s input_schema. Do not silently rename columns in production.

Unexpected score changes

Check the evaluation split, target leakage findings, estimator family, and whether the selected pipeline actually beat the baseline. “No improvement found” is a valid result.