Skip to content

--check

Check a file's values against a rules file, and exit 1 if they fail.

octa --check orders.parquet --rules quality.toml
octa --check orders.csv --rules quality.toml -f json

The rules file

TOML, one [[rule]] table per check:

[[rule]]
column = "order_id"
kind = "unique"

[[rule]]
column = "amount"
kind = "range"
min = 0

[[rule]]
column = "email"
kind = "regex"
pattern = ".+@.+"

[[rule]]
kind = "not_null"

Kinds are not_null, unique, range (with min and max, either optional), regex (with pattern) and max_length (with max_length). A rule without a column applies to every column.

The same file is written and read by the Data -> Data validation... dialog's Save rules and Load rules buttons, so a rule set you built by clicking can be committed and run in a pipeline.

Output

One row per failing rule on stdout:

Column Meaning
rule The kind that failed
column The column it applies to, empty for all columns
failures How many cells failed it
samples Up to three of the offending values

A rule that could not run at all, because it names a column the file does not have, appears as its own row with the problem in the rule field. The summary line goes to stderr.

Exit code

1 on any violation and on any rule that could not run; 0 only when everything passed. That second half matters: a rules file whose columns have since been renamed would otherwise report a clean run over checks it silently skipped.

See Data Validation for the dialog.