# CSV Preflight Kit — Hermes

A small, offline, scriptable CSV check before you hand a file to an importer.
It produces a source-preserving JSON report plus a spreadsheet-friendly review
queue. Useful for batch jobs and CI where a machine-readable exit status matters
more than an upload form. It is **not** an importer, repair tool, universal CSV
validator or a promise that a destination will accept your data.

## Run it

No installation, account, server, network access or third-party dependencies.
Use Python 3.9 or later with its standard library. Execution here was verified
with Python 3.14; other Python versions and operating systems have not been run
in this environment. All included sample names and records are fictional.

From the kit directory, choose new output paths:

```sh
python import_guard.py samples/clean.csv --required id --required name --key id --review clean-review.csv > clean-report.json
python import_guard.py samples/conflict.csv --required id --required name --key id --review conflict-review.csv > conflict-report.json
python import_guard.py samples/malformed.csv --required id --key id --review malformed-review.csv > malformed-report.json
python -m unittest discover -s . -p 'test_*.py' -v
```

The commands above return 0, 2 and 1 respectively. In a CI runner, explicitly
handle 2 as a review decision rather than treating it as a crashing process.
For example, in a POSIX shell without `set -e` around this command:

```sh
python import_guard.py input.csv --required id --key id --review batch-review.csv > batch-report.json
status=$?
case "$status" in
  0) printf '%s\n' 'Selected checks found no issues; destination validation still needed.' ;;
  2) printf '%s\n' 'Review the report; no automatic import or repair was performed.' ;;
  *) printf '%s\n' 'Input, configuration or runtime error; inspect the error JSON.' ;;
esac
```

## Explicit rules, not guesses

- Positional argument: path to a UTF-8 CSV file. A UTF-8 BOM is accepted and
  excluded from the parsed first header. The original bytes are never rewritten.
- `--delimiter CHAR`: one literal character; default comma. For semicolon input
  use `--delimiter ';'`. An actual tab character can be passed as a delimiter.
  Newline, carriage-return and double-quote delimiters are rejected.
- `--required NAME`: repeat for each required header. Header matching is exact
  and case-sensitive. Headers and values are not trimmed or renamed.
- `--key NAME`: optional, single column whose values your workflow explicitly
  requires to be unique. Omit it when repeated identifiers are legitimate.
  No platform-specific uniqueness assumption or store compatibility is claimed.
- `--review PATH`: default `review.csv` in the current directory. The path must
  **not already exist**. Existing files, symlinks and hardlinks are refused,
  including paths pointing at the input. Use a fresh path for every run.
- `--help`: prints usage and returns 0 instead of a JSON report.

Reusable JSON schema/recipe loading is deliberately not included in this small
version. Store the explicit command and its flags in your own batch script.
Nothing is sent to a service and there is no telemetry, API, LLM or database.

## What is checked

Every data record is retained in the JSON report, even when flagged.
The program checks inconsistent row width, blank or duplicate headers and
missing required headers. If a key is configured, it checks a missing field,
empty/whitespace-only key and repeated keys. Keys otherwise use their original
strings: `001`, `1` and ` 001` are distinct. No numeric coercion takes place.

All members of a repeated-key group, including the first occurrence, are
flagged. Matching parsed field lists are `exact_duplicate_row`; differing field
lists within the same key group are `duplicate_key_conflict`. A group containing
both identical copies and a differing version has both kinds of finding.
"Exact" means identical parsed strings, not identical quote syntax or raw bytes.
Without `--key`, this version does not run duplicate-row or duplicate-key checks.
Blank keys are flagged individually, not collapsed into a duplicate group.

CSV is parsed with Python's `csv.reader(strict=True)`, double-quote quoting and
doubled-quote escaping. Quoted commas, escaped quotes and multiline fields are
supported. Unterminated quoted fields and invalid characters after a closing
quote are parse errors. Strict mode is not a complete independent RFC validator:
for example, a quote inside an unquoted field can be treated as literal data.
No automatic delimiter detection, backslash escaping or encoding guessing.

## Record numbers and accounting

`source_record` is the **logical CSV record**, not a physical text line:
header = 1, first data record = 2. A quoted field spanning three text lines still
belongs to one logical record. Blank records are counted and checked for width;
the tool never silently skips them.

The JSON report includes `headers`, `header_issues`, `checks`, `records`,
`duplicate_groups`, `review_cell_limit`, `scope_note`, `status` and `summary`.
A record with duplicates references a numbered `duplicate_group`; that group's
`source_records` lists every member once, avoiding quadratic report growth.
`data_records = records_with_issues + records_without_issues`. `issue_count`
counts individual header findings plus individual record findings, so it can
exceed the number of affected rows. A failed header validation flags every data
record `header_validation_failed`; the raw records are still present. Missing
or ambiguous key headers cannot be used for uniqueness checks.

Status/exit codes:

| Exit | JSON status | Meaning |
| --- | --- | --- |
| 0 | `no_issues_detected` | No issues under these selected structural rules; **not import-ready or safe by guarantee**. A header-only valid file can also return 0. |
| 2 | `needs_review` | Fully parsed within limits, with findings; no changes to source. |
| 1 | `error` | Parse, size, configuration, decoding or runtime problem. Error JSON has an error type and message, no success summary or partial data report. |

## Review CSV and spreadsheet safety

`review.csv` contains only flagged records, plus source record 1 when headers
have findings. A clean result creates a review file containing only its header.
Fixed columns are `source_record`, `issues`, `values_json`, then `cell_1` through
at most `cell_64`. Wide rows retain **all** their original parsed fields in
`values_json` and in the JSON report; only the convenient per-cell display stops
at 64 to bound the padded review output. Those display cells are not import data.

Per-cell display values that start with `=`, `+`, `-` or `@`, including after
leading whitespace, are prefixed with an apostrophe. Leading tab/CR characters,
also after whitespace, get the same prefix. This includes negative-looking
numbers: conservatively treat source values as strings. The JSON report and
`values_json` retain the original strings; JSON arrays begin with `[` and are
not executable formula cells. Quoting alone is not used as formula protection.
Do not strip the apostrophe from the display cells and then treat them as safe.
Spreadsheet applications differ: when opening files, use text import settings;
a spreadsheet may convert `001` to `1` even though the CSV and JSON preserve it.
No spreadsheet app was executed as part of verification.

## Hard bounds and limitations

- Maximum 5 MiB (5,242,880 bytes), measured on the original file including BOM.
  Read is bounded to that limit plus one detection byte. Exactly the limit is
  accepted; one byte over is rejected before any review output is created.
- Maximum **25,000 logical records including the header**, so at most 24,999
  data records. Exactly that bound is accepted; the next record is an error.
- The whole allowed input and parsed report are kept in memory. Memory usage can
  exceed input byte size, especially with many tiny fields. This is not an
  untrusted-file sandbox or a streaming validator for large exports.
- Intended input is a regular local file, not a pipe/device or a concurrently
  modified file. The tool provides no transaction or locking against other apps.
- Output creation is exclusive, not a filesystem transaction. A disk failure
  can leave an incomplete new review file: on exit 1 discard that new artifact.
  Parse/configuration/limit errors do not create a review artifact. Existing
  output artifacts are never considered evidence of a later successful run.
- **Never redirect stdout onto the source file.** Shell `>` truncates its target
  before Python starts, so no CLI can protect an input from that shell action.
  The built-in `--review` writer refuses all existing files.
- No destination integration, field-type validation, composite keys, cross-file
  matching, hidden normalization, automatic deletion or auto-repair.
- Reports can contain sensitive input values. Keep them local and manage their
  permissions and retention yourself. There are no bank, tax or compliance claims.

## License and commercial use

See `LICENSE.txt`: commercial/internal use and charging for your own client work
and reports are permitted; resale, redistribution and repackaging of this kit
or its modified source are not. This is not an open-source license.

This is an original narrow implementation of common CSV preflight checks, not a
claim to unique or exclusive CSV technology. The fictional fixtures and passing
tests demonstrate behavior only, not customer demand, sales or import success.