Datagrunt 4.5.12: Panic-Freedom Property Testing for the Rust Core Catches the Bug Differential Parity Missed

Datagrunt 4.5.12 lands the second half of a testing plan sketched out in issue #315. Where 4.5.9 introduced differential property testing on the Python side — generating thousands of random CSV byte strings with Hypothesis and asserting our compiled Rust core (datagrunt._native) and our pure-Python oracle (_compute_python) agree on every one — this release asks a different question, from the other side of the FFI boundary: not “do the two backends agree with each other?” but “can this compiled code, loaded directly into the host process, ever be made to panic?”

The answer, over 17 new proptest properties exercising every public path-based entry point in datagrunt-core with arbitrary and CSV-biased byte soup, is: almost. On its first run, the suite surfaced a genuine contract violation — leading_rows(path, limit=0) returns one row instead of zero — sitting identically in both backends, invisible to differential parity testing for a structural reason that is the most important idea in this release.

There is no user-facing API change here. Nothing about how you call datagrunt is different. What changes is confidence: the rule that datagrunt’s Rust core must never panic on malformed input — previously enforced by code review and discipline — is now an executable property that runs on every cargo test.

The Stakes: A Panic Is Not a Python Exception

datagrunt’s CSV compute — delimiter inference, dialect sniffing, header probing, row counting — runs by default inside a compiled Rust extension, datagrunt._native, built with PyO3 and loaded directly into the host Python process. This is not an optional accelerator with a safe fallback: since 4.0, _native is the required default, and _compute_python exists as a parity oracle and hidden diagnostics toggle, never a runtime safety net.

That architecture makes one failure mode categorically worse than an ordinary bug: a Rust panic that reaches across the PyO3 boundary on malformed input. It does not behave like a well-formed Python exception that calling code can catch, log, and route around — unwinding through foreign-function-interface code is territory a host application cannot be expected to handle gracefully. For a required, always-on native extension parsing arbitrary user-supplied files, “malformed input” and “a way to disrupt the host process” are uncomfortably close together.

Datagrunt’s engineering rules have said this from the start: avoid unwrap()/expect()/panic!() reachable from user input, because a panic across the PyO3 boundary is a denial of service. Until this release, that rule lived entirely in review discipline. Issue #315 called it out explicitly as a property worth locking in rather than re-auditing by hand every time the crate grows — exactly the kind of invariant a generative test suite is built to hold permanently, instead of a human checking it once per pull request.

Two Property Files, Seventeen New Properties

PR #326 adds proptest = "1" as a dev-dependency of datagrunt-core and lands two integration test files that run inside the existing cargo test job — no CI workflow changes needed.

props_pure.rs — invariants of the functions with no I/O

These properties don’t touch the filesystem — they hand arbitrary strings and byte vectors straight to the functions that do the character-level work, and check what must structurally hold of the result:

take_chars is the sharpest example of why this matters: it exists specifically because naive byte slicing panics mid-codepoint on multi-byte UTF-8. A property asserting char-boundary safety over arbitrary strings is a direct, permanent check on the exact hazard the function was written to eliminate:

#[test]
fn take_chars_never_panics_and_bounds_hold(s in ".*", n in 0usize..(3 * 1024)) {
    let out = take_chars(&s, n);
    prop_assert!(out.chars().count() <= n);
    // Prefix property, char-wise.
    prop_assert!(s.starts_with(&out));
}

And sniff_output_invariants locks down a shape guarantee that the rest of the dialect-inference pipeline depends on — a delimiter that is anything other than exactly one character, or a quotechar outside {", '}`, would be a contract break for every caller downstream:

#[test]
fn sniff_output_invariants(sample in ".*") {
    if let Some(d) = sniff(&sample, None) {
        prop_assert_eq!(d.delimiter.chars().count(), 1);
        // csv.Sniffer only ever yields '"' or '\'' as quotechar.
        prop_assert!(d.quotechar == "\"" || d.quotechar == "'");
    }
}

props_files.rs — panic-freedom across the file-reading surface

This file is the panic-freedom battery proper: arbitrary and CSV-biased bytes, written to a real temp file, driven through every public path-based function the crate exposes.

Between the two files, that’s 13 distinct path-based entry points under test: infer_delimiter, is_legacy_mac_newlines, is_empty, is_tsv, universal_lines, DecodedReader::open, probe_csv_header, leading_rows, first_row, count_leading_comments, count_leading_physical_lines_before_header, row_count_with_header, and check_ragged — every function in the crate that takes a filesystem path and touches bytes it did not generate itself.

The generator feeding all of this is deliberately biased, not purely random, so it actually reaches CSV-shaped code paths instead of bouncing off the first malformed byte:

fn csvish_bytes() -> impl Strategy<Value = Vec<u8>> {
    prop_oneof![
        proptest::collection::vec(any::<u8>(), 0..2048),
        "[a-z0-9,;|\t \"'\r\n#=+-]{0,2048}".prop_map(|s| s.into_bytes()),
        "[ \t\r\n]{1,64}".prop_map(|s| s.into_bytes()),
        Just(Vec::new()),
    ]
}

Four arms: raw arbitrary bytes, a CSV-alphabet-biased string guaranteed to contain delimiters and quotes and newlines, a non-empty whitespace-only string, and the deterministic empty file. The last two arms exist for a reason covered below — they are not filler.

Deliberately Not Differential

It would be easy to read “property testing over the Rust core” and assume it re-derives 4.5.9’s differential comparison one layer down. It doesn’t, on purpose. Rust-equals-Python agreement already has a home: tests/parity/, exercised by the Hypothesis suite that shipped in 4.5.9 and was extended to cover the delimiter field in 4.5.10. Re-deriving expected values inside datagrunt-core’s own test suite would just duplicate that oracle with extra steps.

What props_pure.rs and props_files.rs assert instead is what must hold of the Rust implementation in isolation — panic-freedom, idempotence, character-set membership, uniqueness, prefix relationships — properties that are true (or false) of one function on its own, independent of whatever the Python mirror happens to do:

  graph TD
    A[Hypothesis csv_bytes strategy] -->|same input| B[Rust _native]
    A -->|same input| C[Python _compute_python]
    B --> D{Outcomes equal?}
    C --> D
    D -->|"tests/parity/, 4.5.9 + 4.5.10"| E[Proves: Rust agrees with Python]

    F[proptest arbitrary + CSV-biased bytes] --> G[datagrunt-core function, alone]
    G --> H{Panics? Own invariants hold?}
    H -->|"datagrunt-core/tests/, 4.5.12"| I[Proves: Rust cannot be crashed or broken on its own terms]

The two layers are complementary rather than redundant, and the distinction is not academic — it is exactly why one layer caught a bug the other could not.

The Bug It Found on the First Run

leading_rows(path, limit=0) is supposed to return an empty list. It returns a list with one row in it.

Both implementations check the cap after appending, not before:

def leading_rows(filepath: StrPath, limit: int) -> list[str]:
    """Up to ``limit`` leading non-blank, non-comment rows, each stripped."""
    rows: list[str] = []
    with open(filepath, "r", encoding=FileProperties(filepath).DEFAULT_ENCODING, errors="ignore") as f:
        for line in _capped_lines(f):
            stripped = line.strip()
            if stripped and not stripped.startswith("#"):
                rows.append(stripped)
                if len(rows) >= limit:
                    break
    return rows

pub fn leading_rows(path: &Path, limit: usize) -> std::io::Result<Vec<String>> {
    let mut rows = Vec::new();
    for line in universal_lines(path)? {
        let line = line?;
        let stripped = py_strip(&line);
        if !stripped.is_empty() && !stripped.starts_with('#') {
            rows.push(stripped.to_string());
            if rows.len() >= limit {
                break;
            }
        }
    }
    Ok(rows)
}

On a file containing "a\n" with limit=0: the first data row gets pushed, then rows.len() >= limit (1 >= 0) is checked and trips the break — one row out, not zero. Both docstrings promise “up to limit” rows. Both implementations silently hand back limit + 1 in this corner. It’s filed as issue #325, still open as we publish this.

This is the point of the whole exercise. All of differential testing’s work so far — proving Rust and Python agree — could not have found this, structurally could not, because the two backends agree with each other while both are wrong. Differential parity answers “did the port introduce a divergence?” It cannot answer “did the original design have a bug that the port faithfully reproduced?” Only an independent invariant — “the output must never exceed the caller’s own stated limit” — checked against the implementation itself rather than against a second implementation, can catch a defect two backends share by common ancestry.

The property that found it is leading_rows_respects_limit, and until #325 lands its fix, the assertion is pinned at the current, documented-but-wrong bound rather than the correct one:

#[test]
fn leading_rows_respects_limit(bytes in csvish_bytes(), limit in 0usize..64) {
    let f = file_with(&bytes);
    if let Ok(rows) = datagrunt_core::rows::leading_rows(f.path(), limit) {
        // KNOWN (issue #325): limit=0 currently returns 1 row — the cap is
        // checked after the append in BOTH backends, so differential parity
        // cannot see it. The fix PR for #325 must tighten this back to
        // `rows.len() <= limit`; that flip is its red/green test.
        prop_assert!(rows.len() <= limit.max(1));
    }
}

That comment is a deliberate obligation, not a shrug. Per the project’s own known-divergence convention (the same pattern used to xfail the dialect sniffer bug ahead of its 4.5.10 fix), pinning the weakened bound today means the fix for #325 has a built-in regression test: tightening limit.max(1) back to plain limit is the red/green cycle that proves the fix landed, with no separate test to remember to write. Severity is low in practice — no internal caller passes limit=0, callers use small constants like 5 — but it is reachable from the public PyO3 surface (datagrunt._native.leading_rows(path, 0)), and it is exactly the class of defect this release’s investment exists to catch. It paid for itself before the suite finished its first run in CI.

Vacuity Discipline: A Green Suite Isn’t Proof of Anything

A property that never gets exercised passes just as loudly as one that does. probe_header_shape_is_consistent carries two assertions guarded by probe.empty and probe.blank — but an early check, run before csvish_bytes() had its whitespace-only and empty-file arms, measured those branches firing zero times across 257 generated cases. The assertions were real, the test was green, and neither had ever actually executed.

The fix wasn’t to weaken the assertions — it was to strengthen the generator until the shapes it was supposed to be guarding against actually showed up. Adding a guaranteed-empty arm (Just(Vec::new())) and a non-empty whitespace-only arm ("[ \t\r\n]{1,64}") to csvish_bytes() brought reachability up to 62 hits out of 69 cases — the branches now fire on the overwhelming majority of runs instead of never.

The lesson generalizes past this one test: a property suite that always generates the same shape of input passes thousands of times and produces the exact same exit code as a suite that is genuinely exercising the invariant. Green is necessary but not sufficient — it has to be paired with a measurement that the interesting branch was actually taken. Every property in both files went through this check before merge; the header-probe pair is simply the one where the first measurement came back at zero.

Verification

All numbers below were re-run independently before publishing this post, against the merged state of main:

proptest runs 256 fresh, non-derandomized cases per property on every invocation, mirroring the philosophy already established for the Hypothesis suite: any failure it ever produces is a real bug, and gets promoted into a permanent, committed regression case rather than dismissed as flake. The long-exploration venue for deeper searches remains the existing weekly scheduled job on the Python side; this suite is the fast, always-on PR gate for the Rust core specifically.

Upgrading to Datagrunt v4.5.12

This is an internal quality release. There is nothing to change in your code, no new parameters, no altered return values. The benefit is architectural confidence: the compiled core that every CSV read in datagrunt runs through now has a permanent, generative check that it cannot be crashed by the file you hand it, running on every build rather than resting on a one-time audit.

# Force upgrade via uv
uv pip install --upgrade datagrunt

# Verify installation details
python -c "import datagrunt; print(f'Version: {datagrunt.__version__}')"

If you maintain a fork or embed datagrunt-core directly, leading_rows(path, 0) still returns one row rather than zero until issue #325 lands its fix — worth knowing if any caller of yours relies on limit=0 meaning “give me nothing.” To learn more about how the Rust core and Python oracle divide responsibility, explore the CSV Engines and Rust Acceleration Guide.