Datagrunt 3.1.10: Defaulting to Sequential PDF Execution for Guard-Free Scripts

Concurrency is a powerful tool, but it comes at a cost. When writing small utility scripts or executing in constrained distributed environments, process-spawning overhead and strict multiprocessing rules can get in the way of simple, robust code.

In Datagrunt 3.1.10, we changed the default worker count for PDFReader and PDFWriter from 4 to 1. This defaults PDF parsing to sequential execution, eliminating the need for if __name__ == '__main__': entry-point guards in simple scripts while keeping high-performance multiprocessing fully opt-in.

The Problem: Multiprocessing Overhead and Guards

Previously, Datagrunt default-initialized PDFReader and PDFWriter with workers=4. Because the default PDFium engine is not thread-safe within a single process, the library utilized a process pool (ProcessPoolExecutor) under the hood to parse pages concurrently.

While this drastically improved speed for large, multi-page PDFs, it introduced two major pain points:

1. Mandatory Boilerplate Guards: Under Python’s spawn start method (used on macOS and Windows), child processes import the main module to initialize themselves. Calling PDFReader or PDFWriter in a script without wrapping it in an if __name__ == '__main__': guard caused recursive process spawning, leading to crash loops or errors.
2. Spawning Overhead: For small or single-page PDFs, the time spent spawning child processes and serializing data across IPC boundaries often exceeded the parsing duration itself, making sequential execution faster in practice.

The Solution: Opt-in Parallelism

By changing the default workers count to 1, Datagrunt now executes all PDF parsing and writing sequentially by default.

Sequential Execution (Default)

You can now write simple, guard-free scripts to process your PDFs:

from datagrunt import PDFReader

# No __name__ == '__main__' guard required!
reader = PDFReader("invoice.pdf")
print(reader.to_dicts())

Opt-in Multiprocessing

For large, multi-page documents where parallelism provides a substantial speedup, you can explicitly opt back into multiprocessing by passing a workers count greater than 1. When you do this, you must protect the entry point:

from datagrunt import PDFReader

if __name__ == '__main__':
    # Run with 8 processes to parse pages concurrently
    reader = PDFReader("large_manual.pdf", workers=8)
    print(reader.to_dicts())

Upgrade Today

Datagrunt 3.1.10 has been released. You can pull the latest update:

uv pip install --upgrade datagrunt