Datagrunt 4.6.0: Fully Configurable OCR DPI Scaling and Page-Render Resolution Control

Datagrunt 4.6.0 is here, introducing extensive customization options to our advanced PDF processing suite. In this release, we have completely decoupled our PDF engine from its historical hardcoded rendering and OCR (Optical Character Recognition) threshold rules. Developers now have absolute, fine-grained programmatical control over OCR resolution thresholds, large-format scaling factors, and whole-page rasterization defaults.

Whether you are extracting text from high-resolution financial scans, parsing giant engineering blueprint layouts, or generating high-quality whole-page previews, v4.6.0 allows you to tune memory consumption, OCR accuracy, and CPU footprint to perfectly match your target environment.

The Motivation: Dynamic Resolution Tuning for Complex PDF Workloads

To extract structured data from scanned or image-based PDFs, Datagrunt triggers a robust, fallback OCR pipeline backed by Tesseract. This fallback pipeline first rasterizes the page to an in-memory image, and then runs Tesseract OCR to build structured text blocks.

Previously, the render DPI (dots per inch) for OCR was controlled by hardcoded constants:

• Pages under 1,500 points on either side were rendered at 150 DPI.
• Large-format pages exceeding 1,500 points in width or height (typical for blueprints, CAD drawings, and architectural layouts) were rendered at 75 DPI to avoid out-of-memory (OOM) faults on standard servers.

While this default logic served as a safe baseline, it introduced critical production boundaries:

1. Low-Legibility Scans: For small or highly compressed fonts on standard pages, 150 DPI was occasionally insufficient, leading Tesseract to skip characters or misread similar-looking punctuation.
2. Dense Blueprints: Large-format documents with complex, technical textual annotations suffered when forced down to 75 DPI. In many cases, developers had adequate system RAM to process these pages at 150 or 300 DPI, yet were hindered by the static fallback.
3. Mismatched Render Defaults: Whole-page PDF preview rendering (render_pages_as_images) was locked to a default of 300 DPI unless overriden at every separate call, requiring redundant method arguments.

The v4.6.0 Solution: Class-Level Parameter Injection

Datagrunt 4.6.0 resolves these issues by exposing four new class-level constructor arguments across PDFReader and PDFWriter under the Unified Element Schema, Native Reader, and Engine-Backed writer systems:

Architectural Deep Dive: Thread-Safe Configuration and Process-Pool Boundaries

Under the hood, these parameters are parsed, validated, and consolidated into a frozen _PDFExtractionConfig dataclass inside config.py.

1. Robust At-Construction Validation

Every input is validated immediately upon initial instance construction to guarantee a “fail-fast” developer experience.

# Rejects booleans, non-integers, and zero/negative thresholds at construction
from datagrunt import PDFReader

try:
    reader = PDFReader("document.pdf", ocr_standard_dpi=0)
except ValueError as e:
    print(f"Error caught: {e}")  # ocr_standard_dpi must be >= 1, got 0

2. Multi-Process Pickling and Worker Crossings

When using the default PDFium engine with workers > 1, Datagrunt spins up a process pool to distribute pages across multiple CPU cores. This requires the configuration instance to be completely serializable (picklable) to pass safely across the process pool boundary.

To support this seamlessly, the internal _PDFExtractionConfig maintains a strict pickling contract and is passed to work helpers:

  sequenceDiagram
    participant Main as Main Thread / PDFReader
    participant Pool as Process Pool Managers
    participant Worker as Worker Process (PdfiumNativeReader)
    
    Main->>Main: Validate DPI fields (>=1) & freeze _PDFExtractionConfig
    Main->>Pool: Submit pages with pickled _PDFExtractionConfig
    Pool->>Worker: Deserializes config on worker thread
    Worker->>Worker: dpi_for_page(width, height, config)
    Worker->>Worker: Render PIL at resolved resolution and run Tesseract OCR
    Worker->>Main: Return parsed page elements

If no configuration overrides are specified, the fallback helper dpi_for_page gracefully resorts to the historical single source of truth defaults, guaranteeing backward compatibility.

Code Examples: Harnessing v4.6.0 in Production

Here is how you can utilize the new features to optimize standard data pipelines.

Example 1: Enforcing Ultra-High Fidelity OCR

If you are dealing with small receipts, medical records, or fuzzy scanned tables, you can ramp up the standard OCR resolution to 300 DPI, and drop the large-format ceiling to enforce consistent, high-res extraction:

from datagrunt import PDFReader

# Initialize a PDFReader with high-fidelity OCR parameters
reader = PDFReader(
    "invoice_scan.pdf", 
    ocr_standard_dpi=300,            # Force standard pages to highly detailed 300 DPI (default 150)
    ocr_large_format_dpi=150,        # Elevate large-format pages to 150 DPI (default 75)
    ocr_large_format_dimension=1200, # More aggressive large-format threshold (default 1500)
)

# Parsed blocks are generated at the higher resolution, vastly improving Tesseract's accuracy
extracted_data = reader.to_dicts()

Example 2: Uniform Resolution across All Page Dimensions

For pipelines that must run at equal resolution regardless of page sizes (e.g. strict positional matching against an expected layout template coordinate space), simply configure the standard and large-format DPI parameters to the exact same value:

from datagrunt import PDFReader

# Force a uniform 200 DPI across all page sizes
reader = PDFReader(
    "mixed_sizes.pdf",
    ocr_standard_dpi=200,
    ocr_large_format_dpi=200
)

# No scaling transitions will occur
doc_dict = reader.to_dicts()

Example 3: Streamlining Page Preview Pipelines

With the introduction of the class-level render_dpi parameter, PDFWriter.render_pages_as_images has been upgraded. The dpi parameter of render_pages_as_images now defaults to None, making it automatically respect the class’s default configuration settings. This allows you to set your target resolution once during initialization:

from datagrunt import PDFWriter

# Configure a low-resolution rendering pipeline for fast thumbnail generation (e.g. 100 DPI)
writer = PDFWriter("presentation.pdf", render_dpi=100)

# Generates 100 DPI thumbnail PNGs automatically
writer.render_pages_as_images(output_dir="thumbnails")

# You can still bypass the default using method-level explicit overrides when needed
writer.render_pages_as_images(output_dir="high_res_slides", dpi=400)

Verification and Quality Testing

Our continuous integration suites have been fully expanded to test and secure these new customization options against regressions:

1. Validation Checks: We expanded test_extraction_config.py to assert that any invalid setting (like boolean types sneaky subclassing integer, or negative values) raises a clear TypeError or ValueError at instance creation.
2. Dynamic Behavior Assertions: We added test_ocr.py and test_pdfcomponents.py to monkeypatch and verify that DocumentAssembler passes the customized configuration DPI down to our backend engines.
3. Pickle Integrity Preservation: We verified that _PDFExtractionConfig with custom overrides roundtrips through the pickler perfectly, protecting our multiprocessing execution loops.

Upgrading to Datagrunt v4.6.0

Upgrading to v4.6.0 is trivial and strongly recommended for any system parsing scanned PDFs or rendering whole-page previews:

# Force upgrade via uv
uv pip install --upgrade datagrunt

# Verify the installed version
python -c "import datagrunt; print(datagrunt.__version__)"

To dive deeper into standard PDF extraction methods and structural layouts, check out our updated PDF Parsing Guide and constructor signatures in the Readers Reference and Writers Reference.