Datagrunt 3.1.8: Pre-Parsed JSON & Dict Support for PDF Exports

PDF parsing is computationally heavy. Under the hood, extracting text structures, detecting tables (with pdfplumber), extracting images, and running OCR on scanned pages takes considerable CPU time and memory.

Prior to version 3.1.8, if you wanted to export a PDF to both JSON (for your backend database) and Markdown (to feed into an LLM or RAG pipeline), you had to parse the PDF document twice:

from datagrunt import PDFWriter

# Parses the PDF once to write JSON
writer = PDFWriter("quarterly_report.pdf")
writer.write_json("output.json")

# Parses the PDF a second time to write Markdown!
writer.write_markdown("output.md")

For large or graphics-dense documents, this double-parsing was a significant bottleneck.

Datagrunt 3.1.8 resolves this inefficiency by allowing PDFWriter and PDFReader to accept pre-parsed JSON dictionaries or .json file paths directly in their constructors.

The Solution: Reusing Parsed State

With this update, you can now parse the PDF exactly once, obtain the intermediate JSON/dict representation, and generate all other formats instantly from that state.

Example 1: Decoupled Pipelines (Loading from a JSON File)

If you run a decoupled pipeline (e.g., one microservice extracts PDF data to JSON, and another formats it to Markdown later), you can instantiate PDFWriter directly with the JSON file path:

from datagrunt import PDFWriter

# Initialize directly from the saved JSON file
writer = PDFWriter("output.json")

# Generates the Markdown file instantly without reading the PDF
writer.write_markdown("output.md")

Example 2: Single-Session In-Memory Pipelines

If you are processing everything in a single script, you can extract the parsed dictionary in memory and pass it straight to PDFWriter without ever touching the disk:

from datagrunt import PDFReader, PDFWriter

# 1. Parse the PDF exactly once to memory
reader = PDFReader("quarterly_report.pdf")
doc_dict = reader.to_dicts()

# 2. Export JSON, JSONL, and Markdown instantly from memory
writer = PDFWriter(doc_dict)
writer.write_json("output.json")
writer.write_json_newline_delimited("output.jsonl")
writer.write_markdown("output.md")

Performance Parity

In our benchmarks on a sample 10-page document, the optimization results speak for themselves:

• Before (Double PDF Parse):

  • Time to write JSON: 1.03 seconds
  • Time to write Markdown: 1.01 seconds
  • Total time: 2.04 seconds

• After (Single PDF Parse + In-Memory Formatting):

  • Time to parse PDF & write JSON: 1.00 seconds
  • Time to write Markdown from in-memory dict: 0.0001 seconds (0.1 milliseconds)
  • Total time: 1.00 second (a 50% overall speedup)

Zero File Dependency & Complete Correctness

Because this optimization works directly on the parsed dictionary structure, it is 100% byte-for-byte identical to parsing the PDF directly. It preserves the exact same layout-aware column sorting, markdown table formatting, and MD5 image deduplication logic. Additionally, when initializing with a dictionary, the process runs completely in memory without creating temporary files on disk.

Upgrade Today

Datagrunt 3.1.8 is now live on PyPI. Upgrade your environment to start building faster document parsing pipelines:

uv pip install --upgrade "datagrunt[pdf]"