Polars to Pandas

Datagrunt gives you rails, not a cage — enough structure to move fast, with no lock-in. Readers return standard, open objects — Polars DataFrames, PyArrow Tables, plain Python dicts, and DuckDB relations — so the moment Datagrunt has parsed your file, you can hand the data to whatever ecosystem your project already uses. Nothing about your downstream code has to change to adopt Datagrunt for ingestion.

Pandas is the most common destination, so this guide shows every path from a Datagrunt reader to a pandas DataFrame. The same idea applies to anything else that speaks Arrow or Python dicts.

Setup

Pandas is intentionally not a Datagrunt dependency — you bring it only if you use it:

uv pip install pandas

Every conversion below works identically from CSVReader, ExcelReader, and ParquetReader.

Path 1: Polars → pandas (the direct route)

to_dataframe() returns a genuine Polars DataFrame, and Polars ships its own to_pandas():

from datagrunt import CSVReader

df = CSVReader('data.csv').to_dataframe().to_pandas()

For large data, ask for pandas’ Arrow-backed dtypes — the conversion is near zero-copy instead of materializing NumPy arrays:

df = CSVReader('data.csv').to_dataframe().to_pandas(use_pyarrow_extension_array=True)

Path 2: Arrow → pandas (the interchange route)

to_arrow_table() returns a PyArrow Table — Arrow is the lingua franca of the modern data stack, and its to_pandas() is the same one Polars uses under the hood:

from datagrunt import ParquetReader

df = ParquetReader('data.parquet').to_arrow_table().to_pandas()

Prefer this route when pandas is just one stop — the same Table also feeds DuckDB, Spark, Ray, or anything else Arrow-native, without re-reading the file.

Path 3: SQL → pandas (the query route)

query_data() returns a DuckDB relation, which converts to any of the three formats — so you can filter and aggregate in SQL first and only convert the result:

from datagrunt import CSVReader

dg = CSVReader('vehicles.csv', engine='duckdb')
rel = dg.query_data(f"SELECT city, COUNT(*) AS n FROM {dg.db_table} GROUP BY 1")

df_pandas = rel.df()      # pandas DataFrame
df_polars = rel.pl()      # Polars DataFrame
table     = rel.arrow()   # PyArrow Table

See Querying with DuckDB for the full querying guide.

Path 4: dicts → anywhere (the escape hatch)

to_dicts() returns plain Python dictionaries — no library in the middle at all:

import pandas as pd
from datagrunt import ExcelReader

df = pd.DataFrame(ExcelReader('workbook.xlsx').to_dicts(sheet='Products'))

This is the slowest path (it round-trips through Python objects), but it works with literally anything and is handy for small data, tests, and JSON-adjacent code.

Choosing a path

The reverse trip is just as easy — Datagrunt never holds your data hostage on the way out either: every writer exports to CSV, Excel, JSON, JSONL, and Parquet, all readable by pandas (pd.read_csv, pd.read_parquet, …) or anything else.