How to Optimize Parquet and Columnar Storage for Query Cost

Last updated: June 2026 · Written by Morten Andersen and reviewed by Fredrik Filipsson · See, Cut, Lock, Run method

Columnar formats like Apache Parquet and Apache ORC exist to make analytics cheap, but only if the data is laid out for the way it is queried. A scan-priced engine such as Amazon Athena charges by the data it reads, so two tables holding identical information can cost wildly different amounts to query depending on compression, partitioning, file size, and how queries are written. The good news is that layout is a one-time job with permanent payback: every query against the table runs cheaper afterward. This guide is part of our complete guide to cloud storage and data cost optimization, the cluster pillar it links up to.

What drives query cost on Parquet and columnar data?

Query cost on columnar data is driven by bytes scanned, which compression, partitioning, file layout, and column selection all control. Scan-priced engines bill per terabyte read: Amazon Athena, for example, charges by the amount of data scanned per query, per the Amazon Athena pricing page. That single fact reframes the whole problem. You are not paying for the query, you are paying for the bytes it touches, so anything that reduces bytes touched reduces cost. Parquet helps because it is columnar: a query reading three of forty columns reads roughly three columns worth of data, not the whole row. Compression helps because the engine reads the compressed bytes. Partitioning helps because the engine skips whole partitions that cannot match the filter. The discipline is to make every one of those mechanisms work, rather than dumping raw rows into the lake and paying to scan them forever.

How do I lower Parquet query cost step by step?

Lower query cost by reshaping the data so engines read less of it. Work through these steps:

1. Convert raw data to a columnar format. Convert CSV and JSON to Parquet or ORC so engines read only the columns a query needs instead of every byte of every row. This alone often cuts scanned bytes by an order of magnitude.
2. Compress with an efficient codec. Apply Snappy for a speed and ratio balance, or ZSTD for tighter compression on colder data. Scan-priced engines read the compressed bytes, so smaller files mean a smaller bill.
3. Partition on common filter columns. Partition the table on the columns queries filter by most, usually date, so the engine prunes entire partitions and never reads them. Pruned partitions cost nothing to scan.
4. Right-size files and row groups. Target files of 128MB to 512MB with sensible row group sizes, so the engine reads efficiently and skips irrelevant blocks using column statistics. Tiny files waste overhead; oversized files limit pruning.
5. Select only needed columns. Replace SELECT star with explicit column lists so the columnar layout actually reduces bytes scanned. A query that reads every column pays as if the format were row-based.

This is the same waste-elimination work in our rightsizing and waste elimination service. The storage side of the same files, compacting small objects and tiering cold partitions, is covered in the sibling guide how to cut data lake storage costs with compaction and tiering.

Does compression actually reduce query cost?

Yes, compression reduces query cost directly because scan-priced engines bill on the compressed bytes they read, not the uncompressed size. A Parquet file compressed with Snappy or ZSTD is several times smaller than the raw data, and the engine pays only to read the compressed form, so the per-query charge falls in proportion. The choice of codec is a trade between compression ratio and decode speed: Snappy decodes fast and compresses moderately, which makes it the common default for frequently queried data, while ZSTD compresses tighter at a modest decode cost, which suits colder data queried less often. Crucially, compression also pairs with Parquet's internal encodings such as dictionary and run-length encoding, so columns with low cardinality compress dramatically. Test codecs on your own column types, because results vary, but in nearly every case compressed columnar data is both cheaper to store and cheaper to query than raw rows.

How do partitioning and column pruning work together?

Partitioning skips whole partitions and column pruning skips whole columns, so used together they cut bytes scanned on two axes at once. Partitioning is horizontal: by organizing data into folders keyed on a filter column such as date or region, the engine reads only the partitions whose keys can match the query predicate and ignores the rest. Column projection is vertical: by selecting only the columns a query needs, the engine reads only those column chunks from each file it does open. A well-laid-out table queried with a date filter and a short column list might read a few hundredths of a percent of the total dataset, which is exactly why two tables with identical contents can differ a hundredfold in query cost. The practical rule is to partition on the columns you filter by, keep partition counts reasonable so metadata stays manageable, and never use SELECT star in a cost-sensitive query. This layout discipline is the analytics counterpart to the request-volume work in our broader storage guidance, and it routes to the same fix in our rightsizing and waste elimination service.

Frequently asked questions

Why is Parquet cheaper to query than CSV?

Parquet is columnar and compressed, so a query reads only the columns it needs rather than scanning entire rows. On scan-priced engines that bill by bytes read, reading fewer columns and compressed bytes directly lowers cost. CSV forces a full row scan, which costs far more for the same result.

Does partitioning lower query cost?

Yes. Partitioning on columns that queries filter by, such as date, lets the engine prune whole partitions and never read them. On scan-priced engines this is one of the largest savings available, because pruned partitions cost nothing to scan. Partition on what you filter by most.

What compression codec is best for query cost?

Snappy is the common default for a balance of speed and ratio, while ZSTD often compresses tighter for cold data where read speed matters less. Both shrink bytes scanned, which lowers cost on scan-priced engines. Test on your own data, since results vary by column types and cardinality.

Does SELECT star increase query cost on Parquet?

Yes. SELECT star forces the engine to read every column, which defeats the point of a columnar format. Projecting only the columns you need lets Parquet skip the rest, so bytes scanned and cost both fall. Always select explicit columns in cost-sensitive queries.

The short version

Query cost on columnar data is set by bytes scanned, so read less. Store data as compressed Parquet or ORC, partition on the columns you filter by, right-size files to 128MB to 512MB, and project only the columns you need. Each lever cuts scanned bytes and they compound, often halving analytics spend. When you want the hot tables re-laid and the query saving proven down, that is the work our rightsizing and waste elimination service delivers.