Unbounded, spill-to-disk results

✓ functional

Stream large results with backpressure and disk spillover so one huge answer does not eat the machine.

A very large result set should not turn into a very large memory problem. Rows stream through a back-pressured channel and, past a threshold, spill to disk - so the grid pages from an on-disk store whether a query returns 50 rows or 50 million.

The path a row takes

  1. The driver batches. Rows come off the wire in batches with a flush timer, so a slow query still shows you what it has rather than nothing.
  2. The channel pushes back. The queue between driver and UI is bounded. If the grid cannot keep up, the driver waits - which is what stops a fast server from filling memory faster than the app can drain it.
  3. Past the threshold, it spills. Beyond 50,000 rows (or a 64 MiB per-set memory budget, whichever comes first) the set is written to a typed binary spill file with its own row-offset index, and the in-memory copy is released.
  4. The grid pages. A spill-backed set keeps a window of rows resident around your viewport and re-centres it - on a background task - as you scroll. The index is paged too, so the cost does not grow with the size of the set.

You keep working while this happens. The grid paints as rows arrive rather than waiting for completion, the live row count updates as it goes, and repaints are coalesced so streaming a million rows does not spend its time redrawing.

Honest about the trade

What spilling costs. A spill-backed set is on disk, not in memory, so sort, filter, pivot, and diff are disabled for it - those need the whole set at once, and quietly sorting the visible window would be a lie. Export is fine: it pages the complete set straight from the file.

Spill files are temporary and owned by the run that made them. Closing the tab or starting a new query deletes them; a superseded query cleans up after itself even if its results arrive late.

Tuning it

Both knobs live in Settings › Performance › Query Result Optimizations: the row threshold and the per-set memory budget (1 MiB to 2 GiB). Raise them if you have memory to spare and want sorting on bigger sets; lower them on a modest machine.