Résultats illimités avec déversement sur disque

fonctionnel

Diffusez de grands résultats avec contre-pression et déversement sur disque afin qu'une seule réponse énorme ne monopolise pas la machine.

Un très grand ensemble de résultats ne devrait pas devenir un très gros problème de mémoire. Les lignes transitent par un canal avec contre-pression et, au-delà d'un seuil, se déverser sur le disque, so the grid pages from an on-disk store whether a query returns 50 rows or 50 million.

Le chemin emprunté par une ligne

  1. The driver batches. Rows come off the wire in batches with a flush timer, so a slow query still shows you what it has rather than nothing.
  2. The channel pushes back. The queue between driver and UI is bounded. If the grid cannot keep up, the driver waits, which is what stops a fast server from filling memory faster than the app can drain it.
  3. Past the threshold, it spills. Beyond 50,000 rows (ou un 64 MiB per-set memory budget, whichever comes first) the set is written to a typed binary spill file with its own row-offset index, and the in-memory copy is released.
  4. The grid pages. A spill-backed set keeps a window of rows resident around your viewport and re-centres it, on a background task, as you scroll. The index is paged too, so the cost does not grow with the size of the set.

You keep working while this happens. The grid paints as rows arrive rather than waiting for completion, the live row count updates as it goes, and repaints are coalesced so streaming a million rows does not spend its time redrawing.

Pivot and chart every row

Le Pivot et Graphique tabs work on spill-backed sets too. Each time you change the layout they read the whole spill file on a background task, showing Pivoting… ou Charting… with a percentage, and memory stays flat however many rows there are. Drill-through works as well: double-click a pivot cell, or click a bar, slice, or histogram bin, and SQLly collects exactly the rows behind it into a temporary file of their own and shows them on the Grid tab under a banner. Effacer brings back every row.

Honest about the trade

Ce que coûte le débordement. Un ensemble en mode débordement est sur disque, pas en mémoire, donc sort, filter, and diff are disabled for it. Those need the whole set at once, and quietly sorting the visible window would be a lie. Export is fine: it pages the complete set straight from the file. Pivot and chart are fine too, as above, though on a very large set each change takes a moment longer while the file is read.

Spill files are temporary and owned by the run that made them. Closing the tab or starting a new query deletes them, along with any drill-through files made from them, and a superseded query cleans up after itself even if its results arrive late.

Dans le navigateur

Le version navigateur (the live preview, and SQLly Embedded pages) spills too. A browser tab has no disk to write to, so past the same threshold the rows go to the browser's Origin Private File System, private storage the browser keeps for the site. A small background worker owns the files, because only a worker can read and write them synchronously; the app sends it rows in the same spill format the desktop uses and asks it for the rows on screen as you scroll, so the tab's memory stays flat while a multi-million-row result scrolls like a short one.

  • What works: scrolling, View Row as Form, copy and export (the first 100,000 rows, like an in-memory export on the desktop), Top Values and profiling, Pivot and Chart over every row, and pivot drill-through.
  • Pivot and chart read the stored rows back the first time you open them on a result. The rows are held in the browser's memory, outside the app's own, until the result is replaced.
  • Cleanup: the files go when the result is replaced or closed. A tab that closes before it can tidy up leaves them behind, and the next SQLly page in that browser removes them on startup; files another open tab is still using are left alone.
  • When it can't: a browser without OPFS, a private window that refuses it, or one with less than about 64 MB of storage free keeps the whole result in memory, as before, and says so in Messages. If storage fills up partway through, SQLly keeps the rows it managed to store and tells you how many.

L’ajuster

Les deux réglages se trouvent dans Paramètres › Performance › Optimisations des résultats de requête: le seuil de lignes et le budget de mémoire par ensemble (1 Mio à 2 Gio). Augmentez-les si vous avez de la mémoire à disposition et que vous voulez trier des ensembles plus grands ; réduisez-les sur une machine modeste.