Automatic query caching

The database enables cost-aware query caching by default. A result qualifies when its execution scans at least 128 candidate rows, returns at most 25% as many rows, and stays within the configured output-size bound. Second-use admission avoids encoding and retaining one-off results; a third sequential execution can hit. A concurrent repeat can satisfy second-use admission during the first execution. This is a cheap structural heuristic, not a measured CPU-cost model.

import { openMemory } from 'defuss-db';

const db = await openMemory();
const docs = db.collection<{ id: number; category: number; amount: number }>('docs');
await docs.insertMany(Array.from({ length: 512 }, (_, id) => ({
  id, category: id % 8, amount: id,
})));
const q = docs.q;
const totals = {
  groupBy: { category: q.field('category') },
  measures: { count: q.count(), sum: q.sum(q.field('amount')) },
};
await docs.aggregate(totals); // Compile and compute; remember first use.
await docs.aggregate(totals); // Reuse plan, compute, encode and admit.
await docs.aggregate(totals); // Reuse plan; fresh snapshot, then decode cached result.
console.log(db.queryCacheStats());
db.clearQueryCache();
await db.close();

find, findOne, query, select, aggregate, view, count, and prepared evaluate support result caching. get/getMany retain their direct point-read paths. One-shot primary-key equality queries skip automatic key creation and implicit plan reuse because they cannot meet the minimum scan threshold. Automatic profiling requests execute normally so profiling remains useful. Streams and transaction reads bypass result caching.

Control Behavior
Omitted cache, default database options Automatic qualification and second-use admission
{ cache: 'auto' } Automatic qualification even in manual mode
{ cache: false } Execute without result caching; one-shot methods also skip implicit plan reuse
{ cache: true } Override scan/output-ratio heuristics; retain admission policy and byte limits
{ cache: 'refresh' } Recompute, attempt immediate admission, do not coalesce
Open with { queryCache: { mode: 'manual' } } Use result caching on explicit request. Prepared plans can still be reused
Open with { queryCache: false } Disable both result and compiled-plan caches

Prepared methods accept a second cacheKey boolean (default true). Set false to skip key preparation and plan reuse for plans that will never cache. A previously prepared plan remains usable after clearing the caches.

Bounds and policy

const db = await openMemory({ queryCache: {
  mode: 'auto', minScannedRows: 128, maxOutputRatio: 0.25,
  policy: 'slru', admission: 'second-hit',
  maxEntries: 128, maxBytes: 16 * 1024 * 1024,
  maxEntryBytes: 2 * 1024 * 1024, maxKeyBytes: 16 * 1024,
  maxPlans: 64, maxPlanBytes: 1024 * 1024, maxPendingQueries: 16,
} });

These are the defaults. Storage is allocated on use. policy: 'lru' | 'slru'; admission: 'always' | 'second-hit'. SLRU uses probation and protected recency lists; hits promote entries, and the protected list is limited to 80% of entry count and byte budget. Eviction prefers probation. Second-use history is exact and bounded to twice maxEntries and one eighth of maxBytes. Automatically rejected results remove their history entry.

The 16 MiB result budget accounts for encoded values, UTF-16 keys, live-scope keys, estimated entry overhead and history. Each entry must also fit 2 MiB. Compiled plans have a separate 1 MiB/64-entry LRU budget: key bytes plus a 2 KiB per-plan estimate. Pending queries have a separate count bound of 16 distinct keys and reported key/overhead estimates. These are accounting limits, not total JS heap/RSS bounds. Query working memory, decoded return values, compiler allocations and caller-held prepared plans remain outside these budgets.

Exactness and ownership

A hit reports the caller's pinned database revision in summary.revision. Scan and profile counts describe the computation that populated the entry. Use queryCacheStats() to observe caching.

Concurrent misses and clearing

Live, signal-free concurrent requests with the same collection version, descriptor, page and budgets may share one execution. A follower is returned independent decoded bytes only when the leader's result qualifies and fits the cache. Otherwise followers execute independently; this avoids retaining unbounded intermediate results. Different budgets do not coalesce. Signal-bearing requests execute independently on misses, preserving cancellation behavior. Errors are propagated to waiting callers and the pending slot is always released.

clearQueryCache() clears results, history, version lookups and plans, and advances an epoch. Old pending executions remain tracked until settlement but cannot refill the cleared cache; new requests cannot join them. Existing waiters complete or recompute. Counters remain cumulative. Database close clears retained entries and normal lifetime checks reject use after close.

versionReads counts persisted collection-marker lookups. crossRevisionHits counts decodes whose returned revision was updated to the caller's head. Other stats expose hits, misses, admissions, rejections, evictions, invalidations, coalesced followers, plan hits/misses, entries/bytes, history entries/bytes, plans/plan bytes, and pending queries/bytes. autoBypasses counts post-compute heuristic rejections; early bypasses such as primary-key reads, profiling, unsupported keys and disabled caching do not create result lookups.

Admission costs

Unconditional retention adds avoidable serialization and memory costs to point reads and full-output scans. Automatic qualification, second-use admission, bounded plans and exact miss sharing make repeated compact queries benefit without per-call configuration. Explicit controls remain for workloads the heuristic misclassifies. Cold scans and cached results use the same historical count query.

SLRU plus bounded second-use history is not full LFU or TinyLFU. A Bloom filter before an exact in-memory Map lookup would add work without avoiding storage I/O. Storage-layer negative-read filtering remains a separate, unproven optimization requiring revision-consistent maintenance and its own benchmark.

Design references: Caffeine design, Caffeine eviction, TinyLFU paper, RocksDB Bloom filters. This implementation does not embed those implementations.