[GH-ISSUE #980] perf: shared route scan cache to eliminate 3-5x duplicate directory scans #212

New issue

Closed

opened 2026-05-06 12:38:13 +02:00 by BreizhHardware · 1 comment

BreizhHardware commented

2026-05-06 12:38:13 +02:00

Owner

Originally created by @Divkix on GitHub (Apr 30, 2026).
Original GitHub issue: https://github.com/cloudflare/vinext/issues/980

Problem

The app/pages directories are scanned from scratch at 5+ different phases, with no shared cache:

Phase	File:Line	Function	Impact
Virtual module load (RSC entry)	`index.ts:1656`	`appRouter()`	Full glob scan `/page` + `/route`
Virtual module load (root-params)	`index.ts:1692`	`appRouter()` again	Full glob scan (redundant)
Every API request (dev)	`index.ts:2666`	`apiRouter()`	Per-request glob scan
Every pages SSR request (dev)	`index.ts:2696`	`pagesRouter()`	Per-request glob scan
Build: prerender	`build/run-prerender.ts:204,236-237`	All three routers	Build-time scans
Build: report/nitro	`build/report.ts:926-937`, `build/nitro-route-rules.ts:40-46`	All three routers	Build-time scans

Each appRouter() call runs scanWithExtensions("**/page", ...) + scanWithExtensions("**/route", ...) sequentially. These are full directory glob operations.

Current caching

app-router.ts:144-152: Module-level cache (cachedRoutes, cachedAppDir, cachedPageExtensionsKey). Invalidated entirely by invalidateAppRouteCache() on any file change (all-or-nothing).
pages-router.ts:25-37: Map<string, ...> keyed by pages:${dir}:${extensions}. Also all-or-nothing invalidation.

Both caches exist but are isolated — they don't communicate between the different call sites. The RSC entry load at index.ts:1656 scans, then the root-params load at index.ts:1692 scans the same directory again 30 lines later.

Why it matters

Dev startup: 500-1500ms wasted on redundant glob scans
First request: 300-800ms from regenerating virtual modules (which trigger rescans)
File changes: 200-500ms — full cache clear on any change (invalidateAppRouteCache() at index.ts:2052-2071)
Per-request latency (Pages Router): 20-100ms from pagesRouter() called every request at index.ts:2696

Proposed fix

Create a single shared route cache on the plugin instance closure in index.ts that all consumers reference:

Store cached routes in plugin instance scope (alongside root, appDir, pagesDir, etc. at index.ts:521-560)
Pass cached routes to appRouter(), pagesRouter(), apiRouter() instead of having them scan
Scan once on startup, invalidate incrementally via file watcher
For per-request call sites (index.ts:2666, index.ts:2696): use cached routes, don't rescan

For incremental invalidation: instead of invalidateAppRouteCache() clearing everything, use the file watcher's add/unlink events to add/remove individual routes. The trie (route-trie.ts) can be updated incrementally (insert/remove operations already exist).

Estimated files touched: 5-7

File	Change
`index.ts`	Hold shared cache on plugin closure; wire file watcher for incremental updates; update 5+ call sites
`routing/app-router.ts`	Accept pre-scanned files or expose incremental add/remove APIs
`routing/pages-router.ts`	Same
`entries/pages-server-entry.ts`	Accept routes from caller
`entries/pages-client-entry.ts`	Same
`build/run-prerender.ts`	Pass through shared cache
`build/report.ts`, `build/nitro-route-rules.ts`	Trivial

Difficulty: Medium

Plumbing work. Caching infrastructure exists, just needs to be unified and made incremental.

Expected improvement

Dev startup: 40-60% faster
First request: 50-70% faster
File changes: 30-50% faster
Pages Router requests: 15-30ms saved per request

Originally created by @Divkix on GitHub (Apr 30, 2026). Original GitHub issue: https://github.com/cloudflare/vinext/issues/980 ### Problem The app/pages directories are scanned from scratch at 5+ different phases, with no shared cache: | Phase | File:Line | Function | Impact | |-------|-----------|----------|--------| | Virtual module load (RSC entry) | `index.ts:1656` | `appRouter()` | Full glob scan `**/page` + `**/route` | | Virtual module load (root-params) | `index.ts:1692` | `appRouter()` again | Full glob scan (redundant) | | Every API request (dev) | `index.ts:2666` | `apiRouter()` | Per-request glob scan | | Every pages SSR request (dev) | `index.ts:2696` | `pagesRouter()` | Per-request glob scan | | Build: prerender | `build/run-prerender.ts:204,236-237` | All three routers | Build-time scans | | Build: report/nitro | `build/report.ts:926-937`, `build/nitro-route-rules.ts:40-46` | All three routers | Build-time scans | Each `appRouter()` call runs `scanWithExtensions("**/page", ...)` + `scanWithExtensions("**/route", ...)` sequentially. These are full directory glob operations. ### Current caching - `app-router.ts:144-152`: Module-level cache (`cachedRoutes`, `cachedAppDir`, `cachedPageExtensionsKey`). Invalidated entirely by `invalidateAppRouteCache()` on any file change (all-or-nothing). - `pages-router.ts:25-37`: `Map<string, ...>` keyed by `pages:${dir}:${extensions}`. Also all-or-nothing invalidation. Both caches exist but are isolated — they don't communicate between the different call sites. The RSC entry load at `index.ts:1656` scans, then the root-params load at `index.ts:1692` scans the same directory again 30 lines later. ### Why it matters - **Dev startup:** 500-1500ms wasted on redundant glob scans - **First request:** 300-800ms from regenerating virtual modules (which trigger rescans) - **File changes:** 200-500ms — full cache clear on any change (`invalidateAppRouteCache()` at `index.ts:2052-2071`) - **Per-request latency (Pages Router):** 20-100ms from `pagesRouter()` called every request at `index.ts:2696` ### Proposed fix Create a single shared route cache on the plugin instance closure in `index.ts` that all consumers reference: 1. Store cached routes in plugin instance scope (alongside `root`, `appDir`, `pagesDir`, etc. at `index.ts:521-560`) 2. Pass cached routes to `appRouter()`, `pagesRouter()`, `apiRouter()` instead of having them scan 3. Scan once on startup, invalidate incrementally via file watcher 4. For per-request call sites (`index.ts:2666`, `index.ts:2696`): use cached routes, don't rescan For incremental invalidation: instead of `invalidateAppRouteCache()` clearing everything, use the file watcher's `add`/`unlink` events to add/remove individual routes. The trie (`route-trie.ts`) can be updated incrementally (insert/remove operations already exist). ### Estimated files touched: 5-7 | File | Change | |------|--------| | `index.ts` | Hold shared cache on plugin closure; wire file watcher for incremental updates; update 5+ call sites | | `routing/app-router.ts` | Accept pre-scanned files or expose incremental add/remove APIs | | `routing/pages-router.ts` | Same | | `entries/pages-server-entry.ts` | Accept routes from caller | | `entries/pages-client-entry.ts` | Same | | `build/run-prerender.ts` | Pass through shared cache | | `build/report.ts`, `build/nitro-route-rules.ts` | Trivial | ### Difficulty: Medium Plumbing work. Caching infrastructure exists, just needs to be unified and made incremental. ### Expected improvement - Dev startup: 40-60% faster - First request: 50-70% faster - File changes: 30-50% faster - Pages Router requests: 15-30ms saved per request

BreizhHardware closed this issue

2026-05-06 12:38:13 +02:00

BreizhHardware commented

2026-05-06 12:38:14 +02:00

Author

Owner

@Divkix commented on GitHub (Apr 30, 2026):

Closing: existing module-level caches already prevent the described redundant scans.

app-router.ts:174: cache hit on second call within same invalidation cycle
pages-router.ts:58-59: cached promise returned on all per-request calls
app-router.ts:174 + pages-router.ts:204-206: dev middleware never rescans

All caches invalidate correctly on file watcher add/unlink events. No redundant glob operations observed.

@Divkix commented on GitHub (Apr 30, 2026): Closing: existing module-level caches already prevent the described redundant scans. - `app-router.ts:174`: cache hit on second call within same invalidation cycle - `pages-router.ts:58-59`: cached promise returned on all per-request calls - `app-router.ts:174` + `pages-router.ts:204-206`: dev middleware never rescans All caches invalidate correctly on file watcher add/unlink events. No redundant glob operations observed.

BreizhHardware referenced this issue

2026-05-06 12:39:35 +02:00

[PR #212] [MERGED] fix: use function form of glob exclude for Node < 22.14 compat #392