starred/vinext

Fork 0

mirror of https://github.com/cloudflare/vinext.git synced 2026-05-09 08:25:34 +02:00

[GH-ISSUE #454] Track stable Next.js API surface with a checked-in manifest and CI coverage gate #102

New issue

Closed

opened 2026-05-06 12:37:13 +02:00 by BreizhHardware · 3 comments

BreizhHardware commented

2026-05-06 12:37:13 +02:00

Owner

Originally created by @Divkix on GitHub (Mar 11, 2026).
Original GitHub issue: https://github.com/cloudflare/vinext/issues/454

Summary

vinext already has strong compatibility infrastructure:

upstream-ported tests in tests/nextjs-compat/TRACKING.md
a migration/compatibility scanner in packages/vinext/src/check.ts
CI coverage in .github/workflows/ci.yml
daily ecosystem validation in .github/workflows/ecosystem.yml
advisory canary testing in .github/workflows/tip.yml

What we do not have yet is a machine-readable compatibility model for stable Next.js releases.

Today, compatibility knowledge is spread across:

the inline shim alias map in packages/vinext/src/index.ts
shim source files under packages/vinext/src/shims/
user-facing support maps in packages/vinext/src/check.ts
ported Next.js tests in tests/nextjs-compat/

That works, but it means we do not have a single CI-enforced answer to:

what is the stable public next/* API surface for version x.y.z?
which exports are implemented by vinext?
which missing exports are intentional vs accidental?
when did stable Next.js add or remove something we should track?

This issue proposes a stable-only compatibility tracking system built around a checked-in manifest extracted from the published next npm package.

Goals

Track stable Next.js public runtime API surface from the published package.
Check in a canonical api-manifest.json for the currently supported stable version.
Enforce shim coverage in CI.
Document intentional gaps explicitly.
Add a small behavioral contract suite for high-risk parity cases.
Keep canary as advisory signal only.

Non-Goals

Do not make canary a required merge gate.
Do not use check.ts as the canonical compatibility model.
Do not block this work on type-export tracking in v1.
Do not block this work on a broad ecosystem internal-import scanner.
Do not rewrite the existing Next.js compat suite.

Decisions

v1 tracks runtime exports only.
Weekly stable tracking opens an issue, not an auto-PR.
Ecosystem workflows should pin to the exact manifest version, not a floating minor like "16.1".
check.ts remains a user-facing migration tool, not the internal engineering source of truth.

Current vs Proposed

Area	Current	Proposed
Source of truth	Inline shim map in `index.ts` + hand-maintained support maps in `check.ts`	Checked-in `api-manifest.json` + exported shim registry + `known-gaps.json`
Stable release tracking	Manual / ad hoc	Automated stable-only diff workflow
PR protection	Existing CI/E2E/ecosystem checks	Existing checks + required shim coverage gate
Behavioral drift detection	Ported tests and ecosystem builds	Existing tests + small dual-target contracts
Canary	Advisory only	Still advisory only

Architecture

Current:

             Next.js stable changes
                    |
              humans notice
                    |
   +----------------+----------------+
   |                                 |
ported tests                    manual shim updates
(TRACKING.md)                   (index.ts + shims)
   |                                 |
   +----------------+----------------+
                    |
                   CI

Proposed:

          published next@stable release
                    |
         extract-nextjs-api script
                    |
             api-manifest.json
                    |
   +----------------+----------------+
   |                                 |
weekly diff workflow         PR coverage gate
(open issue)                 (fail on missing exports
                             unless in known-gaps.json)
   |                                 |
   +----------------+----------------+
                    |
        existing CI + contract tests

Registry refactor:

TODAY
index.ts
  - owns inline nextShimMap
  - resolve.alias uses it
  - resolveId uses it

PROPOSED
shims/registry.ts
  - public shim entries
  - internal shim entries
  - shared helper(s)

index.ts
  - imports registry

coverage script
  - imports same registry

Proposed Work

1. Extract a stable public API manifest

Add:

scripts/extract-nextjs-api.mjs
api-manifest.json

Behavior:

download or npm pack the published stable next@x.y.z
inspect public next/* entrypoints from the published package
record runtime exports only in v1
store version metadata in the manifest

Example shape:

{
  "version": "16.1.6",
  "modules": {
    "next/cache": {
      "runtime": [
        "unstable_cache",
        "revalidatePath",
        "revalidateTag",
        "updateTag",
        "refresh"
      ]
    }
  }
}

Notes:

Prefer the published npm package over repo source as the source of truth.
Runtime-only keeps v1 simpler and covers what actually breaks apps.
Wrapper modules are mostly explicit; next/navigation may require a one-hop resolution into compiled output.

2. Move the shim registry into a shared module

Add:

packages/vinext/src/shims/registry.ts

Update:

packages/vinext/src/index.ts

Behavior:

move the inline nextShimMap into a reusable exported registry
keep public next/* entries separate from internal next/dist/* aliases
make both the plugin and the coverage script consume the same registry

This avoids parsing index.ts as source text in CI.

3. Add a required shim coverage check

Add:

scripts/check-shim-coverage.mjs
known-gaps.json

Update:

.github/workflows/ci.yml

Behavior:

read api-manifest.json
read the shared shim registry
inspect actual named/default exports from shim files
fail if a stable public export is missing unless explicitly listed in known-gaps.json
fail if a manifest module has no mapped shim and no documented gap

Suggested known-gaps.json shape:

{
  "next/jest": {
    "exports": ["*"],
    "status": "wont-fix",
    "reason": "vinext uses Vitest, not Jest"
  },
  "next/amp": {
    "exports": ["useAmp"],
    "status": "stub",
    "reason": "Deprecated API; shimmed behavior is intentionally minimal"
  }
}

Suggested statuses:

planned
stub
wont-fix

This lets reviewers distinguish intentional non-goals from temporary gaps.

4. Add stable-only tracking workflow

Add:

.github/workflows/nextjs-api-track.yml
optionally scripts/diff-nextjs-api.mjs

Behavior:

run weekly
check npm view next version
compare against api-manifest.json
if unchanged, exit
if changed, extract new manifest and open one tracking issue with the diff

This should target stable releases only.

5. Add a small dual-target contract suite

Add:

tests/contracts/
tests/fixtures/contract-app/

Initial contracts:

redirect() status behavior
cookies() mutability gating
headers() read-only behavior
middleware request header propagation
generateMetadata() merge chain

Behavior:

same fixture app should run under vinext and real stable Next.js
use as signal, not a required PR gate in v1
run on schedule or workflow dispatch against stable Next.js

6. Keep canary advisory-only

Update:

.github/workflows/tip.yml

Behavior:

keep next@canary as non-blocking early warning
optionally run a focused compat subset
do not make canary failures block PRs or releases

Testing / Validation

This work needs tests at three layers: script correctness, shim coverage enforcement, and behavioral parity.

1. Script-level tests

Add Vitest coverage for the new scripts and helpers.

Suggested files:

tests/api-manifest.test.ts
tests/shim-coverage.test.ts

These should cover:

extracting runtime exports from explicit wrapper modules
handling module.exports = require(...) style wrappers
handling the one-hop next/navigation style case
diffing old vs new manifests
failing when a manifest export has no shim and no known gap
passing when a missing export is explicitly listed in known-gaps.json

Prefer fixture-based tests over hitting the network.

Suggested fixtures:

tests/fixtures/next-api-manifest/

2. Registry refactor safety

The registry extraction from index.ts must not change runtime behavior.

Validation:

existing shim tests continue to pass
existing compat tests continue to pass
resolve.alias and resolveId still handle:
- next/*
- next/*.js
- internal next/dist/* aliases

Minimum targeted test runs:

pnpm test tests/shims.test.ts
pnpm test tests/nextjs-compat/

3. Contract tests

Add a small dual-target suite for behavioral parity.

Suggested files:

tests/contracts/redirect.contract.test.ts
tests/contracts/request-apis.contract.test.ts
tests/contracts/middleware.contract.test.ts
tests/contracts/metadata.contract.test.ts

Initial cases:

redirect() status behavior
cookies() mutability gating
headers() read-only behavior
middleware request header propagation
generateMetadata() merge chain

These should run against the same fixture app under:

vinext
real stable Next.js

4. CI / workflow validation

Required on PRs:

shim coverage check against api-manifest.json

Non-blocking / scheduled:

weekly stable release manifest diff
advisory canary signal in tip.yml
dual-target contract run against stable Next.js

5. Definition of done

This issue is complete when:

api-manifest.json is checked in and versioned
known-gaps.json is checked in and required for intentional omissions
the shim registry no longer lives only as inline state in index.ts
CI fails if a stable public next/* export is missing without a documented gap
new scripts have direct Vitest coverage
the registry refactor does not regress existing shim/compat tests
at least 3-5 dual-target contract tests are in place
a weekly stable-only tracking workflow exists
tip.yml remains advisory only

Follow-ups

Out of scope for v1, but natural follow-ups:

type-export tracking
ecosystem internal-import scanner for next/dist/*
auto-PR generation instead of issue-only tracking
tighter coupling between manifest version and ecosystem workflow version pinning

Why this is worth doing

This does not replace the current test suite or ecosystem jobs. It fills a different gap: machine-enforced API drift detection for stable releases.

That gives us:

a canonical compatibility model
fewer silent regressions
explicit documentation of intentional gaps
a cleaner path to tracking new stable Next.js releases

Originally created by @Divkix on GitHub (Mar 11, 2026). Original GitHub issue: https://github.com/cloudflare/vinext/issues/454 ## Summary vinext already has strong compatibility infrastructure: - upstream-ported tests in `tests/nextjs-compat/TRACKING.md` - a migration/compatibility scanner in `packages/vinext/src/check.ts` - CI coverage in `.github/workflows/ci.yml` - daily ecosystem validation in `.github/workflows/ecosystem.yml` - advisory canary testing in `.github/workflows/tip.yml` What we do not have yet is a machine-readable compatibility model for stable Next.js releases. Today, compatibility knowledge is spread across: - the inline shim alias map in `packages/vinext/src/index.ts` - shim source files under `packages/vinext/src/shims/` - user-facing support maps in `packages/vinext/src/check.ts` - ported Next.js tests in `tests/nextjs-compat/` That works, but it means we do not have a single CI-enforced answer to: - what is the stable public `next/*` API surface for version `x.y.z`? - which exports are implemented by vinext? - which missing exports are intentional vs accidental? - when did stable Next.js add or remove something we should track? This issue proposes a stable-only compatibility tracking system built around a checked-in manifest extracted from the published `next` npm package. ## Goals - Track stable Next.js public runtime API surface from the published package. - Check in a canonical `api-manifest.json` for the currently supported stable version. - Enforce shim coverage in CI. - Document intentional gaps explicitly. - Add a small behavioral contract suite for high-risk parity cases. - Keep canary as advisory signal only. ## Non-Goals - Do not make canary a required merge gate. - Do not use `check.ts` as the canonical compatibility model. - Do not block this work on type-export tracking in v1. - Do not block this work on a broad ecosystem internal-import scanner. - Do not rewrite the existing Next.js compat suite. ## Decisions - v1 tracks runtime exports only. - Weekly stable tracking opens an issue, not an auto-PR. - Ecosystem workflows should pin to the exact manifest version, not a floating minor like `"16.1"`. - `check.ts` remains a user-facing migration tool, not the internal engineering source of truth. ## Current vs Proposed | Area | Current | Proposed | | --- | --- | --- | | Source of truth | Inline shim map in `index.ts` + hand-maintained support maps in `check.ts` | Checked-in `api-manifest.json` + exported shim registry + `known-gaps.json` | | Stable release tracking | Manual / ad hoc | Automated stable-only diff workflow | | PR protection | Existing CI/E2E/ecosystem checks | Existing checks + required shim coverage gate | | Behavioral drift detection | Ported tests and ecosystem builds | Existing tests + small dual-target contracts | | Canary | Advisory only | Still advisory only | ## Architecture Current: ```text Next.js stable changes | humans notice | +----------------+----------------+ | | ported tests manual shim updates (TRACKING.md) (index.ts + shims) | | +----------------+----------------+ | CI ``` Proposed: ```text published next@stable release | extract-nextjs-api script | api-manifest.json | +----------------+----------------+ | | weekly diff workflow PR coverage gate (open issue) (fail on missing exports unless in known-gaps.json) | | +----------------+----------------+ | existing CI + contract tests ``` Registry refactor: ```text TODAY index.ts - owns inline nextShimMap - resolve.alias uses it - resolveId uses it PROPOSED shims/registry.ts - public shim entries - internal shim entries - shared helper(s) index.ts - imports registry coverage script - imports same registry ``` ## Proposed Work ### 1. Extract a stable public API manifest Add: - `scripts/extract-nextjs-api.mjs` - `api-manifest.json` Behavior: - download or `npm pack` the published stable `next@x.y.z` - inspect public `next/*` entrypoints from the published package - record runtime exports only in v1 - store version metadata in the manifest Example shape: ```json { "version": "16.1.6", "modules": { "next/cache": { "runtime": [ "unstable_cache", "revalidatePath", "revalidateTag", "updateTag", "refresh" ] } } } ``` Notes: - Prefer the published npm package over repo source as the source of truth. - Runtime-only keeps v1 simpler and covers what actually breaks apps. - Wrapper modules are mostly explicit; `next/navigation` may require a one-hop resolution into compiled output. ### 2. Move the shim registry into a shared module Add: - `packages/vinext/src/shims/registry.ts` Update: - `packages/vinext/src/index.ts` Behavior: - move the inline `nextShimMap` into a reusable exported registry - keep public `next/*` entries separate from internal `next/dist/*` aliases - make both the plugin and the coverage script consume the same registry This avoids parsing `index.ts` as source text in CI. ### 3. Add a required shim coverage check Add: - `scripts/check-shim-coverage.mjs` - `known-gaps.json` Update: - `.github/workflows/ci.yml` Behavior: - read `api-manifest.json` - read the shared shim registry - inspect actual named/default exports from shim files - fail if a stable public export is missing unless explicitly listed in `known-gaps.json` - fail if a manifest module has no mapped shim and no documented gap Suggested `known-gaps.json` shape: ```json { "next/jest": { "exports": ["*"], "status": "wont-fix", "reason": "vinext uses Vitest, not Jest" }, "next/amp": { "exports": ["useAmp"], "status": "stub", "reason": "Deprecated API; shimmed behavior is intentionally minimal" } } ``` Suggested statuses: - `planned` - `stub` - `wont-fix` This lets reviewers distinguish intentional non-goals from temporary gaps. ### 4. Add stable-only tracking workflow Add: - `.github/workflows/nextjs-api-track.yml` - optionally `scripts/diff-nextjs-api.mjs` Behavior: - run weekly - check `npm view next version` - compare against `api-manifest.json` - if unchanged, exit - if changed, extract new manifest and open one tracking issue with the diff This should target stable releases only. ### 5. Add a small dual-target contract suite Add: - `tests/contracts/` - `tests/fixtures/contract-app/` Initial contracts: - `redirect()` status behavior - `cookies()` mutability gating - `headers()` read-only behavior - middleware request header propagation - `generateMetadata()` merge chain Behavior: - same fixture app should run under vinext and real stable Next.js - use as signal, not a required PR gate in v1 - run on schedule or workflow dispatch against stable Next.js ### 6. Keep canary advisory-only Update: - `.github/workflows/tip.yml` Behavior: - keep `next@canary` as non-blocking early warning - optionally run a focused compat subset - do not make canary failures block PRs or releases ## Testing / Validation This work needs tests at three layers: script correctness, shim coverage enforcement, and behavioral parity. ### 1. Script-level tests Add Vitest coverage for the new scripts and helpers. Suggested files: - `tests/api-manifest.test.ts` - `tests/shim-coverage.test.ts` These should cover: - extracting runtime exports from explicit wrapper modules - handling `module.exports = require(...)` style wrappers - handling the one-hop `next/navigation` style case - diffing old vs new manifests - failing when a manifest export has no shim and no known gap - passing when a missing export is explicitly listed in `known-gaps.json` Prefer fixture-based tests over hitting the network. Suggested fixtures: - `tests/fixtures/next-api-manifest/` ### 2. Registry refactor safety The registry extraction from `index.ts` must not change runtime behavior. Validation: - existing shim tests continue to pass - existing compat tests continue to pass - `resolve.alias` and `resolveId` still handle: - `next/*` - `next/*.js` - internal `next/dist/*` aliases Minimum targeted test runs: - `pnpm test tests/shims.test.ts` - `pnpm test tests/nextjs-compat/` ### 3. Contract tests Add a small dual-target suite for behavioral parity. Suggested files: - `tests/contracts/redirect.contract.test.ts` - `tests/contracts/request-apis.contract.test.ts` - `tests/contracts/middleware.contract.test.ts` - `tests/contracts/metadata.contract.test.ts` Initial cases: - `redirect()` status behavior - `cookies()` mutability gating - `headers()` read-only behavior - middleware request header propagation - `generateMetadata()` merge chain These should run against the same fixture app under: - vinext - real stable Next.js ### 4. CI / workflow validation Required on PRs: - shim coverage check against `api-manifest.json` Non-blocking / scheduled: - weekly stable release manifest diff - advisory canary signal in `tip.yml` - dual-target contract run against stable Next.js ### 5. Definition of done This issue is complete when: - `api-manifest.json` is checked in and versioned - `known-gaps.json` is checked in and required for intentional omissions - the shim registry no longer lives only as inline state in `index.ts` - CI fails if a stable public `next/*` export is missing without a documented gap - new scripts have direct Vitest coverage - the registry refactor does not regress existing shim/compat tests - at least 3-5 dual-target contract tests are in place - a weekly stable-only tracking workflow exists - `tip.yml` remains advisory only ## Follow-ups Out of scope for v1, but natural follow-ups: - type-export tracking - ecosystem internal-import scanner for `next/dist/*` - auto-PR generation instead of issue-only tracking - tighter coupling between manifest version and ecosystem workflow version pinning ## Why this is worth doing This does not replace the current test suite or ecosystem jobs. It fills a different gap: machine-enforced API drift detection for stable releases. That gives us: - a canonical compatibility model - fewer silent regressions - explicit documentation of intentional gaps - a cleaner path to tracking new stable Next.js releases

BreizhHardware closed this issue

2026-05-06 12:37:13 +02:00

BreizhHardware commented

2026-05-06 12:37:14 +02:00

Author

Owner

@Divkix commented on GitHub (Mar 11, 2026):

Implemented on feat/api-surface-tracking. #461

What landed:

Stable API manifest extraction now discovers public runtime entrypoints from the published next package instead of using a hard-coded module list.
The checked-in api-manifest.json now tracks 26 runtime modules for next@16.1.6.
The tracking workflow in .github/workflows/nextjs-api-track.yml now works from the published artifact and de-dupes issues by version marker.
Shim coverage is enforced against the stable manifest plus known-gaps.json.
Shared behavioral contracts are now truly dual-target for portable HTTP behaviors in tests/contracts/http.contract.test.ts and pass against both vinext and real next dev.

Important scope clarifications:

Manifest tracking is runtime-only in v1.
Some discovered public modules are intentionally marked wont-fix in known-gaps.json (next/babel, next/client, next/jest, and the experimental test helpers).
Extra migration shims can still exist outside manifest coverage if they are not part of the current stable published runtime surface.
The server-action redirect 303 check is covered in tests/app-router.test.ts, not in the shared contract suite, because that flow depends on vinext-specific action transport.

Verification:

pnpm test tests/api-manifest.test.ts tests/manifest-diff.test.ts tests/shim-coverage.test.ts
pnpm test tests/app-router.test.ts -t "uses 303 for server action redirects"
pnpm test tests/contracts/http.contract.test.ts
CONTRACT_TARGET=nextjs pnpm test tests/contracts/http.contract.test.ts
pnpm test tests/shims.test.ts
manifest parity against the published next@16.1.6 package
coverage parity check against the updated manifest
pnpm run typecheck
pnpm run lint
pnpm run fmt:check

@Divkix commented on GitHub (Mar 11, 2026): Implemented on `feat/api-surface-tracking`. #461 What landed: - Stable API manifest extraction now discovers public runtime entrypoints from the published `next` package instead of using a hard-coded module list. - The checked-in `api-manifest.json` now tracks 26 runtime modules for `next@16.1.6`. - The tracking workflow in `.github/workflows/nextjs-api-track.yml` now works from the published artifact and de-dupes issues by version marker. - Shim coverage is enforced against the stable manifest plus `known-gaps.json`. - Shared behavioral contracts are now truly dual-target for portable HTTP behaviors in `tests/contracts/http.contract.test.ts` and pass against both vinext and real `next dev`. Important scope clarifications: - Manifest tracking is runtime-only in v1. - Some discovered public modules are intentionally marked `wont-fix` in `known-gaps.json` (`next/babel`, `next/client`, `next/jest`, and the experimental test helpers). - Extra migration shims can still exist outside manifest coverage if they are not part of the current stable published runtime surface. - The server-action redirect `303` check is covered in `tests/app-router.test.ts`, not in the shared contract suite, because that flow depends on vinext-specific action transport. Verification: - `pnpm test tests/api-manifest.test.ts tests/manifest-diff.test.ts tests/shim-coverage.test.ts` - `pnpm test tests/app-router.test.ts -t "uses 303 for server action redirects"` - `pnpm test tests/contracts/http.contract.test.ts` - `CONTRACT_TARGET=nextjs pnpm test tests/contracts/http.contract.test.ts` - `pnpm test tests/shims.test.ts` - manifest parity against the published `next@16.1.6` package - coverage parity check against the updated manifest - `pnpm run typecheck` - `pnpm run lint` - `pnpm run fmt:check`

BreizhHardware commented

2026-05-06 12:37:14 +02:00

Author

Owner

@Divkix commented on GitHub (Mar 11, 2026):

Related to #204. This issue tracks the API surface (what exports exist), while #204 does a behavioral audit (whether implementations match Next.js behavior). They complement each other - the manifest tells us what to implement, the audit verifies we implement it correctly. Not blocking.

@Divkix commented on GitHub (Mar 11, 2026): Related to #204. This issue tracks the API surface (what exports exist), while #204 does a behavioral audit (whether implementations match Next.js behavior). They complement each other - the manifest tells us what to implement, the audit verifies we implement it correctly. Not blocking.

BreizhHardware commented

2026-05-06 12:37:14 +02:00

Author

Owner

@Divkix commented on GitHub (Apr 14, 2026):

Looks like it has already been solved with a github workflow.

@Divkix commented on GitHub (Apr 14, 2026): Looks like it has already been solved with a github workflow.

BreizhHardware referenced this issue

2026-05-06 12:39:08 +02:00

[PR #102] [MERGED] fix: normalize request pathname before middleware and route matching #311