[PR #1047] [MERGED] feat: support non-standard robots directives via `other` field #1047

New issue

Closed

opened 2026-05-06 13:11:44 +02:00 by BreizhHardware · 0 comments

BreizhHardware commented

2026-05-06 13:11:44 +02:00

Owner

📋 Pull Request Information

Original PR: https://github.com/cloudflare/vinext/pull/1047
Author: @NathanDrake2406
Created: 5/4/2026
Status: ✅ Merged
Merged: 5/4/2026
Merged by: @james-elicx

Base: main ← Head: fix/1010-support-robots-other

📝 Commits (1)

069fb66 fix: support non-standard robots directives via other field (#1010)

📊 Changes

2 files changed (+93 additions, -0 deletions)

View changed files

📝 packages/vinext/src/server/metadata-routes.ts (+12 -0)
📝 tests/metadata-routes.test.ts (+81 -0)

📄 Description

Fixes #1010

Summary

Adds other field to RobotsRule type accepting Record<string, string | number | Array<string | number>>
Emits non-standard per-user-agent directives after Crawl-delay and before the blank line separating blocks
Array values expand into repeated lines; null/undefined entries are skipped
Mirrors Next.js behavior from https://github.com/vercel/next.js/pull/93206

Test plan

5 new unit tests in tests/metadata-routes.test.ts covering: directive position, array expansion, null/undefined skipping, multiple directives, numeric values
All 82 metadata-routes tests pass; lint & check clean

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/cloudflare/vinext/pull/1047 **Author:** [@NathanDrake2406](https://github.com/NathanDrake2406) **Created:** 5/4/2026 **Status:** ✅ Merged **Merged:** 5/4/2026 **Merged by:** [@james-elicx](https://github.com/james-elicx) **Base:** `main` ← **Head:** `fix/1010-support-robots-other` --- ### 📝 Commits (1) - [`069fb66`](https://github.com/cloudflare/vinext/commit/069fb66ec3734e6e072a968169290c7c81b1a10c) fix: support non-standard robots directives via `other` field (#1010) ### 📊 Changes **2 files changed** (+93 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `packages/vinext/src/server/metadata-routes.ts` (+12 -0) 📝 `tests/metadata-routes.test.ts` (+81 -0) </details> ### 📄 Description Fixes #1010 ## Summary - Adds `other` field to `RobotsRule` type accepting `Record<string, string | number | Array<string | number>>` - Emits non-standard per-user-agent directives after `Crawl-delay` and before the blank line separating blocks - Array values expand into repeated lines; `null`/`undefined` entries are skipped - Mirrors Next.js behavior from https://github.com/vercel/next.js/pull/93206 ## Test plan - 5 new unit tests in `tests/metadata-routes.test.ts` covering: directive position, array expansion, null/undefined skipping, multiple directives, numeric values - All 82 metadata-routes tests pass; lint & check clean --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>