[GH-ISSUE #1678] S3 attachment uploads fail with HTTP/2 PROTOCOL_ERROR on DigitalOcean Spaces #1169

New issue

Closed

opened 2026-05-07 00:30:46 +02:00 by BreizhHardware · 1 comment

BreizhHardware commented

2026-05-07 00:30:46 +02:00

Owner

Originally created by @binwiederhier on GitHub (Mar 27, 2026).
Original GitHub issue: https://github.com/binwiederhier/ntfy/issues/1678

Bug description

When using DigitalOcean Spaces as the S3 attachment backend, some uploads fail with HTTP/2 stream errors:

uploading object rdHcTL5FTE0s failed: Put "https://nyc3.digitaloceanspaces.com/ntfy-attachments/rdHcTL5FTE0s": 
http2: Transport: cannot retry err [stream error: stream ID 19; PROTOCOL_ERROR; received from peer] 
after Request.Body was written; define Request.GetBody to avoid this error

Users see an HTTP 500 (ntfy error 50001) response when publishing messages with attachments.

Root cause

Go's default HTTP client negotiates HTTP/2 via ALPN. Some S3-compatible providers (DigitalOcean Spaces, MinIO, and others) have incomplete or buggy HTTP/2 implementations that can send RST_STREAM with PROTOCOL_ERROR mid-upload. Go's HTTP/2 transport cannot retry the request because the streaming body has already been consumed (no GetBody is defined).

This is a well-documented issue across multiple projects:

rclone added --s3-disable-http2 to work around it: https://github.com/rclone/rclone/issues/4673
Docker registry hit the same issue with S3 storage: https://github.com/distribution/distribution/issues/2570
Go's HTTP/2 transport has known stream accounting bugs: https://github.com/golang/go/issues/42777
rclone is considering disabling HTTP/2 by default for all S3 backends: https://forum.rclone.org/t/frq-disable-http2-as-standard/49467

Fix

Add a disable_http2=true query parameter to the S3 URL configuration that forces the S3 client to use HTTP/1.1 only:

attachment-cache-dir: "s3://KEY:SECRET@BUCKET/PREFIX?region=nyc3&endpoint=https://nyc3.digitaloceanspaces.com&disable_http2=true"

When set, the HTTP client is constructed with TLSNextProto set to an empty map, which prevents HTTP/2 ALPN negotiation — the same approach used by rclone.

Originally created by @binwiederhier on GitHub (Mar 27, 2026). Original GitHub issue: https://github.com/binwiederhier/ntfy/issues/1678 ## Bug description When using DigitalOcean Spaces as the S3 attachment backend, some uploads fail with HTTP/2 stream errors: ``` uploading object rdHcTL5FTE0s failed: Put "https://nyc3.digitaloceanspaces.com/ntfy-attachments/rdHcTL5FTE0s": http2: Transport: cannot retry err [stream error: stream ID 19; PROTOCOL_ERROR; received from peer] after Request.Body was written; define Request.GetBody to avoid this error ``` Users see an HTTP 500 (ntfy error 50001) response when publishing messages with attachments. ## Root cause Go's default HTTP client negotiates HTTP/2 via ALPN. Some S3-compatible providers (DigitalOcean Spaces, MinIO, and others) have incomplete or buggy HTTP/2 implementations that can send `RST_STREAM` with `PROTOCOL_ERROR` mid-upload. Go's HTTP/2 transport cannot retry the request because the streaming body has already been consumed (no `GetBody` is defined). This is a well-documented issue across multiple projects: - rclone added `--s3-disable-http2` to work around it: https://github.com/rclone/rclone/issues/4673 - Docker registry hit the same issue with S3 storage: https://github.com/distribution/distribution/issues/2570 - Go's HTTP/2 transport has known stream accounting bugs: https://github.com/golang/go/issues/42777 - rclone is considering disabling HTTP/2 by default for all S3 backends: https://forum.rclone.org/t/frq-disable-http2-as-standard/49467 ## Fix Add a `disable_http2=true` query parameter to the S3 URL configuration that forces the S3 client to use HTTP/1.1 only: ```yaml attachment-cache-dir: "s3://KEY:SECRET@BUCKET/PREFIX?region=nyc3&endpoint=https://nyc3.digitaloceanspaces.com&disable_http2=true" ``` When set, the HTTP client is constructed with `TLSNextProto` set to an empty map, which prevents HTTP/2 ALPN negotiation — the same approach used by rclone.