[GH-ISSUE #1051] Races in `v.User()` - Panic in .handleAccountSubscriptionAdd + stats resetter not working #737

New issue

Open

opened 2026-05-07 00:27:01 +02:00 by BreizhHardware · 3 comments

BreizhHardware commented

2026-05-07 00:27:01 +02:00

Owner

Originally created by @binwiederhier on GitHub (Mar 7, 2024).
Original GitHub issue: https://github.com/binwiederhier/ntfy/issues/1051

goroutine 580278315 [running]:
net/http.(*conn).serve.func1()
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:1868 +0xb9
panic({0xf11200?, 0x300e720?})
	/opt/hostedtoolcache/go/1.21.3/x64/src/runtime/panic.go:920 +0x270
heckel.io/ntfy/v2/server.(*Server).handleAccountSubscriptionAdd(0xc000234700, {0x29c2460, 0xc023b935e0}, 0xc02ed53900, 0xc01770edc6?)
	/home/runner/work/ntfy/ntfy/server/server_account.go:344 +0x84
heckel.io/ntfy/v2/server.(*Server).handleInternal.(*Server).withAccountSync.func22({0x29c2460?, 0xc023b935e0?}, 0x0?, 0x1?)
	/home/runner/work/ntfy/ntfy/server/server_middleware.go:126 +0x33
heckel.io/ntfy/v2/server.(*Server).handleInternal.(*Server).ensureUser.func23({0x29c2460, 0xc023b935e0}, 0x29cff68?, 0xf95be0?)
	/home/runner/work/ntfy/ntfy/server/server_middleware.go:84 +0x56
heckel.io/ntfy/v2/server.(*Server).handleInternal.(*Server).ensureUser.(*Server).ensureUserManager.func85({0x29c2460?, 0xc023b935e0?}, 0x3?, 0x4?)
	/home/runner/work/ntfy/ntfy/server/server_middleware.go:75 +0x2f
heckel.io/ntfy/v2/server.(*Server).handleInternal(0xc000234700, {0x29c2460, 0xc023b935e0}, 0xc02ed53900, 0xc02844f320?)
	/home/runner/work/ntfy/ntfy/server/server.go:465 +0x157e
heckel.io/ntfy/v2/server.(*Server).handle.func1()
	/home/runner/work/ntfy/ntfy/server/server.go:379 +0x45
heckel.io/ntfy/v2/log.(*Event).Timing(0xc0088f5aa0, 0xc024007a60)
	/home/runner/work/ntfy/ntfy/log/event.go:88 +0x3e
heckel.io/ntfy/v2/server.(*Server).handle(0xc000234700, {0x29c2460, 0xc023b935e0}, 0xc02ed53900)
	/home/runner/work/ntfy/ntfy/server/server.go:378 +0x1f5
net/http.HandlerFunc.ServeHTTP(0x10?, {0x29c2460?, 0xc023b935e0?}, 0xc01a72b22c?)
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:2136 +0x29
net/http.(*ServeMux).ServeHTTP(0x4124e5?, {0x29c2460, 0xc023b935e0}, 0xc02ed53900)
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:2514 +0x142
net/http.serverHandler.ServeHTTP({0x29bff30?}, {0x29c2460?, 0xc023b935e0?}, 0x6?)
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:2938 +0x8e
net/http.(*conn).serve(0xc0219385a0, {0x29c39d0, 0xc00038ccc0})
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:2009 +0x5f4
created by net/http.(*Server).Serve in goroutine 40
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:3086 +0x5cb

http: panic serving @: runtime error: invalid memory address or nil pointer dereference
goroutine 580387742 [running]:
net/http.(*conn).serve.func1()
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:1868 +0xb9
panic({0xf11200?, 0x300e720?})
	/opt/hostedtoolcache/go/1.21.3/x64/src/runtime/panic.go:920 +0x270
heckel.io/ntfy/v2/server.(*Server).handleAccountTokenUpdate(0xc000234700, {0x29c2460, 0xc021578b60}, 0xc024c86a00, 0xc00f7a84ec?)
	/home/runner/work/ntfy/ntfy/server/server_account.go:253 +0x95
heckel.io/ntfy/v2/server.(*Server).handleInternal.(*Server).withAccountSync.func16({0x29c2460?, 0xc021578b60?}, 0xf?, 0x1?)
	/home/runner/work/ntfy/ntfy/server/server_middleware.go:126 +0x33
heckel.io/ntfy/v2/server.(*Server).handleInternal.(*Server).ensureUser.func17({0x29c2460, 0xc021578b60}, 0x29cff68?, 0xf95be0?)
	/home/runner/work/ntfy/ntfy/server/server_middleware.go:84 +0x56
heckel.io/ntfy/v2/server.(*Server).handleInternal.(*Server).ensureUser.(*Server).ensureUserManager.func82({0x29c2460?, 0xc021578b60?}, 0x3?, 0x4?)
	/home/runner/work/ntfy/ntfy/server/server_middleware.go:75 +0x2f
heckel.io/ntfy/v2/server.(*Server).handleInternal(0xc000234700, {0x29c2460, 0xc021578b60}, 0xc024c86a00, 0xc029a78670?)
	/home/runner/work/ntfy/ntfy/server/server.go:459 +0x113e
heckel.io/ntfy/v2/server.(*Server).handle.func1()
	/home/runner/work/ntfy/ntfy/server/server.go:379 +0x45
heckel.io/ntfy/v2/log.(*Event).Timing(0xc010d25200, 0xc0185d1a60)
	/home/runner/work/ntfy/ntfy/log/event.go:88 +0x3e
heckel.io/ntfy/v2/server.(*Server).handle(0xc000234700, {0x29c2460, 0xc021578b60}, 0xc024c86a00)
	/home/runner/work/ntfy/ntfy/server/server.go:378 +0x1f5
net/http.HandlerFunc.ServeHTTP(0x448220?, {0x29c2460?, 0xc021578b60?}, 0x792f9a?)
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:2136 +0x29
net/http.(*ServeMux).ServeHTTP(0x30d4220?, {0x29c2460, 0xc021578b60}, 0xc024c86a00)
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:2514 +0x142
net/http.serverHandler.ServeHTTP({0xc0094f2c60?}, {0x29c2460?, 0xc021578b60?}, 0x6?)
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:2938 +0x8e
net/http.(*conn).serve(0xc015d125a0, {0x29c39d0, 0xc00038ccc0})
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:2009 +0x5f4
created by net/http.(*Server).Serve in goroutine 40
	/opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:3086 +0x5cb

Originally created by @binwiederhier on GitHub (Mar 7, 2024). Original GitHub issue: https://github.com/binwiederhier/ntfy/issues/1051 ``` goroutine 580278315 [running]: net/http.(*conn).serve.func1() /opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:1868 +0xb9 panic({0xf11200?, 0x300e720?}) /opt/hostedtoolcache/go/1.21.3/x64/src/runtime/panic.go:920 +0x270 heckel.io/ntfy/v2/server.(*Server).handleAccountSubscriptionAdd(0xc000234700, {0x29c2460, 0xc023b935e0}, 0xc02ed53900, 0xc01770edc6?) /home/runner/work/ntfy/ntfy/server/server_account.go:344 +0x84 heckel.io/ntfy/v2/server.(*Server).handleInternal.(*Server).withAccountSync.func22({0x29c2460?, 0xc023b935e0?}, 0x0?, 0x1?) /home/runner/work/ntfy/ntfy/server/server_middleware.go:126 +0x33 heckel.io/ntfy/v2/server.(*Server).handleInternal.(*Server).ensureUser.func23({0x29c2460, 0xc023b935e0}, 0x29cff68?, 0xf95be0?) /home/runner/work/ntfy/ntfy/server/server_middleware.go:84 +0x56 heckel.io/ntfy/v2/server.(*Server).handleInternal.(*Server).ensureUser.(*Server).ensureUserManager.func85({0x29c2460?, 0xc023b935e0?}, 0x3?, 0x4?) /home/runner/work/ntfy/ntfy/server/server_middleware.go:75 +0x2f heckel.io/ntfy/v2/server.(*Server).handleInternal(0xc000234700, {0x29c2460, 0xc023b935e0}, 0xc02ed53900, 0xc02844f320?) /home/runner/work/ntfy/ntfy/server/server.go:465 +0x157e heckel.io/ntfy/v2/server.(*Server).handle.func1() /home/runner/work/ntfy/ntfy/server/server.go:379 +0x45 heckel.io/ntfy/v2/log.(*Event).Timing(0xc0088f5aa0, 0xc024007a60) /home/runner/work/ntfy/ntfy/log/event.go:88 +0x3e heckel.io/ntfy/v2/server.(*Server).handle(0xc000234700, {0x29c2460, 0xc023b935e0}, 0xc02ed53900) /home/runner/work/ntfy/ntfy/server/server.go:378 +0x1f5 net/http.HandlerFunc.ServeHTTP(0x10?, {0x29c2460?, 0xc023b935e0?}, 0xc01a72b22c?) /opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:2136 +0x29 net/http.(*ServeMux).ServeHTTP(0x4124e5?, {0x29c2460, 0xc023b935e0}, 0xc02ed53900) /opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:2514 +0x142 net/http.serverHandler.ServeHTTP({0x29bff30?}, {0x29c2460?, 0xc023b935e0?}, 0x6?) /opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:2938 +0x8e net/http.(*conn).serve(0xc0219385a0, {0x29c39d0, 0xc00038ccc0}) /opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:2009 +0x5f4 created by net/http.(*Server).Serve in goroutine 40 /opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:3086 +0x5cb ``` ``` http: panic serving @: runtime error: invalid memory address or nil pointer dereference goroutine 580387742 [running]: net/http.(*conn).serve.func1() /opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:1868 +0xb9 panic({0xf11200?, 0x300e720?}) /opt/hostedtoolcache/go/1.21.3/x64/src/runtime/panic.go:920 +0x270 heckel.io/ntfy/v2/server.(*Server).handleAccountTokenUpdate(0xc000234700, {0x29c2460, 0xc021578b60}, 0xc024c86a00, 0xc00f7a84ec?) /home/runner/work/ntfy/ntfy/server/server_account.go:253 +0x95 heckel.io/ntfy/v2/server.(*Server).handleInternal.(*Server).withAccountSync.func16({0x29c2460?, 0xc021578b60?}, 0xf?, 0x1?) /home/runner/work/ntfy/ntfy/server/server_middleware.go:126 +0x33 heckel.io/ntfy/v2/server.(*Server).handleInternal.(*Server).ensureUser.func17({0x29c2460, 0xc021578b60}, 0x29cff68?, 0xf95be0?) /home/runner/work/ntfy/ntfy/server/server_middleware.go:84 +0x56 heckel.io/ntfy/v2/server.(*Server).handleInternal.(*Server).ensureUser.(*Server).ensureUserManager.func82({0x29c2460?, 0xc021578b60?}, 0x3?, 0x4?) /home/runner/work/ntfy/ntfy/server/server_middleware.go:75 +0x2f heckel.io/ntfy/v2/server.(*Server).handleInternal(0xc000234700, {0x29c2460, 0xc021578b60}, 0xc024c86a00, 0xc029a78670?) /home/runner/work/ntfy/ntfy/server/server.go:459 +0x113e heckel.io/ntfy/v2/server.(*Server).handle.func1() /home/runner/work/ntfy/ntfy/server/server.go:379 +0x45 heckel.io/ntfy/v2/log.(*Event).Timing(0xc010d25200, 0xc0185d1a60) /home/runner/work/ntfy/ntfy/log/event.go:88 +0x3e heckel.io/ntfy/v2/server.(*Server).handle(0xc000234700, {0x29c2460, 0xc021578b60}, 0xc024c86a00) /home/runner/work/ntfy/ntfy/server/server.go:378 +0x1f5 net/http.HandlerFunc.ServeHTTP(0x448220?, {0x29c2460?, 0xc021578b60?}, 0x792f9a?) /opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:2136 +0x29 net/http.(*ServeMux).ServeHTTP(0x30d4220?, {0x29c2460, 0xc021578b60}, 0xc024c86a00) /opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:2514 +0x142 net/http.serverHandler.ServeHTTP({0xc0094f2c60?}, {0x29c2460?, 0xc021578b60?}, 0x6?) /opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:2938 +0x8e net/http.(*conn).serve(0xc015d125a0, {0x29c39d0, 0xc00038ccc0}) /opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:2009 +0x5f4 created by net/http.(*Server).Serve in goroutine 40 /opt/hostedtoolcache/go/1.21.3/x64/src/net/http/server.go:3086 +0x5cb ```

BreizhHardware added the

🪲 bug

server

labels

2026-05-07 00:27:01 +02:00

BreizhHardware commented

2026-05-07 00:27:02 +02:00

Author

Owner

@binwiederhier commented on GitHub (Mar 7, 2024):

A theory: I think it's this:

ensureUser() checks if v.User() is nil -> not nil
Some other request does v.SetUser(nil) // in visitor()
When the request gets to the actual handler, v.User() is nil

@binwiederhier commented on GitHub (Mar 7, 2024): A theory: I think it's this: - ensureUser() checks if v.User() is nil -> not nil - Some other request does v.SetUser(nil) // in visitor() - When the request gets to the actual handler, v.User() is nil

BreizhHardware commented

2026-05-07 00:27:02 +02:00

Author

Owner

@binwiederhier commented on GitHub (Apr 5, 2024):

There may be a race condition issue with resetting in-memory stats as well that is likely related to this race.

@binwiederhier commented on GitHub (Apr 5, 2024): There may be a race condition issue with resetting in-memory stats as well that is likely related to this race.

BreizhHardware commented

2026-05-07 00:27:03 +02:00

Author

Owner

@winneratwin commented on GitHub (Mar 18, 2025):

Just wanted to hop in and say that subscribing to topics(only tested websockets) that require authentication before reading also experience this race condition(or a similar one) where the connection will be dropped even when the account is an admin.

Found this out when creating a rust implementation of the ntfy client to minimize concurrent connections(currently it opens a connection for each subscription when less could be used). With 3 connections initiated simultaneously, there is an almost 1/2 chance of getting this race condition leading to one or more of the connections being rejected for being "unauthorized".

Example log line:

2025/03/18 15:33:51 DEBUG Access to topic testing not authorized (error=unauthorized, http_method=GET, http_path=/testing/ws?title=test, tag=http, topic=testing, topic_last_access=2025-03-18T15:33:51.426Z, topic_subscribers=2, user_id=u_66RaB8V7Sw, user_name=winner, [...])

The result of ntfy access is:

user winner (role: admin, tier: none)
- read-write access to all topics (admin role)
user * (role: anonymous, tier: none)
- read-write access to topic pub*
- write-only access to topic up*
- no access to any (other) topics (server config)

@winneratwin commented on GitHub (Mar 18, 2025): Just wanted to hop in and say that subscribing to topics(only tested websockets) that require authentication before reading also experience this race condition(or a similar one) where the connection will be dropped even when the account is an admin. Found this out when creating a rust implementation of the ntfy client to minimize concurrent connections(currently it opens a connection for each subscription when less could be used). With 3 connections initiated simultaneously, there is an almost 1/2 chance of getting this race condition leading to one or more of the connections being rejected for being "unauthorized". Example log line: ``` 2025/03/18 15:33:51 DEBUG Access to topic testing not authorized (error=unauthorized, http_method=GET, http_path=/testing/ws?title=test, tag=http, topic=testing, topic_last_access=2025-03-18T15:33:51.426Z, topic_subscribers=2, user_id=u_66RaB8V7Sw, user_name=winner, [...]) ``` The result of `ntfy access` is: ``` user winner (role: admin, tier: none) - read-write access to all topics (admin role) user * (role: anonymous, tier: none) - read-write access to topic pub* - write-only access to topic up* - no access to any (other) topics (server config) ```