mirror of
https://github.com/binwiederhier/ntfy.git
synced 2026-05-09 08:26:00 +02:00
[GH-ISSUE #19] Auth #18
Labels
No labels
ai-generated
android-app
android-app
android-app
🪲 bug
build
build
dependencies
docs
enhancement
enhancement
🔥 HOT
in-progress 🏃
ios
prio:low
prio:low
pull-request
question
🔒 security
server
server
unified-push
web-app
website
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/ntfy#18
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @dwmccheyne on GitHub (Nov 24, 2021).
Original GitHub issue: https://github.com/binwiederhier/ntfy/issues/19
I would like to require authentication to pub/sub to topics at least on private servers.
I think Google Identity would be a good provider here since authenticating to it is a familiar part of the types of workflows I would like to incorporate this into.
@binwiederhier commented on GitHub (Nov 24, 2021):
I'm not against adding optional authentication, but using Google seems a little against the selfhosting spirit, no?
I'll think about this a little and do some research. I think there are a lot of difficulties implementing any auth. Maybe auth shouldn't be done by ntfy but rather by nginx or some other proxy...
@aslmx commented on GitHub (Nov 25, 2021):
👍
Interim step could be to support basic auth. Haven't tried if that would work?
Curl should support it, but would the android app?
@binwiederhier commented on GitHub (Nov 26, 2021):
I think I'll look at something with basic auth to begin with. Especially since that's incredibly easy to do with curl too.
@aslmx commented on GitHub (Nov 26, 2021):
It would maybe be enough to just make sure the App supports Basic Auth - which your http library, if you use one, might already support.
Then it would be up to the maintainer of the selfhosted instance to put http basic auth in the config of the reverse proxy.
I guess most people will run ntfy behind nginx or Traefik, etc.
They all support basic auth.
So that would be the "cheapest" way of achieving some kind of protection - it would protect everything in one go.
@shadow00 commented on GitHub (Dec 1, 2021):
What implementing authentication via api tokens? For example, the creator of a topic can optionally include a token, and then whoever tries to subscribe to that topic afterwards has to provide the same token (this is effectively a password-protected topic). The content of said token is completely arbitrary (though perhaps should be limited to, say 64-128 characters/bytes).
This could be done for example via the
"Authorization"header, or by simply including a"token"field along with the message/subscription request. Note that the server would not include the token when relaying a message. For public topics this field could be omitted, or assumed/left empty by default.More complex but more flexible variant: topic creator includes an "admin" token and a list of allowed "client" tokens. Anyone providing a valid client token can publish subscribe to the topic, and the list of allowed tokens can me changed only by providing the admin token as authentication. The list of client tokens could even be split into two, separating the clients that can publish to the topic and clients that can subscribe (the same token can be on both lists, and the two capabilities are separate - that way, we have clients that are publish-only, subscribe-only, and both).
This way:
https://ntfy.sh/mytopic/tokens,https://ntfy.sh/mytopic/addtoken,https://ntfy.sh/mytopic/rmtoken,The server administrator can decide via the config if the auth tokens expire with the topic or if they are persistent (can be easily saved in a dedicated json/yaml file).
Example
auth.json:Note how
mytopichas two admin tokens,iqp5qgmKenWjd1I6can only publish,ahPaEtqoY0i8MWwNcan only subscribe, and the other three can do both.As for
topic2, there are no admin tokens - so only those two client tokens would be able to pub/sub, and nobody would be able to change that list except the server administrator.@binwiederhier commented on GitHub (Dec 2, 2021):
Thank you very much for the incredibly detailed suggestion. That's awesome!
So technically a topic is never created actively:
I think this is solvable though.
As you may have noticed from the rest of ntfy, I like simple things, so I am not a big fan of the admin + client token idea (for now at least). I can totally imagine this being useful eventually, but for now I think your first idea can be adapted to work with the ephemeral topics.
How about this:
Topic-specfic auth
When you publish or subscribe to a topic, you can additionally send a
Authorization: token ...header as you suggested, e.g.or
Authorizationheader is present and the topic is not known, the topic is reserved fortopic-auth-reservation: 24h(configurable), during which time the topic will not be reaped and only requests with a valid auth can publish/subscribetopic-auth-reservationto reconnect before the topic expires.Server-wide auth
Instead of doing a topic specific
Authorization, you configure aserver-auth: mysecrettokenor something which applies to all topics. This would be the simplest way to make a private server.All configs:
Thoughts?
@shadow00 commented on GitHub (Dec 2, 2021):
Lol, I guess I got a little carried away. I like your idea very much - both the topic-specific auth, as well as the server-wide auth (and it doesn't preclude the implementation of some kind of authorization logic on top of it in the future ;) )
How would you handle the case of
topic-auth-reservation < cache-duration? I see the following options:What about making the per-topic auth duration dynamically configurable? Eg. by providing an additional (possibly optional)
Reservation-periodheader (or maybe something with a better name). If the header is provided, use that when setting the grace period and renewing the topic; otherwise, the server-configured value is used by default. Server-side, this second parameter would be stored along with the auth token inside your reservation list.@aslmx commented on GitHub (Dec 2, 2021):
Haven't fully understood the last comments about tokens.
But please consider: adding API Tokens that I have to setup first for a topic will void the key selling point of ntfy for me and basically I could stick to use gotify.
@shadow00 commented on GitHub (Dec 2, 2021):
@aslmx it would be an optional feature. If you want, you can create a topic that requires a custom api key to publish/subscribe. If you don't want, you can keep using ntfy the same way you can do now, without any api key.
@rigrig commented on GitHub (Dec 3, 2021):
Maybe there could also be a
auth-mode: publish: everybody can subscribe (if you know the topic), but you need theserver-authtoken to publish?And I guess it would be nice if
server-authcould be a list, so you can hand out different tokens to different publishers.@binwiederhier commented on GitHub (Dec 3, 2021):
@aslmx Yeah don't worry, I'll leave the "no auth" a default. Every auth option I'll add will be opt-in, just like @shadow00 said
@rigrig If you read this comment from @shadow00, he had very similar ideas. I desperately want to keep things as simple as possible, and I also don't want to re-invent the wheel. I think I may need to read up on how others do auth, and how to do this properly. I don't really want to necessarily roll my own here.
That said, I think I can iterate on this. I can implement the
server-wideone pretty easily and push that out, but I think I'll need a little bit of time for the others to get it right. Please keep suggestions coming!!@rigrig commented on GitHub (Dec 3, 2021):
I understand wanting to keep it simple, I just figured it would be nice to restrict what gets published to my server without clients needing to bother with tokens. (i.e.
publishwould be the same asserver-wide, except you don't need a token to subscribe.)@binwiederhier commented on GitHub (Dec 30, 2021):
Thank you for your comment. Fascinating! NATS sounds great. Didn't know it existed. It looks like a classic pubsub implementation. I'll look at it further, but I think it's probably a little too much for this little project, don't you think?
I would have to replace the guts of ntfy with NATS and for what? I am a strong believer in KISS, and this seems like the opposite of KISS. I'll mull it over a little though.
Also: This ticket is about Auth, so maybe we should move this convo somewhere else. Feel free to open a NATS ticket :-D
@binwiederhier commented on GitHub (Dec 30, 2021):
Just an update on the general status of this ticket: After the unified push ticket (#9), I'll work on this.
@gc-ss commented on GitHub (Dec 31, 2021):
On the topic of NATS (yes! Definitely use it!) I have had great experience with an OSS layer on top of NATS: LiftBridge/NATS
https://github.com/liftbridge-io/liftbridge
Tyler and his team is fantastic and has OSS the very product he is building his company on.
Just drop by on their Slack to ask any questions you have - no matter how basic.
LiftBridge is a great product that makes use of NATS.
@binwiederhier commented on GitHub (Jan 1, 2022):
I kindly ask to move NATS discussions here: #78 :-)
@nmoseman commented on GitHub (Jan 20, 2022):
The only thing I need would be to have basic auth support in the client. I'm going to be running it behind a proxy like nginx anyways to enforce https-only and I can use that to set auth.
I have made the mistake of running things 'open' on the internet before and they just up ripe for abuse. Somebody will figure out how to abuse it. I won't do that for anything that allows people to use a service to send data.
@binwiederhier commented on GitHub (Jan 20, 2022):
I have read a lot about auth these last few days and the more I read, the less I want to do anything fancy. I will likely try to iterate on this, starting with basic auth with a static key for everything. The Android app has to support it obviously, so you can't just put configs in nginx.
I don't disagree. People on the Internet are evil and auth is definitely better than no auth if it's a service just for you. That said, I have jumped through endless hoops to limit everything to the extreme: Subscription rate limits, requests per second, attachment size limits, cache size limits, topic limits, ... (see https://ntfy.sh/docs/config/#rate-limiting). And then there's also fail2ban to lock out the abusers (https://ntfy.sh/docs/config/#banning-bad-actors-fail2ban).
The problem is typically though as a self-hoster, you want NO LIMITS, you want to upload a 500 MB file to your phone whenever you want, and that's obviously only possible with auth.
Long story short. I get it. I'll do auth, soon :-D
@binwiederhier commented on GitHub (Jan 22, 2022):
This WIP PR (https://github.com/binwiederhier/ntfy/pull/114) implements this approach:
In this example,
philcan do everything,bencan read and write only toalertsandmariancan read fromalerts. Anonymous users can read fromannouncements.There will be the following config params:
ntfy.sh will be set to
auth-default-read/write: true, since everyone can write/read from everything.I like this design a lot. It's super simple, yet infinitely flexible.
Feedback required: What I am unsure about is
/var/cache/ntfy/cache.dbthat contains messages. And now there will be/var/lib/ntfy/user.db. We could just combine them to/var/lib/ntfy/database.db.@binwiederhier commented on GitHub (Jan 24, 2022):
Thank you for your response. I think you must have misunderstood my comment. I was merely talking about one sqlite database vs. two, not about two services/binaries. For now I opted to a different database for user+access management.
Here's an excerpt from the user/access management as I have implemented it currently:
@gc-ss commented on GitHub (Jan 24, 2022):
@binwiederhier, a different and dedicated sqlite database per usecase makes a lot of sense and is really appreciated.
For example, I can now apply (https://litestream.io/) to the user+access management sqlite database and have a fairly reliable user+access serdes for ntfy on less reliable commodity servers.
Still not as reliable as LiftBridge/NATS but way simpler and practically close for very low user velocity!
@cmeis commented on GitHub (Jan 24, 2022):
Oops, haven't pulled over my comments from the discord, will do now:
+1 for separate databases (user auth / cache)
I would suggest making access rules with wildcards at least a possibility. For starters (to be honest: for most cases) I'd like to see an asterisk wildcard, i.e. an access rule for "datacenter1-*" matching all topics starting with "datacenter1-".
@binwiederhier commented on GitHub (Jan 26, 2022):
Still working on this. The server side is solid and working, and I've written tests for pretty much everything. I've also started wireframing the Android changes. It's gonna be a lot of work unfortunately.
@julianfoad commented on GitHub (Jan 27, 2022):
Even though you've gone down this road, I still want to record what I mentioned in the chat a few weeks ago, about how I'm disillusioned with trying to self-host services that take the Do-It-Yourself Auth approach with their own user-password database, their own rules for valid usernames, their own system of roles and permissions, their own security model, and expect me to directly expose their own security implementation to the internet while running on my home server. Mainly my concern is not about security against hackers but about my ability to manage consistent logins, usernames, password updates (and preferably single-sign-on too) across several small self-hosted services.
The best way to support self-hosting (and I mean to say this to all self-hostable projects) is to provide the simple ability to connect to standard auth mechanisms.
I wish you could even now reconsider instead to make the client compatible with external (reverse-proxy based) auth standards such as http-basic (the simplest) and (the next step up) the session-cookie based "forward auth" flow which is described well in Authelia's architecture doc page. Authelia is just one such auth front-end, not a special case.
With those kinds of auth schemes, adding awareness of "users" to the server is completely optional. You can start with a system where just "successful authentication" according to the external auth front-end is enough and once past that then whoever authenticated will have full access to the server (the old existing ntfy server before you started adding auth awareness). Then if you want to add "user accounts" to the server you don't need to add authentication code but merely look at the "Remote-User" HTTP header on every request, which is added for you by the reverse proxy in combination with the auth front-end -- explained in Authelia - How can the backend be aware of the authenticated users?
@julianfoad commented on GitHub (Jan 27, 2022):
Actually the "forward auth" flow is documented perhaps more generically in Traefik docs.
Example, snippets from my Authelia configuration.yml:
and users_database.yml:
and my Traefik-2 middleware rule, which reference from the routing rule of each service that I want protected:
@binwiederhier commented on GitHub (Jan 27, 2022):
I appreciate all the input. I will review all of this later. I did do a whole lot of research and asked coworkers about what to do here. My conclusion was that there is no standard way of doing this, at least not for the use case o want.
That said, the implementation I made is very easy to replace or even augment. For example, the authentication can likely be delegated entirely to another system (i e. username and password), while the authorization is kept in ntfy. I am more than happy to integrate other systems, i just didn't know which since they all didn't seem to fit.
Like, for instance: oauth2 had a web based flow. If I want to support curl with user/pass, then do I return "unauthorized" with a web link in curl..?! You get the problem.
I'd really appreciate detailed help on how to integrate one of these solutions. I am happy to accommodate them, as long as they can stand next to the no-dependency-integrated one.
@binwiederhier commented on GitHub (Jan 27, 2022):
I gotta say Authelia and Traefik look very interesting, though this is what I found with most of these solutions:
Neither the curl use case nor the Android app are "web applications". If there is an easy way to accommodate these two, then I'd be game. I'll look some more. @julianfoad I'm also more than happy to discuss on Discord/Matrix.
As for Zitadel, @gedw99, it says
The "only" here for me is almost ironic. :-)
@gc-ss commented on GitHub (Jan 27, 2022):
Concur that Zitadelle is a fantastic piece of software. There used to be issues of running it just with docker but I worked with their team to tease those issues out -
docker-composeshould work right OOB now (If not, please let me know).They also have a generous free tier.
That said, for something lightweight like
ntfyauthentik is extremely simple and more interestingly - wouldn't requirentfyto be auth aware at all (depends on what level of authz we wantntfyto support) - which checks the no-dependency-integrated boxIf Authelia looks like a good fit, but you need "non webapp" support - authentik should be a great solution. They don't have any getting started guides (I will be trying to fix that soon in a few months) - so just run the
docker-composeand check it out right away or jump in on the chat with any questions.https://goauthentik.io/
@binwiederhier commented on GitHub (Jan 28, 2022):
Progress
@binwiederhier commented on GitHub (Jan 30, 2022):
Progress:

@binwiederhier commented on GitHub (Jan 31, 2022):
Alright folks this is "it".
I am soliciting feedback now that it's feature complete. I find it a little awkward and complicated. It's definitely not the "just works" kinda thing you'd expect.
Notes:
a) The "subscription settings" are a little weird, because they only contain the login stuff right now. There will be topic-specific settings (notification sound, paused notifications, min priority, ...) too later.
b) I was thinking about visual indicators for which user a topic is using, e.g. little person indicators with a number or color. What do you think?
c) I think I definitely have to indicate in the main view when the auth fails. Right now it just says "reconnecting ..." forever.
Here's a full video of the entire flow:
https://phil.nopaste.net/7xfchbUB31?a=0zi01nlZsJ
@gc-ss commented on GitHub (Jan 31, 2022):
I prefer things that are accessible to majority of people - so number over color (colorblindness)
@Mek101 commented on GitHub (Jan 31, 2022):
I would remove "You can also add a user when adding a topic"
@Mek101 commented on GitHub (Jan 31, 2022):
What does the Material Design guidelines say about that?
@karmanyaahm commented on GitHub (Jan 31, 2022):
When anonymous has no access to any topics, does that include write access?
If so, shouldn't all topics that are UnifiedPush have write access for anonymous (so app servers can POST messages)? Maybe giving topics /up.* anonymous write by default could solve that.
@binwiederhier commented on GitHub (Jan 31, 2022):
I agree. I'll do that.
There's a setting
auth-default-access = [deny-all|read-only|read-write]that defines the anonymous access behavior. I could obviously also make awrite-onlyfor anonymous access, I didn't think there was a use case. Now there is one :-)A UnifiedPush topic is just a topic, so if you make it
/up*, then you effectively let people use the server as if it was fully open (with a small name restriction). Right now, pattern based ACL rules are not supported, though it has been asked for by @cmeis before, and since you're effectively the second person to ask, I may just implement that.For UP, I'd say the best we can do if you want a "private server" is what you described:
auth-default-access = deny-allinserver.ymlntfy access everyone "up*" write-onlyntfy user add --role admin myuserand assign that in the Android app (not done yet).@binwiederhier commented on GitHub (Jan 31, 2022):
The more I think about it, the more I think I should disallow multiple users per server (in the app), i.e. you can only have one user per server and that is used for all topics. What do you think about that? It feels quite unnatural to allow multiple users for the same servers from within one app.
@cmeis commented on GitHub (Jan 31, 2022):
Yeah, that would make sense. Thought about it a new times, and I don't come up with a situation where one might need different users to the same server (testing scenarios aside).
@binwiederhier commented on GitHub (Feb 1, 2022):
Video:
https://phil.nopaste.net/YhbTac9AoL?a=YgMgHVdXRv
Alright I am much happier with this. Now the app only supports one user per host, meaning there is no more awkward dropdown and the "login required" screen is only shown once when the user does not exist.
I also removed the "Subscription settings" entirely (for now) and added the mechanism that PeterCxy suggested to avoid exposing messages for non-public Firebase topics via a "poll_request" message (not shown in the video).
I think this is what i'll stick with. Now "all I need to do" is polish it a little more and add lots of docs.
For those who want to test, here are binaries:
ntfy binary: https://phil.nopaste.net/ntfy-auth-1?a=BA0OLM8HiE
ntfy Android app: https://phil.nopaste.net/ntfy-auth-app-1.apk?a=AuUjHSlgpk
@binwiederhier commented on GitHub (Feb 4, 2022):
Server side auth code is released in https://github.com/binwiederhier/ntfy/releases/tag/v1.14.0
Note to future self, so I don't lose the links:
@binwiederhier commented on GitHub (Feb 5, 2022):
The Android app will be released soon, so I'd consider this done for now. Thanks everyone for the fantastic discussion and the input. 👍 👍 👍
If you have any other desires regarding auth, please open another ticket. One that I would see would be "allow integrating X" to use as external auth backend, an I'll likely do it if it's reasonable.
@Kranzes commented on GitHub (Mar 14, 2024):
Hey, is it possible to authenticate via my own OpenID Connect Identity Provider?