[GH-ISSUE #26] [SELinux] TOTP don't seems to work

BreizhHardware commented

2026-05-07 00:17:09 +02:00

Owner

Originally created by @doubleailes on GitHub (Oct 31, 2020).
Original GitHub issue: https://github.com/ovh/the-bastion/issues/26

I set TOTP on my account.

But after the sucessfull registration, all my verification are refused.

Is there any way to get my account back?
With the scratch code?

Originally created by @doubleailes on GitHub (Oct 31, 2020). Original GitHub issue: https://github.com/ovh/the-bastion/issues/26 I set TOTP on my account. But after the sucessfull registration, all my verification are refused. Is there any way to get my account back? With the scratch code?

BreizhHardware

2026-05-07 00:17:09 +02:00

closed this issue
added the
enhancement
label

BreizhHardware commented

2026-05-07 00:17:11 +02:00

Author

Owner

@speed47 commented on GitHub (Oct 31, 2020):

The scratch codes can help if for some reason you no longer have the TOTP app or it no longer works correctly. Each code can only be used once, however.

To disable TOTP for your account you can use the --osh selfMFAResetTOTP command. It requires the TOTP of course, but you might want to try a scratch code there.

Once you've unlocked yourself, to be able to help you better, please describe the steps you followed to enable TOTP on your account.

@speed47 commented on GitHub (Oct 31, 2020): The scratch codes can help if for some reason you no longer have the TOTP app or it no longer works correctly. Each code can only be used once, however. To disable TOTP for your account you can use the `--osh selfMFAResetTOTP` command. It requires the TOTP of course, but you might want to try a scratch code there. Once you've unlocked yourself, to be able to help you better, please describe the steps you followed to enable TOTP on your account.

BreizhHardware commented

2026-05-07 00:17:11 +02:00

Author

Owner

@doubleailes commented on GitHub (Oct 31, 2020):

Ok i tried the --osh selfMFAResetTOTP and insert my scratch code but i did not work
Of course i tried a couple ones in the list.

I'm running Centos8 in VM. My timedate seems update.

I used the command --osh selfMFASetupTOTP

and get the result:

Warning: pasting the following URL into your browser exposes the OTP secret to Google:
*****
Failed to use libqrencode to show QR code visually for scanning.
Consider typing the OTP secret into your app manually.
Your new secret key is: ****..***
Enter code from app (-1 to skip): ******
Code confirmed
Your emergency scratch codes are:
  *******
  *******
  *******
  *******
  *******

@doubleailes commented on GitHub (Oct 31, 2020): Ok i tried the `--osh selfMFAResetTOTP` and insert my **scratch code** but i did not work Of course i tried a couple ones in the list. I'm running Centos8 in VM. My **timedate** seems update. I used the command `--osh selfMFASetupTOTP` and get the result: ```bash Warning: pasting the following URL into your browser exposes the OTP secret to Google: ***** Failed to use libqrencode to show QR code visually for scanning. Consider typing the OTP secret into your app manually. Your new secret key is: ****..*** Enter code from app (-1 to skip): ****** Code confirmed Your emergency scratch codes are: ******* ******* ******* ******* ******* ```

BreizhHardware commented

2026-05-07 00:17:12 +02:00

Author

Owner

@speed47 commented on GitHub (Oct 31, 2020):

Did you install pamtester?
Can you give a screenshot of the error you get when trying to login with the TOTP?

@speed47 commented on GitHub (Oct 31, 2020): Did you install `pamtester`? Can you give a screenshot of the error you get when trying to login with the TOTP?

BreizhHardware commented

2026-05-07 00:17:12 +02:00

Author

Owner

@doubleailes commented on GitHub (Oct 31, 2020):

nope i did install pamtester should it be install on the bastion?

here is the login exemple:

$ bssh --osh selfMFAResetTOTP
*------------------------------------------------------------------------------*
|THIS IS A PRIVATE COMPUTER SYSTEM, UNAUTHORIZED ACCESS IS STRICTLY PROHIBITED.|
|ALL CONNECTIONS ARE LOGGED. IF YOU ARE NOT AUTHORIZED, DISCONNECT NOW.        |
*------------------------------------------------------------------------------*
Enter passphrase for key '/home/******/.ssh/id_ed25519': 
Multi-Factor Authentication enabled, an additional authentication factor is required (OTP).
Verification code: 
Multi-Factor Authentication enabled, an additional authentication factor is required (OTP).
Verification code: 
Multi-Factor Authentication enabled, an additional authentication factor is required (OTP).
Verification code: 
***@***.***.***.***: Permission denied (keyboard-interactive).

@doubleailes commented on GitHub (Oct 31, 2020): nope i did install `pamtester` should it be install on the **bastion**? here is the login exemple: ```bash $ bssh --osh selfMFAResetTOTP *------------------------------------------------------------------------------* |THIS IS A PRIVATE COMPUTER SYSTEM, UNAUTHORIZED ACCESS IS STRICTLY PROHIBITED.| |ALL CONNECTIONS ARE LOGGED. IF YOU ARE NOT AUTHORIZED, DISCONNECT NOW. | *------------------------------------------------------------------------------* Enter passphrase for key '/home/******/.ssh/id_ed25519': Multi-Factor Authentication enabled, an additional authentication factor is required (OTP). Verification code: Multi-Factor Authentication enabled, an additional authentication factor is required (OTP). Verification code: Multi-Factor Authentication enabled, an additional authentication factor is required (OTP). Verification code: ***@***.***.***.***: Permission denied (keyboard-interactive). ```

BreizhHardware commented

2026-05-07 00:17:12 +02:00

Author

Owner

@doubleailes commented on GitHub (Oct 31, 2020):

I just checked. pamtester is install into my bastion.

@doubleailes commented on GitHub (Oct 31, 2020): I just checked. `pamtester` is install into my bastion.

BreizhHardware commented

2026-05-07 00:17:13 +02:00

Author

Owner

@speed47 commented on GitHub (Oct 31, 2020):

There might be something wrong on the configuration of the google_authenticator PAM module on your centos machine. When there is something wrong in its config, it'll always deny authentication. Centos 8 is part of the OSes that are automatically tested on each release, including for MFA, so it should work if the configuration is ok and the proper packages are installed.

If there is the "debug" keyword on the google_authenticator pam module line in /etc/pam.d/sshd file, you should get some diagnostic information in the system logs, that should give a hint about the problem.

@speed47 commented on GitHub (Oct 31, 2020): There might be something wrong on the configuration of the google_authenticator PAM module on your centos machine. When there is something wrong in its config, it'll always deny authentication. Centos 8 is part of the OSes that are automatically tested on each release, including for MFA, so it should work if the configuration is ok and the proper packages are installed. If there is the "debug" keyword on the google_authenticator pam module line in `/etc/pam.d/sshd` file, you should get some diagnostic information in the system logs, that should give a hint about the problem.

BreizhHardware commented

2026-05-07 00:17:13 +02:00

Author

Owner

@doubleailes commented on GitHub (Oct 31, 2020):

There is no real equivalent of system logs in CentOs8 can you be more specific in which log it will be store.
the only line talking about google_authenticator is:
auth [success=ok new_authok_reqd=ok ignore=ignore default=bad module_unknow=ignore] pam_google_authenticator.so secret=~/.otp

I guess debug should be add here:
auth [success=ok new_authok_reqd=ok ignore=ignore default=bad module_unknow=ignore] pam_google_authenticator.so debug secret=~/.otp

@doubleailes commented on GitHub (Oct 31, 2020): There is no real equivalent of system logs in CentOs8 can you be more specific in which log it will be store. the only line talking about google_authenticator is: `auth [success=ok new_authok_reqd=ok ignore=ignore default=bad module_unknow=ignore] pam_google_authenticator.so secret=~/.otp` I guess debug should be add here: `auth [success=ok new_authok_reqd=ok ignore=ignore default=bad module_unknow=ignore] pam_google_authenticator.so debug secret=~/.otp`

BreizhHardware commented

2026-05-07 00:17:14 +02:00

Author

Owner

@speed47 commented on GitHub (Oct 31, 2020):

Just booted a brand new CentOS 8 VM to be sure, and I can indeed get OTP to work by following the documentation (just wanted to be sure).

You are right, the "debug" keyword is to be added to the line you specified.

When you've done this, your PAM will log using your system's syslog, under the "debug" level. Here's what I get with mine:

Oct 31 22:01:54 c1bfbf5957bc sshd[2239]: Accepted key ED25519 SHA256:r0e/w4HzCzPArzE3bbqu3fSI7Sko29ODO7yjNFt08QY found at /home/joe/.ssh/authorized_keys2:1
Oct 31 22:01:54 c1bfbf5957bc sshd[2239]: Partial publickey for joe from 127.0.0.1 port 46634 ssh2: ED25519 SHA256:r0e/w4HzCzPArzE3bbqu3fSI7Sko29ODO7yjNFt08QY
Oct 31 22:01:54 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: start of google_authenticator for "joe"
Oct 31 22:01:54 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: Secret file permissions are 0400. Allowed permissions are 0600
Oct 31 22:01:54 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: "/home/joe/.otp" read
Oct 31 22:01:54 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: shared secret in "/home/joe/.otp" processed
Oct 31 22:01:54 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: google_authenticator for host "127.0.0.1"
Oct 31 22:01:54 c1bfbf5957bc sshd[2239]: Postponed keyboard-interactive for joe from 127.0.0.1 port 46634 ssh2 [preauth]
Oct 31 22:02:08 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: no scratch code used from "/home/joe/.otp"
Oct 31 22:02:08 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: Accepted google_authenticator for joe
Oct 31 22:02:08 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: "/home/joe/.otp" written
Oct 31 22:02:08 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: end of google_authenticator for "joe". Result: Success
Oct 31 22:02:08 c1bfbf5957bc sshd[2239]: Postponed keyboard-interactive/pam for joe from 127.0.0.1 port 46634 ssh2 [preauth]
Oct 31 22:02:08 c1bfbf5957bc sshd[2239]: Accepted keyboard-interactive/pam for joe from 127.0.0.1 port 46634 ssh2

The location of the file will depend on your system's syslog configuration.
On my test system I've just put up, I installed syslog-ng, and added these 2 configuration lines to get all the logs into the same file:

destination d_all { file("/var/log/all.log"); };
log { source(s_sys); destination(d_all); };

(at the end of /etc/syslog-ng/syslog-ng.conf)

@speed47 commented on GitHub (Oct 31, 2020): Just booted a brand new CentOS 8 VM to be sure, and I can indeed get OTP to work by following the documentation (just wanted to be sure). You are right, the "debug" keyword is to be added to the line you specified. When you've done this, your PAM will log using your system's syslog, under the "debug" level. Here's what I get with mine: ``` Oct 31 22:01:54 c1bfbf5957bc sshd[2239]: Accepted key ED25519 SHA256:r0e/w4HzCzPArzE3bbqu3fSI7Sko29ODO7yjNFt08QY found at /home/joe/.ssh/authorized_keys2:1 Oct 31 22:01:54 c1bfbf5957bc sshd[2239]: Partial publickey for joe from 127.0.0.1 port 46634 ssh2: ED25519 SHA256:r0e/w4HzCzPArzE3bbqu3fSI7Sko29ODO7yjNFt08QY Oct 31 22:01:54 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: start of google_authenticator for "joe" Oct 31 22:01:54 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: Secret file permissions are 0400. Allowed permissions are 0600 Oct 31 22:01:54 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: "/home/joe/.otp" read Oct 31 22:01:54 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: shared secret in "/home/joe/.otp" processed Oct 31 22:01:54 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: google_authenticator for host "127.0.0.1" Oct 31 22:01:54 c1bfbf5957bc sshd[2239]: Postponed keyboard-interactive for joe from 127.0.0.1 port 46634 ssh2 [preauth] Oct 31 22:02:08 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: no scratch code used from "/home/joe/.otp" Oct 31 22:02:08 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: Accepted google_authenticator for joe Oct 31 22:02:08 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: "/home/joe/.otp" written Oct 31 22:02:08 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: end of google_authenticator for "joe". Result: Success Oct 31 22:02:08 c1bfbf5957bc sshd[2239]: Postponed keyboard-interactive/pam for joe from 127.0.0.1 port 46634 ssh2 [preauth] Oct 31 22:02:08 c1bfbf5957bc sshd[2239]: Accepted keyboard-interactive/pam for joe from 127.0.0.1 port 46634 ssh2 ``` The location of the file will depend on your system's syslog configuration. On my test system I've just put up, I installed syslog-ng, and added these 2 configuration lines to get all the logs into the same file: ``` destination d_all { file("/var/log/all.log"); }; log { source(s_sys); destination(d_all); }; ``` (at the end of /etc/syslog-ng/syslog-ng.conf)

BreizhHardware commented

2026-05-07 00:17:14 +02:00

Author

Owner

@doubleailes commented on GitHub (Oct 31, 2020):

ok it's a permission issue.

@doubleailes commented on GitHub (Oct 31, 2020): ok it's a permission issue. ![2020-11-01_00-20](https://user-images.githubusercontent.com/23233470/97791925-1094b880-1bd8-11eb-9ed9-c8e0c5d77246.png)

BreizhHardware commented

2026-05-07 00:17:14 +02:00

Author

Owner

@speed47 commented on GitHub (Nov 1, 2020):

Indeed, it seems your user can't write to its own home directory. I suppose you fixed it successfully?

@speed47 commented on GitHub (Nov 1, 2020): Indeed, it seems your user can't write to its own home directory. I suppose you fixed it successfully?

BreizhHardware commented

2026-05-07 00:17:15 +02:00

Author

Owner

@doubleailes commented on GitHub (Nov 1, 2020):

Nope. I change folder permission other than 700 the TOTP check is refuse.

@doubleailes commented on GitHub (Nov 1, 2020): Nope. I change folder permission other than **700** the TOTP check is refuse.

BreizhHardware commented

2026-05-07 00:17:15 +02:00

Author

Owner

@doubleailes commented on GitHub (Nov 1, 2020):

How can i be sure the pam process is running under my user?

@doubleailes commented on GitHub (Nov 1, 2020): How can i be sure the pam process is running under my user?

BreizhHardware commented

2026-05-07 00:17:15 +02:00

Author

Owner

@speed47 commented on GitHub (Nov 1, 2020):

How can i be sure the pam process is running under my user?

It's running under your user when pamtester is called, which is what the bastion does:

github.com/ovh/the-bastion@b0eaf15bdb/bin/shell/osh.pl (L878)

You can try it manually this way, running it directly on your centos machine under your user:

su - yourusername -s /bin/bash
pamtester sshd yourusername authenticate

@speed47 commented on GitHub (Nov 1, 2020): > > > How can i be sure the pam process is running under my user? It's running under your user when `pamtester` is called, which is what the bastion does: https://github.com/ovh/the-bastion/blob/b0eaf15bdb617f5f6cb0f2859eb5b8952f85287f/bin/shell/osh.pl#L878 You can try it manually this way, running it directly on your centos machine under your user: ``` su - yourusername -s /bin/bash pamtester sshd yourusername authenticate ```

BreizhHardware commented

2026-05-07 00:17:16 +02:00

Author

Owner

@doubleailes commented on GitHub (Nov 1, 2020):

You can try it manually this way, running it directly on your centos machine under your user:

su - yourusername -s /bin/bash
pamtester sshd yourusername authenticate

It worked.
So i checked my home user folder is 700 and my .opt is 600

@doubleailes commented on GitHub (Nov 1, 2020): > You can try it manually this way, running it directly on your centos machine under your user: >su - yourusername -s /bin/bash >pamtester sshd yourusername authenticate It worked. So i checked my home user folder is 700 and my .opt is 600

BreizhHardware commented

2026-05-07 00:17:16 +02:00

Author

Owner

@doubleailes commented on GitHub (Nov 1, 2020):

Did change selinux on your centos8 VM?

@doubleailes commented on GitHub (Nov 1, 2020): Did change selinux on your centos8 VM?

BreizhHardware commented

2026-05-07 00:17:17 +02:00

Author

Owner

@doubleailes commented on GitHub (Nov 1, 2020):

I changed selinux to permissive and it worked.

@doubleailes commented on GitHub (Nov 1, 2020): I changed selinux to permissive and it worked.

BreizhHardware commented

2026-05-07 00:17:17 +02:00

Author

Owner

@speed47 commented on GitHub (Nov 1, 2020):

Ah! Interesting.

This isn't catched during the tests because we use Docker to test on several distro flavors, so of course SELinux doesn't apply there.

We never stumbled upon that because we use Debian in production. Falling back to permissive is not acceptable for a security asset, so I think we'll have to take some time to write a proper SELinux policy.

@speed47 commented on GitHub (Nov 1, 2020): Ah! Interesting. This isn't catched during the tests because we use Docker to test on several distro flavors, so of course SELinux doesn't apply there. We never stumbled upon that because we use Debian in production. Falling back to `permissive` is not acceptable for a security asset, so I think we'll have to take some time to write a proper SELinux policy.

BreizhHardware commented

2026-05-07 00:17:18 +02:00

Author

Owner

@doubleailes commented on GitHub (Nov 1, 2020):

Thank for all your help.
It's really a great tool.

I'll be happy to put it in production when the policy will be written.

@doubleailes commented on GitHub (Nov 1, 2020): Thank for all your help. **It's really a great tool.** I'll be happy to put it in production when the policy will be written.

BreizhHardware commented

2026-05-07 00:17:19 +02:00

Author

Owner

@speed47 commented on GitHub (Nov 6, 2020):

pam, called by sshd, needs to be able to write the otp files. The following policy seems to be sufficient with our tests:

cat >the-bastion.te <<EOF
module the-bastion 1.0;

require {
	type var_t;
	type sshd_t;
	type user_home_t;
	type user_home_dir_t;
	class file { create getattr rename setattr unlink open read write };
}

# needed for user TOTP (~/.totp and ~/.totp~XXXXXX temporary file)
allow sshd_t user_home_dir_t:file { create getattr rename setattr unlink open read write };
allow sshd_t user_home_t:file     unlink;
# needed for root TOTP (/var/otp/root and /var/otp/root~XXXXXX temporary file)
allow sshd_t var_t:file           { create getattr rename setattr unlink open read write };
EOF

checkmodule -M -m -o the-bastion.mod the-bastion.te
semodule_package -o the-bastion.pp -m the-bastion.mod
semodule -i the-bastion.pp

@speed47 commented on GitHub (Nov 6, 2020): `pam`, called by `sshd`, needs to be able to write the otp files. The following policy seems to be sufficient with our tests: ``` cat >the-bastion.te <<EOF module the-bastion 1.0; require { type var_t; type sshd_t; type user_home_t; type user_home_dir_t; class file { create getattr rename setattr unlink open read write }; } # needed for user TOTP (~/.totp and ~/.totp~XXXXXX temporary file) allow sshd_t user_home_dir_t:file { create getattr rename setattr unlink open read write }; allow sshd_t user_home_t:file unlink; # needed for root TOTP (/var/otp/root and /var/otp/root~XXXXXX temporary file) allow sshd_t var_t:file { create getattr rename setattr unlink open read write }; EOF checkmodule -M -m -o the-bastion.mod the-bastion.te semodule_package -o the-bastion.pp -m the-bastion.mod semodule -i the-bastion.pp ```

BreizhHardware referenced this issue

2026-05-07 00:19:38 +02:00

[PR #9] [MERGED] clush: change description for --no-pause-on-failure to represent the actual behavior #166

Rows
Columns

[GH-ISSUE #26] [SELinux] TOTP don't seems to work #9