Repository: privacyidea/privacyidea

Issue:
Policy authentication bug in 3.9
After upgrading from 3.8.1 to 3.9 the `policy -> authentication -> passthru` **is not working**

```
# echo "User-Name = xxx, User-Password =xxx" | radclient -sx localhost:1812 auth xxx
Sent Access-Request Id 185 from 0.0.0.0:42733 to 127.0.0.1:1812 length 46
        User-Name = "xxx"
        User-Password = "xxx"
        Cleartext-Password = "xxx"
Received Access-Reject Id 185 from 127.0.0.1:1812 to 127.0.0.1:42733 length 53
        Reply-Message = "The user has no tokens assigned"
(0) -: Expected Access-Accept got Access-Reject
Packet summary:
        Accepted      : 0
        Rejected      : 1
        Lost          : 0
        Passed filter : 0
        Failed filter : 1
```

After rollback (VM snapshot)
```
# echo "User-Name = xxx, User-Password = xxxx" | radclient -sx localhost:1812 auth xxx
Sent Access-Request Id 225 from 0.0.0.0:54408 to 127.0.0.1:1812 length 48
        User-Name = "xxx"
        User-Password = "xxx"
        Cleartext-Password = "xxx"
Received Access-Accept Id 225 from 127.0.0.1:1812 to 127.0.0.1:54408 length 48
        Reply-Message = "privacyIDEA access granted"
Packet summary:
        Accepted      : 1
        Rejected      : 0
        Lost          : 0
        Passed filter : 1
        Failed filter : 0
```


Discussion:
Thank you for filing an issue and sharing your observations or ideas. Please be sure to provide as much information as possible to help us to work on this issue.
> Thank you for filing an issue and sharing your observations or ideas. Please be sure to provide as much information as possible to help us to work on this issue.

When I don't want to assign a token, validation must take place via password (in local realm):

_Reply-Message = "privacyIDEA access granted"_

with the upgrade this no longer works (even if passthru is enabled):

_Reply-Message = "The user has no tokens assigned"_
Hi @ctosae 
could You try to ask the API directly?
`curl -d "user=<user>" -d "pass=<password>" http://localhost:1812/validate/check`
You should get something like:
```
  "detail": {
    "message": "against userstore due to 'passthru'",
    "threadid": 140399037761344
  },
  "id": 2,
  "jsonrpc": "2.0",
  "result": {
    "authentication": "ACCEPT",
    "status": true,
    "value": true
  },
```
@plettich thanks for asking

1812 is the Radius port (UDP) usable with radclient 

Here the web commands (pre-upgrare, upgrade, post-upgrade):

```
# curl -k -d "user=xxx" -d "pass=xxx" https://localhost/validate/check`

"detail":{
"message":"against userstore due to 'xxx_authentication'",
"threadid":139938323842624
},
"id":2,
"jsonrpc":"2.0",
"result":{
"authentication":"ACCEPT",
"status":true,
"value":true
},
"time":1698402028.4214945,
"version":"privacyIDEA 3.8.1",
...
```

```
# apt update; apt install privacyidea privacyidea-apache2 -y

[..]
Considering dependency socache_shmcb for ssl:
Module socache_shmcb already enabled
Module ssl already enabled
Module rewrite already enabled

             _                    _______  _______
   ___  ____(_)  _____ _______ __/  _/ _ \/ __/ _ |
  / _ \/ __/ / |/ / _ `/ __/ // // // // / _// __ |
 / .__/_/ /_/|___/\_,_/\__/\_, /___/____/___/_/ |_|
/_/                       /___/
                                               v3.9


The file
        /etc/privacyidea/enckey
already exist. We do not overwrite it!

             _                    _______  _______
   ___  ____(_)  _____ _______ __/  _/ _ \/ __/ _ |
  / _ \/ __/ / |/ / _ `/ __/ // // // // / _// __ |
 / .__/_/ /_/|___/\_,_/\__/\_, /___/____/___/_/ |_|
/_/                       /___/
                                               v3.9

<SQLAlchemy engine=mysql+pymysql://pi:***@localhost/pi?charset=utf8>

             _                    _______  _______
[..]
```

```
# curl -k -d "user=xxx" -d "pass=xxx" https://localhost/validate/check

{
"detail":{
"message":"The user has no tokens assigned",
"threadid":140537173894720
},
"id":2,
"jsonrpc":"2.0",
"result":{
"authentication":"REJECT",
"status":true,
"value":false
},
"time":1698402130.8997238,
"version":"privacyIDEA 3.9",
"versionnumber":"3.9",
...```
Note. you can also see versions pre and post upgrade
Please check your policies.
Also check in the audit log, which policy actually matches in the /validate/check request.

You are looking at the last link in the chain. 
I suspect, that you have a somehow more complex policy setting, which results in the observed behaviour, which would not be hat passthru does not work anymore, but that your policy does not **match** anymore.
Hey @cornelinux how are you?
I've been using privacyIDEA since the first versions.
The problem only appeared with this update.
The same policy is always matched:
![image](https://github.com/privacyidea/privacyidea/assets/75297085/711a6d77-fc77-4870-8176-e04ff1404bf2)

And here the policy. There is only one "auth" with the realm:

![image](https://github.com/privacyidea/privacyidea/assets/75297085/c0f6165b-a74b-4d9e-a1ec-c917eb33d77c)

with action:

`otppin=tokenpin, passthru=userstore, reset_all_user_tokens`
@plettich Please check if the otppin=tokenpin creates an unwanted side effect
@cornelinux The `otppin=tokenpin` does not affect the `passthru` policy. It is checked after the `passthru` policy is handled.

@ctosae Could You check in the privacyIDEA logs if the user actually successfully authenticated against the userstore?

@cornelinux @plettich i found the issue!
After this migration, privacyIDEA is no longer validating local users with passwords with :   _ ! $
Changing it with the same one (Users->Realm->User->Change user->Password) starts working again
Something in the skip format has probably been changed (like `\$` etc..)

@ctosae which user resolver do You use? Database (which kind of db)?
There shouldn't have been any changes in how the passwords are formatted except for the `%` sign since we use percent-encoded (url-encoded) parameters.
I'm using the standard mysql (apt install).
```
# mysql -V
mysql  Ver 8.0.32-0ubuntu0.22.04.2 for Linux on x86_64 ((Ubuntu))
```

```
# cat /etc/privacyidea/pi.cfg | grep sql
#SQLALCHEMY_DATABASE_URI = 'sqlite:////etc/privacyidea/data.sqlite'
#PI_AUDIT_MODULE = 'privacyidea.lib.auditmodules.sqlaudit'
SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://pi:xxxxx@localhost/pi'
```
@ctosae And which user resolver? Where are the users (and passwords) stored? Just to rule out that we somehow broke the password checking.
I'm using the internal "Users" section (creating and management) since the first privacyIDEA versions.

Try to import this in a 3.8.1 version (my current version before the upgrade)
```
SET SQL_MODE = "NO_AUTO_VALUE_ON_ZERO";
START TRANSACTION;
SET time_zone = "+00:00";

CREATE TABLE `accounts_vpn` (
  `userid` bigint NOT NULL,
  `username` varchar(60) NOT NULL,
  `phone` varchar(60) DEFAULT NULL,
  `mobile` varchar(60) DEFAULT NULL,
  `email` varchar(60) DEFAULT NULL,
  `givenname` varchar(60) DEFAULT NULL,
  `surname` varchar(60) DEFAULT NULL,
  `password` varchar(100) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

INSERT INTO `accounts_vpn` (`userid`, `username`, `phone`, `mobile`, `email`, `givenname`, `surname`, `password`) VALUES
(50, 'test', NULL, NULL, 'test@test', NULL, NULL, '{SSHA256}Zl8HdEif0dJF4JZXrdSM2yd0zStpL69qIpTxTEqj873iky7qsOhCGh2cJxluHg0BJlgmRFLnP2stA3VWAQdZKQ==');

ALTER TABLE `accounts_vpn`
  ADD PRIMARY KEY (`userid`),
  ADD UNIQUE KEY `username` (`username`);

ALTER TABLE `accounts_vpn`
  MODIFY `userid` bigint NOT NULL AUTO_INCREMENT, AUTO_INCREMENT=56;
COMMIT;
```

Then query with `https://IP/validate/check?user=test&pass=SRB_$_2023!`

Now upgrade to 3.9 and check again.

Thanks

Resolving patch:
diff --git a/privacyidea/lib/resolvers/SQLIdResolver.py b/privacyidea/lib/resolvers/SQLIdResolver.py
--- a/privacyidea/lib/resolvers/SQLIdResolver.py
+++ b/privacyidea/lib/resolvers/SQLIdResolver.py
@@ -55,6 +55,7 @@
 import passlib.utils.handlers as uh
 import passlib.exc as exc
 from passlib.registry import register_crypt_handler
+from passlib.handlers.ldap_digests import _SaltedBase64DigestHelper
 
 
 class phpass_drupal(uh.HasRounds, uh.HasSalt, uh.GenericHandler):  # pragma: no cover
@@ -108,13 +109,29 @@ def _calc_checksum(self, secret):
         return h64.encode_bytes(result).decode("ascii")[:self.checksum_size]
 
 
+class ldap_salted_sha256_pi(_SaltedBase64DigestHelper):
+    name = 'ldap_salted_sha256_pi'
+    ident = '{SSHA256}'
+    checksum_size = 32
+    # passlib sets the max_salt_size for SSHA256 to 16:
+    # <https://foss.heptapod.net/python-libs/passlib/-/blob/branch/stable/passlib/handlers/ldap_digests.py#L71>
+    # But we have encountered hashes with longer salt sizes. Since we set this
+    # password hashing function when we create an internal db resolver,
+    # we should be able to verify these hashes as well.
+    max_salt_size = 32
+    _hash_func = hashlib.sha256
+    _hash_regex = re.compile(r"^\{SSHA256\}(?P<tmp>[+/a-zA-Z0-9]{48,}={0,2})$")
+
+
 register_crypt_handler(phpass_drupal)
+register_crypt_handler(ldap_salted_sha256_pi)
+
 
 # The list of supported password hash types for verification (passlib handler)
 pw_ctx = CryptContext(schemes=['phpass',
                                'phpass_drupal',
                                'ldap_salted_sha1',
-                               'ldap_salted_sha256',
+                               'ldap_salted_sha256_pi',
                                'ldap_salted_sha512',
                                'ldap_sha1',
                                'md5_crypt',
@@ -128,7 +145,7 @@ def _calc_checksum(self, secret):
 hash_type_dict = {"PHPASS": 'phpass',
                   "SHA": 'ldap_sha1',
                   "SSHA": 'ldap_salted_sha1',
-                  "SSHA256": 'ldap_salted_sha256',
+                  "SSHA256": 'ldap_salted_sha256_pi',
                   "SSHA512": 'ldap_salted_sha512',
                   "OTRS": 'hex_sha256',
                   "SHA256CRYPT": 'sha256_crypt',
@@ -279,7 +296,7 @@ def getUserInfo(self, userId):
             log.error("Could not get the user information: {0!r}".format(exx))
 
         return userinfo
-    
+
     def _get_userid_filter(self, userId):
         column = self.TABLE.columns[self.map.get("userid")]
         if isinstance(column.type, String):


Test patch: