internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: privacyidea/privacyidea

Issue:
Node from policy can not be removed
A policy has a condition with a node.
If I remove all nodes (select none) and save the policy the node is still contained in the policy.

Reopen the policy and check: The node is still there.


Discussion:

Resolving patch:
diff --git a/doc/conf.py b/doc/conf.py
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -23,7 +23,7 @@
 # built documents.
 #
 # The short X.Y version.
-version = '3.8'
+version = '3.8.1'
 # The full version, including alpha/beta/rc tags.
 #release = '2.16dev5'
 release = version
diff --git a/privacyidea/api/policy.py b/privacyidea/api/policy.py
--- a/privacyidea/api/policy.py
+++ b/privacyidea/api/policy.py
@@ -120,6 +120,7 @@ def set_policy_api(name=None):
     :jsonparam user: The policy is valid for these users.
         string with wild cards or list of strings
     :jsonparam time: on which time does this policy hold
+    :jsonparam pinode: The privacyIDEA node (or list of nodes) for which this policy is valid
     :jsonparam client: for which requesting client this should be
     :jsontype client: IP address with subnet
     :jsonparam active: bool, whether this policy is active or not
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -6,7 +6,7 @@
 import sys
 
 #VERSION = "2.1dev4"
-VERSION = "3.8.1dev1"
+VERSION = "3.8.1"
 
 # Taken from kennethreitz/requests/setup.py
 package_directory = os.path.realpath(os.path.dirname(__file__))


Test patch:
Repository: privacyidea/privacyidea

Issue:
multiselect policies in webUI
Since we now have a multiselect in policies, we could adapt some policies like

* challenge_response
* ...

That already take a list of e.g. tokentypes.
Since the multiselect and the existing policy both save this as a whitespace separated list, this should simply be an adaption of the policy definition here:
https://github.com/privacyidea/privacyidea/blob/d166281ce3fc226242952be9384814d78fa0d3c2/privacyidea/lib/policy.py#L2237

To generate the list of tokenclasses for the multiselect we could use a check of the class attribute "mode" to see if this can be a challene response token.


Discussion:
Actually we nee need to support policy actions written with capital letters to avoid breaking old installations.
The idea would be to change the entries in `allowed_tokentypes_dict` to lower case [here](https://github.com/privacyidea/privacyidea/blob/d6a63494b2894290adff39a4aa6f82df7fe7be43/privacyidea/lib/policydecorators.py#L127).
Also the loop [here](https://github.com/privacyidea/privacyidea/blob/d6a63494b2894290adff39a4aa6f82df7fe7be43/privacyidea/lib/policydecorators.py#L126) is not necessary.

Resolving patch:
diff --git a/privacyidea/lib/policydecorators.py b/privacyidea/lib/policydecorators.py
--- a/privacyidea/lib/policydecorators.py
+++ b/privacyidea/lib/policydecorators.py
@@ -121,16 +121,16 @@ def challenge_response_wrapper(*args, **kwds):
                                                  action=ACTION.CHALLENGERESPONSE, user_object=user_object)\
                 .action_values(unique=False, write_to_audit_log=False)
             log.debug("Found these allowed tokentypes: {0!s}".format(list(allowed_tokentypes_dict)))
-            # allowed_tokentypes_dict.keys() is a list of actions from several policies.
+            allowed_tokentypes_dict = {k.lower(): v for k, v in allowed_tokentypes_dict.items()}
+            token = token.get_tokentype().lower()
             chal_resp_found = False
-            for toks in allowed_tokentypes_dict:
-                if token.get_tokentype() in allowed_tokentypes_dict:
-                    # This token is allowed to to chal resp
-                    chal_resp_found = True
-                    g.audit_object.add_policy(allowed_tokentypes_dict.get(toks))
+            if token in allowed_tokentypes_dict:
+                # This token is allowed to do challenge-response
+                chal_resp_found = True
+                g.audit_object.add_policy(allowed_tokentypes_dict.get(token))
 
             if not chal_resp_found:
-                # No policy to allow this token to do challenge response
+                # No policy to allow this token to do challenge-response
                 return False
 
         f_result = func(*args, **kwds)


Test patch:
Repository: privacyidea/privacyidea

Issue:
Typo in policy definition disables revoke-permission
Due to a typo here:
https://github.com/privacyidea/privacyidea/blob/ac9135f1c48449a8121afa30ea67b41dea215dea/privacyidea/lib/policy.py#L1612
the UI was never able to correctly set the `revoke` permission (which was never noticed since no one really used that permission).
We could use Python [`dataclasses`](https://docs.python.org/3.8/library/dataclasses.html) to avoid issues like this in the future.


Discussion:

Resolving patch:
diff --git a/privacyidea/lib/policy.py b/privacyidea/lib/policy.py
--- a/privacyidea/lib/policy.py
+++ b/privacyidea/lib/policy.py
@@ -1609,7 +1609,7 @@ def get_static_policy_definitions(scope=None):
                                'a token.'),
                            'mainmenu': [MAIN_MENU.TOKENS],
                            'group': GROUP.TOKEN},
-            ACTION.REVOKE: {'tpye': 'bool',
+            ACTION.REVOKE: {'type': 'bool',
                             'desc': _("Admin is allowed to revoke a token"),
                             'mainmenu': [MAIN_MENU.TOKENS],
                             'group': GROUP.TOKEN},


Test patch:
Repository: privacyidea/privacyidea

Issue:
Unclear Policy/API description
The documentation of the [`/token/description`](https://privacyidea.readthedocs.io/en/latest/modules/api/token.html#post--token-description-(serial)) endpoint describes, that it can be used by a user or admin. This is only true for the admin if no admin policies are set.
There is only a user policy for [`setdescription`](https://privacyidea.readthedocs.io/en/latest/policies/user.html#setdescription). The admin usually uses the [`/token/set`](https://privacyidea.readthedocs.io/en/latest/modules/api/token.html#post--token-set-(serial)) endpoint.
We should either add a `setdescription` admin policy or update the documentation of the endpoint.


Discussion:
The historical background is, that there was a request, that a user should only be able to set a description.
The user is not allowed to use the endpoint /token/set anyways.

It might be good, that "admins" like helpdesk users, could also be policified this way, that the would only be allowed to set the description.

* Add an admin policy "setdescription".
* An admin then is allowed to use the endpoint /token/description if he had the "setdescription" policy set.

Resolving patch:
diff --git a/privacyidea/lib/policy.py b/privacyidea/lib/policy.py
--- a/privacyidea/lib/policy.py
+++ b/privacyidea/lib/policy.py
@@ -1580,6 +1580,10 @@ def get_static_policy_definitions(scope=None):
                              'Admin is allowed to set token properties.'),
                          'mainmenu': [MAIN_MENU.TOKENS],
                          'group': GROUP.TOKEN},
+            ACTION.SETDESCRIPTION: {'type': 'bool',
+                                    'desc': _('The admin is allowed to set the token description.'),
+                                    'mainmenu': [MAIN_MENU.TOKENS],
+                                    'group': GROUP.TOKEN},
             ACTION.SETPIN: {'type': 'bool',
                             'desc': _(
                                 'Admin is allowed to set the OTP PIN of '


Test patch:
Repository: privacyidea/privacyidea

Issue:
url open for permitted schemes
Check if we can provide a specific scheme here. This should only be a `data:` scheme.

Tracking issue for:
- [x] https://github.com/privacyidea/privacyidea/security/code-scanning/27



Discussion:

Resolving patch:
diff --git a/privacyidea/lib/eventhandler/usernotification.py b/privacyidea/lib/eventhandler/usernotification.py
--- a/privacyidea/lib/eventhandler/usernotification.py
+++ b/privacyidea/lib/eventhandler/usernotification.py
@@ -422,7 +422,7 @@ def do(self, action, options=None):
 
                 if attach_qrcode and googleurl_img:
                     # get the image part of the googleurl
-                    googleurl = urlopen(googleurl_img)
+                    googleurl = urlopen(googleurl_img)  #  nosec B310   # no user input
                     mail_body = MIMEMultipart('related')
                     mail_body.attach(MIMEText(body, mimetype))
                     mail_img = MIMEImage(googleurl.read())


Test patch:
Repository: privacyidea/privacyidea

Issue:
Add event handler to poll transaction
We need to be able to connect event handlers to `poll transcation`. This way we can act, in case polling a transaction from a PUSH token returns a declined push authentication.

The event handler then could e.g. notify admins, that a PUSH was declined, which might indicate that credentials are compromized.


Discussion:

Resolving patch:
diff --git a/privacyidea/api/validate.py b/privacyidea/api/validate.py
--- a/privacyidea/api/validate.py
+++ b/privacyidea/api/validate.py
@@ -601,6 +601,7 @@ def trigger_challenge():
 @prepolicy(mangle, request=request)
 @CheckSubscription(request)
 @prepolicy(api_key_required, request=request)
+@event("validate_poll_transaction", request, g)
 def poll_transaction(transaction_id=None):
     """
     Given a mandatory transaction ID, check if any non-expired challenge for this transaction ID


Test patch:
Repository: privacyidea/privacyidea

Issue:
Improve System Config documentation
There are some misleading/wrong system settings described in the [System Config Docs](https://privacyidea.readthedocs.io/en/latest/configuration/system_config.html):
- `FailCounterIncOnFalsePin` was renamed to `IncFailCountOnFalsePin` (or the variable could be dropped completely)
- `ResetFailcounterOnPIN` can not be set through the WebUI. It defaults to `False`
- `SerialLength` can not be set through the WebUI, only via API (defaults to 8)
- `DefaultResetFailCount` can not be set through the WebUI and is only used in tests
- `ReturnSamlAttributesOnFail` not documented
- `recovery.identifier` for email recovery not documented


Discussion:

Resolving patch:
diff --git a/privacyidea/lib/config.py b/privacyidea/lib/config.py
--- a/privacyidea/lib/config.py
+++ b/privacyidea/lib/config.py
@@ -282,6 +282,7 @@ class SYSCONF(object):
     SPLITATSIGN = "splitAtSign"
     INCFAILCOUNTER = "IncFailCountOnFalsePin"
     RETURNSAML = "ReturnSamlAttributes"
+    RETURNSAMLONFAIL = "ReturnSamlAttributesOnFail"
     RESET_FAILCOUNTER_ON_PIN_ONLY = "ResetFailcounterOnPIN"
 
 
@@ -608,7 +609,7 @@ def get_resolver_class_dict():
     """
     get a dictionary of the resolver classes and a dictionary
     of the resolver types:
-    
+
     ({'privacyidea.lib.resolvers.PasswdIdResolver.IdResolver':
       <class 'privacyidea.lib.resolvers.PasswdIdResolver.IdResolver'>,
       'privacyidea.lib.resolvers.PasswdIdResolver.UserIdResolver':
@@ -971,13 +972,13 @@ def set_prepend_pin(prepend=True):
 
 
 def return_saml_attributes():
-    r = get_from_config(key="ReturnSamlAttributes", default= False,
+    r = get_from_config(key=SYSCONF.RETURNSAML, default=False,
                         return_bool=True)
     return r
 
 
 def return_saml_attributes_on_fail():
-    r = get_from_config(key="ReturnSamlAttributesOnFail", default=False,
+    r = get_from_config(key=SYSCONF.RETURNSAMLONFAIL, default=False,
                         return_bool=True)
     return r
 


Test patch:
Repository: privacyidea/privacyidea

Issue:
4 chars ID of day password token.
The ID of the day password token should only be 4 chars. Like "DYPW" oder "PIDP".

Currently it is "DayPassword", which leads to a very long serial number.

![image](https://github.com/privacyidea/privacyidea/assets/1908620/bfbd9a72-fad4-4563-a77c-9dc1b40e1b74)



Discussion:

Resolving patch:
diff --git a/privacyidea/lib/tokens/daypasswordtoken.py b/privacyidea/lib/tokens/daypasswordtoken.py
--- a/privacyidea/lib/tokens/daypasswordtoken.py
+++ b/privacyidea/lib/tokens/daypasswordtoken.py
@@ -64,7 +64,7 @@ def get_class_prefix():
 
         :return: DayPasswordToken
         """
-        return "DayPassword"
+        return "DYPW"
 
     @staticmethod
     @log_with(log)


Test patch:
diff --git a/tests/test_lib_tokens_daypassword.py b/tests/test_lib_tokens_daypassword.py
--- a/tests/test_lib_tokens_daypassword.py
+++ b/tests/test_lib_tokens_daypassword.py
@@ -75,7 +75,7 @@ def test_01_create_token(self):
         self.assertTrue(token.token.tokentype == "daypassword", token.token.tokentype)
         self.assertTrue(token.type == "daypassword", token)
         class_prefix = token.get_class_prefix()
-        self.assertTrue(class_prefix == "DayPassword", class_prefix)
+        self.assertTrue(class_prefix == "DYPW", class_prefix)
         self.assertTrue(token.get_class_type() == "daypassword", token)
 
     def test_02_set_user(self):
Repository: privacyidea/privacyidea

Issue:
Better label for day password policy
The label in the policy for the `daypassword_timestep` should be more helpful.

Please replace "text..." with something like "24h".

![image](https://github.com/privacyidea/privacyidea/assets/1908620/2895a46c-3c70-4131-b3bc-eceaa4efbffc)



Discussion:

Resolving patch:
diff --git a/privacyidea/lib/tokens/daypasswordtoken.py b/privacyidea/lib/tokens/daypasswordtoken.py
--- a/privacyidea/lib/tokens/daypasswordtoken.py
+++ b/privacyidea/lib/tokens/daypasswordtoken.py
@@ -33,7 +33,7 @@
 class DayPasswordTokenClass(TotpTokenClass):
     previous_otp_offset = 0
 
-    desc_timestep = _('Specify the time step of the DayPassword token.')
+    desc_timestep = _('Specify the time step of the DayPassword token. For example: "24h"')
 
     @log_with(log)
     def __init__(self, db_token):


Test patch:
Repository: privacyidea/privacyidea

Issue:
Update documentation for v3.9



Discussion:
@plettich  what do we still need to update?

Resolving patch:
diff --git a/doc/conf.py b/doc/conf.py
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -76,7 +76,7 @@ def wrapper(decorator):
 
 # General information about the project.
 project = u'privacyIDEA'
-copyright = u'2014-2022, Cornelius Kölbel'
+copyright = u'2014-2023, Cornelius Kölbel'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.
@@ -146,7 +146,7 @@ def wrapper(decorator):
 # The name of an image file (within the static path) to use as favicon of the
 # docs.  This file should be a Windows icon file (.ico) being 16x16 or 32x32
 # pixels large.
-#html_favicon = None
+html_favicon = "images/privacyidea-favicon_book.png"
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
diff --git a/privacyidea/webui/login.py b/privacyidea/webui/login.py
--- a/privacyidea/webui/login.py
+++ b/privacyidea/webui/login.py
@@ -45,7 +45,8 @@
 from privacyidea.lib.queue import has_job_queue
 
 DEFAULT_THEME = "/static/contrib/css/bootstrap-theme.css"
-DEFAULT_LANGUAGE_LIST = ['en', 'de', 'nl', 'zh_Hant', 'fr', 'es', 'tr', 'cs', 'it']
+# note: the comment in the following line allows to include it in the docs
+DEFAULT_LANGUAGE_LIST = ['en', 'de', 'nl', 'zh_Hant', 'fr', 'es', 'tr', 'cs', 'it']  #:
 
 login_blueprint = Blueprint('login_blueprint', __name__)
 


Test patch:
Repository: privacyidea/privacyidea

Issue:
LDAP resolver: missing config description multivalueattribute
When saving the LDAP resolver the config values are checked.

The log file contains a warning about the missing config value MULTIVALUEATTRIBUTE; that originates from here:

https://github.com/privacyidea/privacyidea/blob/0d32822cf7e90710960c4f1811b9e57fd50a367c/privacyidea/lib/utils/__init__.py#L424

The reason is, that the "MULTIVALUEATTRIBUTE" is not contained in the LDAP Resolver class descriptor.
https://github.com/privacyidea/privacyidea/blob/d8abb69b8df65dc31501cab9def4428ec5576dc5/privacyidea/lib/resolvers/LDAPIdResolver.py#L1006




Discussion:

Resolving patch:
diff --git a/privacyidea/lib/resolvers/LDAPIdResolver.py b/privacyidea/lib/resolvers/LDAPIdResolver.py
--- a/privacyidea/lib/resolvers/LDAPIdResolver.py
+++ b/privacyidea/lib/resolvers/LDAPIdResolver.py
@@ -1042,7 +1042,8 @@ def getResolverClassDescriptor(cls):
                                 'SERVERPOOL_SKIP': 'int',
                                 'SERVERPOOL_PERSISTENT': 'bool',
                                 'OBJECT_CLASSES': 'string',
-                                'DN_TEMPLATE': 'string'}
+                                'DN_TEMPLATE': 'string',
+                                'MULTIVALUEATTRIBUTES': 'string'}
         return {typ: descriptor}
 
     @classmethod


Test patch:
Repository: privacyidea/privacyidea

Issue:
Label policies for PUSH token enroll via validate/check
When enrolling via validate/check the token_label etc policies are not applied to the PUSH token.

It works for HOTP and TOTP, since these policies are added here:

https://github.com/privacyidea/privacyidea/blob/662774047d37fdfe335fcf523892eff11ca5f881/privacyidea/lib/tokens/hotptoken.py#L839

Thus, we need to add `get_init_tokenlabel_parameters`  also to the `enroll_via_validate` method of the Push-Token:
https://github.com/privacyidea/privacyidea/blob/662774047d37fdfe335fcf523892eff11ca5f881/privacyidea/lib/tokens/pushtoken.py#L1056


Discussion:

Resolving patch:
diff --git a/privacyidea/lib/tokens/pushtoken.py b/privacyidea/lib/tokens/pushtoken.py
--- a/privacyidea/lib/tokens/pushtoken.py
+++ b/privacyidea/lib/tokens/pushtoken.py
@@ -34,7 +34,7 @@
 import traceback
 
 from privacyidea.api.lib.utils import getParam
-from privacyidea.api.lib.policyhelper import get_pushtoken_add_config
+from privacyidea.api.lib.policyhelper import get_pushtoken_add_config, get_init_tokenlabel_parameters
 from privacyidea.lib.token import get_one_token, init_token
 from privacyidea.lib.utils import prepare_result, to_bytes, is_true, create_tag_dict
 from privacyidea.lib.error import (ResourceNotFoundError, ValidateError,
@@ -1068,7 +1068,11 @@ def enroll_via_validate(cls, g, content, user_obj):
         # Create a challenge!
         c = token_obj.create_challenge(options={"session": CHALLENGE_SESSION.ENROLLMENT})
         # get details of token
-        init_details = token_obj.get_init_detail(params=params)
+        enroll_params = get_init_tokenlabel_parameters(g, user_object=user_obj,
+                                                       token_type=cls.get_class_type())
+        params.update(enroll_params)
+        init_details = token_obj.get_init_detail(params=params,
+                                                 user=user_obj)
         detail["transaction_ids"] = [c[2]]
         chal = {"transaction_id": c[2],
                 "image": init_details.get("pushurl", {}).get("img"),


Test patch:
diff --git a/tests/test_api_push_validate.py b/tests/test_api_push_validate.py
--- a/tests/test_api_push_validate.py
+++ b/tests/test_api_push_validate.py
@@ -380,6 +380,8 @@ def test_10_enroll_push(self):
         # 1. set policies.
         set_policy("pol_passthru", scope=SCOPE.AUTH, action=ACTION.PASSTHRU)
 
+        set_policy("pol_tokenlabel", scope=SCOPE.ENROLL, action="{0!s}=Pushy".format(ACTION.TOKENLABEL))
+
         # 2. authenticate user via passthru
         with self.app.test_request_context('/validate/check',
                                            method='POST',
Repository: plone/Products.CMFPlone

Issue:
Old-style profile with JS/CSS imported but not reliably used within rendered pages after a while
Situation: Plone 5.0a3 buildout with 

git@bitbucket.org:onkopedia/zopyx.existdb.git 

('dexterity' branch).

I created a new Plone site and added 'zopyx.existdb' as add-on.

As expected the JS/CSS resources show up in the resource registry control panels.

For debugging purposes I added an alert() to the local.js file of the package.

After the some operations in the control panel and some content management work in Plone UI the 
zopyx.existdb resources are no longer loaded - they do not longer show up in the HTML markup and of course the alert() is no longer trigger for each page load.



Discussion:
The resources are properly configured

![plone site](https://cloud.githubusercontent.com/assets/594239/5316900/56bb3dbc-7c91-11e4-82bb-e2a170073fec.png)

The HTML of the rendered front-page is here and does not contain the zopyx.existdb resources!?

https://bitbucket.org/onkopedia/zopyx.existdb/raw/8b701639460f3c5be2b215137dda35be1b4ba564/welcome.html

look at the ++resource++plonetheme.barceloneta-legacy.js bundle

are there errors in console?

And just because it's a resource, don't mean it's loaded. Look at the bundle definitions. Old style imports are supposed to be used in the legacy bundle.

@zopyx I would recomend to create a bundle for your resources for plone 5, if you want to use the legacy support can you check at plone-legacy bundle if your resources are loaded there?

More information on : https://github.com/plone/documentation/blob/6ef432fa385f44d63ac0a2021541ffa381c03c47/adapt-and-extend/theming/resources.rst


Resolving patch:
diff --git a/Products/CMFPlone/controlpanel/browser/resourceregistry.py b/Products/CMFPlone/controlpanel/browser/resourceregistry.py
--- a/Products/CMFPlone/controlpanel/browser/resourceregistry.py
+++ b/Products/CMFPlone/controlpanel/browser/resourceregistry.py
@@ -240,6 +240,7 @@ def save_js_build(self):
         bundle = self.get_bundles().get(req.form['bundle'])
         if bundle:
             bundle.last_compilation = datetime.now()
+            bundle.jscompilation = '++plone++{}'.format(filepath)
         return json.dumps({
             'success': True,
             'filepath': '++plone++' + filepath
@@ -255,6 +256,7 @@ def save_less_build(self):
         bundle = self.get_bundles().get(req.form['bundle'])
         if bundle:
             bundle.last_compilation = datetime.now()
+            bundle.csscompilation = '++plone++{}'.format(filepath)
         return json.dumps({
             'success': True,
             'filepath': '++plone++' + filepath
diff --git a/Products/CMFPlone/resources/viewlets/__init__.py b/Products/CMFPlone/resources/viewlets/__init__.py
--- a/Products/CMFPlone/resources/viewlets/__init__.py
+++ b/Products/CMFPlone/resources/viewlets/__init__.py
@@ -1,4 +1,3 @@
-
 from Products.ResourceRegistries.interfaces.registries import IResourceRegistry
 from Products.ResourceRegistries.interfaces.registries import ICSSRegistry
 from Products.ResourceRegistries.interfaces.registries import IKSSRegistry


Test patch: