Repository: privacyidea/privacyidea

Issue:
Event handler to remove response parameters to allow enrolling softtokens without seeing the QR Code
The helpdesk user or admin could be able to enroll a token for a user, without seeing the qrcode.

The qrcode could be sent to a user via an event handler.
Or the qrcode could be made available in the self service portal of the user, so that the user can "pick up" his token in the selfservice portal.


Discussion:
This is a similar problem like #1196 

There is some normal data in the response, but we need to remove this data from the response.
In this case it is the seed and the QR code, in the case of #1196 it is the resetted PIN.

We could use a POST handler, that removes arbitrary ``details`` or ``results`` from the response. The helpdesk user in this case also should not see the seed!

In regards to the user "picking up" the QR code, this might mean we need some internal notification. The QR code could be sent via the event handler to the internal notification system. See #614
This is a similar problem like the resetting of the PIN. #1196 
This could mean that the notification handler could modify an information from the response.
It would somehow redirect the information: Do not send it via response anymore, but send it to an email...



The mangle pre-event handler from #1810 modifies the request before it is passed to the privacyIDEA API. This issue would, similarly, implement a post-event handler that modifies the response before it is passed to the webserver. Maybe this could even be the same event handler?
Such an Event-Handler would only allow the position ``[post]``.

It could modify, add or delete any existing entry in the JSON response.
We need to somehow specify the value in the JSON response. Maybe we simply do this by specifing it from the top like:

* result:value
* detail:message

Or we could specify a complete tree - which would only make sense in the case of ``delete``:

* detail

The different actions would require different attributes:

**delete**

Only requires the key to delete like ``detail`` or ``detail:message``.

**add**

Would require the key and the value:

* key: ``detail:message``
* value: "This is our little secret" (string)

If the key already exists, the existing key would be overwritten.

**modify**

*TBD*
  


We could also check if it makes sense to use a pre-baked solution like [JSON Patch](http://jsonpatch.com/), which seems to have a Python binding.
Interesting, JSON Pointer (https://tools.ietf.org/html/rfc6901) and JSON Patch (https://tools.ietf.org/html/rfc6902) is specified in RFC. Thanks a lot for this hint! I think it makes sense to adhere to the pointer definition so the previsou example would be ``/detail`` and ``/detail/message``.
I think this is nice.

But I do not think that we should ask the user to really use a JSON patch to add a value. In roughly 99% this would be to overcomplicated for the user.

I would also like to keep it a bit similar to #1810. And in this case JSON patch does not work out.

Yes, I guess using JSON Patch would be overkill. But using a notation similar to JSON Pointer, or even the "real" JSON pointer notation, sounds nice. :)
I also just checked: "pre" and "post" have no influence on the actions of an event handler. Thus we need to have an event handler, that mangels the request parameters (RequestMangler) and a handler that mangles the response (ResponseMangler).
There is a [jsonpointer module](https://github.com/stefankoegl/python-json-pointer) which could be used to modify the JSON response. But it has some drawbacks:

1. You can not set a key within an non-existing subpath. If you want to set /detail/message, /detail has to exist
2. You can not delete a pointer. You can only set it empty.
3. The implementation was not maintained since 2018.

I think we will do it on our own.

Resolving patch:
diff --git a/privacyidea/api/event.py b/privacyidea/api/event.py
--- a/privacyidea/api/event.py
+++ b/privacyidea/api/event.py
@@ -67,6 +67,7 @@ def get_eventhandling(eventid=None):
     elif eventid == "handlermodules":
         # TODO: We need to provide a dynamic list of event handlers
         res = ["UserNotification", "Token", "Federation", "Script", "Counter"]
+        res.append("ResponseMangler")
     else:
         res = g.event_config.get_event(eventid)
     g.audit_object.log({"success": True})
diff --git a/privacyidea/lib/event.py b/privacyidea/lib/event.py
--- a/privacyidea/lib/event.py
+++ b/privacyidea/lib/event.py
@@ -159,17 +159,20 @@ def get_handler_object(handlername):
     from privacyidea.lib.eventhandler.federationhandler import \
         FederationEventHandler
     from privacyidea.lib.eventhandler.counterhandler import CounterEventHandler
+    from privacyidea.lib.eventhandler.responsemangler import ResponseManglerEventHandler
     h_obj = None
     if handlername == "UserNotification":
         h_obj = UserNotificationEventHandler()
-    if handlername == "Token":
+    elif handlername == "Token":
         h_obj = TokenEventHandler()
-    if handlername == "Script":
+    elif handlername == "Script":
         h_obj = ScriptEventHandler()
-    if handlername == "Federation":
+    elif handlername == "Federation":
         h_obj = FederationEventHandler()
-    if handlername == "Counter":
+    elif handlername == "Counter":
         h_obj = CounterEventHandler()
+    elif handlername == "ResponseMangler":
+        h_obj = ResponseManglerEventHandler()
     return h_obj
 
 
diff --git a/privacyidea/lib/eventhandler/responsemangler.py b/privacyidea/lib/eventhandler/responsemangler.py
new file mode 100644
--- /dev/null
+++ b/privacyidea/lib/eventhandler/responsemangler.py
@@ -0,0 +1,169 @@
+# -*- coding: utf-8 -*-
+#
+#  2019-09-14 Cornelius Kölbel <cornelius.koelbel@netknights.it>
+#             Initial writup
+#
+# License:  AGPLv3
+# (c) 2019. Cornelius Kölbel
+#
+# This code is free software; you can redistribute it and/or
+# modify it under the terms of the GNU AFFERO GENERAL PUBLIC LICENSE
+# License as published by the Free Software Foundation; either
+# version 3 of the License, or any later version.
+#
+# This code is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU AFFERO GENERAL PUBLIC LICENSE for more details.
+#
+# You should have received a copy of the GNU Affero General Public
+# License along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+#
+__doc__ = """This is the event handler module that can mangle the JSON response.
+We can add or delete key or even subtrees in the JSON response of a request.
+
+The key is identified by a JSON Pointer
+(see https://tools.ietf.org/html/rfc6901)
+"""
+from privacyidea.lib.eventhandler.base import BaseEventHandler
+from privacyidea.lib.utils import is_true
+from privacyidea.lib import _
+import json
+import logging
+
+
+log = logging.getLogger(__name__)
+
+
+class ACTION_TYPE(object):
+    """
+    Allowed actions
+    """
+    SET = "set"
+    DELETE = "delete"
+
+
+class ResponseManglerEventHandler(BaseEventHandler):
+    """
+    An Eventhandler needs to return a list of actions, which it can handle.
+
+    It also returns a list of allowed action and conditions
+
+    It returns an identifier, which can be used in the eventhandlig definitions
+    """
+
+    identifier = "ResponseMangler"
+    description = "This event handler can mangle the JSON response."
+
+    @property
+    def allowed_positions(cls):
+        """
+        This returns the allowed positions of the event handler definition.
+        The ResponseMangler can only be located at the "post" position
+
+        :return: list of allowed positions
+        """
+        return ["post"]
+
+    @property
+    def actions(cls):
+        """
+        This method returns a dictionary of allowed actions and possible
+        options in this handler module.
+
+        :return: dict with actions
+        """
+        actions = {ACTION_TYPE.DELETE:
+                       {"JSON pointer":
+                            {"type": "str",
+                             "required": True,
+                             "description": _("The JSON pointer (key) that should be deleted. Please "
+                                              "specify in the format '/detail/message'.")}
+                        },
+                   ACTION_TYPE.SET:
+                       {"JSON pointer":
+                            {"type": "str",
+                             "required": True,
+                             "description": _("The JSON pointer (key) that should be set. "
+                                              "Please specify in the format '/detail/message'.")
+                             },
+                        "type":
+                            {"type": "str",
+                             "required": True,
+                             "description": _("The type of the value."),
+                             "value": ["string", "integer", "bool"]
+                            },
+                        "value":
+                            {"type": "str",
+                             "required": True,
+                             "description": _("The value of the JSON key that should be set.")
+                            }
+                        }
+                   }
+        return actions
+
+    def do(self, action, options=None):
+        """
+        This method executes the defined action in the given event.
+
+        :param action:
+        :param options: Contains the flask parameters g, request, response
+            and the handler_def configuration
+        :type options: dict
+        :return:
+        """
+        ret = True
+        response = options.get("response")
+        try:
+            content = self._get_response_content(response)
+        except Exception:
+            # The body could not be decoded to JSON. Actually this would be
+            # a BadRequest, but we refrain from importing from werkzeug, here.
+            log.info("The body could not be decoded to JSON.")
+            # Early exist
+            return ret
+        handler_def = options.get("handler_def")
+        handler_options = handler_def.get("options", {})
+
+        json_pointer = handler_options.get("JSON pointer", "")
+        value = handler_options.get("value")
+        type = handler_options.get("type")
+        comp = json_pointer.split("/")
+        comp = [x for x in comp if x]
+        if action.lower() == ACTION_TYPE.DELETE:
+            try:
+                if len(comp) == 1:
+                    del(content[comp[0]])
+                elif len(comp) == 2:
+                    del(content[comp[0]][comp[1]])
+                elif len(comp) == 3:
+                    del (content[comp[0]][comp[1]][comp[2]])
+                else:
+                    log.warning("JSON pointer length of {0!s} not supported.".format(len(comp)))
+                options.get("response").data = json.dumps(content)
+            except KeyError:
+                log.warning("Can not delete response JSON Pointer {0!s}.".format(json_pointer))
+        elif action.lower() == ACTION_TYPE.SET:
+            try:
+                if type == "integer":
+                    value = int(value)
+                elif type == "bool":
+                    value = is_true(value)
+            except ValueError:
+                log.warning("Failed to convert value")
+
+            if len(comp) == 1:
+                content[comp[0]] = value
+            elif len(comp) == 2:
+                content[comp[0]] = content.get(comp[0]) or {}
+                content[comp[0]][comp[1]] = value
+            elif len(comp) == 3:
+                content[comp[0]] = content.get(comp[0]) or {}
+                content[comp[0]][comp[1]] = content[comp[0]].get(comp[1]) or {}
+                content[comp[0]][comp[1]][comp[2]] = value
+            else:
+                log.warning("JSON pointer of length {0!s} not supported.".format(len(comp)))
+            options.get("response").data = json.dumps(content)
+
+        return ret


Test patch:
diff --git a/tests/test_lib_events.py b/tests/test_lib_events.py
--- a/tests/test_lib_events.py
+++ b/tests/test_lib_events.py
@@ -17,6 +17,7 @@
                                                        ACTION_TYPE, VALIDITY)
 from privacyidea.lib.eventhandler.scripthandler import ScriptEventHandler, SCRIPT_WAIT, SCRIPT_BACKGROUND
 from privacyidea.lib.eventhandler.counterhandler import CounterEventHandler
+from privacyidea.lib.eventhandler.responsemangler import ResponseManglerEventHandler
 from privacyidea.models import EventCounter, TokenOwner
 from privacyidea.lib.eventhandler.federationhandler import FederationEventHandler
 from privacyidea.lib.eventhandler.base import BaseEventHandler, CONDITION
@@ -41,6 +42,7 @@
 from dateutil.parser import parse as parse_date_string
 from dateutil.tz import tzlocal
 from privacyidea.app import PiResponseClass as Response
+import json
 
 
 class EventHandlerLibTestCase(MyTestCase):
@@ -909,6 +911,235 @@ def test_02_forward_admin_request(self):
                          "https://remote/token/init")
 
 
+class ResponseManglerTestCase(MyTestCase):
+
+    def test_01_delete_response(self):
+        actions = ResponseManglerEventHandler().actions
+        self.assertTrue("delete" in actions, actions)
+        self.assertTrue("set" in actions, actions)
+
+        pos = ResponseManglerEventHandler().allowed_positions
+        self.assertEqual(set(pos), {"post"})
+
+        g = FakeFlaskG()
+        audit_object = FakeAudit()
+        builder = EnvironBuilder(method='POST',
+                                 data={'serial': "SPASS01"},
+                                 headers={})
+        env = builder.get_environ()
+        # Set the remote address so that we can filter for it
+        env["REMOTE_ADDR"] = "10.0.0.1"
+        g.client_ip = env["REMOTE_ADDR"]
+        req = Request(env)
+        req.all_data = {"serial": "SPASS01", "type": "spass"}
+        resp = Response(mimetype='application/json')
+
+        # delete JSON pointer with two components
+        resp.data = """{"result": {"value": true}, "detail": {"message": "Du", "error": 1}}"""
+        options = {"g": g,
+                   "request": req,
+                   "response": resp,
+                   "handler_def": {"options":
+                                       {"JSON pointer": "/detail/message"}
+                                   }
+                   }
+        r_handler = ResponseManglerEventHandler()
+        res = r_handler.do("delete", options=options)
+        self.assertTrue(res)
+        self.assertEqual(resp.json["detail"]["error"], 1)
+        self.assertNotIn("message", resp.json["detail"])
+
+        # delete JSON pointer with one component
+        resp.data = """{"result": {"value": true}, "detail": {"message": "Du", "error": 1}}"""
+        options = {"g": g,
+                   "request": req,
+                   "response": resp,
+                   "handler_def": {"options":
+                                       {"JSON pointer": "/result"}
+                                   }
+                   }
+        r_handler = ResponseManglerEventHandler()
+        res = r_handler.do("delete", options=options)
+        self.assertTrue(res)
+        self.assertIn("message", resp.json["detail"])
+        self.assertNotIn("result", resp.json)
+
+        # delete JSON pointer with three components
+        resp.data = """{"result": {"value": true}, "detail": {"data": {"Du": "Da"}, "error": 1}}"""
+        options = {"g": g,
+                   "request": req,
+                   "response": resp,
+                   "handler_def": {"options":
+                                       {"JSON pointer": "/detail/data/Du"}
+                                   }
+                   }
+        r_handler = ResponseManglerEventHandler()
+        res = r_handler.do("delete", options=options)
+        self.assertTrue(res)
+        self.assertIn("error", resp.json["detail"])
+        self.assertNotIn("Du", resp.json["detail"]["data"])
+
+        # JSON pointer with more than 3 components not supported
+        resp.data = """{"result": {"value": true},
+                        "detail": {"message": {"comp1": {"comp2": "test"}}, "error": 1}}"""
+        options = {"g": g,
+                   "request": req,
+                   "response": resp,
+                   "handler_def": {"options":
+                                       {"JSON pointer": "/detail/message/comp1/comp2"}
+                                   }
+                   }
+        r_handler = ResponseManglerEventHandler()
+        res = r_handler.do("delete", options=options)
+        self.assertTrue(res)
+        self.assertIn("comp2", resp.json["detail"]["message"]["comp1"])
+        self.assertIn("result", resp.json)
+
+        # Invalid JSON pointer will cause a log warning but will not change the response
+        resp.data = """{"result": {"value": true}, "detail": {"message": "Du", "error": 1}}"""
+        options = {"g": g,
+                   "request": req,
+                   "response": resp,
+                   "handler_def": {"options":
+                                       {"JSON pointer": "/notexist"}
+                                   }
+                   }
+        r_handler = ResponseManglerEventHandler()
+        res = r_handler.do("delete", options=options)
+        self.assertTrue(res)
+        self.assertIn("message", resp.json["detail"])
+        self.assertIn("result", resp.json)
+
+        # What happens if we have a non-json response, like in GET /token?outform=csv
+        # Nothing is changed!
+        csv = b"""column1, column2, column3
+        column1, column2, column3
+        """
+        resp.data = csv
+        options = {"g": g,
+                   "request": req,
+                   "response": resp,
+                   "handler_def": {"options":
+                                       {"JSON pointer": "/notexist"}
+                                   }
+                   }
+        r_handler = ResponseManglerEventHandler()
+        res = r_handler.do("delete", options=options)
+        self.assertTrue(res)
+        self.assertEqual(resp.data, csv)
+
+    def test_02_set_response(self):
+
+        g = FakeFlaskG()
+        audit_object = FakeAudit()
+        builder = EnvironBuilder(method='POST',
+                                 data={'serial': "SPASS01"},
+                                 headers={})
+        env = builder.get_environ()
+        # Set the remote address so that we can filter for it
+        env["REMOTE_ADDR"] = "10.0.0.1"
+        g.client_ip = env["REMOTE_ADDR"]
+        req = Request(env)
+        req.all_data = {"serial": "SPASS01", "type": "spass"}
+        resp = Response(mimetype='application/json')
+
+        # add JSON pointer with one component
+        resp.data = """{"result": {"value": true}, "detail": {"message": "Du", "error": 1}}"""
+        options = {"g": g,
+                   "request": req,
+                   "response": resp,
+                   "handler_def": {"options":
+                                       {"JSON pointer": "/something",
+                                        "value": "special",
+                                        "type": "string"}
+                                   }
+                   }
+        r_handler = ResponseManglerEventHandler()
+        res = r_handler.do("set", options=options)
+        self.assertTrue(res)
+        self.assertEqual(resp.json["something"], "special")
+
+        # add JSON pointer with two components
+        resp.data = """{"result": {"value": true}, "detail": {"message": "Du", "error": 1}}"""
+        options = {"g": g,
+                   "request": req,
+                   "response": resp,
+                   "handler_def": {"options":
+                                       {"JSON pointer": "/detail/something",
+                                        "value": "special",
+                                        "type": "string"}
+                                   }
+                   }
+        r_handler = ResponseManglerEventHandler()
+        res = r_handler.do("set", options=options)
+        self.assertTrue(res)
+        self.assertEqual(resp.json["detail"]["something"], "special")
+
+        # change JSON pointer with two components
+        resp.data = """{"result": {"value": true}, "detail": {"message": "Du", "error": 1}}"""
+        options = {"g": g,
+                   "request": req,
+                   "response": resp,
+                   "handler_def": {"options":
+                                       {"JSON pointer": "/detail/message",
+                                        "value": "special",
+                                        "type": "string"}
+                                   }
+                   }
+        r_handler = ResponseManglerEventHandler()
+        res = r_handler.do("set", options=options)
+        self.assertTrue(res)
+        self.assertEqual(resp.json["detail"]["message"], "special")
+
+        # add the components, that do not yet exist
+        resp.data = """{"result": {"value": true}, "detail": {"message": "Du", "error": 1}}"""
+        options = {"g": g,
+                   "request": req,
+                   "response": resp,
+                   "handler_def": {"options":
+                                       {"JSON pointer": "/comp1/comp2/comp3",
+                                        "value": "1",
+                                        "type": "bool"}
+                                   }
+                   }
+        r_handler = ResponseManglerEventHandler()
+        res = r_handler.do("set", options=options)
+        self.assertTrue(res)
+        self.assertEqual(resp.json["comp1"]["comp2"]["comp3"], True)
+
+        # JSON pointer with more than 3 components not supported
+        resp.data = """{"result": {"value": true}, "detail": {"message": "Du", "error": 1}}"""
+        options = {"g": g,
+                   "request": req,
+                   "response": resp,
+                   "handler_def": {"options":
+                                       {"JSON pointer": "/comp1/comp2/comp3/comp4",
+                                        "value": "1",
+                                        "type": "integer"}
+                                   }
+                   }
+        r_handler = ResponseManglerEventHandler()
+        res = r_handler.do("set", options=options)
+        self.assertTrue(res)
+        self.assertNotIn("comp1", resp.json)
+
+        # Wrong type declaration
+        resp.data = """{"result": {"value": true}, "detail": {"message": "Du", "error": 1}}"""
+        options = {"g": g,
+                   "request": req,
+                   "response": resp,
+                   "handler_def": {"options":
+                                       {"JSON pointer": "/comp1/comp2",
+                                        "value": "notint",
+                                        "type": "integer"}
+                                   }
+                   }
+        r_handler = ResponseManglerEventHandler()
+        res = r_handler.do("set", options=options)
+        self.assertTrue(res)
+        self.assertEqual(resp.json["comp1"]["comp2"], "notint")
+
+
 class TokenEventTestCase(MyTestCase):
 
     def test_01_set_tokenrealm(self):