Repository: zulip/zulip

Issue:
Mattermost import - support importing Guests
In our Mattermost import system, we currently import all users who aren't Team Admin as Zulip Members. That's not quite right, because per https://docs.mattermost.com/welcome/about-user-roles.html, we're not respecting Guest user status properly - a Mattermost Guest should be imported as a Zulip Guest too.

This should be relatively simple to do: in `process_user` in `zerver/data_import/mattermost.py` we have these two lines at the time of writing this, where user `role` is determined:
```
    role = UserProfile.ROLE_MEMBER
    if is_team_admin(user_dict):
        role = UserProfile.ROLE_REALM_OWNER
```

Similarly to `is_team_admin`, the code should inspect the `user_dict` here to check if this might be a guest user, and if so, set `role = UserProfile.ROLE_GUEST`.

For reference, https://docs.mattermost.com/manage/bulk-export-tool.html has some documentation about the data format of the export. Though it doesn't seem to explain how exactly Guest status is encoded, so that might require looking up somewhere else or just manually generating a Mattermost export with a guest user and inspecting the data to determine how it's marked.

Mattermost import - support importing Guests
In our Mattermost import system, we currently import all users who aren't Team Admin as Zulip Members. That's not quite right, because per https://docs.mattermost.com/welcome/about-user-roles.html, we're not respecting Guest user status properly - a Mattermost Guest should be imported as a Zulip Guest too.

This should be relatively simple to do: in `process_user` in `zerver/data_import/mattermost.py` we have these two lines at the time of writing this, where user `role` is determined:
```
    role = UserProfile.ROLE_MEMBER
    if is_team_admin(user_dict):
        role = UserProfile.ROLE_REALM_OWNER
```

Similarly to `is_team_admin`, the code should inspect the `user_dict` here to check if this might be a guest user, and if so, set `role = UserProfile.ROLE_GUEST`.

For reference, https://docs.mattermost.com/manage/bulk-export-tool.html has some documentation about the data format of the export. Though it doesn't seem to explain how exactly Guest status is encoded, so that might require looking up somewhere else or just manually generating a Mattermost export with a guest user and inspecting the data to determine how it's marked.



Discussion:
Hello @zulip/server-misc members, this issue was labeled with the "area: export/import" label, so you may want to check it out!

<!-- areaLabelAddition -->

@zulipbot claim
Hello @mpedro12, it looks like you've currently claimed 1 issue in this repository. We encourage new contributors to focus their efforts on at most 1 issue at a time, so please complete your work on your other claimed issues before trying to claim this issue again.

We look forward to your valuable contributions!

@zulipbot claim
Hello @zulip/server-misc members, this issue was labeled with the "area: export/import" label, so you may want to check it out!

<!-- areaLabelAddition -->

@zulipbot claim
Hello @mpedro12, it looks like you've currently claimed 1 issue in this repository. We encourage new contributors to focus their efforts on at most 1 issue at a time, so please complete your work on your other claimed issues before trying to claim this issue again.

We look forward to your valuable contributions!

@zulipbot claim

Resolving patch:
diff --git a/zerver/data_import/mattermost.py b/zerver/data_import/mattermost.py
--- a/zerver/data_import/mattermost.py
+++ b/zerver/data_import/mattermost.py
@@ -70,6 +70,14 @@ def is_team_admin(user_dict: Dict[str, Any]) -> bool:
             for team in user_dict["teams"]
         )
 
+    def is_team_guest(user_dict: Dict[str, Any]) -> bool:
+        if user_dict["teams"] is None:
+            return False
+        for team in user_dict["teams"]:
+            if team["name"] == team_name and "team_guest" in team["roles"]:
+                return True
+        return False
+
     def get_full_name(user_dict: Dict[str, Any]) -> str:
         full_name = "{} {}".format(user_dict["first_name"], user_dict["last_name"])
         if full_name.strip():
@@ -85,9 +93,12 @@ def get_full_name(user_dict: Dict[str, Any]) -> str:
     date_joined = int(timezone_now().timestamp())
     timezone = "UTC"
 
-    role = UserProfile.ROLE_MEMBER
     if is_team_admin(user_dict):
         role = UserProfile.ROLE_REALM_OWNER
+    elif is_team_guest(user_dict):
+        role = UserProfile.ROLE_GUEST
+    else:
+        role = UserProfile.ROLE_MEMBER
 
     if user_dict["is_mirror_dummy"]:
         is_active = False


Test patch:
diff --git a/zerver/tests/fixtures/mattermost_fixtures/guestExport.json b/zerver/tests/fixtures/mattermost_fixtures/guestExport.json
new file mode 100644
--- /dev/null
+++ b/zerver/tests/fixtures/mattermost_fixtures/guestExport.json
@@ -0,0 +1,36 @@
+{"type":"version","version":1}
+{"type":"team","team":{"name":"gryffindor","display_name":"Iago Realm","type":"O","description":"","allow_open_invite":true}}
+{"type":"team","team":{"name":"slytherin","display_name":"Othello Team","type":"O","description":"","allow_open_invite":true}}
+{"type":"channel","channel":{"team":"gryffindor","name":"gryffindor-common-room","display_name":"Gryffindor common room","type":"O","header":"","purpose":"A place for talking about Gryffindor common room"}}
+{"type":"channel","channel":{"team":"gryffindor","name":"gryffindor-quidditch-team","display_name":"Gryffindor quidditch team","type":"O","header":"","purpose":"A place for talking about Gryffindor quidditch team"}}
+{"type":"channel","channel":{"team":"slytherin","name":"slytherin-common-room","display_name":"Slytherin common room","type":"O","header":"","purpose":""}}
+{"type":"channel","channel":{"team":"gryffindor","name":"dumbledores-army","display_name":"Dumbledores army","type":"P","header":"https//:github.com/zulip/zulip","purpose":"A place for talking about Dumbledores army"}}
+{"type":"channel","channel":{"team":"slytherin","name":"slytherin-quidditch-team","display_name":"Slytherin quidditch team","type":"O","header":"","purpose":""}}
+{"type":"user","user":{"username":"ron","email":"ron@zulip.com","auth_service":"","nickname":"","first_name":"Ron","last_name":"Weasley","position":"","roles":"system_user","locale":"en","teams":[{"name":"gryffindor","roles":"team_user","channels":[{"name":"gryffindor-quidditch-team","roles":"channel_user","notify_props":{"desktop":"default","mobile":"default","mark_unread":"all"},"favorite":false},{"name":"gryffindor-common-room","roles":"channel_user","notify_props":{"desktop":"default","mobile":"default","mark_unread":"all"},"favorite":false},{"name":"dumbledores-army","roles":"channel_user","notify_props":{"desktop":"default","mobile":"default","mark_unread":"all"},"favorite":false}]}],"notify_props":{"desktop":"mention","desktop_sound":"true","email":"true","mobile":"mention","mobile_push_status":"away","channel":"true","comments":"never","mention_keys":"ron,@ron"}}}
+{"type":"user","user":{"username":"harry","email":"harry@zulip.com","auth_service":"","nickname":"","first_name":"Harry","last_name":"Potter","position":"","roles":"system_admin system_user","locale":"en","teams":[{"name":"gryffindor","roles":"team_admin team_user","channels":[{"name":"dumbledores-army","roles":"channel_admin channel_user","notify_props":{"desktop":"default","mobile":"default","mark_unread":"all"},"favorite":false},{"name":"gryffindor-common-room","roles":"channel_admin channel_user","notify_props":{"desktop":"default","mobile":"default","mark_unread":"all"},"favorite":false},{"name":"gryffindor-quidditch-team","roles":"channel_admin channel_user","notify_props":{"desktop":"default","mobile":"default","mark_unread":"all"},"favorite":false}]}],"notify_props":{"desktop":"mention","desktop_sound":"true","email":"true","mobile":"mention","mobile_push_status":"away","channel":"true","comments":"never","mention_keys":"harry,@harry"}}}
+{"type":"user","user":{"username":"malfoy","email":"malfoy@zulip.com","auth_service":"","nickname":"","first_name":"","last_name":"","position":"","roles":"system_user","locale":"en","teams":[{"name":"slytherin","roles":"team_admin team_user","channels":[{"name":"slytherin-common-room","roles":"channel_admin channel_user","notify_props":{"desktop":"default","mobile":"default","mark_unread":"all"},"favorite":false},{"name":"slytherin-quidditch-team","roles":"channel_admin channel_user","notify_props":{"desktop":"default","mobile":"default","mark_unread":"all"},"favorite":false}]}],"notify_props":{"desktop":"mention","desktop_sound":"true","email":"true","mobile":"mention","mobile_push_status":"away","channel":"true","comments":"never","mention_keys":"malfoy,@malfoy"}}}
+{"type":"user","user":{"username":"pansy","email":"pansy@zulip.com","auth_service":"","nickname":"","first_name":"","last_name":"","position":"","roles":"system_user","locale":"en","teams":[{"name":"slytherin","roles":"team_admin team_user","channels":[{"name":"slytherin-common-room","roles":"channel_admin channel_user","notify_props":{"desktop":"default","mobile":"default","mark_unread":"all"},"favorite":false},{"name":"slytherin-quidditch-team","roles":"channel_admin channel_user","notify_props":{"desktop":"default","mobile":"default","mark_unread":"all"},"favorite":false}]}],"notify_props":{"desktop":"mention","desktop_sound":"true","email":"true","mobile":"mention","mobile_push_status":"away","channel":"true","comments":"never","mention_keys":"malfoy,@malfoy"}}}
+{"type":"user","user":{"username":"snape","email":"snape@zulip.com","auth_service":"","nickname":"","first_name":"Severus","last_name":"Snape","position":"","roles":"system_user","locale":"en","teams":[{"name":"slytherin","roles":"team_user","channels":[{"name":"slytherin-common-room","roles":"channel_user","notify_props":{"desktop":"default","mobile":"default","mark_unread":"all"},"favorite":false}]}],"notify_props":{"desktop":"mention","desktop_sound":"true","email":"true","mobile":"mention","mobile_push_status":"away","channel":"true","comments":"never","mention_keys":"snape,@snape"}}}
+{"type":"user","user":{"username":"sirius","email":"sirius@zulip.com","auth_service":"","nickname":"","first_name":"Sirius","last_name":"Black","position":"","roles":"system_guest","locale":"en","teams":[{"name":"slytherin","roles":"team_guest","channels":[{"name":"slytherin-common-room","roles":"channel_guest","notify_props":{"desktop":"default","mobile":"default","mark_unread":"all"},"favorite":false}]}],"notify_props":{"desktop":"mention","desktop_sound":"true","email":"true","mobile":"mention","mobile_push_status":"away","channel":"true","comments":"never","mention_keys":"sirius,@sirius"}}}
+{"type":"post","post":{"team":"gryffindor","channel":"dumbledores-army","user":"harry","message":"harry joined the channel.","create_at":1553166657086,"reactions":null,"replies":[{"user":"ron","message":"The weather is so hot!","create_at":1553166584976}]}}
+{"type":"post","post":{"team":"gryffindor","channel":"gryffindor-common-room","user":"ron","message":"ron joined the channel.","create_at":1553166512493,"reactions":null,"replies":null}}
+{"type":"post","post":{"team":"gryffindor","channel":"gryffindor-quidditch-team","user":"harry","message":"harry joined the team.","create_at":1553165141670,"reactions":null,"replies":null}}
+{"type":"post","post":{"team":"gryffindor","channel":"gryffindor-quidditch-team","user":"harry","message":"Awesome!","create_at":1553166557928,"reactions":[{"user":"malfoy","create_at":1553166812156,"emoji_name":"tick"}],"replies":null}}
+{"type":"post","post":{"team":"slytherin","channel":"slytherin-quidditch-team","user":"malfoy","message":"malfoy joined the team.","create_at":1553166852598,"reactions":null,"replies":null}}
+{"type":"post","post":{"team":"gryffindor","channel":"gryffindor-quidditch-team","user":"ron","message":"ron joined the team.","create_at":1553166512482,"reactions":null,"replies":null}}
+{"type":"post","post":{"team":"gryffindor","channel":"gryffindor-quidditch-team","user":"ron","message":"Hey folks","create_at":1553166519720,"reactions":null,"replies":null}}
+{"type":"post","post":{"team":"gryffindor","channel":"gryffindor-quidditch-team","user":"harry","message":"@ron Welcome mate!","create_at":1553166519726,"reactions":null,"replies":null}}
+{"type":"post","post":{"team":"gryffindor","channel":"dumbledores-army","user":"harry","message":"ron added to the channel by harry.","create_at":1553166681045,"reactions":null,"replies":null}}
+{"type":"post","post":{"team":"gryffindor","channel":"gryffindor-quidditch-team","user":"harry","message":"Hello world","create_at":1553165193242,"reactions":[{"user":"harry","create_at":1553165521410,"emoji_name":"tick"},{"user":"ron","create_at":1553166530805,"emoji_name":"smile"},{"user":"ron","create_at":1553166540953,"emoji_name":"world_map"}],"replies":null}}
+{"type":"post","post":{"team":"gryffindor","channel":"gryffindor-common-room","user":"harry","message":"Looks like this channel is empty","create_at":1553166567370,"reactions":[{"user":"ron","create_at":1553166584976,"emoji_name":"rocket"}],"replies":null}}
+{"type":"post","post":{"team":"gryffindor","channel":"gryffindor-quidditch-team","user":"ron","message":"How is everything going","create_at":1553166525124,"reactions":[{"user":"harry","create_at":1553166552827,"emoji_name":"apple"}],"replies":null}}
+{"type":"post","post":{"team":"gryffindor","channel":"gryffindor-common-room","user":"ron","message":"Not really","create_at":1553166593455,"reactions":null,"replies":null}}
+{"type":"post","post":{"team":"gryffindor","channel":"dumbledores-army","user":"ron","message":"hello","create_at":1553166686344,"reactions":null,"replies":null}}
+{"type":"post","post":{"team":"gryffindor","channel":"dumbledores-army","user":"harry","message":"hey everyone","create_at":1553166668668,"reactions":[{"user":"ron","create_at":1553166695260,"emoji_name":"grin"}],"replies":null}}
+{"type":"post","post":{"team":"slytherin","channel":"slytherin-common-room","user":"malfoy","message":"malfoy joined the channel.","create_at":1553166852612,"reactions":null,"replies":null}}
+{"type":"post","post":{"team":"slytherin","channel":"slytherin-quidditch-team","user":"malfoy","message":":rofl:4","create_at":1553166916448,"reactions":[{"user":"harry","create_at":1553167016056,"emoji_name":"peerdium"}],"replies":null}}
+{"type":"post","post":{"team":"slytherin","channel":"slytherin-quidditch-team","user":"malfoy","message":"Hello folks","create_at":1553166858280,"reactions":[{"user":"harry","create_at":1553166903980,"emoji_name":"joy"}],"replies":null}}
+{"type":"post","post":{"team":"gryffindor","channel":"gryffindor-common-room","user":"harry","message":"harry joined the channel.","create_at":1553165141689,"reactions":null,"replies":null}}
+{"type":"post","post":{"team":"gryffindor","channel":"gryffindor-quidditch-team","user":"snape","message":"Hey folks! I was always in your team. Time to go now.","create_at":1553166740759,"reactions":null,"replies":null}}
+{"type":"emoji","emoji":{"name":"peerdium","image":"exported_emoji/h15ni7kf1bnj7jeua4qhmctsdo/image.png"}}
+{"type":"emoji","emoji":{"name":"tick","image":"exported_emoji/7u7x8ytgp78q8jir81o9ejwwnr/image.png"}}
diff --git a/zerver/tests/test_mattermost_importer.py b/zerver/tests/test_mattermost_importer.py
--- a/zerver/tests/test_mattermost_importer.py
+++ b/zerver/tests/test_mattermost_importer.py
@@ -127,6 +127,37 @@ def test_process_user(self) -> None:
         self.assertEqual(user["short_name"], "snape")
         self.assertEqual(user["timezone"], "UTC")
 
+    def test_process_guest_user(self) -> None:
+        user_id_mapper = IdMapper()
+        fixture_file_name = self.fixture_file_name("guestExport.json", "mattermost_fixtures")
+        mattermost_data = mattermost_data_file_to_dict(fixture_file_name)
+        username_to_user = create_username_to_user_mapping(mattermost_data["user"])
+        reset_mirror_dummy_users(username_to_user)
+
+        sirius_dict = username_to_user["sirius"]
+        sirius_dict["is_mirror_dummy"] = False
+
+        realm_id = 3
+
+        team_name = "slytherin"
+        user = process_user(sirius_dict, realm_id, team_name, user_id_mapper)
+        self.assertEqual(user["avatar_source"], "G")
+        self.assertEqual(user["delivery_email"], "sirius@zulip.com")
+        self.assertEqual(user["email"], "sirius@zulip.com")
+        self.assertEqual(user["full_name"], "Sirius Black")
+        self.assertEqual(user["role"], UserProfile.ROLE_GUEST)
+        self.assertEqual(user["is_mirror_dummy"], False)
+        self.assertEqual(user["realm"], 3)
+        self.assertEqual(user["short_name"], "sirius")
+        self.assertEqual(user["timezone"], "UTC")
+
+        # A guest user with a `null` team value should be a regular
+        # user. (It's a bit of a mystery why the Mattermost export
+        # tool generates such `teams` lists).
+        sirius_dict["teams"] = None
+        user = process_user(sirius_dict, realm_id, team_name, user_id_mapper)
+        self.assertEqual(user["role"], UserProfile.ROLE_MEMBER)
+
     def test_convert_user_data(self) -> None:
         user_id_mapper = IdMapper()
         realm_id = 3