internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: matrix-org/synapse

Issue:
Support /sync?set_presence=unavailable
as per https://github.com/matrix-org/matrix-doc/pull/780
r0.4.0 TODO
* [ ] https://github.com/matrix-org/synapse/issues/3772 - "unavailable" presence status for /sync
* [ ] declare support for it in `/versions`


Discussion:

Resolving patch:
diff --git a/synapse/rest/client/v2_alpha/sync.py b/synapse/rest/client/v2_alpha/sync.py
--- a/synapse/rest/client/v2_alpha/sync.py
+++ b/synapse/rest/client/v2_alpha/sync.py
@@ -75,7 +75,7 @@ class SyncRestServlet(RestServlet):
     """
 
     PATTERNS = client_v2_patterns("/sync$")
-    ALLOWED_PRESENCE = set(["online", "offline"])
+    ALLOWED_PRESENCE = set(["online", "offline", "unavailable"])
 
     def __init__(self, hs):
         super(SyncRestServlet, self).__init__()
diff --git a/synapse/rest/client/versions.py b/synapse/rest/client/versions.py
--- a/synapse/rest/client/versions.py
+++ b/synapse/rest/client/versions.py
@@ -38,6 +38,7 @@ def on_GET(self, request):
                 "r0.1.0",
                 "r0.2.0",
                 "r0.3.0",
+                "r0.4.0",
             ],
             # as per MSC1497:
             "unstable_features": {


Test patch:
Repository: matrix-org/synapse

Issue:
Synapse returns a 500 error when requesting /state or /state_ids over federation with no event_id
eg: `GET /_matrix/federation/v1/state/!room:domain.com/` (lack of `?event_id=$whatever`)

This should probably be a 400 error, which appears to be the intention given the stacktrace. However, a 500 is returned.

```
synapse.http.server - 101 - ERROR - GET-74- Failed handle request via <function _async_render at 0x7f0817ae3e60>: <XForwardedForRequest at 0x7f0729263cf8 method='GET' uri='/_matrix/federation/v1/state/!redacted/' clientproto='HTTP/1.0' site=8448>: Traceback (most recent call last):
  File "/root/.synapse/local/lib/python2.7/site-packages/synapse/federation/transport/server.py", line 227, in new_func
    origin, content, request.args, *args, **kwargs
  File "/root/.synapse/local/lib/python2.7/site-packages/synapse/federation/transport/server.py", line 343, in on_GET
    query.get("event_id", [None])[0],
  File "/root/.synapse/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1613, in unwindGenerator
    return _cancellableInlineCallbacks(gen)
  File "/root/.synapse/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1529, in _cancellableInlineCallbacks
    _inlineCallbacks(None, g, status)
--- <exception caught here> ---
  File "/root/.synapse/local/lib/python2.7/site-packages/synapse/http/server.py", line 78, in wrapped_request_handler
    yield h(self, request)
  File "/root/.synapse/local/lib/python2.7/site-packages/synapse/http/server.py", line 304, in _async_render
    callback_return = yield callback(request, **kwargs)
  File "/root/.synapse/local/lib/python2.7/site-packages/synapse/federation/transport/server.py", line 227, in new_func
    origin, content, request.args, *args, **kwargs
  File "/root/.synapse/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/root/.synapse/local/lib/python2.7/site-packages/synapse/federation/federation_server.py", line 230, in on_context_state_request
    raise NotImplementedError("Specify an event")
exceptions.NotImplementedError: Specify an event
```


Discussion:
I would like to work on this, can someone help me please.


Resolving patch:
diff --git a/synapse/federation/transport/server.py b/synapse/federation/transport/server.py
--- a/synapse/federation/transport/server.py
+++ b/synapse/federation/transport/server.py
@@ -393,7 +393,7 @@ def on_GET(self, origin, content, query, context):
         return self.handler.on_context_state_request(
             origin,
             context,
-            parse_string_from_args(query, "event_id", None),
+            parse_string_from_args(query, "event_id", None, required=True),
         )
 
 
@@ -404,7 +404,7 @@ def on_GET(self, origin, content, query, room_id):
         return self.handler.on_state_ids_request(
             origin,
             room_id,
-            parse_string_from_args(query, "event_id", None),
+            parse_string_from_args(query, "event_id", None, required=True),
         )
 
 


Test patch:
Repository: matrix-org/synapse

Issue:
remove #4733 debug
Now that #4733 is supposedly fixed, we should remove the debug code for it


Discussion:

Resolving patch:
diff --git a/synapse/replication/tcp/protocol.py b/synapse/replication/tcp/protocol.py
--- a/synapse/replication/tcp/protocol.py
+++ b/synapse/replication/tcp/protocol.py
@@ -52,7 +52,6 @@
 import fcntl
 import logging
 import struct
-import traceback
 from collections import defaultdict
 
 from six import iteritems, iterkeys
@@ -242,7 +241,6 @@ def close(self):
     def send_error(self, error_string, *args):
         """Send an error to remote and close the connection.
         """
-        logger.error("[%s] Sending error: %s", self.id(), error_string % args)
         self.send_command(ErrorCommand(error_string % args))
         self.close()
 
@@ -335,8 +333,6 @@ def stopProducing(self):
         we or the remote has closed the connection)
         """
         logger.info("[%s] Stop producing", self.id())
-        # debug for #4733
-        logger.info("Traceback: %s", "".join(traceback.format_stack()))
         self.on_connection_closed()
 
     def connectionLost(self, reason):


Test patch:
Repository: matrix-org/synapse

Issue:
Doc: Make clear that sharing registration_shared_secret is dangerous
In the config file it reads about `registration_shared_secret`:
> If set, allows registration by anyone who also has the shared secret, even if registration is otherwise disabled.

Especially the last part implies that this is just a secret one can share to users, who want to create an account. It then enables the registration, but only for the user, who has this secret. Also sounds very convenient.

What you do not mention there is one big fact: With that secret, you can **register admin accounts**!
That means that this secret is much more dangerous than one expects from reading the config description! (I also thought it is harmless.)

And, BTW, I am not the only one, who had this idea: https://github.com/vector-im/riot-web/issues/2090

So better adjust the text there, to specifically point that out! (and make sure that no admin shares it)


Discussion:
a pr to reword it would be welcome.
When you point me to the location, that needs to be adjusted... 😃 
@rugk I'm not finding in the code where users can register as admins via the REST interface?  Per your post:
"With that secret, you can register admin accounts" can you tell me how/where they would be able to do such?

And secondarily, is this the desired behavior?
See also the issue I linked. There someone also said one can do this. I don't know whether I tested it, but I'm actually sure it is possible this way.

And, I guess, yes, it is intended. There is this Python script for creating accounts.
> When you point me to the location, that needs to be adjusted... 😃

https://github.com/matrix-org/synapse/blob/d69decd5c78c72abef50b597a689e2bc55a39702/synapse/config/registration.py#L74

and the README probably
Also this issue is a related to #1659 but since it has more info maybe that issue should be closed in favor of this one.

Resolving patch:
diff --git a/synapse/config/registration.py b/synapse/config/registration.py
--- a/synapse/config/registration.py
+++ b/synapse/config/registration.py
@@ -92,8 +92,8 @@ def default_config(self, generate_secrets=False, **kwargs):
         #  - medium: msisdn
         #    pattern: '\\+44'
 
-        # If set, allows registration by anyone who also has the shared
-        # secret, even if registration is otherwise disabled.
+        # If set, allows registration of standard or admin accounts by anyone who
+        # has the shared secret, even if registration is otherwise disabled.
         #
         %(registration_shared_secret)s
 


Test patch:
Repository: matrix-org/synapse

Issue:
KeyError: 'expiry_template_html'
```
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/rav/work/synapse/synapse/app/homeserver.py", line 661, in <module>
    main()
  File "/home/rav/work/synapse/synapse/app/homeserver.py", line 656, in main
    hs = setup(sys.argv[1:])
  File "/home/rav/work/synapse/synapse/app/homeserver.py", line 329, in setup
    config_options,
  File "/home/rav/work/synapse/synapse/config/_base.py", line 362, in load_or_generate_config
    generate_keys=generate_keys,
  File "/home/rav/work/synapse/synapse/config/_base.py", line 408, in read_config_files
    self.parse_config_dict(config)
  File "/home/rav/work/synapse/synapse/config/_base.py", line 411, in parse_config_dict
    self.invoke_all("read_config", config_dict)
  File "/home/rav/work/synapse/synapse/config/_base.py", line 146, in invoke_all
    results.append(getattr(cls, name)(self, *args, **kargs))
  File "/home/rav/work/synapse/synapse/config/emailconfig.py", line 74, in read_config
    self.email_expiry_template_html = email_config["expiry_template_html"]
KeyError: 'expiry_template_html'
```


Discussion:

Resolving patch:
diff --git a/synapse/config/emailconfig.py b/synapse/config/emailconfig.py
--- a/synapse/config/emailconfig.py
+++ b/synapse/config/emailconfig.py
@@ -71,8 +71,12 @@ def read_config(self, config):
             self.email_notif_from = email_config["notif_from"]
             self.email_notif_template_html = email_config["notif_template_html"]
             self.email_notif_template_text = email_config["notif_template_text"]
-            self.email_expiry_template_html = email_config["expiry_template_html"]
-            self.email_expiry_template_text = email_config["expiry_template_text"]
+            self.email_expiry_template_html = email_config.get(
+                "expiry_template_html", "notice_expiry.html",
+            )
+            self.email_expiry_template_text = email_config.get(
+                "expiry_template_text", "notice_expiry.txt",
+            )
 
             template_dir = email_config.get("template_dir")
             # we need an absolute path, because we change directory after starting (and


Test patch:
Repository: matrix-org/synapse

Issue:
Lazy loading: the CS API should provide heroes if room name is ""
### Description
Riot clients remove an existing display name by setting an empty string (`""`).
If that happens on live, matrix clients will be likely to compute a display name like "Bob and 3 others" as specced by the room summary.
But if a matrix client does an initial sync, it will give an "Empty room" name to the room because it receives no heroes.

### Steps to reproduce

On a Riot client:
- create a room with people in there
- set a room display name
- remove it
- Do an initial /sync by clearing the cache.

-> The room will be named "Empty room" instead of  "Bob and 3 others" or stg.









Discussion:
I kinda wish matrix client's set the name state to either empty content or nulled the name rather than an empty string as it's probably a valid thing in itself to have your room title be empty. However, it's probably not worth trying to change that behaviour now...

Resolving patch:
diff --git a/synapse/handlers/sync.py b/synapse/handlers/sync.py
--- a/synapse/handlers/sync.py
+++ b/synapse/handlers/sync.py
@@ -583,19 +583,18 @@ def compute_summary(self, room_id, sync_config, batch, state, now_token):
         )
 
         # if the room has a name or canonical_alias set, we can skip
-        # calculating heroes.  we assume that if the event has contents, it'll
-        # be a valid name or canonical_alias - i.e. we're checking that they
-        # haven't been "deleted" by blatting {} over the top.
+        # calculating heroes. Empty strings are falsey, so we check
+        # for the "name" value and default to an empty string.
         if name_id:
             name = yield self.store.get_event(name_id, allow_none=True)
-            if name and name.content:
+            if name and name.content.get("name"):
                 defer.returnValue(summary)
 
         if canonical_alias_id:
             canonical_alias = yield self.store.get_event(
                 canonical_alias_id, allow_none=True,
             )
-            if canonical_alias and canonical_alias.content:
+            if canonical_alias and canonical_alias.content.get("alias"):
                 defer.returnValue(summary)
 
         joined_user_ids = [


Test patch:
Repository: matrix-org/synapse

Issue:
logout returns M_UNKNOWN_TOKEN instead of M_MISSING_TOKEN
Using `/_matrix/client/r0/logout` without an access token returns the error

`M_UNKNOWN_TOKEN` `Access Token unknown or expired`

while every other unauthorized request, including `/_matrix/client/r0/logout/all` returns

`M_MISSING_TOKEN` `Missing access token.`


Discussion:
This is a synapse bug - I'll move this there.

Resolving patch:
diff --git a/synapse/rest/client/v1/logout.py b/synapse/rest/client/v1/logout.py
--- a/synapse/rest/client/v1/logout.py
+++ b/synapse/rest/client/v1/logout.py
@@ -17,8 +17,6 @@
 
 from twisted.internet import defer
 
-from synapse.api.errors import AuthError
-
 from .base import ClientV1RestServlet, client_path_patterns
 
 logger = logging.getLogger(__name__)
@@ -38,23 +36,16 @@ def on_OPTIONS(self, request):
 
     @defer.inlineCallbacks
     def on_POST(self, request):
-        try:
-            requester = yield self.auth.get_user_by_req(request)
-        except AuthError:
-            # this implies the access token has already been deleted.
-            defer.returnValue((401, {
-                "errcode": "M_UNKNOWN_TOKEN",
-                "error": "Access Token unknown or expired"
-            }))
+        requester = yield self.auth.get_user_by_req(request)
+
+        if requester.device_id is None:
+            # the acccess token wasn't associated with a device.
+            # Just delete the access token
+            access_token = self._auth.get_access_token_from_request(request)
+            yield self._auth_handler.delete_access_token(access_token)
         else:
-            if requester.device_id is None:
-                # the acccess token wasn't associated with a device.
-                # Just delete the access token
-                access_token = self._auth.get_access_token_from_request(request)
-                yield self._auth_handler.delete_access_token(access_token)
-            else:
-                yield self._device_handler.delete_device(
-                    requester.user.to_string(), requester.device_id)
+            yield self._device_handler.delete_device(
+                requester.user.to_string(), requester.device_id)
 
         defer.returnValue((200, {}))
 


Test patch:
Repository: matrix-org/synapse

Issue:
Unexpected error code for invalid param on "/_matrix/client/r0/publicRooms" endpoint
<!-- 

**IF YOU HAVE SUPPORT QUESTIONS ABOUT RUNNING OR CONFIGURING YOUR OWN HOME SERVER**: 
You will likely get better support more quickly if you ask in ** #matrix:matrix.org ** ;)


This is a bug report template. By following the instructions below and
filling out the sections with your information, you will help the us to get all
the necessary data to fix your issue.

You can also preview your report before submitting it. You may remove sections
that aren't relevant to your particular case.
-->

### Description

<!-- Describe here the problem that you are experiencing -->

API returns the error code `M_UNKNOWN` instead of `M_INVALID_PARAM` for an invalid parameter in the request.

### Steps to reproduce

Call _listing public rooms_ endpoint with the invalid type of parameter: string instead of integer for the `limit`.

```sh
curl "https://matrix.org/_matrix/client/r0/publicRooms?limit=NOT_INTEGER"
```
#### Expected Result

```json
{
    "errcode": "M_INVALID_PARAM",
    "error": "Query parameter 'limit' must be an integer"
}
```

#### Actual Result

```json
{
    "errcode": "M_UNKNOWN",
    "error": "Query parameter b'limit' must be an integer"
}
```


<!-- 
Describe how what happens differs from what you expected.

If you can identify any relevant log snippets from _homeserver.log_, please include
those (please be careful to remove any personal or private data). Please surround them with
``` (three backticks, on a line on their own), so that they are formatted legibly.
-->

### Version information

<!-- IMPORTANT: please answer the following questions, to help us narrow down the problem -->

<!-- Was this issue identified on matrix.org or another homeserver? -->
- **Homeserver**:  matrix.og



Discussion:

Resolving patch:
diff --git a/synapse/http/servlet.py b/synapse/http/servlet.py
--- a/synapse/http/servlet.py
+++ b/synapse/http/servlet.py
@@ -55,7 +55,7 @@ def parse_integer_from_args(args, name, default=None, required=False):
             return int(args[name][0])
         except Exception:
             message = "Query parameter %r must be an integer" % (name,)
-            raise SynapseError(400, message)
+            raise SynapseError(400, message, errcode=Codes.INVALID_PARAM)
     else:
         if required:
             message = "Missing integer query parameter %r" % (name,)


Test patch:
Repository: matrix-org/synapse

Issue:
something in the federation sender is dropping logcontexts
Lots of this sort of stuff going on:
```
2019-05-28 10:32:06,362 - synapse.util.metrics - 111 - WARNING - - Context has unexpectedly changed from 'federation_transaction_transmission_loop-6779' to 'sentinel'. ('_send_new_transaction')
2019-05-28 10:32:06,362 - synapse.util.logcontext - 279 - WARNING - - Expected logging context federation_transaction_transmission_loop-6779 has been lost
2019-05-28 10:32:06,365 - synapse.util.metrics - 111 - WARNING - - Context has unexpectedly changed from 'federation_transaction_transmission_loop-6721' to 'sentinel'. ('outbound_request')
```

As with all logcontext drops, this is problematic because (a) it means that logs are hard to trace and (b) metrics will be incorrect


Discussion:

Resolving patch:
diff --git a/synapse/app/_base.py b/synapse/app/_base.py
--- a/synapse/app/_base.py
+++ b/synapse/app/_base.py
@@ -344,15 +344,21 @@ def __init__(self, resolver, max_dns_requests_in_flight):
 
     def resolveHostName(self, resolutionReceiver, hostName, portNumber=0,
                         addressTypes=None, transportSemantics='TCP'):
-        # Note this is happening deep within the reactor, so we don't need to
-        # worry about log contexts.
-
         # We need this function to return `resolutionReceiver` so we do all the
         # actual logic involving deferreds in a separate function.
-        self._resolve(
-            resolutionReceiver, hostName, portNumber,
-            addressTypes, transportSemantics,
-        )
+
+        # even though this is happening within the depths of twisted, we need to drop
+        # our logcontext before starting _resolve, otherwise: (a) _resolve will drop
+        # the logcontext if it returns an incomplete deferred; (b) _resolve will
+        # call the resolutionReceiver *with* a logcontext, which it won't be expecting.
+        with PreserveLoggingContext():
+            self._resolve(
+                resolutionReceiver,
+                hostName,
+                portNumber,
+                addressTypes,
+                transportSemantics,
+            )
 
         return resolutionReceiver
 


Test patch:
Repository: matrix-org/synapse

Issue:
User Directory not regenerating
Originally, our homeserver was federated and able to connect to the matrix network. I decided to remove federation per another Github issue by just whitelisting the local HS for federation. Following this, the `search_all_users` was set to `true` and I performed the psql command `UPDATE user_directory_stream_pos SET stream_id = NULL;` to regenerate the user directory as I wanted the HS to only search local users and forget about all other users it had encountered during federation. However, after a restart of Synapse the user directory still contains entries for users outside of the local installation. How do I clear these entries and force a regeneration of the user directory?


Discussion:
For support please ask in #synapse:matrix.org
For the record they already asked about 5 times in `#matrix:matrix.org` and `#synapse:matrix.org` and @ara4n told them to file an issue. It seems likely that the new user directory in 0.99.3 changed how this works but didn't update the documentation.
Well that's not ideal, right I see, this is from the user_dir docs - fair enough I'll run with it
@drseussofporn You will need to manually rerun the migration, via https://github.com/matrix-org/synapse/blob/master/synapse/storage/schema/delta/53/user_dir_populate.sql , and restart Synapse.
@hawkowl Thank you. That was exactly what I needed.

Resolving patch:
diff --git a/synapse/config/user_directory.py b/synapse/config/user_directory.py
--- a/synapse/config/user_directory.py
+++ b/synapse/config/user_directory.py
@@ -43,9 +43,9 @@ def default_config(self, config_dir_path, server_name, **kwargs):
         #
         # 'search_all_users' defines whether to search all users visible to your HS
         # when searching the user directory, rather than limiting to users visible
-        # in public rooms.  Defaults to false.  If you set it True, you'll have to run
-        # UPDATE user_directory_stream_pos SET stream_id = NULL;
-        # on your database to tell it to rebuild the user_directory search indexes.
+        # in public rooms.  Defaults to false.  If you set it True, you'll have to
+        # rebuild the user_directory search indexes, see
+        # https://github.com/matrix-org/synapse/blob/master/docs/user_directory.md
         #
         #user_directory:
         #  enabled: true


Test patch: