Repository: getsentry/sentry

Issue:
Unable to add GitLab repo. Error Communicating with GitLab (HTTP 422): unknown error
## Important Details

How are you running Sentry?

* [x] On-Premise docker [Version 9.1.1]
* [ ] Saas (sentry.io)
* [ ] Other [briefly describe your environment]

## Description

I successfully make GitLab Integration through the Settings -> ProjectName -> Intergations-> GitLab. But I can't add Repository through the "Add repository" button (Settings -> ProjectName -> Intergations-> GitLab -> Configure).
My user in Sentry is main admin, my user in GitLab is Owner and has permission to target repo.
When I click to "Add repository button" displayed "Saving changes" message and repository does not added. I open console and find this request.

#### Request
- Request URL: http://x.x.x.x:9000/api/0/organizations/groupname/repos/
- Request Method: POST
- Status Code: 400 BAD REQUEST
- POST content: { identifier: 34, installation: "1", provider: "integrations:gitlab" }

#### Response
{"error_type": "validation", "errors": {"__all__": "Error Communicating with GitLab (HTTP 422): unknown error"}}

## Steps to Reproduce
Configured by this manual page: https://docs.sentry.io/workflow/integrations/global-integrations/gitlab/

1. In Sentry, navigate to Organization Settings > Integrations.
2. Within Integrations, find the GitLab icon and click Install.
3. In the resulting modal, click Add Installation.
4. In the pop-up window, complete the instructions to create a Sentry app within GitLab. Once you’re finished, click Next.
5. Fill out the resulting GitLab Configuration form with the following information:
The GitLab URL is the base URL for your GitLab instance. If using gitlab.com, enter https://gitlab.com/.
Find the GitLab Group Path in your group’s GitLab page.
Find the GitLab Application ID and Secret in the Sentry app within GitLab.
Use this information to fill out the GitLab Configuration and click Submit.
6. In the resulting panel, click Authorize.
7. In Sentry, return to Organization Settings > Integrations. You’ll see a new instance of GitLab underneath the list of integrations.
8. Next to your GitLab Instance, click Configure. Note: It’s important to configure to receive the full benefits of commit tracking.
9. On the resulting page, click Add Repository to select which repositories in which you’d like to begin tracking commits.

### What you expected to happen

Excepted selected repository appears in repository list on this page, but nothing happens.

### Error in "internal"

In "internal" project I received this errors:

**TypeError - Cannot read property 'errors' of undefined**

```
./app/actionCreators/integrations.jsx at line 108:43
        repo: repo.name,
      });
      addSuccessMessage(message);
    },
    errorData => {
      const text = errorData.responseJSON.errors
        ? errorData.responseJSON.errors.__all__
        : t('Unable to add repository.');
      addErrorMessage(text);
    }
  );
```

### Versions

OS: Ubuntu 18.04.2 LTS
Docker Server Engine: 18.09.2
Docker Server API: 1.39
Docker Client: 18.09.2
Docker Client API: 1.39 
Sentry: 9.1.1

### Possible Solution

May be it needed to fix frontend TypeError error and make some backend requests after it.


Discussion:
GitLab will return a 422 when it cannot validate the certificate of your sentry instance or if your sentry instance does not have a publicly reachable domain name/IP address.
@markstory My GitLab instance and Sentry instance both located in one local network and sentry instance does not have a publicly reachable domain name/IP address. Sentry instance available at http (not htttps). How can I connect them?
Hello,
I have exactly the same problem. Small differences :
- Sentry ( sentry.xxx.com ) and Gitlab ( gitlab.xxx.com ) are reachable through https only.
- They are on the same server.
- They are publicly accessible through internet.
  - On the internet, these addresses resolves to a public IP address.
  - On our local network, due to limitations of our (great) router, we can't use our public IP address, so we have a DNS local resolver which resolves sentry.xxx.com as 192.168.0.199 and gitlab.xxx.com as 192.168.0.199. Sadly, we don't have any alternative.
@netandreus @husim0 You'll have to add some additional logging [here](https://github.com/getsentry/sentry/blob/4cea625cc989b53742f0f8a39cbb645e09853667/src/sentry/integrations/gitlab/client.py#L92-L118) to get the full response payload from GitLab. I'm not going to be of much more help in solving this issue as I can't see your networks or anything running on them.
@husim0 @markstory I try to investigate this issue. First of all I run
`tail -f /var/log/gitlab/gitlab-rails/api_json.log`
on my GitLab VM to see what happens.
I found this this log entry after try to add repo in sentry ui:
```
{
  "time": "2019-05-08T07:05:56.332Z",
  "severity": "INFO",
  "duration": 7.38,
  "db": 1.05,
  "view": 6.33,
  "status": 422,
  "method": "POST",
  "path": "/api/v4/projects/34/hooks",
  "params": [
    {
      "key": "url",
      "value": "http://x.x.x.x:9000/extensions/gitlab/webhook/"
    },
    {
      "key": "token",
      "value": "xxxxxxxxxxxx"
    },
    {
      "key": "push_events",
      "value": null
    },
    {
      "key": "enable_ssl_verification",
      "value": null
    },
    {
      "key": "merge_requests_events",
      "value": null
    }
  ],
  "host": "git.my.domain",
  "ip": "192.168.11.30, 127.0.0.1",
  "ua": "sentry/9.1.1 (https://sentry.io)",
  "route": "/api/:version/projects/:id/hooks",
  "user_id": 1,
  "username": "sentry",
  "queue_duration": 2.6,
  "gitaly_calls": 0,
  "gitaly_duration": 0,
  "correlation_id": "xxxxxxxxxx"
}
```
I try to reproduce this API request through GitLab web UI and  and saw there "Url is blocked: Requests to the local network are not allowed." error message.

I'm trying to google this error and found this issue https://gitlab.com/gitlab-org/gitlab-ce/issues/45134
Yeah, there is GitLab option that allows all local webhooks requests and it's disable by default.

**Finally solution** is to enable this option in GitLab:
GitLab -> Admim area -> Settings -> Network -> Outbound requests -> Allow requests to the local network from hooks and services

@markstory Should this ticket be closed?  I see a note was added to the docs regarding a specific Gitlab setting, but isn't this original error report highlighting lack of proper error handling on the add repository screen?  I don't see where this error is being handled:
```
TypeError - Cannot read property 'errors' of undefined
```

@bdw429s Here is link to the docs. https://github.com/getsentry/sentry-docs/pull/972 Status: Review required
About Type error - may be this is just another error.
@netandreus Either you didn't understand my post, or I don't understand yours.  

I'm pointing out that the "typeerror" itself appears to be a bug in Sentry's handling of failures while adding a repository and Sentry needs to fix this bug.  It's great that we updated the docs, but that is pointless if all we did was add a note somewhere to help someone figure out when Sentry's error handling fails to perform, when we should have fixed the error handling in the first place so the user actually got a useful message in their browser as opposed to the current behavior of a spinner popping up for a few seconds and then... nothing.  I stared at my screen for quite some time with zero feedback from Sentry before finding this thread.  So yes, it's "just another error" and "let's actually fix it" and not just throw a note in the docs and call it a day :)

IMO, this ticket isn't closed until Sentry properly handles situations in which the Gitlab calls fail (or more broadly, whatever the backend Repo endpoint is).  In fact, I would suggest a two pronged approach to fixing this:

1. Fix the type error so when an error happens while adding a repo, the user sees some sort of actual feedback on the screen such as _"There was an error adding your repository, please check the XYZ log"_
2. Improve the Gitlab integration (and others as necessary) so a more useful message comes back other than just the status code.  The Ajax response simply says things like "403 forbidden", but I assume the actual Gitlab API sent back more than that.  Let's show it to the user!
@bdw429s I totally agree with you. This is another error, and it is not normal that the processing of responses from GitLab does not work properly. Because of this, other errors related to GtiLab can also be hidden.
Glad we're on the same page.  @markstory can this ticket be reopened until such a time as the original error reported is resolved?
We can make the 422 scenario better. I agree it would be good to bubble that error up the the frontend.

Resolving patch:
diff --git a/src/sentry/integrations/gitlab/integration.py b/src/sentry/integrations/gitlab/integration.py
--- a/src/sentry/integrations/gitlab/integration.py
+++ b/src/sentry/integrations/gitlab/integration.py
@@ -103,6 +103,19 @@ def search_issues(self, project_id, query, iids):
         client = self.get_client()
         return client.search_project_issues(project_id, query, iids)
 
+    def error_message_from_json(self, data):
+        """
+        Extract error messages from gitlab API errors.
+        Generic errors come in the `error` key while validation errors
+        are generally in `message`.
+
+        See https://docs.gitlab.com/ee/api/#data-validation-and-error-reporting
+        """
+        if 'message' in data:
+            return data['message']
+        if 'error' in data:
+            return data['error']
+
 
 class InstallationForm(forms.Form):
     url = forms.CharField(


Test patch:
diff --git a/tests/sentry/integrations/gitlab/test_repository.py b/tests/sentry/integrations/gitlab/test_repository.py
--- a/tests/sentry/integrations/gitlab/test_repository.py
+++ b/tests/sentry/integrations/gitlab/test_repository.py
@@ -134,6 +134,30 @@ def request_callback(request):
         assert response.status_code == 201
         self.assert_repository(self.default_repository_config)
 
+    @responses.activate
+    def test_create_repository_request_invalid_url(self):
+        responses.add(
+            responses.GET,
+            u'https://example.gitlab.com/api/v4/projects/%s' % self.gitlab_id,
+            status=200,
+            json=self.default_repository_config
+        )
+        responses.add(
+            responses.POST,
+            u'https://example.gitlab.com/api/v4/projects/%s/hooks' % self.gitlab_id,
+            status=422,
+            json={'error': 'Invalid url given'}
+        )
+        response = self.create_repository(
+            self.default_repository_config,
+            self.integration.id,
+            add_responses=False)
+        assert response.status_code == 400
+        self.assert_error_message(
+            response,
+            'validation',
+            'Error Communicating with GitLab (HTTP 422): Invalid url given')
+
     def test_create_repository_data_no_installation_id(self):
         response = self.create_repository(self.default_repository_config, None)
         assert response.status_code == 400