Error synchronising users: Problem communicating with Crowd
Asked by Igor Gali?:

From <https://support.atlassian.com/browse/FSH-7575> which I was unable to cross-post here at the same time in my first attempt.

In trying to secure our installation I changed all services to listen 127.0.0.1. This, of course, broke the authentication.  
I followed <http://confluence.atlassian.com/display/JIRA044/Integrating+JIRA+with+FishEye> to re-establish the connection using the external URIs only so the applications have a clean interface to talk to. When trying to re-sync the users in FishEye however, I get the following error:

```
2011-12-18 18:01:31,601 WARN  - Error synchronising users: com.cenqua.fisheye.user.AuthenticationException: Problem communicating with Crowd	at com.cenqua.fisheye.user.crowd.CrowdAuth.listUserNames(CrowdAuth.java:487)	at com.cenqua.fisheye.user.DefaultUserManager.synchroniseUsers(DefaultUserManager.java:678)	at com.cenqua.fisheye.user.DefaultUserManager.synchroniseUsers(DefaultUserManager.java:653)	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)	at java.lang.reflect.Method.invoke(Method.java:597)	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)	at com.googlecode.ehcache.annotations.interceptor.EhCacheInterceptor.invokeTriggersRemove(EhCacheInterceptor.java:191)	at com.googlecode.ehcache.annotations.interceptor.EhCacheInterceptor.invoke(EhCacheInterceptor.java:90)	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)	at $Proxy36.synchroniseUsers(Unknown Source)	at com.cenqua.fisheye.web.admin.actions.sec.ReSyncAuthAction.execute(ReSyncAuthAction.java:13)	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)	at java.lang.reflect.Method.invoke(Method.java:597)	at com.opensymphony.xwork.DefaultActionInvocation.invokeAction(DefaultActionInvocation.java:358)	at com.opensymphony.xwork.DefaultActionInvocation.invokeActionOnly(DefaultActionInvocation.java:218)	at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:192)	at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:31)	at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:190)	at com.cenqua.fisheye.web.admin.interceptors.LoginInterceptor.intercept(LoginInterceptor.java:33)	at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:190)	at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:31)	at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:190)	at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:31)	at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:190)	at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:31)	at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:190)	at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:31)	at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:190)	at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:31)	at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:190)	at com.opensymphony.xwork.interceptor.ExceptionMappingInterceptor.intercept(ExceptionMappingInterceptor.java:186)	at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:190)	at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:31)	at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:190)	at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:31)	at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:190)	at com.opensymphony.xwork.DefaultActionProxy.execute(DefaultActionProxy.java:116)	at com.opensymphony.webwork.dispatcher.DispatcherUtils.serviceAction(DispatcherUtils.java:273)	at com.opensymphony.webwork.dispatcher.ServletDispatcher.service(ServletDispatcher.java:111)	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)	at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1272)	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)	at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:25)	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)	at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1264)	at com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:118)	at com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:54)	at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1264)	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)	at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1264)	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:83)	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)	at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1264)	at com.atlassian.crucible.filters.CrucibleFilter.doFilter(CrucibleFilter.java:146)	at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1264)	at com.cenqua.fisheye.web.filters.TotalityFilter.doFilter(TotalityFilter.java:278)	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)	at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1264)	at com.atlassian.security.auth.trustedapps.filter.TrustedApplicationsFilter.doFilter(TrustedApplicationsFilter.java:98)	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)	at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1264)	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)	at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:71)	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)	at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1264)	at org.mortbay.servlet.UserAgentFilter.doFilter(UserAgentFilter.java:78)	at org.mortbay.servlet.GzipFilter.doFilter(GzipFilter.java:131)	at com.cenqua.fisheye.web.filters.CustomIncludableGzipFilter.doFilter(CustomIncludableGzipFilter.java:27)	at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1264)	at com.cenqua.fisheye.web.filters.ProductInfoFilter.doFilter(ProductInfoFilter.java:44)	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)	at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1264)	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)	at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1264)	at com.cenqua.fisheye.web.filters.UpfrontFilter.doFilter(UpfrontFilter.java:54)	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)	at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1264)	at com.atlassian.fecru.profiling.ProfilingServletFilter.doFilter(ProfilingServletFilter.java:73)	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)	at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1264)	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)	at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)	at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)	at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)	at org.mortbay.jetty.Server.handle(Server.java:326)	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)	at org.mortbay.jetty.ajp.Ajp13Connection.access$2900(Ajp13Connection.java:42)	at org.mortbay.jetty.ajp.Ajp13Connection$RequestHandler.headerComplete(Ajp13Connection.java:228)	at org.mortbay.jetty.ajp.Ajp13Parser.parseNext(Ajp13Parser.java:498)	at org.mortbay.jetty.ajp.Ajp13Parser.parseAvailable(Ajp13Parser.java:151)	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)	at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)	at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:451)Caused by: com.atlassian.crowd.exception.InvalidAuthenticationException: Application failed to authenticate	at com.atlassian.crowd.integration.rest.service.RestExecutor$MethodExecutor.throwError(RestExecutor.java:418)	at com.atlassian.crowd.integration.rest.service.RestExecutor$MethodExecutor.andReceive(RestExecutor.java:300)	at com.atlassian.crowd.integration.rest.service.RestCrowdClient.searchUserNames(RestCrowdClient.java:479)	at com.cenqua.fisheye.user.crowd.CrowdAuth$9.call(CrowdAuth.java:483)	at com.cenqua.fisheye.user.crowd.CrowdAuth$9.call(CrowdAuth.java:467)	at com.cenqua.fisheye.user.crowd.CrowdAuth.withTimer(CrowdAuth.java:98)	at com.cenqua.fisheye.user.crowd.CrowdAuth.listUserNames(CrowdAuth.java:467)	... 121 more
```

Or, in essence:

```
2011-12-18 18:01:31,601 WARN  - Error synchronising users: com.cenqua.fisheye.user.AuthenticationException: Problem communicating with CrowdCaused by: com.atlassian.crowd.exception.InvalidAuthenticationException: Application failed to authenticate
```


Answers:
author: rpamfil
body: Same issue here, and the root cause was also the fact that the IP was not whitelisted. In our case, the servers were on the same machine, but there was no 127.0.0.1 alias for [jira.ourdomain.com](http://jira.ourdomain.com) in /etc/hosts, so it was going through the external IP address.


author: rj
body: I had a very similar problem after migrating our Jira and Fisheye servers which involved changing their IP addresses. Jira version 5.0.6; Fisheye version 2.7.13.

In addition to the steps above you may need to check Administration--\>Users--\>Jira User Server on your Jira installation. The IP address for your Fisheye installation (and maybe others) will probably need changing. Also check that the application name and password match up between this screen on Jira and what you specify when configuring authentication via Jira on Fisheye.

FWIW, the logging for this probelm is abysmal. Instead of a helpful "IP address is not allowed, have you checked XYZ" there's just a lot of technobabble.


author: Thomas Keller
body: Another potential pitfall here is btw. if the JIRA instance that serves as user directory is wrongly configued as "Crowd Instance" in Fisheye. Changing that made my login working again.


author: Igor Gali?
body: It appears that it's actually necessary to specify IP addresses - these are whitelists, and they are not optional.


author: Colin Goudie
body: My guess with that error is that that the application password in the fisheye's crowd setup is incorrect or the password was possibly changed in Crowd?

comments: author: Igor Gali?
body: What is that password supposed to be? Where do I set it in Jira?

When establishing the Link in Jira, I did this with the user "admin" and its password. When verifying that connection in Fisheye, there is no application user, however, only an application password.


author: Colin Goudie
body: It appears to be fisheye's application password isn't matching the application setup in Crowd so not JIRA related as such. Check out Crowd and go to Applications tab. Under there there should be the crowd application. The password setup there (which won't be displayed) needs to match what is in Fisheye Admin-\>Security Settings-\>Authentication under the JIRA/Crowd settings.

Just to check I'm not confusing you, are you using Crowd or connecting Fisheye direct to JIRA for User admin?


author: Igor Gali?
body: Okay, done that, and the error I'm getting from Fisheye is now different:

```
2011-12-19 03:55:42,952 WARN  - Error synchronising users: com.cenqua.fisheye.user.AuthenticationException: Problem communicating with Crow[snip]Caused by: com.atlassian.crowd.exception.ApplicationPermissionException: &lt;html&gt;&lt;head&gt;&lt;title&gt;Apache Tomcat/6.0.32 - Error report&lt;/title&gt;&lt;style&gt;&lt;!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--&gt;&lt;/style&gt; &lt;/head&gt;&lt;body&gt;&lt;h1&gt;HTTP Status 403 - client.forbidden.exception&lt;/h1&gt;&lt;HR size="1" noshade="noshade"&gt;&lt;p&gt;&lt;b&gt;type&lt;/b&gt; Status report&lt;/p&gt;&lt;p&gt;&lt;b&gt;message&lt;/b&gt; &lt;u&gt;client.forbidden.exception&lt;/u&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;description&lt;/b&gt; &lt;u&gt;Access to the specified resource (client.forbidden.exception) has been forbidden.&lt;/u&gt;&lt;/p&gt;&lt;HR size="1" noshade="noshade"&gt;&lt;h3&gt;Apache Tomcat/6.0.32&lt;/h3&gt;&lt;/body&gt;&lt;/html&gt;        at com.atlassian.crowd.integration.rest.service.RestExecutor$MethodExecutor.throwError(RestExecutor.java:414)        at com.atlassian.crowd.integration.rest.service.RestExecutor$MethodExecutor.andReceive(RestExecutor.java:300)        at com.atlassian.crowd.integration.rest.service.RestCrowdClient.searchUserNames(RestCrowdClient.java:479)        at com.cenqua.fisheye.user.crowd.CrowdAuth$9.call(CrowdAuth.java:483)        at com.cenqua.fisheye.user.crowd.CrowdAuth$9.call(CrowdAuth.java:467)        at com.cenqua.fisheye.user.crowd.CrowdAuth.withTimer(CrowdAuth.java:98)        at com.cenqua.fisheye.user.crowd.CrowdAuth.listUserNames(CrowdAuth.java:467)        ... 121 more
```


Tags: authentication

fisheye-crucible