Inital import of repository to Crucible using light Fisheye fails with java.util.ConcurrentModificationException
Asked by Fesifaz Ino:

I'm having problems trying to do the initial import of my perforce depot into Crucible using light Fisheye. (This is version fecru-2.10.3.) It's a very large repository, so I'm starting only a few months back and using hundreds of exclude patterns to get it down to a manageable size. (Without the exclude patterns, I run out of memory before the import finishes on my machine with 16GB of RAM.)  

After Crucible using light Fisheye finally downloads all the file names, I get this error in the logs:  

<br />

<br />

2013-04-11 21:13:14,112 ERROR \[InitialPinger1 try11\] fisheye BaseRepositoryScanner-handleSlurpException - Problem processing revisions from repo try11 due to class java.util.ConcurrentModificationException - null  
java.util.ConcurrentModificationException  
at java.util.TreeMap$PrivateEntryIterator.nextEntry(TreeMap.java:1117)  
at java.util.TreeMap$ValueIterator.next(TreeMap.java:1162)  
at com.cenqua.fisheye.perforce.client.P4Client.addFileInfo(P4Client.java:281)  
at com.cenqua.fisheye.perforce.client.P4Client.addChangeFileInfo(P4Client.java:197)  
at com.cenqua.fisheye.perforce.P4Scanner.createInitialImport(P4Scanner.java:851)  
at com.cenqua.fisheye.rep.RepositoryScanner.processRevisions(RepositoryScanner.java:132)  
at com.cenqua.fisheye.rep.BaseRepositoryScanner.slurpRepository(BaseRepositoryScanner.java:254)  
at com.cenqua.fisheye.rep.BaseRepositoryScanner.doSlurpTransaction(BaseRepositoryScanner.java:221)  
at com.cenqua.fisheye.rep.BaseRepositoryScanner.ping(BaseRepositoryScanner.java:180)  
at com.cenqua.fisheye.rep.BaseRepositoryEngine.doSlurp(BaseRepositoryEngine.java:92)  
at com.cenqua.fisheye.rep.RepositoryEngine.slurp(RepositoryEngine.java:382)  
at com.cenqua.fisheye.rep.ping.OneOffPingRequest.doRequest(OneOffPingRequest.java:28)  
at com.cenqua.fisheye.rep.ping.PingRequest.process(PingRequest.java:58)  
at com.cenqua.fisheye.rep.RepositoryHandle.processPingRequests(RepositoryHandle.java:198)  
at com.cenqua.fisheye.rep.RepositoryHandle.access$100(RepositoryHandle.java:50)  
at com.cenqua.fisheye.rep.RepositoryHandle$2.run(RepositoryHandle.java:156)  
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)  
at java.lang.Thread.run(Thread.java:636)  

<br />

<br />

And then Crucible using light Fisheye starts the import all over again from the beginning.

What's going on? And, more importantly, how can I get past this?


Answers:
author: Fesifaz Ino
body: Hopefully this comment appears under Eddie Webb's comment. There's no link to comment on a comment.

Addressing your comment back to front:

I don't know if something is seriously wrong. I do know that it's a very big repository. Before I used the hundreds of exclude patterns, I kept getting errors like

2013-04-07 02:16:02,769 ERROR \[InitialPinger1 try11\] fisheye BaseRepositoryScanner-handleSlurpException - Problem processing revisions from repo try11 due to class java.lang.OutOfMemoryError -

Java heap space

java.lang.OutOfMemoryError: Java heap space

which would abort the initial import. I used FISHEYE_OPTS to gradually increase the Java heap, eventually getting to "-Xmx12288m" before I gave up. I logged all the perforce commands, and I could tell that it got further and further into the repository as I increased the memory.

With the exclude patterns, though, I have FISHEYE_OPTS set to "-Xmx4096m" and things go fine until I get the java.util.ConcurrentModificationException.

As to the number of threads I use for initial scan: I never set it. I assume it was whatever the default was. I just checked <http://localhost:8060/admin/viewServerSettings.do> and it said under Resource Limits that Initial Indexing Threads was not set and Incremental Indexing Threads was not set. I just changed them both to 1 and restarted the server. It's trying to do the initial import again. We can see what happens in another 8 hours or so.

Some other data from <http://localhost:8060/admin/sysinfo.do> :

Resource Limits - initial threads: 1 (max)

Resource Limits - incremental threads: 1 (max)

Database Type: MySQL

Database Driver: com.mysql.jdbc.Driver

Database Url: jdbc:mysql://localhost:3306/fisheye

Database Version: 80

JDBC Pool (min): 5

JDBC Pool (max): 20

JDBC Pool (effective max): 18

JDBC Pool (partitions): 3

JDBC Pool (max per partition): 6


author: Conor
body: This looks like a bug so please raise a support request at [support.atlassian.com](http://support.atlassian.com)

comments: author: Fesifaz Ino
body: I raised support request CRC-5275 . It's not publickly viewable at this time, though.


author: EddieW
body: How many threads do you use for initial scan? What is the FECRU heap set to? There is no way a scan would consume 16 gigs unless something is seriously wrong.


Tags: crucible-without-fisheye

perforce