Repository: mockito/mockito
Issue: Deadlock after upgrading to Mockito 2 and parallel tests execution

I updated Mockito and ScalaTest today from ScalaTest 2.x and Mockito 1.x and my tests suddenly froze in SBT (not in IntelliJ). After some playing around, I noticed every test class by itself passes and when I disabled running tests in parallel in SBT, the tests would work again.

I used jstack and came across this deadlock:
```
Found one Java-level deadlock:
=============================
"pool-9-thread-10-ScalaTest-running-ApiKeyVerificationFilterSpec":
  waiting to lock monitor 0x00007fea6f039538 (object 0x00000007959e4a78, a sbt.classpath.ClasspathUtilities$$anon$1),
  which is held by "pool-9-thread-2-ScalaTest-running-BRequestFilterSpec"
"pool-9-thread-2-ScalaTest-running-BRequestFilterSpec":
  waiting to lock monitor 0x00007fea70e95548 (object 0x0000000795a868e8, a sbt.classpath.ClasspathFilter),
  which is held by "pool-9-thread-10-ScalaTest-running-ApiKeyVerificationFilterSpec"

Java stack information for the threads listed above:
===================================================
"pool-9-thread-10-ScalaTest-running-ApiKeyVerificationFilterSpec":
	at java.lang.ClassLoader.loadClass(ClassLoader.java:404)
	- waiting to lock <0x00000007959e4a78> (a sbt.classpath.ClasspathUtilities$$anon$1)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
	- locked <0x0000000795a868e8> (a sbt.classpath.ClasspathFilter)
	at sbt.classpath.ClasspathFilter.loadClass(ClassLoaders.scala:59)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at net.bytebuddy.dynamic.loading.MultipleParentClassLoader.loadClass(MultipleParentClassLoader.java:68)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at sun.reflect.GeneratedSerializationConstructorAccessor35.newInstance(Unknown Source)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
	at org.objenesis.instantiator.sun.SunReflectionFactoryInstantiator.newInstance(SunReflectionFactoryInstantiator.java:48)
	at org.objenesis.ObjenesisBase.newInstance(ObjenesisBase.java:73)
	at org.mockito.internal.creation.instance.ObjenesisInstantiator.newInstance(ObjenesisInstantiator.java:18)
	at org.mockito.internal.creation.bytebuddy.SubclassByteBuddyMockMaker.createMock(SubclassByteBuddyMockMaker.java:47)
	at org.mockito.internal.creation.bytebuddy.ByteBuddyMockMaker.createMock(ByteBuddyMockMaker.java:25)
	at org.mockito.internal.util.MockUtil.createMock(MockUtil.java:35)
	at org.mockito.internal.MockitoCore.mock(MockitoCore.java:63)
	at org.mockito.Mockito.mock(Mockito.java:1729)
	at org.mockito.Mockito.mock(Mockito.java:1642)
	at org.scalatest.mockito.MockitoSugar$class.mock(MockitoSugar.scala:73)
	at com.b.n.filter.ApiKeyVerificationFilterSpec.mock(ApiKeyVerificationFilterSpec.scala:25)
	at com.b.n.filter.ApiKeyVerificationFilterSpec$ApiKeyFetchSetup$class.$init$(ApiKeyVerificationFilterSpec.scala:77)
	at com.b.n.filter.ApiKeyVerificationFilterSpec$$anonfun$15$$anon$11.<init>(ApiKeyVerificationFilterSpec.scala:80)
	at com.b.n.filter.ApiKeyVerificationFilterSpec$$anonfun$15.apply(ApiKeyVerificationFilterSpec.scala:80)
	at com.b.n.filter.ApiKeyVerificationFilterSpec$$anonfun$15.apply(ApiKeyVerificationFilterSpec.scala:80)
	at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
	at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
	at org.scalatest.Transformer.apply(Transformer.scala:22)
	at org.scalatest.Transformer.apply(Transformer.scala:20)
	at org.scalatest.FunSpecLike$$anon$1.apply(FunSpecLike.scala:454)
	at org.scalatest.TestSuite$class.withFixture(TestSuite.scala:196)
	at org.scalatest.FunSpec.withFixture(FunSpec.scala:1630)
	at org.scalatest.FunSpecLike$class.invokeWithFixture$1(FunSpecLike.scala:451)
	at org.scalatest.FunSpecLike$$anonfun$runTest$1.apply(FunSpecLike.scala:464)
	at org.scalatest.FunSpecLike$$anonfun$runTest$1.apply(FunSpecLike.scala:464)
	at org.scalatest.SuperEngine.runTestImpl(Engine.scala:289)
	at org.scalatest.FunSpecLike$class.runTest(FunSpecLike.scala:464)
	at com.b.n.filter.ApiKeyVerificationFilterSpec.org$scalatest$BeforeAndAfter$$super$runTest(ApiKeyVerificationFilterSpec.scala:25)
	at org.scalatest.BeforeAndAfter$class.runTest(BeforeAndAfter.scala:203)
	at com.b.n.filter.ApiKeyVerificationFilterSpec.runTest(ApiKeyVerificationFilterSpec.scala:25)
	at org.scalatest.FunSpecLike$$anonfun$runTests$1.apply(FunSpecLike.scala:497)
	at org.scalatest.FunSpecLike$$anonfun$runTests$1.apply(FunSpecLike.scala:497)
	at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:396)
	at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:384)
	at scala.collection.immutable.List.foreach(List.scala:381)
	at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:384)
	at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:379)
	at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:461)
	at org.scalatest.FunSpecLike$class.runTests(FunSpecLike.scala:497)
	at org.scalatest.FunSpec.runTests(FunSpec.scala:1630)
	at org.scalatest.Suite$class.run(Suite.scala:1147)
	at org.scalatest.FunSpec.org$scalatest$FunSpecLike$$super$run(FunSpec.scala:1630)
	at org.scalatest.FunSpecLike$$anonfun$run$1.apply(FunSpecLike.scala:501)
	at org.scalatest.FunSpecLike$$anonfun$run$1.apply(FunSpecLike.scala:501)
	at org.scalatest.SuperEngine.runImpl(Engine.scala:521)
	at org.scalatest.FunSpecLike$class.run(FunSpecLike.scala:501)
	at com.b.n.filter.ApiKeyVerificationFilterSpec.org$scalatest$BeforeAndAfter$$super$run(ApiKeyVerificationFilterSpec.scala:25)
	at org.scalatest.BeforeAndAfter$class.run(BeforeAndAfter.scala:258)
	at com.b.n.filter.ApiKeyVerificationFilterSpec.run(ApiKeyVerificationFilterSpec.scala:25)
	at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:314)
	at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:480)
	at sbt.TestRunner.runTest$1(TestFramework.scala:76)
	at sbt.TestRunner.run(TestFramework.scala:85)
	at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
	at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
	at sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:185)
	at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
	at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
	at sbt.TestFunction.apply(TestFramework.scala:207)
	at sbt.Tests$$anonfun$9.apply(Tests.scala:216)
	at sbt.Tests$$anonfun$9.apply(Tests.scala:216)
	at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
	at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
	at sbt.std.Transform$$anon$4.work(System.scala:63)
	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
	at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
	at sbt.Execute.work(Execute.scala:237)
	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
	at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
	at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
"pool-9-thread-2-ScalaTest-running-BRequestFilterSpec":
	at java.lang.ClassLoader.loadClass(ClassLoader.java:404)
	- waiting to lock <0x0000000795a868e8> (a sbt.classpath.ClasspathFilter)
	at sbt.classpath.ClasspathFilter.loadClass(ClassLoaders.scala:59)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at net.bytebuddy.dynamic.loading.MultipleParentClassLoader.loadClass(MultipleParentClassLoader.java:68)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
	at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at net.bytebuddy.dynamic.loading.ClassInjector$UsingReflection$Dispatcher$Direct.defineClass(ClassInjector.java:408)
	at net.bytebuddy.dynamic.loading.ClassInjector$UsingReflection.inject(ClassInjector.java:185)
	- locked <0x0000000781ee0ff0> (a net.bytebuddy.dynamic.loading.MultipleParentClassLoader)
	at net.bytebuddy.dynamic.loading.ClassLoadingStrategy$Default$InjectionDispatcher.load(ClassLoadingStrategy.java:187)
	at net.bytebuddy.dynamic.TypeResolutionStrategy$Passive.initialize(TypeResolutionStrategy.java:79)
	at net.bytebuddy.dynamic.DynamicType$Default$Unloaded.load(DynamicType.java:4376)
	at org.mockito.internal.creation.bytebuddy.SubclassBytecodeGenerator.mockClass(SubclassBytecodeGenerator.java:94)
	at org.mockito.internal.creation.bytebuddy.TypeCachingBytecodeGenerator$1.call(TypeCachingBytecodeGenerator.java:37)
	at org.mockito.internal.creation.bytebuddy.TypeCachingBytecodeGenerator$1.call(TypeCachingBytecodeGenerator.java:34)
	at net.bytebuddy.TypeCache.findOrInsert(TypeCache.java:138)
	at net.bytebuddy.TypeCache$WithInlineExpunction.findOrInsert(TypeCache.java:346)
	at net.bytebuddy.TypeCache.findOrInsert(TypeCache.java:161)
	- locked <0x00000007959e4a78> (a sbt.classpath.ClasspathUtilities$$anon$1)
	at net.bytebuddy.TypeCache$WithInlineExpunction.findOrInsert(TypeCache.java:355)
	at org.mockito.internal.creation.bytebuddy.TypeCachingBytecodeGenerator.mockClass(TypeCachingBytecodeGenerator.java:32)
	at org.mockito.internal.creation.bytebuddy.SubclassByteBuddyMockMaker.createMockType(SubclassByteBuddyMockMaker.java:71)
	at org.mockito.internal.creation.bytebuddy.SubclassByteBuddyMockMaker.createMock(SubclassByteBuddyMockMaker.java:42)
	at org.mockito.internal.creation.bytebuddy.ByteBuddyMockMaker.createMock(ByteBuddyMockMaker.java:25)
	at org.mockito.internal.util.MockUtil.createMock(MockUtil.java:35)
	at org.mockito.internal.MockitoCore.mock(MockitoCore.java:63)
	at org.mockito.Mockito.mock(Mockito.java:1729)
	at org.mockito.Mockito.mock(Mockito.java:1642)
	at org.scalatest.mockito.MockitoSugar$class.mock(MockitoSugar.scala:73)
	at com.b.n.filter.BRequestFilterSpec.mock(BRequestFilterSpec.scala:18)
	at com.b.n.filter.BRequestFilterSpec$Base$class.$init$(BRequestFilterSpec.scala:31)
	at com.b.n.filter.BRequestFilterSpec$$anonfun$7$$anon$10.<init>(BRequestFilterSpec.scala:43)
	at com.b.n.filter.BRequestFilterSpec$$anonfun$7.apply(BRequestFilterSpec.scala:43)
	at com.b.n.filter.BRequestFilterSpec$$anonfun$7.apply(BRequestFilterSpec.scala:43)
	at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
	at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
	at org.scalatest.Transformer.apply(Transformer.scala:22)
	at org.scalatest.Transformer.apply(Transformer.scala:20)
	at org.scalatest.FunSpecLike$$anon$1.apply(FunSpecLike.scala:454)
	at org.scalatest.TestSuite$class.withFixture(TestSuite.scala:196)
	at org.scalatest.FunSpec.withFixture(FunSpec.scala:1630)
	at org.scalatest.FunSpecLike$class.invokeWithFixture$1(FunSpecLike.scala:451)
	at org.scalatest.FunSpecLike$$anonfun$runTest$1.apply(FunSpecLike.scala:464)
	at org.scalatest.FunSpecLike$$anonfun$runTest$1.apply(FunSpecLike.scala:464)
	at org.scalatest.SuperEngine.runTestImpl(Engine.scala:289)
	at org.scalatest.FunSpecLike$class.runTest(FunSpecLike.scala:464)
	at com.b.n.filter.BRequestFilterSpec.org$scalatest$BeforeAndAfter$$super$runTest(BRequestFilterSpec.scala:18)
	at org.scalatest.BeforeAndAfter$class.runTest(BeforeAndAfter.scala:203)
	at com.b.n.filter.BRequestFilterSpec.runTest(BRequestFilterSpec.scala:18)
	at org.scalatest.FunSpecLike$$anonfun$runTests$1.apply(FunSpecLike.scala:497)
	at org.scalatest.FunSpecLike$$anonfun$runTests$1.apply(FunSpecLike.scala:497)
	at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:396)
	at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:384)
	at scala.collection.immutable.List.foreach(List.scala:381)
	at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:384)
	at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:379)
	at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:461)
	at org.scalatest.FunSpecLike$class.runTests(FunSpecLike.scala:497)
	at org.scalatest.FunSpec.runTests(FunSpec.scala:1630)
	at org.scalatest.Suite$class.run(Suite.scala:1147)
	at org.scalatest.FunSpec.org$scalatest$FunSpecLike$$super$run(FunSpec.scala:1630)
	at org.scalatest.FunSpecLike$$anonfun$run$1.apply(FunSpecLike.scala:501)
	at org.scalatest.FunSpecLike$$anonfun$run$1.apply(FunSpecLike.scala:501)
	at org.scalatest.SuperEngine.runImpl(Engine.scala:521)
	at org.scalatest.FunSpecLike$class.run(FunSpecLike.scala:501)
	at com.b.n.filter.BRequestFilterSpec.org$scalatest$BeforeAndAfter$$super$run(BRequestFilterSpec.scala:18)
	at org.scalatest.BeforeAndAfter$class.run(BeforeAndAfter.scala:258)
	at com.b.n.filter.BRequestFilterSpec.run(BRequestFilterSpec.scala:18)
	at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:314)
	at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:480)
	at sbt.TestRunner.runTest$1(TestFramework.scala:76)
	at sbt.TestRunner.run(TestFramework.scala:85)
	at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
	at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
	at sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:185)
	at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
	at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
	at sbt.TestFunction.apply(TestFramework.scala:207)
	at sbt.Tests$$anonfun$9.apply(Tests.scala:216)
	at sbt.Tests$$anonfun$9.apply(Tests.scala:216)
	at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
	at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
	at sbt.std.Transform$$anon$4.work(System.scala:63)
	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
	at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
	at sbt.Execute.work(Execute.scala:237)
	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
	at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
	at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

Found 1 deadlock.

```



ApiKeyVerificationFilter, line 77 is `val apikey = mock[ImmutableMap[String, ChildData]]` and BRequestFilterSpec line 31 is val pmap = mock[ParamMap].


sbt prompt looks like this:
```
$ sbt8             
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256M; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
[info] Loading global plugins from /Users/m/.sbt/0.13/plugins
[info] Loading project definition from  ...
[info] Set current project to newton-api (in build file:...)
> test
[info] NServiceModuleSpec:
[info] NServiceModule
[info] - should have a test !!! IGNORED !!!

-- hangs here --
```

Mac OS 10.11.6
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
sbt 0.13.9
"org.scalatest"           %%   "scalatest"        %   "3.0.1"   %   "test",
"org.mockito"              %   "mockito-core"   %   "2.7.22"  %   "test"


`parallelExecution in Test := true` in build.sbt  -  setting this to false circumvents the issue.

Issue comments:
It seems to me as if you are defining circular class loading hierarchies. With Mockito 2, we are applying more fine grained locking to improve concurrency. The difference is displayed in the stack trace where `findOrInsert` aquires a lock for the class loader of the mock class before creating a mock. The previous solution always aquired a global lock what made it impossible to create to mock classes concurrently.

The relevant part:

```
at java.lang.ClassLoader.loadClass(ClassLoader.java:404)
- waiting to lock <0x00000007959e4a78> (a sbt.classpath.ClasspathUtilities$$anon$1)
at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
- locked <0x0000000795a868e8> (a sbt.classpath.ClasspathFilter)
	
at java.lang.ClassLoader.loadClass(ClassLoader.java:404)
- waiting to lock <0x0000000795a868e8> (a sbt.classpath.ClasspathFilter)
at net.bytebuddy.TypeCache.findOrInsert(TypeCache.java:161)
- locked <0x00000007959e4a78> (a sbt.classpath.ClasspathUtilities$$anon$1)
```

How can it be that `sbt.classpath.ClasspathFilter` delegates to `sbt.classpath.ClasspathUtilities$$anon$1` where the latter apparently also reqires locking the former? Class loaders should only be used in strict hierarchies.
So both Maps are abstract, I'll try mocking concrete implementations. That might have caused the classloaders to kick in. I try to rewrite some of the tests to use specific implementations instead of abstract classes, but for i.e. ImmutableMap that's gonna be difficult as far as I can see.

edit:
Switched to a concrete Map, just deadlocks somewhere else now, while mocking a Finagle Service[Request, Response] and a simple trait, same locks as before though.

```
Found one Java-level deadlock:
=============================
"Timer-0":
  waiting to lock monitor 0x00007f4d08002a48 (object 0x00000000c2857fb8, a sbt.classpath.ClasspathUtilities$$anon$1),
  which is held by "pool-9-thread-6-ScalaTest-running-BRequestFilterSpec"
"pool-9-thread-6-ScalaTest-running-BRequestFilterSpec":
  waiting to lock monitor 0x00007f4d34003f08 (object 0x00000000c2857f60, a sbt.classpath.ClasspathFilter),
  which is held by "pool-9-thread-7-ScalaTest-running-JFilterSpec"
"pool-9-thread-7-ScalaTest-running-JFilterSpec":
  waiting to lock monitor 0x00007f4d08002a48 (object 0x00000000c2857fb8, a sbt.classpath.ClasspathUtilities$$anon$1),
  which is held by "pool-9-thread-6-ScalaTest-running-BRequestFilterSpec"
```
I've run into this same problem recently as well and my jstack was identical to the OP's. I've started playing around with the mockito versions and I'm not sure if this helps, but using mockito 2.6.6 in my tests didn't result in a deadlock, while using 2.6.7+ did result in deadlock.
Maybe we need to add a property that can be enabled to avoid the more fine-grained locking for the type cache. I would still like to know what is causing this as I even stress-tested the cache lock but I cannot reproduce the locking.

Is there any project that I can build that reliably reproduces this?
I'm not sure if it's the same problem but I got similar deadlock after specs2 update:
```
17:57:32 	at java.lang.ClassLoader.loadClass(ClassLoader.java:404)
17:57:32 	- waiting to lock <0x00000000d7fda228> (a java.lang.Object)
17:57:32 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
17:57:32 	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
17:57:32 	at java.lang.Class.getDeclaredMethods0(Native Method)
17:57:32 	at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
17:57:32 	at java.lang.Class.getDeclaredMethods(Class.java:1975)
17:57:32 	at net.bytebuddy.description.method.MethodList$ForLoadedMethods.<init>(MethodList.java:90)
17:57:32 	at net.bytebuddy.description.type.TypeDescription$ForLoadedType.getDeclaredMethods(TypeDescription.java:7232)
17:57:32 	at net.bytebuddy.description.type.TypeDescription$Generic$OfNonGenericType.getDeclaredMethods(TypeDescription.java:3349)
17:57:32 	at net.bytebuddy.dynamic.scaffold.MethodGraph$Compiler$Default.doAnalyze(MethodGraph.java:576)
17:57:32 	at net.bytebuddy.dynamic.scaffold.MethodGraph$Compiler$Default.analyze(MethodGraph.java:537)
17:57:32 	at net.bytebuddy.dynamic.scaffold.MethodGraph$Compiler$Default.doAnalyze(MethodGraph.java:573)
17:57:32 	at net.bytebuddy.dynamic.scaffold.MethodGraph$Compiler$Default.compile(MethodGraph.java:508)
17:57:32 	at net.bytebuddy.dynamic.scaffold.MethodGraph$Compiler$AbstractBase.compile(MethodGraph.java:408)
17:57:32 	at net.bytebuddy.dynamic.scaffold.MethodRegistry$Default.prepare(MethodRegistry.java:418)
17:57:32 	at net.bytebuddy.dynamic.scaffold.subclass.SubclassDynamicTypeBuilder.make(SubclassDynamicTypeBuilder.java:162)
17:57:32 	at net.bytebuddy.dynamic.scaffold.subclass.SubclassDynamicTypeBuilder.make(SubclassDynamicTypeBuilder.java:155)
17:57:32 	at net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:2560)
17:57:32 	at net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$Delegator.make(DynamicType.java:2662)
17:57:32 	at org.mockito.internal.creation.bytebuddy.SubclassBytecodeGenerator.mockClass(SubclassBytecodeGenerator.java:94)
17:57:32 	at org.mockito.internal.creation.bytebuddy.TypeCachingBytecodeGenerator$1.call(TypeCachingBytecodeGenerator.java:37)
17:57:32 	at org.mockito.internal.creation.bytebuddy.TypeCachingBytecodeGenerator$1.call(TypeCachingBytecodeGenerator.java:34)
17:57:32 	at net.bytebuddy.TypeCache.findOrInsert(TypeCache.java:138)
17:57:32 	at net.bytebuddy.TypeCache$WithInlineExpunction.findOrInsert(TypeCache.java:346)
17:57:32 	at net.bytebuddy.TypeCache.findOrInsert(TypeCache.java:161)
17:57:32 	- locked <0x0000000080591100> (a sun.misc.Launcher$AppClassLoader)
17:57:32 	at net.bytebuddy.TypeCache$WithInlineExpunction.findOrInsert(TypeCache.java:355)
17:57:32 	at org.mockito.internal.creation.bytebuddy.TypeCachingBytecodeGenerator.mockClass(TypeCachingBytecodeGenerator.java:32)
17:57:32 	at org.mockito.internal.creation.bytebuddy.SubclassByteBuddyMockMaker.createMockType(SubclassByteBuddyMockMaker.java:71)
17:57:32 	at org.mockito.internal.creation.bytebuddy.SubclassByteBuddyMockMaker.createMock(SubclassByteBuddyMockMaker.java:42)
17:57:32 	at org.mockito.internal.creation.bytebuddy.ByteBuddyMockMaker.createMock(ByteBuddyMockMaker.java:25)
17:57:32 	at org.mockito.internal.util.MockUtil.createMock(MockUtil.java:35)
17:57:32 	at org.mockito.internal.MockitoCore.mock(MockitoCore.java:63)
17:57:32 	at org.mockito.Mockito.mock(Mockito.java:1729)
17:57:32 	at org.mockito.Mockito.mock(Mockito.java:1642)
17:57:32 	at org.specs2.mock.MockitoMocker$class.mock(MockitoMocker.scala:21)
17:57:32 	at org.specs2.mock.mockito.TheMockitoMocker$$anon$1.mock(TheMockitoMocker.scala:8)
17:57:32 	at org.specs2.mock.mockito.MocksCreation$class.mock(MocksCreation.scala:18)
```
@raphw 
> Is there any project that I can build that reliably reproduces this?

This seems to be happening for every travis build for https://github.com/lucidsoftware/xtract

However, it doesn't happen when I build locally.

I was able to reproduce by building it in the travisci/ci-garnet:packer-1512502276-986baf0 docker image. Which is how I was able to get a thread dump and figure out that this is a likely cause.

Now that I have identified what the problem probably is, I'll try to fix future builds, but a commit id that is broken is 1939ab6798c974e4238439d04ba54a71ece5c31c.

From the thread dump, I think the applicable code is:

```scala
  private class PureXmlReader[T](result: T) extends XmlReader[T] {
    def read(elem: scala.xml.NodeSeq): ParseResult[T] = ParseSuccess(result)
  }

  private class PartialSuccessXmlReader[T](result: T, errors: Seq[ParseError]) extends XmlReader[T] {
    def read(elem: scala.xml.NodeSeq): ParseResult[T] = PartialParseSuccess(result, errors)
  }

  def successXmlReader[T](result: T): XmlReader[T] =
    spy(new PureXmlReader(result))

  def partialSuccessXmlReader[T](result: T, errors: Seq[ParseError]): XmlReader[T] =
    spy(new PartialSuccessXmlReader[T](result, errors))
```

It seems successXmlReader and partialSuccessXmlReader are being called simultaneously, and creating the spies concurrently results in a deadlock. The fact that both PureXmlReader and PartialSuccessXmlReader extend XmlReader is suspicious.

I can give you the full thread dump if you want, but it is pretty similar to the original post.
In my case I had to fix it by adding `sequential` to the Specification to preven examples from being executed in parallel.
Hey guys, we're planning to merge the fix soon. Thank you for patience and reporting another use case here!

Pull request review comments:
@raphw, can we lock always using BOOTSTRAP_LOCK?

In JVM that runs tests, we have one instance of BOOTSTRAP_LOCK and (typically) one instance of the classloader. If using classloader is a performance tweak, do we have an evidence how much value it adds? Also, the lock is only used for "new" classes (not yet in cache). I am not convinced using "classloader" as monitor adds value.

It is a best practice to always use internal objects as synchronization monitors (private final Object). This way, nobody else will use that object for monitoring and we're safe from deadlocks.

I'm not a fan of adding a system property to control this. It does not help with user experience because the user is impacted by a hanging build. After some googling, the user might be able to find an answer or he needs to reach out to us to learn about the property. But then what - he needs to remember to use the property in codebases that are exposed to this problem? Also, a system property is yet another method to configure Mockito - let's limit the various ways Mockito can be configured.

Hope you don't mind me complaining :) I'm super grateful you debugged this issue and offered a solution! For Mockito, it's best if we keep things simple, add new configuration options only when we really need to, and strive for great user experience.

Patch that resolved the issue:
diff --git a/src/main/java/org/mockito/internal/creation/bytebuddy/TypeCachingBytecodeGenerator.java b/src/main/java/org/mockito/internal/creation/bytebuddy/TypeCachingBytecodeGenerator.java
index 34c31feef2..ea51edf780 100644
--- a/src/main/java/org/mockito/internal/creation/bytebuddy/TypeCachingBytecodeGenerator.java
+++ b/src/main/java/org/mockito/internal/creation/bytebuddy/TypeCachingBytecodeGenerator.java
@@ -36,7 +36,7 @@ public <T> Class<T> mockClass(final MockFeatures<T> params) {
                         public Class<?> call() throws Exception {
                             return bytecodeGenerator.mockClass(params);
                         }
-                    }, classLoader == null ? BOOTSTRAP_LOCK : classLoader);
+                    }, BOOTSTRAP_LOCK);
         } catch (IllegalArgumentException exception) {
             Throwable cause = exception.getCause();
             if (cause instanceof RuntimeException) {


Test patch: