Repository: mockito/mockito
Issue: Any way to enable Strict Stubbing when using Mockito.mock() without using @Mock?

Hi,
I am using mockito-core 4.5.1 and JUnit 5. I can use `MockitoExtension` and `@Mock` annotation and by default I get Strict Stubbing behaviour. However, if I use `Mockito.Mock(MyClass.class)` to create mock it seems Strictness is not enabled. Even if I stub unnecessary methods it is not giving any error. 

I know I can work around this by using MockitoExtension to create mocks using @Mock, but is there any way to enable Strict Stubbing while using `Mockito.mock(MyClass.class);`

```
class ProductServiceTest {

    private ProductRepository productRepository;
    private ProductService productService;

    @BeforeEach
    void setUp() {
        productRepository = Mockito.mock(ProductRepository.class);
        productService = new ProductService(productRepository);
        // this is unnecessary method stubbing but it is not giving any error
        Mockito.when(productRepository.findAllActiveProducts()).thenReturn(List.of());
    }

    @Test
    void shouldReturnOnlyActiveProducts() {
        BDDMockito.given(productRepository.findAll()).willReturn(List.of(p1, p2));
        productService.findAll(); // which calls productRepository.findAll()
    }
}
```

 

Issue comments:
I would have though that we supported this feature with our `MockSettings`. While looking that up, I discovered that we do ship [`lenient()`](https://javadoc.io/static/org.mockito/mockito-core/4.5.1/org/mockito/MockSettings.html#lenient()), but we don't ship an equivalent API to set the strictness to `STRICT_STUBS`. I consider that a deficiency of the API, for which we should add an option. Therefore, +1 to this feature request.

In hindsight, we shouldn't have shipped `mockSettings.lenient()`, but instead add `mockSettings.strictness(Strictness)`. We have the same issue with `@Mock`: https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mock.html#lenient()

Therefore, I propose we update both `MockSettings` and `@Mock` to accept a new setting called `strictness` which sets the `Strictness`. We can then deprecate both `lenient()` options, as they would be a shortcut for `strictness(Strictness.LENIENT)`.

@sivaprasadreddy Would you like to contribute this feature to Mockito?
@TimvdLippe Sure, I will contribute to this feature.
@TimvdLippe I have raised a PR https://github.com/mockito/mockito/pull/2650 and the GH Actions pipeline is waiting for approval https://github.com/mockito/mockito/actions/runs/2368951973.
@TimvdLippe The PR build failed (https://github.com/mockito/mockito/actions/runs/2368951973) because of an error in Javadoc comment, which I have fixed now.

Pull request review comments:
Nit: `@since 4.6.0` since we will include this in the next minor release
Nit: the default is the default of the runner. If you use the lenient runner, it will be lenient. I suggest to formulate it the following way:

```
The default strictness level is determined by the rule/runner used. If you are using no rule/runner, the default strictness level is LENIENT.
```
Do we still use this method somewhere else? If yes, we should refactor these to call `getStrictness` and deprecate this method (since we ship it in the `MockSettings` interface, we can't remove it just yet).
Nit: 4.6.0
This line isn't covered yet by the tests. Therefore, let's add a test that attempts to set the `strictness` to `null` and ensures that an Exception is thrown.
We should deprecate this method and point users to the `strictness` method.
We should deprecate this method as well.
Nit: mocks by default are lenient. I would omit that part altogether and reword it to:

```
You can now customize the strictness level for a single mock, either using `@Mock` annotation strictness attribute or using `MockSettings.strictness()`. This can be useful if you want all of your mocks to be strict, but one of the mocks to be lenient.
```
Nit: let's extract separate test methods for these cases. E.g. we have 2 tests here, one which ensures the `lenientMock` is allowed and one where the `regularMock` throws.
Nit: I think we can use a lambda here, can we not?

```java
() -> ProductionCode.simpleMethod(regularMock, "4")
```
Same here, let's have 3 separate 3 test cases here.
Updated.
Updated.
Yes, this isLenient() is being used in one place in main code. I have refactored that to use getStrictness().

But this isLenient() is also being used in few tests, I think we can remove those reference while removing this deprecated method.
Updated.
Added test to verify this.
Added deprecation marker and suggestion to use strictness().
Added deprecation marker and suggestion to use `strictness()`.
Updated.
Separated this into multiple tests.
Updated.
Separated this into multiple tests.
Quick Q - was it intentional for this to change mocks defined by `mock(Blah.class)`  i.e with no rule/Junit `MockitoSettings` to strict by default?

I believe the previous behaviour was lenient by default?
We have not changed the behavior here. This test is using a Strict rule explicitly (https://github.com/mockito/mockito/pull/2650/files/d0907808e9fce24f6dcddc94d1ea2c3ebfa15676#diff-3040268f6b18ef562dd495ac7fc13465723311889d5f326bfb615cc3aed37415R21) to verify that we handle that case correctly.

In case you are hitting any regressions, please file a new issue. You are most likely running into a different problem.
OK, thanks. We definitely have regressions (previously lenient mocks now behaving as strict) but I may have misidentified the possible cause. Will need to dig further if I get time.
Another person has beat me to raising it at #2656 but this seems to be the same issue we are having.

Patch that resolved the issue:
diff --git a/src/main/java/org/mockito/Mock.java b/src/main/java/org/mockito/Mock.java
index 62c4faa2df..7c3e036083 100644
--- a/src/main/java/org/mockito/Mock.java
+++ b/src/main/java/org/mockito/Mock.java
@@ -13,6 +13,7 @@
 import java.lang.annotation.Target;
 
 import org.mockito.junit.MockitoJUnitRunner;
+import org.mockito.quality.Strictness;
 import org.mockito.stubbing.Answer;
 
 /**
@@ -105,10 +106,21 @@
     boolean serializable() default false;
 
     /**
+     * @deprecated Use {@link Mock#strictness()} instead.
+     *
      * Mock will be lenient, see {@link MockSettings#lenient()}.
      * For examples how to use 'Mock' annotation and parameters see {@link Mock}.
      *
      * @since 2.23.3
      */
+    @Deprecated
     boolean lenient() default false;
+
+    /**
+     * Mock will have custom strictness, see {@link MockSettings#strictness(Strictness)}.
+     * For examples how to use 'Mock' annotation and parameters see {@link Mock}.
+     *
+     * @since 4.6.0
+     */
+    Strictness strictness() default Strictness.STRICT_STUBS;
 }
diff --git a/src/main/java/org/mockito/MockSettings.java b/src/main/java/org/mockito/MockSettings.java
index fb6554070f..f838a8d121 100644
--- a/src/main/java/org/mockito/MockSettings.java
+++ b/src/main/java/org/mockito/MockSettings.java
@@ -350,6 +350,8 @@ public interface MockSettings extends Serializable {
     <T> MockCreationSettings<T> buildStatic(Class<T> classToMock);
 
     /**
+     * @deprecated Use {@link MockSettings#strictness(Strictness)} instead.
+     *
      * Lenient mocks bypass "strict stubbing" validation (see {@link Strictness#STRICT_STUBS}).
      * When mock is declared as lenient none of its stubbings will be checked for potential stubbing problems such as
      * 'unnecessary stubbing' ({@link UnnecessaryStubbingException}) or for 'stubbing argument mismatch' {@link PotentialStubbingProblem}.
@@ -360,5 +362,23 @@ public interface MockSettings extends Serializable {
      *
      * For more information and an elaborate example, see {@link Mockito#lenient()}.
      */
+    @Deprecated
     MockSettings lenient();
+
+    /**
+     * Specifies strictness level for the mock.
+     * The default strictness level is determined by the rule/runner used.
+     * If you are using no rule/runner, the default strictness level is LENIENT
+     *
+     * <pre class="code"><code class="java">
+     *   Foo defaultStrictMock = mock(Foo.class);
+     *   Foo explicitStrictMock = mock(Foo.class, withSettings().strictness(Strictness.STRICT_STUBS));
+     *   Foo lenientMock = mock(Foo.class, withSettings().strictness(Strictness.LENIENT));
+     * </code></pre>
+     *
+     * @param strictness the strictness level to set on mock
+     * @return settings instance so that you can fluently specify other settings
+     * @since 4.6.0
+     */
+    MockSettings strictness(Strictness strictness);
 }
diff --git a/src/main/java/org/mockito/Mockito.java b/src/main/java/org/mockito/Mockito.java
index 8c1a0cff9b..8b7b92776c 100644
--- a/src/main/java/org/mockito/Mockito.java
+++ b/src/main/java/org/mockito/Mockito.java
@@ -105,6 +105,7 @@
  *      <a href="#49">49. New API for mocking object construction (Since 3.5.0)</a><br/>
  *      <a href="#50">50. Avoiding code generation when restricting mocks to interfaces (Since 3.12.2)</a><br/>
  *      <a href="#51">51. New API for marking classes as unmockable (Since 4.1.0)</a><br/>
+ *      <a href="#51">52. New strictness attribute for @Mock annotation and <code>MockSettings.strictness()</code> methods (Since 4.6.0)</a><br/>
  * </b>
  *
  * <h3 id="0">0. <a class="meaningful_link" href="#mockito2" name="mockito2">Migrating to Mockito 2</a></h3>
@@ -1606,6 +1607,21 @@
  * For any class/interface you own that is problematic to mock, you can now mark the class with {@link org.mockito.DoNotMock @DoNotMock}. For usage
  * of the annotation and how to ship your own (to avoid a compile time dependency on a test artifact), please see its JavaDoc.
  * <p>
+ *
+ * <h3 id="52">52. <a class="meaningful_link" href="#mockito_strictness" name="mockito_strictness">
+ *  New strictness attribute for @Mock annotation and <code>MockSettings.strictness()</code> methods (Since 4.6.0)</a></h3>
+ *
+ * You can now customize the strictness level for a single mock, either using `@Mock` annotation strictness attribute or
+ * using `MockSettings.strictness()`. This can be useful if you want all of your mocks to be strict,
+ * but one of the mocks to be lenient.
+ *
+ * <pre class="code"><code class="java">
+ *   &#064;Mock(strictness = Strictness.LENIENT)
+ *   Foo mock;
+ *   // using MockSettings.withSettings()
+ *   Foo mock = Mockito.mock(Foo.class, withSettings().strictness(Strictness.WARN));
+ * </code></pre>
+ *
  */
 @CheckReturnValue
 @SuppressWarnings("unchecked")
diff --git a/src/main/java/org/mockito/internal/configuration/MockAnnotationProcessor.java b/src/main/java/org/mockito/internal/configuration/MockAnnotationProcessor.java
index e48291e5d5..484c9102e4 100644
--- a/src/main/java/org/mockito/internal/configuration/MockAnnotationProcessor.java
+++ b/src/main/java/org/mockito/internal/configuration/MockAnnotationProcessor.java
@@ -45,6 +45,7 @@ public static Object processAnnotationForMock(
         if (annotation.stubOnly()) {
             mockSettings.stubOnly();
         }
+        mockSettings.strictness(annotation.strictness());
         if (annotation.lenient()) {
             mockSettings.lenient();
         }
diff --git a/src/main/java/org/mockito/internal/creation/MockSettingsImpl.java b/src/main/java/org/mockito/internal/creation/MockSettingsImpl.java
index 3aa746c065..e1dcd48f9e 100644
--- a/src/main/java/org/mockito/internal/creation/MockSettingsImpl.java
+++ b/src/main/java/org/mockito/internal/creation/MockSettingsImpl.java
@@ -12,6 +12,7 @@
 import static org.mockito.internal.exceptions.Reporter.extraInterfacesRequiresAtLeastOneInterface;
 import static org.mockito.internal.exceptions.Reporter.methodDoesNotAcceptParameter;
 import static org.mockito.internal.exceptions.Reporter.requiresAtLeastOneListener;
+import static org.mockito.internal.exceptions.Reporter.strictnessDoesNotAcceptNullParameter;
 import static org.mockito.internal.util.collections.Sets.newSet;
 
 import java.io.Serializable;
@@ -33,6 +34,7 @@
 import org.mockito.mock.MockCreationSettings;
 import org.mockito.mock.MockName;
 import org.mockito.mock.SerializableMode;
+import org.mockito.quality.Strictness;
 import org.mockito.stubbing.Answer;
 
 @SuppressWarnings("unchecked")
@@ -239,7 +241,16 @@ public <T2> MockCreationSettings<T2> buildStatic(Class<T2> classToMock) {
 
     @Override
     public MockSettings lenient() {
-        this.lenient = true;
+        this.strictness = Strictness.LENIENT;
+        return this;
+    }
+
+    @Override
+    public MockSettings strictness(Strictness strictness) {
+        this.strictness = strictness;
+        if (strictness == null) {
+            throw strictnessDoesNotAcceptNullParameter();
+        }
         return this;
     }
 
diff --git a/src/main/java/org/mockito/internal/creation/settings/CreationSettings.java b/src/main/java/org/mockito/internal/creation/settings/CreationSettings.java
index 9da7311c87..50bb937696 100644
--- a/src/main/java/org/mockito/internal/creation/settings/CreationSettings.java
+++ b/src/main/java/org/mockito/internal/creation/settings/CreationSettings.java
@@ -18,6 +18,7 @@
 import org.mockito.mock.MockCreationSettings;
 import org.mockito.mock.MockName;
 import org.mockito.mock.SerializableMode;
+import org.mockito.quality.Strictness;
 import org.mockito.stubbing.Answer;
 
 public class CreationSettings<T> implements MockCreationSettings<T>, Serializable {
@@ -44,7 +45,7 @@ public class CreationSettings<T> implements MockCreationSettings<T>, Serializabl
     private boolean useConstructor;
     private Object outerClassInstance;
     private Object[] constructorArgs;
-    protected boolean lenient;
+    protected Strictness strictness = Strictness.STRICT_STUBS;
 
     public CreationSettings() {}
 
@@ -65,7 +66,7 @@ public CreationSettings(CreationSettings copy) {
         this.useConstructor = copy.isUsingConstructor();
         this.outerClassInstance = copy.getOuterClassInstance();
         this.constructorArgs = copy.getConstructorArgs();
-        this.lenient = copy.lenient;
+        this.strictness = copy.strictness;
         this.stripAnnotations = copy.stripAnnotations;
     }
 
@@ -170,6 +171,11 @@ public boolean isStubOnly() {
 
     @Override
     public boolean isLenient() {
-        return lenient;
+        return strictness == Strictness.LENIENT;
+    }
+
+    @Override
+    public Strictness getStrictness() {
+        return strictness;
     }
 }
diff --git a/src/main/java/org/mockito/internal/exceptions/Reporter.java b/src/main/java/org/mockito/internal/exceptions/Reporter.java
index f91c6e1f0d..ea670407e7 100644
--- a/src/main/java/org/mockito/internal/exceptions/Reporter.java
+++ b/src/main/java/org/mockito/internal/exceptions/Reporter.java
@@ -964,6 +964,10 @@ public static MockitoException defaultAnswerDoesNotAcceptNullParameter() {
         return new MockitoException("defaultAnswer() does not accept null parameter");
     }
 
+    public static MockitoException strictnessDoesNotAcceptNullParameter() {
+        return new MockitoException("strictness() does not accept null parameter");
+    }
+
     public static MockitoException serializableWontWorkForObjectsThatDontImplementSerializable(
             Class<?> classToMock) {
         return new MockitoException(
diff --git a/src/main/java/org/mockito/internal/stubbing/InvocationContainerImpl.java b/src/main/java/org/mockito/internal/stubbing/InvocationContainerImpl.java
index 3091670fb1..5aedbb912e 100644
--- a/src/main/java/org/mockito/internal/stubbing/InvocationContainerImpl.java
+++ b/src/main/java/org/mockito/internal/stubbing/InvocationContainerImpl.java
@@ -38,7 +38,7 @@ public class InvocationContainerImpl implements InvocationContainer, Serializabl
 
     public InvocationContainerImpl(MockCreationSettings mockSettings) {
         this.registeredInvocations = createRegisteredInvocations(mockSettings);
-        this.mockStrictness = mockSettings.isLenient() ? Strictness.LENIENT : null;
+        this.mockStrictness = mockSettings.getStrictness();
         this.doAnswerStyleStubbing = new DoAnswerStyleStubbing();
     }
 
diff --git a/src/main/java/org/mockito/internal/stubbing/StrictnessSelector.java b/src/main/java/org/mockito/internal/stubbing/StrictnessSelector.java
index 6422fd53ae..c8e7e44400 100644
--- a/src/main/java/org/mockito/internal/stubbing/StrictnessSelector.java
+++ b/src/main/java/org/mockito/internal/stubbing/StrictnessSelector.java
@@ -31,8 +31,8 @@ public static Strictness determineStrictness(
             return stubbing.getStrictness();
         }
 
-        if (mockSettings.isLenient()) {
-            return Strictness.LENIENT;
+        if (mockSettings.getStrictness() != null) {
+            return mockSettings.getStrictness();
         }
 
         return testLevelStrictness;
diff --git a/src/main/java/org/mockito/mock/MockCreationSettings.java b/src/main/java/org/mockito/mock/MockCreationSettings.java
index 23d5418a14..f7f0b96028 100644
--- a/src/main/java/org/mockito/mock/MockCreationSettings.java
+++ b/src/main/java/org/mockito/mock/MockCreationSettings.java
@@ -118,10 +118,21 @@ public interface MockCreationSettings<T> {
     Object getOuterClassInstance();
 
     /**
+     *  @deprecated Use {@link MockCreationSettings#getStrictness()} instead.
+     *
      * Informs if the mock was created with "lenient" strictness, e.g. having {@link Strictness#LENIENT} characteristic.
      * For more information about using mocks with lenient strictness, see {@link MockSettings#lenient()}.
      *
      * @since 2.20.0
      */
+    @Deprecated
     boolean isLenient();
+
+    /**
+     * Sets strictness level for the mock, e.g. having {@link Strictness#STRICT_STUBS} characteristic.
+     * For more information about using mocks with custom strictness, see {@link MockSettings#strictness(Strictness)}.
+     *
+     * @since 4.6.0
+     */
+    Strictness getStrictness();
 }


Test patch:
diff --git a/src/test/java/org/mockito/internal/creation/MockSettingsImplTest.java b/src/test/java/org/mockito/internal/creation/MockSettingsImplTest.java
index 9e8d68d5d8..dc8af35423 100644
--- a/src/test/java/org/mockito/internal/creation/MockSettingsImplTest.java
+++ b/src/test/java/org/mockito/internal/creation/MockSettingsImplTest.java
@@ -260,4 +260,10 @@ public void addListeners_canAddDuplicateMockObjectListeners_ItsNotOurBusinessThe
         assertThat(mockSettingsImpl.getStubbingLookupListeners())
                 .containsSequence(stubbingLookupListener, stubbingLookupListener);
     }
+
+    @Test
+    public void validates_strictness() {
+        assertThatThrownBy(() -> mockSettingsImpl.strictness(null))
+                .hasMessageContaining("strictness() does not accept null parameter");
+    }
 }
diff --git a/src/test/java/org/mockitousage/strictness/StrictnessMockAnnotationTest.java b/src/test/java/org/mockitousage/strictness/StrictnessMockAnnotationTest.java
new file mode 100644
index 0000000000..c035b3d708
--- /dev/null
+++ b/src/test/java/org/mockitousage/strictness/StrictnessMockAnnotationTest.java
@@ -0,0 +1,43 @@
+/*
+ * Copyright (c) 2022 Mockito contributors
+ * This program is made available under the terms of the MIT License.
+ */
+package org.mockitousage.strictness;
+
+import org.assertj.core.api.Assertions;
+import org.junit.Rule;
+import org.junit.Test;
+import org.mockito.Mock;
+import org.mockito.exceptions.misusing.PotentialStubbingProblem;
+import org.mockito.junit.MockitoJUnit;
+import org.mockito.junit.MockitoRule;
+import org.mockito.quality.Strictness;
+import org.mockitousage.IMethods;
+
+import static org.mockito.Mockito.when;
+
+public class StrictnessMockAnnotationTest {
+
+    public @Rule MockitoRule rule = MockitoJUnit.rule().strictness(Strictness.STRICT_STUBS);
+
+    @Mock(strictness = Strictness.LENIENT)
+    IMethods lenientMock;
+
+    @Mock IMethods regularMock;
+
+    @Test
+    public void mock_is_lenient() {
+        when(lenientMock.simpleMethod("1")).thenReturn("1");
+
+        // then lenient mock does not throw:
+        ProductionCode.simpleMethod(lenientMock, "3");
+    }
+
+    @Test
+    public void mock_is_strict() {
+        when(regularMock.simpleMethod("2")).thenReturn("2");
+
+        Assertions.assertThatThrownBy(() -> ProductionCode.simpleMethod(regularMock, "4"))
+                .isInstanceOf(PotentialStubbingProblem.class);
+    }
+}
diff --git a/src/test/java/org/mockitousage/strictness/StrictnessWithSettingsTest.java b/src/test/java/org/mockitousage/strictness/StrictnessWithSettingsTest.java
new file mode 100644
index 0000000000..854792cd50
--- /dev/null
+++ b/src/test/java/org/mockitousage/strictness/StrictnessWithSettingsTest.java
@@ -0,0 +1,57 @@
+/*
+ * Copyright (c) 2022 Mockito contributors
+ * This program is made available under the terms of the MIT License.
+ */
+package org.mockitousage.strictness;
+
+import org.assertj.core.api.Assertions;
+import org.junit.Before;
+import org.junit.Rule;
+import org.junit.Test;
+import org.mockito.exceptions.misusing.PotentialStubbingProblem;
+import org.mockito.junit.MockitoJUnit;
+import org.mockito.junit.MockitoRule;
+import org.mockito.quality.Strictness;
+import org.mockitousage.IMethods;
+
+import static org.mockito.Mockito.*;
+
+public class StrictnessWithSettingsTest {
+
+    public @Rule MockitoRule rule = MockitoJUnit.rule().strictness(Strictness.STRICT_STUBS);
+
+    IMethods lenientMock;
+    IMethods regularMock;
+    IMethods strictMock;
+
+    @Before
+    public void before() {
+        lenientMock = mock(IMethods.class, withSettings().strictness(Strictness.LENIENT));
+        regularMock = mock(IMethods.class);
+        strictMock = mock(IMethods.class, withSettings().strictness(Strictness.STRICT_STUBS));
+    }
+
+    @Test
+    public void mock_is_lenient() {
+        when(lenientMock.simpleMethod("1")).thenReturn("1");
+
+        // lenient mock does not throw
+        ProductionCode.simpleMethod(lenientMock, "3");
+    }
+
+    @Test
+    public void mock_is_strict_with_default_settings() {
+        when(regularMock.simpleMethod("3")).thenReturn("3");
+
+        Assertions.assertThatThrownBy(() -> ProductionCode.simpleMethod(regularMock, "4"))
+                .isInstanceOf(PotentialStubbingProblem.class);
+    }
+
+    @Test
+    public void mock_is_strict_with_explicit_settings() {
+        when(strictMock.simpleMethod("2")).thenReturn("2");
+
+        Assertions.assertThatThrownBy(() -> ProductionCode.simpleMethod(strictMock, "5"))
+                .isInstanceOf(PotentialStubbingProblem.class);
+    }
+}