Repository: chainer/chainer

Issue:
flaky test: `chainer_tests/functions_tests/loss_tests/test_negative_sampling.py`
https://ci.preferred.jp/chainer.py37.gpu/34311/

Occurred in #8248

`[FAILED] /chainer/tests/chainer_tests/functions_tests/loss_tests/test_negative_sampling.py (1 failed, 96 passed, 49 deselected in 29.36 seconds)`

`TestNegativeSamplingFunction_use_chainerx_true__chainerx_device_cuda:1__use_cuda_false__cuda_device_None__use_cudnn_never__cudnn_deterministic_false__autotune_false__cudnn_fast_batch_normalization_false__use_ideep_never_param_1_{dtype=float16, reduce='no', t=[0, 2]}.test_forward `

```

2019-10-11 12:23:04.308787 STDOUT 2327] | E           AssertionError: Parameterized test failed.
-- | --
2019-10-11 12:23:04.308788 STDOUT 2327] | E
2019-10-11 12:23:04.308790 STDOUT 2327] | E           Base test method: TestNegativeSamplingFunction_use_chainerx_true__chainerx_device_cuda:1__use_cuda_false__cuda_device_None__use_cudnn_never__cudnn_deterministic_false__autotune_false__cudnn_fast_batch_normalization_false__use_ideep_never.test_forward
2019-10-11 12:23:04.308796 STDOUT 2327] | E           Test parameters:
2019-10-11 12:23:04.308798 STDOUT 2327] | E             dtype: <class 'numpy.float16'>
2019-10-11 12:23:04.308908 STDOUT 2327] | E             reduce: no
2019-10-11 12:23:04.308911 STDOUT 2327] | E             t: [0, 2]
2019-10-11 12:23:04.308912 STDOUT 2327] | E
2019-10-11 12:23:04.308913 STDOUT 2327] | E
2019-10-11 12:23:04.308915 STDOUT 2327] | E           (caused by)
2019-10-11 12:23:04.308917 STDOUT 2327] | E           AssertionError:
2019-10-11 12:23:04.308918 STDOUT 2327] | E           Not equal to tolerance rtol=0.001, atol=0.001
2019-10-11 12:23:04.308920 STDOUT 2327] | E
2019-10-11 12:23:04.308950 STDOUT 2327] | E           Mismatch: 50%
2019-10-11 12:23:04.308953 STDOUT 2327] | E           Max absolute difference: 0.003906
2019-10-11 12:23:04.309004 STDOUT 2327] | E           Max relative difference: 0.001363
2019-10-11 12:23:04.309006 STDOUT 2327] | E            x: array([1.697, 2.863], dtype=float16)
2019-10-11 12:23:04.309007 STDOUT 2327] | E            y: array([1.696, 2.867], dtype=float16)
2019-10-11 12:23:04.309009 STDOUT 2327] | E
2019-10-11 12:23:04.309052 STDOUT 2327] | E           assert_allclose failed:
2019-10-11 12:23:04.309054 STDOUT 2327] | E             shape: (2,) (2,)
2019-10-11 12:23:04.309055 STDOUT 2327] | E             dtype: float16 float16
2019-10-11 12:23:04.309162 STDOUT 2327] | E             i: (1,)
2019-10-11 12:23:04.309164 STDOUT 2327] | E             x[i]: 2.86328125
2019-10-11 12:23:04.309165 STDOUT 2327] | E             y[i]: 2.8671875
2019-10-11 12:23:04.309176 STDOUT 2327] | E             relative error[i]: 0.0013628005981445312
2019-10-11 12:23:04.309178 STDOUT 2327] | E             absolute error[i]: 0.00390625
2019-10-11 12:23:04.309179 STDOUT 2327] | E             relative tolerance * \|y[i]\|: 0.0028671875
2019-10-11 12:23:04.309181 STDOUT 2327] | E             absolute tolerance: 0.001
2019-10-11 12:23:04.309182 STDOUT 2327] | E             total tolerance: 0.0038671875
2019-10-11 12:23:04.309273 STDOUT 2327] | E           x: [1.697 2.863]
2019-10-11 12:23:04.309275 STDOUT 2327] | E           y: [1.696 2.867]
```


Discussion:

Resolving patch:
diff --git a/chainer/functions/loss/negative_sampling.py b/chainer/functions/loss/negative_sampling.py
--- a/chainer/functions/loss/negative_sampling.py
+++ b/chainer/functions/loss/negative_sampling.py
@@ -7,6 +7,7 @@
 from chainer import function_node
 from chainer import utils
 from chainer.utils import argument
+from chainer.utils import precision
 from chainer.utils import type_check
 
 
@@ -53,6 +54,8 @@ def check_type_forward(self, in_types):
             w_type.ndim == 2,
         )
 
+    # Avoid fp16 computation to keep the precision in reduction operations.
+    @precision._fp16_mixed_precision_helper
     def forward_cpu(self, inputs):
         self.retain_inputs((0, 1, 2))
         x, t, W = inputs
@@ -74,6 +77,7 @@ def forward_cpu(self, inputs):
         self.samples = samples
         return loss,
 
+    @precision._fp16_mixed_precision_helper
     def forward_gpu(self, inputs):
         self.retain_inputs((0, 1, 2))
         x, t, W = inputs


Test patch: