internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: chainer/chainer

Issue:
pytest 2x2 fails
Pytest with `2x2` (2 containers with 2 processes each, 4 processes in total) fails with the following error.

```
tests/chainermn_tests/communicator_tests/test_communicator.py::test_communicator_gpu[param6] FAILED

<snip>

communicator = <chainermn.communicators.two_dimensional_communicator.TwoDimensionalCommunicator object at 0x7efd4882dc50>
model = <test_communicator.ExampleModel object at 0x7efd4882dbd0>

<snip>

tests/chainermn_tests/communicator_tests/test_communicator.py:316:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

x = array([3.5, 3.5, 3.5, 5. , 5. ], dtype=float32)
y = array([3.5, 3.5, 3.5, 3.5, 3.5]), atol = 1e-05, rtol = 0.0001
verbose = True

<snip>

E           AssertionError:
E           Not equal to tolerance rtol=0.0001, atol=1e-05
E
E           Mismatch: 40%
E           Max absolute difference: 1.5
E           Max relative difference: 0.42857143
E            x: array([3.5, 3.5, 3.5, 5. , 5. ], dtype=float32)
E            y: array([3.5, 3.5, 3.5, 3.5, 3.5])
E
E           assert_allclose failed:
E             shape: (5,) (5,)
E             dtype: float32 float64
E             i: (3,)
E             x[i]: 5.0
E             y[i]: 3.5
E             relative error[i]: 0.428571428571
E             absolute error[i]: 1.5
E           x: [3.5 3.5 3.5 5.  5. ]
E           y: [3.5 3.5 3.5 3.5 3.5]

chainer/testing/array.py:59: AssertionError

```

The full output is shown here: https://gist.github.com/keisukefukuda/4e5a89b039f4e4083769ba8303bbf3b2


Discussion:

Resolving patch:
diff --git a/chainermn/communicators/two_dimensional_communicator.py b/chainermn/communicators/two_dimensional_communicator.py
--- a/chainermn/communicators/two_dimensional_communicator.py
+++ b/chainermn/communicators/two_dimensional_communicator.py
@@ -54,7 +54,7 @@ def allreduce_grad(self, model, zero_fill=False):
         itemsize = 4
         n_elems_total = _memory_utility.count_grad_elements(params,
                                                             zero_fill)
-        n_elems_per_node_2d = int(math.ceil(n_elems_total / self.size))
+        n_elems_per_node_2d = int(math.ceil(n_elems_total / self.inter_size))
         n_elems_per_node_1d = n_elems_per_node_2d * self.inter_size
         n_bytes_per_node_1d = n_elems_per_node_1d * itemsize
         n_bytes_per_node_2d = n_elems_per_node_2d * itemsize


Test patch:
Repository: chainer/chainer

Issue:
AdamW behavior on CPU changed at v6.0.0 and is now inconsistent to GPU logic
I just found that there is a code path in Adam (AdamW) on CPU that changes the behavior between v5.4.0 and v6.0.0.
(The problem has been originally reported by @harujoh on Twitter. Thank you!)

**Code to reproduce:**
```py3
import chainer as ch
import numpy as np  # s/numpy/cupy/ for cupy case

n = ch.Link(x=(2,))
x = n.x

o = ch.optimizers.Adam(eta=0.5, weight_decay_rate=0.1)
o.setup(n)

x.data.fill(1)
x.grad = np.ones_like(x.data)
o.update()

print(x.data)
```

chainer==5.4.0 (numpy): `[0.9495 0.9495]`
chainer==6.0.0 (numpy): `[0.89949995 0.89949995]`
chainer==6.0.0 (cupy): `[0.9495 0.9495]`

**Cause**
[Original logic](https://github.com/chainer/chainer/blob/v5/chainer/optimizers/adam.py#L132-L133) multiplies `eta` to `weight_decay_rate` while [new logic](https://github.com/chainer/chainer/blob/master/chainer/optimizers/adam.py#L219-L220) does not. It looks intel64 path in v5 is also inconsistent to the original CPU logic.

As the above result tells, [the GPU logic](https://github.com/chainer/chainer/blob/master/chainer/optimizers/adam.py#L303-L304) behaves same as the "original" logic.


Discussion:

Resolving patch:
diff --git a/chainer/optimizers/adam.py b/chainer/optimizers/adam.py
--- a/chainer/optimizers/adam.py
+++ b/chainer/optimizers/adam.py
@@ -217,7 +217,7 @@ def update_core_cpu(self, param):
         # param -=
         #  eta * (step * m - weight_decay_rate * param)
         _inplace_axpby(
-            param.data, 1.0 - hp.weight_decay_rate, -hp.eta, step * m)
+            param.data, 1.0 - hp.eta * hp.weight_decay_rate, -hp.eta, step * m)
 
     def update_core_gpu(self, param):
         grad = param.grad


Test patch:
diff --git a/tests/chainer_tests/optimizers_tests/test_optimizers.py b/tests/chainer_tests/optimizers_tests/test_optimizers.py
--- a/tests/chainer_tests/optimizers_tests/test_optimizers.py
+++ b/tests/chainer_tests/optimizers_tests/test_optimizers.py
@@ -145,4 +145,40 @@ def test_invalid_configs(self):
             self.optimizer.loss_scaling(scale=-1)
 
 
+@testing.backend.inject_backend_tests(
+    None,
+    [
+        # CPU
+        {},
+        # Intel
+        {'use_ideep': True},
+        # CUDA
+        {'use_cuda': True, 'cuda_device': 0},
+        # ChainerX
+        {'use_chainerx': True, 'chainerx_device': 'native:0'},
+        {'use_chainerx': True, 'chainerx_device': 'cuda:0'},
+    ]
+)
+class TestAdamW(unittest.TestCase):
+
+    def test_adam_w(self, backend_config):
+        xp = backend_config.xp
+        device = backend_config.device
+
+        link = chainer.Link(x=(1,))
+        link.to_device(device)
+
+        opt = optimizers.Adam(eta=0.5, weight_decay_rate=0.1)
+        opt.setup(link)
+
+        link.x.data.fill(1)
+        link.x.grad = device.send(xp.ones_like(link.x.data))
+
+        opt.update()
+
+        # compare against the value computed with v5 impl
+        testing.assert_allclose(link.x.data, np.array([0.9495]),
+                                atol=1e-7, rtol=1e-7)
+
+
 testing.run_module(__name__, __file__)
Repository: chainer/chainer

Issue:
Add a function which tells why ChainerX is not available
Although ChainerX has `chainerx.is_available()`, ChainerX does not tell why ChainerX is unavailable. For example, when I changed `LD_LIBRARY_PATH` by mistake, `chainerx.is_available()` told only one bit information, but `import chainerx._core` solved my problem.



Discussion:

Resolving patch:
diff --git a/chainerx/__init__.py b/chainerx/__init__.py
--- a/chainerx/__init__.py
+++ b/chainerx/__init__.py
@@ -1,16 +1,14 @@
 import os
-import sys
 import warnings
 
+from chainerx import _build_info
 
-if sys.version_info[0] < 3:
-    _available = False
+
+if _build_info.build_chainerx:
+    from chainerx import _core
+    _available = True
 else:
-    try:
-        from chainerx import _core
-        _available = True
-    except Exception:
-        _available = False
+    _available = False
 
 
 if _available:
diff --git a/chainerx_build_helper.py b/chainerx_build_helper.py
--- a/chainerx_build_helper.py
+++ b/chainerx_build_helper.py
@@ -12,6 +12,13 @@
 from setuptools.command import build_ext
 
 
+def emit_build_info(build_chainerx):
+    dirname = os.path.dirname(__file__)
+    filename = os.path.join(dirname, 'chainerx/_build_info.py')
+    with open(filename, mode='w') as f:
+        f.write('build_chainerx = {}\n'.format(build_chainerx))
+
+
 class CMakeExtension(setuptools.Extension):
 
     def __init__(self, name, build_targets, sourcedir=''):
@@ -89,6 +96,10 @@ def build_extension(self, ext):
 
 
 def config_setup_kwargs(setup_kwargs, build_chainerx):
+
+    # TODO(imanishi): Call this function with setuptools.
+    emit_build_info(build_chainerx)
+
     if not build_chainerx:
         # `chainerx` package needs to be able to be imported even if ChainerX
         # is unavailable.


Test patch:
Repository: chainer/chainer

Issue:
PTB custom loop examples are broken
The custom loop version of PTB example causes the following error:
```
AttributeError: 'ParallelSequentialIterator' object has no attribute 'reset'
```

`ParallelSequentialIterator` is a custom iterator that is defined in the example and does not implement `reset` method.

PTB example of the static graph optimization also has the same problem.

- [examples/ptb/train_ptb_custom_loop.py](https://github.com/chainer/chainer/blob/master/examples/ptb/train_ptb_custom_loop.py)
- [examples/static_graph_optimizations/ptb/train_ptb_custom_loop.py](https://github.com/chainer/chainer/blob/master/examples/static_graph_optimizations/ptb/train_ptb_custom_loop.py)

It looks that the change in #5834 was not enough.



Discussion:
We need to test these examples.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs. Thank you for your contributions.

@kmaehashi Is this ongoing?

Resolving patch:
diff --git a/examples/ptb/train_ptb.py b/examples/ptb/train_ptb.py
--- a/examples/ptb/train_ptb.py
+++ b/examples/ptb/train_ptb.py
@@ -57,18 +57,22 @@ def forward(self, x):
 class ParallelSequentialIterator(chainer.dataset.Iterator):
 
     def __init__(self, dataset, batch_size, repeat=True):
+        super(ParallelSequentialIterator, self).__init__()
         self.dataset = dataset
         self.batch_size = batch_size  # batch size
+        self.repeat = repeat
+        length = len(dataset)
+        # Offsets maintain the position of each sequence in the mini-batch.
+        self.offsets = [i * length // batch_size for i in range(batch_size)]
+        self.reset()
+
+    def reset(self):
         # Number of completed sweeps over the dataset. In this case, it is
         # incremented if every word is visited at least once after the last
         # increment.
         self.epoch = 0
         # True if the epoch is incremented at the last iteration.
         self.is_new_epoch = False
-        self.repeat = repeat
-        length = len(dataset)
-        # Offsets maintain the position of each sequence in the mini-batch.
-        self.offsets = [i * length // batch_size for i in range(batch_size)]
         # NOTE: this is not a count of parameter updates. It is just a count of
         # calls of ``__next__``.
         self.iteration = 0


Test patch:
Repository: chainer/chainer

Issue:
Support ellipsis in `Array::At` and `__getitem__`
Depends on #7559 because `py::ellipsis` is supported from v2.3.0.


Discussion:

Resolving patch:
diff --git a/chainerx/_fallback_workarounds.py b/chainerx/_fallback_workarounds.py
--- a/chainerx/_fallback_workarounds.py
+++ b/chainerx/_fallback_workarounds.py
@@ -118,10 +118,8 @@ def _populate_ndarray():
     old_getitem = ndarray.__getitem__
 
     def __getitem__(arr, key):
-        try:
+        if not isinstance(key, chainerx.ndarray):
             return old_getitem(arr, key)
-        except (IndexError, chainerx.DimensionError):
-            pass
 
         is_backprop_required = arr.is_backprop_required()
 


Test patch:
diff --git a/tests/chainerx_tests/unit_tests/routines_tests/test_indexing.py b/tests/chainerx_tests/unit_tests/routines_tests/test_indexing.py
--- a/tests/chainerx_tests/unit_tests/routines_tests/test_indexing.py
+++ b/tests/chainerx_tests/unit_tests/routines_tests/test_indexing.py
@@ -107,6 +107,12 @@
     ((), (chainerx.newaxis,)),
     ((3,), (chainerx.newaxis,)),
     ((2, 3), (chainerx.newaxis, chainerx.newaxis)),
+    # ellipsis indexing - non-tuple indexing
+    ((), Ellipsis),
+    ((3,), Ellipsis),
+    # ellipsis indexing - tuple indexing
+    ((), (Ellipsis,)),
+    ((2, 3), (Ellipsis,)),
     # mixed indexing - tuple indexing
     ((2, 3), (0, slice(1, 3))),
     ((4, 3), (slice(1, 3), 1)),
@@ -118,6 +124,14 @@
     ((2, 3, 4), (chainerx.newaxis, slice(0, 1), slice(1, 2), slice(1, 3))),
     ((2, 3, 4),
      (1, slice(2,), chainerx.newaxis, slice(1, 3), chainerx.newaxis)),
+    ((2, 3, 4), (0, Ellipsis)),
+    ((2, 3, 4), (Ellipsis, 2)),
+    ((2, 3, 4), (1, Ellipsis, 2)),
+    ((2, 3, 4), (1, Ellipsis, 2, 3)),
+    ((2, 3, 4), (chainerx.newaxis, Ellipsis, chainerx.newaxis)),
+    ((2, 3, 4), (1, Ellipsis, chainerx.newaxis, 3)),
+    ((2, 3, 4), (1, Ellipsis, 2, chainerx.newaxis, 3)),
+    ((2, 3, 4), (slice(0, 1), Ellipsis, slice(1, 3))),
 ])
 class TestGetitem(op_utils.NumpyOpTest):
     # TODO(niboshi): Remove this
@@ -141,6 +155,9 @@ def forward_xp(self, inputs, xp):
     ((3,), 3),
     ((3,), (0, 1)),
     ((2, 3,), (2, 0)),
+    ((2,), (2, chainerx.newaxis, 3)),
+    ((2,), (2, Ellipsis, chainerx.newaxis, 3)),
+    ((2,), (Ellipsis, Ellipsis)),
 ])
 def test_getitem_index_error(device, shape, indices):
     a = array_utils.create_dummy_ndarray(chainerx, shape, 'float32')
Repository: chainer/chainer

Issue:
Confusing error message about `FunctionNode.forward` output type
Code:
```python
import chainer, numpy

class MyFunc(chainer.FunctionNode):
    def forward_cpu(self, inputs):
        x, = inputs
        y = numpy.square(x),
        return y,  # nested tuple

x = chainer.Variable(numpy.ones((2,), 'float32'))
MyFunc().apply((x,))
```

Error message:
```
TypeError: incompatible array types are mixed in the forward output (MyFunc).
Actual: <class 'tuple'>
```

The message does not correctly describe the mistake.


Discussion:

Resolving patch:
diff --git a/chainer/function_node.py b/chainer/function_node.py
--- a/chainer/function_node.py
+++ b/chainer/function_node.py
@@ -331,6 +331,17 @@ def apply(self, inputs):
                 'Actual: {}'.format(self.label, type(outputs)))
 
         if not chainer.is_arrays_compatible(outputs):
+            if not all(
+                    isinstance(y, chainer.get_array_types())
+                    for y in outputs):
+                raise TypeError(
+                    'forward output must be a tuple of ndarrays.\n'
+                    'Function: {}\n'
+                    'Actual output types: {}'
+                    .format(
+                        self.label,
+                        tuple(type(y) for y in outputs)))
+
             raise TypeError(
                 'incompatible array types are mixed in the forward output '
                 '({}).\n'


Test patch:
diff --git a/tests/chainer_tests/test_function_node.py b/tests/chainer_tests/test_function_node.py
--- a/tests/chainer_tests/test_function_node.py
+++ b/tests/chainer_tests/test_function_node.py
@@ -426,14 +426,29 @@ def forward(self, inputs):
             f.apply((v,))
 
 
-class TestFunctionNodeInconsistentBackends(unittest.TestCase):
+class TestFunctionNodeForwardTypeCheck(unittest.TestCase):
 
     def setUp(self):
         self.x1 = numpy.random.rand(2, 3).astype(numpy.float32)
         self.x2 = numpy.random.rand(2, 3).astype(numpy.float32)
 
+    def test_invalid_output_type(self):
+        class FunctionNode(chainer.FunctionNode):
+
+            def forward(self, inputs):
+                return object(),
+
+        f = FunctionNode()
+        x1 = chainer.Variable(self.x1)
+
+        with six.assertRaisesRegex(
+                self,
+                TypeError,
+                'forward output must be a tuple of ndarrays'):
+            f.apply((x1,))
+
     @attr.gpu
-    def test_inconsistent_inputs(self):
+    def test_inconsistent_input_backends(self):
         class FunctionNode(chainer.FunctionNode):
 
             def forward(self, inputs):
@@ -451,7 +466,7 @@ def forward(self, inputs):
             f.apply((x1, x2))
 
     @attr.gpu
-    def test_inconsistent_outputs(self):
+    def test_inconsistent_output_backends(self):
         class FunctionNode(chainer.FunctionNode):
 
             def forward(self, inputs):
Repository: chainer/chainer

Issue:
Cannot read dtype of uninitialized `Parameter` with explicit dtype
Code to reproduce
```python
import chainer
import numpy


class MyInitializer:
    dtype = numpy.float32

    def __call__(self, array):
        array[...] = 0

param = chainer.Parameter(MyInitializer())  # dtype should have been determined

# print(param.dtype)  # This would cause "AttributeError: 'NoneType' object has no attribute 'dtype'"


param.initialize((3, 2))  # Make it initialized

print(param.dtype)
```
I expect `param.dtype` to return `float32` even if uninitialized, because it's explicitly given.


Discussion:

Resolving patch:
diff --git a/chainer/variable.py b/chainer/variable.py
--- a/chainer/variable.py
+++ b/chainer/variable.py
@@ -1700,6 +1700,19 @@ def __reduce__(self):
             self.initializer, self.update_rule, self.device)
         return _recover_parameter, args
 
+    @property
+    def dtype(self):
+        array = self.array
+        if array is not None:
+            return array.dtype
+        # uninitialized
+        initializer = self.initializer
+        if hasattr(initializer, 'dtype'):
+            return numpy.dtype(initializer.dtype)
+        raise RuntimeError(
+            'Dtype of the parameter is not determined yet because it\'s '
+            'uninitialized and dtype was not explicitly given.')
+
     def to_cpu(self):
         return self.to_device(backend.CpuDevice())
 


Test patch:
diff --git a/tests/chainer_tests/test_variable.py b/tests/chainer_tests/test_variable.py
--- a/tests/chainer_tests/test_variable.py
+++ b/tests/chainer_tests/test_variable.py
@@ -1989,6 +1989,25 @@ def test_addgrad_to_uninitialized_parameter_cpu_to_chx(self):
         # TODO(sonots): Support addgrad with ChainerX
         raise unittest.SkipTest('ChainerX does not support addgrad')
 
+    def test_dtype_given_by_initializer(self):
+        class MyInitializer(object):
+            dtype = 'float16'
+
+            def __call__(self, array):
+                assert False  # never called
+
+        param = chainer.Parameter(MyInitializer())
+        assert param.dtype == np.float16
+
+    def test_dtype_not_given(self):
+        class MyInitializer(object):
+            def __call__(self, array):
+                assert False  # never called
+
+        param = chainer.Parameter(MyInitializer())
+        with pytest.raises(RuntimeError):
+            param.dtype
+
 
 class TestDebugPrint(unittest.TestCase):
Repository: chainer/chainer

Issue:
Improve chainerx import check
#7518 has disabled the following:
1. Check out the source code and `import chainer` without pip install.
2. pip install chainer in non-editable mode (with/without chainerx) and import chainer from the source root directory.

\1. should be supported. In 2., we should give more comprehensible error.


Discussion:

Resolving patch:
diff --git a/chainerx/__init__.py b/chainerx/__init__.py
--- a/chainerx/__init__.py
+++ b/chainerx/__init__.py
@@ -1,8 +1,22 @@
 import os
 import warnings
 
-from chainerx import _build_info
-
+try:
+    from chainerx import _build_info
+except ImportError:
+    raise ImportError(
+        '''\
+Cannot import chainerx because _build_info.py cannot be found.
+
+The chainer and chainerx module being imported was not correctly \
+installed by `pip install`.
+
+It may be caused by either of the following reasons.
+
+1. You are directly importing chainer source files without installing it with \
+`pip install`.
+2. You installed chainer in non-editable mode (`pip install` without -e) and \
+are importing chainer source files instead of the installed module.''')
 
 if _build_info.build_chainerx:
     from chainerx import _core


Test patch:
Repository: chainer/chainer

Issue:
Difficulty in debugging in `check_type_forward`
It's difficult to debug in `check_type_forward` because you cannot write something like
```py
if x_type.dtype == numpy.float32:
    print("OK")
```
which would result in:
```
RuntimeError: Don't convert Expr to bool. Please call Expr.eval method to evaluate expression.
```

You cannot write something like this either (which is suggested by the error message above):
```py
if (x_type.dtype == numpy.float32).eval():
    print("OK")
```
which would result in:
```
AttributeError: 'bool' object has no attribute 'eval'
```

That's because `check_type_forward` is called twice in different modes, with different way of evaluation.
This is not intuitive.

Related: #6414 (reverted in #7772)


Discussion:
You can use `type_check.eval()`.
Error message can be improved.

Resolving patch:
diff --git a/chainer/utils/type_check.py b/chainer/utils/type_check.py
--- a/chainer/utils/type_check.py
+++ b/chainer/utils/type_check.py
@@ -208,8 +208,9 @@ def __nonzero__(self):
         # `z == w`. It is confusing.
         # See also:
         # https://docs.python.org/3/library/stdtypes.html
-        msg = ('Don\'t convert Expr to bool. '
-               'Please call Expr.eval method to evaluate expression.')
+        msg = ('An Expr instance cannot be evaluated as bool. '
+               'Please use chainer.type_check.eval() to evaluate an '
+               'expression.')
         raise RuntimeError(msg)
 
     def __bool__(self):


Test patch:
Repository: chainer/chainer

Issue:
`static_graph` documentation is broken
https://docs.chainer.org/en/v7.0.0b2/reference/static_graph.html

Code example is broken.


Discussion:

Resolving patch:
diff --git a/examples/static_graph_optimizations/mnist/train_mnist.py b/examples/static_graph_optimizations/mnist/train_mnist.py
--- a/examples/static_graph_optimizations/mnist/train_mnist.py
+++ b/examples/static_graph_optimizations/mnist/train_mnist.py
@@ -4,7 +4,13 @@
 to support the static subgraph optimizations feature. Note that
 the code is mostly unchanged except for the addition of the
 `@static_graph` decorator to the model chain's `__call__()` method.
+
+Note for contributors:
+This example code is referred to from the documentation.
+If this file is to be modified, please also update the line numbers in
+`docs/source/reference/static_graph.rst` accordingly.
 """
+
 from __future__ import print_function
 
 import argparse


Test patch:
Repository: chainer/chainer

Issue:
`chainerx.to_numpy` is not documented
`chainerx.to_numpy` is not documented.


Discussion:

Resolving patch:
diff --git a/chainerx/_docs/__init__.py b/chainerx/_docs/__init__.py
--- a/chainerx/_docs/__init__.py
+++ b/chainerx/_docs/__init__.py
@@ -7,6 +7,7 @@
 from chainerx._docs import context
 from chainerx._docs import device
 from chainerx._docs import routines
+from chainerx._docs import utils
 
 
 def set_doc(obj, docstring):
@@ -20,5 +21,5 @@ def set_doc(obj, docstring):
 
 
 def set_docs():
-    for m in (array, backend, backprop, context, device, routines):
+    for m in (array, backend, backprop, context, device, routines, utils):
         m.set_docs()
diff --git a/chainerx/_docs/utils.py b/chainerx/_docs/utils.py
new file mode 100644
--- /dev/null
+++ b/chainerx/_docs/utils.py
@@ -0,0 +1,18 @@
+import chainerx
+from chainerx import _docs
+
+
+def set_docs():
+    _docs.set_doc(
+        chainerx.to_numpy,
+        """to_numpy(array, copy=True)
+Converts a ChainerX array to NumPy
+
+Args:
+    array (~chainerx.ndarray): ChainerX array.
+    copy (bool): If ``True``, a copy is always made. Otherwise, the resulting
+        array may be aliased with the input array.
+
+Returns:
+    numpy.ndarray: NumPy array.
+""")


Test patch: