Repository: vyperlang/vyper

Issue:
Malformed data when passing a tuple or struct that contains an array
### Version Information

* vyper Version (output of `vyper --version`): `0.1.0-beta16` (current master)
* OS: linux
* Python Version: 3.7

### What's your issue about?

As reported by @michwill on Gitter, passing a struct between functions, where the struct contains an array, results in malformed data. WIth the following contract:

```python
struct A:
    many: uint256[4]
    one: uint256

@private
@constant
def _foo(_many: uint256[4], _one: uint256) -> A:
    return A({many: _many, one: _one})

@public
@constant
def foo() -> A:
    return self._foo([1, 2, 3, 4], 5)

@public
@constant
def bar() -> (uint256[4], uint256):
    out: A = self._foo([1, 2, 3, 4], 5)
    return out.many, out.one
```

Here is the output of each public function:

```python
>>> contract.foo()
((1, 2, 5, 0), 0)
>>> contract.bar()
((0, 0, 5, 4), 3)
```

Observations:
1. Generating the struct within the public function and returning it, or a component of it, works as expected.
2. Passing the struct between public functions of two different contracts works as expected.
3. Structs that do not contain arrays work as expected.
4. Removing the `@constant` modifier has no effect.

### How can it be fixed?
It looks like an off-by-one (or 2?) error to me, but I'm unsure how to fix it.


Discussion:
Do you get the same answers if you reverse the calling order?
The answers seems different if the order is changed in the declaration of struct
I get differently incorrect answers. Placing the non-array before the array does NOT result in the non-array being correctly located so I think the issue is with the struct itself, not just how the array is handled inside the struct...  if that makes sense.
Yes, that's exactly what I am observing, too
Any hints of why could that be happening?
Been looking at this some more.  The issue isn't only with structs, it also happens when passing a tuple that contains an array.

Looking at the IR it seems that there's an extra `MSTORE` which overwrites one of the return values. The section of the code where the instruction is generated is: [vyper/parser/self_call.py#L227](https://github.com/vyperlang/vyper/blob/master/vyper/parser/self_call.py#L227)

I think this is a lingering issue from #1838 - @charles-cooper any chance you could take a look?
Good call, seems like we can't draft a new release until we figure out this issue
@iamdefinitelyahuman thanks - I will take a look later today or tomorrow.
Any luck?
hey @michwill - looking into this now
probably declared victory on https://github.com/vyperlang/vyper/issues/1839 and #1409 a little too soon
> Been looking at this some more. The issue isn't only with structs, it also happens when passing a tuple that contains an array.
> 
> Looking at the IR it seems that there's an extra `MSTORE` which overwrites one of the return values. The section of the code where the instruction is generated is: [vyper/parser/self_call.py#L227](https://github.com/vyperlang/vyper/blob/master/vyper/parser/self_call.py#L227)
> 
> I think this is a lingering issue from #1838 - @charles-cooper any chance you could take a look?

@iamdefinitelyahuman which extra `mstore` do you see?
i see the issue - call-to-self code assumes that tuple/struct members will always be 1 word

https://github.com/vyperlang/vyper/blob/6ea96f0903e5a6a7f3eb418c3126d3a4d966d395/vyper/parser/self_call.py#L242
So - I could fix the unpacking code in `self_call.py` but I think what I want to do is change our internal calling convention to use unpacked data instead of ABI-encoded data. It will simplify the compiler and reduce code size.

Resolving patch:
diff --git a/vyper/parser/self_call.py b/vyper/parser/self_call.py
--- a/vyper/parser/self_call.py
+++ b/vyper/parser/self_call.py
@@ -220,19 +220,22 @@ def call_self_private(stmt_expr, context, sig):
             elif isinstance(sig.output_type, TupleLike):
                 static_offset = 0
                 pop_return_values = []
-                for out_type in sig.output_type.members:
-                    if isinstance(out_type, ByteArrayLike):
+                for name, typ in sig.output_type.tuple_items():
+                    if isinstance(typ, ByteArrayLike):
                         pop_return_values.append(
                             ['mstore', ['add', output_placeholder, static_offset], 'pass']
                         )
                         dynamic_offsets.append(
-                            (['mload', ['add', output_placeholder, static_offset]], out_type)
+                            (['mload', ['add', output_placeholder, static_offset]], name)
                         )
+                        static_offset += 32
                     else:
-                        pop_return_values.append(
-                            ['mstore', ['add', output_placeholder, static_offset], 'pass']
-                        )
-                    static_offset += 32
+                        member_output_size = get_size_of_type(typ) * 32
+                        pop_return_values.extend([
+                            ['mstore', ['add', output_placeholder, pos], 'pass']
+                            for pos in range(static_offset, static_offset + member_output_size, 32)
+                        ])
+                        static_offset += member_output_size
 
             # append dynamic unpacker.
             dyn_idx = 0


Test patch:
diff --git a/tests/parser/features/decorators/test_private.py b/tests/parser/features/decorators/test_private.py
--- a/tests/parser/features/decorators/test_private.py
+++ b/tests/parser/features/decorators/test_private.py
@@ -1,3 +1,6 @@
+import pytest
+
+
 def test_private_test(get_contract_with_gas_estimation):
     private_test_code = """
 @private
@@ -209,23 +212,6 @@ def added(a: uint256, b: uint256) -> uint256:
     assert c.added(10, 20) == 30
 
 
-def test_private_return_tuple(get_contract_with_gas_estimation):
-    code = """
-@private
-def _test(a: int128) -> (int128, int128):
-    return a + 2, 2
-
-
-@public
-def test(a: int128) -> (int128, int128):
-    return self._test(a)
-    """
-
-    c = get_contract_with_gas_estimation(code)
-
-    assert c.test(11) == [13, 2]
-
-
 def test_private_return_bytes(get_contract_with_gas_estimation):
     code = """
 a_message: bytes[50]
@@ -562,3 +548,79 @@ def outer(xs: bytes[256] = b"") -> bool:
 
     c = get_contract(private_test_code)
     assert c.outer()
+
+
+tuple_return_sources = [
+    ("""
+@private
+def _test(a: int128) -> (int128, int128):
+    return a + 2, 2
+
+
+@public
+def foo(a: int128) -> (int128, int128):
+    return self._test(a)
+    """, (11,), [13, 2]),
+    ("""
+struct A:
+    many: uint256[4]
+    one: uint256
+
+@private
+def _foo(_many: uint256[4], _one: uint256) -> A:
+    return A({many: _many, one: _one})
+
+@public
+def foo() -> A:
+    return self._foo([1, 2, 3, 4], 5)
+    """, (), ([1, 2, 3, 4], 5)),
+    ("""
+struct A:
+    many: uint256[4]
+    one: uint256
+
+@private
+def _foo(_many: uint256[4], _one: uint256) -> A:
+    return A({many: _many, one: _one})
+
+@public
+def foo() -> (uint256[4], uint256):
+    out: A = self._foo([1, 2, 3, 4], 5)
+    return out.many, out.one
+    """, (), [[1, 2, 3, 4], 5]),
+    ("""
+@private
+def _foo() -> (uint256[2], uint256[2]):
+    return [1, 2], [5, 6]
+
+@public
+def foo() -> (uint256[2], uint256[2], uint256[2]):
+    return self._foo()[0], [3, 4], self._foo()[1]
+    """, (), [[1, 2], [3, 4], [5, 6]]),
+    ("""
+@private
+def _foo(a: int128, b: int128[3], c: int128[3]) -> (int128[3], int128, int128[3]):
+    return c, 4, [b[1], a, b[0]]
+
+@public
+def foo(a: int128, b: int128[3], c: int128[3]) -> (int128[3], int128, int128[3]):
+    return self._foo(a, b, c)
+    """, (6, [7, 5, 8], [1, 2, 3]), [[1, 2, 3], 4, [5, 6, 7]]),
+    ("""
+@private
+def _foo(a: int128, b: int128[3], c: int128[3]) -> (int128[3], int128, int128[3]):
+    return c, 4, [b[1], a, b[0]]
+
+@public
+def foo(a: int128, b: int128[3], c: int128[3]) -> (int128[3], int128, int128[3]):
+    return c, 4, self._foo(a, b, c)[2]
+    """, (6, [7, 5, 8], [1, 2, 3]), [[1, 2, 3], 4, [5, 6, 7]]),
+
+]
+
+
+@pytest.mark.parametrize("source_code,args,expected", tuple_return_sources)
+def test_tuple_return_types(get_contract, source_code, args, expected):
+    c = get_contract(source_code)
+
+    assert c.foo(*args) == expected