Repository: vyperlang/vyper

Issue:
Off-by-one error in `zero_pad`
### Version Information

* vyper Version (output of `vyper --version`): 0.1.0b12+commit.8663ac5

### What's your issue about?

Off-by-one error in `zero_pad()`:

https://github.com/ethereum/vyper/blob/ab39d4e9c5168eff2646dd29e23d7212654b636d/vyper/parser/parser_utils.py#L904

Logically, the loop termination condition should use `ge` instead of `gt`.

### How can it be fixed?

It can be fixed by replace `gt` by `ge` in the above, but I'm not sure if such a fix does not affect others.



Discussion:
@daejunpark could you please take a look at https://github.com/ethereum/vyper/pull/1605/commits/5a845710f999e07cea4ea66922b7dbd7adee4586 and let me know if you think it is a reasonable solution?
@charles-cooper thanks for this. It is a really clever trick, and I like it (from my hacker instinct)!

But I have a concern about this kind of off-label uses of opcodes for the long term in general. For example, if people discuss the behavior change of `calldatacopy` in a future hard fork, they may be not aware of this off-label use, and fail to consider/analyze the effect of their change to this.

I have a question. Is the zero padding size always less than 32? (I cannot think of a counterexample.)

If so, you may want to consider to use the `ID` precompiled for the zero padding. That is,
```
CALL <gas> 4 0 <src> <len> <dest> <len>
```
where `<src>` points to the zero bytes that can be preallocated in the first region of the memory.
Although this will consume more gas than the `calldatacopy` approach, but it is an expected use case, so would be safer for the long term.
> If so, you may want to consider to use the ID precompiled for the zero padding. That is,
> `CALL <gas> 4 0 <src> <len> <dest> <len>`

Yes, I've been thinking about this approach for awhile. In this case the zero-pad is always <32 bytes, but having this technique for zeroing arbitrary amounts of memory would be handy. So we could always guarantee that the source is a zeroed block of memory simply by reading from past-the-end of where we have allocated, or some number so large it would be impractical to write to (e.g. 2**32). But I think reasoning about where the past-the-end pointer should be for `CALLDATACOPY` is easier - it is always `CALLDATASIZE`!

> But I have a concern about this kind of off-label uses of opcodes for the long term in general. For example, if people discuss the behavior change of calldatacopy in a future hard fork, they may be not aware of this off-label use, and fail to consider/analyze the effect of their change to this.

I was also concerned about this. However, I feel like the existing semantics were carefully considered and were written that way for a reason. Note the difference in semantics between `CALLDATACOPY` and `CODECOPY` - `CODECOPY` issues a `STOP` if you try to read from past-the-end. So even if it may not be the "intended" use-case for `CALLDATACOPY`, it is not exactly "off-label" either! A change would be quite a significant - almost pathological - change to the EVM and would definitely break the semantics of existing programs. But in any case, I think what we can do is create a new pseudo-opcode `mzero <dst> <len>`  which zeroes out the destination block (analogously to stdlib's `bzero`) and then choose to implement it however we want. Then even if `CALLDATACOPY` is changed for whatever reason, the overhead of swapping out the implementation of `mzero` is fairly low.
@charles-cooper I agree with all of your points, but please note that the overhead of updating the `mzero` implementation may not be low from the contract owners' perspective, because they will need to redeploy the updated bytecode if that happens, and the redeployment may not be easy in certain situations.

@CarlBeek what do you think about this in terms of the deposit contract operation?
FYI, this is also related: https://github.com/ethereum/vyper/issues/1610
> @charles-cooper I agree with all of your points, but please note that the overhead of updating the mzero implementation may not be low from the contract owners' perspective, because they will need to redeploy the updated bytecode if that happens, and the redeployment may not be easy in certain situations.

Thanks for the note. My assumption was that, if the opcode semantics are ever changed, it would be as an EVM version update so that only contracts deployed after `fork_blocknum` would have the new semantics, while existing contracts would retain the old semantics.
I want to do the quick fix for the `gt` issue though since it seems more pressing. As it stands, the loop can write a `\0` byte to one past the end of the bytearray.

Resolving patch:
diff --git a/vyper/parser/events.py b/vyper/parser/events.py
--- a/vyper/parser/events.py
+++ b/vyper/parser/events.py
@@ -108,9 +108,10 @@ def pack_args_by_32(holder, maxlen, arg, typ, context, placeholder,
             typ=typ, location='memory', annotation="pack_args_by_32:dest_placeholder")
         copier = make_byte_array_copier(dest_placeholder, source_lll, pos=pos)
         holder.append(copier)
+        item_maxlen = source_lll.typ.maxlen
         # Add zero padding.
         holder.append(
-            zero_pad(dest_placeholder, maxlen, zero_pad_i=zero_pad_i)
+            zero_pad(dest_placeholder, item_maxlen, zero_pad_i=zero_pad_i)
         )
 
         # Increment offset counter.
diff --git a/vyper/parser/parser_utils.py b/vyper/parser/parser_utils.py
--- a/vyper/parser/parser_utils.py
+++ b/vyper/parser/parser_utils.py
@@ -899,6 +899,8 @@ def annotate_ast(
     RewriteUnarySubVisitor().visit(parsed_ast)
 
 
+# zero pad a bytearray according to the ABI spec. The last word
+# of the byte array needs to be right-padded with zeroes.
 def zero_pad(bytez_placeholder, maxlen, context=None, zero_pad_i=None):
     zero_padder = LLLnode.from_list(['pass'])
     if maxlen > 0:
@@ -906,12 +908,18 @@ def zero_pad(bytez_placeholder, maxlen, context=None, zero_pad_i=None):
         if zero_pad_i is None:
             zero_pad_i = context.new_placeholder(BaseType('uint256'))
         zero_padder = LLLnode.from_list([
-            'repeat', zero_pad_i, ['mload', bytez_placeholder], ceil32(maxlen), [
+            # the runtime length of the data rounded up to nearest 32
+            # from spec:
+            #   the actual value of X as a byte sequence,
+            #   followed by the *minimum* number of zero-bytes
+            #   such that len(enc(X)) is a multiple of 32.
+            'with', '_ceil32_end', ['ceil32', ['mload', bytez_placeholder]],
+            ['repeat', zero_pad_i, ['mload', bytez_placeholder], ceil32(maxlen), [
                 'seq',
                 # stay within allocated bounds
-                ['if', ['gt', ['mload', zero_pad_i], ceil32(maxlen)], 'break'],
+                ['if', ['ge', ['mload', zero_pad_i], '_ceil32_end'], 'break'],
                 ['mstore8', ['add', ['add', 32, bytez_placeholder], ['mload', zero_pad_i]], 0]
-            ]],
+                ]]],
             annotation="Zero pad",
         )
     return zero_padder


Test patch: