Repository: web2py/web2py

Issue:
python 3 upload getting uft8 invalid continuation byte error
Environment: 2.15.2-stable+timestamp.2017.07.19.12.18.41 (Running on Rocket 1.2.6, Python 3.6.2)

used example from http://web2py.com/books/default/chapter/29/07/forms-and-validators#SQLFORM-and-uploads

```
db.define_table('person',
    Field('name', requires=IS_NOT_EMPTY()),
    Field('image', 'upload'))
```

```
def display_form():
   record = db.person(request.args(0))
   form = SQLFORM(db.person, record, deletable=True,
                  upload=URL('download'))
   if form.process().accepted:
       response.flash = 'form accepted'
   elif form.errors:
       response.flash = 'form has errors'
   return dict(form=form)
```

Upload a simple text file, all is well.  Upload a binary file of any kind (PDF, MP4, etc.), and get an error like the following:

```
<class 'UnicodeDecodeError'> 'utf-8' codec can't decode byte 0xd3 in position 33402: invalid continuation byte

Traceback (most recent call last):
  File "[redacted]\web2py\gluon\restricted.py", line 219, in restricted
    exec(ccode, environment)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd3 in position 33402: invalid continuation byte
```

Edit: fixed code block formatting


Discussion:
Can't reproduce this. This works fine for me.

I simply copy pasted your code into the welcome application and run with python 3.   
Thanks for trying to reproduce it..  I can't imagine why it would matter, but I tried copy/pasting the code into the welcome app instead of my test app, same result.  Tried mysql as the database, same result.  But it's possible I've messed something up in my web2py setup, and I was using pycharm as my IDE.  So I started from scratch and used only the built in web2py editor.  Here are my steps to reproduce.

- download and unzip web2py source from website
- install python 3.6.2 64 bit for Windows
- cd c:\python36
- python [path to web2py.py]
- put in password, start server, welcome app loads in Chrome browser
- this app -> db model
- log in
- copy/paste the following to the bottom of db.py

```
db.define_table('person',
    Field('name', requires=IS_NOT_EMPTY()),
    Field('image', 'upload'))
```

- click save
- hover over left hand toolbar, click on controllers -> default.py
- copy/paste the following after index() and before user()

```
def display_form():
   record = db.person(request.args(0))
   form = SQLFORM(db.person, record, deletable=True,
                  upload=URL('download'))
   if form.process().accepted:
       response.flash = 'form accepted'
   elif form.errors:
       response.flash = 'form has errors'
   return dict(form=form)
```

- click save
- go to http://127.0.0.1:8000/welcome/default/display_form
- type "ascii file" in "name" box
- click choose file button, select a TEXT ONLY file that contains no characters over chr(127)
- click submit
- it works, world peace ensues
- type "binary file" in "name" box
- click choose file button, select a binary file of some kind (PDF, JPG, MP4, etc.)
- click submit
- same error as in original post (though not necessarily with the same "byte" and "position" values, of course), mass hysteria ensues

I should emphasize that I'm using python 3.6.2 for Windows specifically, and probably more importantly, this problem only arises when trying to upload binary files.
Is it possible this was already fixed in trunk, I was using the github version, can you try that?  
I tried that based on your suggestion, but got the same error.  However, admin tool shows version 2.15.2-stable+timestamp.2017.07.19.01.21.31, which I find odd, since that's actually older than my 2.15.2 install.  I just did a git clone https://github.com/web2py/web2py.git followed by git submodule update --init --recursive.  This seems to be the right place to get the latest version, and I even compared the code I got from github versus some changes merged within the last couple days, and all seemed to be correct.
@bjquinn I managed to reproduce it. Apparently you need to be using the generic.html view for this to happen and I had created a display_form.html view. Thanks!!

Apparently it's caused by =response.toolbar()

EDIT: Namely the BEAUTIFY(request) it does

EDIT2: This problem actually also happens in python 2 although it does not cause a ticket error because the problem is that BEAUTIFY is showing the file contents (which could be many MB) when it shouldn't.

EDIT3: Fount it! :)

Resolving patch:
diff --git a/gluon/html.py b/gluon/html.py
--- a/gluon/html.py
+++ b/gluon/html.py
@@ -2431,7 +2431,7 @@ def __init__(self, component, **attributes):
         if level == 0:
             return
         for c in self.components:
-            if hasattr(c, 'value') and not callable(c.value):
+            if hasattr(c, 'value') and not callable(c.value) and not isinstance(c, cgi.FieldStorage):
                 if c.value:
                     components.append(c.value)
             if hasattr(c, 'xml') and callable(c.xml):


Test patch: