Deployment Blues : Why Won’t My Flask Web App Just @#$% Deploy On Google App Engine ?
By: Zhijing Eu
Tags: Deployment, Python, Web App Development, Google App Engine, Flask

Introduction

Fair warning, this article is a bit of a rant and you will probably be rolling your eyes if you are already an experienced DevOps person or familar with Google Cloud App Engine

I will be detailing my journey down the Dev Ops “rabbit hole” and the troubleshooting I went through to deploy a simple Flask Web App online using Google’s App Engine along with any online resources (Stackoverflow posts or other tech blogs) I found useful as a n00b data science enthusiast.

Ultimately I didn’t manage to fix my problem as I ended up deploying NOT ON GOOGLE APP ENGINE but a GOOGLE COMPUTE ENGINE instance (will probably cost me more $ but hey it’s still something I guess)

However I did learn a few things while troubleshooting the other issues so hopefully by writing this up , someone else out there might benefit.

I know Medium.com is NOT StackOverflow but anyone reading this who might have pointers — please drop me a note in the comments.

My GitHub repo here (Note — it already contains the refactored versions of the files after I fixed most of the issues below) :-

Outline

0.Background

In a recent article I shared a Text Analysis Flask App that I was running locally on my own machine.

Unfortunately this is not very useful for others who may want to make use of your machine learning model , code or whatever else without them having to learn python or replicating your development environment.

Therefore , I started exploring my deployment options on how to put the app online. Previously I’ve used pythonanywhere.com on their free tier service. However this time , I decided to try out Google Cloud’s App Engine instead

Based on the documentation and the example tutorial below (please have a read through it before the rest of the article as it will become relevant later), it seemed like a simple enough process…

I got as far as this step. It looked promising as the message said the deployment went through…

Looked simple enough…

Uh-oh.

…but no : (

1.Pickling & Attribute Errors

I did a quick trawl through the error logs in Google Cloud and found this message.

The Model class was for a custom class that used a RandomForestRegression and RandomForestClassifier to predict Big 5 Personality Traits. It was a model that someone had posted online and I only incorporated their pre-trained (and pickled) models into my web-app.

So some stackoverdrive trawling uncovered this nugget:

“When you define a Class directly in a file you run, it’s pickled as coming from the module main. However when you unpickle it in the Flask web app, there is no main module, so the code can’t find the definition”

Turns out this is a fairly common novice mistake covered in a few different articles.

The gist of it is that if you define a class and function in one project (e.g. let’s call it FileA.py ) and try to load it in another project FileB.py , Python will have trouble finding the relevant class in the module " __main__” , which corresponds now with the FileB and not with the FileA where it was pickled.

The reason I never noticed this message when I ran this locally on my machine was because I was always running the app using the >python main.py command. Therefore this portion at the bottom was always getting executed:

if __name__ == "__main__":

main()

However if I were instead of use flask run and set FLASK_APP=main.py I managed to at least replicate the error I saw in Google App Engine’s logs because the flask run command works by importing an app object from your main.py module. This StackOverflow thread probably explains it better….

Thankfully, there is always someone out there who has a solution at hand.. After more digging through StackOverflow, I found a user who posted a very nice Custom Unpickler class

import pickle class CustomUnpickler(pickle.Unpickler):



def find_class(self, module, name):

try:

return super().find_class(__name__, name)

except AttributeError:

return super().find_class(module, name)



pickle_data = CustomUnpickler(open('file_path.pkl', 'rb')).load()

This works because it overwrites the find_class function of pickle.Unpickler and explicitly directs it to look in the current module's namespace so the whole confusion about which "___main___" to refer to.

So now at the very least I could ‘Flask Run’ the app on my local machine without the error. I updated the main.py and uploaded this to Google App Engine and eagerly awaited the results.

ARRRRGGGGGGGGHHHHHH

2.Resolving Library Compatibility Issues

Back to the error logs then — I found this cryptic message.

???

Back to trawling the internet for help , I stumbled onto some relevant github posts:

Apparently , it was because I built my app using Python 3.6.4 but I was using the “free tier” of the Google App Engine that defaults to a Python 3.7 environment and this “typing” package is redundant after Python 3.6 and is only used to backport this functionality in earlier versions.

I don’t even remember ever installing this particular library in my local virtual environment when building my app but sure enough when I used the pip freeze > requirements.txt command to pull out all the relevant packages this was in there.

So a quick deletion and re-upload later, I waited for the results to load.

Noooooooooooo

3. (Explicitly) Adding In The WSGI Server…

I managed to get rid of all the AttributeError messages. The app still doesn’t work but this was progress of sorts.

I went back to the logs and noticed that there was a message that was already showing up earlier but I never looked at it carefully as I was dealing with the Attribute Error issues.

In the example tutorial I was using and even Google Cloud’s own “Hello World” example , you only had to specify Flask in the requirements.txt file and the only thing in the app.yaml file (which is like a config file that tells Google App Engine how to set up the containers) was a ONE LINER runtime:Python37

Of course what I was building was a bit more complex but I thought GAE would “guess” what I was trying to do without me being explicit.

Anyway I went back to my requirements.txt file and added: gunicorn==20.0.4 and also added this line to my app.yaml file: entrypoint: gunicorn -b :$PORT main:app

For good measure to make sure I could “see” in the logs if the application at least loaded up, I wrote in a line at the very end of the main.py file : print(“Hey, At Least I Loaded Up…”)

As you may have expected, things did not go as planned.

Hey — this is new… Which is good ?

and back to the log files…

Not a total loss I guess. At least I know it was doing SOMETHING even if the app wasn’t loading.

4 Memory Loss ?

According to the help files:

Expanding on the memory limit exceeded warning, I had a message that said “While handling this request, the process that handled this request was found to be using too much memory and was terminated. This is likely to cause a new process to be used for the next request to your application. If you see this message frequently, you may have a memory leak in your application or may be using an instance with insufficient memory. Consider setting a larger instance class in app.yaml.”

Problem is, sometime in the midst of fixing all the other errors, I ALREADY upgraded to the BIGGEST instance (i.e F4_1G which has 2048MB) just to rule out any issues with memory so I wasn’t sure how I could set anything bigger

It appeared like I was facing some sort of Memory Leak in the way the app processed information.

I found this thread in StackOverflow where someone wrote a function that helps to identify the “Top” lines in the code that are consuming memory Python’s default tracemalloc module

So I added a custom function based on that StackOverflow post into my code that pulls out the Top 10 memory gobblers…

import linecache

import os

import tracemalloc



def display_top(snapshot, key_type='lineno', limit=10):

snapshot = snapshot.filter_traces((

tracemalloc.Filter(False, "<frozen importlib._bootstrap>"),

tracemalloc.Filter(False, "<unknown>"),

))

top_stats = snapshot.statistics(key_type)



print("Top %s lines" % limit)

for index, stat in enumerate(top_stats[:limit], 1):

frame = stat.traceback[0]

# replace "/path/to/module/file.py" with "module/file.py"

filename = os.sep.join(frame.filename.split(os.sep)[-2:])

print("#%s: %s:%s: %.1f KiB"

% (index, filename, frame.lineno, stat.size / 1024))

line = linecache.getline(frame.filename, frame.lineno).strip()

if line:

print(' %s' % line)



other = top_stats[limit:]

if other:

size = sum(stat.size for stat in other)

print("%s other: %.1f KiB" % (len(other), size / 1024))

total = sum(stat.size for stat in top_stats)

print("Total allocated size: %.1f KiB" % (total / 1024)) # OTHER CODE GOES HERE

# ... if __name__ == '__main__': tracemalloc.start() app.run(debug = True) print('Top prefixes:', most_common)

snapshot = tracemalloc.take_snapshot()

display_top(snapshot)



and it spat this out:-

TOP 10

#1: <frozen importlib._bootstrap_external>:487: 66267.8 KiB #2: gensim\utils.py:1398: 65934.3 KiB

return _pickle.load(f, encoding='latin1') #3: main.py:107: 10321.5 KiB

cv = CustomUnpickler(open('./MTBIModels/cv.pickle', 'rb')).load() #4: core\completer.py:174: 9945.4 KiB

names.append(unicodedata.name(chr(c))) #5: xlrd\xlsx.py:144: 5979.9 KiB

t = elem.text #6: main.py:161: 2161.0 KiB

models['OPN'] = CustomUnpickler(open('BigFiveModels/OPN_model.pkl', 'rb')).load() #7: main.py:164: 2160.9 KiB

models['AGR'] = CustomUnpickler(open('BigFiveModels/AGR_model.pkl', 'rb')).load() #8: main.py:165: 2160.9 KiB

models['NEU'] = CustomUnpickler(open('BigFiveModels/NEU_model.pkl', 'rb')).load() #9: main.py:162: 2160.4 KiB

models['CON'] = CustomUnpickler(open('BigFiveModels/CON_model.pkl', 'rb')).load() #10: main.py:163: 2160.2 KiB

models['EXT'] = CustomUnpickler(open('BigFiveModels/CON_model.pkl', 'rb')).load() 74040 other: 61565.3 KiB Total allocated size: 230817.6 KiB

Looking at the output, the key culprit being the loading of the pre-trained Gensim Doc2Vec model at 65MB worth and another 66MB for importlib to just load up all the relevant modules but Tracemalloc was saying I only used 231MiB worth of memory which should be far below the 2048MB in the F4_1G instance I chose…

Hmmm…that was odd.

5. Deployment On Google Compute Engine

I wasn’t getting anywhere with Google App Engine so I thought I’d try something different and tried to get my flask app running on a GOOGLE COMPUTE ENGINE (GCE) versus Google App Engine.

I followed the instructions in this article :

This is obviously a very crude (and more expensive ) way of deployment and will not scale but I wanted more insight into what was going on.

I finally found out what was going wrong — it wasn’t anything to do with memory leak.

It was because I was using NLTK_data ‘punkt’ and ‘stopwords’ corpora that had been downloaded on my local machine earlier when I was working on another project but I never installed this on Google App Engine !

It was strange that the error message below never appeared in the Google App Engine logs as it came up on my Google Compute Engine command prompt when I tried to flask run the app.

These two lines fixed my app when I ran this on Google Compute Engine

python -m nltk.downloader punkt

python -m nltk.downloader stopwords

So I tried to “replicate” this fix back to the Google App Engine by adding a ntlk_data folder and changing the app.yaml file like so:-

runtime: python37 entrypoint: gunicorn -b :$PORT main:app --timeout 120 instance_class: F4_1G env_variables:

NLTK_DATA: './nltk_data'

Fingers crossed….it should work this time.

FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF

6. Conclusion

Waaaaaah

I know Medium.com is NOT StackOverflow but anyone reading this who might have pointers — please drop me a note in the comments.