How To Host Your Own Python Models | by Emmett Boudreau | Towards Data Science

Under the topic of deployment with machine-learning, there are a lot of things to consider and a lot of different options that will provide you with a different result. Firstly, there are a lot of standard VPS and semi-shared hosts that you could go with for deploying your models. These are usually not great options for a beginner but allow for more freedom in what applications are used and how the file-system is arranged. Another option is, of course, a deployment solution like AWS or Google Cloud. While these are both great options, neither of them will give you the price-to-performance ratio and convenience of hosting a server yourself. Although there are of course drawbacks to hosting a server yourself, it is certainly a great option if you happen to have good and reliable internet and an extra computer lying around.
When you’re running your own server, something very important to consider is what it is actually going to run on. You might not have a full-blown server sitting in your closet like many computing maniacs, but that is fine. For most applications, it’s unlikely that you’re going to have a need for 128GB of memory and a ridiculously high core count processor. For example, if you wish to only deploy a few endpoints, maybe serve some static files, you could go with essentially any computer with more than 1GB of memory.
That being said, another thing you’ll want to consider is heat. How hot does it get where you live, and do you have centralized air-conditioning? It’s important to keep your server components cool, and server processors generate a lot of heat and will require a lot more environmental cooling than your average computer. As a result, you could even have a server but not want to use it because of the heat in your environment.
Another thing that you might want to consider is power consumption. After all, if you’re not using all of your server’s power, but running a high TDP processor, it might not even be cost-effective to host it yourself — you might end up paying more for electricity than you bargained for.
A great suggestion that I have when you don’t need all that much power is to use a laptop. Many people have an old I-7 or I-5 laptop with about 4GB of memory sitting around somewhere, and these are a great option for a simple server that can perform a lot of operations for incredibly cheap. Laptop processors have dramatically lower TDP compared to desktop processors, and as a result, they don’t generate nearly as much heat or use very much power.
The first thing you’re going to need to do in order to serve any sort of files to the world wide web is set up port forwarding. Port forwarding allows your local connection to your internet service provider and network to be not only incoming from networks, but also outgoing into networks. Web-pages and websites are always configured to run on port 80. Sometimes port 80 acts as a socket port for a port like 8000, for example, and this is precisely how your model is likely going to use your ports.
A default gateway is the default device IP that all of the other devices are going to use in order to pass through to your ISP. In other words, the default gateway of your network is your router’s local IP address. On Linux, you can obtain this information using
route -n
On Windows, you’re going to first need to run command prompt as administrator. Next, you’ll simply type
ipconfig
into the command prompt, return, and then search for default gateway in the list of values provided. Grab the default gateway for your network and paste it into your web-browser. This should bring you to the login page for your router, which you will of course now need to login to.
After logging in, your router’s setup will almost certainly vary, so it will be difficult to explain where port forwarding is going to be on every single router. Usually, it is either in its own tab or under the “ advanced” tab.
I’m not really sure why, but for some reason, it is likely your network is going to request an application name for forwarding your ports. This can be anything, it doesn’t really matter at all what the application is called. The only purpose this serves is for organization is my guess. You will want to forward ports 60–82 and ports 8000–8030. As to the question of TCP versus UDP, TCP is slower but will eat a lot less bandwidth, whereas UDP is fast but can cause unstable connections when used sparingly. To be clear, UDP is usually used for things that need a high-pinging connection like video-games and things of that nature. So for hosting a website, it’s probably a good idea to just use TCP. I usually just forward for both TCP and UDP, though, because the protocol doesn’t really take effect anywhere in this regard. The protocol is determined by the application, not the router, so all you would be doing by selecting TCP or UDP here is limiting your ability to use one or the other.
Next, you will, of course, need a domain. This is a self-explanatory process —
buy a domain.
Alternatively, you could always just use your server’s public IP address as your domain, as well. The next thing you will need to do is get your domain name server off of your router. This will let the domain registry know where to send incoming connections. You will need to then put this into the name-server on your chosen domain registration site.
Before getting into actually creating and deploying an endpoint, it’s important to test your server to see if it actually works. For this article, I’m going to be using Ubuntu as it is absolutely dominating the server market and beginner-friendly compared to RHEL or Arch servers. After connecting our new server to wifi, we’re going to want to either use dig or curl to get our public address. The problem with dig is that it often relies on name-servers that might identify you by your hardware address, not your IP address, so I usually do:
curl ifconfig.co
Now that we have the IP of our server, copy it into another computer. This is a public IP address, so it won’t work through a local area connection. In other words, we are not making a request to a local server, so we wouldn’t get a local server — so it doesn’t matter if the computer is on the same network or not, this will not work if the ports are not forwarded properly. Now back on the server run
python -im http.server
This will start up a development server that will serve the files stored on your machine statically. Now go ahead and navigate to that public IP address on your other computer, and if you end up in ~/ on your server, congratulations!
In order to set up the virtual part of your server, your first going to need to have an idea of what options you have to do so. The two biggest and most used options are NGINX and Apache. I published an article a while ago which compared the two if you’d like to learn more about them individually to make your decision:
medium.com
You’re also going to need to use Supervisor and set one up to run your server. I’m not going to be including configurations here, as this article is more networking-based, but here are articles that will walk you through how to deploy both Gunicorn3 (Python production servers) and Genie (Julia production servers) with NGINX and Supervisor:
towardsdatascience.com
towardsdatascience.com
And if you wanted to serve static files, I have got you covered for that, as well:
medium.com
The last step I would take in this project is installing Openssh on both machines with
(my daily machine runs Fedora)sudo dnf install openssh-client(The server runs Ubuntu)sudo apt-get install openssh-server
This will allow you to connect to the server while it is headless and not connected to any peripherals. This means that you will be able to log into it without ever needing to actually use the computer itself, which is pretty handy! Another cool thing is that you can also login to your public IPV4 address from anywhere in the world using SSH, as well, as long as the SSH port, port 22, is forwarded as well.
I’d say that depending on your situation, and what exactly it is that you want to do with your server, running your own server off your router can be a fantastic and incredibly cost-effective approach. Running your own server allows you to manage your own hardware, rather than rent it. Using a low TDP processor can really save you a few bucks as well, as you could end up spending pennies in power each month to keep your websites and endpoints up. Need more storage? Buy a bigger hard-drive. Need more memory? Buy more memory. It is simple and cuts out an often unwanted middleman who is running your website from behind the curtain, giving you the freedom and the flexibility to take your website anywhere!