Running computational tasks indefinitely on the cloud
Tags: cloud, devjournal, python, machinelearning
Published: 2021-06-13 04:11:23

I have been running my most resource-intensive computations on the cloud (that is, on commercial external servers) rather than on using the CPUs from my personal computer. This is advantageous for many reasons such as liberating your computer from resources and in doing so, avoiding lagging or other undesired effects. These 2 reasons are most unique to the application: 
1. Computational tasks can be kept running in the cloud even after your computer is turned off. 
2. If more cloud resources are needed, you can upgrade your resources while keeping your same virtual environment at all times (i.e. programs / applications installed on the cloud space)

I have outlined the process to sending your calculations to the cloud with DigitalOcean as well as other useful features you may use for cloud computing below. 

# Outline 
1. [Sign-up to DigitalOcean](#link1)
2. [Python Users](#link2)
3. [Installing Git on droplet](#link3)
4. [Transferring files between droplet and computer](#link4)
5. [Run processes in the cloud even after turning off your computer](#link5)
6. [How to take a snapshot of your created droplet](#link6)
7. [Machine Learning: Installing Tensorflow](#link7)

Let's begin.

## Sign-up to DigitalOcean <a name="link1"></a>
[Referral link: Sign up with a $100 credit over 60 days through here](https://m.do.co/c/b86f53f9a50f)

I have used **DigitalOcean** virtual machines known as **droplets** for this cloud computing activity. I consider the DigitalOcean platform quite user-friendly and their pricing of droplets is also very reasonable and flexible; the **cheapest droplet**, for instance, will only cost about **$5 a month** and if you decide to **only use it for 2 days**, the price would be around $(2/30) x 5 = **33 cents TOTAL** and **not $5** (as long as the droplet is destroyed after the 2 days, see more on this below) You may click on the link above to sign up

<a href="https://www.digitalocean.com/?refcode=b86f53f9a50f&utm_campaign=Referral_Invite&utm_medium=Referral_Program&utm_source=badge"><img src="https://web-platforms.sfo2.digitaloceanspaces.com/WWW/Badge%202.svg" alt="DigitalOcean Referral Badge" /></a>

## Python users: Installing Anaconda python on droplet <a name="link2"></a>
https://www.digitalocean.com/community/tutorials/how-to-install-the-anaconda-python-distribution-on-ubuntu-20-04

Once you have the droplet, I recommend installing the Anaconda suite which comes with several python modules pre-packaged. The tutorial above allows you to do so with command lines alone. 


## Installing Git on droplet <a name="link3"></a>
https://www.digitalocean.com/community/tutorials/how-to-install-git-on-ubuntu-20-04#setting-up-git

Git may already be installed in your DigitalOcean droplet, but above is the tutorial nonetheless. Git can be used to communicate with your **Github** repository, so that Python programs developed in your computer can be pulled to your droplet, and viceversa. (P.S. You may also need a Github account)


## Transferring files between droplet and computer <a name="link4"></a>
https://docs.digitalocean.com/products/droplets/how-to/transfer-files/

This approach uses **Filezilla** to do this. I use this in the case my programs generate files and I need to transfer them from droplet to computer. 

## Use screen to run process on droplet even after turning off your computer or exiting ssh <a name="link5"></a>
https://www.digitalocean.com/community/tutorials/how-to-install-and-use-screen-on-an-ubuntu-cloud-server

Screen is a program which allows you to do just that. You may need to install it INSIDE your droplet (i.e. ssh  protocol) and then call "screen". After calling screen you can run your program (e.g. "python myprogram.py") and then press "Ctrl+A", followed by letting go off the keys, followed by pressing "D" to detach the screen.

Then you may exit the droplet or even turn off the computer and the python process will keep on running! To access again, access the droplet, then type "screen -r" to re-attach the screen and see how far your process has been running on the server. 


## How to take a snapshot of created droplet <a name="link6"></a>
https://docs.digitalocean.com/products/images/snapshots/how-to/snapshot-droplets/

In case you have been using an expensive droplet and you do not want to be charged for it for a period of time, you may need to destroy it. No worries! If you want to "revive" your droplet later on preserving all its installed programs and environment*, you may do so by taking a snapshot. Click on the link above for this option. 


##  How to install TensorFlow on droplet <a name="link7"></a>
https://www.digitalocean.com/community/tutorial_collections/how-to-install-and-use-tensorflow

If you're ready to take your droplet to the next level, machine learning, then this tutorial may be just for you. The current recommendations are to install TensorFlow in a droplet with **at least 4GB RAM**.

Let's say you created your whole development environment on a smaller droplet. That's ok. You may take a snapshot of the smaller droplet and then re-execute it on a larger memory droplet. This strategy may also be useful if you would want to take time setting up your development platform on a smaller droplet to spend less money before moving to the larger one. 

# Conclusion

DigitalOcean droplets are designed to be suitable for developers and small applications. Their application to cloud computing goes beyond python, as it can be extended to as many programming languages as can be run on a computer.