10-Step guide to schedule your script using cloud services (for free) | by Ekapope Viriyakovithya | Towards Data Science

Well..., you can run it locally and I believe many people probably know task scheduler function if you are Windows users. We set the trigger time and conditions, the script will be run based on our pre-defined criteria. However, the drawback on this is that you need to keep your computer ‘power on’ at the scheduled time (if not all the time).
I encountered this problem when I wanted to run my tiny script to retrieve weather forecast every morning. It is not practical for me to keep my laptop power on, just to run this simple task on daily schedule.
Please consider this as the complimentary guide for Dashboarding with Notebooks: Day 3 / Day 4. The full credit goes to Dr. Rachael and Kaggle team. Thanks for their efforts in the original content, they also published live-streaming recorded here.
This article intends to help someone who is not a hardcore programmer and does not familiar with the file system in Bash and cloud services. Therefore, I included a lot of screenshots and detail explanations here.
Kaggle account, we will use the kernels to host and run our Python script.PythonAnywhere account, we will use the task scheduling to trigger our script hosted on Kaggle.
Kaggle account, we will use the kernels to host and run our Python script.
PythonAnywhere account, we will use the task scheduling to trigger our script hosted on Kaggle.
Setup Kaggle account and go to ‘Kernels’ tab, then ‘New Kernel’. You can select Script or Notebook.
Setup Kaggle account and go to ‘Kernels’ tab, then ‘New Kernel’. You can select Script or Notebook.
It will redirect to the console, with settings tab. Here, you can upload data, select language (Python/R /RMarkdown), add custom packages and also set GPU/Internet option for your script here.
Save and commit your code then return to the Kernels page, you will see your work here. In this example, my kernel name is ‘WeatherAlertFB’.
Now, your script is ready...
2. Go to the ‘Account’ tab of your user profile, scroll down until you find ‘Create API Token’. This will download of kaggle.json, a file containing your API credentials. More detail about Kaggle API here.
Open your kaggle.json file with notepad.
Add the line "proxy": "http://proxy.server:3128" to the end of your .json file. It should look like this.
3. Register to PythonAnywhere. In your Dashboard page, ‘New console:’ section, click $ Bash. You will see new Bash console created. Run the command pip install kaggle --user to install the Kaggle API.
Upload your kaggle.json file. Check that your file has been uploaded by running ls. You should see kaggle.json listed.
4. Make .kaggle directory;
mkdir --parents /home/[Your_Python_Anywhere_Username]/.kaggle
move the kaggle.json file by running the command;
mv kaggle.json /home/[Your_Python_Anywhere_Username]/.kaggle/kaggle.json
5. Make your credentials private by running; chmod 600 /home/[Your_Python_Anywhere_Username]/.kaggle/kaggle.json
6. You can search for your kernels, ranked by how recently you ran them, by running; kaggle kernels list --user [YOUR KAGGLE USERNAME HERE] --sort-by dateRun
Pull a copy of your kernel by running; kaggle kernels pull [AUTHOR'S KAGGLE USERNAME]/[KERNEL SLUG FROM URL] -m
Running ls. You should see that the notebook file and metadata file are both in your current working directory.
7. Push your notebook by running kaggle kernels push
This command will re-run the script and commit new version on Kaggle.
8. Create new text file using any text editor, put below code in it.
#!/usr/bin/env bashkaggle kernels pull pull [KAGGLE USERNAME]/[KERNEL SLUG] -mkaggle kernels push -p /home/[Your_Python_Anywhere_Username]
Save as ‘run_kernel.sh’, then upload to PythonAnywhere.
9. Run the command chmod u+x run_kernel.sh.
Then run your script by running ./run_kernel.sh. This should update your kernel on Kaggle.
10. Go the Task tab on PythonAnywhere to schedule your script.
Times are UTC, not your local time zone. Enter the file path to your .sh script. /home/[Your_Python_Anywhere_Username]/run_kernel.sh, hit “Create” button.
And...it’s done! Your script will run daily at the specified UTC time.
Please note that for the free tier, your script will stop running after a month unless you manually renew it, so be careful about that.
Thank you for reading. Please give it a try, have fun and let me know your feedback! Get in touch with me to discuss your inspiring projects.
If you like what I did, consider following me on GitHub, Medium, and Twitter. :P