How to write a bash script for your reproducible data analysis pipeline (CC247)
Channel: Riffomonas Project
Date: 2022-09-12

Bash and Zsh are tremendously powerful tools for doing file operations from the command line. You can combine multiple bash programs in a script to automate your data analysis pipelines. In this Code Club, Pat uses bash scripts and a driver script to download files from a website and will share some tricks for using wget and tar within his bash scripts. The goal of this project is to highlight reproducible research practices using a number of tools. The specific output from this project will be a map-based visual that shows the level of drought across the globe.

You can find my blog post for this episode at https://www.riffomonas.org/code_club/2022-09-12-bash-scripts.

#bash #wget #tar #R #Rstats

Support Riffomonas by becoming a Patreon member!
https://www.patreon.com/riffomonas

Want more practice on the concepts covered in Code Club? You can sign up for my weekly newsletter at https://shop.riffomonas.org/youtube to get practice problems, tips, and insights.

If you're interested in taking an upcoming 3 day R workshop be sure to check out our schedule at https://riffomonas.org/workshops/

You can also find complete tutorials for learning R with the tidyverse using...
Microbial ecology data: https://www.riffomonas.org/minimalR/
General data: https://www.riffomonas.org/generalR/

0:00 Introduction
1:09 Navigating NOAA's GHCN data portal
3:47 Creating an executable bash script
6:27 Creating a bash script to download data with wget
14:15 Creating command line arguments for bash scripts
17:43 Creating a driver script for pipeline
19:57 Add another step to pipeline with tar