Creating data-driven fabric designs // Part 1: Scraping & data preparation with NodeJS
Channel: Creating with Data
Date: 01/06/2023

This screencast is part 1 of 2. It shows how to scrape imagery and data from the USDA's online collection of pomological watercolours - https://naldc.nal.usda.gov/usda_pomological_watercolor?per_page=100

This sets up an images-set we can use for the generative pattern designer described here: https://creatingwithdata.com/fruit-of-the-algorthim-a-generative-fabric-designer/

Here's a link to the revamped generative shirt designer: https://creatingwithdata.com/fruit-of-the-algorthim-a-generative-fabric-designer/

The scraping is done using NodeJS with the node-fetch and cheerio libraries. Once all the images and accompanying data are gathered we further process the images by cropping them slightly (10 pixels each way). To this end I demonstrate using ImageMagick's 'chop' command.

See blog post for further notes and code - https://creatingwithdata.com/creating-data-driven-fabric-designs-with-javascript-p1-scraping-and-data-preparation-with-nodejs/

0:00 Intro
3:08 Exploring the watercolour collection
5:59 Project init and starter code
10:29 Parsing results with Cheerio
18:15 Filtering results and parsing common name from results
22:28 Paging through and saving results
24:34 Bulk cropping images with ImageMagick