Open a Dialogue About Your Data
By: Xristos Katsaros
Tags: Data, Music, Open Data, Version Control, Issues

Open a Dialogue About Your Data

Ever wonder who has performed the most at your favorite venue? Or which artist has been booked the most in your city? Maybe you suspect there is some unfair favoritism at play at a specific venue? As a fan and a musician, I’ve wondered about these questions myself, so I started to put together a dataset that would help answer them.

The first place that came to mind when thinking about where to get this data, was Resident Advisor. I’m a big fan of electronic music, and Resident Advisor has been a major hub for electronic music news, reviews, and events going as far back as 2001. So I scraped the event pages for lineup information and made a count of how many times an artist appeared for each venue and city.

After I gathered data on some of the more popular venues across Europe, I made the CSV into a Qri dataset so I could start keeping track of the data as the project moves forward, and to make it easy for future collaborators to pull down the latest version of the dataset. Along with those benefits, there’s also a new feature on Qri Cloud that immediately added more value to the data and the project as a whole: issues!

GIF as Demo of Opening an issue on Qri

Qri Issues, and the discussions they organize, stay with the dataset

There’s a mantra in open-source software, “Many eyes make all bugs shallow,” and that applies to data too. Data doesn’t necessarily have bugs, but there are sometimes problems with datatypes, missing values, duplicate rows, or other inconsistencies. Having your data open to the public also allows for others to make suggestions for a new column, a new application for the data, or methodology in creating the data. The dataset I made for this project had 2 issues filed, and they helped me improve the data and my workflow.

How Issues Improved my Dataset

The first issue pointed out that row 1 repeated the column names in the body and in the preview on Qri Cloud, and came with an easy solution. After following their instructions, the dataset was fixed, and looked great. I also ran into this problem of a repeated header again and was able to apply the same fix.

Issue that points out repeated header row

The second issue was a question to the methodology of the venues I selected. It was really interesting, because it led to discussion of an entirely new dataset to compliment this one, which can be found here.

Suggestion for an additional dataset

Where the data goes from here

That started turning some gears in my brain: I could utilize the Qri python package in the scraper I made to collect the data. From the suggested clubs dataset, I had the club IDs that Resident Advisor uses for the club pages. This resulted in my script making fewer requests to their site, speeding up the data collecting process. I will go into more details on this in my next blog. For now, feel free to scope out tthe dataset, download it, and make it your own:

Datasets on Qri are more than just the CSV file — there are readme, meta, and structure files that add more context to your work. Issues can help improve all of these components and make your data more accessible and organized, whether it’s for yourself, a lab partner, coworker, or collaborator.