The Shifting Definition of Newsworthiness
Tags: Python, News, Analytics, Data Visualization, New York Times

The Politics of Intolerance

So, what was the Times covering in 2015? What was it covering then that it isn’t now, and what is it covering now that it wasn’t then?

We can look at this in a few ways. In this piece I’m going to look at the geography of coverage (at least, what I could find from some simple text mining), but some others include text analysis of both headlines and keywords, or looking at bylines and representation at the Times in more detail.

Technical notes

I’ve generated the data under analysis by parsing bylines in archived articles. This is, as far as I can tell, the easiest way to get geography from historical articles. The Times does offer a wire service that includes geographical information, but what I’m interested in here is historical data, not incoming stories.

The script I use to parse geography relies on the journalistic convention of the dateline — something like KABUL, Afghanistan; LONDON; or, as in one recent article, CENTER OF THE WORLD, Ohio (in an article about Ohio State football). Given the available data, this seemed like the best (/only) way to get the information we’re interested in.

A few caveats to this approach: articles that have a dateline like this are less and less common, not only at the Times but elsewhere in the news media as well. In many cases, we also miss out on geographic information from the dateline because there’s no geographic focus to the article — think opinion pieces, magazine posts, podcasts and other multimedia posts, as well as “geographically diffuse” stories about things like the coronavirus, or Biden’s VP pick.

More specifically, if we look at the number of articles with extractable geographic information by year, we have the following:

Not surprising, given that we know the number of stories overall is decreasing. But the same is true if we look at these kinds of stories as a percentage of all stories published, we get a clearer picture:

Looks like the percentage of stories with identifiable geography in the byline has remained roughly constant with a dip in 2018.

This suggests a few things: first, that there may have been a shift to more named places after Trump’s election (2017 has the highest percentage of “geographic” bylines at 28.5%) but it wasn’t major (the average was 26% of bylines containing geographic information). Second, though we’ve seen the Times drop the overall number of articles, they don’t seem to have shifted significantly away from “locatable” stories.

There’s one more thing we should look at with these aggregate measures: which sections are the most “diverse?” By “diversity” here I mean place diversity, without reference to the subjects or authors of the pieces in question (that material should be saved for a future analysis). I’m going to exclude foreign desk coverage here—it’s natural that a desk with a remit to cover stories from all over the world would feature articles from… all over the world—and focus on domestic stories. So, what does the diversity by news desk look like?

And finally, what is the count of unique places by year?

(Note that the 2020 number is low because we’re only halfway through the year. On a pro-rated basis, 451 places would translate to about 770 locations, putting 2020 in line with 2019.) It would seem that the Weiss was wrong about the Times’ direction (or at least, her hire didn’t exactly presage a new era of geographic diversity in their pages).