Read Less!
Tags: Naturallanguageprocessing, Internet, Fun, Programming, Reading

Everybody says reading is good for you. And it is. But sometimes, authors simply get carried away with their writing and make things much more verbose than they need to be, which means you end up reading a lot of fluff in between useful pieces of text which in turn means you’ve ended up wasting time that could’ve been better spent elsewhere, such as actually reading the important bits of the text and gaining useful information.

And if you haven’t already noticed, a lot of the previous paragraph was just that — fluff. Sorry.

So how do you ensure you aren’t wasting time reading words that have no impact? Well, for starters, you could be picky with your reading lists, choosing only well-edited sources from respectable authors. Or you could simply extract out the useful bits!

Automatic Text Summarization

Instead of reading everything you see, get your computer to feed you only the important bits! Computers can summarize in two ways:

Extractive — An algorithm decides the most important sentences from a piece of text and provides them to you verbatim. Abstractive — This is how people (usually) summarize — using their own words to describe passages. Computers can do this too, thanks to neural nets.

Extractive Techniques

Curious to know how the “important” sentences are identified? Here are some ways. Skip this bit if you just want to read less.

Positional — Just pick out the first and last sentences from each paragraph. Useful for some articles, not so much for others. Luhn’s method — identifies the most important terms (using tf-idf) and picks the sentences with higher concentrations of these important terms. Edmundson method — Basically Luhn, but also uses bonus words that are more important, stigma words that have a negative effect on a sentence’s importance, and null words that don’t have any impact (unlike this article). LexRank — Creates a similarity matrix to represent how close two sentences are to each other. Represent each sentence as a node in a graph with edges based on their similarity. Remove edges with lower weights. Sentences that are connected to several other sentences are more important. Also, a sentence’s importance depends on the importance of other sentences recommending it.

Extract Sum(y) Stuff!

Now that you (maybe) know how the extractive techniques work, let’s do some actual extraction. The sumy library is particularly handy for this.

I’m not going to summarize this for you here, so explore it yourself!

Make it less Cumbersome

sumy can summarize text, hyperlinks, and files. But sometimes you just want to summarize a particular piece of text. You could copy it over to a file and have sumy summarize that, but it’s going to be really tedious if you plan on doing that all the time. Here are some ideas on making it more convenient -

A Chrome Extension! — I’ve written the SumFly chrome extension just for this purpose! Select some text, right-click and hit summarize!

You can get the extension from the Chrome Web Store:

I’ve touched upon extensions here:

And here’s where you can get the source code:

Of course, the actual summarization takes place in a back-end server, and all the extension does is call that API. Writing the API shouldn’t be too hard — just refer to the sumy docs. And feel free to modify things as you please!

2. Run a watch command that listens for changes to a particular file, and run a script to summarize the contents every time the file changes.

3. Ask a friend — you don’t have to do any of the work yourself, just ask a friend to brief you. You might need to persuade him though…

Read More!

As ironic as it may sound, reading less actually allows you to read more since you spend less time reading the fluff on each article, letting you read more articles. Of course, summarization isn’t perfect, and sometimes you’d want to read the whole thing. But the next time you find something interesting on the internet, do summarize it first and see if you want to go in-depth later.