ITP - exercises - exercise - 2

Got two errors

Encoding.default_external = "utf-8"

and

`validate_sheet_name':

To wrap up this chapter we are going to work through an exercise that
will show us how to interact with these common file types. We will pull
news articles from two different sources. One in XML format. One in
JSON. We will combine the two lists to a common format, sort them, and
save them to CSV and XLS files.

Let's get started. In your command line run these two commands.

$ wget -O feedzilla.json
http://api.feedzilla.com/v1/categories/8/articles.json
$ wget -O slashdot.xml http://rss.slashdot.org/Slashdot/slashdot

You should have two new files in your directory now. The above commands
saved feedzilla.json and slashdot.xml into your directory. Get a good
look at these files.

$ cat feedzilla.json
$ cat slashdot.xml

From the command line, let's install the AXLSX gem for our XLS file, and
nogokiri for parsing XML.

First, install axlsx gem.

$ gem install axlsx

After that installs, install nokogiri gem.

$ gem install nokogiri

Great, now open irb. Let's first write some code to open the
slashdot.xml file and get its articles' title, url, and summary.

First, we will require the libraries to deal with our mix of file types
that we will use throughout this example. Type the following into irb.
If you get an error, it probably means one of the gems didn't install
correctly. Make sure the gem install instructions above successfully
installed the required gems.

require 'json'
require 'nokogiri'
require 'axlsx'
require 'csv'

Time to open up that XML file, find the item elements, and parse them
with Nokogiri

slashdot_articles = []
File.open("slashdot.xml", "r") do |f|
  doc = Nokogiri::XML(f)
  slashdot_articles = doc.css('item').map { |i|
    {
      title: i.at_css('title').content,
      link: i.at_css('link').content,
      summary: i.at_css('description').content
    }
  }
end

At this point, the slashdot_articles array should be populated with the
articles from the slashdot XML RSS feed. We have an array of hashes with
fields we want to export. We found all the articles with the
doc.css('item') call and then chained it through the map block to pull
out the fields we want to export. We are ready to do a similar manauver
on the feedzilla.json file.

feedzilla_articles =[]
File.open("feedzilla.json", "r") do |f|
  items = JSON.parse(f.read)
  feedzilla_articles= items['articles'].map { |a|
    {
      title: a['title'],
      link: a['url'],
      summary: a['summary']
    }
  }
end

sorted_articles =  (feedzilla_articles + slashdot_articles).sort_by {|a|
a[:title]}

As you can see, we pulled out the articles from the feedzilla json file
just like we did to the slashdot xml feed. After that we sorted the
combined results based on the title.

That just leaves us the task of writing the two export files.

CSV.open("article.csv", "wb") do |csv|
  sorted_articles.each { |a| csv << [ a[:title], a[:link], a[:summary] ]
}
end

pkg = Axlsx::Package.new
pkg.workbook.add_worksheet(:name => "Articles #{Time.now}") do |sheet|
 sorted_articles.each { |a| sheet.add_row [a[:title], a[:link],
a[:summary]] }
end
pkg.serialize("articles.xlsx")

To fully understand the examples above requires studying the various
libraries we were using, and that's beyond the scope of this chapter.