Chicago/RSocrata — pull request #3 by DASpringate (2014-07-23 10:26:08+00:00)
Title: Cope with missing columns converting from json to df

Hi,

Thanks for writing this package, it is really useful!

I was using it and found that in some cases where not all of the fields are present in every row of the data, the converter chokes as not all of the rows are the same length. e.g. try this:

```
url <- "http://data.undp.org/resource/wxub-qc5k.json"
read.socrata(url)
```

To deal with this I added a function `content_to_df`, which is called from `getContentAsDataFrame` and looks like:

```
content_to_df <- function(con){
lengths <- sapply(con, length)
if(all(lengths == length(con[[1]]))){
data.frame(t(sapply(con, unlist)), stringsAsFactors = FALSE)
} else {
all_cols <- names(con[[which(sapply(con, length) == max(sapply(con, length)))[1]]])
con <- lapply(con, function(x){
r <- c(x, sapply(all_cols[!all_cols %in% names(x)], function(xx) NA, simplify = FALSE))
r[all_cols]
})
data.frame(t(sapply(con, unlist)), stringsAsFactors = FALSE)
}
}
```

If the rows are all the same length, it converts as before, but if they are not, it assumes that the first longest length row has all of the correct column names in it and adds empty fields to any rows that do not have all of them in, then it sorts the columns back to the original order and returns a dataframe from this as before.

I added a test for this condition to the test file as well.

I also changed the json parser to `jsonlite` (I think you have already done this in the CRAN version).

If you can bring these changes into the CRAN version as well I would be really grateful as I want to call it in in a package I am developing now.

Cheers,

David