dpalmisano/NoTube-Beancounter-2.0 — Issue #4: Twitter crawl sometimes fails
Author: libbymiller
Created: 2012-03-07 12:26:09+00:00
State: open

error is com.google.gson.JsonParseException: The JsonDeserializer tv.notube.crawler.requester.request.twitter.TwitterResponseAdapter@430b6 failed to deserialize json object when I run crawler/run.sh

I've narrowed the offending code down to

crawler/src/main/java/tv/notube/crawler/requester/request/twitter/TwitterResponseAdapter.java

JsonArray urlsArray = tweetElement
.getAsJsonObject()
.get("entities")
.getAsJsonObject()
.get("urls")
.getAsJsonArray();
for(JsonElement jsonUrl : urlsArray) {
try {
tweet.addUrl(new URL(
jsonUrl.getAsJsonObject()
.get("expanded_url")
.getAsString())
);
} catch (MalformedURLException e) {
// could happen that a url is not well formed
// just skip
}
}

-> currently commented out.