devxoul/hippo — pull request #1 by devxoul (2014-10-11 05:25:02+00:00)
Title: Bulk upsert

#### `crawl_webtoon_list`

Crawl default webtoon data.

##### Redis Key

`webtoon-naver-335885`

##### Redis Value

```json
+ {
+ "portal": "naver",
+ "portal_id": 335885,
+ "title": "가우스전자",
+ "picture_url": "http://thumb.comic.naver.net/webtoon/335885/thumbnail/title_thumbnail_20110527212550_t125x101.jpg",
+ "concluded": false,
+ "mon": true,
+ "tue": true,
+ "wed": true,
+ "tue": true,
+ "fri": true
+ }
```

#### `crawl_webtoon_detail(key)`

Crawl webtoon detail data: *introduction*, *artists*, *total_no*

##### Redis Key

`webtoon-naver-335885`

##### Redis Value

```json
{
"portal": "naver",
"portal_id": 335885,
"title": "가우스전자",
"picture_url": "http://thumb.comic.naver.net/webtoon/335885/thumbnail/title_thumbnail_20110527212550_t125x101.jpg",
"concluded": false,
"mon": true,
"tue": true,
"wed": true,
"tue": true,
"fri": true,
+ "introduction": "다국적 문어발 기업 가우스전자 마케팅3부 이야기\n<트라우마> 곽백수 작가가 선보이는 본격 직장인 만화!",
+ "artists": ["곽백수"],
+ "total_no": 833
}
```

#### `crawl_episode_page(key, page)`

Crawl webtoon for each pages.

##### Redis Key

`webtoon-naver-335885`

##### Redis Value

```json
{
"portal": "naver",
"portal_id": 335885,
"title": "가우스전자",
"picture_url": "http://thumb.comic.naver.net/webtoon/335885/thumbnail/title_thumbnail_20110527212550_t125x101.jpg",
"concluded": false,
"mon": true,
"tue": true,
"wed": true,
"tue": true,
"fri": true,
"introduction": "다국적 문어발 기업 가우스전자 마케팅3부 이야기\n<트라우마> 곽백수 작가가 선보이는 본격 직장인 만화!",
"artists": ["곽백수"],
"total_no": 833,
+ "episodes": {
+ "1": {
+ "no": 1,
+ "title": "1화",
+ "picture_url": "...",
+ "published_at": "2011.06.06 00:00:00"
+ },
+ "2": {
+ "no": 2,
+ "title": "2화",
+ "picture_url": "...",
+ "published_at": "2011.06.07 00:00:00"
+ },
+ ...
+ }
}
```

#### `upsert_crawl_result`

Upsert crawling results if `total_no` == `episodes.count`