Taiwan-News-Crawler 2020.2.15
A political news crawler for 34 Taiwanese stream media.
License: MIT
Keywords: news crawler Taiwan politics

---
title: 'Taiwan_News_Crawler'
disqus: hackmd
---
**GitHub**
https://github.com/milkpool/Taiwan_News_Crawler

![info](https://img.shields.io/badge/release%20date-Feb.%202020-green) ![python](https://img.shields.io/badge/python-2-blue) ![python](https://img.shields.io/badge/python-3-blue)

**pypl**
https://pypi.org/project/Taiwan_News_Crawler/

![info](https://img.shields.io/badge/license-MIT-orange)

## Introduction
This open-source library is a political news crawler for 34 Taiwanese mainstream media.
The crawlered media is listed below.

Media Type|Meida Name (CN)|Media Name (EN)|ID|Abbreviation
:---:|:---:|:---:|:---:|:---:
Print Media|自由時報|Liberty News|0|ltn
 Print Media|蘋果日報|Apple Daily|1|appledaily
|Print Media|聯合報|UDN News|2|udn
|Print Media|中國時報|China Times|3|chinatimes
|Broadcast Media|TVBS|TVBS|4|tvbs
|Broadcast Media|ETtoday|ETtoday|5|ettoday
|Broadcast Media|台視|TTV|6|ttv
|Broadcast Media|中視|CTV|7|ctv
|Broadcast Media|華視|CTS|8|cts
|Broadcast Media|民視|FTV News|9|ftv
|Broadcast Media|公視|PTS|10|pts
|Broadcast Media|三立新聞|STEN|11|sten
|Broadcast Media|中天新聞|CTITV|12|ctitv
|Broadcast Media|年代新聞|ERA News|13|era
|Broadcast Media|非凡新聞|USTV|14|ustv
|Electronic Media|中央通訊社|CNA|15|cna
|Electronic Media|關鍵評論網|The News Lens|16|thenewslens
|Electronic Media|民報|People News|17|peoplenews
|Electronic Media|上報|Up Media|18|upmedia
|Electronic Media|大紀元|Epoch Times|19|epochtimes
|Electronic Media|信傳媒|CM Media|20|cmmedia
|Electronic Media|匯流新聞網|CNEWS|21|cnews
|Electronic Media|新頭殼|Newtalk|22|newtalk
|Electronic Media|風傳媒|Storm Media|23|storm
|Electronic Media|今日新聞|NOW News|24|nownews
|Electronic Media|鏡週刊|Mirror Media|25|mirrormedia
|Electronic Media|台灣好新聞|Taiwan Hot|26|taiwanhot
|Electronic Media|中央廣播電台|RTI News|27|rti
|Electronic Media|世界日報|World Journal|28|worldjournal
|Electronic Media|風向新聞|Kairos News|29|kairos
|Electronic Media|民眾日報|Mypeople News|30|mypeople
|Electronic Media|芋傳媒|Taro News|31|taronews
|News Website|Pchome新聞|Pchome News|32|pchome
|News Website|Yahoo!奇摩新聞|YAHOO! News|33|yahoo

## Installation
#### 1. Install the library package with pip.
```
pip install Taiwan_News_Crawler
```
#### 2. Download the webdriver for Chrome on the [official website](https://chromedriver.chromium.org/downloads).


## Usage
#### 1. Build a crawler with the webdriver path inputted.
```python
import Taiwan_News_Crawler

## Build news crawler
webdriver_path = 'WEBDRIVER_PATH'
mycrawler = NewsCrawler.crawler(webdriver_path)
```

#### 2. Crawl political news of certain media.
The `crawler_news` crawls latest news with specified media id or name. 
There are two parameters to input:
+  **media**: int/str, the media id or name needed to be crawlered
+  **amount**: int, the amount of crawlering news. Default number is 20.
```python
## Crawl new with media id
news_id_0 = mycrawler.crawler_news(media=0)

## Crawl new with media name
news_ltn = mycrawler.crawler_news(media='ltn', amount=10)
news_udn = mycrawler.crawler_news(media="聯合報", amount=50)
```

#### 3. Crawl political news with news url.
The `crawler_by_url` identifies the news media with url and gets the information.
The url parameter is a list of string. Url with different media is acceptable. 
```python
news = mycrawler.crawler_by_url(url=['NEWS_URL_1', 'NEWS_URL_2'])
```

#### 4. Print the crawled news.
The output of our crawler is in json format.
There are fields of the output, which are shown below:
+  **title**: str, the news title
+  **url**: str, the news full url
+  **author**: list, the news author. More than one author is possible. Shown as an empty list if it's not avaiable.
+  **time**: list, the published time. 
    +  time (str): the complete published time. ex. "2020/01/10 13:17"
    +  time_year (str): the published year. ex. "2020"
    +  time_month (str): the published month. ex. "01"
    +  time_day (str): the published day. ex. "10"
    +  time_hour_min (str): the published timing. ex. "13:17"
+  **context**: str, the news article.
+  **tag**: list, the tags of the news. Empty list for not avaiable. More than one tag is possible.
+  **related_news**: list, the related or recommended news the media provides.
    +  related_title: str, the related news' title.
    +  related_url: str, the related news' url link.
+  **source_img**: list, the pictures in the report.
    +  img_title: str, the related news' title. None for not avaliable.
    +  img_url: str, the related news' url link.
+  **sourcce_video**: list, the video in the report.
    +  video_title: str, thr video title. None for not avaliable.
    +  video_url: str, the video url link.
```python3
news_ltn = mycrawler.crawler_news(media='CTS', amount=10)
# print the first news information
news_no_1 = news_ltn[0]
for key, value in new_no_1.items():
    print(key)
    print(value)
    print()
```
```
title
'菲律賓禁台令 總統:若政治考量請三思'

url
'https://news.cts.com.tw/cts/politics/202002/202002141990582.html'


author
[u'陳詩雅', u'李鴻杰']

time
[{'time_day': '14', 'time_hour_min': '19:39', 'time_year': '2020', 'time_month': '02', 'time': '2020/02/14 19:39'}]

context
'華視新聞 陳詩雅 李鴻杰 台南報導  / 台南市面對武漢肺炎疫情，總統蔡英文、行政院長蘇貞昌和副院長陳其邁，今(14)日分頭前進工廠視察。總統下午就南下視察防疫用酒精生產。面對菲律賓發出禁台令，總統表示，若是因為政治考量，要求菲律賓三思，台灣不能容忍這樣的事情，也必然會有相應的處理，最新消息，菲律賓已經撤回對台灣的禁令。75度防疫酒精台酒一天就可以產20萬瓶，面對武漢肺癌疫情，總統蔡英文再度前進工廠生產線，這次看的是酒精，成為70年來，第一位造訪台南隆田酒廠的現任總統，目前酒精產量穩定，可以支撐現階段需求，不過防疫戰火延燒到菲律賓，對於菲律賓在10日無預警禁止台灣人入境，總統說如果是政治考量菲律賓要三思。總統 蔡英文：「如果是基於政治考量的話，我們就要請他們三思，因為我們不能夠容忍這樣的事情，也必然會有一些相應的處理，」在總統受訪之後，菲律賓在晚間取消對台禁令，記者 vs. 總統 蔡英文：「台灣最紅的小孩子就是小明。」總統一聽到小明忍不住笑了，但是又立刻收起微笑，因為被問到對於無我國國籍中配子女禁止入境，馬前總統PO文說不要讓歧視凌駕人道，總統 蔡英文：「這沒有歧視的問題，只有疫情處理跟疫情掌控，跟保護我們國人的健康，是最重要的原則，我是覺得既然已經做過總統，應該知道說在做一個相關的決策，現在所最重要的還是以疫情的掌控為最優先」，另外還有國人滯留湖北，總統則再次強調，弱勢優先檢疫優先兩大原則，會持續溝通。'

tag
[u'撤回禁令', u'菲律賓', u'蔡英文', u'酒精', u'武漢肺炎']

related_news
[{u'好消息! 傳菲將解除對台灣旅行禁令': 'https://news.cts.com.tw/cts/politics/202002/202002141990583.html'}, {u'菲律賓發布禁台令 對移工衝擊大': 'https://news.cts.com.tw/cts/international/202002/202002131990465.html'}, {u'菲律賓禁台入境 我方擬祭出反制': 'https://news.cts.com.tw/cts/international/202002/202002131990381.html'}, {u'菲內閣重議禁台措施 我研擬反擊': 'https://news.cts.com.tw/cts/international/202002/202002131990380.html'}, {u'出國怕被誤認 「來自台灣」小物熱賣': 'https://news.cts.com.tw/cts/general/202002/202002121990318.html'}, {u'菲律賓突發禁台令 台灣恐爆缺工潮': 'https://news.cts.com.tw/cts/society/202002/202002121990317.html'}, {u'菲禁令滯留長灘島 部落客:如歷險記': 'https://news.cts.com.tw/cts/society/202002/202002121990316.html'}, {u'菲禁台團入境 當地旅遊業損失逾50萬': 'https://news.cts.com.tw/cts/society/202002/202002121990257.html'}, {u'禁台入境轉折 外交部:菲國內部不同調': 'https://news.cts.com.tw/cts/general/202002/202002111990194.html'}, {u'菲禁台客入境 旅客到機場無法登機': 'https://news.cts.com.tw/cts/international/202002/202002111990193.html'}, {u'菲律賓移工無法入台 勞動部祭出因應措施': 'https://news.cts.com.tw/cts/life/202002/202002111990179.html'}, {u'菲遵循「一個中國」 國民黨：政府應強硬展立場並反制': 'https://news.cts.com.tw/cts/politics/202002/202002111990154.html'}]

source_img
[{None: 'https://news.cts.com.tw/photo/cts/202002/202002141990582_l.jpg'}]

source_video
[{None: 'https://www.youtube.com/embed/6cV1YNTOjyI?rel=0&playsinline=1'}]

media
'華視'

```