Package: ghananews-scraper
GhanaNews ScraperA simple unofficial python package to scrape data fromGhanaWeb,MyJoyOnline,DailyGraphic,CitiBusinessNews,YenGH,3News,MyNewsGh,PulseGhAffiliated toBank of Ghana Fx RatesandGhanaShops-ScraperNOTE:This library may keep changing due to changes to the respective websites.How to installpipinstallghananews-scraperExample Google Colab NotebookClick Here:Warning: DO NOT RUN GHANAWEB CODE IN ONLINE Google Colabs)Some GhanaWeb Urls:urls = [
    "https://www.ghanaweb.com/GhanaHomePage/regional/"	
    "https://www.ghanaweb.com/GhanaHomePage/editorial/"
    "https://www.ghanaweb.com/GhanaHomePage/health/"
    "https://www.ghanaweb.com/GhanaHomePage/diaspora/"
    "https://www.ghanaweb.com/GhanaHomePage/tabloid/"
    "https://www.ghanaweb.com/GhanaHomePage/africa/"
    "https://www.ghanaweb.com/GhanaHomePage/religion/"
    "https://www.ghanaweb.com/GhanaHomePage/NewsArchive/"
    "https://www.ghanaweb.com/GhanaHomePage/business/"
    "https://www.ghanaweb.com/GhanaHomePage/SportsArchive/"
    "https://www.ghanaweb.com/GhanaHomePage/entertainment/"
    "https://www.ghanaweb.com/GhanaHomePage/africa/"
    "https://www.ghanaweb.com/GhanaHomePage/television/"
]OutputsAll outputs will be saved in a.csvfile. Other file formats not yet supported.Usagefromghanaweb.scraperimportGhanaWeburl='https://www.ghanaweb.com/GhanaHomePage/politics/'# url = "https://www.ghanaweb.com/GhanaHomePage/NewsArchive/"# url = 'https://www.ghanaweb.com/GhanaHomePage/health/'# url = 'https://www.ghanaweb.com/GhanaHomePage/crime/'# url = 'https://www.ghanaweb.com/GhanaHomePage/regional/'# url = 'https://www.ghanaweb.com/GhanaHomePage/year-in-review/'# web = GhanaWeb(url='https://www.ghanaweb.com/GhanaHomePage/politics/')web=GhanaWeb(url=url)# scrape data and save to `current working dir`web.download(output_dir=None)Scrape list of articles fromGhanaWebfromghanaweb.scraperimportGhanaWeburls=['https://www.ghanaweb.com/GhanaHomePage/politics/','https://www.ghanaweb.com/GhanaHomePage/health/','https://www.ghanaweb.com/GhanaHomePage/crime/','https://www.ghanaweb.com/GhanaHomePage/regional/','https://www.ghanaweb.com/GhanaHomePage/year-in-review/']forurlinurls:print(f"Downloading:{url}")web=GhanaWeb(url=url)# download to current working directory# if no location is specified# web.download(output_dir="/Users/tsiameh/Desktop/")web.download(output_dir=None)Scrape data fromMyJoyOnlineIt is recommended to use option 1. Option 2 might run into timeout issues.DO NOT RUN THIS INGoogle Colab NOTEBOOK, either in .py script, visual studio code or in terminal due toseleniumpackage.You may pass driver_name =chromeorfirefox# Option 1.frommyjoyonline.scraperimportMyJoyOnlineNewsurl='https://myjoyonline.com/politics'print(f"Downloading data from:{url}")joy=MyJoyOnlineNews(url=url)# joy = MyJoyOnlineNews(url=url, driver_name="firefox")joy.download()# Option 2.frommyjoyonline.scraperimportMyJoyOnlineNewsurls=['https://myjoyonline.com/news','https://myjoyonline.com/politics''https://myjoyonline.com/entertainment','https://myjoyonline.com/business','https://myjoyonline.com/sports','https://myjoyonline.com/opinion','https://myjoyonline.com/technology']forurlinurls:print(f"Downloading data from:{url}")joy=MyJoyOnlineNews(url=url)# download to current working directory# if no location is specified# joy.download(output_dir="/Users/tsiameh/Desktop/")joy.download()Scrape data fromCitiBusinessNewsHere are some list of publisher names:citibusinessnewsaklamaellenemmanuel-oppongnerteleyedna-agnes-boakyenii-larte-larteynaa-shika-caesarogboduNote: using publisher names fetches more data than the url.fromcitionline.scraperimportCitiBusinessOnlineurls=["https://citibusinessnews.com/ghanabusinessnews/features/","https://citibusinessnews.com/ghanabusinessnews/telecoms-technology/","https://citibusinessnews.com/ghanabusinessnews/international/","https://citibusinessnews.com/ghanabusinessnews/news/government/","https://citibusinessnews.com/ghanabusinessnews/news/","https://citibusinessnews.com/ghanabusinessnews/business/","https://citibusinessnews.com/ghanabusinessnews/news/economy/","https://citibusinessnews.com/ghanabusinessnews/news/general/","https://citibusinessnews.com/ghanabusinessnews/news/top-stories/","https://citibusinessnews.com/ghanabusinessnews/business/tourism/"]forurlinurls:print(f"Downloading data from:{url}")citi=CitiBusinessOnline(url=url)citi.download()# OR: scrape using publisher namefromcitionline.authorsimportCitiBusinessciti=CitiBusiness(author="citibusinessnews",limit_pages=4)citi.download()Scrape data fromDailyGraphicfromgraphiconline.scraperimportGraphicOnlineurls=["https://www.graphic.com.gh/news.html","https://www.graphic.com.gh/news/politics.html","https://www.graphic.com.gh/lifestyle.html","https://www.graphic.com.gh/news/education.html","https://www.graphic.com.gh/native-daughter.html","https://www.graphic.com.gh/international.html"]forurlinurls:print(f"Downloading data from:{url}")graphic=GraphicOnline(url=url)graphic.download()Scrape data fromYenGH# OPTION: 1fromyen.scrapyimportYenNewsurl='https://www.yen.com.gh/'print(f"Downloading data from:{url}")yen=YenNews(url=url)yen.download()# OPTION: 2fromyen.scrapyimportYenNewsurls=['https://www.yen.com.gh/','https://yen.com.gh/politics/','https://yen.com.gh/world/europe/','https://yen.com.gh/education/','https://yen.com.gh/ghana/','https://yen.com.gh/people/','https://yen.com.gh/world/asia/','https://yen.com.gh/world/africa/','https://yen.com.gh/entertainment/','https://yen.com.gh/business-economy/money/','https://yen.com.gh/business-economy/technology/']forurlinurls:print(f"Downloading data from:{url}")yen=YenNews(url=url)yen.download()Scrape data fromMyNewsGhfrommynewsgh.scraperimportMyNewsGh# scrape from multiple URLsurls=["https://www.mynewsgh.com/category/politics/","https://www.mynewsgh.com/category/news/","https://www.mynewsgh.com/category/entertainment/","https://www.mynewsgh.com/category/business/","https://www.mynewsgh.com/category/lifestyle/","https://www.mynewsgh.com/tag/feature/","https://www.mynewsgh.com/category/world/","https://www.mynewsgh.com/category/sports/"]forurlinurls:print(f"Downloading data from:{url}")my_news=MyNewsGh(url=url,limit_pages=50)my_news.download()# scrape from a single URLfrommynewsgh.scraperimportMyNewsGhurl="https://www.mynewsgh.com/category/politics/"my_news=MyNewsGh(url=url,limit_pages=None)my_news.download()Scrape data from3Newsfromthreenews.scraperimportThreeNews# DO NOT RUN ALL AUTHORS: select ONLY few# DO NOT CHANGE THE AUTHOR NAMESauthors=["laud-nartey","3xtra","essel-issac","arabaincoom","bbc","betty-kankam-boadu","kwameamoh","fiifi_forson","fdoku","frankappiah","godwin-asediba","afua-somuah","irene","joyce-sesi","3news_user","ntollo","pwaberi-denis","sonia-amade","effah-steven","michael-tetteh"]forauthorinauthors:print(f"Downloading data from author:{author}")three_news=ThreeNews(author=author,limit_pages=50)three_news.download()# ORfromthreenews.scraperimportThreeNewsthree=ThreeNews(author="laud-nartey",limit_pages=None)three.download()Scrape data fromPulseGhselect ONLY few urlsNote: these values may changeCategoryNumber of PagesNews40Entertainment40Business40Lifestyle40Business/Domestic26Business/International40Sports/Football99News/Politics40News/Local40News/World40News/Filla38Entertainment/Celebrities40Lifestyle/Fashion40frompulsegh.scraperimportPulseGhurls=["https://www.pulse.com.gh/news","https://www.pulse.com.gh/news/politics","https://www.pulse.com.gh/entertainment","https://www.pulse.com.gh/lifestyle","https://www.pulse.com.gh/sports","https://www.pulse.com.gh/sports/football","https://www.pulse.com.gh/business/international","https://www.pulse.com.gh/business/domestic","https://www.pulse.com.gh/business","https://www.pulse.com.gh/quizzes","https://www.pulse.com.gh/news/filla","https://www.pulse.com.gh/news/world"]forurlinurls:print(f"Downloading data from:{url}")pulse=PulseGh(url=url,limit_pages=5)pulse.download()# news has 40 pagesfrompulsegh.scraperimportPulseGhpulse=PulseGh(url="https://www.pulse.com.gh/news",total_pages=40,limit_pages=20)pulse.download()# Sports/football has 99 pagesfrompulsegh.scraperimportPulseGhpulse=PulseGh(url="https://www.pulse.com.gh/sports/football",total_pages=99,limit_pages=None)pulse.download()BuyMeCoffeeCreditsTheophilus Siameh