Scrapping Google Playstore
Published: 2022-09-27 11:34:40

## Purpose Of Project :

#### To analyze how each fin tech apps is being used and widely accepted by Nigerians, as well as understanding the impact fin tech companies has made on Nigerians finances and Nigeri's Tech Sector.

## Project Process:
- Web scraping data from playstore
- Performing Exporatory Data Analysis On data
- Visualizing data on Power BI and Tableau

## Project Tools:
- Notebook
- Power BI
- Tableau

#### Importing `Necessary Libraries`

```
import time
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
```
#### Accessing `PlayStore` Using `Selenium`

```
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get('https://play.google.com/store/search?q=fintech%20in%20nigeria&c=apps')
time.sleep(10)
```
#### Getting All `The Apps` In The Required Page

```
SCROLL_PAUSE_TIME = 5
 
# Get scroll height
last_height = driver.execute_script("return document.body.scrollHeight")
time.sleep(SCROLL_PAUSE_TIME)
 
while True:
    # Scroll down to bottom
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
 
    # Wait to load page
    time.sleep(SCROLL_PAUSE_TIME)
 
    # Calculate new scroll height and compare with last scroll height
    new_height = driver.execute_script("return document.body.scrollHeight")
    if new_height == last_height:
        break
    last_height = new_height
```
#### `Scraping` App Links

```
links_fintech = []
elems = driver.find_elements_by_xpath("//a[@href]")
for elem in elems:
    if "details?id" in elem.get_attribute("href"):
        links_fintech.append((elem.get_attribute("href")))
        
links_fintech = list(dict.fromkeys(links_fintech))
```
#### Scraping The `Necessary Informations` From Each App

```
list_all_elements = []
for iteration in links_fintech:
    try:
        driver.get(iteration)
        print(iteration)
        time.sleep(3)
 
        header1 = driver.find_element_by_tag_name("h1")
        
        downloads = driver.find_elements_by_class_name("htlgb")
        list_downloads = []
        for x in range (len(downloads)):
            if x % 2 == 0:
                list_downloads.append(downloads[x].text)
    

        titles = driver.find_elements_by_class_name("AHFaub")
        comments = driver.find_element_by_class_name("EymY4b")
 
        list_elements = [iteration,header1.text, list_downloads.append(downloads[x].text), comments.text.split()[0]]
        for x in range (len(titles)):
            if titles[x].text == "Download":
                list_elements.append(list_others[x])
            if titles[x].text == "Developer":
                for y in list_others[x].split("\n"):
                    if "@" in y:
                        list_elements.append(y)
                        break
 
        list_all_elements.append(list_elements)
    except Exception as e:
        print(e)
```
`https://play.google.com/store/apps/details?id=ng.gov.cbn.speed.wallet.temp
https://play.google.com/store/apps/details?id=com.oxloan.loan.money.credit.nigeria
https://play.google.com/store/apps/details?id=com.cowrywise.android
https://play.google.com/store/apps/details?id=com.fastmoney.loan.credit.cash.nigeria
https://play.google.com/store/apps/details?id=com.wemabank.alat.prod
https://play.google.com/store/apps/details?id=com.ikonik.lamp_pay......`
#### Creating `A CSV File` For `Scraped DataFrame`

```
import pandas as pd
 
df = pd.DataFrame(list_all_elements,columns=['URL', 'Name', 'downloads', 'install']) 
df_1 = df.to_csv('fintech_playstore.csv', header = True, index=False, encoding="utf-8")
```

```
df_1
```

https://www.danielherediamejias.com/scraping-google-play-store-with-python-and-selenium/

https://play.google.com/store/search?q=fintechs%20in%20nigeria&c=apps