Package: linkedin-scraper
Linkedin ScraperScrapes Linkedin User DataLinkedin ScraperInstallationSetupUsageSample UsageUser ScrapingCompany ScrapingJob ScrapingJob Search ScrapingScraping sites where login is required firstScraping sites and login automaticallyAPIPersonlinkedin_urlnameaboutexperienceseducationsinterestsaccomplishmentcompanyjob_titledriverscrapescrape(close_on_complete=True)Companylinkedin_urlnameabout_uswebsiteheadquartersfoundedcompany_typecompany_sizespecialtiesshowcase_pagesaffiliated_companiesdriverget_employeesscrape(close_on_complete=True)ContributionInstallationpip3install--userlinkedin_scraperVersion2.0.0and before is calledlinkedin_user_scraperand can be installed viapip3 install --user linkedin_user_scraperSetupFirst, you must set your chromedriver location byexportCHROMEDRIVER=~/chromedriverSponsorScrape public LinkedIn profile data at scale withProxycurl APIs.• Scraping Public profiles are battle tested in court in HiQ VS LinkedIn case.• GDPR, CCPA, SOC2 compliant• High rate limit - 300 requests/minute• Fast - APIs respond in ~2s• Fresh data - 88% of data is scraped real-time, other 12% are not older than 29 days• High accuracy• Tons of data points returned per profileBuilt for developers, by developers.UsageTo use it, just create the class.Sample Usagefromlinkedin_scraperimportPerson,actionsfromseleniumimportwebdriverdriver=webdriver.Chrome()email="some-email@email.address"password="password123"actions.login(driver,email,password)# if email and password isnt given, it'll prompt in terminalperson=Person("https://www.linkedin.com/in/joey-sham-aa2a50122",driver=driver)NOTE: The account used to log-in should have it's language set English to make sure everything works as expected.User Scrapingfromlinkedin_scraperimportPersonperson=Person("https://www.linkedin.com/in/andre-iguodala-65b48ab5")Company Scrapingfromlinkedin_scraperimportCompanycompany=Company("https://ca.linkedin.com/company/google")Job Scrapingfromlinkedin_scraperimportJobSearch,actionsfromseleniumimportwebdriverdriver=webdriver.Chrome()email="some-email@email.address"password="password123"actions.login(driver,email,password)# if email and password isnt given, it'll prompt in terminalinput("Press Enter")job=Job("https://www.linkedin.com/jobs/collections/recommended/?currentJobId=3456898261",driver=driver,close_on_complete=False)Job Search Scrapingfromlinkedin_scraperimportJobSearch,actionsfromseleniumimportwebdriverdriver=webdriver.Chrome()email="some-email@email.address"password="password123"actions.login(driver,email,password)# if email and password isnt given, it'll prompt in terminalinput("Press Enter")job_search=JobSearch(driver=driver,close_on_complete=False,scrape=False)# job_search contains jobs from your logged in front page:# - job_search.recommended_jobs# - job_search.still_hiring# - job_search.more_jobsjob_listings=job_search.search("Machine Learning Engineer")# returns the list of `Job` from the first pageScraping sites where login is required firstRunipythonorpythonInipython/python, run the following code (you can modify it if you need to specify your driver)fromlinkedin_scraperimportPersonfromseleniumimportwebdriverdriver=webdriver.Chrome()person=Person("https://www.linkedin.com/in/andre-iguodala-65b48ab5",driver=driver,scrape=False)Login to Linkedin[OPTIONAL] Logout of LinkedinIn the sameipython/pythoncode, runperson.scrape()The reason is that LinkedIn has recently blocked people from viewing certain profiles without having previously signed in. So by settingscrape=False, it doesn't automatically scrape the profile, but Chrome will open the linkedin page anyways. You can login and logout, and the cookie will stay in the browser and it won't affect your profile views. Then when you runperson.scrape(), it'll scrape and close the browser. If you want to keep the browser on so you can scrape others, run it asNOTE: For version >=2.1.0, scraping can also occur while logged in. Beware that users will be able to see that you viewed their profile.person.scrape(close_on_complete=False)so it doesn't close.Scraping sites and login automaticallyFrom verison2.4.0on,actionsis a part of the library that allows signing into Linkedin first. The email and password can be provided as a variable into the function. If not provided, both will be prompted in terminal.fromlinkedin_scraperimportPerson,actionsfromseleniumimportwebdriverdriver=webdriver.Chrome()email="some-email@email.address"password="password123"actions.login(driver,email,password)# if email and password isnt given, it'll prompt in terminalperson=Person("https://www.linkedin.com/in/andre-iguodala-65b48ab5",driver=driver)APIPersonA Person object can be created with the following inputs:Person(linkedin_url=None,name=None,about=[],experiences=[],educations=[],interests=[],accomplishments=[],company=None,job_title=None,driver=None,scrape=True)linkedin_urlThis is the linkedin url of their profilenameThis is the name of the personaboutThis is the small paragraph about the personexperiencesThis is the past experiences they have. A list oflinkedin_scraper.scraper.ExperienceeducationsThis is the past educations they have. A list oflinkedin_scraper.scraper.EducationinterestsThis is the interests they have. A list oflinkedin_scraper.scraper.InterestaccomplishmentThis is the accomplishments they have. A list oflinkedin_scraper.scraper.AccomplishmentcompanyThis the most recent company or institution they have worked at.job_titleThis the most recent job title they have.driverThis is the driver from which to scraper the Linkedin profile. A driver using Chrome is created by default. However, if a driver is passed in, that will be used instead.For exampledriver=webdriver.Chrome()person=Person("https://www.linkedin.com/in/andre-iguodala-65b48ab5",driver=driver)scrapeWhen this isTrue, the scraping happens automatically. To scrape afterwards, that can be run by thescrape()function from thePersonobject.scrape(close_on_complete=True)This is the meat of the code, where execution of this function scrapes the profile. Ifclose_on_completeis True (which it is by default), then the browser will close upon completion. If scraping of other profiles are desired, then you might want to set that to false so you can keep using the same driver.CompanyCompany(linkedin_url=None,name=None,about_us=None,website=None,headquarters=None,founded=None,company_type=None,company_size=None,specialties=None,showcase_pages=[],affiliated_companies=[],driver=None,scrape=True,get_employees=True)linkedin_urlThis is the linkedin url of their profilenameThis is the name of the companyabout_usThe description of the companywebsiteThe website of the companyheadquartersThe headquarters location of the companyfoundedWhen the company was foundedcompany_typeThe type of the companycompany_sizeHow many people are employeed at the companyspecialtiesWhat the company specializes inshowcase_pagesPages that the company owns to showcase their productsaffiliated_companiesOther companies that are affiliated with this onedriverThis is the driver from which to scraper the Linkedin profile. A driver using Chrome is created by default. However, if a driver is passed in, that will be used instead.get_employeesWhether to get all the employees of companyFor exampledriver=webdriver.Chrome()company=Company("https://ca.linkedin.com/company/google",driver=driver)scrape(close_on_complete=True)This is the meat of the code, where execution of this function scrapes the company. Ifclose_on_completeis True (which it is by default), then the browser will close upon completion. If scraping of other companies are desired, then you might want to set that to false so you can keep using the same driver.Contribution