Package: peviitor-pyscraper
Peviitor ScraperDescriptionpeviitor_pyscraperis a Python-based scraping library that relies on HTML parsing libraries, Beautiful Soup, and Requests. It allows you to extract the required data from web pages and save them in an easily usable format such as CSV or JSON. Withpeviitor_pyscraper, you can select specific HTML elements from a web page and extract necessary information like text, links, images, etc.Features ofpeviitor_pyscraper:Utilizes popular Python libraries, BeautifulSoup and Requests, to facilitate web scraping.Extracts the required data from a web page using specific HTML selections.Provides a variety of storage options for the scraped data, including JSON.Is easy to use and integrate into existing Python projects.It can render pages with dynamically generated elements.peviitor_pyscraperis an excellent choice for Python developers seeking a powerful and flexible web scraping library. Withpeviitor_pyscraper, you can automate the process of extracting data from web pages, saving time and effort.InstallationYou need to have Python 3.6 or higher installed on your computer.pip install peviitor-pyscraperNode JS is required for rendering pages with dynamically generated elements.npm i peviitor_jsscraperUsage ExamplesDownloading the content from a specific URL:fromscraper.ScraperimportScraperscraper=Scraper()html=scraper.get_from_url('https://www.example.ro')print(html.prettify())The two lines of code create a Scraper object with the URLhttps://www.example.roand then download the HTML code from that URL using theget_from_url()method, which returns a BeautifulSoup object that can be later used to search for specific elements within the web page.To extract all "a" tags that contain an "href" attribute starting with "https://" from the downloaded HTML code, you can use the following code:fromscraper.ScraperimportScraperscraper=Scraper()html=scraper.get_from_url('https://www.example.ro')links=html.find_all('a',href=re.compile('^https://'))forlinkinlinks:print(link.get('href'))To extract the first "h1" tag from the page:fromscraper.ScraperimportScraperscraper=Scraper()html=scraper.get_from_url('https://www.example.ro')h1=html.find('h1')print(h1.text)Downloading JSON content from a specific URL:fromscraper.ScraperimportScraperscraper=Scraper()json=scraper.get_from_url('https://api.example.ro',type='JSON')print(json)These lines of code create a Scraper object with the URLhttps://api.example.roand then download the JSON content from that URL using theget_from_url()method, which returns a JSON object that can be later used to search for specific elements within the web page.To make a POST request to a specific URL:fromscraper.ScraperimportScraperscraper=Scraper()data={'key1':'value1','key2':'value2'}response=scraper.post('https://api.example.ro',data=data)json=response.json()print(json)Thepeviitor_pyscrapercan render pages with dynamically generated elements.
To render a page with dynamically generated elements you need to install Node JS and
thepeviitor_jsscraperpackage. To install the package runnpm i peviitor_jsscraperand then use therender_page()method.fromscraper.ScraperimportScraperscraper=Scraper()html=scraper.render_page('https://www.example.ro')print(html.prettify())Contains all BeautifulSoup methods and attributes.ContributingIf you want to contribute to the development of the scraper, there are several ways you can do so. First, you can help by contributing to the source code by adding new features or fixing existing issues. Second, you can contribute to improving the documentation or translating it into other languages. Additionally, if you want to help but are unsure how to get started, you can check our list of open issues and ask us how you can assist. For more information, please refer to the "Contribute" section in our documentation.LicenseThis project is licensed under the MIT License - see the LICENSE file for details.ContactIf you have any questions or suggestions, please contact us atEmail:contact@laurentiumarian.roWebsite:https://laurentiumarian.roGitHub:https://github.com/lalalaurentiuAcknowledgementsBeautiful SoupRequests