Package: django-easy-scraper
Django Easy ScraperAn standalone django app that can be used/intigrated with both django and no-django application easily. the scraping mechanism is  onRegular Expressionandxpathwhich is mean you can use what you are familiar with very easily.It requires to install pythonrequestsmodulesInstallpip install django-easy-scraperBasic Usesif you use regex:from django_easy_scraper import scraper

class ScrapeExampleDotCom(scraper.Scraper):

    regex_fields = {
        'price': "Write Your Regex pattern for price here",
        'title': "Write your regex pattern for title here",
        # Like above way you can add as much fields/keys as you want
    }if you use xpath:from django_easy_scraper import scraper

class ScrapeExampleDotCom(scraper.Scraper):

    xpath_fields = {
        'price': "Write Your xpath pattern for price here",
        'title': "Write your xpath pattern for title here",
        # Like above way you can add as much fields/keys as you want
    }Scrape nowurl = 'www.example.com/bla-bla-details-page/
data = ScrapeExampleDotCom.regex_url_scraper(url)

print(data)and the response should look like this if your regex pattern are correct:{
    'price': 4,
    'title': 'an scraped title',
}regex_url_scrapermethod always gives you json response,So if you add many regex pattern inregex_fields, it will give you response that number of dictionary key with result that you added in that dictionary.Multiple Sites Scraping togetherYou don't need to call different method for different site all the time !! Just call once and scrape all, something fun, right?Like you are gonna scrape three sites:www.example.comwww.exampletwo.comwww.examplethree.comBut how will those site product scrape automatically, it scares you ?Wirte Regex pattern for all above site with the fields that you want to scrape:from django_easy_scraper import scraper

class ScrapeExampleDotCom(scraper.Scraper):
    regex_fields = {
        'price': "Write Your Regex pattern for price here",
        'title': "Write your regex pattern for title here",
        # Like above way you can add as much fields/keys as you want
    }

class ScrapeExampleTwo(scraper.Scraper):
    regex_fields = {
        'price': "Write Your Regex pattern for price here",
        'title': "Write your regex pattern for title here",
        # Like above way you can add as much fields/keys as you want
    }

class ScrapeExampleThree(scraper.Scraper):
    regex_fields = {
        'price': "Write Your Regex pattern for price here",
        'title': "Write your regex pattern for title here",
        # Like above way you can add as much fields/keys as you want
    }You have written regular expression for you all the site you are gonna scrape,Now it's time use ourSwitchclass that will route your script/class based on the site you are gonna scrape? Cool, right ? !!It's where the magic really begins:Just place all your class in the dictionaryswitcher.Important Note:keyname should be domain name, pure domain name, no www or http or slash, dont add anything prefix/suffixvalueshould be the class of that domain you have written for and place it's method `regex_url_scraperfrom django_easy_scraper import switch

class Switch(switch.BaseSwitch):
    switcher = {
        'example.com': ScrapeExampleDotCom.regex_url_scraper,
        'exampletwo.com': ScrapeExampleTwo.regex_url_scraper,
        'examplethree.com': ScrapeExampleThree.regex_url_scraper,
    }If you use xpath, you have passxpath_scraperinstead ofregex_url_scraperSo you have done routing your script/class based on the url it gets.Get response data as python dictionary like above site:url = 'Any of site you have written class for the site and added in switch class'

response = Switch.get_data(url=url, raise_exception=False)

print(response) # Will give you an object of data that you trying to scrapeSwitch class is giving you facilities to route your scraping class automacally based whatever site link pass to it'sget_datamethod.get_datamethod'sraise_exceptionis it handle if you want to raise excepiton when your expected fields not foundGot an issue ?Please open an issue on our github repo:https://github.com/dearopen/django-easy-scraperDon't forget to star to this project if you like this.Happy Scraping !!