Package: bookshops
Web scraping to get book informationThis library is to get book information. We can search withkeywords, with theisbn, with anadvanced search, and dopagination.We get the data from existing websites. We scrape:for French books,http://www.librairie-de-paris.fr(also Decitre, but
it’s less complete)for Spain:http://www.casadellibro.comfor Germany:http://www.buchlentner.dewe get: the title and authors, the price, the publisher(s), the cover,
etcImport data from an ods or csv fileAnother goal of this lib is to easily fetch a large amount of books you
have listed in an ods or csv file.If your file has an ‘isbn’ and a ‘quantity’ column, it’s easy, we will
find all the book information.If it only has the title and the publisher, it’s doable but error prone.
We can still do it, but you shall do an inventory of your stock
afterwards.See theodsimportmodule. It gives back a json. It’s your
responsibility to add what you want in your database (this is done in
Abelujohttps://gitlab.com/vindarel/abelujo).Usable, but work in progress.Accepted format and columnsWe can read ods and csv files.a file with an “isbn” and “quantity” column,a file with columns “title”, “publisher”, “isbn” (optionnal in this
case), “shelf”, “distributor”, “quantity”. There isno“price”
column. “authors” is optionnal (it can help to fetch the correct
book).If the file has no headers, use the “odsettings.py” configuration file
(in particular, to set the csv delimiter, either “,” or “;”).Why not Amazon ?Amazon kills the book industry and its employees. But moreover, we can
add value to our results. We can link to a good and independent bookshop
from within our application, we could command books from it, we could
say if it has exemplaries in stock or not, etc. And… we learn a lot in
doing this !Technically speaking, the Amazon API web service can be too limitating
and not appropriate. One must register to Amazon Product Advertising and
to AWS, and the requests rate is limited to 1 request per second.Why not Google books ?It has very few data.Why not the BNF (Bibliothèque Nationale de France) ?Because, for bookshops, we need recent books (they enter the BNF
database after a few months), up to date information. There isn’t a lot
of tools either.InstallInstall from pypi:pip install bookshopsUseCommand lineYou can try this lib on the command line with the following commands: -livres: french books -libros: spanish books -bucher:
german books - come and ask for more :)For example:livres antigoneorlivres 9782918059363and you get the above screenshot.Options: (this may vary according to the scrapers, check them with-h) --ior--isbnto ensure to get all the isbn. The
command line tool won’t get them by default if they need to be fetched
with another http request for each book. That depends on the websites.As a libraryBut most of all, from within your program:from bookshops.frFR.librairiedeparis.librairiedeparisScraper import Scraper as frenchScraper

scraper = frenchScraper("search keywords")
cards = scraper.search()
# we get a list of dictionnaries with the title, the authors, etc.Advanced searchWork in progress.You can searched:agoneto search for a specific publisher.PaginationWe do pagination:scraper = frenchScraper("search keywords", PAGE=2)Develop and testSeehttp://dev.abelujo.cc/webscraping.htmlDevelopment mode:pip install -e .Now you can edit the project and run the development version like the
lib is meant to be run, i.e. with theentry_points:livres,libros, etc.doc:https://python-packaging-user-guide.readthedocs.org/en/latest/distributing/#working-in-development-modeBugs and shortcomingsThis is webscraping, so it doesn’t go without pitfalls:the site can go down. It happened already.the site can change, it which case we would have to change our sraper
too. This can be catched early with automated and frequent tests
(work ongoing).Changelog0.2.1german scraper: search by isbn0.2.0German scrapermultiprocessing for the german scraper (from 15 to 9s) (seeissue
#1)--isbnoption for it0.1.xfrench, spanish scraperscommand line toolLicenceLGPLv3