Download All Free Textbooks from Springer using Python | by Joe T. Santhanavanich | Towards Data Science

Good news to all data scientists and researchers🎉🎉 Springer has announced to give hundreds of expensive books on Science and technology worth thousands of dollars available for free download during this COVID-19 lockdown. Over more than 500 textbooks are available!
If you are interested only in some of these books, you may download them one by one. But this chance won’t happen often, what about getting them all?😊 Download manually yourself would be boring and tiresome. Why don’t let Python do this job for you?🤔 According to this post, these Spring textbooks are only available until the end of July. The golden time might be extended according to the pandemic situation...
This article will show you how to automatically download all Springer textbooks from the list table using Python with a step-by-step guide. So, no worry if you are new to programming or Python.
Proof:
As you can see, only 11 lines of Python script can download all these textbooks for you in some minutes. After you read this article, you will able to do this process yourself and able to apply this technique to download a list of any items or files in the future too. ✌
The coronavirus outbreak is having an unprecedented impact on education, putting particular strain on academics and their students. In the midst of much uncertainty and rapidly changing information, we know many educators are working to make the transition to a virtual classroom environment, finding ways to engage with students online and make sure they have the resources they need to continue their studies. To try and ease the tremendous pressure which is being put on academia right now, Springer Nature has made a range of essential textbooks from all disciplines freely available to help support students and instructors worldwide. [1]
Niels Peter Thomas, Managing Director Springer Nature Books, said: “As the global impact of the crisis intensifies, remote access to educational resources has become essential. We want to support lecturers, teachers and students during this challenging period and hope that this initiative, which will see over 500 key textbooks available for free online, will go some way to help.” [2]
Now, let’s take a look at how we can use Python to do this job!
First, let’s prepare all requirements for your python. If you don't have Python in your PC yet, you may download it at the official site here.
Please, prepare Python libraries using pipby opening your command line and using the following commands:
$ pip install pandas$ pip install wget$ pip install requests$ pip install xlrd
After the installation is done, create a project folder and download an excel list of the available free textbooks from Springer (Free+English+textbooks.xlsx) here.
The link URL may change over time and you can find an updated version of this excel sheet at this page and click the link “Free English textbook titles” as a figure below.
Then, create a Python file download_textbooks.py and prepare the folder named download inside the project folder. Inside our project folder will look like this:
Then, edit the download_textbook.py using the editor you like and adding the following lines to import Python modules and to load Springer textbook excel into the pandas dataframe. And let’s explore it by using df.head(10)command to check the first 10 rows of dataframe.
import requestsimport wgetimport pandas as pddf = pd.read_excel("Free+English+textbooks.xlsx")print(df.head(10))
In this dataframe, the main columns we need at the moment are “Book Title” and “Edition” for forming a filename, and “OpenURL” to get a download link. So our plan now is to loop through the excel list in each row using for loop.
What shall we do for each loop:
Create a filename variable for each book by “Book Title” and “Edition” column. Please note that we can use any mixture of columns to generate the file name.
Download a book using wget command and give a name with filename variable.
So we can convert this plan to Python Script:
for index, row in df.iterrows():    file_name = f"{row.loc['Book Title']}_{row.loc['Edition']}"    url = f"{row.loc['OpenURL']}"    wget.download(download_url, f"./download/{file_name}.pdf")
Looks good, right? However, this method would not work because the “OpenURL” column does not directly represent the final URL for downloading PDF files yet. So, we need to change this OpenURL to the correct endpoint URL. For example, if we open the link of one textbook from “OpenURL” column on our web browser:
1.http://link.springer.com/openurl?genre=book&isbn=978-0-306-48048-5
Oh, it is redirected! 🤨 It is automatically redirected to
2.https://link.springer.com/book/10.1007%2Fb100747
Then if we click on the download PDF button in the website, the real endpoint PDF file will be:
3.https://link.springer.com/content/pdf/10.1007%2F0-387-36274-6.pdf
That means we should convert from 1. to 3. first in each loop! So, additional steps we should do in the for loop are
Open the link from the “OpenURL” column.
Get the redirected URL: I found the simplest to do this is to use request module.
Reformat the structure of the URL string to be the endpoint URL for downloading the PDF file. We can do it using str.replace(old,new) .
Download a book using wget as planned.
Then, the overall Python codes would look like this:
After you run the Python script with python download_textbooks.py , you would get all the textbooks in your download folder.
So, that’s about it. I hope you enjoy this article and able to apply this idea to automate some of your workflows in the future. Please, note that there are several ways to do this with Python as several modules can get the job done. Feel free to share with me if you have some questions, comments, or suggestions.
Be Safe and Healthy!
Thank you for Reading. 👋😄
[1] Lucy Frisch, Here’s how you can access textbooks for free during the coronavirus lockdown (Apr 16, 2020), Springer Nature
[2] Felicitas Behrendt, Springer Nature Group, Springer Nature makes key textbooks freely accessible for educators, students and academics affected by coronavirus lockdown (2020), Springer Nature