Reading Manga with Python
Tags: python, manga
Published: 2020-05-12 20:21:39



Photo by Miika Laaksonen on Unsplash

## What is Manga ?
> **Manga** (漫画, *manga*) are [comics](https://en.wikipedia.org/wiki/Comics) or [graphic novels](https://en.wikipedia.org/wiki/Graphic_novel) created in [Japan](https://en.wikipedia.org/wiki/Japan) or using the [Japanese language](https://en.wikipedia.org/wiki/Japanese_language) and conforming to a style developed in Japan in the late 19th century. They have a long and complex pre-history in earlier [Japanese art](https://en.wikipedia.org/wiki/Japanese_art).

let’s say manga is Japanese comics which are more popular and interesting than most of the main stream comics.

### Scouting

Let’s learn some WebScraping and get some value instead of just getting data, let us download some manga from internet and try to read it.
> Reading manga online is easy, you just go to some site like mangapanda.com search some comics and read it. what if you want to download the entire comic compress each chapter to a particular volume and read it offline.

when we go to mangapanda.com and search for a particular comic like say naruto here’s what the URL we are directed to

![](https://cdn-images-1.medium.com/max/2000/1*uVD-rmR0l2HYjPy-qA8qLg.png)

Notice the naruto at the end of the URL, now if we go to the first chapter of naruto the URL transforms to [http://www.mangapanda.com/naruto/1](http://www.mangapanda.com/naruto/1) that’s just great for us. Note that this doesn’t happen with all the manga sites out there and watch out for that before trying to scrape any other manga site. we are trying to download the images that exists in naruto chapter 1

![](https://cdn-images-1.medium.com/max/4096/1*yyVS6eGaKVRkm03KYZiePQ.png)

Let’s write a small function to get the image from the URL

![](https://cdn-images-1.medium.com/max/2556/1*C9T0jR6SuRLjC73qj4qSDg.png)

OK, what is happening here. for the _download_image we are giving URL say mangapanda.com/naruto/1/3 according to our observation we are downloading naruto’s chapter 1 image 3 . let’s breakdown the function and understand what’s going on for each line.

* requests.get download the source of the given URL

* convert the source code html document into lxml html tree this helps us to parse tags easily

* get the tags with img with id=’img’ the expression, ensures that.

    ".//img[@id='img']/@src"

* after we get the image URL download the image with requests.get(URL).content

### Downloading the entire chapter

It’s good that the chapters are in the format /chapter/page_number so how can we download all the images of a particular chapter if we don’t know the ending chapter number. if we know the ending chapter then we can simply using range and loop over the image number to download.

if we see the source code there is this interesting tag.

![](https://cdn-images-1.medium.com/max/3804/1*UnxK9n429w-hB4ydJ3o0Gw.png)

There wrote this so that users can select the page number in the form of a dropdown. we can use the lxml format tree for this .//*[@id=’pageMenu’]/option[last()]/text() and get the last occurence of the pageMenu id which is the end page of the chapter.

let’s write wrap this up in a small function

![](https://cdn-images-1.medium.com/max/3400/1*ESbc2hiRJV8Ig91o86kA-A.png)

now, we know the page numbers of the chapters we are going to download. we can just get all the images from the chapter in parallel, sort them and then compress them to make a single volume.

let’s use ThreadPoolExecutor and write an async function for the following job.

![](https://cdn-images-1.medium.com/max/3940/1*FJTHvAc0YoJAg_JaGJYQ9A.png)

    properties = json.load(open("configs.json"))

    base_url = properties.get("base_url") + "/" + properties.get("manga_name")

we can define manga_name and base_url in configs.json so that we don’t have to give name of the manga every time we download a chapter.

download_chapter function creates directories based on the manga_name and chapter number

    ➜  naruto git:(master) ✗ tree
    .
    └── 1
        ├── 1.jpg
        ├── 10.jpg
        ├── 11.jpg

Now that we’ve downloaded all the pages in the chapter. let’s compress it in CBZ format and ensure that the order of the page numbers is sorted properly

![](https://cdn-images-1.medium.com/max/3332/1*L2SVuq1n6wVmb_1e97G07w.png)

we can wrap everything up with a classic __main__ so that if we give chapter number we will download the entire comic

![](https://cdn-images-1.medium.com/max/3432/1*fS8K4iF2sRm4X9XGSddJsA.png)

### In action

we can run the script in the following way

![](https://cdn-images-1.medium.com/max/2120/1*gamhdHl-KANkKYZTwIm-iA.png)
> Disclaimer: this is for pure educational purpose only. Do not use this commercially for piracy or for attacking mangapanda.com