Title: bat-country: an extendible, lightweight Python package for deep dreaming with Caffe and Convolutional Neural Networks - PyImageSearch

URL Source: https://pyimagesearch.com/2015/07/06/bat-country-an-extendible-lightweight-python-package-for-deep-dreaming-with-caffe-and-convolutional-neural-networks/

Published Time: 2015-07-06T14:00:24+00:00

Markdown Content:
[![Image 1: batcountry_fal_header_animated](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20262'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/batcountry_fal_header_animated.gif)

> We can’t stop here, this is bat country.

Just a few days ago, the [Google Research blog published a post](http://googleresearch.blogspot.com/2015/06/inceptionism-going-deeper-into-neural.html) demonstrating a unique, interesting, and perhaps even disturbing method to visualize what’s going inside the layers of a Convolutional Neural Network (CNN).

_**Note:** Before you go, [I suggest taking a look at the images generated using bat-country](#show_and_tell) — most of them came out fantastic, especially the Jurassic Park images._

Their approach works by turning the CNN upside down, inputting an image, and gradually tweaking the image to what the network “thinks” a particular object or class looks like.

_The results are breathtaking to say the least._ Lower levels reveal edge-like regions in the images. Intermediate layers are able to represent basic shapes and components of objects (doorknob, eye, nose, etc.). And lastly, the final layers are able to form the complete interpretation (dog, cat, tree, etc.) — _and often in a psychedelic, spaced out manner._

Along with their results, Google also published an [excellent IPython Notebook](https://github.com/google/deepdream) allowing you to play around and create some trippy images of your own.

The IPython Notebook is indeed fantastic. It’s fun to play around with. And since it’s an IPython Notebook, it’s fairly easy to get started with. _But I wanted to take it a step further_. Make it **modular**. More **customizable**. More like a Python modules that acts and behaves like one. And of course, it has to be **pip-installable** (you’ll need to bring your own Caffe installation).

**That’s why I put together [bat-country](https://github.com/jrosebr1/bat-country), an easy to use, highly extendible, lightweight Python module for inceptionism and deep dreaming with Convolutional Neural Networks and Caffe.**

Comparatively, my contributions here are honestly pretty minimal. All the real research has been done by Google — I’m simply taking the IPython Notebook, turning it into a Python module, while keeping in mind the importance of extensibility, _such as custom step functions_.

Before we dive into the rest of this post, I would like to take a second and call attention to Justin Johnson’s [cnn-vis](https://github.com/jcjohnson/cnn-vis), a command line tool for generating inceptionism images. His tool is quite powerful and more like what Google is (probably) using for their own research publications. If you’re looking for a more advanced, complete package, definitely go take a look at cnn-vis. You also might be interested in Vision.ai’s co-founder Tomasz Malisiewicz’s [clouddream](https://github.com/VISIONAI/clouddream) docker image to quickly get Caffe up and running.

But in the meantime, if you’re interested in playing around with a simple, easy to use Python package, go [grab the source from GitHub](https://github.com/jrosebr1/bat-country) or install it via `pip install bat-country`

**The rest of this blog post is organized as follows:**

*   **A simple example.** 3 lines of code to generate your own deep dream/inceptionism images.
*   **Requirements.** Libraries and packages required to run `bat-country` (mostly just Caffe and its associated dependencies).
*   **What’s going on under the hood?** The anatomy of `bat-country` and how to extend it.
*   **Show and tell.** If there is any section of this post that you don’t want to miss, _**it’s this one**_. I have put together a gallery of some really awesome images generated I generated over the weekend using `bat-country` . The results are quite surreal, to say the least.

Again, I want make it clear that the code for [bat-country](https://github.com/jrosebr1/bat-country) is heavily based on the work from the [Google Research Team](http://googleresearch.blogspot.com/2015/06/inceptionism-going-deeper-into-neural.html). My contributions here are mainly refactoring the code into a usable Python package, making the package easily extendible via custom preprocessing, deprocessing, step functions, etc., and ensuring that the package is pip-installable. With that said, let’s go ahead and get our first look at [bat-country](https://github.com/jrosebr1/bat-country).

A simple example.
-----------------

As I mentioned, one of the goals of `bat-country` is simplicity. Provided you have already installed Caffe and `bat-country` on your system, it only takes 3 lines of Python code to generate a deep dream/inceptionism image:

\# we can't stop here...
bc = BatCountry("caffe/models/bvlc\_googlenet")
image = bc.dream(np.float32(Image.open("/path/to/image.jpg")))
bc.cleanup()

After executing this code, you can then take the `image` returned by the `dream` method and write it to file:

result = Image.fromarray(np.uint8(image))
result.save("/path/to/output.jpg")

And that’s it! You can see the view source code of `demo.py` [here on GitHub](https://github.com/jrosebr1/bat-country/blob/master/demo.py).

Requirements.
-------------

The bat-country packages requires [Caffe, an open-source CNN implementation from Berkeley](http://caffe.berkeleyvision.org/), to be already installed on your system. This section will detail the basic steps to get Caffe setup on your system. However, an excellent alternative is to use the [Docker image provided by Tomasz of Vision.ai](https://github.com/VISIONAI/clouddream). Using the Docker image will get you up and running quite painlessly. But for those who would like their own install, keep reading.

### Step 1: Install Caffe

Take a look at the [official installation instructions](http://caffe.berkeleyvision.org/installation.html) to get Caffe up and running. Instead of installing Caffe on your own system, I recommend spinning up an [Amazon EC2 g2.2xlarge instance](https://pyimagesearch.com/2014/10/13/deep-learning-amazon-ec2-gpu-python-nolearn/) (so you have access to the GPU) and working from there.

### Step 2: Compile Python bindings for Caffe

Again, use the [official install instructions from Caffe](http://caffe.berkeleyvision.org/installation.html#prerequisites). Creating a separate virtual environment for all the packages from `requirements.txt` is a good idea, but certainly not required.

An important step to do here is update your `$PYTHONPATH` to include your Caffe installation directory:

export PYTHONPATH=/path/to/caffe/python:$PYTHONPATH

On Ubuntu, I also like to but this export in my `.bashrc` file so that it’s loaded each time I login or open up a new terminal, but that’s up to you.

### Step 3: Optionally install cuDNN

Caffe works fine out of the box on the CPU. But if you really want to make Caffe scream, you should be using the GPU. [Installing cuDNN](https://developer.nvidia.com/cudnn) isn’t too difficult of a process, but if you’ve never done it before, be prepared to spend some time working through this step.

### Step 4: Set your $CAFFE\_ROOT

The `$CAFFE_ROOT` directory is the base directory of your Caffe install:

export CAFEE\_ROOT=/path/to/caffe

Here’s what my `$CAFFE_ROOT` looks like:

export CAFFE\_ROOT=/home/ubuntu/libraries/caffe

Again, I would suggest putting this in your `.bashrc` file so it’s loaded each time you login.

### Step 5: Download the pre-trained GoogLeNet model

You’ll need a pre-trained model to generate deep dream images. Let’s go ahead and use the GoogLeNet model which Google used in their blog post. The Caffe package provides a script that downloads the model for you:

$ cd $CAFFE\_ROOT
$ ./scripts/download\_model\_binary.py models/bvlc\_googlenet/

### Step 6: Install bat-country

The `bat-country` package is dead simple to install. The easiest way is to use pip:

$ pip install bat-country

But you can also pull down the [source from GitHub](https://github.com/jrosebr1/bat-country) if you want to do some hacking:

$ git clone https://github.com/jrosebr1/bat-country.git
$ cd bat-country
... do some hacking ...
$ python setup.py install

What’s going on under the hood — and how to extend bat-country
--------------------------------------------------------------

The vast majority of the `bat-country` code is from Google’s IPython Notebook. My contributions are pretty minimal, just re-factoring the code to make it act and behave like a Python module — and to facilitate easy modifications and customizability.

The first important method to consider is the `BatCountry` constructor which allows you to pass in custom CNNs like GoogLeNet, MIT Places, or other models from the [Caffe Model Zoo](https://github.com/BVLC/caffe/wiki/Model-Zoo). All you need to do is modify the `base_path` , `deploy_path` , `model_path` , and image `mean` . The `mean` itself will have to be computed from the original training set. Take a look at the [BatCountry constructor](https://github.com/jrosebr1/bat-country/blob/master/batcountry/batcountry.py) for more details.

The internals of `BatCountry` take care of patching the model to compute gradients, along with loading the network itself.

Now, let’s say you wanted to override the standard gradient ascent function for maximizing the L2-norm activations for a given layer. All you would need to do is provide your custom function to the `dream` method. Here’s a trivial example of overriding the default behavior of the gradient ascent function to use a smaller `step` , and larger `jitter` :

def custom\_step(net, step\_size=1.25, end="inception\_4c/output",
	jitter=48, clip=True):
	src = net.blobs\["data"\]
	dst = net.blobs\[end\]

	ox, oy = np.random.randint(-jitter, jitter + 1, 2)
	src.data\[0\] = np.roll(np.roll(src.data\[0\], ox, -1), oy, -2)

	net.forward(end=end)
	dst.diff\[:\] = dst.data
	net.backward(start=end)
	g = src.diff\[0\]

	src.data\[:\] += step\_size / np.abs(g).mean() \* g
	src.data\[0\] = np.roll(np.roll(src.data\[0\], -ox, -1), -oy, -2)

	if clip:
		bias = net.transformer.mean\["data"\]
		src.data\[:\] = np.clip(src.data, -bias, 255 - bias)

image = bc.dream(np.float32(Image.open("image.jpg")),
	step\_fn=custom\_step)

Again, this is just a demonstration of implementing a custom step function and not meant to be anything too exciting.

You can also override the default `preprocess` and `deprocess` functions by passing in a custom `preprocess_fn` and `deprocess_fn` to `dream` :

def custom\_preprocess(net, img):
	# do something interesting here...
	pass

def custom\_deprocess(net, img):
	# do something interesting here...
	pass

image = bc.dream(np.float32(Image.open("image.jpg")),
	preprocess\_fn=custom\_preocess, deprocess\_fn=custom\_deprocess)

Finally, bat-country also supports visualizing each octave, iteration, and layer of the network:

bc = BatCountry(args.base\_model)
(image, visualizations) = bc.dream(np.float32(Image.open(args.image)),
	end=args.layer, visualize=True)
bc.cleanup()

for (k, vis) in visualizations:
	outputPath = "{}/{}.jpg".format(args.vis, k)
	result = Image.fromarray(np.uint8(vis))
	result.save(outputPath)

To see the full `demo_vis.py` script on GitHub, [just click here](https://github.com/jrosebr1/bat-country/blob/master/demo_vis.py).

Show and tell.
--------------

I had a lot of fun playing with `bat-country` over the weekend, specifically with images from _Fear and Loathing in Las Vegas_, _The Matrix_, and _Jurassic Park_. I also included a few of my favorite desktop wallpapers and photos from my recent vacation on the western part of the United States for fun.

For each of the original images _(top)_, I have generated a “deep dream” using the `conv2/3x3` , `inception_3b/5x5_reduce` , `inception_4c/output` layers, respectively.

The `conv2/3x3` and `inception_3b/5x5_reduce` layers are lower level layers in the network that give more “edge-like” features. The `inception_4c/output` layer is the final output that generates trippy hallucinations of dogs, snails, birds, and fish.

### Fear and Loathing in Las Vegas

[![Image 2: output_fal_01](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%201078'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/output_fal_01.jpg) [![Image 3: output_fal_02](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%201078'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/output_fal_02.jpg) [![Image 4: output_fal_03](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%201044'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/output_fal_03.jpg) [![Image 5: output_fal_04](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%201046'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/output_fal_04.jpg)

### Jurassic Park

[![Image 6: output_jp_06](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%201375'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/output_jp_06.jpg)

**This is DEFINITELY my personal favorite.**

[![Image 7: output_jp_01](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%201224'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/output_jp_01.jpg) [![Image 8: output_jp_02](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%201322'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/output_jp_02.jpg) [![Image 9: output_jp_03](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%201375'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/output_jp_03.jpg) [![Image 10: output_jp_04](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%201306'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/output_jp_04.jpg) [![Image 11: output_jp_05](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%201638'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/output_jp_05.jpg)

### The Matrix

[![Image 12: output_matrix_01](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%201015'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/output_matrix_01.jpg)

[![Image 13: output_matrix_02](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%201628'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/output_matrix_02.jpg) [![Image 14: output_matrix_03](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%201375'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/output_matrix_03.jpg) [![Image 15: output_matrix_04](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%201015'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/output_matrix_04.jpg)

### Antelope Canyon (Page, AZ)

[![Image 16: antelope_canyon_01](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20223'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/antelope_canyon_01.jpg)

[![Image 17: antelope_canyon_02](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20395'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/antelope_canyon_02.jpg)

### Angels Landing (Zion Canyon; Springdale, UT)

[![Image 18: angels_landing_01](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20395'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/angels_landing_01.jpg)

[![Image 19: angels_landing_02](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20223'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/angels_landing_02.jpg)

### Zoo (Phoenix, AZ)

[![Image 20: giraffe](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20223'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2015/07/giraffe.jpg)

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

Summary
-------

In this blog post I introduced [bat-country](https://github.com/jrosebr1/bat-country), an easy to use, highly extendible, lightweight Python module for inceptionism and deep dreaming with Convolutional Neural Networks and Caffe.

The vast majority of the code is based on the [IPython Notebook](http://nbviewer.ipython.org/github/google/deepdream/blob/master/dream.ipynb) published by the [Google Research blog](http://googleresearch.blogspot.com/2015/06/inceptionism-going-deeper-into-neural.html). My own contribution isn’t too exciting, I have simply (1) wrapped the code in a Python class, (2) made it easier to extend and customize, and (3) pushed it to PyPI to make it pip-installable.

If you are looking for a more robust deep dreaming tool using Caffe, I really suggest taking a look at Justin Johnson’s [cnn-vis](https://github.com/jcjohnson/cnn-vis). And if you want to get your own Caffe installation up and running quickly (not to mention, one that has a web interface for deep dreaming), take a look a [Tomasz’s docker instance](https://github.com/VISIONAI/clouddream).

Anyway, I hope you enjoy the `bat-country` ! Feel free to comment on this post with your own images generated using the package.

![Image 21](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Join the PyImageSearch Newsletter and Grab My FREE 17-page Resource Guide PDF

Enter your email address below to **join the PyImageSearch Newsletter** and **download my FREE 17-page Resource Guide PDF** on Computer Vision, OpenCV, and Deep Learning.