Moving towards Containers: Deployment of Jupyter Notebook using OpenShift
Tags: Blog Post, Openshift, Jupyter Notebook, Big Data, Containers

Technology innovation is often driven by two things: agility and mobility. The technology that exists today will steadily surpass itself into a much-advanced form in the future. We all have started from the age of GPRS and are now advancing towards 6G. Similarly, the field of Container technology emerged with Unix, then came Linux and then the momentum continued with the introduction of Kubernetes in 2017.

In this digital age, industries are moving towards a container-based platform. If you are unfamiliar with the term container, don’t worry we’ve got you covered! Containers are the basic units. It is where the application instances are running. This link explains about the container technology.

Benefits of containers :

Less overhead

Consistency in operation

Accelerate Application Development

Greater efficiency

Flexibility

To put it in simple terms, a container consists of an entire run time environment which comprises an application, plus all its dependencies, the libraries and other binaries, and configuration files needed to run it, which are all bundled into one package.

Containers solve the problem of getting the software to run reliably when moved from one computing environment to another. Like migrating from laptop to a test environment and from test environment to production. Or maybe from a physical machine to a virtual machine in the cloud. The problem of non-identical supporting software environments while migrating from one system to another is solved by containers.

OpenShift Platform!

Enterprises across the world are using micro services and containers to build applications faster and to deliver and scale intelligently across hybrid cloud environments. But before proceeding they need the right platform. Open Shift is the Kubernetes platform that provides a trusted foundation for hybrid and multi-cloud deployments that today’s industries demand. With automated operations and streamlined life-cycle management, Redhat Openshift empowers development teams to build and deploy new applications. They can access to the data in real time and get native access to a wide range of third-party cloud services all through a single platform.

OpenShift is the industry-leading Kubernetes based container platform that provides a self-service seamless experience for Data Scientists to easily deploy and use the Machine Learning tools, empowering them to create and share the modeling results in a consistent way with peers and software developers without dependency.

Why OpenShift platform?

Most of the platforms such as Amazon’s ECS or Google Kubernetes engine are all built for their specific environment. OpenShift container platform, on the other hand, is built in such a way that allows customers to deploy many platforms and provide the same features and tools for developers. OpenShift provides consistency in its usage, meaning, deployment across multiple public and private cloud environments can be used by developers and other operations personnel in exactly the same way as it is intended to be. OpenShift provides for both flexibility as well as simplicity and allows to deploy applications in different programming languages and handle different database. Moreover, OpenShift comes with Red Hat Enterprise Linux (RHEL) as part of its support. All the elements from the host operating systems to the application services layer is supported by the Red Hat.

High efficiency of containerization at the enterprise level comes from the usage of containers throughout the environment. OpenShift provides a platform for managing the containers across various environments, which also reduces the time needed for building, deploying and scaling them. In conclusion, the level of support for the OpenShift platform is unlike any other. There is support for the infrastructure and run time for applications. You will rarely find services with such support for the container or the system hosting the containers.

Now that we know what OpenShift is and how the containers work, let us begin by running an application on it. The following content shows how to deploy Jupyter Notebook on OpenShift.

About Jupyter Notebook

Jupyter Notebook is an open-source web application that serves as a single entity that facilitates creating and sharing documents that can contain codes, data, any mathematical or statistical equations, visualizations, explanations, all embedded into one. Jupyter Notebook is well-suited for demonstrations of programming concepts and training. It may not be wrong to say that Data Scientists love the notebook interface. Jupyter notebook provides an instinctive workflow that promotes iterative and rapid development, making them a popular choice at the heart of contemporary data science and related analysis. In a single window, we enter the inputs and generate outputs. It is a fundamental way of telling stories. Stories that describe the insights into the relevant data.

How to deploy Jupyter Notebook ?

The following images guide through a step by step procedure in deploying Jupyter Notebook in OpenShift.

Launch the OpenShift(OKD) console in Developer Mode by running Local OpenShift Cluster with Minishift. To know more about how to run a local OpenShift cluster, please click here.

2. Create a Project by giving appropriate Name, Display Name and Description:

3. Copy the Docker image name from here. For your reference, the Image name will be — jupyter/minimal-notebook

Now go to Overview menu and click on Deploy Image:

4. Paste the copied name in the search bar of the Image name and click on the🔍 icon. After a successful search, the image will be loaded. Give an appropriate Image Name and click Deploy

5. The Jupyter Notebook will be successfully created as shown below:

6. You can find your Deployments in the Applications tab as shown in the image below:

7. As Health Check is a very important aspect of OpenShift, configure the health checks as shown in the image below. Update both the Initial Delay field as highlighted and keep the other fields same:

8. Click Save and OpenShift will trigger a build generation:

9. Now, Create a Route for your application using Route option under Applications:

10. After the successful creation of the route, you will get a URL so as to use the deployed application like Jupyter Notebook over the internet. Please refer to the image below:

11. The Jupyter Notebook will ask you for a password. As we did not add a password(Option is not available!), click on Logs which is under the Deployments tab.

12. Copy the URL as highlighted in Logs below and paste it in your browser.

13. Next, copy the token from the URL which is needed for authentication and paste it in Password or tokens and Log In:

14. Now your Jupyter Notebook is ready to use.

Adding a Persistent Volume !

While working with a Jupyter Notebook, one has the flexibility to create new notebooks or upload an existing notebook. But the catch here is that any changes made will be saved to the local file system within the container. As a result, if the container running the Jupyter Notebook instance is restarted, all the work will be lost. Hence, this should be taken care of.

To avoid this, we must configure the OpenShift cluster with persistent volumes. To claim and mount the persistent volume, you can use the oc set volume command. A persistent volume claim can also be made, and associated with the Jupyter Notebook application from the web console by going to the Deployment Config for the Jupyter Notebook application.

Running a Machine Learning Model in Jupyter Notebook

Now that we’ve looked at how to deploy Jupyter Notebook on OpenShift, it’s time to look at how they’re used in practice, which should give you a clearer understanding of why they are so popular among the Data Scientists and Analysts.

Analyzing the Customer Churn Behavior in the Telecom industry

So let’s say you have been tasked with finding out whether the customer will continue the services with the company or else will switch (churn) to other Telecom service provider. Following section shows how to design and implement an interactive Machine Learning project for the above mentioned task using Jupyter Notebook on OpenShift. The Notebook and code comes from an actual project that was done by the team. The Github link is mentioned here. Video snippet of the code is as follows.

Churn Prediction: GITHUB

We have trained Gradient Boosting model for classification with an AUC score of 0.85. Detailed explanation about the steps implemented for predicting whether the customer will churn or not has been mentioned in the Notebook which you can download and have a look.

Conclusion

Containers are highly efficient vehicles for developing and deploying apps. OpenShift brings professional and production-oriented experience. It’s about giving users a professional and efficient way of deploying and manage containerized applications. OpenShift offers a platform for managing your containers across a variety of operating environments, significantly reducing the time necessary to build, deploy, and scale them.

While on Openshift everything is based on conatiners and kubernetes. One downside is that your application must implement kubernetes pattern design. We may hope in the near future OpenShift comes up with much more flexibilty to address the issue.

Now that you know so much, go ahead and try it out yourself!

Thanks for reading our blog. If you like it, Clap, Follow and Share!

References: