Connecting to files in Box and analyze- Unstructured data analysis
By: Rajesh Gudikoti
Tags: NLP, Unstructured Data, Machine Learning

Box files

What is Box?

Box is cloud content management and file sharing service. It provides Secure collaboration with anyone, anywhere, on any device.

Problem statement

Organization uses Box to store the data in files. It is one of the best to collaborate and share contents with team or across organization. Once information gets stored in Box files, there will be business need to extract information from the box files.

How do we address this problem?

One of the easiest way is to integrate Box with Watson Discovery and Discovery will do the analysis for you.

We will see how this can be achieved…..

In previous article I had mentioned about analysing selective contents from web. In this article we will connect Box data source and analyse the contents. The Box can hold folders and files and we will extract information from unstructured data stored in files.

For connecting to Box source, first thing is, we need to have right access for box. Once we have access, we can create folders and files. If you do not have box account, you can navigate below link and sign up to create your personal Box account(sign up link →Individual Plans tab- free/other option). Login into account and create folder/files as shown in screenshot below.

https://account.box.com/login

Box folders and files

I have contents in Box and want to analyse the contents. I am using Watson discovery to do the analysis and extract info for me. Later I can query discovery and get the required information. To integrate Box with discovery, there are pre-requisites to be followed. The pre-requisites are mentioned in below link.

https://cloud.ibm.com/docs/discovery?topic=discovery-sources

Creating custom application

As part of pre-requisite, we need to create Box custom application.

For article, I will be referring my personal Box account. The same can be done with your enterprise account which has access to Box source in your organisation. When we create application in Box, we can set Application level access or Enterprise level access.

Note: If you have Enterprise level access, modified documents are crawled during a refresh and for Application level it has to go through admin approval process (Refer admin Authorization process steps).

Navigate to below link to create custom application

https://app.box.com/developers/console

choose Custom App

Choose JWT option

Change Application to Enterprise Access and enable Advanced features and Save changes

Change Application to Enterprise Access and enable Advanced features and Save changes

Need to need to generate Public-Private Keys. It will ask for two level authentication.

json configuration file is generated, download it

Please keep configuration handy, we need it in next steps.

Navigate to General Settings tab and click Review and Submit. This will create review request and mail this request to box admin. In my case, I am using personal account and I am admin, so i will get mail to my registered email address.

Please observe the state of application (it is Disabled).

Admin Review and Approval Process

Click Authorize after reviewing

(Refresh your Box Custom application)Post Approval observe the status change. It is Enabled now.

We are done with creating Box Custom application.

Integration of Box with Watson Discovery

The first pre-requisite here needed, is to have IBM cloud account. You can create new IBM cloud account navigating to below link.

https://cloud.ibm.com/registration

Login into IBM cloud account, navigate to catalog page, choose AI/Machine Learning on left navigation present under category, select Discovery and create Discover service.

Click your Discovery Service

Launch Discovery Service

Choose Connect to data source to connect to different data sources

Different Data Sources

Click Box and fill connection details. The complete private key has to placed(including — — Begin ***** to END ENCRYPTED PRIVATE KEY-----

)

Once Box is connected successfully, Click View Content

Choose folders and files to be ingested into discovery.

choose folders and files and click Save and Synch

After clicking on Save and Synch button, files are ingested into discovery. It takes some time based on your content size. Once ingestion and enrichment are done, you can view the important information like entities, concepts etc.

You can see the files in Box folders have been crawled and ingested in Discovery.

You can change sync settings any time after initial ingestion.

You can query your content and look for results.

Contents of pdf file ingested in discovery which is shown as result based on your query.

Note: You can parse contents in table and get results using SDU feature in Discovery. Also, there is option to show complete table as part of your query result in Cloud Pak for Data offering.