vania-janet/multiturn-rag-retrieval-data huggingface dataset:
* Test dataset: Calculates the score of the trained model.



13. Click Create.

It might take several minutes for model training to complete.







 Deploying the document classifier model 

After the model is trained, deploy the model as an enrichment.



1. Click the overflow menu icon in the Actions column, and then click Deploy model. Specify the name and other details, and then click Deploy.
2. Do one of the following things:



* To apply the document classifier to a collection in your Content Mining project, see [Enriching your collection](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-cm-edit-collectioncm-enrichments).
* To apply the document classifier to a collection in a different project, complete the following steps:



1. In Discovery, create or open the collection that has the documents that you want to classify.

The data in the collection where you apply the enrichment must have the same fields as the collection that you used to train the model.
2. In the Enrichments tab, locate your classifier in the Name column. From the Fields to enrich field, choose the same text field that was used to train the model. (This field is determined by the system and is indexed as the Analyzable text content field. For more information, see [Identifying the text field](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-cm-edit-collectiontext-field).)
3. Click Apply changes and reprocess.











 Results of classification 

After the enrichment is applied to a collection, a facet is generated that you can use to find the predicted classes. In this example, the predicted field is named label_answer_predicted.

Zoom

!
[Download and install the Data Crawler](https://cloud.ibm.com/docs/discovery?topic=discovery-downloading-and-installing-the-data-crawler) on a supported Linux system that has access to the content that you want to crawl.
3. [Connect the Data Crawler](https://cloud.ibm.com/docs/discovery?topic=discovery-configuring-connector-and-seed-options) to your content.
4. [Configure the Data Crawler](https://cloud.ibm.com/docs/discovery?topic=discovery-configuring-the-data-crawler) to connect to the Discovery Service.
5. [Crawl your content](https://cloud.ibm.com/docs/discovery?topic=discovery-crawling-your-data-repository).



You can get started quickly with the Data Crawler by following the example in [Getting started with the Data Crawler](https://cloud.ibm.com/docs/discovery?topic=discovery-getting-started-with-the-data-crawler).
Searching Discovery data from Watson Assistant 

Your Discovery project can provide answers to questions that stump your assistant. Instead of answering with “I don't know”, your assistant can say, “I'm not sure, but I searched my knowledge base and found these answers which might help.”

For more information about how to search a Discovery project from an assistant, read the appropriate Watson Assistant documentation for your situation.



* From the new experience user interface, see [Search trigger](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-search-addsearch-add-trigger).
* From an actions skill in the classic user interface, see [Configuring the search for an answer](https://cloud.ibm.com/docs/assistant?topic=assistant-actionsactions-what-next-search).
* From a dialog skill, see [Adding a search skill response type](https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-overviewdialog-overview-add-search-skill).



If you use the built-in web chat, you can use answer finding by enabling the Emphasize the answer feature. Answer finding highlights the word or phrase in the search result that is determined to be the exact answer to the customer's question.

For a more detailed look at the steps to take to connect to a Discovery project from Watson Assistant, take a tutorial that walks you through them. For more information, see [Power your assistant with answers from web resources](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-tutorial-assistant-fred).

Alternatively, you can add a generative language service named NeuralSeek between the Watson Discovery and Watson Assistant services. For more information, see [Use NeuralSeek to return polished answers from existing help content](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-tutorial-neuralseek).



 How the assistant calls Discovery 

When a user asks your assistant a question that triggers a search, the following API request is sent to Discovery if Emphasize the answer is enabled.
You are prompted to choose a collection to train.



2. On the Train Watson screen, click Add a natural language query, for example: "IBM Watson in healthcare", and add it. Make sure your queries are written the way your users would ask them. Also, it is recommended that training queries be written with some term overlap between the query and the desired answer. This overlap improves initial results, when the natural language query is run. Relevance training only uses natural language queries, so do not enter queries written in the Discovery Query Language.
3. To view the results of your query, click the Rate Results button next to it. If you don't think there are enough results, you could try rewriting the query, or adding more documents to this collection via the Manage data screen.
4. Begin rating results as either Relevant or Not relevant. When you are done, click Back to queries. In the Discovery tooling, Relevant has a score of 10 and Not relevanthas a score of 0. If you already started rating results for this collection, using the API, and used a different scoring scale, a warning is displayed, with options to fix the issue. At the top of the screen, Watson tracks the training status and provides tips about what you can do to improve results. "Add more variety to your ratings" means that you might want to use both the Relevant and Not relevant ratings. After you meet the requirements, training starts updating periodically. Most training takes less than 30 minutes to complete, but you can continue working while training is underway.
5. Continue adding queries and rating results.



To return to the main Build queries screen at any time, click Build queries on the upper left. To return to the Manage data screen, click the name of the collection on the upper right.

If you would like to delete all of the training data in your collection at one time, you must do so via the API. See [Delete all training data for a collection](https://cloud.ibm.com/apidocs/discoverydelete-all-training-data) for more information.
[Shows a custom machine learning facet.](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/nlu-search.png)

Figure 2. Machine learning facet for filtering search results with custom enrichment values

If extracting meaning is your goal, complete the steps that are listed in the following table.



Checklist for extracting meaning

 Step Task Related information 

 ![checkbox](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/checkbox.png) Create any project type. [Creating projects](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-projects) 
 ![checkbox](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/checkbox.png) Add collections that connect to external data sources or contain uploaded files. [Creating collections](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-collections) 
 ![checkbox](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/checkbox.png) Chunk large documents into many smaller documents so you can apply more targeted enrichments to the content. [Using Smart Document Understanding](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-configuring-fields) 
 !
When the JSON file is ingested, each item in the array is added as a separate document with a seprarate document ID. Each document shares the same parent ID, which identifies the relationship between them.

You can quickly find documents that share the same parent ID or other common metadata value from the Manage data page. Customize the view to show the field, such as extracted_metadata.parent_document_id or extracted_metadata.foldername, that the documents share in common.
* Can I customize Discovery to understand my data

Yes. Use the intuitive tools provided with the product to teach Discovery about the unique terminology of your domain. For example, you can teach it to recognize patterns, such as BOM or part numbers that you use, or add dictionaries that recognize your product names and other industry-specific word meanings. For more information, see [Adding domain-specific resources](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-domain).
* How does the Smart Document Understanding tool work?

You can use the Smart Document Understanding tool to teach Discovery about sections in your documents with distinct format and structure that you want Discovery to index. You can define a new field, and then annotate documents to train Discovery to understand what type of information is typically stored in the field. For more information, see [Using Smart Document Understanding](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-configuring-fields).
* What's the best way to add synonyms?

You can use two different methods to define synonyms.



* To define synonyms that are recognized and tagged when a document is ingested and that can be retrieved by search, create a dictionary and add synonyms for the dictionary term entry. A dictionary defines special terms that you want to tag in your documents, such as product names or industry-specific terminology. You can use the dictionary terms later to create facets and to filter documents. For more information, see [Dictionary](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-domain-dictionary).
Or you can apply a custom model that recognizes terms and categories that are unique to your data.

[Extend your analysis by adding your own facets](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-cm-add-facets).





 Getting started 

Before you can use the application, you must create a Discovery Content Mining project. After the project is created and data is uploaded, you can open the Content Mining application.

For more information, see [Creating projects](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-projects).

Of course, you can't get out useful insights if you don't put the right type of information in. Be sure to include consistent data. If you want to find trends over time, your data must include data points that specify a date.

Data that is submitted in CSV file format is optimal. For a sample of a CSV file that provides interesting analysis capabilities, see [Analyzing CSV files](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-cm-csv-file).
The dictionary is downloaded as a CSV file.



2. From your Discovery Content mining project, open the Content Mining application.
3. From the analysis view of your collection, click the Collections link in the breadcrumb to open the Create a collection for your analytics solutions page of the Content Mining application.
4. To create an annotator, click collection, and then select custom annotator from the list.

Zoom

![Shows the collection menu](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/cm-collection-menu.png)

Figure 2. Collection menu
5. Click Create custom annotator.
6. Name your annotator, and then optionally add a description.
7. From the Annotator Type menu, select Dictionary, and then click Next.
8. Click the Import button, and then upload the CSV file of the dictionary that was downloaded from oneWEX.
9. Click Import, and then click Save.







 Importing character patterns from Watson Explorer oneWEX 

You can import [character patterns](https://www.ibm.com/support/knowledgecenter/en/SS8NLW_12.0.0/com.ibm.watson.wex.ee.doc/c_ce_adm_char_ann.html) from IBM Watson Explorer oneWEX.



1. From your Discovery Content mining project, open the Content Mining application.
2. From the analysis view of your collection, click the Collections link in the breadcrumb to open the Create a collection for your analytics solutions page of the Content Mining application.
3. To create an annotator, click collection, and then select custom annotator from the list.

Zoom

!
(https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/cm-dict-lift-up.png)

Figure 3. Dictionary enrichment application

However, a user must click the Food>Vegetables facet explicitly to get documents that mention terms in the Vegetables dictionary, such as lettuce, to be returned.

Zoom

![Shows that only documents that mention vegetables are returned when the Vegetables facet is selected.](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/cm-dict-veg.png)

Figure 4. Subfacets
7. Repeat previous steps to add more dictionaries.
8. Click Save.



From the custom annotator page, you can see dictionaries that were created in other projects, including non-Content Mining projects. Dictionaries from other project types show the enrichment name as the annotator name. The Ignore case and Lift up words settings are disabled and the dictionary is named custom dict.



 Dictionary limits 



Dictionary plan limits

 Plan Number of dictionaries per service instance Number of base words per dictionary Number of terms for which suggestions can be generated 

 Cloud Pak for Data Unlimited Unlimited 1,000 
 Premium 100 10,000 1,000 
 Enterprise 100 10,000 1,000 



Totals include enrichments that you create in this Content Mining project and in other projects in the same service instance.







 Machine learning configuration 

You can import an existing machine learning model.

To use Discovery to create a model, see [Entity extractor](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-entity-extractor).

To import a model, complete the following steps:



1. Click Select file, and then browse for the machine learning model file.
To find trends, complete the following steps:



1. From the initial search page, enter a keyword or select a facet with number values to filter the documents.
2. Click Find trends and anomaly from the list of suggested next steps that is displayed in the guided mode view.

The resulting bar graph shows the number of documents that mention the term or facet value that you specified in the search query over time.

Zoom

![Facet trend heat map](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/cm-heatmap.png)

Figure 3. Facet trend graph

The time series chart is rendered as a heat map. Each cell color indicates a level of relevancy.
3. You can click a facet to investigate it more closely. The facet is shown in a bar graph.

Zoom

![Facet trend graph](https://cloud.ibm.com/docs-content/v1/content/f16b1f78a92498247223b337bd9303c2758b9ba8/discovery-data/images/cm-bar-graph.png)

Figure 4. Facet trend detail in bar graph

Each individual bar graph highlights trends in your data that deviate from the normal distribution by displaying increase indicators.

Increase indicators measure how much the frequency of a facet value on a specific date or in a particular time interval deviates from the expected average frequency. The average is calculated based on the changes in the past time interval frequencies.



You can click individual items in a visualization or click and drag the cursor to select contiguous items.

The cyclical data is calculated from the current time zone setting of your collection.
* [Cloud Pak for Data](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-exportimportwks_exportimport_expimp_doc)



Although you must download both the document set and type system to include annotations in documents that you upload to another Knowledge Studio workspace, the same is not true in this use case. You import only the document set to Discovery. Any annotations in the documents are recreated in Discovery. The Knowledge Studio type system is not needed.

To import a Knowledge Studio corpus, complete the following steps:



1. Open the project where you want to import the corpus.
2. From the Improvement tools panel of the Improve and customize page, expand Teach domain concepts, and then click Extract entities.
3. Click the arrow that is associated with the New button. and then click Import a Knowledge Studio corpus.
4. Add an extractor name and optionally a description.

This name is used as the model name and as the name of the enrichment that is created when you publish the model. The name is displayed as the enrichment name in the Enrichments page where you and others can apply it to collections. It also is displayed as the model name in the JSON representation of documents where custom entities are found. The name is stored with the capitalization and spacing that you specify.
5. Click Upload, and then browse to find and select the .zip file that you exported from Knowledge Studio. Click Create.

The annotated documents that you upload are stored with the entity extractor workspace, not as a new collection in the project. You can continue to annotate the documents.



Give Discovery some time to import and process the machine learning model corpus. After the entity extractor is created, the extractor is opened to the Label documents page.





 Training the extractor 

After you label documents, review the training data that will be used to train the entity extractor model.

To train the extractor, complete the following step:



1. Decide whether you want to apply an advanced option.
* [Get the Code](https://github.com/IBM/watson-discovery-analyze-data-breaches?cm_sp=IBMCode-_-import-enrich-and-gain-insight-from-data-_-Get-the-Code)
* [View the Demo](https://www.youtube.com/watch?v=zAu9tHefdDc&cm_sp=IBMCode-_-import-enrich-and-gain-insight-from-data-_-View-the-Demo)



[Cognitive Retail Chatbot](https://developer.ibm.com/patterns/create-cognitive-retail-chatbot/?cm_sp=Developer-_-code-_-retail_chatbot) Create a chatbot dialog using Watson Assistant, a Cloudant NoSQL database, Watson Discovery, and a Slack group.



* [Get the Code](https://github.com/IBM/watson-online-store/?cm_sp=IBMCode-_-create-cognitive-retail-chatbot-_-Get-the-Code)
* [View the Demo](https://www.youtube.com/watch?v=b-94B3O1czU&cm_sp=IBMCode-_-create-cognitive-retail-chatbot-_-View-the-Demo)



[Cognitive News Search App](https://developer.ibm.com/patterns/create-a-cognitive-news-search-app/?cm_sp=Developer-_-code-_-trending_news) Build your own news mining web application using JavaScript, Node.js, and the Watson Discovery service. Use the Watson Node.js SDK to build your news app to search the latest news, find trends, and even integrate it with other applications, such as Slack.



* [Get the Code](https://github.com/IBM/watson-discovery-news/?cm_sp=IBMCode-_-create-a-cognitive-news-search-app-_-Get-the-Code)
* [Restoring an ordering node](https://cloud.ibm.com/docs/blockchain?topic=blockchain-backup-restorebackup-restore-restore-orderer)









 Building a network with APIs 

[Building a network with APIs](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-v2-apisibp-v2-apis)



* [Swagger](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-v2-apisibp-v2-apis-swagger)
* [Before you begin](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-v2-apisibp-v2-apis-prereq)
* [Authentication](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-v2-apisibp-v2-apis-authentication)



* [Retrieving service credentials](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-v2-apisibp-v2-apis-retrieve-service-credentials)
* [Retrieving an access token](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-v2-apisibp-v2-apis-retrieve-token)



* [Forming your API request](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-v2-apisibp-v2-apis-form-api-request)
* [Limitations](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-v2-apisibp-v2-apis-limitations)
Where is the part of the contract that I need for my task? Quickly extract critical information from contracts. Document Retrieval for Contracts 
 I want the chatbot I'm building to use knowledge that I own. Give a virtual assistant quick access to technical information that is stored in various external data sources and document formats to answer customer questions. Conversational Search 
 I want to uncover insights I didn't know to ask about. Gain insights from pattern analysis or perform root cause analysis. Content Mining 



For more information, see [Creating projects](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-projects).
Watson Discovery docs 

This documentation helps you use Watson Discovery to find critical answers and insights from your business data.

 Recommended content 

[Creating projects Figure out which project type best fits your use case.](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-projects)[Creating collections Connect to your structured and unstructured data where it is.](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-collections)[Adding custom fields with Smart Document Understanding Identify meaningful information based on the structure of your documents.](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-configuring-fields)[Adding domain-specific resources Teach Watson about the information that matters to your use case.](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-domain)

 Video library 

[!
On the Improvement tools panel, expand Teach domain concepts, and then choose the resource that you want to add.

After you create the resource, it becomes a new type of enrichment that you can apply to your data.
3. Specify the collection and field in which to apply the enrichment.

You can apply enrichments to the text and html fields, and to custom fields that were added from uploaded JSON or CSV files or from the Smart Document Understanding (SDU) tool. Only the first 50,000 characters of a custom field from a JSON file are enriched.

For example, if you add a dictionary and choose to apply it to the text field of a collection, the documents in the collection are reprocessed. If the term vehicle is specified as a synonym of the car dictionary entry and occurs in the document text, vehicle is tagged as a mention of the car dictionary entry type. If a customer later searches for car, the passage that contains the vehicle mention is included in the search results.

If the field that you choose comes from a JSON file, after you apply the enrichment, the field data type is converted to an array. The field is converted to an array even if it contains a single value. For example, "field1": "Discovery" becomes "field1": ["Discovery"].



You can choose to apply resource-derived enrichments to your data later. Enrichments that you add to a project are available for use from any collection in the project. Go to the Manage collections page, choose the collection where you want to apply the enrichment, and then open the Enrichments tab. Make sure the status of the enrichment shows that it is Ready, and then apply the enrichment to a field in the collection. Enrichments that you enable are applied to the documents in random order. For more information, see [Managing enrichments](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-manage-enrichments).

From the deployed Content Mining application, you can create a classifier or a custom annotator from a dictionary, regular expression, machine learning, or PEAR file and use it as an enrichment in collections that are stored in other project types.