vania-janet/multiturn-rag-retrieval-data huggingface dataset:
The connector does not support cookie-based crawling.

All Discovery data source connectors are read-only. Regardless of the permissions that are granted to the crawl account, Discovery never writes, updates, or deletes any content in the original data source.

The following table illustrates the objects that Discovery can crawl.



Table 1. Data sources crawling support

 Objects that are crawled 

 Websites, website subdirectories 







 Prerequisite step 

If you want to connect to a website that is hosted behind a firewall, set up an IBM® Secure Gateway for IBM Cloud® connection first.

Valuable content is often stored on your company's internal website. Typically, such intranet websites are accessible only from a computer that is connected to your office network or through a VPN connection. You can establish a persistent and more secure connection between the web crawler and this type of internal site by using Secure Gateway.

For more information about how to set up the connection, see [Installing IBM Secure Gateway for on-premises data](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-sourcesgatewaypublic).





 Connecting to the data source 

To configure the web crawl collection, complete the following steps:



1. From the navigation pane, choose Manage collections.
2. Click New collection.
3. Click Web crawl, and then click Next.
4. Name the collection.
5. If the language of the content on the website is not English, select the appropriate language.

For a list of supported languages, see [Language support](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-language-support).
6. Optional: You can change the synchronization schedule.

For more information, see [Crawl schedule options](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-collectionscrawlschedule).
7. Specify the URL of the website that you want to crawl.
For example, if a subdirectory that you expect the crawl to access, such as https://www.example.com/banking/pdfs, isn't being crawled, check whether you can access the subdirectory URL from a web browser directly. If you can't access it, the crawl can't access it.

During subsequent scheduled recrawls, a full recrawl is performed and any changes are reflected in your collection. Documents that were added to your collection from website pages that are later deleted from the external website are not deleted from the collection. However, starting with collections that were created after April 2022, when you remove a starting URL from the web crawl configuration, any associated documents are deleted. Deleted documents include indexed documents that were added to the collection based on the content of the web page at the starting URL and documents that were derived from web pages that the starting URL linked to. You cannot limit the number of indexed documents by changing other settings, such as changing the existing URL to include a path with a more limited scope than before or reducing the maximum number of links to follow to 0. Only by deleting the URL can you remove the indexed documents that are associated with it.

The web crawler can crawl web pages that use JavaScript to render content, but the crawler works best on individual pages, not entire websites. It cannot crawl sites that use dynamic URLs; if you can't see any content when you view the source code of a web page in your browser, then the service cannot crawl it.

If you want to crawl a group of URLs that includes some websites that require authentication and some that don't, consider creating a different collection for each authentication type. The connector does not support cookie-based crawling.

All Discovery data source connectors are read-only. Regardless of the permissions that are granted to the crawl account, Discovery never writes, updates, or deletes any content in the original data source.

The following table illustrates the objects that Discovery can crawl.



Table 1. Data sources crawling support

 Objects that are crawled 

 Websites, website subdirectories 







 Prerequisite step
The common name that is used in the certificate is the host name for the website. The host name and the common name of the certificate must match, or browsers issue a warning. If your site is web1.mydomain.com, then make that your common name. If you also use images.mydomain.com, you need either a wildcard certificate (which is not available from IBM Cloud®) or you need to set up multiple certificates. If you choose the wrong common name, there are steps that can be taken to fix a certificate order or to revoke and reissue with the correct common name.





 Learn about the socket rule 

SSL certificates are limited to one certificate per socket. A socket is an IP address and port combination, such as 1.2.3.4:443 (1.2.3.4:444 would be a different socket). For applications like SMTP/POP3 or FTP, the socket rule does not matter. However, it matters for HTTP because of its involvement with virtual hosting.

Virtual hosting is the method by which you can host 20, 30, 100 websites on one IP address. Virtual hosting works because modern browsers pass a field that is called the host reader as part of their request. This field looks like “Host: web1.mydomain.com” and tells the web server which site you are trying to access. In the case of HTTPS (HTTP over SSL), the web server must select the SSL certificate to send to the client before seeing the host header, which is why a socket can have only one certificate.

You can assign each SSL-enabled website to its own socket by varying the IP address or varying the port. Keep in mind if you change the port from 443/tcp, users must include the port number in their URL like https://web1.mydomain.com:444.





 Generate the CSR 

You can generate the certificate signing request (CSR) by using software on the web server. For UNIX systems, use the OpenSSL package. For Windows, there is a wizard that is accessed from the Directory Security tab of the website properties in IIS Manager. If you are using a control panel, refer to specific information for that control panel.
Any integration that is not listed ignores the response. If the channels array is absent, all integrations handle the response.







 Response types 

The following response types are available. (For detailed information about how to specify each response type, see [Response types reference](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-response-types-reference).)

Not all channel integrations support all response types. For information about which integrations support which response types, see [Channel integration support for response types](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-assistant-responses-jsonassistant-responses-json-integration-support).

audio
: Plays an audio clip that is specified by a URL.

channel_transfer
: Requests that the conversation is transferred to a different integration. (Currently, only the web chat integration can be the target of a channel transfer.)

connect_to_agent
: Requests that the conversation is transferred to a contact center with live agents for help.

date
: Requests that the channel collects a date value from the customer (for example, by displaying an interactive calendar).

dtmf
: Sends commands to the phone integration to control input or output by using dual-tone multi-frequency (DTMF) signals. (DTMF is a protocol that is used to transmit the tones that are generated when a user presses keys on a push-button phone.)

end_session
: Sends a command to the channel to end the session. This response type instructs the phone integration to hang up the call.

iframe
: Embeds content from an external website as an HTML iframe element.

image
: Displays an image that is specified by a URL.

option
: Presents a set of options (such as buttons or a drop-down list) that users can choose from. The selected value is then sent to the assistant as user input.

pause
: Pauses before sending the next message to the channel, and optionally sends a "user is typing" event (for channels that support it).

speech_to_text
Add the full URL to the hosted image file into the Image source field.

The image must be in .jpg, .gif, or .png format. The image file must be stored in a location that is publicly addressable by an https: URL.

For example: https://www.example.com/assets/common/logo.png.

If you want to display an image title and description above the embedded image in the response, then add them in the fields provided.

To access an image that is stored in IBM Cloud® Object Storage, enable public access to the individual image storage object, and then reference it by specifying the image source with syntax like this: https://s3.eu.cloud-object-storage.appdomain.cloud/your-bucket-name/image-name.png.

Some integration channels ignore titles or descriptions.







 Adding a Video response type 

Include videos in your response to share how-to demonstrations, promotional clips, and so forth. In the web chat, a video response renders as an embedded video player.

To add a Video response type, complete the following steps:



1. Choose Video.
2. Add the full URL to the hosted video into the Video source field:



* To link directly to a video file, specify the URL to a file in any standard format such as MPEG or AVI. In the web chat, the linked video will render as an embedded video player.

HLS (.m3u8) and DASH (MPD) streaming videos are not supported.
* To link to a video hosted on a supported video hosting service, specify the URL to the video. In the web chat, the linked video will render using the embeddable player for the hosting service.

Specify the URL you would use to view the video in your browser (for example, https://www.youtube.com/watch?v=52bpMKVigGU). You do not need to convert the URL to an embeddable form; the web chat will do this automatically.

You can embed videos hosted on the following services:



* [YouTube](https://youtube.com)
[Histogram](https://cloud.ibm.com/docs-content/v1/content/f84e2238288b7f0355715a4c355609ce7ac48fb0/solution-tutorials/images/solution55-openshift-microservices/board-img-11.png)

Histogram
9. Name the dashboard by hitting the pencil Edit Board button next to the New Board N name".



* Enter patientui as the name of the dashboard
* Enter a category, for example, yourname then click Add this as a new board category
* Click Save





A new category appears on the left navigation panel.





 Create a screen to monitor your app 

You can create a screen to monitor your app graphically through metrics (counters), operational KPIs (gauges), tables, and time-shifted graphs (graphs that you can use to analyze patterns and trends for comparison analysis).

Complete the following steps to create a dashboard to monitor logs from the lab's sample app:



1. In the Log Analysis web UI, click the screens icon.
2. Select NEW SCREEN.
3. Add a count of the patient health frontend log lines for the last two weeks:



* Click Add Widget at the top and select Count
* Click the newly created widget to reveal the configuration fields for the widget on the right
* In the Data section



* Select the field app, and set the value to patient-health-frontend
* Keep Operation at the default Counts
* In the Duration drop down select Last 2 Weeks



* In the Appearance section



* In the Label text box type patient-health-frontend





The widget should look similar to the following one:

Zoom

![New widget](https://cloud.ibm.com/docs-content/v1/content/f84e2238288b7f0355715a4c355609ce7ac48fb0/solution-tutorials/images/solution55-openshift-microservices/screen-img-7.png)

New widget
* Virtual Server for VPC



Because emissions are reported after the close of each billing cycle, data for newly added services and current quarter results take about two months to populate.





 How do I use this data with Envizi or other third party ESG reporting tools? 

IBM Envizi is a robust data management foundation that is designed to create a single, trusted data source for all your ESG reporting and opportunity identification. Use ESG reporting to meet compliance and reporting requirements. For more information, see [IBM Envizi ESG Suite](https://www.ibm.com/products/envizi).

Envizi integration currently requires creation of a custom connector. Reach out to your IBM Cloud representative for more information.





 How do I submit feedback on Carbon Calculator? 

Talk to your CSM or create case on support center. When you open a support casae, choose billing and usage topics and subtopics, but specify carbon calculator in case description.





 How do I view the carbon intensity of my data center? 

In the carbon calculator, on the locations widget, you can hover over the location of their data center to see carbon intensity.
* How do I configure archiving for my instance?

To configure archiving see [Archiving events to IBM Cloud Object Storage](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-archiving).
* How do I access the logging web UI? I cannot see the logging instance in the Observability page.

To lauch the logging web UI, complete the following steps:



1. Check that you have IAM permissions to view the logging instances through the Logging dashboard. You must have viewer platform role permissions for the IBM Log Analysis service. [Learn more about managing IAM policies and access groups](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-work_iam).
2. [Launch the logging web UI](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-launch).



* How do I reset an ingestion key?

An ingestion key can be reset or new ones created in the logging web UI or using the API. You can have a maximum of 10 ingestion keys active at the same time in a logging instance.

Learn more about [resetting using the UI](https://cloud.ibm.com/docs/log-analysis?topic=log-analysis-ingestion_keyreset) or [resetting using the API](https://cloud.ibm.com/apidocs/log-analysislist-key).
* How can I get the ID of a logging instance?

To get the ID of a logging instance, run the following command:

ic resource service-instance <INSTANCE_NAME>

To get the name of the logging instance, run the following command ibmcloud resource service-instances --all-resource-groups.
* Can I import archived data into the UI?

Archived data cannot be imported to be searched or used in the IBM Log Analysis UI.
How do I deploy an OVF file by using the vSphere Web Client? 

  What’s happening 

I want to use the VMware vSphere® Web Client to deploy an Open Virtualization Format (OVF).

  How to fix it 

Use the following steps to use the vSphere Web Client to deploy an OVF file:



1.  Before you attempt to deploy an OVF file, add the following host information to the /etc/hosts file:



*  Host information for Platform Services Controller (PSC)
*  Host information for VMware ESXi® server
*  Host information for vCenter



2.  Check your /etc/hosts file to ensure that this information is successfully added. An example of a hosts file might look like the following list:

10.131.9.201  psc-myinstance.myinstance.mydomain.com
10.131.7.26   host0.myinstance.mydomain.com
10.131.7.39   host1.myinstance.mydomain.com
10.131.7.6    host2.myinstance.mydomain.com
10.131.7.48   host3.myinstance.mydomain.com
10.131.9.196  vcenter-1.myinstance.mydomain.com
3.  Ensure that you have access to all the necessary VPNs (Virtual Private Networks) in the IBM Cloud® infrastructure customer portal. In this example, it would be 10.131.7.xx and 10.131.9.xxx.
4.  Sign in the VPN for your data center.
5.  Click vCenter to access the vSphere Web Client.
6.  Right-click on a host and deploy an .ovf file.
See [Deleting a resource group](https://cloud.ibm.com/docs/account?topic=account-delete_rgs) for more information.





 How do I add users to a resource group? 

Resource groups are a method of organizing resources and are not directly associated with the management of users. For information on creating a resource group, see [Adding resources to a resource group](https://cloud.ibm.com/docs/account?topic=account-rgsadd_to_rgs). After your resource group is created, an account administrator can grant access to a specific user. Or, an account administrator can create an access group to provide access to a resource group. For information, see [Creating an Access Group](https://cloud.ibm.com/docs/account?topic=account-groupscreate_ag) in the console. After an access group is created, complete the following steps to associate the access group with a resource group:



1. In the IBM Cloud console, click Manage > Access (IAM) and click Access groups.
2. Select an access group.
3. Click Access > Assign access.
4. Select a service, or group of services.
5. Click Next.
6. Scope the access to all resources, or select specific resources and select the resource groups attribute. Then, click Next.
7. Select the platform, service and resource group access for the resource group.
8. Click Review.
9. Click Add to add your policy configuration to your policy summary.
10. Click Assign.



You can't use access groups with infrastructure service resources or permissions.





 Can I move service instances between resource groups? 

You can't move service instances between resource groups. If you assign a service instance incorrectly, you must delete and recreate the instance to assign it to another resource group.





 How do I delete a service from my account? 

You can delete a service instance by using the following steps:



1. From your dashboard in the IBM Cloud console, click View resources within the Resources summary widget.
* How do I delete apps?

To delete an app that you created, complete these steps:



1. From the [IBM Cloud console](https://%7BDomainName%7D), click the Menu icon ![Menu icon](https://cloud.ibm.com/icons/icon_hamburger.svg), and select Resource list.
2. Click the Actions icon ![Actions icon](https://cloud.ibm.com/icons/action-menu-icon.svg) for the app that you want to delete, and click Delete.



* What are toolchains?

A toolchain is a set of tool integrations that support continuous development, deployment, monitoring, operations, and more. For more information about toolchains, see [Creating toolchains](https://cloud.ibm.com/docs/ContinuousDelivery?topic=ContinuousDelivery-toolchains_getting_started).
* How do I extract a toolchain template from an existing toolchain?

You can run the [toolchain-to-template script](https://github.com/open-toolchain/toolchain-to-templatesetup) to extract a toolchain template from an existing toolchain. This script uses a toolchain URL and generates an open toolchain (OTC) template in the current folder. When the script runs, it creates a clone of your original toolchain.



Getting support



* How do I check for planned maintenance?

You can check for upcoming maintenance from your dashboard in the IBM Cloud® console at least one time every 24 hours. Use one of the following options:



* Check out the Planned maintenance widget on the [Dashboard](https://%7BDomainName%7D), and click View all.
* Go directly to the [Status - Planned maintenance page](https://cloud.ibm.com/status?selected=maintenance).



* What steps can I take before I open a support case?

Before you open a support case, explore the following resources:
Branding your app 

With IBM Cloud® App ID, you can customize the entire sign-up experience of your application by using your own branded screens. You can replace the provided sign-in widget with your own, add extra fields, validate password strength, and verify email addresses against a blocklist!



 Understanding screen reuse 

When you reuse your existing UIs, you can create a cohesive sign-in flow for your app. By using the same imagery, colors, and branding, your users are more likely to recognize your brand, even when not directly interacting with your app.

Want to use a [language](https://cloud.ibm.com/docs/appid?topic=appid-cd-typescd-languages) other than English? You can choose another language by using the [language management APIs](https://us-south.appid.cloud.ibm.com/swagger-ui//Config/updateLocalization) to display your own translated content.





 What information do I need to obtain with my sign in or sign up screen? 

When you make the request to submit your sign-in form, it must contain both username and password parameters in your request body. Your sign-in or sign-up page must ask that information from your users.



 Can I use some of my own and some default screens? 

Yes! You can create a hybrid flow that uses some of your screens and some default screens. However, you can use only one option per flow. As an example, you can use your own sign-in screen and also use the default sign-up screen. But, if you choose to use the default sign-up screen, then you must continue to use the default through entire sign-up flow, including sign-up verification.





 How are the flows technically different? 

The service uses OAuth 2.0 grant flows to map the authorization process. When you configure social identity providers such as Facebook, the [Authorization Grant flow](https://oauthlib.readthedocs.io/en/latest/oauth2/grants/authcode.html) is used to call the Login Widget.
Service API endpoint change
: As explained in [December 2019](https://cloud.ibm.com/docs/assistant?topic=assistant-release-notesassistant-dec122019), as part of work done to fully support IAM authentication, the endpoint you use to access your Watson Assistant service programmatically is changing. The old endpoint URLs are deprecated and will be retired on 26 May 2021. Update your API calls to use the new URLs.

The pattern for the endpoint URL changes from gateway-{location}.watsonplatform.net/assistant/api/ to api.{location}.assistant.watson.cloud.ibm.com/. The domain, location, and offering identifier are different in the new endpoint. For more information, see [Updating endpoint URLs from watsonplatform.net](https://cloud.ibm.com/docs/watson?topic=watson-endpoint-change).



* If your service instance API credentials show the old endpoint, create a new credential and start using it today. After you update your custom applications to use the new credential, you can delete the old one.
* For a web chat integration, you might need to take action depending on when and how you created your integration.



* If you tied your deployment to a specific web chat version by using the clientVersion parameter and specified a version earlier than version 3.3.0, update the parameter value to use version 3.3.0 or later. Web chat integrations that use the latest or 3.3.0 and later versions will not be impacted by the endpoint deprecation.
* If you created your web chat integration before May 2020, check the code snippet that you embedded in your web page to see if it refers to watsonplatform.net. If so, you must edit the code snippet to use the new URL syntax. For example, change the following URL:

<script src="https://assistant-web.watsonplatform.net/loadWatsonAssistantChat.js"></script>