Jay-Rajput/oci_data huggingface dataset:
Using Notebooks to Connect to Data Flow


You can connect to Data Flow and run an Apache Spark application from a Data Science notebook session. These sessions allow you to run interactive Spark workloads on a long lasting Data Flow cluster through an Apache Livy integration.

Data Flow uses fully managed Jupyter Notebooks to enable data scientists and data engineers to create, visualize, collaborate, and debug data engineering and data science applications. You can write these applications in Python, Scala, and PySpark. You can also connect a Data Science notebook session to Data Flow to run applications. The Data Flow kernels and applications run on Oracle Cloud Infrastructure Data Flow. Data Flow is a fully managed Apache Spark service that performs processing tasks on extremely large datasets, without the need to deploy or manage infrastructure. For more information, see the Data Flow documentation.

Apache Spark is a distributed compute system designed to process data at scale. It supports large-scale SQL, batch, and stream processing, and machine learning tasks. Spark SQL provides database-like support. To query structured data, use Spark SQL. It is an ANSI standard SQL implementation.

Data Flow is a fully managed Apache Spark service that performs processing tasks on extremely large datasets, without infrastructure to deploy or manage. You can use Spark Streaming to perform cloud ETL on your continuously produced streaming data. It enables rapid application delivery because you can focus on application development, not infrastructure management.

Apache Livy is a REST interface to Spark. Submit fault-tolerant Spark jobs from the notebook using synchronous and asynchronous methods to retrieve the output.

SparkMagic allows for interactive communication with Spark using Livy. Using the `%%spark` magic directive within a JupyterLab code cell. The SparkMagic commands are avilable for Spark 3.2.1 and the Data Flow conda environment.

Data Flow Sessions support auto-scaling Data Flow cluster capabilities. For more information, see Autoscaling in the Data Flow documentation. Data Flow Sessions support the use of conda environments as customizable Spark runtime environments.

Figure 1. Data Science Notebook to Data Flow
Limitations

Data Flow Sessions last up to 7 days or 10,080 mins (maxDurationInMinutes).

Data Flow Sessions have a default idle timeout value of 480 mins (8 hours) (idleTimeoutInMinutes). You can configure a different value.
The Data Flow Session is only available through a Data Science Notebook Session.
Only Spark version 3.2.1 is supported.
 Tip

Watch the tutorial video on using Data Science with Data Flow. Also see the Oracle Accelerated Data Science SDK documentation for more information on integrating Data Science and Data Flow.
Example Connector Messages

Email (Raw) and Slack
Email (Formatted)
SMS
Logging Search

Required IAM Policy
Required Permissions for Searching Logs
Basic Search Queries
Advanced Search Queries
Oracle Cloud Migrations Work Requests


The Oracle Cloud Migrations work requests enable you to monitor long-running operations such as the creation of a migration plan. When you create an operation, the Oracle Cloud Migrations runs a work request. A work request is an activity log that helps you to track each step in the operation's progress. Each work request has an OCID that allows you to interact with it programmatically and use it for automation.

Work Request Status

Many Oracle Cloud Migrations requests don't take effect immediately. The request runs an asynchronous workflow for fulfillment. Some operations depend on the completion of a preceding operation before completing. You can monitor each operation’s work request and confirm that the operation succeeds before proceeding to the next operation.

For a detailed list of status values and what they mean, see Work Requests.
Networking scenarios

Basic scenarios

All but the first of these basic routing scenarios send traffic from a subnet in the VCN to the DRG. To accomplish this, you must set up a rule in the subnet's route table. The rule's destination CIDR is the CIDR of the network that is being reached through the DRG, and the rule's target is the DRG. For more information, see VCN Route Tables.

Scenario A: Public Subnet (no DRG required)
Scenario B: Private Subnet with a VPN
Scenario C: Public and Private Subnets with a VPN
Peering

These scenarios all allow traffic to flow from one VCN to another.

Peering VCNs in the same region through a DRG (Upgraded DRG)
Peering VCNs in different regions through a DRG (Upgraded DRG)
Local VCN Peering using Local Peering Gateways (Legacy DRG)
Remote VCN Peering using a Legacy DRG (Legacy DRG)
Advanced scenarios using a single DRG

VCN Ingress Routing: Some of the following advanced scenarios require ingress routing of traffic entering the VCN from a DRG through the VCN attachment. To accomplish this, you must associate a VCN route table (this is a route table created inside your VCN) with the VCN attachment. After a VCN route table is associated with a VCN attachment, there must always be a VCN route table associated with that attachment (that is, you cannot update the field in the corresponding data object to "Null"). Removing the VCN ingress routing functionality from a VCN attachment can only be accomplished by emptying the associated VCN route table or updating the attachment to refer to an empty VCN route table.

Remote on-ramp (Upgraded DRG)
Using a DRG to route traffic through a centralized network virtual appliance (Upgraded DRG)
Private Access to Oracle Services
Transit Routing inside a hub VCN
Advanced scenario with multiple DRGs and multiple VCNs

There's an extra advanced scenario that illustrates the use of multiple DRGs and VCNs. In this case, each VCN has its own dynamic routing gateway (DRG)  and its own FastConnect private virtual circuit . Contrast this with Transit Routing inside a hub VCN, in which there's a single DRG with either Site-to-Site VPN or a single FastConnect private virtual circuit.

Here are some restrictions for using the scenario that has multiple DRGs:

The scenario works only with FastConnect through a third-party provider or through colocation with Oracle. It is not supported for FastConnect through an Oracle partner.
The scenario is supported only for VCNs in the same region and same tenancy. This is because all the virtual circuits use a single cross-connect, a regional resource.

See FastConnect with Multiple DRGs and VCNs.
Resource Settings
 page to edit that option. You can also change the default approval duration from this page.

You can perform the following approval template tasks:

Editing a Resource Setting
Deleting a Resource Setting
Oracle Linux KVM Image

Release Information
Creating and Connecting to an Instance
Allocating Block Storage and VNICs
Creating and Attaching Block Storage
Creating and Attaching VNICs
Working with Virtual Machines
Creating a Virtual Machine
Removing a Virtual Machine
Using the OCI-KVM Utilities
oci-kvm create
oci-kvm destroy
oci-kvm create-pool
oci-kvm list-pool
oci-kvm create-network
oci-kvm delete-network
Finding Resources


Several tools can help you find resources in your tenancy.

Console Search

Use Search to find resources in a tenancy, Console pages in services, and documentation in the OCI documentation.

To use Search, start typing in the search bar in the header of the Console. You are immediately shown matching results in the region you are currently viewing. To see matching resources in another region, switch to that region.

For more information about using Search in the Console, see Search documentation.

Tenancy Explorer

The tenancy explorer provides a cross-region view of all your resources. You can choose to view just the resources that reside in the selected compartment, or you can choose to view all the resources in all the subcompartments as well, to get a full view of the compartment tree.

To navigate to the tenancy explorer, open the navigation menu and click Governance & Administration. Under Tenancy Management, click Tenancy Explorer.

For more information about the tenancy explorer, see Viewing All Resources in a Compartment.

Dashboard Resource Explorer

The resource explorer displays the list of services and count of resources in a selected compartment and region.

To set up a dashboard with the resource explorer, click the Dashboard tab and follow the prompts.

For more information about dashboards, see Console Dashboards.
Modifying a Cluster


Modify clusters involves adding and removing cloud SQL and running and updating bootstrap scripts.

See the following:

Adding Cloud SQL
Removing Cloud SQL from a Cluster
Bootstrap Script Helper Functions
Deleting a Knowledge Base


Delete a knowledge base using the Console, CLI, or API.

Console
CLI
API
Open the navigation menu and click Developer Services. Under App Dependency Management, click Knowledge Bases.
On the Knowledge bases page, from the list of compartments on the left side, select a compartment.
From the list of knowledge bases, click the knowledge base that you want to delete.
On the Knowledge base details page, click Delete.
In the Delete knowledge base dialog box, enter the name of the knowledge base and click Delete.
Typical Use Cases


Review some of the scenarios where you can use Oracle Logging Analytics.

Topics:

Perform Advanced Analytics with Link

Parse Log Records with Multiple Timestamps

Perform Advanced Analytics with Cluster Compare

Project the time series data of EBS Concurrent Request logs using the Link Trend feature. See Visualize Time Series Data Using the Link Trend Feature.

Examples of Semantic Clustering

Examples of Leveraging Log Partitioning Feature

Examples of Tile Layout in Link
Using the SQL Worksheet
 🔗

The SQL Worksheet provides a web-based SQL workspace where you can enter SQL statements directly in the browser using a database connection. From the SQL Worksheet, you can run SQL statements or scripts against the database, and create database objects.

Some of the SQL Worksheet features are not available for MySQL database services. This is indicated with a note in the following topics where applicable.

Topics:

About the SQL Worksheet
Using the Oracle Cloud Infrastructure Console
Service Overview


Document Understanding is a serverless, multitenant service, accessible using the Console, REST APIs, SDK, or CLI.

You can upload documents to detect and classify text and objects in them. You can process individual files or batches of documents using the ProcessorJob API endpoint. The following pretrained models are supported:

Optical Character Recognition (OCR): Document Understanding can detect and recognize text in a document.
Text extraction: Document Understanding provides the word level and line level text, and the bounding box coordinates of where the text is found.
Key-value extraction: Document Understanding extracts a predefined list of key-value pair information from receipts, invoices, passports, and driver IDs.
Table extraction: Document Understanding extracts content in tabular format, maintaining the row and column relationships of cells.
Document classification: Document Understanding classifies documents into different types based on visual appearance, high-level features, and extracted keywords. For example, document types such as invoice, receipt, and resume.
Optical Character Recognition (OCR) PDF: Document Understanding generates a searchable PDF file in Object Storage.
Set Up Administration

Setting Up Data Labeling
Load Balancer Session Persistence

Sticky Cookies
Fallback
Artifact RegistryKnown Issues


Artifact Registry has no known issues.
Managing Dynamic Groups

About Dynamic Groups
Required IAM Policy
Tagging Resources
Working with Dynamic Groups
Updating Dynamic Groups
Limits on Dynamic Groups
Using the Console
Writing Matching Rules to Define Dynamic Groups
Using the API
Using the Oracle Cloud Console

Discover Your Applications
Add Users
Explore Learning Resources
Navigate to Applications and Related Services
Understand Regions
Setting the Console Time Zone for Fusion Applications
Use Support Chat to Get Help
Open a Support Request
Manage Support Requests
View the Infrastructure Console
View the Dashboard
Required Permissions to View the List of Applications
Passing Custom Configuration Parameters to Functions


Find out about passing custom configuration parameters to running functions with OCI Functions.

The code in functions you deploy to OCI Functions will typically require values for different parameters. Some pre-defined parameters are available to your functions as environment variables. But you'll often want your functions to use parameters that you've defined yourself. For example, you might create a function that reads from and writes to a database. The function will require a database connect string, comprising a username, password, and hostname. You'll probably want to define username, password, and hostname as parameters that are passed to the function when it's invoked.

To pass user-defined parameters to a function deployed in OCI Functions, you create key-value pairs known as custom configuration parameters. You can create custom configuration parameters that are:

application-wide, meaning they are passed to every function in an application
function-specific, meaning they are passed to the particular function for which they are defined (function-specific parameters override application-wide parameters with the same name)

To create custom configuration parameters, you can use:

the config: section of a function's func.yaml file, to define function-specific custom configuration parameters
the Console and the Fn Project CLI, to define both application-wide and function-specific custom configuration parameters

OCI Functions combines all the custom configuration parameters (both application-wide and function-specific) in the application into a single, serially encoded configuration object with a maximum allowable size of 4 Kb.

You can specify custom configuration parameters to pass to functions using the Console, the Fn Project CLI, and the API. See Specifying Custom Configuration Parameters to Pass to Functions.
Managing Source Environments


A source environment in Oracle Cloud Migrations represents an on-premises environment (outside of OCI), such as an on-premises datacenter with VMware vCenter deployment.

A source environment contains logically or physically grouped set of external assets. You can specify information about compartment and network access policies for a source environment. These environment information is then applied to all the external assets.

After creating a source environment in the OCI Console, you install agents. Agents are virtual machines that provide network connectivity between your on-premises environment and OCI. They support VM data migration. You then register a remote agent appliance for the source environment, and begin the discovery and migration process.

Before you create a source environment, ensure that you complete the prerequisites for using Oracle Cloud Migrations.

Manage a source environment by performing the following tasks:

Creating an Environment
Renaming an Environment
Deleting an Environment
Visual Builder
 Resources
Web Applications Learning Path
Mobile Applications Learning Path
Videos
Groovy Scripting Reference
Troubleshooting Versioning


This topic provides troubleshooting solutions for issues you might encounter using versioning.



For more information click here https://docs.oracle.com/en-us/iaas/Content/Object/Troubleshooting/usingversioning_topic-Troubleshooting_Versioning.htm#TroubleshootVersioning
Policy-Based Backups

User Defined Backup Policies
Oracle Defined Backup Policies
Working with Backup Policies
Tagging Resources
Creating and Configuring User Defined Backup Policies
Using the Console
Managing Backup Policy Assignments to Volumes
Tracking the Status of Backup Operations with Events
Overview of IAM

Customer Responsibility
IAM Components
The Administrators Group, Policy, and Administrator Roles
Ways to Access Oracle Cloud Infrastructure
Documentation to Use for Cloud Identity
Do You Have Access to Identity Domains?
Which Documentation Do You Need?
Viewing All Resources in a Compartment

Tenancy Explorer Highlights
Work Requests
Resources Supported by the Tenancy Explorer
Required IAM Policy to Work with Resources in the Tenancy Explorer
Navigating to the Tenancy Explorer and Viewing Resources
Filtering Displayed Resources
Opening the Resource Details Page
Moving Resources to a Different Compartment
Deleting Resources
Using the API
About Database Tools


This section provides an overview of the Database Tools service.

Overview
Database Tools Resources
Required IAM Policies
Oracle Database Use Cases
MySQL Database Use Cases
Known Issues for IAM without Identity Domains

Security questions are not available as MFA factors
Permissions granted through policies that specify groups or dynamic groups by name persist through name changes
New permissions in resource-types are not propagated
Deleted compartments continue to count against service limits
OCI Modules for PowerShell

Requirements
Supported PowerShell Versions
Services Supported
Contact Us
Maintenance
 of Read Replicas
MySQL Server Upgrades



For more information click here https://docs.oracle.com/en-us/iaas/Content/postgresql/maintenance.htm
About Data Labeling


Find out about Data Labeling, what it is and how to use it.

Data labeling is the process of identifying properties (labels) of documents, text, and images (records), and annotating (labeling) them with those properties. The topic of a news article, the sentiment of a tweet, the caption of an image, important words spoken in an audio recording, the genre of a video are all examples of a data label.

Many machine learning techniques require labeled data before they can be used to train machines to complete an autonomous task. Data labeling is thus an integral part of an Artificial Intelligence (AI) or Machine Learning (ML) project.

Data Labeling lets you create and browse datasets, view data records (documents, text, and images), and apply labels to build AI/ML models. Datasets can be exported as line-delimited JSON for use in machine learning model development. Datasets are accessible and interoperable across other Data and AI services to support supervised training. For example, Oracle Cloud Infrastructure Language can be used to create specialized models, but only if labeled data is available for training the model. Data Labeling lets you quickly start labeling raw datasets with a minimal number of configuration steps. Thus, it also provides the data labeling experience for Oracle Cloud Infrastructure AI services.

Information on the supported file types and content types for documents, text, and images is given in Supported File Formats.

Datasets are the core resource available in Data Labeling. They consist of data records and their associated labels. Data records represent a document, a single image, or a piece of text. Labels are strings of text, which become annotations when associated with a data record. Annotations have other associated data, for example, with object detection, bounding box coordinates. Data records can exist without an annotation. Datasets can be exported as a JSON manifest for use as an input to machine learning model development.
 Tip

Watch a video introduction to the service..
To use Data Labeling:
Set it up, including creating buckets in Object Storage and setting up your user policies.
Create a dataset.
Generate records in your dataset.
Add labels to your documents, images, or pieces of text.
Export the dataset to Object Storage for use elsewhere.
Testing DNS Using BIND'S dig Tool

Using dig
dig Commands
Moving a Repository Between Compartments


Find out how to move a repository in Container Registry from one compartment to another.

When you create a new repository in Container Registry, you specify the compartment in which to create it. Having created the repository in one compartment, you can subsequently move it to a different compartment. For example, to change the users who are authorized to use the repository, or to change how billing for a repository is charged.

Only users with appropriate permissions can access the repository in the compartment that you move it to.

Your permissions control which repositories you can move, and the compartments that you can move them to (see Policies to Control Repository Access). You can move repositories that you've created (and repositories that the groups to which you belong have been granted access by IAM policies) to any compartment to which you have access. If you belong to the Administrators group, you can move any repository in the tenancy to any compartment.

Console
CLI
API

To move a repository in Container Registry from one compartment to another using the Console:

Open the navigation menu and click Developer Services. Under Containers & Artifacts, click Container Registry.
Select the region that contains the repository.

Select the compartment that contains the repository.

The Repositories and images field lists the repositories in the selected region and compartment to which you have access.

From the list, select the repository that you want to move.
Click Move compartment.
Select the compartment to which you want to move the repository.
Click Submit to move the repository.

The repository is moved to the compartment you selected.

Only users with appropriate permissions can now access the repository in the compartment that you've moved it to.
Monitoring Query Language (MQL) Reference

MQL Syntax
Valid Values for MQL Expressions
Interval
Statistic
Predicate Operators
Arithmetic Operators
Resource Identifiers

Oracle Cloud IDs (OCIDs)
Where to Find Your Tenancy's OCID
Name and Description
Display Name
Oracle Linux Cloud Developer Image

x86_64
Notices
Preinstalled Components
Yum Repositories Enabled by Default
aarch64
Notices
Preinstalled Components
Yum Repositories Enabled by Default
Creating and Connecting to an Instance
Updating the Image
More Information
Service Catalog

Browse and launch service catalog applications
Create a service catalog
Update a service catalog
Delete a service catalog
View all service catalogs in a compartment
Support
Get help and contact Support
Create a service request
Work with Private Applications
Create a private application
Update a private application
Delete a private application
View all private applications in a compartment
Community
Oracle Cloud Infrastructure blog
Cloud infrastructure community forum
Developer Tutorials
 show you how to complete development related tasks with step-by-step instructions.

Applications and Platforms
Free Tier
Functions
Kubernetes
Terraform
Data and AI
Big Data
Data Catalog
Data Flow
Data Integration
Data Science
 Expand All Expandable Areas
Terraform Provider

Availability
Contributions
Notifications
Questions or Feedback
Understanding Object Storage Namespaces

Using the Console
Using the Command Line Interface (CLI)
Using the API
Oracle Enterprise Landing Zone v2 - Workload Expansion Implementation

Prerequisites
Deployment
User
Region
Known Issues for Object Storage


Object Storage has no known issues.
Pulse
 Reports
Learn about reports
Storage reports
Availability reports
Incident management reports
Change management reports
Support
Get help and contact Support
Create a service request
User Provisioning for Federated Users

Overview
Understanding User Types
Who Should Set Up This Integration?
Prerequisite
Enabling User Provisioning
Instructions for Oracle Identity Cloud Service Federations
Instructions for Okta Federations
What to Expect After the Upgrade
Data Import - Disk

Disk-Based Data Import Concepts
Roles and Responsibilities
Task Flow for Disk-Based Data Import
Secure Disk Data Transfer to Oracle Cloud Infrastructure
Ways to Manage Disk Data Transfers
What's Next
Fusion Analytics Warehouse
 Community
Copyright © 2021, Oracle and/or its affiliates.
About Oracle
Contact Us
Legal Notices
Terms of Use & Privacy
Document Conventions
Cookie Preferences
A DevOps Engineer's Guide to OCI

The DevOps Life Cycle
The DevOps Approach
DevOps Services and Tools
Additional Resources