DB4AI: Database-driven AI-Full-process AI
Tags: opengauss
Published: 2023-04-13 10:29:47

A traditional AI task usually has multiple processes. For example, a data collection process includes data collection, data cleaning, and data storage, an algorithm training process includes data preprocessing, training, and model storage and management, and a model training process includes hyperparameter tuning. The entire lifecycle of such machine learning models can be integrated into the database. The model training, management, and optimization processes are performed closest to the data storage. The out-of-the-box AI lifecycle management function of SQL statements is provided on the database side, which is called full-process AI.

openGauss implements some full-process AI functions, which will be described in detail in this section.

**PLPython Fenced Mode**
PL/Python is added to the fenced mode, which is insecure. During database compilation, to integrate Python into the database, you can add the **–with-python** option to **configure**, or specify the Python path for installing PL/Python and add the **–with-includes='/python-dir=path'** option.

Before starting the database, set the GUC parameter **unix_socket_directory** to specify the file address for communication between unix_socket processes. You need to create a folder in **user-set-dir-path** in advance and grant read, write, and execute permissions on the folder.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/syzvoqro74e8gdq8a5vk.png)

After the configuration is complete, start the database.

After PL/Python is added to the database compilation and the GUC parameter **unix_socket_directory** is set, the **fenced-Master** process is automatically created during database startup. If Python compilation is not performed for the database, you need to manually start the master process in fenced mode. After the GUC parameter is set, run the command to create the master process.

Run the following command to start the **fenced-Master** process:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/88hj8x3qmlhv5yflef2y.png)

After the fenced mode is configured, the UDF calculation is performed in the fenced-worker process for the PL/Python-fenced UDF database.

**Usage Guide**

**·** Create an extension.

**·** When the compiled PL/Python is Python 2:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5x5kmuft7n2tvxnway6m.png)

**·** When the compiled PL/Python is Python 3:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vxe8jkdpvdevn3bpptav.png)

The following uses Python 2 as an example.

**·** Create a PL/Python-fenced UDF database.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/23griiyath5762q2h9bj.png)

**·** View UDF information.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8p3n5hyaoslf30puttfo.png)

**·** Run the UDF.

**·** Create a data table.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/99s30dbs9qlhgl9dduo9.png)

**·** Run the UDF.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tp0ri3592vn5cwylv19t.png)