movielens search with Vertex AI Agent Builder Media search

## 概要

Vertex AI Agent Builder Media searchを使ってSearchエンジンを作ってみる。

内容は、[Get started with media search](https://cloud.google.com/generative-ai-app-builder/docs/try-media-search)の中身をベースにしています。

## まとめ

- VertexAI Agent Builderで簡単にSearchエンジンを作成できる
- Data SourceはGCSやBQに対応している
    - 必要な形への変換はBQのViewを使うことで簡単に利用可能
- Recommendationも同様のデータソースを利用できるので、検索とレコメンデーションを高速に実装できる -> Recommendationは[こちら](https://qiita.com/nakamasato/items/26a60eca99b5323d440c)
- Applicationはlibraryがあるので他のGoogle Service同様に簡単にクライアントで呼び出しができる
- BigQueryからのDataのLoadが定期インポートがサポートされている :thumbsup: ([Import from BigQuery - Periodic ingestion](https://cloud.google.com/generative-ai-app-builder/docs/create-data-store-es#bigquery))


## Prepare BQ Dataset

bqのdataset `movielens`作成

```
bq mk $PROJECT_ID:movielens
```

csvからmovies テーブルをload

```
bq load --skip_leading_rows=1 movielens.movies \
  gs://cloud-samples-data/gen-app-builder/media-recommendations/movies.csv \
  movieId:integer,title,genres
```

csvからratingテーブルをload

```
bq load --skip_leading_rows=1 movielens.ratings \
  gs://cloud-samples-data/gen-app-builder/media-recommendations/ratings.csv \
  userId:integer,movieId:integer,rating:float,time:timestamp
```

view `movies_view`を作成

```
bq mk --project_id=$PROJECT_ID \
 --use_legacy_sql=false \
 --view "
  WITH t AS (
    SELECT
      CAST(movieId AS string) AS id,
      SUBSTR(title, 0, 128) AS title,
      SPLIT(genres, \"|\") AS categories
      FROM \`$PROJECT_ID.movielens.movies\`)
    SELECT
      id, \"default_schema\" as schemaId, null as parentDocumentId,
      TO_JSON_STRING(STRUCT(title as title, categories as categories,
      CONCAT(\"http://mytestdomain.movie/content/\", id) as uri,
      \"2023-01-01T00:00:00Z\" as available_time,
      \"2033-01-01T00:00:00Z\" as expire_time,
      \"movie\" as media_type)) as jsonData
    FROM t;" \
$PROJECT_ID:movielens.movies_view
```

![Screenshot 2024-08-30 at 7.59.15.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/7b1aa49b-83ec-a74c-cd72-7d86ecbb21b7.png)


view `user_events_for_search` (`user_events` is already used in https://qiita.com/nakamasato/items/012ea7159d3e3fc8e30e)

```
bq mk --project_id=$PROJECT_ID \
 --use_legacy_sql=false \
 --view "
 WITH t AS (
  SELECT
    MIN(UNIX_SECONDS(time)) AS old_start,
    MAX(UNIX_SECONDS(time)) AS old_end,
    UNIX_SECONDS(TIMESTAMP_SUB(
    CURRENT_TIMESTAMP(), INTERVAL 90 DAY)) AS new_start,
    UNIX_SECONDS(CURRENT_TIMESTAMP()) AS new_end
  FROM \`$PROJECT_ID.movielens.ratings\`)
  SELECT
    CAST(userId AS STRING) AS userPseudoId,
    \"view-item\" AS eventType,
    FORMAT_TIMESTAMP(\"%Y-%m-%dT%X%Ez\",
    TIMESTAMP_SECONDS(CAST(
      (t.new_start + (UNIX_SECONDS(time) - t.old_start) *
      (t.new_end - t.new_start) / (t.old_end - t.old_start))
    AS int64))) AS eventTime,
    [STRUCT(movieId AS id, null AS name)] AS documents,
  FROM \`$PROJECT_ID.movielens.ratings\`, t
  WHERE rating >= 4;" \
$PROJECT_ID:movielens.user_events_for_search
```

document nameは常にnullが入り、document idにはmovie idが入っている

![Screenshot 2024-08-30 at 8.01.28.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/053b3e90-50bf-459c-1a01-c03518c9cf03.png)


## Search appの作成

https://console.cloud.google.com/gen-app-builder/engines このページからSearchを選択 (RecommendationをやりたいばあいにはRecommendationを使う)

![Screenshot 2024-08-30 at 7.27.21.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/90e7a220-8d89-f915-3449-c2b04c149c81.png)

Mediaを選択し、appに名前をつける

![Screenshot 2024-08-30 at 7.28.39.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/98076376-6ae9-3096-90be-43c169e34c55.png)

Data storeの作成

![Screenshot 2024-08-30 at 7.29.46.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/79c13259-36f1-c078-e74d-219df1b5c7d4.png)

作成したdata sourceを選択してappの作成を完了
![Screenshot 2024-08-30 at 7.30.29.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/1ebfc514-b5b3-bae7-ade4-7f7d6ec1279e.png)

## データのインポート

### Documentのインポート

BigQueryまたはCloud Storageが選択できるので、今回はbigqueryを選択

![Screenshot 2024-08-30 at 7.31.41.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/e1a99b17-13d4-ecc5-55f1-07eba2c60a68.png)

Datasetを選択

![Screenshot 2024-08-30 at 7.32.10.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/f3f15ef1-93f8-0d3c-f031-22283de7d1bc.png)

先ほど作成したmovielensの`movies_view` を選択してimportする

![Screenshot 2024-08-30 at 7.34.30.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/f9d72ae4-bef2-f596-a1de-48e00ebac53e.png)

importには15分くらいかかるので完了を待つ

![Screenshot 2024-08-30 at 7.36.08.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/15abe72d-d6dc-9dcb-777e-b3fb4370c2ef.png)

`86,537` documents importされる

![Screenshot 2024-08-30 at 7.49.49.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/e9094c6a-369b-4087-194e-5327a22833af.png)

### User Eventのインポート

次にEventタブから `Import Events`をクリックする

![Screenshot 2024-08-30 at 7.50.16.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/4ec05142-c039-a7d5-b06b-5444deffff5f.png)

同様にBigQueryを選択して、`user_events_for_search` (上でインポートしたテーブル)を指定して、インポート

<!-- ![Screenshot 2024-08-30 at 7.33.01.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/219bef00-7fe9-403c-8a95-2c4bc667ff57.png) -->

![Screenshot 2024-08-30 at 7.51.52.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/69a26e49-b974-e3ad-63a8-f6b1038e4c64.png)

完了

![Screenshot 2024-08-30 at 8.09.44.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/1a2b2c6b-79d5-642d-132f-0141d5b35aaa.png)

## Search Configuration

`The Lord of the Rings` を検索してみる

![Screenshot 2024-08-30 at 8.10.49.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/bd3a4f3a-c7b6-f2dc-07b2-99494793240d.png)

検索のコンディションを選択する

![Screenshot 2024-08-30 at 8.13.20.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/96ed5bd6-4a33-a668-db35-8585dd39c859.png)

決定後[Save and Publish]をクリック

## Search Widgetの設定

↓のような画面でJWT or OAuth based またはpublic accessを選択できる

![Screenshot 2024-08-30 at 9.44.21.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/0282b5d5-2ffb-0004-08fc-13a524e0aeb4.png)

Domainの設定も可能。

## Search APIを呼ぶ

![Screenshot 2024-08-30 at 9.48.29.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/7b43470e-2308-fa1e-2ee7-395928ff2837.png)

`Run in Cloud Shell` を実行すると実際に検索結果を取得することができる

![Screenshot 2024-08-30 at 9.50.52.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/7059/cd3f4361-e8b1-ad3b-3376-7d7dceaa2ff3.png)

使われたQueryはこれ:

```
curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
"https://discoveryengine.googleapis.com/v1alpha/projects/<project_number>/locations/global/collections/default_collection/engines/quickstart-media-search/servingConfigs/default_search:search" \
-d '{"query":"load of the ring","pageSize":10,"queryExpansionSpec":{"condition":"AUTO"},"spellCorrectionSpec":{"mode":"AUTO"}}'
```

## Search APIをApplicationに組み込む

https://cloud.google.com/generative-ai-app-builder/docs/preview-search-results

こちらを参考にADCと一緒にClientを実装するとできる

例:

```go
package main

import (
	"context"
	"fmt"
	"os"

	discoveryengine "cloud.google.com/go/discoveryengine/apiv1beta"
	discoveryenginepb "cloud.google.com/go/discoveryengine/apiv1beta/discoveryenginepb"
	"google.golang.org/api/iterator"
	"google.golang.org/api/option"
)

// search searches for a query in a search engine given the Google Cloud Project ID,
// Location, and Search Engine ID.
//
// This example uses the default search engine.
func search(projectID, location, dataStoreID, query string) error {

	ctx := context.Background()

	// Create a client
	endpoint := "discoveryengine.googleapis.com:443" // Default to global endpoint
	if location != "global" {
		endpoint = fmt.Sprintf("%s-%s", location, endpoint)
	}
	client, err := discoveryengine.NewSearchClient(ctx, option.WithEndpoint(endpoint))
	if err != nil {
		return fmt.Errorf("creating Vertex AI Search client: %w", err)
	}
	defer client.Close()

	// Full resource name of search engine serving config
	servingConfig := fmt.Sprintf("projects/%s/locations/%s/collections/default_collection/dataStores/%s/servingConfigs/default_serving_config",
		projectID, location, dataStoreID)

	searchRequest := &discoveryenginepb.SearchRequest{
		ServingConfig: servingConfig,
		Query:         query,
	}

	it := client.Search(ctx, searchRequest)
	for {
		resp, err := it.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			return err
		}
		fmt.Printf("%+v\n", resp)
	}

	return nil
}

func main() {
	query := "Terminator"

	if err := search(os.Getenv("GCP_PROJECT_ID"), os.Getenv("LOCATION"), os.Getenv("DATA_STORE_ID"), query); err != nil {
		fmt.Println(err)
	}
}
```

```
export GCP_PROJECT_ID=xxx
export LOCATION=global
export DATA_STORE_ID=quickstart-media-data-store
```

```
go run main.go    
id:"175137" document:{struct_data:{fields:{key:"available_time" value:{string_value:"2023-01-01T00:00:00Z"}} fields:{key:"categories" value:{list_value:{values:{string_value:"Documentary"}}}} fields:{key:"expire_time" value:{string_value:"2033-01-01T00:00:00Z"}} fields:{key:"media_type" value:{string_value:"movie"}} fields:{key:"title" value:{string_value:"The Making of 'The Terminator': A Retrospective (1992)"}} fields:{key:"uri" value:{string_value:"http://mytestdomain.movie/content/175137"}}} name:"projects/267676219654/locations/global/collections/default_collection/dataStores/quickstart-media-data-store/branches/0/documents/175137" id:"175137"}
id:"207830" document:{struct_data:{fields:{key:"available_time" value:{string_value:"2023-01-01T00:00:00Z"}} fields:{key:"categories" value:{list_value:{values:{string_value:"Action"} values:{string_value:"Sci-Fi"}}}} fields:{key:"expire_time" value:{string_value:"2033-01-01T00:00:00Z"}} fields:{key:"media_type" value:{string_value:"movie"}} fields:{key:"title" value:{string_value:"Terminator: Dark Fate (2019)"}} fields:{key:"uri" value:{string_value:"http://mytestdomain.movie/content/207830"}}} name:"projects/267676219654/locations/global/collections/default_collection/dataStores/quickstart-media-data-store/branches/0/documents/207830" id:"207830"}
id:"6537" document:{struct_data:{fields:{key:"available_time" value:{string_value:"2023-01-01T00:00:00Z"}} fields:{key:"categories" value:{list_value:{values:{string_value:"Action"} values:{string_value:"Adventure"} values:{string_value:"Sci-Fi"}}}} fields:{key:"expire_time" value:{string_value:"2033-01-01T00:00:00Z"}} fields:{key:"media_type" value:{string_value:"movie"}} fields:{key:"title" value:{string_value:"Terminator 3: Rise of the Machines (2003)"}} fields:{key:"uri" value:{string_value:"http://mytestdomain.movie/content/6537"}}} name:"projects/267676219654/locations/global/collections/default_collection/dataStores/quickstart-media-data-store/branches/0/documents/6537" id:"6537"}
id:"120799" document:{struct_data:{fields:{key:"available_time" value:{string_value:"2023-01-01T00:00:00Z"}} fields:{key:"categories" value:{list_value:{values:{string_value:"Action"} values:{string_value:"Adventure"} values:{string_value:"Sci-Fi"} values:{string_value:"Thriller"}}}} fields:{key:"expire_time" value:{string_value:"2033-01-01T00:00:00Z"}} fields:{key:"media_type" value:{string_value:"movie"}} fields:{key:"title" value:{string_value:"Terminator Genisys (2015)"}} fields:{key:"uri" value:{string_value:"http://mytestdomain.movie/content/120799"}}} name:"projects/267676219654/locations/global/collections/default_collection/dataStores/quickstart-media-data-store/branches/0/documents/120799" id:"120799"}
id:"212887" document:{struct_data:{fields:{key:"available_time" value:{string_value:"2023-01-01T00:00:00Z"}} fields:{key:"categories" value:{list_value:{values:{string_value:"(no genres listed)"}}}} fields:{key:"expire_time" value:{string_value:"2033-01-01T00:00:00Z"}} fields:{key:"media_type" value:{string_value:"movie"}} fields:{key:"title" value:{string_value:"Other Voices: Creating 'The Terminator' (2001)"}} fields:{key:"uri" value:{string_value:"http://mytestdomain.movie/content/212887"}}} name:"projects/267676219654/locations/global/collections/default_collection/dataStores/quickstart-media-data-store/branches/0/documents/212887" id:"212887"}
id:"589" document:{struct_data:{fields:{key:"available_time" value:{string_value:"2023-01-01T00:00:00Z"}} fields:{key:"categories" value:{list_value:{values:{string_value:"Action"} values:{string_value:"Sci-Fi"}}}} fields:{key:"expire_time" value:{string_value:"2033-01-01T00:00:00Z"}} fields:{key:"media_type" value:{string_value:"movie"}} fields:{key:"title" value:{string_value:"Terminator 2: Judgment Day (1991)"}} fields:{key:"uri" value:{string_value:"http://mytestdomain.movie/content/589"}}} name:"projects/267676219654/locations/global/collections/default_collection/dataStores/quickstart-media-data-store/branches/0/documents/589" id:"589"}
id:"177415" document:{struct_data:{fields:{key:"available_time" value:{string_value:"2023-01-01T00:00:00Z"}} fields:{key:"categories" value:{list_value:{values:{string_value:"Documentary"}}}} fields:{key:"expire_time" value:{string_value:"2033-01-01T00:00:00Z"}} fields:{key:"media_type" value:{string_value:"movie"}} fields:{key:"title" value:{string_value:"The Making of 'Terminator 2: Judgment Day' (1991)"}} fields:{key:"uri" value:{string_value:"http://mytestdomain.movie/content/177415"}}} name:"projects/267676219654/locations/global/collections/default_collection/dataStores/quickstart-media-data-store/branches/0/documents/177415" id:"177415"}
id:"68791" document:{struct_data:{fields:{key:"available_time" value:{string_value:"2023-01-01T00:00:00Z"}} fields:{key:"categories" value:{list_value:{values:{string_value:"Action"} values:{string_value:"Adventure"} values:{string_value:"Sci-Fi"} values:{string_value:"Thriller"}}}} fields:{key:"expire_time" value:{string_value:"2033-01-01T00:00:00Z"}} fields:{key:"media_type" value:{string_value:"movie"}} fields:{key:"title" value:{string_value:"Terminator Salvation (2009)"}} fields:{key:"uri" value:{string_value:"http://mytestdomain.movie/content/68791"}}} name:"projects/267676219654/locations/global/collections/default_collection/dataStores/quickstart-media-data-store/branches/0/documents/68791" id:"68791"}
id:"136200" document:{struct_data:{fields:{key:"available_time" value:{string_value:"2023-01-01T00:00:00Z"}} fields:{key:"categories" value:{list_value:{values:{string_value:"Action"} values:{string_value:"Horror"} values:{string_value:"Sci-Fi"}}}} fields:{key:"expire_time" value:{string_value:"2033-01-01T00:00:00Z"}} fields:{key:"media_type" value:{string_value:"movie"}} fields:{key:"title" value:{string_value:"The Terminators (2009)"}} fields:{key:"uri" value:{string_value:"http://mytestdomain.movie/content/136200"}}} name:"projects/267676219654/locations/global/collections/default_collection/dataStores/quickstart-media-data-store/branches/0/documents/136200" id:"136200"}
id:"228409" document:{struct_data:{fields:{key:"available_time" value:{string_value:"2023-01-01T00:00:00Z"}} fields:{key:"categories" value:{list_value:{values:{string_value:"Action"}}}} fields:{key:"expire_time" value:{string_value:"2033-01-01T00:00:00Z"}} fields:{key:"media_type" value:{string_value:"movie"}} fields:{key:"title" value:{string_value:"Ninja Terminator (1985)"}} fields:{key:"uri" value:{string_value:"http://mytestdomain.movie/content/228409"}}} name:"projects/267676219654/locations/global/collections/default_collection/dataStores/quickstart-media-data-store/branches/0/documents/228409" id:"228409"}
id:"102425" document:{struct_data:{fields:{key:"available_time" value:{string_value:"2023-01-01T00:00:00Z"}} fields:{key:"categories" value:{list_value:{values:{string_value:"Action"} values:{string_value:"Adventure"} values:{string_value:"Horror"} values:{string_value:"Sci-Fi"} values:{string_value:"Thriller"}}}} fields:{key:"expire_time" value:{string_value:"2033-01-01T00:00:00Z"}} fields:{key:"media_type" value:{string_value:"movie"}} fields:{key:"title" value:{string_value:"Lady Terminator (Pembalasan ratu pantai selatan) (1989)"}} fields:{key:"uri" value:{string_value:"http://mytestdomain.movie/content/102425"}}} name:"projects/267676219654/locations/global/collections/default_collection/dataStores/quickstart-media-data-store/branches/0/documents/102425" id:"102425"}
id:"139909" document:{struct_data:{fields:{key:"available_time" value:{string_value:"2023-01-01T00:00:00Z"}} fields:{key:"categories" value:{list_value:{values:{string_value:"Action"}}}} fields:{key:"expire_time" value:{string_value:"2033-01-01T00:00:00Z"}} fields:{key:"media_type" value:{string_value:"movie"}} fields:{key:"title" value:{string_value:"Russian Terminator (1989)"}} fields:{key:"uri" value:{string_value:"http://mytestdomain.movie/content/139909"}}} name:"projects/267676219654/locations/global/collections/default_collection/dataStores/quickstart-media-data-store/branches/0/documents/139909" id:"139909"}
id:"214344" document:{struct_data:{fields:{key:"available_time" value:{string_value:"2023-01-01T00:00:00Z"}} fields:{key:"categories" value:{list_value:{values:{string_value:"Action"} values:{string_value:"Drama"}}}} fields:{key:"expire_time" value:{string_value:"2033-01-01T00:00:00Z"}} fields:{key:"media_type" value:{string_value:"movie"}} fields:{key:"title" value:{string_value:"Angel Terminators 2 (1993)"}} fields:{key:"uri" value:{string_value:"http://mytestdomain.movie/content/214344"}}} name:"projects/267676219654/locations/global/collections/default_collection/dataStores/quickstart-media-data-store/branches/0/documents/214344" id:"214344"}
```

## DataのImport

[Import from BigQuery](https://cloud.google.com/generative-ai-app-builder/docs/create-data-store-es#bigquery)、[Import from Cloud Storage](https://cloud.google.com/generative-ai-app-builder/docs/create-data-store-es#cloud-storage)では `Periodic ingestion` (Public Preview) がサポートされているので、定期的にデータを更新することも可能 (1,3,5日に1回のどれか)

> Periodic ingestion: You import data from one or more BigQuery tables, and you set a sync frequency that determines how often the data stores are updated with the most recent data from the BigQuery dataset.
> Data updates automatically every 1, 3, or 5 days. Data cannot be manually refreshed.

詳細は、[Create a search data store](https://cloud.google.com/generative-ai-app-builder/docs/create-data-store-es#cloud-sql-procedure)

## Ref

- [Get started with media search](https://cloud.google.com/generative-ai-app-builder/docs/try-media-search)
- [Get search results](https://cloud.google.com/generative-ai-app-builder/docs/preview-search-results)
- [Create a search data store](https://cloud.google.com/generative-ai-app-builder/docs/create-data-store-es#cloud-sql-procedure)
- [movielens recommendation with VertexAI Search for Retail](https://qiita.com/nakamasato/items/012ea7159d3e3fc8e30e)
- [Get started with media recommendations](https://cloud.google.com/generative-ai-app-builder/docs/try-media-recommendations)