Repository: Automattic/jetpack
Pull request #43390: AI Assistant: Track model used in toolbar extension and AI excerpt features
Author: login: vianasw
Labels: name: Enhancement
description: Changes to an existing feature — removing, adding, or changing parts of it
name: [Plugin] Jetpack
description: Issues about the Jetpack plugin. https://wordpress.org/plugins/jetpack/
name: RNA
name: [Block] AI Assistant
name: [JS Package] AI Client
name: [Extension] AI Content Lens
name: I don't care about code coverage for this PR
description: Use this label to ignore the check for insufficient code coveage.
Created: 2025-05-07T09:52:18Z | Merged: 2025-05-12T16:21:44Z

<!--- Provide a general summary of your changes in the Title above -->
<!-- Would you like this feature to be tested by Beta testers?
Please add testing instructions to projects/plugins/jetpack/to-test.md in a new commit as part of your PR. -->
<!-- a12s: If you have an expected version that you're aiming for the PR to add, please use the Milestone field to communicate it. If you leave it blank, that indicates there isn't a preference. -->
We need to track when we're using our default AI model and when we're using Chrome's AI built-in model. We also need to update the code to adapt to Chrome AI API changes.

Fixes JETPACK-326

## Proposed changes:
<!--- Explain what functional changes your PR includes -->
*  Added two new events to track toolbar extension generate and undo actions (similar to events that we already have but these will have the model used)
* Added a model property in the AI excerpt tracking events
* Modified the code to adapt to Chrome AI API changes in the languageDetector and Translator API.

### Other information:

- [ ] Have you written new tests for your changes, if applicable?
- [ ] Have you checked the E2E test CI results, and verified that your changes do not break them?
- [ ] Have you tested your changes on WordPress.com, if applicable (if so, you'll see a generated comment below with a script to run)?

## Jetpack product discussion
<!-- If you're an Automattician, include a shortlink to the p2 discussion with Jetpack Product here. -->
<!-- Make sure any changes to existing products have been discussed and agreed upon -->

## Does this pull request change what data or activity we track or use?
<!--- If so, please add the "[Status] Needs Privacy Updates" label and explain what changes there are. -->
<!--- Check existing Jetpack support documents for a preview of the information we need. -->

## Testing instructions:
<!-- If you were reviewing this PR, how would you like the instructions to be presented? -->
<!-- Please include detailed testing steps, explaining how to test your change. -->
<!-- Bear in mind that context you working on is not obvious for everyone.  -->
<!-- Adding "simple" configuration steps will help reviewers to get to your PR as quickly as possible. -->
<!-- "Before / After" screenshots can also be very helpful when the change is visual. -->

Prerequisites:
* You'll need to test it using Chrome version 133 or higher.
* First you'll need a token to use the Chrome AI API. Check the comments on the issue JETPACK-326 for the tokens you need to use in the following snippet.
* Check out this branch, add the following snippet to enable beta features, the feature flag, and add your token in place of [ your trial token goes here ]:
```php
define( 'JETPACK_BLOCKS_VARIATION', 'beta' );
add_filter( 'ai_chrome_ai_enabled', '__return_true' );

$token_transient_name = 'jetpack-ai-chrome-ai-summarization-token';

set_transient(
    $token_transient_name,
    "[ your translation trial token goes here ]",
    3600
);


$token_transient_name = 'jetpack-ai-chrome-ai-translation-token';
set_transient(
    $token_transient_name,
    "[ your translation trial token goes here ]",
    3600
);

$token_transient_name = 'jetpack-ai-chrome-ai-language-detection-token';
set_transient(
    $token_transient_name,
    "[ your translation trial token goes here ]",
    3600
);
```
* Once this is set up, you should be able to create an excerpt using Chrome's AI API.
* Open the javascript console and enable debug mode.
* Create a new post and add some text.
* Go to the Excerpt section in the post sidebar and click on generate.
* The first time you click on it, the browser needs to download the model, which can take several seconds (close to a minute for me). There are no UI progress indicators for this, so you'll just have to be patient the first time. The following requests are almost instantaneous.
* Tone, word count, and language settings should work.
* The summarization API only returns summaries in English, check that if the content is in a different language, we fallback to using our default AI model.
* For the excerpt feature check that the following events have the `model` property and is `gemini-nano` or `default` for the fallback case:
  * jetpack_ai_assistant_block_generate
  * jetpack_ai_assistant_block_accept
  * jetpack_ai_assistant_block_discard
* Follow the instructions on https://github.com/Automattic/jetpack/pull/41724 to test the translation feature, but ignore the token setup part.
* For the translation feature check for these events: 
  * jetpack_ai_assistant_toolbar_extension_generate
  * jetpack_ai_assistant_toolbar_extension_undo


Diff:
diff --git a/projects/js-packages/ai-client/changelog/ai-model-used b/projects/js-packages/ai-client/changelog/ai-model-used
new file mode 100644
index 000000000000..59c99dadbd98
--- /dev/null
+++ b/projects/js-packages/ai-client/changelog/ai-model-used
@@ -0,0 +1,4 @@
+Significance: minor
+Type: changed
+
+AI Assistant: propagate the AI model used in the AI requests
diff --git a/projects/js-packages/ai-client/src/chrome-ai/factory.ts b/projects/js-packages/ai-client/src/chrome-ai/factory.ts
index 798ebc023a4b..0738fd17001d 100644
--- a/projects/js-packages/ai-client/src/chrome-ai/factory.ts
+++ b/projects/js-packages/ai-client/src/chrome-ai/factory.ts
@@ -75,13 +75,41 @@ export default async function ChromeAIFactory( promptArg: PromptProp ) {
 		}
 	}
 
+	// Early return if the prompt type is not supported.
+	if (
+		! promptType.startsWith( 'ai-assistant-change-language' ) &&
+		! promptType.startsWith( 'ai-content-lens' )
+	) {
+		return false;
+	}
+
+	// If the languageDetector is not available, we can't use the translation or summary features—it's safer to fall back
+	// to the default AI model than to risk an unexpected error.
+	if (
+		! ( 'LanguageDetector' in self ) ||
+		! self.LanguageDetector.create ||
+		! self.LanguageDetector.availability
+	) {
+		return false;
+	}
+
+	const languageDetectorAvailability = await self.LanguageDetector.availability();
+	if ( languageDetectorAvailability === 'unavailable' ) {
+		return false;
+	}
+
+	const detector = await self.LanguageDetector.create();
+	if ( languageDetectorAvailability !== 'available' ) {
+		await detector.ready;
+	}
+
 	if ( promptType.startsWith( 'ai-assistant-change-language' ) ) {
 		const [ language ] = context.language.split( ' ' );
 
 		if (
-			! ( 'translation' in self ) ||
-			! self.translation.createTranslator ||
-			! self.translation.canTranslate
+			! ( 'Translator' in self ) ||
+			! self.Translator.create ||
+			! self.Translator.availability
 		) {
 			return false;
 		}
@@ -91,26 +119,22 @@ export default async function ChromeAIFactory( promptArg: PromptProp ) {
 			targetLanguage: language,
 		};
 
-		// see if we can detect the source language
-		if ( 'ai' in self && self.ai.languageDetector ) {
-			const detector = await self.ai.languageDetector.create();
-			const confidences = await detector.detect( context.content );
-
-			for ( const confidence of confidences ) {
-				// 75% confidence is just a value that was picked. Generally
-				// 80% of higher is pretty safe, but the source language is
-				// required for the translator to work at all, which is also
-				// why en is the default language.
-				if ( confidence.confidence > 0.75 ) {
-					languageOpts.sourceLanguage = confidence.detectedLanguage;
-					break;
-				}
+		const confidences = await detector.detect( context.content );
+
+		for ( const confidence of confidences ) {
+			// 75% confidence is just a value that was picked. Generally
+			// 80% of higher is pretty safe, but the source language is
+			// required for the translator to work at all, which is also
+			// why en is the default language.
+			if ( confidence.confidence > 0.75 ) {
+				languageOpts.sourceLanguage = confidence.detectedLanguage;
+				break;
 			}
 		}
 
-		const canTranslate = await self.translation.canTranslate( languageOpts );
+		const translationAvailability = await self.Translator.availability( languageOpts );
 
-		if ( canTranslate === 'no' ) {
+		if ( translationAvailability === 'unavailable' ) {
 			return false;
 		}
 
@@ -123,14 +147,33 @@ export default async function ChromeAIFactory( promptArg: PromptProp ) {
 		return chromeAI;
 	}
 
-	// TODO: consider also using ChromeAI for ai-assistant-summarize
 	if ( promptType.startsWith( 'ai-content-lens' ) ) {
+		if ( ! ( 'Summarizer' in self ) ) {
+			return false;
+		}
+
+		if ( context.language && context.language !== 'en (English)' ) {
+			return false;
+		}
+
+		const confidences = await detector.detect( context.content );
+
+		// if it doesn't look like the content is in English, we can't use the summary feature
+		for ( const confidence of confidences ) {
+			// 75% confidence is just a value that was picked. Generally
+			// 80% of higher is pretty safe, but the source language is
+			// required for the translator to work at all, which is also
+			// why en is the default language.
+			if ( confidence.confidence > 0.75 && confidence.detectedLanguage !== 'en' ) {
+				return false;
+			}
+		}
+
 		const summaryOpts = {
 			tone: tone,
 			wordCount: wordCount,
 		};
 
-		// TODO: detect if the content is in English and fallback if it's not
 		return new ChromeAISuggestionsEventSource( {
 			content: context.content,
 			promptType: PROMPT_TYPE_SUMMARIZE,
diff --git a/projects/js-packages/ai-client/src/chrome-ai/suggestions.ts b/projects/js-packages/ai-client/src/chrome-ai/suggestions.ts
index ce4c6f0f812f..215ae3b8b696 100644
--- a/projects/js-packages/ai-client/src/chrome-ai/suggestions.ts
+++ b/projects/js-packages/ai-client/src/chrome-ai/suggestions.ts
@@ -110,11 +110,11 @@ export default class ChromeAISuggestionsEventSource extends EventTarget {
 
 	// use the Chrome AI translator
 	async translate( text: string, target: string, source: string = '' ) {
-		if ( ! ( 'translation' in self ) ) {
+		if ( ! ( 'Translator' in self ) ) {
 			return;
 		}
 
-		const translator = await self.translation.createTranslator( {
+		const translator = await self.Translator.create( {
 			sourceLanguage: source,
 			targetLanguage: target,
 		} );
@@ -125,6 +125,7 @@ export default class ChromeAISuggestionsEventSource extends EventTarget {
 
 		try {
 			const translation = await translator.translate( renderHTMLFromMarkdown( { content: text } ) );
+
 			this.processEvent( {
 				id: '',
 				event: 'translation',
@@ -160,20 +161,22 @@ export default class ChromeAISuggestionsEventSource extends EventTarget {
 
 	// use the Chrome AI summarizer
 	async summarize( text: string, tone?: string, wordCount?: number ) {
-		if ( ! ( 'ai' in self ) || ! ( 'summarizer' in self.ai ) ) {
+		if ( ! ( 'Summarizer' in self ) ) {
 			return;
 		}
-		const available = ( await self.ai.summarizer.capabilities() ).available;
+		// eslint-disable-next-line no-console
+		console.log( 'Summarizer is available' );
+		const availability = await self.Summarizer.availability();
 
-		if ( available === 'no' ) {
+		if ( availability === 'unavailable' ) {
 			return;
 		}
 
-		const options = this.getSummarizerOptions( tone, wordCount );
+		const summarizerOptions = this.getSummarizerOptions( tone, wordCount );
 
-		const summarizer = await self.ai.summarizer.create( options );
+		const summarizer = await self.Summarizer.create( summarizerOptions );
 
-		if ( available === 'after-download' ) {
+		if ( availability !== 'available' ) {
 			await summarizer.ready;
 		}
 
diff --git a/projects/js-packages/ai-client/src/hooks/use-ai-suggestions/index.ts b/projects/js-packages/ai-client/src/hooks/use-ai-suggestions/index.ts
index 8e26024f0181..f36edadcf884 100644
--- a/projects/js-packages/ai-client/src/hooks/use-ai-suggestions/index.ts
+++ b/projects/js-packages/ai-client/src/hooks/use-ai-suggestions/index.ts
@@ -16,13 +16,20 @@ import {
 	ERROR_SERVICE_UNAVAILABLE,
 	ERROR_UNCLEAR_PROMPT,
 	ERROR_RESPONSE,
+	AI_MODEL_DEFAULT,
+	AI_MODEL_GEMINI_NANO,
 } from '../../types.ts';
 /**
  * Types & constants
  */
 import type { AskQuestionOptionsArgProps } from '../../ask-question/index.ts';
 import type SuggestionsEventSource from '../../suggestions-event-source/index.ts';
-import type { PromptProp, SuggestionErrorCode, RequestingStateProp } from '../../types.ts';
+import type {
+	PromptProp,
+	SuggestionErrorCode,
+	RequestingStateProp,
+	AiModelTypeProp,
+} from '../../types.ts';
 
 export type RequestingErrorProps = {
 	/*
@@ -70,7 +77,7 @@ type useAiSuggestionsOptions = {
 	/*
 	 * onDone callback.
 	 */
-	onDone?: ( content: string, skipRequestCount?: boolean ) => void;
+	onDone?: ( content: string, skipRequestCount?: boolean, modelUsed?: AiModelTypeProp ) => void;
 
 	/*
 	 * onStop callback.
@@ -85,7 +92,7 @@ type useAiSuggestionsOptions = {
 	/*
 	 * Error callback common for all errors.
 	 */
-	onAllErrors?: ( error: RequestingErrorProps ) => void;
+	onAllErrors?: ( error: RequestingErrorProps, skipRequestCount?: boolean ) => void;
 };
 
 type useAiSuggestionsProps = {
@@ -94,6 +101,11 @@ type useAiSuggestionsProps = {
 	 */
 	suggestion: string;
 
+	/*
+	 * The model.
+	 */
+	model: AiModelTypeProp;
+
 	/*
 	 * The error.
 	 */
@@ -221,6 +233,12 @@ export default function useAiSuggestions( {
 	const [ requestingState, setRequestingState ] =
 		useState< RequestingStateProp >( initialRequestingState );
 	const [ suggestion, setSuggestion ] = useState< string >( '' );
+	const [ model, setModel ] = useState< AiModelTypeProp >( AI_MODEL_DEFAULT );
+	const modelRef = useRef< AiModelTypeProp >( AI_MODEL_DEFAULT );
+	const setModelAndRef = ( value: AiModelTypeProp ) => {
+		setModel( value );
+		modelRef.current = value;
+	};
 	const [ error, setError ] = useState< RequestingErrorProps >();
 
 	// Store the event source in a ref, so we can handle it if needed.
@@ -258,7 +276,7 @@ export default function useAiSuggestions( {
 
 			const fullSuggestion = removeLlamaArtifact( event?.detail?.message ?? event?.detail );
 
-			onDone?.( fullSuggestion, event?.detail?.source === 'chromeAI' );
+			onDone?.( fullSuggestion, event?.detail?.source === 'chromeAI', modelRef.current );
 			setRequestingState( 'done' );
 		},
 		[ onDone ]
@@ -266,7 +284,7 @@ export default function useAiSuggestions( {
 
 	const handleAnyError = useCallback(
 		( event: CustomEvent ) => {
-			onAllErrors?.( event?.detail );
+			onAllErrors?.( event?.detail, event?.detail?.source === 'chromeAI' );
 		},
 		[ onAllErrors ]
 	);
@@ -319,8 +337,10 @@ export default function useAiSuggestions( {
 			const chromeAI = await ChromeAIFactory( promptArg );
 
 			if ( chromeAI !== false ) {
+				setModelAndRef( AI_MODEL_GEMINI_NANO );
 				eventSourceRef.current = chromeAI;
 			} else {
+				setModelAndRef( AI_MODEL_DEFAULT );
 				eventSourceRef.current = await askQuestion( promptArg, options );
 			}
 
@@ -436,6 +456,7 @@ export default function useAiSuggestions( {
 	return {
 		// Data
 		suggestion,
+		model,
 		error,
 		requestingState,
 
diff --git a/projects/js-packages/ai-client/src/types.ts b/projects/js-packages/ai-client/src/types.ts
index b2ab7e47fd27..12107a724be9 100644
--- a/projects/js-packages/ai-client/src/types.ts
+++ b/projects/js-packages/ai-client/src/types.ts
@@ -93,8 +93,14 @@ export type RequestingStateProp = ( typeof REQUESTING_STATES )[ number ];
  */
 export const AI_MODEL_GPT_3_5_Turbo_16K = 'gpt-3.5-turbo-16k' as const;
 export const AI_MODEL_GPT_4 = 'gpt-4' as const;
+export const AI_MODEL_DEFAULT = 'default' as const;
+export const AI_MODEL_GEMINI_NANO = 'gemini-nano' as const;
 
-export type AiModelTypeProp = typeof AI_MODEL_GPT_3_5_Turbo_16K | typeof AI_MODEL_GPT_4;
+export type AiModelTypeProp =
+	| typeof AI_MODEL_GPT_3_5_Turbo_16K
+	| typeof AI_MODEL_GPT_4
+	| typeof AI_MODEL_GEMINI_NANO
+	| typeof AI_MODEL_DEFAULT;
 
 /*
  * Media recording types
@@ -135,43 +141,43 @@ export interface BlockEditorStore {
 
 declare global {
 	interface Window {
-		translation?: {
-			canTranslate: ( options: {
+		LanguageDetector?: {
+			create: () => Promise< {
+				detect: ( text: string ) => Promise<
+					{
+						detectedLanguage: string;
+						confidence: number;
+					}[]
+				>;
+				ready: Promise< void >;
+			} >;
+			availability: () => Promise<
+				'unavailable' | 'available' | 'downloadable' | 'downloading' | string
+			>;
+		};
+		Translator?: {
+			create: ( options: {
 				sourceLanguage: string;
 				targetLanguage: string;
-			} ) => Promise< 'no' | 'yes' | string >;
-			createTranslator: ( options: {
+			} ) => Promise< { translate: ( text: string ) => Promise< string > } >;
+			availability: ( options: {
 				sourceLanguage: string;
 				targetLanguage: string;
+			} ) => Promise< 'unavailable' | 'available' | 'downloadable' | 'downloading' | string >;
+		};
+		Summarizer?: {
+			availability: () => Promise<
+				'unavailable' | 'available' | 'downloadable' | 'downloading' | string
+			>;
+			create: ( options: {
+				sharedContext?: string;
+				type?: string;
+				format?: string;
+				length?: string;
 			} ) => Promise< {
-				translate: ( text: string ) => Promise< string >;
+				ready: Promise< void >;
+				summarize: ( text: string, summarizeOptions?: { context?: string } ) => Promise< string >;
 			} >;
 		};
-		ai?: {
-			languageDetector: {
-				create: () => Promise< {
-					detect: ( text: string ) => Promise<
-						{
-							detectedLanguage: string;
-							confidence: number;
-						}[]
-					>;
-				} >;
-			};
-			summarizer?: {
-				capabilities: () => Promise< {
-					available: 'no' | 'yes' | 'after-download';
-				} >;
-				create: ( options: {
-					sharedContext?: string;
-					type?: string;
-					format?: string;
-					length?: string;
-				} ) => Promise< {
-					ready: Promise< void >;
-					summarize: ( text: string, summarizeOptions?: { context?: string } ) => Promise< string >;
-				} >;
-			};
-		};
 	}
 }
diff --git a/projects/plugins/jetpack/changelog/add-chrome-ai-event-properties b/projects/plugins/jetpack/changelog/add-chrome-ai-event-properties
new file mode 100644
index 000000000000..717b172e5c5c
--- /dev/null
+++ b/projects/plugins/jetpack/changelog/add-chrome-ai-event-properties
@@ -0,0 +1,4 @@
+Significance: minor
+Type: other
+
+AI Assistant: Added tracking events to know when we're using built-in AI models in some requests and adapted to Chrome AI API changes
diff --git a/projects/plugins/jetpack/extensions/blocks/ai-assistant/ai-assistant.php b/projects/plugins/jetpack/extensions/blocks/ai-assistant/ai-assistant.php
index ed0388376da7..0eb60bc768ba 100644
--- a/projects/plugins/jetpack/extensions/blocks/ai-assistant/ai-assistant.php
+++ b/projects/plugins/jetpack/extensions/blocks/ai-assistant/ai-assistant.php
@@ -61,7 +61,7 @@ function load_assets( $attr, $content ) {
  * This ultimately sets an Origin-Trial header with the token.
  */
 function add_chrome_ai_token_headers() {
-	$token_transient_names = array( 'jetpack-ai-chrome-ai-translation-token', 'jetpack-ai-chrome-ai-summarization-token' );
+	$token_transient_names = array( 'jetpack-ai-chrome-ai-translation-token', 'jetpack-ai-chrome-ai-summarization-token', 'jetpack-ai-chrome-ai-language-detection-token' );
 
 	foreach ( $token_transient_names as $token_transient_name ) {
 		$cached_token = get_transient( $token_transient_name );
diff --git a/projects/plugins/jetpack/extensions/blocks/ai-assistant/extensions/text-blocks/with-ai-text-extension.tsx b/projects/plugins/jetpack/extensions/blocks/ai-assistant/extensions/text-blocks/with-ai-text-extension.tsx
index 1528c4644948..7a76cb412b2f 100644
--- a/projects/plugins/jetpack/extensions/blocks/ai-assistant/extensions/text-blocks/with-ai-text-extension.tsx
+++ b/projects/plugins/jetpack/extensions/blocks/ai-assistant/extensions/text-blocks/with-ai-text-extension.tsx
@@ -8,6 +8,7 @@ import {
 	useAiSuggestions,
 	useAiFeature,
 } from '@automattic/jetpack-ai-client';
+import { useAnalytics } from '@automattic/jetpack-shared-extension-utils';
 import { BlockControls, useBlockProps } from '@wordpress/block-editor';
 import { createHigherOrderComponent } from '@wordpress/compose';
 import { useDispatch, useSelect } from '@wordpress/data';
@@ -39,6 +40,7 @@ import type {
 	PromptMessagesProp,
 	PromptItemProps,
 	RequestingStateProp,
+	AiModelTypeProp,
 } from '@automattic/jetpack-ai-client';
 
 const debug = debugFactory( 'jetpack-ai-assistant:extensions:with-ai-extension' );
@@ -89,6 +91,8 @@ const blockEditWithAiComponents = createHigherOrderComponent( BlockEdit => {
 		const [ action, setAction ] = useState< string >( '' );
 		// The last human-readable action performed by the user, to be used by the thumbs up/down buttons.
 		const [ lastAction, setLastAction ] = useState< string | null >( null );
+		// The last prompt type used by the user, to be used for tracking purposes.
+		const lastPromptType = useRef< string | null >( null );
 		// The last request made by the user, to be used when the user clicks the "Try Again" button.
 		const lastRequest = useRef< RequestOptions | null >( null );
 		// Ref to the requesting state to use it in the hideOnBlockFocus effect.
@@ -123,6 +127,8 @@ const blockEditWithAiComponents = createHigherOrderComponent( BlockEdit => {
 			inputRef.current?.focus();
 		}, [] );
 
+		const { tracks } = useAnalytics();
+
 		// Data and functions with block-specific implementations.
 		const {
 			onSuggestion: onBlockSuggestion,
@@ -214,7 +220,7 @@ const blockEditWithAiComponents = createHigherOrderComponent( BlockEdit => {
 
 		// Called after the last suggestion chunk is received.
 		const onDone = useCallback(
-			( suggestion: string, skipRequestCount?: boolean ) => {
+			( suggestion: string, skipRequestCount?: boolean, modelUsed?: AiModelTypeProp ) => {
 				disableAutoScroll();
 				onBlockDone( suggestion );
 				if ( ! skipRequestCount ) {
@@ -222,6 +228,11 @@ const blockEditWithAiComponents = createHigherOrderComponent( BlockEdit => {
 				}
 				setAction( '' );
 
+				tracks.recordEvent( 'jetpack_ai_assistant_toolbar_extension_generate', {
+					prompt_type: lastPromptType.current,
+					model: modelUsed,
+				} );
+
 				if ( lastRequest.current?.message ) {
 					const assistantMessage = {
 						role: 'assistant' as const,
@@ -268,6 +279,8 @@ const blockEditWithAiComponents = createHigherOrderComponent( BlockEdit => {
 				adjustPosition,
 				focusInput,
 				adjustBlockPadding,
+				tracks,
+				lastPromptType,
 			]
 		);
 
@@ -293,6 +306,7 @@ const blockEditWithAiComponents = createHigherOrderComponent( BlockEdit => {
 			request,
 			stopSuggestion,
 			requestingState,
+			model,
 			error,
 			reset: resetSuggestions,
 		} = useAiSuggestions( {
@@ -320,6 +334,8 @@ const blockEditWithAiComponents = createHigherOrderComponent( BlockEdit => {
 					return;
 				}
 
+				lastPromptType.current = promptType;
+
 				const humanText = mapActionToHumanText( promptType, options );
 
 				if ( humanText ) {
@@ -387,10 +403,13 @@ const blockEditWithAiComponents = createHigherOrderComponent( BlockEdit => {
 
 		// Called when the user clicks the "Undo" button after a successful request.
 		const handleUndo = useCallback( async () => {
+			tracks.recordEvent( 'jetpack_ai_assistant_toolbar_extension_undo', {
+				prompt_type: lastPromptType.current,
+				model: model,
+			} );
 			await undo();
-
 			handleClose();
-		}, [ undo, handleClose ] );
+		}, [ undo, handleClose, tracks, model, lastPromptType ] );
 
 		// Closes the AI Control if the block is deselected.
 		useEffect( () => {
diff --git a/projects/plugins/jetpack/extensions/plugins/ai-content-lens/components/ai-excerpt-control/index.tsx b/projects/plugins/jetpack/extensions/plugins/ai-content-lens/components/ai-excerpt-control/index.tsx
index a5839bfcaf1d..24080594b62d 100644
--- a/projects/plugins/jetpack/extensions/plugins/ai-content-lens/components/ai-excerpt-control/index.tsx
+++ b/projects/plugins/jetpack/extensions/plugins/ai-content-lens/components/ai-excerpt-control/index.tsx
@@ -15,7 +15,6 @@ import { ToneDropdownMenu } from '../../../../blocks/ai-assistant/components/ton
  */
 import type { LanguageProp } from '../../../../blocks/ai-assistant/components/i18n-dropdown-control';
 import type { ToneProp } from '../../../../blocks/ai-assistant/components/tone-dropdown-control';
-import type { AiModelTypeProp } from '@automattic/jetpack-ai-client';
 
 export type AiExcerptControlProps = {
 	/*
@@ -48,9 +47,6 @@ export type AiExcerptControlProps = {
 
 	tone?: ToneProp;
 	onToneChange?: ( tone: ToneProp ) => void;
-
-	model?: AiModelTypeProp;
-	onModelChange?: ( model: AiModelTypeProp ) => void;
 };
 
 import './style.scss';
diff --git a/projects/plugins/jetpack/extensions/plugins/ai-content-lens/extend/ai-post-excerpt/index.tsx b/projects/plugins/jetpack/extensions/plugins/ai-content-lens/extend/ai-post-excerpt/index.tsx
index 0c126de2c1f9..4896de608cac 100644
--- a/projects/plugins/jetpack/extensions/plugins/ai-content-lens/extend/ai-post-excerpt/index.tsx
+++ b/projects/plugins/jetpack/extensions/plugins/ai-content-lens/extend/ai-post-excerpt/index.tsx
@@ -31,7 +31,7 @@ import { AiExcerptControl } from '../../components/ai-excerpt-control';
  */
 import type { LanguageProp } from '../../../../blocks/ai-assistant/components/i18n-dropdown-control';
 import type { ToneProp } from '../../../../blocks/ai-assistant/components/tone-dropdown-control';
-import type { AiModelTypeProp, PromptProp } from '@automattic/jetpack-ai-client';
+import type { PromptProp } from '@automattic/jetpack-ai-client';
 import type { ReactElement } from 'react';
 
 import './style.scss';
@@ -45,7 +45,6 @@ type ContentLensMessageContextProps = {
 	content?: string;
 	language?: LanguageProp;
 	tone?: ToneProp;
-	model?: AiModelTypeProp;
 };
 
 function AiPostExcerpt() {
@@ -71,36 +70,53 @@ function AiPostExcerpt() {
 	const [ reenable, setReenable ] = useState( false );
 	const [ language, setLanguage ] = useState< LanguageProp >();
 	const [ tone, setTone ] = useState< ToneProp >();
-	const [ model, setModel ] = useState< AiModelTypeProp >( null );
-
-	const { request, stopSuggestion, suggestion, requestingState, error, reset } = useAiSuggestions( {
-		onDone: useCallback( () => {
-			/*
-			 * Increase the AI Suggestion counter.
-			 * @todo: move this at store level.
-			 */
-			increaseAiAssistantRequestsCount();
-		}, [ increaseAiAssistantRequestsCount ] ),
-		onError: useCallback(
-			suggestionError => {
-				/*
-				 * Incrses AI Suggestion counter
-				 * only for valid errors.
-				 * @todo: move this at store level.
-				 */
-				if (
-					suggestionError.code === 'error_network' ||
-					suggestionError.code === 'error_quota_exceeded'
-				) {
-					return;
-				}
-
-				// Increase the AI Suggestion counter.
-				increaseAiAssistantRequestsCount();
-			},
-			[ increaseAiAssistantRequestsCount ]
-		),
-	} );
+
+	const { request, stopSuggestion, suggestion, requestingState, error, reset, model } =
+		useAiSuggestions( {
+			onDone: useCallback(
+				( _content, skipRequestCount, modelUsed ) => {
+					/*
+					 * Increase the AI Suggestion counter.
+					 * @todo: move this at store level.
+					 */
+					if ( ! skipRequestCount ) {
+						increaseAiAssistantRequestsCount();
+					}
+					tracks.recordEvent( 'jetpack_ai_assistant_block_generate', {
+						feature: 'jetpack-ai-content-lens',
+						model: modelUsed,
+					} );
+				},
+				[ increaseAiAssistantRequestsCount, tracks ]
+			),
+			onError: useCallback(
+				suggestionError => {
+					/*
+					 * Incrses AI Suggestion counter
+					 * only for valid errors.
+					 * @todo: move this at store level.
+					 */
+					if (
+						suggestionError.code === 'error_network' ||
+						suggestionError.code === 'error_quota_exceeded'
+					) {
+						return;
+					}
+
+					// Increase the AI Suggestion counter.
+					increaseAiAssistantRequestsCount();
+				},
+				[ increaseAiAssistantRequestsCount ]
+			),
+			onAllErrors: useCallback(
+				( _suggestionError, skipRequestCount ) => {
+					if ( ! skipRequestCount ) {
+						increaseAiAssistantRequestsCount();
+					}
+				},
+				[ increaseAiAssistantRequestsCount ]
+			),
+		} );
 
 	// Cancel and reset AI suggestion when the component is unmounted
 	useEffect( () => {
@@ -179,17 +195,14 @@ ${ postContent }
 		 * when performing a new AI suggestion request.
 		 */
 		dequeueAiAssistantFeatureAsyncRequest();
-
 		request( prompt, { feature: 'jetpack-ai-content-lens', model } );
-		tracks.recordEvent( 'jetpack_ai_assistant_block_generate', {
-			feature: 'jetpack-ai-content-lens',
-		} );
 	}
 
 	function setExcerpt() {
 		editPost( { excerpt: suggestion } );
 		tracks.recordEvent( 'jetpack_ai_assistant_block_accept', {
 			feature: 'jetpack-ai-content-lens',
+			model: model,
 		} );
 		reset();
 	}
@@ -198,6 +211,7 @@ ${ postContent }
 		editPost( { excerpt: excerpt } );
 		tracks.recordEvent( 'jetpack_ai_assistant_block_discard', {
 			feature: 'jetpack-ai-content-lens',
+			model: model,
 		} );
 		reset();
 	}
@@ -254,11 +268,6 @@ ${ postContent }
 						setTone( newTone );
 						setReenable( true );
 					} }
-					model={ model }
-					onModelChange={ newModel => {
-						setModel( newModel );
-						setReenable( true );
-					} }
 					disabled={ isBusy || requireUpgrade }
 				/>
 


Reviews:
user: login: dhasilva
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-07T22:07:59Z
user: login: dhasilva
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-07T22:13:32Z
user: login: vianasw
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-08T10:25:51Z
user: login: CGastrell
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-09T18:07:04Z
user: login: CGastrell
body: Looking good! Worked as described on all scenarios.
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T16:17:34Z

Review comments:
diff_hunk: @@ -123,14 +124,39 @@ export default async function ChromeAIFactory( promptArg: PromptProp ) {
 		return chromeAI;
 	}
 
-	// TODO: consider also using ChromeAI for ai-assistant-summarize
 	if ( promptType.startsWith( 'ai-content-lens' ) ) {
+		if ( ! ( 'ai' in self ) || ! ( 'summarizer' in self.ai ) ) {
+			return false;
+		}
+
+		if ( context.language ) {
path: projects/js-packages/ai-client/src/chrome-ai/factory.ts
user: login: dhasilva
body: This outer condition is not necessary, the inner condition is enough.
created_at: 2025-05-07T22:07:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -165,15 +166,17 @@ export default class ChromeAISuggestionsEventSource extends EventTarget {
 		}
 		const available = ( await self.ai.summarizer.capabilities() ).available;
 
-		if ( available === 'no' ) {
+		if ( available === 'unavailable' ) {
 			return;
 		}
 
-		const options = this.getSummarizerOptions( tone, wordCount );
-
-		const summarizer = await self.ai.summarizer.create( options );
+		const summarizerOptions = this.getSummarizerOptions( tone, wordCount );
 
-		if ( available === 'after-download' ) {
+		let summarizer;
+		if ( available === 'available' ) {
+			summarizer = await self.ai.summarizer.create( summarizerOptions );
+		} else {
+			summarizer = await self.ai.summarizer.create( summarizerOptions );
 			await summarizer.ready;
 		}
path: projects/js-packages/ai-client/src/chrome-ai/suggestions.ts
user: login: dhasilva
body: This can be simplified

```suggestion
		summarizer = await self.ai.summarizer.create( summarizerOptions );

		if ( available !== 'available' ) {
			await summarizer.ready;
		} 
```
created_at: 2025-05-07T22:13:32Z
author_association: CONTRIBUTOR
diff_hunk: @@ -123,14 +124,39 @@ export default async function ChromeAIFactory( promptArg: PromptProp ) {
 		return chromeAI;
 	}
 
-	// TODO: consider also using ChromeAI for ai-assistant-summarize
 	if ( promptType.startsWith( 'ai-content-lens' ) ) {
+		if ( ! ( 'ai' in self ) || ! ( 'summarizer' in self.ai ) ) {
+			return false;
+		}
+
+		if ( context.language ) {
path: projects/js-packages/ai-client/src/chrome-ai/factory.ts
user: login: vianasw
body: Yep, this is the result of rewrite number 4 or 5. I'll change it.
created_at: 2025-05-08T10:25:51Z
author_association: CONTRIBUTOR
diff_hunk: @@ -75,13 +75,43 @@ export default async function ChromeAIFactory( promptArg: PromptProp ) {
 		}
 	}
 
+	// Early return if the prompt type is not supported.
+	if (
+		! promptType.startsWith( 'ai-assistant-change-language' ) &&
+		! promptType.startsWith( 'ai-content-lens' )
+	) {
+		return false;
+	}
+
+	// If the languageDetector is not available, we can't use the translation or summary features—it's safer to fall back
+	// to the default AI model than to risk an unexpected error.
+	if (
+		! ( 'ai' in self ) ||
path: projects/js-packages/ai-client/src/chrome-ai/factory.ts
user: login: CGastrell
body: This check is failing, while I get `ai` object, `languageDetector` is not there. What I could dig about it is the [LanguageDetector API](https://developer.chrome.com/docs/ai/language-detection) might have changed its implementation. Once I enable the experimental flag, I get LanguageDetector's create and availability. 
created_at: 2025-05-09T18:07:04Z
author_association: CONTRIBUTOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-05-07T09:53:18Z
body: <!-- wpcom-reminder-comment -->
Are you an Automattician? Please test your changes on all WordPress.com environments to help mitigate accidental explosions.

- To test on WoA, go to the Plugins menu on a WoA dev site. Click on the "Upload" button and follow the upgrade flow to be able to upload, install, and activate [the Jetpack Beta plugin](https://jetpack.com/download-jetpack-beta/). Once the plugin is active, go to Jetpack > Jetpack Beta, select your plugin (Jetpack), and enable the `add/chrome-ai-event-properties` branch.
- To test on Simple, run the following command on your sandbox:
```
bin/jetpack-downloader test jetpack add/chrome-ai-event-properties
```

*Interested in more tips and information?*

- In your local development environment, use the `jetpack rsync` command to sync your changes to a WoA dev blog.
- Read more about our development workflow here: PCYsg-eg0-p2
- Figure out **when your changes will be shipped to customers** here: PCYsg-eg5-p2
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-05-07T09:53:22Z
body: **Thank you for your PR!**

When contributing to Jetpack, we have [a few suggestions](https://github.com/Automattic/jetpack/blob/trunk/.github/PULL_REQUEST_TEMPLATE.md) that can help us test and review your patch:<br>
- :white_check_mark: Include a description of your PR changes.<br>
- :white_check_mark: Add a "[Status]" label (In Progress, Needs Review, ...).<br>
- :white_check_mark: Add a "[Type]" label (Bug, Enhancement, Janitorial, Task).<br>
- :white_check_mark: Add testing instructions.<br>
- :white_check_mark: Specify whether this PR includes any changes to data or privacy.<br>
- :white_check_mark: Add changelog entries to affected projects<br>


This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation :robot:

******

**Follow this PR Review Process:**

1. Ensure all required checks appearing at the bottom of this PR are passing.
2. Make sure to test your changes on all platforms that it applies to. *You're responsible for the quality of the code you ship*.
3. You can use [GitHub's Reviewers functionality](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/requesting-a-pull-request-review) to request a review.
4. When it's reviewed and merged, you will be pinged in Slack to deploy the changes to WordPress.com simple once the build is done.

If you have questions about anything, reach out in #jetpack-developers for guidance!
******

**Jetpack plugin:**

No scheduled milestone found for this plugin.

If you have any questions about the release process, please ask in the #jetpack-releases channel on Slack.

author_association: CONTRIBUTOR
user: login: jp-launch-control[bot]
created_at: 2025-05-07T10:01:32Z
body: ### Code Coverage Summary

Coverage changed in 5 files.

| File | Coverage | Δ% | Δ Uncovered |
| :--- | :------: | :---: | :---: |
| projects/js-packages/ai-client/src/chrome-ai/factory.ts | 0/62 (0.00%) | 0.00% | 16 💔 |
| projects/js-packages/ai-client/src/hooks/use-ai-suggestions/index.ts | 0/91 (0.00%) | 0.00% | 7 💔 |
| projects/plugins/jetpack/extensions/blocks/ai-assistant/extensions/text-blocks/with-ai-text-extension.tsx | 0/178 (0.00%) | 0.00% | 5 💔 |
| projects/js-packages/ai-client/src/types.ts | 0/21 (0.00%) | 0.00% | 2 ❤️‍🩹 |
| projects/js-packages/ai-client/src/chrome-ai/suggestions.ts | 0/49 (0.00%) | 0.00% | 1 ❤️‍🩹 |



[Full summary](https://jetpackcodecoverage.atomicsites.blog/prs/43390/) · [PHP report](https://jetpackcodecoverage.atomicsites.blog/prs/43390/php/) · [JS report](https://jetpackcodecoverage.atomicsites.blog/prs/43390/js/)

<sub>Coverage check overridden by https://github.com/Automattic/jetpack/labels/I%20don%27t%20care%20about%20code%20coverage%20for%20this%20PR.</sub>
author_association: NONE