Bug: MozIntGemm is not working for larger translation models
Status: VERIFIED
Priority: P1
Resolution: FIXED
Created: 2024-10-21 19:33:05+00:00
Resolved: 2025-01-31 07:42:25+00:00

### Description

We are experimenting with the tradeoffs between model size, translation quality, and translation performance. In [#169](https://github.com/mozilla/firefox-translations-models/commit/6d33a2784c8690618eb72cb60ba04ee997830c89) we added a new `en-ru` model that is roughly twice the size (~40MB) of the current `en-ru` model (~20MB). 

Unfortunately, the new larger-size model doesn't seem to be working in Firefox's Translations ecosystem. I've narrowed down the cause to be related to [MozIntGemm](https://searchfox.org/mozilla-central/search?q=MozIntGemm). The model translates correctly when the `javascript.options.wasm_moz_intgemm` pref is set to `false`, but the Translation fails when the pref is set to `true`. 

`MozIntGemm` is a critical component for maintaining performant translations, so it is imperative that we attempt to fix the issue if we are going to support larger-size models. 

I ran our tentative manual [benchmarks](https://gregtatum.github.io/taskcluster-tools/src/benchmark/) with the following results:

**Small Model | MozIntGemm**

- **9.03** seconds (base metric)
- **786.66** words per second (base metric)

**Small Model | Fallback IntGemm**

- **16.49** seconds (**82.61%** increase from base)
- **420.92** words per second (**46.49%** decrease from base)

**Large Model | Fallback IntGemm**

- **29.19** seconds(**223.25%** increase from base)
- **237.79** words per second (**69.77%** decrease from base)

---

### Steps to reproduce

> Note: This STR is based on the current state of the new larger-size `en-ru` model being available in the `Prod (preview)` channel of Remote Settings. When this bug is resolved and that model is no longer actively in the `Prod (preview)` channel, this STR will no longer be valid. 

1) Open a Nightly or locally built version of Firefox to `about:config` and ensure that the `browser.translations.LogLevel` pref is set to `All` instead of its default `Error`.  This will enable Translations logging in the browser console. Also ensure that the `javascript.options.wasm_moz_intgemm` pref is set to `true`, which should be the default.
2) Navigate to the `remote-settings-devtools` [releases page](https://github.com/mozilla-extensions/remote-settings-devtools/releases) and download and install the latest XPI file for the extension. 
3) Open the Remote Settings Devtools extension. The currently selected Environment should be `Prod`. Click the `Clear all` button, then switch the environment to `Prod (preview)`. Click the `Clear all` button again in the `Prod (preview)` environment and then click the `Sync` button once the clear finishes. Note, if the `Sync` button appears to do nothing, then you may have to restart the browser, re-open the extension, and try the sync again. I think this is a bug in Remote Settings. 
4) Navigate to a translatable web page, e.g. https://en.wikipedia.org/wiki/Firefox.
5) Open the Application (hamburger) menu near the top-right of the browser.
6) Click the "Translate page..." menu item in the Application menu.
7) When the Full Page Translations Panel opens, set the source language to English and the target language to Russian.
8) Click Translate. 
9) Check the browser console log to ensure that the `en-ru` models that are downloaded are version `1.0a3`. 

**Expected Results**
The page translates from English to Russian

**Actual Results**
Much of the page content is lost due to Translation requests coming back empty. 

10) Open `about:config` and set the `javascript.options.wasm_moz_intgemm` pref to `false`.
11) Restart the browser.
12) Repeat steps 4) through 9).
13) Observe that the translation from English to Russian works with the `javascript.options.wasm_moz_intgemm` set to `false.`