Repository: symfony/ux
Pull request #2206 (2024-09-25 13:42:44)

[Translator] Many performance improvements on the cache warming
| Q             | A
| ------------- | ---
| Bug fix?      | no
| New feature?  | no <!-- please update src/**/CHANGELOG.md files -->
| Issues        | Fix #... <!-- prefix each issue number with "Fix #", no need to create an issue if none exist, explain below instead -->
| License       | MIT

The Symfony UX Translator cache warmer can be an long task, especially if your app has a lot of translations.

I was able to identify some bottlenecks and fix them, with a dedicated commit for each of them.

My test application have ~25k translation keys (3.119 keys * 8 locales). Of course that's a lot, but I believe some people needs to dump all their translations (e.g. if they build an SPA on Symfony). For "more classic" applications, it is recommended to [use dedicated domains for translations JS-side](https://symfony.com/bundles/ux-translator/current/index.html#configuring-the-dumped-translations).

Anyway, in my application, I was able to reduce the Translator cache warming **from 2m11s** (higher than the reality, because of Blackfire profiling) **to 41s** (higher than the reality, because of Blackfire profiling):
<img width="1339" alt="image" src="https://github.com/user-attachments/assets/98b5546b-0ba9-457d-b007-e8afdee7e5ac">

The vast majority of improvements come from better use of [the Symfony String component](https://symfony.com/doc/current/string.html).

Blackfire profiles:
1. [Initial](https://app.blackfire.io/profiles/2e264f01-c3a6-46de-92fa-0856b2078010/graph) 
2. [Re-using `s()` instances](https://app.blackfire.io/profiles/b4c5dea7-4309-4cb8-b99f-584ec3a4ddc9/graph)
3. [Removing useless `s()->slice()`](https://app.blackfire.io/profiles/f66b6bdf-39da-4d98-a966-781418b3a6f5/graph)
4. [Caching `$this->message->length()`](https://app.blackfire.io/profiles/1334e3e1-4405-44d1-a327-e446dad7d1d8/graph) as it will always return the same length
5. [Before/After](https://blackfire.io/profiles/compare/8a0df8d0-5d01-47bf-a064-bfed6783c5a3/graph): ~63% 🚀 

Also, since the Translator component is still experimental, I've marked some classes as final and internal, as they should only be used internally. :) 


Patch:
diff --git a/src/Translator/src/Intl/ErrorKind.php b/src/Translator/src/Intl/ErrorKind.php
index cabe240e..94066bba 100644
--- a/src/Translator/src/Intl/ErrorKind.php
+++ b/src/Translator/src/Intl/ErrorKind.php
@@ -14,7 +14,7 @@ namespace Symfony\UX\Translator\Intl;
 /**
  * Adapted from https://github.com/formatjs/formatjs/blob/590f1f81b26934c6dc7a55fff938df5436c6f158/packages/icu-messageformat-parser/error.ts#L9-L77.
  *
- * @experimental
+ * @internal
  */
 final class ErrorKind
 {
diff --git a/src/Translator/src/Intl/IntlMessageParser.php b/src/Translator/src/Intl/IntlMessageParser.php
index 839de42f..560b883e 100644
--- a/src/Translator/src/Intl/IntlMessageParser.php
+++ b/src/Translator/src/Intl/IntlMessageParser.php
@@ -11,16 +11,21 @@
 
 namespace Symfony\UX\Translator\Intl;
 
+use Symfony\Component\String\AbstractString;
+
 use function Symfony\Component\String\s;
 
 /**
  * Adapted from https://github.com/formatjs/formatjs/blob/590f1f81b26934c6dc7a55fff938df5436c6f158/packages/icu-messageformat-parser/parser.ts.
  *
- * @experimental
+ * @internal
  */
-class IntlMessageParser
+final class IntlMessageParser
 {
-    private string $message;
+    private readonly AbstractString $message;
+    // Minor optimization, this avoid a lot of calls to `$this->message->length()`
+    private readonly int $messageLength;
+
     private Position $position;
     private bool $ignoreTag;
     private bool $requiresOtherClause;
@@ -28,7 +33,8 @@ class IntlMessageParser
     public function __construct(
         string $message,
     ) {
-        $this->message = $message;
+        $this->message = s($message);
+        $this->messageLength = $this->message->length();
         $this->position = new Position(0, 1, 1);
         $this->ignoreTag = true;
         $this->requiresOtherClause = true;
@@ -112,14 +118,14 @@ class IntlMessageParser
      */
     private function parseTagName(): string
     {
-        $startOffset = $this->offset();
+        $startOffset = $this->position->offset;
 
         $this->bump(); // the first tag name character
         while (!$this->isEOF() && Utils::isPotentialElementNameChar($this->char())) {
             $this->bump();
         }
 
-        return s($this->message)->slice($startOffset, $this->offset() - $startOffset)->toString();
+        return $this->message->slice($startOffset, $this->position->offset - $startOffset)->toString();
     }
 
     /**
@@ -365,7 +371,7 @@ class IntlMessageParser
     {
         $startingPosition = clone $this->position;
 
-        $startOffset = $this->offset();
+        $startOffset = $this->position->offset;
         $value = Utils::matchIdentifierAtIndex($this->message, $startOffset);
         $endOffset = $startOffset + s($value)->length();
 
@@ -445,9 +451,9 @@ class IntlMessageParser
                 );
 
                 // Extract style or skeleton
-                if ($styleAndLocation && s($styleAndLocation['style'] ?? '')->startsWith('::')) {
+                if ($styleAndLocation && ($style = s($styleAndLocation['style'] ?? ''))->startsWith('::')) {
                     // Skeleton starts with `::`.
-                    $skeleton = s($styleAndLocation['style'])->slice(2)->trimStart()->toString();
+                    $skeleton = $style->slice(2)->trimStart()->toString();
 
                     if ('number' === $argType) {
                         $result = $this->parseNumberSkeletonFromString(
@@ -663,7 +669,7 @@ class IntlMessageParser
                         --$nestedBraces;
                     } else {
                         return [
-                            'val' => s($this->message)->slice($startPosition->offset, $this->offset() - $startPosition->offset)->toString(),
+                            'val' => $this->message->slice($startPosition->offset, $this->position->offset - $startPosition->offset)->toString(),
                             'err' => null,
                         ];
                     }
@@ -676,7 +682,7 @@ class IntlMessageParser
         }
 
         return [
-            'val' => s($this->message)->slice($startPosition->offset, $this->offset() - $startPosition->offset)->toString(),
+            'val' => $this->message->slice($startPosition->offset, $this->position->offset - $startPosition->offset)->toString(),
             'err' => null,
         ];
     }
@@ -735,7 +741,7 @@ class IntlMessageParser
                         return $result;
                     }
                     $selectorLocation = new Location($startPosition, clone $this->position);
-                    $selector = s($this->message)->slice($startPosition->offset, $this->offset() - $startPosition->offset)->toString();
+                    $selector = $this->message->slice($startPosition->offset, $this->position->offset - $startPosition->offset)->toString();
                 } else {
                     break;
                 }
@@ -864,14 +870,9 @@ class IntlMessageParser
         ];
     }
 
-    private function offset(): int
-    {
-        return $this->position->offset;
-    }
-
     private function isEOF(): bool
     {
-        return $this->offset() === s($this->message)->length();
+        return $this->position->offset === $this->messageLength;
     }
 
     /**
@@ -882,16 +883,14 @@ class IntlMessageParser
      */
     private function char(): int
     {
-        $message = s($this->message);
-
         $offset = $this->position->offset;
-        if ($offset >= $message->length()) {
+        if ($offset >= $this->messageLength) {
             throw new \OutOfBoundsException();
         }
 
-        $code = $message->slice($offset, 1)->codePointsAt(0)[0] ?? null;
+        $code = $this->message->codePointsAt($offset)[0] ?? null;
         if (null === $code) {
-            throw new \Exception("Offset {$offset} is at invalid UTF-16 code unit boundary");
+            throw new \Exception("Offset {$offset} is at invalid UTF-16 code unit boundary.");
         }
 
         return $code;
@@ -909,7 +908,7 @@ class IntlMessageParser
             'err' => [
                 'kind' => $kind,
                 'location' => $location,
-                'message' => $this->message,
+                'message' => $this->message->toString(),
             ],
         ];
     }
@@ -941,7 +940,7 @@ class IntlMessageParser
      */
     private function bumpIf(string $prefix): bool
     {
-        if (s($this->message)->slice($this->offset())->startsWith($prefix)) {
+        if ($this->message->slice($this->position->offset)->startsWith($prefix)) {
             for ($i = 0, $len = \strlen($prefix); $i < $len; ++$i) {
                 $this->bump();
             }
@@ -958,14 +957,13 @@ class IntlMessageParser
      */
     private function bumpUntil(string $pattern): bool
     {
-        $currentOffset = $this->offset();
-        $index = s($this->message)->indexOf($pattern, $currentOffset);
+        $index = $this->message->indexOf($pattern, $this->position->offset);
         if ($index >= 0) {
             $this->bumpTo($index);
 
             return true;
         } else {
-            $this->bumpTo(s($this->message)->length());
+            $this->bumpTo($this->messageLength);
 
             return false;
         }
@@ -979,18 +977,18 @@ class IntlMessageParser
      */
     private function bumpTo(int $targetOffset)
     {
-        if ($this->offset() > $targetOffset) {
-            throw new \Exception(\sprintf('targetOffset %s must be greater than or equal to the current offset %d', $targetOffset, $this->offset()));
+        if ($this->position->offset > $targetOffset) {
+            throw new \Exception(\sprintf('targetOffset "%s" must be greater than or equal to the current offset %d', $targetOffset, $this->position->offset));
         }
 
-        $targetOffset = min($targetOffset, s($this->message)->length());
+        $targetOffset = min($targetOffset, $this->messageLength);
         while (true) {
-            $offset = $this->offset();
+            $offset = $this->position->offset;
             if ($offset === $targetOffset) {
                 break;
             }
             if ($offset > $targetOffset) {
-                throw new \Exception("targetOffset {$targetOffset} is at invalid UTF-16 code unit boundary");
+                throw new \Exception("targetOffset {$targetOffset} is at invalid UTF-16 code unit boundary.");
             }
 
             $this->bump();
@@ -1019,8 +1017,7 @@ class IntlMessageParser
         }
 
         $code = $this->char();
-        $offset = $this->offset();
-        $nextCodes = s($this->message)->codePointsAt($offset + ($code >= 0x10000 ? 2 : 1));
+        $nextCodes = $this->message->codePointsAt($this->position->offset + ($code >= 0x10000 ? 2 : 1));
 
         return $nextCodes[0] ?? null;
     }
diff --git a/src/Translator/src/Intl/Location.php b/src/Translator/src/Intl/Location.php
index e342a2ff..270a1bfe 100644
--- a/src/Translator/src/Intl/Location.php
+++ b/src/Translator/src/Intl/Location.php
@@ -14,7 +14,7 @@ namespace Symfony\UX\Translator\Intl;
 /**
  * Adapted from https://github.com/formatjs/formatjs/blob/590f1f81b26934c6dc7a55fff938df5436c6f158/packages/icu-messageformat-parser/types.ts#L58-L61.
  *
- * @experimental
+ * @internal
  */
 final class Location
 {
diff --git a/src/Translator/src/Intl/Position.php b/src/Translator/src/Intl/Position.php
index bc633622..0772508c 100644
--- a/src/Translator/src/Intl/Position.php
+++ b/src/Translator/src/Intl/Position.php
@@ -14,7 +14,7 @@ namespace Symfony\UX\Translator\Intl;
 /**
  * Adapted from https://github.com/formatjs/formatjs/blob/590f1f81b26934c6dc7a55fff938df5436c6f158/packages/icu-messageformat-parser/types.ts#L53-L57.
  *
- * @experimental
+ * @internal
  */
 final class Position
 {
diff --git a/src/Translator/src/Intl/SkeletonType.php b/src/Translator/src/Intl/SkeletonType.php
index b10daeb0..089c87fe 100644
--- a/src/Translator/src/Intl/SkeletonType.php
+++ b/src/Translator/src/Intl/SkeletonType.php
@@ -14,7 +14,7 @@ namespace Symfony\UX\Translator\Intl;
 /**
  * Adapted from https://github.com/formatjs/formatjs/blob/590f1f81b26934c6dc7a55fff938df5436c6f158/packages/icu-messageformat-parser/types.ts#L48-L51.
  *
- * @experimental
+ * @internal
  */
 final class SkeletonType
 {
diff --git a/src/Translator/src/Intl/Type.php b/src/Translator/src/Intl/Type.php
index 12927c34..d02bab4a 100644
--- a/src/Translator/src/Intl/Type.php
+++ b/src/Translator/src/Intl/Type.php
@@ -14,7 +14,7 @@ namespace Symfony\UX\Translator\Intl;
 /**
  * Adapted from https://github.com/formatjs/formatjs/blob/590f1f81b26934c6dc7a55fff938df5436c6f158/packages/icu-messageformat-parser/types.ts#L8-L46.
  *
- * @experimental
+ * @internal
  */
 final class Type
 {
diff --git a/src/Translator/src/Intl/Utils.php b/src/Translator/src/Intl/Utils.php
index af363073..b5c44a5f 100644
--- a/src/Translator/src/Intl/Utils.php
+++ b/src/Translator/src/Intl/Utils.php
@@ -11,10 +11,10 @@
 
 namespace Symfony\UX\Translator\Intl;
 
-use function Symfony\Component\String\s;
+use Symfony\Component\String\AbstractString;
 
 /**
- * @experimental
+ * @internal
  */
 final class Utils
 {
@@ -355,12 +355,12 @@ final class Utils
         return $elements;
     }
 
-    public static function matchIdentifierAtIndex(string $s, int $index): string
+    public static function matchIdentifierAtIndex(AbstractString $s, int $index): string
     {
         $match = [];
 
         while (true) {
-            $c = s($s)->codePointsAt($index)[0] ?? null;
+            $c = $s->codePointsAt($index)[0] ?? null;
             if (null === $c || self::isWhiteSpace($c) || self::isPatternSyntax($c)) {
                 break;
             }
diff --git a/src/Translator/src/MessageParameters/Extractor/ExtractorInterface.php b/src/Translator/src/MessageParameters/Extractor/ExtractorInterface.php
index 93d2b1d5..18c709c3 100644
--- a/src/Translator/src/MessageParameters/Extractor/ExtractorInterface.php
+++ b/src/Translator/src/MessageParameters/Extractor/ExtractorInterface.php
@@ -14,7 +14,7 @@ namespace Symfony\UX\Translator\MessageParameters\Extractor;
 /**
  * @author Hugo Alliaume <hugo@alliau.me>
  *
- * @experimental
+ * @internal
  */
 interface ExtractorInterface
 {
diff --git a/src/Translator/src/MessageParameters/Extractor/IntlMessageParametersExtractor.php b/src/Translator/src/MessageParameters/Extractor/IntlMessageParametersExtractor.php
index 6722f882..091e2789 100644
--- a/src/Translator/src/MessageParameters/Extractor/IntlMessageParametersExtractor.php
+++ b/src/Translator/src/MessageParameters/Extractor/IntlMessageParametersExtractor.php
@@ -17,7 +17,7 @@ use Symfony\UX\Translator\Intl\Type;
 /**
  * @author Hugo Alliaume <hugo@alliau.me>
  *
- * @experimental
+ * @internal
  */
 final class IntlMessageParametersExtractor implements ExtractorInterface
 {
diff --git a/src/Translator/src/MessageParameters/Extractor/MessageParametersExtractor.php b/src/Translator/src/MessageParameters/Extractor/MessageParametersExtractor.php
index 289c32db..38988ea7 100644
--- a/src/Translator/src/MessageParameters/Extractor/MessageParametersExtractor.php
+++ b/src/Translator/src/MessageParameters/Extractor/MessageParametersExtractor.php
@@ -14,7 +14,7 @@ namespace Symfony\UX\Translator\MessageParameters\Extractor;
 /**
  * @author Hugo Alliaume <hugo@alliau.me>
  *
- * @experimental
+ * @internal
  */
 final class MessageParametersExtractor implements ExtractorInterface
 {
diff --git a/src/Translator/src/MessageParameters/Printer/TypeScriptMessageParametersPrinter.php b/src/Translator/src/MessageParameters/Printer/TypeScriptMessageParametersPrinter.php
index 4eaf9c89..2981142e 100644
--- a/src/Translator/src/MessageParameters/Printer/TypeScriptMessageParametersPrinter.php
+++ b/src/Translator/src/MessageParameters/Printer/TypeScriptMessageParametersPrinter.php
@@ -14,7 +14,7 @@ namespace Symfony\UX\Translator\MessageParameters\Printer;
 /**
  * @author Hugo Alliaume <hugo@alliau.me>
  *
- * @experimental
+ * @internal
  */
 final class TypeScriptMessageParametersPrinter
 {
diff --git a/src/Translator/src/TranslationsDumper.php b/src/Translator/src/TranslationsDumper.php
index cba61117..a986c7c0 100644
--- a/src/Translator/src/TranslationsDumper.php
+++ b/src/Translator/src/TranslationsDumper.php
@@ -155,21 +155,18 @@ TS
                 }
 
                 $parametersTypes[$domain] = $this->typeScriptMessageParametersPrinter->print($parameters);
-
                 $locales[] = $locale;
             }
         }
 
+        $typeScriptParametersType = [];
+        foreach ($parametersTypes as $domain => $parametersType) {
+            $typeScriptParametersType[] = \sprintf("'%s': { parameters: %s }", $domain, $parametersType);
+        }
+
         return \sprintf(
             'Message<{ %s }, %s>',
-            implode(', ', array_reduce(
-                array_keys($parametersTypes),
-                fn (array $carry, string $domain) => [
-                    ...$carry,
-                    \sprintf("'%s': { parameters: %s }", $domain, $parametersTypes[$domain]),
-                ],
-                [],
-            )),
+            implode(', ', $typeScriptParametersType),
             implode('|', array_map(fn (string $locale) => "'$locale'", array_unique($locales))),
         );
     }
@@ -189,13 +186,14 @@ TS
     {
         static $alreadyGenerated = [];
 
+        $translationId = s($translationId)->ascii()->snake()->upper()->replaceMatches('/^(\d)/', '_$1')->toString();
         $prefix = 0;
         do {
-            $constantName = s($translationId)->ascii()->snake()->upper()->replaceMatches('/^(\d)/', '_$1')->toString().($prefix > 0 ? '_'.$prefix : '');
+            $constantName = $translationId.($prefix > 0 ? '_'.$prefix : '');
             ++$prefix;
-        } while (\in_array($constantName, $alreadyGenerated, true));
+        } while ($alreadyGenerated[$constantName] ?? false);
 
-        $alreadyGenerated[] = $constantName;
+        $alreadyGenerated[$constantName] = true;
 
         return $constantName;
     }


Test patch:
diff --git a/src/Translator/tests/TranslationsDumperTest.php b/src/Translator/tests/TranslationsDumperTest.php
index c68a9fef..10b13d90 100644
--- a/src/Translator/tests/TranslationsDumperTest.php
+++ b/src/Translator/tests/TranslationsDumperTest.php
@@ -57,6 +57,7 @@ export const NOTIFICATION_COMMENT_CREATED = {"id":"notification.comment_created"
 export const NOTIFICATION_COMMENT_CREATED_DESCRIPTION = {"id":"notification.comment_created.description","translations":{"messages+intl-icu":{"en":"Your post \"{title}\" has received a new comment. You can read the comment by following <a href=\"{link}\">this link<\/a>","fr":"Votre article \"{title}\" a re\u00e7u un nouveau commentaire. Vous pouvez lire le commentaire en suivant <a href=\"{link}\">ce lien<\/a>"}}};
 export const POST_NUM_COMMENTS = {"id":"post.num_comments","translations":{"messages+intl-icu":{"en":"{count, plural, one {# comment} other {# comments}}","fr":"{count, plural, one {# commentaire} other {# commentaires}}"},"foobar":{"en":"There is 1 comment|There are %count% comments","fr":"Il y a 1 comment|Il y a %count% comments"}}};
 export const POST_NUM_COMMENTS_1 = {"id":"post.num_comments.","translations":{"messages+intl-icu":{"en":"{count, plural, one {# comment} other {# comments}} (should not conflict with the previous one.)","fr":"{count, plural, one {# commentaire} other {# commentaires}} (ne doit pas rentrer en conflit avec la traduction pr\u00e9c\u00e9dente)"}}};
+export const POST_NUM_COMMENTS_2 = {"id":"post.num_comments..","translations":{"messages+intl-icu":{"en":"{count, plural, one {# comment} other {# comments}} (should not conflict with the previous one.)","fr":"{count, plural, one {# commentaire} other {# commentaires}} (ne doit pas rentrer en conflit avec la traduction pr\u00e9c\u00e9dente)"}}};
 export const SYMFONY_GREAT = {"id":"symfony.great","translations":{"messages":{"en":"Symfony is awesome!","fr":"Symfony est g\u00e9nial !"}}};
 export const SYMFONY_WHAT = {"id":"symfony.what","translations":{"messages":{"en":"Symfony is %what%!","fr":"Symfony est %what%!"}}};
 export const SYMFONY_WHAT_1 = {"id":"symfony.what!","translations":{"messages":{"en":"Symfony is %what%! (should not conflict with the previous one.)","fr":"Symfony est %what%! (ne doit pas rentrer en conflit avec la traduction pr\u00e9c\u00e9dente)"}}};
@@ -83,6 +84,7 @@ export declare const NOTIFICATION_COMMENT_CREATED: Message<{ 'messages+intl-icu'
 export declare const NOTIFICATION_COMMENT_CREATED_DESCRIPTION: Message<{ 'messages+intl-icu': { parameters: { 'title': string, 'link': string } } }, 'en'|'fr'>;
 export declare const POST_NUM_COMMENTS: Message<{ 'messages+intl-icu': { parameters: { 'count': number } }, 'foobar': { parameters: { '%count%': number } } }, 'en'|'fr'>;
 export declare const POST_NUM_COMMENTS_1: Message<{ 'messages+intl-icu': { parameters: { 'count': number } } }, 'en'|'fr'>;
+export declare const POST_NUM_COMMENTS_2: Message<{ 'messages+intl-icu': { parameters: { 'count': number } } }, 'en'|'fr'>;
 export declare const SYMFONY_GREAT: Message<{ 'messages': { parameters: NoParametersType } }, 'en'|'fr'>;
 export declare const SYMFONY_WHAT: Message<{ 'messages': { parameters: { '%what%': string } } }, 'en'|'fr'>;
 export declare const SYMFONY_WHAT_1: Message<{ 'messages': { parameters: { '%what%': string } } }, 'en'|'fr'>;
@@ -149,6 +151,7 @@ TYPESCRIPT);
                     'notification.comment_created.description' => 'Your post "{title}" has received a new comment. You can read the comment by following <a href="{link}">this link</a>',
                     'post.num_comments' => '{count, plural, one {# comment} other {# comments}}',
                     'post.num_comments.' => '{count, plural, one {# comment} other {# comments}} (should not conflict with the previous one.)',
+                    'post.num_comments..' => '{count, plural, one {# comment} other {# comments}} (should not conflict with the previous one.)',
                 ],
                 'messages' => [
                     'symfony.great' => 'Symfony is awesome!',
@@ -178,6 +181,7 @@ TYPESCRIPT);
                     'notification.comment_created.description' => 'Votre article "{title}" a reçu un nouveau commentaire. Vous pouvez lire le commentaire en suivant <a href="{link}">ce lien</a>',
                     'post.num_comments' => '{count, plural, one {# commentaire} other {# commentaires}}',
                     'post.num_comments.' => '{count, plural, one {# commentaire} other {# commentaires}} (ne doit pas rentrer en conflit avec la traduction précédente)',
+                    'post.num_comments..' => '{count, plural, one {# commentaire} other {# commentaires}} (ne doit pas rentrer en conflit avec la traduction précédente)',
                 ],
                 'messages' => [
                     'symfony.great' => 'Symfony est génial !',