Repository: CSCfi/rems
Pull request #3295 (2024-05-15 14:09:51)

Faster translations
relates #3283 

Internal performance update to how translations are fetched.

Motivation
---
Email template transformations take quite a bit of time when large-ish number of emails are produced. For example `application.event/decided` creates 1 email for each workflow handler:

event-to-emails, disabled cache
---
| handler count | mean          | lower-q 2.5%  | upper-q 97.5% |
| ------------- | ------------- | ------------- | ------------- |
| 1             | 4,780888 ms   | 4,097078 ms   | 6,170799 ms   |
| 10            | 52,443133 ms  | 49,652797 ms  | 55,356244 ms  |
| 50            | 223,241818 ms | 203,445667 ms | 249,808264 ms |
| 100           | 441,938544 ms | 413,564604 ms | 478,001015 ms |
| 200           | 834,458932 ms | 804,961543 ms | 914,037245 ms |

(using 2019 Macbook Pro with 2,3 GHz 8-Core Intel Core i9)

![Screenshot 2024-05-15 at 17 22 06](https://github.com/CSCfi/rems/assets/794087/2213b01a-cc01-4d4e-b84d-6b561f2ee4cc)

Caching dictionary for tempura function reduces mean time by almost 100x. For same event, but using cached translation function:

event-to-emails, cache
---
| handler count | mean          | lower-q 2.5%  | upper-q 97.5% | cache size |
| ------------- | ------------- | ------------- | ------------- | ---------- |
| 1             | 40,996545 µs  | 39,542207 µs  | 43,635827 µs  | 882,5 KiB  |
| 10            | 397,217641 µs | 367,224108 µs | 435,834954 µs | 882,5 KiB  |
| 50            | 2,158321 ms   | 1,904828 ms   | 2,440271 ms   | 882,5 KiB  |
| 100           | 4,206048 ms   | 3,757476 ms   | 4,956851 ms   | 882,5 KiB  |
| 200           | 8,656116 ms   | 7,626819 ms   | 10,665969 ms  | 882,5 KiB  |

![Screenshot 2024-05-15 at 16 26 41](https://github.com/CSCfi/rems/assets/794087/d6a77cd6-e4c0-4ee6-96b0-ef20031d74e0)

# Checklist for author

Remove items that aren't applicable, check items that are done.

## Reviewability
- [ ] Link to issue (no issue number)
- [x] Consider adding screenshots for ease of review (clj-async-profiler)

## Documentation
- [ ] Update changelog if necessary

## Testing
- [x] Complex logic is unit tested
- [x] Valuable features are integration / browser / acceptance tested automatically

## Follow-up
- [ ] New tasks are created for pending or remaining tasks



Patch:
diff --git a/project.clj b/project.clj
index 22de8606c..c32674faf 100644
--- a/project.clj
+++ b/project.clj
@@ -113,6 +113,7 @@
    :test [:project/dev :project/test :profiles/test]
 
    :project/dev {:dependencies [[binaryage/devtools "1.0.7"]
+                                [com.clojure-goes-fast/clj-async-profiler "1.2.0"] ; also check extra :jvm-opts https://github.com/clojure-goes-fast/clj-async-profiler?tab=readme-ov-file#tuning-for-better-accuracy
                                 [com.clojure-goes-fast/clj-memory-meter "0.3.0"]
                                 [criterium "0.4.6"]
                                 [lambdaisland/kaocha "1.87.1366"]
diff --git a/src/clj/rems/markdown.clj b/src/clj/rems/markdown.clj
index e90de0dcf..a99783ef1 100644
--- a/src/clj/rems/markdown.clj
+++ b/src/clj/rems/markdown.clj
@@ -89,3 +89,40 @@
                                 "```"
                                 "the end"]))
       "keeps Clojure code intact"))
+
+(defn- apply-padding [coll]
+  (let [max-len (apply max (map count coll))
+        get-padding #(repeat (- max-len (count %)) " ")]
+    (->> coll
+         (mapv #(apply str % (get-padding %))))))
+
+(defn- add-table-header-separator [coll]
+  (let [make-separator (comp str/join #(repeat (count %) "-"))]
+    (vec
+     (concat (take 1 coll)
+             (conj [] (mapv make-separator (first coll)))
+             (drop 1 coll)))))
+
+(defn markdown-table [{:keys [header rows row-fn]}]
+  (let [transpose #(apply mapv vector %)
+        format-rows #(->> (str/join " | " %)
+                          (format "| %s |"))
+        row-fn (or row-fn identity)
+        cols (mapv str header)
+        rows (vec (for [row rows]
+                    (mapv str (row-fn row))))]
+    (->> (into [cols] rows)
+         transpose
+         (mapv apply-padding) ; align column width
+         transpose
+         add-table-header-separator
+         (mapv format-rows))))
+
+(comment
+  (doseq [row (markdown-table {:header ["handler count" "mean time" "lower-q 2.5%" "upper-q 97.5%"]
+                               :rows [[1 0.049 0.039 0.067]
+                                      [10 0.423 0.401 0.452]
+                                      [50 2.098 1.910 2.294]
+                                      [100 4.145 3.849 4.673]
+                                      [200 8.539 8.025 9.374]]})]
+    (println row)))
diff --git a/src/cljc/rems/tempura.cljc b/src/cljc/rems/tempura.cljc
index 8dbd5f889..7f184bd17 100644
--- a/src/cljc/rems/tempura.cljc
+++ b/src/cljc/rems/tempura.cljc
@@ -66,52 +66,55 @@
                        vargs)]
         (f res-args)))))
 
-(defn- get-map-compiler [{:keys [resource resource-keys]}]
-  (let [f (get-default-resource-compiler resource)]
+(defn- get-map-compiler [resource-with-map-params]
+  (let [{:keys [resource resource-keys]} (replace-map-args resource-with-map-params)
+        f (get-default-resource-compiler resource)]
     (fn compile-map-args [vargs]
-      (assert (map? (first vargs)) {:resource resource
-                                    :vargs vargs})
-      (let [res-args (mapv (first vargs) resource-keys)]
-        (f res-args)))))
+      (f (cond
+           (empty? vargs) []
+           (map? (first vargs)) (mapv (first vargs) resource-keys)
+           :else (assert (map? (first vargs))
+                         {:resource resource-with-map-params
+                          :resource-keys resource-keys
+                          :vargs vargs}))))))
+
 
 (defn- get-resource-compiler [resource]
   (if (seq (find-map-params resource))
-    (get-map-compiler (replace-map-args resource))
+    (get-map-compiler resource)
     (get-vec-compiler resource)))
 
-(defn tr
-  "Wrapper for `taoensso.tempura/tr`."
-  ([translations language ks] (taoensso.tempura/tr {:dict translations}
-                                                   [language]
-                                                   (vec ks)))
-  ([translations language ks args] (taoensso.tempura/tr {:dict translations
-                                                         :resource-compiler get-resource-compiler}
-                                                        [language]
-                                                        (vec ks)
-                                                        (vec args))))
+(def default-tr-opts {:cache-dict? :fn-local
+                      :resource-compiler get-resource-compiler})
+
+(defn get-cached-tr [translations & [opts]]
+  (taoensso.tempura/new-tr-fn (-> default-tr-opts
+                                  (merge opts)
+                                  (assoc :dict translations))))
 
-(deftest test-resource-compiler
+(deftest test-get-cached-tr
   (let [dict {:en
               {:string {:no-args "test"
                         :vector "%1 %2 %1"
                         :map "%:x% %:y% %:x%"}
                :hiccup {:no-args [:div {:aria-label "test"} [:span "test"]]
                         :vector [:div {:aria-label "%1 %2 %1"} [:span "%1 %2 %1"]]
-                        :map [:div {:aria-label "%:x% %:y% %:x%"} [:span "%:x% %:y% %:x%"]]}}}]
+                        :map [:div {:aria-label "%:x% %:y% %:x%"} [:span "%:x% %:y% %:x%"]]}}}
+        tr (get-cached-tr dict)]
     (testing "no args"
       (is (= "test"
-             (tr dict :en [:string/no-args])))
+             (tr [:en] [:string/no-args])))
       (is (= [:div {:aria-label "test"} [:span "test"]]
-             (tr dict :en [:hiccup/no-args]))))
+             (tr [:en] [:hiccup/no-args]))))
     (testing "index parameters"
       (is (= "1 2 1"
-             (tr dict :en [:string/vector] [1 2 3])))
+             (tr [:en] [:string/vector] [1 2 3])))
       ;; map attributes are identified by custom resource compiler, but tempura does not use them
       (is (= [:div {:aria-label "%1 %2 %1"} [:span "" 1 " " 2 " " 1]]
-             (tr dict :en [:hiccup/vector] [1 2 3]))))
+             (tr [:en] [:hiccup/vector] [1 2 3]))))
     (testing "named parameters"
       (is (= "1 2 1"
-             (tr dict :en [:string/map] [{:x 1 :y 2 :z 3}])))
+             (tr [:en] [:string/map] [{:x 1 :y 2 :z 3}])))
       ;; map attributes are identified by custom resource compiler, but tempura does not use them
       (is (= [:div {:aria-label "%1 %2 %1"} [:span "" 1 " " 2 " " 1]]
-             (tr dict :en [:hiccup/map] [{:x 1 :y 2 :z 3}]))))))
+             (tr [:en] [:hiccup/map] [{:x 1 :y 2 :z 3}]))))))
diff --git a/src/cljc/rems/text.cljc b/src/cljc/rems/text.cljc
index 4db9f9901..6524fbfa9 100644
--- a/src/cljc/rems/text.cljc
+++ b/src/cljc/rems/text.cljc
@@ -1,6 +1,5 @@
 (ns rems.text
-  (:require [better-cond.core :as b]
-            [clojure.string :as str]
+  (:require [clojure.string :as str]
             [clojure.test :refer [deftest is]]
             #?(:clj [clj-time.core :as time]
                :cljs [cljs-time.core :as time])
@@ -27,19 +26,31 @@
                  (first args)
                  (cons k args)))))
 
+(def cached-tr (atom nil))
+(defn reset-cached-tr! [] (reset! cached-tr nil))
+
+(defn ensure-cached-tr! []
+  (when-not (fn? @cached-tr)
+    (reset! cached-tr (rems.tempura/get-cached-tr #?(:clj rems.locales/translations
+                                                     :cljs @(rf/subscribe [:translations])))))
+  @cached-tr)
+
+(defn- tr [ks & [args]]
+  (let [language #?(:clj context/*lang*
+                    :cljs @(rf/subscribe [:language]))]
+    ((ensure-cached-tr!) [language]
+                         (vec ks)
+                         (some-> args vec))))
+
 (defn text-format
   "Return the tempura translation for a given key and arguments:
 
    `(text-format :key 1 2)`"
   [k & args]
-  #?(:clj (rems.tempura/tr rems.locales/translations
-                           context/*lang*
-                           [k :t/missing]
-                           args)
-     :cljs (rems.tempura/tr @(rf/subscribe [:translations])
-                            @(rf/subscribe [:language])
-                            [k :t/missing (failsafe-fallback k args)]
-                            args)))
+  #?(:clj (tr [k :t/missing]
+              args)
+     :cljs (tr [k :t/missing (failsafe-fallback k args)]
+               args)))
 
 (defn text-format-map
   "Return the tempura translation for a given key and argument map:
@@ -58,13 +69,9 @@
   "Return the tempura translation for a given key. Additional fallback
   keys can be given but there is no default fallback text."
   [& ks]
-  #?(:clj (rems.tempura/tr rems.locales/translations
-                           context/*lang*
-                           ks)
+  #?(:clj (tr ks)
      :cljs (try
-             (rems.tempura/tr @(rf/subscribe [:translations])
-                              @(rf/subscribe [:language])
-                              ks)
+             (tr ks)
              (catch js/Object e
                ;; fail gracefully if the re-frame state is incomplete
                (.error js/console e)
@@ -74,15 +81,11 @@
   "Return the tempura translation for a given key. Additional fallback
   keys can be given."
   [& ks]
-  #?(:clj (rems.tempura/tr rems.locales/translations
-                           context/*lang*
-                           (conj (vec ks) (text-format :t/missing (vec ks))))
+  #?(:clj (tr (conj (vec ks) (text-format :t/missing (vec ks))))
      ;; NB: we can't call the text-no-fallback here as in CLJS
      ;; we can both call this as function or use as a React component
      :cljs (try
-             (rems.tempura/tr @(rf/subscribe [:translations])
-                              @(rf/subscribe [:language])
-                              (conj (vec ks) (text-format :t/missing (vec ks))))
+             (tr (conj (vec ks) (text-format :t/missing (vec ks))))
              (catch js/Object e
                ;; fail gracefully if the re-frame state is incomplete
                (.error js/console e)


Test patch:
diff --git a/test/clj/rems/application/test_eraser.clj b/test/clj/rems/application/test_eraser.clj
index f6d79c53e..4349c28ab 100644
--- a/test/clj/rems/application/test_eraser.clj
+++ b/test/clj/rems/application/test_eraser.clj
@@ -6,7 +6,6 @@
             [rems.application.eraser :as eraser]
             [rems.application.expirer-bot :as expirer-bot]
             [rems.config :refer [env]]
-            [rems.locales :refer [translations]]
             [rems.db.applications :as applications]
             [rems.db.outbox :as outbox]
             [rems.db.roles :as roles]
@@ -223,10 +222,7 @@
                     user-settings/get-user-settings (constantly {:language :en})
                     env {:application-id-column :id
                          :public-url "localhost/"
-                         :application-expiration expiration-config}
-                    translations (-> translations
-                                     (assoc-in [:en :t :email :footer] "")
-                                     (assoc-in [:en :t :email :regards] ""))]
+                         :application-expiration expiration-config}]
         (with-fixed-time test-time
           (log-test/with-log
             (reset! outbox-emails [])
@@ -253,14 +249,16 @@
                          :outbox/email {:subject (str "Your unsubmitted application " draft-expires-in-6d " will be deleted soon")
                                         :body (str "Dear alice,"
                                                    "\n\nYour unsubmitted application has been inactive since 2022-10-09 and it will be deleted after 2023-01-08, if it is not edited."
-                                                   "\n\nYou can view and edit the application at localhost/application/" draft-expires-in-6d)
+                                                   "\n\nYou can view and edit the application at localhost/application/" draft-expires-in-6d
+                                                   "\n\nKind regards,\n\nREMS\n\n\nPlease do not reply to this automatically generated message.")
                                         :to-user "alice"}
                          :outbox/type :email}
                         {:outbox/deadline test-time
                          :outbox/email {:subject (str "Your unsubmitted application " draft-expires-in-6d " will be deleted soon")
                                         :body (str "Dear member,"
                                                    "\n\nYour unsubmitted application has been inactive since 2022-10-09 and it will be deleted after 2023-01-08, if it is not edited."
-                                                   "\n\nYou can view and edit the application at localhost/application/" draft-expires-in-6d)
+                                                   "\n\nYou can view and edit the application at localhost/application/" draft-expires-in-6d
+                                                   "\n\nKind regards,\n\nREMS\n\n\nPlease do not reply to this automatically generated message.")
                                         :to-user "member"}
                          :outbox/type :email}]
                        @outbox-emails))))))
@@ -327,9 +325,10 @@
             (testing "email is sent to applicant for expiring draft"
               (is (= [{:outbox/deadline test-time-plus-ten-days
                        :outbox/email {:subject (str "Your unsubmitted application " draft-expires-in-8d " will be deleted soon")
-                                      :body (str "Dear alice,\n\n"
-                                                 "Your unsubmitted application has been inactive since 2022-10-11 and it will be deleted after 2023-01-18, if it is not edited.\n\n"
-                                                 "You can view and edit the application at localhost/application/" draft-expires-in-8d)
+                                      :body (str "Dear alice,"
+                                                 "\n\nYour unsubmitted application has been inactive since 2022-10-11 and it will be deleted after 2023-01-18, if it is not edited."
+                                                 "\n\nYou can view and edit the application at localhost/application/" draft-expires-in-8d
+                                                 "\n\nKind regards,\n\nREMS\n\n\nPlease do not reply to this automatically generated message.")
                                       :to-user "alice"}
                        :outbox/type :email}]
                      @outbox-emails)))))))))
diff --git a/test/clj/rems/email/test_template.clj b/test/clj/rems/email/test_template.clj
index b939b4b05..47299d387 100644
--- a/test/clj/rems/email/test_template.clj
+++ b/test/clj/rems/email/test_template.clj
@@ -7,15 +7,9 @@
             [rems.config]
             [rems.db.user-settings :as user-settings]
             [rems.email.template :as template]
-            [rems.locales]))
-
-(defn empty-footer [f]
-  (with-redefs [rems.locales/translations (assoc-in rems.locales/translations [:en :t :email :footer] "")]
-    (f)))
-
-(defn empty-signature [f]
-  (with-redefs [rems.locales/translations (assoc-in rems.locales/translations [:en :t :email :regards] "")]
-    (f)))
+            [rems.locales]
+            [rems.text]
+            [rems.testing-util :refer [with-translations]]))
 
 (use-fixtures
   :once
@@ -27,8 +21,11 @@
 
 (use-fixtures
   :each
-  empty-signature
-  empty-footer)
+  (fn [f]
+    (with-translations (-> rems.locales/translations
+                           (update-vals #(assoc-in % [:t :email :footer] ""))
+                           (update-vals #(assoc-in % [:t :email :regards] "")))
+      (f))))
 
 (def ^:private get-catalogue-item
   {10 {:localizations {:en {:langcode :en
@@ -473,14 +470,13 @@
 
 (deftest test-finnish-emails
   ;; only one test case so far, more of a smoke test
-  (with-redefs [rems.locales/translations (assoc-in rems.locales/translations [:fi :t :email :regards] "")]
-    (testing "submitted"
-      (let [mails (emails :fi created-events submit-event)]
-        (is (= #{"applicant" "assistant" "handler"} (email-recipients mails)))
-        (is (= {:to-user "handler"
-                :subject "(2001/3, \"Application title\") Uusi hakemus"
-                :body "Hyvä Hannah Handler,\n\nAlice Applicant on lähettänyt käyttöoikeushakemuksen 2001/3, \"Application title\" resurss(e)ille fi title 11, fi title 21.\n\nVoit tarkastella hakemusta osoitteessa http://example.com/application/7\n\nTämä on automaattinen viesti. Älä vastaa."}
-               (email-to "handler" mails)))))))
+  (testing "submitted"
+    (let [mails (emails :fi created-events submit-event)]
+      (is (= #{"applicant" "assistant" "handler"} (email-recipients mails)))
+      (is (= {:to-user "handler"
+              :subject "(2001/3, \"Application title\") Uusi hakemus"
+              :body "Hyvä Hannah Handler,\n\nAlice Applicant on lähettänyt käyttöoikeushakemuksen 2001/3, \"Application title\" resurss(e)ille fi title 11, fi title 21.\n\nVoit tarkastella hakemusta osoitteessa http://example.com/application/7"}
+             (email-to "handler" mails))))))
 
 (deftest test-handler-reminder-email
   (with-redefs [rems.config/env (assoc rems.config/env :public-url "http://example.com/")]
@@ -561,7 +557,7 @@
             {:title "Template Workflow"})))))
 
 (deftest test-disabled-email
-  (with-redefs [rems.locales/translations (assoc-in rems.locales/translations [:en :t :email :application-submitted :message-to-handler] "")]
+  (with-translations (assoc-in rems.locales/translations [:en :t :email :application-submitted :message-to-handler] "")
     (let [mails (emails created-events submit-event)]
       (is (= #{"applicant"} (email-recipients mails))
           "applicant gets the email but handler messages are not sent")
@@ -599,13 +595,23 @@
                  :subject "Your unsubmitted application 2001/3, \"Application title\" will be deleted soon"
                  :body "Dear member,\n\nYour unsubmitted application has been inactive since 2023-03-28 and it will be deleted after 2023-06-26, if it is not edited.\n\nYou can view and edit the application at http://example.com/application/7"}]
                (emails member-events notifications-sent)))
-        (with-redefs [rems.locales/translations (assoc-in rems.locales/translations
-                                                          [:fi :t :email :application-expiration-notification :message-to-member]
-                                                          "Lähettämätön hakemuksesi %5 poistetaan automaattisesti %4 jälkeen, jos sitä ei muokata.")]
-          (is (= [{:to-user "applicant"
-                   :subject "Lähettämätön hakemuksesi 2001/3, \"Application title\" poistetaan pian"
-                   :body "Lähettämätön hakemuksesi http://example.com/application/7 poistetaan automaattisesti 2023-06-26 jälkeen, jos sitä ei muokata.\n\nYstävällisin terveisin,\n\nREMS\n\n\nTämä on automaattinen viesti. Älä vastaa."}
-                  {:to-user "member"
-                   :subject "Lähettämätön hakemuksesi 2001/3, \"Application title\" poistetaan pian"
-                   :body "Lähettämätön hakemuksesi http://example.com/application/7 poistetaan automaattisesti 2023-06-26 jälkeen, jos sitä ei muokata.\n\nYstävällisin terveisin,\n\nREMS\n\n\nTämä on automaattinen viesti. Älä vastaa."}]
-                 (emails :fi member-events notifications-sent))))))))
+        (testing "localization overrides"
+          (with-translations (-> rems.locales/translations
+                                 (assoc-in [:fi :t :email :application-expiration-notification :subject-to-member]
+                                           "Hakemus %2 poistetaan %4 jälkeen")
+                                 (assoc-in [:sv :t :email :application-expiration-notification :subject-to-member]
+                                           "Ansökan %:application-id% kommer att raderas efter %:expires-on%"))
+            (is (= [{:to-user "applicant"
+                     :subject "Hakemus 2001/3, \"Application title\" poistetaan 2023-06-26 jälkeen"
+                     :body "Hyvä Alice Applicant,\n\nLähettämätön hakemuksesi on ollut passiivisena 2023-03-28 lähtien ja se poistetaan 2023-06-26 jälkeen, jos sitä ei muokata. \n\nVoit tarkastella ja muokata hakemusta osoitteessa http://example.com/application/7"}
+                    {:to-user "member"
+                     :subject "Hakemus 2001/3, \"Application title\" poistetaan 2023-06-26 jälkeen"
+                     :body "Hyvä member,\n\nLähettämätön hakemuksesi on ollut passiivisena 2023-03-28 lähtien ja se poistetaan 2023-06-26 jälkeen, jos sitä ei muokata. \n\nVoit tarkastella ja muokata hakemusta osoitteessa http://example.com/application/7"}]
+                   (emails :fi member-events notifications-sent)))
+            (is (= [{:to-user "applicant"
+                     :subject "Ansökan 2001/3, \"Application title\" kommer att raderas efter 2023-06-26"
+                     :body "Alice Applicant,\n\nDin ej inlämnad ansökan har varit inaktiv sedan 2023-03-28 och kommer att raderas efter 2023-06-26, om det inte ändras. \n\nDu kan granska ansökningen här: http://example.com/application/7"}
+                    {:to-user "member"
+                     :subject "Ansökan 2001/3, \"Application title\" kommer att raderas efter 2023-06-26"
+                     :body "member,\n\nDin ej inlämnad ansökan har varit inaktiv sedan 2023-03-28 och kommer att raderas efter 2023-06-26, om det inte ändras. \n\nDu kan granska ansökningen här: http://example.com/application/7"}]
+                   (emails :sv member-events notifications-sent)))))))))
diff --git a/test/clj/rems/test_performance.clj b/test/clj/rems/test_performance.clj
index 822dc4390..16771b221 100644
--- a/test/clj/rems/test_performance.clj
+++ b/test/clj/rems/test_performance.clj
@@ -1,11 +1,20 @@
 (ns rems.test-performance
-  (:require [clj-memory-meter.core :as mm]
+  (:require [clj-async-profiler.core :as prof]
+            [clj-memory-meter.core :as mm]
             [criterium.core :as criterium]
             [medley.core :refer [map-vals]]
             [mount.core :as mount]
-            [rems.service.todos :as todos]
+            [rems.application.model]
+            [rems.config]
             [rems.db.applications :as applications]
-            [rems.db.events :as events])
+            [rems.db.events :as events]
+            [rems.db.test-data-users :refer [+fake-users+ +fake-user-data+]]
+            [rems.email.template]
+            [rems.locales]
+            [rems.markdown]
+            [rems.service.todos :as todos]
+            [rems.tempura]
+            [rems.text])
   (:import [java.util Locale]))
 
 (defn run-benchmark [benchmark]
@@ -53,10 +62,10 @@
         ;; developer can view much more applications than alice, so it takes longer to filter reviews from all apps
         test-get-todos #(doall (todos/get-todos "developer"))
         no-cache (fn []
-                   (mount/stop #'applications/all-applications-cache #'rems.db.events/low-level-events-cache))
+                   (mount/stop #'rems.db.applications/all-applications-cache #'rems.db.events/low-level-events-cache))
         cached (fn []
-                 (mount/stop #'applications/all-applications-cache #'rems.db.events/low-level-events-cache)
-                 (mount/start #'applications/all-applications-cache #'rems.db.events/low-level-events-cache)
+                 (mount/stop #'rems.db.applications/all-applications-cache #'rems.db.events/low-level-events-cache)
+                 (mount/start #'rems.db.applications/all-applications-cache #'rems.db.events/low-level-events-cache)
                  (test-get-all-unrestricted-applications))]
     (run-benchmarks [{:name "get-all-unrestricted-applications, no cache"
                       :benchmark test-get-all-unrestricted-applications
@@ -83,11 +92,102 @@
     (run-benchmarks [{:name "get-application"
                       :benchmark test-get-application}])))
 
+(defn- get-template-app-and-event [handler-count]
+  (let [fake-handlers (for [n (range handler-count)
+                            :let [userid (str "benchmark-handler-" n)]]
+                        [userid {:userid userid
+                                 :name (str "Benchmark handler " n)
+                                 :email "fake.handler.email@rems.performance.test"}])
+        application {:application/id 7
+                     :application/external-id "2024-05-15/1"
+                     :application/applicant (get +fake-user-data+ (:alice +fake-users+))
+                     :application/description "Template benchmark"
+                     :application/resources [{:catalogue-item/title {:en "Benchmark resource (EN)"
+                                                                     :fi "Benchmark resource (FI)"
+                                                                     :sv "Benchmark resource (SV)"}}]
+                     :application/workflow {:workflow.dynamic/handlers (set (map first fake-handlers))}}
+        decided (-> {:application/id 7
+                     :event/type :application.event/decided
+                     :event/actor (:decider +fake-users+)
+                     :application/decision :approved}
+                    (rems.application.model/enrich-event (into +fake-user-data+ fake-handlers) nil))]
+    {:application application
+     :event decided}))
+
+(defn benchmark-template-performance [{:keys [benchmark-handlers]}]
+  (let [setup-benchmark (fn setup-benchmark-f []
+                          (mount/start #'rems.config/env ; :enable-handler-emails is enabled by default
+                                       #'rems.locales/translations)
+                          (rems.text/reset-cached-tr!))
+        get-benchmark-fn (fn [handler-count]
+                           (let [{:keys [application event]} (get-template-app-and-event handler-count)]
+                             (fn benchmark-event-to-emails []
+                               (doall (rems.email.template/event-to-emails event application)))))
+        all-stats (atom nil)]
+
+    ;; Execution time mean : 9,400171 ms (2019 Macbook Pro with 2,3 GHz 8-Core Intel Core i9)
+    ;; translations cache size 882,5 KiB
+    (with-redefs [rems.db.user-settings/get-user-settings (fn [& _] {:language (rand-nth [:en :fi :sv])})]
+      (doall
+       (for [n benchmark-handlers
+             :let [test-event-to-emails (get-benchmark-fn n)
+                   stats (run-benchmark {:name (format "event-to-emails, cache, %s handlers" n)
+                                         :benchmark test-event-to-emails
+                                         :setup setup-benchmark})
+                   cache-size (mm/measure rems.text/cached-tr)]]
+         #_(prof/profile (dotimes [_ 100] (test-event-to-emails)))
+         (swap! all-stats update :cached (fnil conj []) (merge stats {:handler-count n
+                                                                      :cache-size cache-size})))))
+
+    ;; XXX: potentially very slow and interesting only for comparison with cached version (default).
+    ;; translations are cached using taoensso.tempura/new-tr-fn because (big & nested) dictionary compilation is expensive.
+    ;; Execution time mean : 870,499348 ms (2019 Macbook Pro with 2,3 GHz 8-Core Intel Core i9)
+    (let [get-cached-tr rems.tempura/get-cached-tr]
+      (with-redefs [rems.db.user-settings/get-user-settings (fn [& _] {:language (rand-nth [:en :fi :sv])})
+                    rems.tempura/get-cached-tr #(get-cached-tr % {:cache-dict? false})]
+        (doall
+         (for [n benchmark-handlers
+               :let [test-event-to-emails (get-benchmark-fn n)
+                     stats (run-benchmark {:name (format "event-to-emails, disabled cache, %s handlers" n)
+                                           :benchmark test-event-to-emails
+                                           :setup setup-benchmark})]]
+           #_(prof/profile (dotimes [_ 10] (test-event-to-emails)))
+           (swap! all-stats update :disabled (fnil conj []) (merge stats {:handler-count n}))))))
+    @all-stats))
+
+(defn- print-template-performance-tables [all-stats]
+  (let [format-time #(apply criterium/format-value % (criterium/scale-time %))
+        mean (comp format-time :mean)
+        low (comp format-time :low)
+        high (comp format-time :high)]
+    (when (:cached all-stats)
+      (println "")
+      (println "event-to-emails, cache")
+      (println "---")
+      (doseq [row (rems.markdown/markdown-table
+                   {:header ["handler count" "mean" "lower-q 2.5%" "upper-q 97.5%" "cache size"]
+                    :rows (->> (:cached all-stats)
+                               (mapv (juxt :handler-count mean low high :cache-size)))})]
+        (println row)))
+    (when (:disabled all-stats)
+      (println "")
+      (println "event-to-emails, disabled cache")
+      (println "---")
+      (doseq [row (rems.markdown/markdown-table
+                   {:header ["handler count" "mean" "lower-q 2.5%" "upper-q 97.5%"]
+                    :rows (->> (:disabled all-stats)
+                               (mapv (juxt :handler-count mean low high)))})]
+        (println row)))))
+
 (comment
-  ;; Note: If clj-memory-meter throws InaccessibleObjectException on Java 9+,
-  ;;       you *could* work around it by adding `--add-opens` JVM options, but
-  ;;       the root cause is probably that there is a lazy sequence that could
-  ;;       easily be avoided.
   (benchmark-get-events)
   (benchmark-get-all-applications)
-  (benchmark-get-application))
+  (benchmark-get-application)
+
+  ;; handler count increases email count linearly
+  (def stats (benchmark-template-performance {:benchmark-handlers [1 10 50 100 200]}))
+  ;; additional table formatting
+  (print-template-performance-tables stats)
+
+  (prof/clear-results)
+  (prof/serve-ui 8080))
diff --git a/test/clj/rems/testing_util.clj b/test/clj/rems/testing_util.clj
index 650ac7734..5f5160153 100644
--- a/test/clj/rems/testing_util.clj
+++ b/test/clj/rems/testing_util.clj
@@ -5,7 +5,9 @@
             [rems.db.applications :as applications]
             [rems.db.roles :as roles]
             [rems.db.users :as users]
-            [rems.db.organizations :as organizations])
+            [rems.db.organizations :as organizations]
+            [rems.locales]
+            [rems.text])
   (:import [ch.qos.logback.classic Level Logger]
            [java.nio.file Files]
            [java.nio.file.attribute FileAttribute]
@@ -73,3 +75,10 @@
                                                                                                :description (pr-str v)}))}])]
      ~@body))
 
+(defmacro with-translations [translations & body]
+  `(try
+     (with-redefs [rems.locales/translations ~translations]
+       (rems.text/reset-cached-tr!) ; ensure next call gets new cache
+       ~@body)
+     (finally ; clean up stale cache
+       (rems.text/reset-cached-tr!))))