Repository: CSCfi/rems
Pull request #3206 (2023-10-26 21:17:10)

Relax catalogue create
Something noticed from Swagger API

`:form` key is required here although the value is optional.

![image](https://github.com/CSCfi/rems/assets/823661/20a577bd-00c7-46fc-b45e-d9dcf7f6f2d2)
On top of #3203 

# Checklist for author

Remove items that aren't applicable, check items that are done.

## Reviewability
- [ ] Link to issue
- [x] Note if PR is on top of other PR
- [x] Consider adding screenshots for ease of review

## Backwards compatibility
- [x] API is backwards compatible or completely new

## Documentation
- [x] Update changelog if necessary

## Testing
- [x] Complex logic is unit tested



Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index b31030889..333bc4376 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -10,11 +10,15 @@ Changes since v2.33
 
 **NB: This release contains migrations!**
 **NB: `:application/public` is renamed to `:event/public` (in `application_event` table `eventdata` column).**
+**NB: `:expires-on` is renamed to `:application/expires-on` (in `application_event` table `eventdata` column).**
+**NB: `:last-activity` is redundant and removed from the event (in `application_event` table `eventdata` column).**
 
 ### Changes
 - "Show related events" has been removed from events. (#3156)
 - Handler can now redact other handlers attachments. (#3190)
 - The tables take advantage of a new paging feature. Especially, the handler Actions page and its handled applications table uses paging and only fetches handled applications when so requested. The first 50 rows are only fetched by default. Paging can be also configured per table, if the default is not good (page size 50 rows). (#3191)
+- If reminder email is configured for the, expirer bot, it will delete an old draft application if and only if the reminder email is sent and the specified amount of time has passed. Previously it would just delete if the application was very old (esp. when enabling expiry for the first time).
+- Allow leaving `:form` field away from catalogue item creation API calls. Previously the value was optional but now the key too.
 
 ### Additions
 - (Experimental) Workflow can be configured to enable voting for the approval. Currently all handlers can vote (including bots). Use `:enable-voting`. (#3174)
@@ -22,6 +26,7 @@ Changes since v2.33
 - Added experimental support for named format parameters in translations. (#3183)
 - Added extended logging option (`:enable-extended-logging`), for additionally logging the content of mutating operations. (#3184)
 - Cache reloading can be configured using the new `:buzy-hours` config. REMS will then try to avoid reloading during the specified time spans. (#3194)
+- Application expiration respects `:buzy hours` config too.
 - The handling users can now see whether an event is shown to the applicant from a small eye icon in the event history. (#3156)
 - Event now shows which attachments were redacted when viewing as handling user. (#3190)
 - Workflow has new option to anonymize handling users. When enabled, applying users may only see "Handler" in events where handling user's name would appear. Event emails to applying users similarly show only anonymized name where handling user's name would appear. Application API also returns only anonymized name to applying users. Anonymized name is customizable with translation key `:t.roles/anonymous-handler`, which defaults to `:t.roles/handler`.
diff --git a/resources/migrations/20231026151640-change-expiration-events.edn b/resources/migrations/20231026151640-change-expiration-events.edn
new file mode 100644
index 000000000..759177482
--- /dev/null
+++ b/resources/migrations/20231026151640-change-expiration-events.edn
@@ -0,0 +1,3 @@
+{:ns rems.migrations.change-expiration-events
+ :up-fn migrate-up
+ :down-fn migrate-down}
diff --git a/src/clj/rems/api/catalogue_items.clj b/src/clj/rems/api/catalogue_items.clj
index a836f0feb..caba4b6b8 100644
--- a/src/clj/rems/api/catalogue_items.clj
+++ b/src/clj/rems/api/catalogue_items.clj
@@ -30,7 +30,7 @@
 ;; TODO resid is misleading: it's the internal id, not the string id
 ;; Should we take the string id instead?
 (s/defschema CreateCatalogueItemCommand
-  {:form (s/maybe s/Int)
+  {(s/optional-key :form) (s/maybe s/Int)
    :resid s/Int
    :wfid s/Int
    :organization schema-base/OrganizationId
diff --git a/src/clj/rems/application/commands.clj b/src/clj/rems/application/commands.clj
index 5f6174aca..ad39840be 100644
--- a/src/clj/rems/application/commands.clj
+++ b/src/clj/rems/application/commands.clj
@@ -110,7 +110,6 @@
          (s/optional-key :duo-codes) [schema-base/DuoCode]))
 (s/defschema SendExpirationNotificationsCommand
   (assoc CommandBase
-         :last-activity DateTime
          :expires-on DateTime))
 (s/defschema SubmitCommand
   CommandBase)
@@ -721,8 +720,7 @@
   [cmd _application _injections]
   (or (invalid-expiration-error cmd)
       (ok {:event/type :application.event/expiration-notifications-sent
-           :last-activity (:last-activity cmd)
-           :expires-on (:expires-on cmd)})))
+           :application/expires-on (:expires-on cmd)})))
 
 (defn- add-common-event-fields-from-command [event cmd]
   (-> event
diff --git a/src/clj/rems/application/eraser.clj b/src/clj/rems/application/eraser.clj
index 35b1949af..dac40a0ac 100644
--- a/src/clj/rems/application/eraser.clj
+++ b/src/clj/rems/application/eraser.clj
@@ -25,7 +25,8 @@
   :start (when (:application-expiration env)
            (scheduler/start! "expired-application-poller"
                              process-applications!
-                             (.toStandardDuration (time/days 1))))
+                             (.toStandardDuration (time/hours 1))
+                             (select-keys env [:buzy-hours])))
   :stop (when expired-application-poller
           (scheduler/stop! expired-application-poller)))
 
diff --git a/src/clj/rems/application/events.clj b/src/clj/rems/application/events.clj
index 5dc0e6e50..ea2422673 100644
--- a/src/clj/rems/application/events.clj
+++ b/src/clj/rems/application/events.clj
@@ -84,8 +84,7 @@
 (s/defschema ExpirationNotificationsSentEvent
   (assoc schema-base/EventBase
          :event/type (s/enum :application.event/expiration-notifications-sent)
-         :last-activity DateTime
-         :expires-on DateTime))
+         :application/expires-on DateTime))
 (s/defschema LicensesAcceptedEvent
   (assoc schema-base/EventBase
          :event/type (s/enum :application.event/licenses-accepted)
diff --git a/src/clj/rems/application/expirer_bot.clj b/src/clj/rems/application/expirer_bot.clj
index e555f766e..00f4b49e7 100644
--- a/src/clj/rems/application/expirer_bot.clj
+++ b/src/clj/rems/application/expirer_bot.clj
@@ -5,101 +5,199 @@
    See also: docs/bots.md"
   (:require [clojure.test :refer [deftest testing is]]
             [clj-time.core :as time]
+            [rems.common.application-util :as application-util]
+            [rems.common.util :refer [getx]]
             [rems.config :refer [env]])
   (:import [org.joda.time Period]))
 
 (def bot-userid "expirer-bot")
 
-(defn- get-last-activity [application]
-  (let [last-event (last (:application/events application))]
-    (if (= (:event/type last-event)
-           :application.event/expiration-notifications-sent)
-      (:last-activity last-event) ; because events update :application/last-activity timestamp
-      (:application/last-activity application))))
-
-(defn- expiration-time [application]
-  (when-some [delete-after (get-in env [:application-expiration
-                                        (:application/state application)
-                                        :delete-after])]
-    (-> (get-last-activity application)
-        (time/plus (Period/parse delete-after)))))
-
-(defn- expire-application [application]
-  (when (some-> (expiration-time application)
-                (time/before? (time/now)))
-    {:type :application.command/delete
-     :time (time/now)
-     :actor bot-userid
-     :application-id (:application/id application)}))
+(defn- get-last-event [application]
+  (last (:application/events application)))
 
-(deftest test-expire-application
-  (with-redefs [env {:application-expiration {:application.state/draft {:delete-after "P90D"}
-                                              :application.state/closed {:delete-after "P7D"}}}]
-    (testing "should identify expired application by state"
-      (let [over-90d-ago (time/minus (time/now) (time/days 90) (time/seconds 1))
-            over-7d-ago (time/minus (time/now) (time/days 7) (time/seconds 1))
-            over-1d-ago (time/minus (time/now) (time/days 1) (time/seconds 1))]
-        (is (some? (expire-application {:application/state :application.state/draft
-                                        :application/last-activity over-90d-ago})))
-        (is (not (expire-application {:application/state :application.state/draft
-                                      :application/last-activity over-7d-ago})))
-        (is (some? (expire-application {:application/state :application.state/closed
-                                        :application/last-activity over-7d-ago})))
-        (is (not (expire-application {:application/state :application.state/closed
-                                      :application/last-activity over-1d-ago})))
-        (is (not (expire-application {:application/state :application.state/rejected
-                                      :application/last-activity over-90d-ago})))))))
-
-(defn- reminder-time [application]
-  (let [expiration (get-in env [:application-expiration
-                                (:application/state application)])
-        delete-after (:delete-after expiration)
+(defn- get-last-event-when-notification [application]
+  (let [last-event (get-last-event application)]
+    (when (= (:event/type last-event)
+             :application.event/expiration-notifications-sent)
+      last-event)))
+
+(defn- calculate-expiration-time [expiration application]
+  (when-let [delete-after (:delete-after expiration)]
+    (some-> application
+            application-util/get-last-applying-user-event ; NB: often :application/last-activity is good too
+            (getx :event/time)
+            (time/plus (Period/parse delete-after)))))
+
+(defn- get-expiration-time [application]
+  (some-> application
+          get-last-event-when-notification
+          :application/expires-on))
+
+(defn- is-reminder-configured? [expiration]
+  (:reminder-before expiration))
+
+(defn- expiration-notification-sent?
+  "Has the expiration notification been sent and nothing else has happened since."
+  [expiration application]
+  (some? (get-last-event-when-notification application)))
+
+(defn- calculate-reminder-time [expiration application]
+  (let [delete-after (:delete-after expiration)
         reminder-before (:reminder-before expiration)]
     (when (and delete-after reminder-before)
-      (-> (:application/last-activity application)
+      (-> (:event/time (application-util/get-last-applying-user-event application))
           (time/plus (Period/parse delete-after))
           (time/minus (Period/parse reminder-before))))))
 
-(deftest test-reminder-time
-  (with-redefs [env {:application-expiration {:application.state/draft {:delete-after "P90D"
-                                                                        :reminder-before "P7D"}
-                                              :application.state/closed {:delete-after "P7D"}
-                                              :application.state/rejected {:reminder-before "P7D"}}}]
-    (let [now (time/now)]
-      (testing "should return reminder time when config is valid"
-        (is (time/equal? (time/plus now (time/days 83))
-                         (reminder-time {:application/state :application.state/draft
-                                         :application/last-activity now})))
-        (is (time/equal? now
-                         (reminder-time {:application/state :application.state/draft
-                                         :application/last-activity (time/minus now (time/days 83))})))
-        (is (time/equal? (time/minus now (time/days 1))
-                         (reminder-time {:application/state :application.state/draft
-                                         :application/last-activity (time/minus now (time/days 84))})))
-        (doseq [state [:application.state/closed
-                       :application.state/rejected
-                       :application.state/submitted]]
-          (is (nil? (reminder-time {:application/state state
-                                    :application/last-activity now}))
-              (str state " returns nil because configuration is missing")))))))
-
-(defn- should-send-email [application]
-  (let [last-event-type (-> application :application/events last :event/type)]
-    (and (not= last-event-type
-               :application.event/expiration-notifications-sent)
-         (some-> (reminder-time application)
-                 (time/before? (time/now))))))
-
-(defn- send-expiration-notifications [application]
-  (when (should-send-email application)
+(defn- enough-time-has-passed-since-notification?
+  "Has the user had enough time to react to the reminder?"
+  [expiration application now]
+  (when-let [notification-time (:event/time (get-last-event-when-notification application))]
+    (when-let [reminder-before (:reminder-before expiration)]
+      (time/before? (time/plus notification-time (Period/parse reminder-before))
+                    now))))
+
+(defn- need-to-wait-for-notification? [expiration application now]
+  (and (is-reminder-configured? expiration)
+       (or (not (expiration-notification-sent? expiration application))
+           (not (enough-time-has-passed-since-notification? expiration application now)))))
+
+(defn- expire-application [expiration application now]
+  (when-not (need-to-wait-for-notification? expiration application now)
+    (when-let [expiration-time (or (get-expiration-time application)
+                                   (calculate-expiration-time expiration application))]
+      (when (time/before? expiration-time now)
+        {:type :application.command/delete
+         :time (time/now)
+         :actor bot-userid
+         :application-id (:application/id application)}))))
+
+(deftest test-expire-application
+  (let [now (time/now)
+        over-90d-ago (time/minus now (time/days 90) (time/seconds 2))
+        over-7d-ago (time/minus now (time/days 7) (time/seconds 2))
+        over-1d-ago (time/minus now (time/days 1) (time/seconds 2))]
+    (testing "without reminders"
+      (is (some? (expire-application {:delete-after "P90D"}
+                                     {:application/applicant {:userid "alice"}
+                                      :application/state :application.state/draft
+                                      :application/last-activity over-90d-ago
+                                      :application/events [{:event/type :application.event/created
+                                                            :event/time over-90d-ago
+                                                            :event/actor "alice"}
+                                                           ;; skip because not applying user
+                                                           {:event/type :application.event/remarked
+                                                            :event/time over-1d-ago
+                                                            :event/actor "hannah"}]}
+                                     now)))
+      (is (nil? (expire-application {:delete-after "P90D"}
+                                    {:application/state :application.state/draft
+                                     :application/last-activity over-7d-ago}
+                                    now))))
+
+    (testing "with reminder set"
+      (is (nil? (expire-application {:delete-after "P90D"
+                                     :reminder-before "P7D"}
+                                    {:application/state :application.state/draft
+                                     :application/last-activity over-90d-ago}
+                                    now))
+          "reminder not yet sent")
+
+      (is (nil? (expire-application {:delete-after "P90D"
+                                     :reminder-before "P7D"}
+                                    {:application/state :application.state/draft
+                                     :application/last-activity over-90d-ago
+                                     :application/events [{:type :application.command/send-expiration-notifications
+                                                           :time over-1d-ago
+                                                           :expires-on now}]}
+                                    now))
+          "reminder sent but time not passed")
+      (is (some? (expire-application {:delete-after "P90D"
+                                      :reminder-before "P7D"}
+                                     {:application/state :application.state/draft
+                                      :application/last-activity over-90d-ago
+                                      :application/events [{:event/type :application.event/expiration-notifications-sent
+                                                            :event/time over-7d-ago
+                                                            :application/expires-on now}]}
+                                     (time/plus now (time/seconds 1))))
+          "reminder sent and enough time passed"))
+
+    (testing "if not configured"
+      (is (nil? (expire-application nil
+                                    {:application/state :application.state/draft
+                                     :application/last-activity over-90d-ago}
+                                    now))))))
+
+(deftest test-calculate-reminder-time
+  (let [now (time/now)]
+    (testing "should return reminder time when config is valid"
+      (is (time/equal? (time/plus now (time/days 83))
+                       (calculate-reminder-time {:delete-after "P90D"
+                                                 :reminder-before "P7D"}
+                                                {:application/applicant {:userid "alice"}
+                                                 :application/state :application.state/draft
+                                                 :application/last-activity now
+                                                 :application/events [{:event/type :application.event/created
+                                                                       :event/time now
+                                                                       :event/actor "alice"}]})))
+      (is (time/equal? now
+                       (calculate-reminder-time {:delete-after "P90D"
+                                                 :reminder-before "P7D"}
+                                                {:application/applicant {:userid "alice"}
+                                                 :application/state :application.state/draft
+                                                 :application/last-activity (time/minus now (time/days 83))
+                                                 :application/events [{:event/type :application.event/created
+                                                                       :event/time (time/minus now (time/days 83))
+                                                                       :event/actor "alice"}]})))
+      (is (time/equal? (time/minus now (time/days 1))
+                       (calculate-reminder-time {:delete-after "P90D"
+                                                 :reminder-before "P7D"}
+                                                {:application/applicant {:userid "alice"}
+                                                 :application/state :application.state/draft
+                                                 :application/last-activity (time/minus now (time/days 84))
+                                                 :application/events [{:event/type :application.event/created
+                                                                       :event/time (time/minus now (time/days 84))
+                                                                       :event/actor "alice"}]})))
+      (is (nil? (calculate-reminder-time nil
+                                         {:application/applicant {:userid "alice"}
+                                          :application/state :application.state/draft
+                                          :application/last-activity now
+                                          :application/events [{:event/type :application.event/created
+                                                                :event/time now}]}))
+          "returns nil because configuration is missing")
+
+      (is (nil? (calculate-reminder-time {:delete-after "P90D"}
+                                         {:application/applicant {:userid "alice"}
+                                          :application/state :application.state/draft
+                                          :application/last-activity now
+                                          :application/events [{:event/type :application.event/created
+                                                                :event/time now}]}))
+          "returns nil because configuration is missing")
+
+      (is (nil? (calculate-reminder-time {:delete-after "P90D"
+                                          :reminder-before nil}
+                                         {:application/applicant {:userid "alice"}
+                                          :application/state :application.state/draft
+                                          :application/last-activity now
+                                          :application/events [{:event/type :application.event/created
+                                                                :event/time now}]}))
+          "returns nil because configuration is missing"))))
+
+(defn- should-send-notification-email? [expiration application now]
+  (when-let [reminder-time (calculate-reminder-time expiration application)]
+    (and (not (expiration-notification-sent? expiration application))
+         (time/before? reminder-time now))))
+
+(defn- send-expiration-notifications [expiration application now]
+  (when (should-send-notification-email? expiration application now)
     {:type :application.command/send-expiration-notifications
      :time (time/now)
      :actor bot-userid
      :application-id (:application/id application)
-     :last-activity (:application/last-activity application)
-     :expires-on (expiration-time application)}))
+     :expires-on (calculate-expiration-time expiration application)}))
 
 (defn run-expirer-bot [application]
-  (or (expire-application application)
-      (send-expiration-notifications application)))
-
+  (let [expiration (get-in env [:application-expiration (:application/state application)])
+        now (time/now)]
+    (or (expire-application expiration application now)
+        (send-expiration-notifications expiration application now))))
diff --git a/src/clj/rems/application/process_managers.clj b/src/clj/rems/application/process_managers.clj
index 7a6bddb3a..fd78cee9f 100644
--- a/src/clj/rems/application/process_managers.clj
+++ b/src/clj/rems/application/process_managers.clj
@@ -3,11 +3,12 @@
 
   NB: An event manager should return an empty sequence (or `nil`) if it doesn't create new events itself."
   (:require [clojure.set :refer [difference]]
-            [rems.service.attachment :as attachment]
-            [rems.service.blacklist :as blacklist]
+            [rems.application.expirer-bot :as expirer-bot]
             [rems.common.application-util :as application-util]
             [rems.db.applications :as applications]
-            [rems.db.attachments :as attachments]))
+            [rems.db.attachments :as attachments]
+            [rems.service.attachment :as attachment]
+            [rems.service.blacklist :as blacklist]))
 
 (defn revokes-to-blacklist
   "Revokation causes the users to be blacklisted."
@@ -27,7 +28,9 @@
   [new-events]
   (doseq [event new-events]
     (when (= :application.event/deleted (:event/type event))
-      (applications/delete-application-and-reload-cache! (:application/id event)))))
+      (if (= expirer-bot/bot-userid (:event/actor event)) ; will reload in the end
+        (applications/delete-application! (:application/id event))
+        (applications/delete-application-and-reload-cache! (:application/id event))))))
 
 (defn delete-orphan-attachments [application-id]
   (let [application (applications/get-application-internal application-id)
diff --git a/src/clj/rems/email/template.clj b/src/clj/rems/email/template.clj
index 466e31805..66a7e1251 100644
--- a/src/clj/rems/email/template.clj
+++ b/src/clj/rems/email/template.clj
@@ -1,7 +1,7 @@
 (ns rems.email.template
   (:require [clojure.string :as str]
-            [rems.common.application-util :as application-util]
             [rems.application.model :as model]
+            [rems.common.application-util :as application-util]
             [rems.config :refer [env]]
             [rems.context :as context]
             [rems.db.user-settings :as user-settings]
@@ -263,24 +263,25 @@
 
 (defmethod event-to-emails :application.event/expiration-notifications-sent [event application]
   (vec
-   (for [recipient (application-util/applicant-and-members application)]
-     (with-language (:language (user-settings/get-user-settings (:userid recipient)))
-       {:to-user (:userid recipient)
-        :subject (text-format :t.email.application-expiration-notification/subject-to-member
+   (let [last-activity (:event/time (application-util/get-last-applying-user-event application))]
+     (for [recipient (application-util/applicant-and-members application)]
+       (with-language (:language (user-settings/get-user-settings (:userid recipient)))
+         {:to-user (:userid recipient)
+          :subject (text-format :t.email.application-expiration-notification/subject-to-member
+                                (application-util/get-member-name recipient)
+                                (format-application-for-email application)
+                                (localize-utc-date last-activity)
+                                (localize-utc-date (:application/expires-on event))
+                                (link-to-application (:application/id application)))
+          :body (str
+                 (text-format :t.email.application-expiration-notification/message-to-member
                               (application-util/get-member-name recipient)
                               (format-application-for-email application)
-                              (localize-utc-date (:last-activity event))
-                              (localize-utc-date (:expires-on event))
+                              (localize-utc-date last-activity)
+                              (localize-utc-date (:application/expires-on event))
                               (link-to-application (:application/id application)))
-        :body (str
-               (text-format :t.email.application-expiration-notification/message-to-member
-                            (application-util/get-member-name recipient)
-                            (format-application-for-email application)
-                            (localize-utc-date (:last-activity event))
-                            (localize-utc-date (:expires-on event))
-                            (link-to-application (:application/id application)))
-               (text :t.email/regards)
-               (text :t.email/footer))}))))
+                 (text :t.email/regards)
+                 (text :t.email/footer))})))))
 
 ;; TODO member-joined?
 
diff --git a/src/clj/rems/migrations/change_expiration_events.clj b/src/clj/rems/migrations/change_expiration_events.clj
new file mode 100644
index 000000000..46822cb45
--- /dev/null
+++ b/src/clj/rems/migrations/change_expiration_events.clj
@@ -0,0 +1,58 @@
+(ns ^{:added "2.34"} rems.migrations.change-expiration-events
+  "Migrates event `:expires-on` to `:application/expires-on`
+  and removes the `:last-modified`."
+  (:require [clojure.set]
+            [hugsql.core :as hugsql]
+            [medley.core :refer [assoc-some]]
+            [rems.db.applications]
+            [rems.db.core]
+            [rems.json :as json]))
+
+(def migration-id 20231026151640)
+
+(declare get-events set-event!)
+(hugsql/def-db-fns-from-string
+  "
+-- :name get-events :? :*
+SELECT id, eventdata::TEXT
+FROM application_event;
+
+-- :name set-event! :!
+UPDATE application_event
+SET eventdata = :eventdata::jsonb
+WHERE id = :id;
+")
+
+(defn attributes-up [eventdata]
+  (-> eventdata
+      (assoc-some :application/expires-on (:expires-on eventdata))
+      (dissoc :last-activity)))
+
+(defn attributes-down [eventdata]
+  (-> eventdata
+      (assoc-some :expires-on (:application/expires-on eventdata))
+      (assoc :last-activity (:event/time eventdata)))) ; NB: not the same but it would be difficult to restore
+
+(defn migrate-event [event migrator]
+  (update event :eventdata migrator))
+
+(def event-from-db #(update % :eventdata json/parse-string))
+
+(defn migrate-events! [conn xf migrator]
+  (doseq [event (sequence (comp (map event-from-db) xf) (get-events conn))
+          :let [new-event (migrate-event event migrator)]]
+    (set-event! conn
+                (update new-event :eventdata json/generate-string))))
+
+(def old-event? #(boolean? (get-in % [:eventdata :expires-on])))
+(def new-event? #(boolean? (get-in % [:eventdata :application/expires-on])))
+
+(defn migrate-up [{:keys [conn]}]
+  (migrate-events! conn (filter old-event?) attributes-up))
+
+(defn migrate-down [{:keys [conn]}]
+  (migrate-events! conn (filter new-event?) attributes-down))
+
+(comment
+  (migrate-up {:conn rems.db.core/*db*})
+  (migrate-down {:conn rems.db.core/*db*}))
diff --git a/src/clj/rems/service/catalogue.clj b/src/clj/rems/service/catalogue.clj
index d611b85c4..8f2e3249d 100644
--- a/src/clj/rems/service/catalogue.clj
+++ b/src/clj/rems/service/catalogue.clj
@@ -13,7 +13,8 @@
 ;; TODO move catalogue item localizations into the catalogueitemdata
 (defn create-catalogue-item! [{:keys [localizations organization] :as command}]
   (util/check-allowed-organization! organization)
-  (let [id (:id (db/create-catalogue-item! (merge {:organization (:organization/id organization "default")}
+  (let [id (:id (db/create-catalogue-item! (merge {:form nil} ; SQL requires, API doesn't
+                                                  {:organization (:organization/id organization "default")}
                                                   (select-keys command [:form :resid :wfid :enabled :archived :start])
                                                   {:catalogueitemdata (catalogue/catalogueitemdata->json command)})))
         loc-ids
diff --git a/src/cljc/rems/common/application_util.cljc b/src/cljc/rems/common/application_util.cljc
index 59652b32b..8b68b35c6 100644
--- a/src/cljc/rems/common/application_util.cljc
+++ b/src/cljc/rems/common/application_util.cljc
@@ -1,6 +1,7 @@
 (ns rems.common.application-util
   (:require [clojure.set]
             [clojure.test :refer [deftest is testing]]
+            [medley.core :refer [find-first]]
             [rems.common.util :as util]))
 
 (def states
@@ -46,6 +47,11 @@
 (def +applying-user-roles+ #{:applicant :member})
 (def +handling-user-roles+ #{:handler :reviewer :decider :past-reviewer :past-decider})
 
+;; TODO combine these functions
+(defn is-applying-userid? [application userid]
+  (contains? (set (map :userid (applicant-and-members application)))
+             userid))
+
 (defn is-applying-user?
   "Returns true if current user is applying for `application`.
 
@@ -138,3 +144,9 @@
     (let [applicant-attachment {:attachment/user {:userid "alice"}}]
       (is (not (can-redact-attachment? applicant-attachment #{} "alice")))
       (is (not (can-redact-attachment? applicant-attachment #{:handler} "handler"))))))
+
+(defn get-last-applying-user-event [application]
+  (->> application
+       :application/events
+       reverse
+       (find-first #(is-applying-userid? application (:event/actor %)))))


Test patch:
diff --git a/test/clj/rems/api/test_catalogue_items.clj b/test/clj/rems/api/test_catalogue_items.clj
index ea86d6dc5..78c373f47 100644
--- a/test/clj/rems/api/test_catalogue_items.clj
+++ b/test/clj/rems/api/test_catalogue_items.clj
@@ -86,6 +86,23 @@
                               :localizations {}}
                              +test-api-key+ "owner")]
           (is (:success data))
+          (testing "and fetch"
+            (is (= {:formid nil
+                    :form-name nil}
+                   (select-keys
+                    (api-call :get (str "/api/catalogue-items/" (:id data)) nil
+                              +test-api-key+ "owner")
+                    [:formid :form-name])))))
+
+        (let [data (api-call :post "/api/catalogue-items/create"
+                             {;; no :form necessary
+                              :resid res-id
+                              :wfid wf-id
+                              :organization {:organization/id "organization1"}
+                              :archived false
+                              :localizations {}}
+                             +test-api-key+ "owner")]
+          (is (:success data))
           (testing "and fetch"
             (is (= {:formid nil
                     :form-name nil}
diff --git a/test/clj/rems/application/test_eraser.clj b/test/clj/rems/application/test_eraser.clj
index 3b2fc8691..d193923b3 100644
--- a/test/clj/rems/application/test_eraser.clj
+++ b/test/clj/rems/application/test_eraser.clj
@@ -185,8 +185,8 @@
               (testing "expiration notifications sent event was created for draft expiring in 6 days"
                 (let [events (expiration-notification-events draft-expires-in-6d)]
                   (is (= 1 (count events)))
-                  (is (time/equal? (time/plus now (time/days 6)) (:expires-on (first events))))
-                  (is (time/equal? (time/minus now (time/days 84)) (:last-activity (first events))))
+                  (is (time/equal? (time/plus now (time/days 6)) (:application/expires-on (first events))))
+
                   (is (time/equal? now (:event-time (first events))))))
               (is (= [{:outbox/deadline (time/date-time 2022)
                        :outbox/email {:subject (str "Your unsubmitted application " draft-expires-in-6d " will be deleted soon")
@@ -217,7 +217,5 @@
                 (testing "expiration notifications events have not changed for draft expiring in 6 days"
                   (let [events (expiration-notification-events draft-expires-in-6d)]
                     (is (= 1 (count events)))
-                    (is (time/equal? (time/plus now (time/days 6)) (:expires-on (first events))))
-                    (is (time/equal? (time/minus now (time/days 84)) (:last-activity (first events))))
+                    (is (time/equal? (time/plus now (time/days 6)) (:application/expires-on (first events))))
                     (is (time/equal? now (:event-time (first events))))))))))))))
-
diff --git a/test/clj/rems/email/test_template.clj b/test/clj/rems/email/test_template.clj
index 459de900d..b939b4b05 100644
--- a/test/clj/rems/email/test_template.clj
+++ b/test/clj/rems/email/test_template.clj
@@ -3,11 +3,11 @@
             [clj-time.core :as time]
             [mount.core :as mount]
             [rems.application.model :as model]
+            [rems.common.application-util :as application-util]
             [rems.config]
             [rems.db.user-settings :as user-settings]
             [rems.email.template :as template]
-            [rems.locales]
-            [rems.testing-util :refer [with-fixed-time]]))
+            [rems.locales]))
 
 (defn empty-footer [f]
   (with-redefs [rems.locales/translations (assoc-in rems.locales/translations [:en :t :email :footer] "")]
@@ -580,32 +580,32 @@
              (email-to "applicant" mails))))))
 
 (deftest test-expiration-notifications-sent-email
-  (with-redefs [rems.config/env (assoc rems.config/env :application-expiration {:application.state/draft {:delete-after "P90D"
-                                                                                                          :reminder-before "P7D"}})]
-    (let [add-member {:application/id 7
-                      :event/type :application.event/member-added
-                      :event/actor "handler"
-                      :application/member {:userid "member"}}
-          member-events (conj created-events add-member)
-          notifications-sent (with-fixed-time (time/date-time 2023 6 19)
-                               {:application/id 7
+  (let [now (time/date-time 2023 6 19)]
+    (with-redefs [rems.config/env (assoc rems.config/env :application-expiration {:application.state/draft {:delete-after "P90D"
+                                                                                                            :reminder-before "P7D"}})
+                  application-util/get-last-applying-user-event (constantly {:event/time (time/minus now (time/days 83))})]
+      (let [add-member {:application/id 7
+                        :event/type :application.event/member-added
+                        :event/actor "handler"
+                        :application/member {:userid "member"}}
+            member-events (conj created-events add-member)
+            notifications-sent {:application/id 7
                                 :event/type :application.event/expiration-notifications-sent
-                                :expires-on (time/plus (time/now) (time/days 7))
-                                :last-activity (time/minus (time/now) (time/days 83))})]
-      (is (= [{:to-user "applicant"
-               :subject "Your unsubmitted application 2001/3, \"Application title\" will be deleted soon"
-               :body "Dear Alice Applicant,\n\nYour unsubmitted application has been inactive since 2023-03-28 and it will be deleted after 2023-06-26, if it is not edited.\n\nYou can view and edit the application at http://example.com/application/7"}
-              {:to-user "member"
-               :subject "Your unsubmitted application 2001/3, \"Application title\" will be deleted soon"
-               :body "Dear member,\n\nYour unsubmitted application has been inactive since 2023-03-28 and it will be deleted after 2023-06-26, if it is not edited.\n\nYou can view and edit the application at http://example.com/application/7"}]
-             (emails member-events notifications-sent)))
-      (with-redefs [rems.locales/translations (assoc-in rems.locales/translations
-                                                        [:fi :t :email :application-expiration-notification :message-to-member]
-                                                        "Lähettämätön hakemuksesi %5 poistetaan automaattisesti %4 jälkeen, jos sitä ei muokata.")]
+                                :application/expires-on (time/plus now (time/days 7))}]
         (is (= [{:to-user "applicant"
-                 :subject "Lähettämätön hakemuksesi 2001/3, \"Application title\" poistetaan pian"
-                 :body "Lähettämätön hakemuksesi http://example.com/application/7 poistetaan automaattisesti 2023-06-26 jälkeen, jos sitä ei muokata.\n\nYstävällisin terveisin,\n\nREMS\n\n\nTämä on automaattinen viesti. Älä vastaa."}
+                 :subject "Your unsubmitted application 2001/3, \"Application title\" will be deleted soon"
+                 :body "Dear Alice Applicant,\n\nYour unsubmitted application has been inactive since 2023-03-28 and it will be deleted after 2023-06-26, if it is not edited.\n\nYou can view and edit the application at http://example.com/application/7"}
                 {:to-user "member"
-                 :subject "Lähettämätön hakemuksesi 2001/3, \"Application title\" poistetaan pian"
-                 :body "Lähettämätön hakemuksesi http://example.com/application/7 poistetaan automaattisesti 2023-06-26 jälkeen, jos sitä ei muokata.\n\nYstävällisin terveisin,\n\nREMS\n\n\nTämä on automaattinen viesti. Älä vastaa."}]
-               (emails :fi member-events notifications-sent)))))))
+                 :subject "Your unsubmitted application 2001/3, \"Application title\" will be deleted soon"
+                 :body "Dear member,\n\nYour unsubmitted application has been inactive since 2023-03-28 and it will be deleted after 2023-06-26, if it is not edited.\n\nYou can view and edit the application at http://example.com/application/7"}]
+               (emails member-events notifications-sent)))
+        (with-redefs [rems.locales/translations (assoc-in rems.locales/translations
+                                                          [:fi :t :email :application-expiration-notification :message-to-member]
+                                                          "Lähettämätön hakemuksesi %5 poistetaan automaattisesti %4 jälkeen, jos sitä ei muokata.")]
+          (is (= [{:to-user "applicant"
+                   :subject "Lähettämätön hakemuksesi 2001/3, \"Application title\" poistetaan pian"
+                   :body "Lähettämätön hakemuksesi http://example.com/application/7 poistetaan automaattisesti 2023-06-26 jälkeen, jos sitä ei muokata.\n\nYstävällisin terveisin,\n\nREMS\n\n\nTämä on automaattinen viesti. Älä vastaa."}
+                  {:to-user "member"
+                   :subject "Lähettämätön hakemuksesi 2001/3, \"Application title\" poistetaan pian"
+                   :body "Lähettämätön hakemuksesi http://example.com/application/7 poistetaan automaattisesti 2023-06-26 jälkeen, jos sitä ei muokata.\n\nYstävällisin terveisin,\n\nREMS\n\n\nTämä on automaattinen viesti. Älä vastaa."}]
+                 (emails :fi member-events notifications-sent))))))))