Repository: doomeer/kalandralang
Pull request #35: feat: add mod type to correctly compute tiers
Author: login: Fuco1
Created: 2025-04-19T00:08:48Z | Merged: 2025-04-22T19:29:56Z

- [x] add loading of type to the mod
- [x] use the mod type instead of mod groups to determine tiers

fixes #34


Linked issue #34: Tiers are not calculated correctly
Because multiple mods can share the "family", you sometimes create "tier 22" which clearly never exists.  This can happen for example for armors with defense mods which are shared (flat es, % es and hybrids).

Similarly I think the `tier "Strength"` instruction won't work correctly because this is not the mod group but mod family, at least that's how I understand it from the code.

(side issue, but it would be really great to be able to write T1 mana instead of IncreasedMana12 or 13 since the max tier is different "rank" based on the item... they did this in PoE 2 and it's so stupid)

![Image](https://github.com/user-attachments/assets/bc58cf76-8631-49e5-9245-3c7109bea069)


kalandralang output

```
--------
Sadist Garb (Rare)
--------
(prefix) [T11] +9 to maximum Energy Shield
               +18 to maximum Life
(prefix) [T3] +106 to maximum Life
(prefix) [T1] Reflects 36 Physical Damage to Melee Attackers

(suffix) [T7] +18 to Dexterity
--------
Cost:
Total: 0.00div (0c)
```


recipe

```
buy "Metadata/Items/Armours/BodyArmours/BodyDexInt16"
  with "LocalBaseEnergyShieldAndLife1"
```
Linked issue comments: user: login: Fuco1
created_at: 2025-04-18T23:37:43Z
body: I think these properties identify the "mod group" of just that particular mod.  It is important to check prefix / suffix / unique (there's a ton of unique mods which we don't want) and also domain = item, otherwise it would include crafted mods.

![Image](https://github.com/user-attachments/assets/002250fa-48cf-460c-bcd6-40ad942a618e)
author_association: CONTRIBUTOR
user: login: doomeer
created_at: 2025-04-19T16:45:06Z
body: Very interesting, thanks!

It's been a while so I don't remember why I chose not to use `"type"`. Maybe the field did not exist yet, or maybe I just overlooked it — it was a lot of guesswork to try to understand the data.

> (side issue, but it would be really great to be able to write T1 mana instead of IncreasedMana12 or 13 since the max tier is different "rank" based on the item... they did this in PoE 2 and it's so stupid)

Maybe I am misunderstanding, but you should already be able to do this (see https://doomeer.github.io/kalandralang/doc/manual.html#modifier-tiers). It probably suffers from the same issue that you raised, but in my experience it is usually fine when trying to only keep items with the best tiers (like T1 to T3).
author_association: OWNER
user: login: Fuco1
created_at: 2025-04-19T19:05:36Z
body: The problem is that `tier "BaseLocalDefencesAndLife"` is actually three different modifiers (ev + life, es + life and somewhat weirdly es + mana), and since the ordering is total only one of these mods (which?) will be tier 1.

I think this will have to be changed to behave in a way that is useful and it will probably be backwards incompatible change.  In my PR #35 I've so far added the mod type ID and I'm going to try to change the tier to use that instead of the groups.  Do you want me to continue or do you have some points to discuss?

> Maybe the field did not exist yet, or maybe I just overlooked it — it was a lot of guesswork to try to understand the data.

I'm not sure but I think before the groups were at least unique on a modifier... now one mod can also has 3 different "families" and then it excludes all of them.  Mostly a lot of the combo veiled/delve/temple mods use this.

author_association: CONTRIBUTOR
user: login: doomeer
created_at: 2025-04-19T19:41:25Z
body: It's ok to make a backward-incompatible change here, the current behavior is bad anyway.

> Do you want me to continue or do you have some points to discuss?

I would be very happy to see you continue! I knew I didn't get it right for all mods but I didn't take the time to look into it. You seem to have a better idea of what to do than me ;)
author_association: OWNER

Diff:
diff --git a/src/cache.ml b/src/cache.ml
index adffce7..a700da5 100644
--- a/src/cache.ml
+++ b/src/cache.ml
@@ -1,4 +1,4 @@
-let version = "KKASH006"
+let version = "KKASH007"
 
 exception Wrong_version
 
@@ -205,10 +205,11 @@ let i_mod_stat i: Mod.stat =
   { id; min; max }
 
 let o_mod o (
-    { id; domain; generation_type; groups; required_level; spawn_weights;
+    { id; mod_type; domain; generation_type; groups; required_level; spawn_weights;
       generation_weights; tags; adds_tags; stats; is_essence_only }: Mod.t
   ) =
   o_id o id;
+  o_id o mod_type;
   o_base_item_domain o domain;
   o_mod_generation_type o generation_type;
   o_id_set o groups;
@@ -222,6 +223,7 @@ let o_mod o (
 
 let i_mod i: Mod.t =
   let id = i_id i in
+  let mod_type = i_id i in
   let domain = i_base_item_domain i in
   let generation_type = i_mod_generation_type i in
   let groups = i_id_set i in
@@ -232,7 +234,7 @@ let i_mod i: Mod.t =
   let adds_tags = i_id_set i in
   let stats = i_list i_mod_stat i in
   let is_essence_only = i_bool i in
-  { id; domain; generation_type; groups; required_level; spawn_weights;
+  { id; mod_type; domain; generation_type; groups; required_level; spawn_weights;
     generation_weights; tags; adds_tags; stats; is_essence_only }
 
 let o_stat_translation_condition o ({ min; max }: Stat_translation.condition) =
diff --git a/src/check.ml b/src/check.ml
index c30c55c..ae3e59c 100644
--- a/src/check.ml
+++ b/src/check.ml
@@ -56,8 +56,16 @@ let rec check_arithmetic_expression ({ node; loc }: AST.arithmetic_expression) =
     | Binary_arithmetic_operator (a, _, b) ->
         check_arithmetic_expression a;
         check_arithmetic_expression b
-    | Tier mod_group ->
-        check_mod_group loc mod_group
+    | Tier mod_type ->
+        (* Mod groups are replaced with mod types.  If the mod type does not
+           exist, check if the identifier is not a mod group, in which case warn
+           the user that mod groups are no longer supported and that they need
+           to switch *)
+       if not (Mod.mod_type_exists mod_type) then
+         if (Mod.group_exists mod_type) then
+           error loc "you are using modifier group: %S.  Modifier groups were replaced by modifier type." (Id.show mod_type)
+         else
+           error loc "unknown modifier type: %S" (Id.show mod_type)
     | Int_of_bool condition ->
         check_condition condition
 
diff --git a/src/interpreter.ml b/src/interpreter.ml
index 6a107be..dace992 100644
--- a/src/interpreter.ml
+++ b/src/interpreter.ml
@@ -201,20 +201,20 @@ let rec eval_arithmetic_expression state (expression: AST.arithmetic_expression)
     | Affix_count ->
         with_item state @@ fun item ->
         Q.of_int (Item.affix_count item)
-    | Tier mod_group ->
+    | Tier mod_type ->
         with_item state @@ fun item ->
         (
-          let has_mod_group { Item.modifier; fractured = _ } =
+          let has_mod_type { Item.modifier; fractured = _ } =
             match modifier.generation_type with
               | Prefix | Suffix ->
-                  Id.Set.mem mod_group modifier.groups
+                  Id.compare mod_type modifier.mod_type = 0
               | Exarch_implicit _ | Eater_implicit _ ->
                   (* Those could have the same mod group as a prefix or suffix,
                      and we wouldn't know what to do. So currently "tier" only
                      supports prefixes and suffixes. *)
                   false
           in
-          match List.filter has_mod_group item.mods with
+          match List.filter has_mod_type item.mods with
             | [] ->
                 (* Item has no modifier of this group. *)
                 Q.of_int 999
@@ -222,14 +222,14 @@ let rec eval_arithmetic_expression state (expression: AST.arithmetic_expression)
                 (
                   match Item.mod_tier item modifier with
                     | None ->
-                        fail "don't know how to compute tier for %S in group %S"
-                          (Id.show modifier.id) (Id.show mod_group)
+                        fail "don't know how to compute tier for %S in type %S"
+                          (Id.show modifier.id) (Id.show mod_type)
                     | Some tier ->
                         Q.of_int tier
                 )
             | _ :: _ :: _ ->
                 (* Items are not supposed to have several modifiers of the same group?? *)
-                fail "item has multiple affixes for mod group: %S" (Id.show mod_group)
+                fail "item has multiple affixes for mod type: %S" (Id.show mod_type)
         )
     | Int_of_bool condition ->
         if eval_condition state condition then
diff --git a/src/item.ml b/src/item.ml
index a0e284d..26c3c05 100644
--- a/src/item.ml
+++ b/src/item.ml
@@ -230,12 +230,12 @@ let full_mod_pool = memoize @@ fun (domain, item_tags, keep_weight_0_tag) ->
   in
   List.filter_map can_spawn_mod !Mod.pool
 
-let sort_mod_group = memoize @@ fun (domain, item_tags, mod_groups, keep_weight_0_tag) ->
+let sort_tier_group = memoize @@ fun (domain, item_tags, mod_type, keep_weight_0_tag) ->
   let pool =
-    let has_mod_group (_, modifier) =
+    let has_mod_type (_, modifier) =
       match modifier.Mod.generation_type with
         | Prefix | Suffix ->
-            if not (Id.Set.disjoint mod_groups modifier.Mod.groups) then
+            if Id.compare mod_type modifier.Mod.mod_type = 0 then
               Some modifier
             else
               None
@@ -244,7 +244,7 @@ let sort_mod_group = memoize @@ fun (domain, item_tags, mod_groups, keep_weight_
             None
     in
     full_mod_pool (domain, item_tags, keep_weight_0_tag)
-    |> List.filter_map has_mod_group
+    |> List.filter_map has_mod_type
   in
   (* [mod2] and [mod1] are reversed because we want to sort in reverse order. *)
   let by_ilvl_or_stat (mod2: Mod.t) (mod1: Mod.t) =
@@ -262,8 +262,8 @@ let sort_mod_group = memoize @@ fun (domain, item_tags, mod_groups, keep_weight_
   List.sort by_ilvl_or_stat pool |> list_group by_ilvl_or_stat
 
 let mod_tier =
-  let mod_tier_memoized = memoize @@ fun (domain, item_tags, mod_groups, mod_id) ->
-    let sorted_group = sort_mod_group (domain, item_tags, mod_groups, None) in
+  let mod_tier_memoized = memoize @@ fun (domain, item_tags, mod_type, mod_id) ->
+    let sorted_group = sort_tier_group (domain, item_tags, mod_type, None) in
     let rec find_mod tier = function
       | [] ->
           (* echo "cannot find mod %s" (Id.show mod_id); *)
@@ -299,7 +299,7 @@ let mod_tier =
     if modifier.is_essence_only then
       essence_mod_tier modifier.id
     else
-      mod_tier_memoized (modifier.domain, base_tags item, modifier.groups, modifier.id)
+      mod_tier_memoized (modifier.domain, base_tags item, modifier.mod_type, modifier.id)
 
 let show_modifier item { modifier; fractured } =
   let tier = mod_tier item modifier in
@@ -1011,10 +1011,10 @@ let apply_orb_of_dominance item =
               let has_weight (tag, _) = Id.compare tag influence_tag = 0 in
               if List.exists has_weight modifier.spawn_weights then
                 let mod_group =
-                  sort_mod_group (
+                  sort_tier_group (
                     modifier.domain,
                     base_tags item,
-                    modifier.groups,
+                    modifier.mod_type,
                     Some influence_tag
                   )
                 in
diff --git a/src/mod.ml b/src/mod.ml
index 13d0889..4b879fb 100644
--- a/src/mod.ml
+++ b/src/mod.ml
@@ -54,6 +54,7 @@ type stat =
 type t =
   {
     id: Id.t;
+    mod_type: Id.t; (* "type" in data file *)
     domain: Base_item.domain;
     generation_type: generation_type;
     (* Only one mod per [group] can spawn on an item. *)
@@ -145,12 +146,13 @@ let pp_stat { id; min; max } =
   ]
 
 let pp {
-    id; domain; generation_type; groups; required_level;
+    id; mod_type; domain; generation_type; groups; required_level;
     spawn_weights; generation_weights;
     tags; adds_tags; stats; is_essence_only;
   } =
   Pretext.OCaml.record [
     "id", Id.pp id;
+    "type", Id.pp mod_type;
     "domain", Base_item.pp_domain domain;
     "generation_type", pp_generation_type generation_type;
     "groups", Id.Set.pp groups;
@@ -166,6 +168,7 @@ let pp {
 let pool = ref []
 let id_map = ref Id.Map.empty
 let mod_groups = ref Id.Set.empty
+let mod_types = ref Id.Set.empty
 
 type data = t list
 
@@ -175,7 +178,8 @@ let import (x: data) =
   pool := x;
   let add modifier =
     id_map := Id.Map.add modifier.id modifier !id_map;
-    mod_groups := Id.Set.union modifier.groups !mod_groups
+    mod_groups := Id.Set.union modifier.groups !mod_groups;
+    mod_types := Id.Set.add modifier.mod_type !mod_types
   in
   List.iter add x
 
@@ -217,6 +221,7 @@ let load filename =
   let add_entry (id, json) =
     if id =~! royale_rex then
       let id = Id.make id in
+      let mod_type = ref Id.empty in
       let domain = ref None in
       let generation_type = ref None in
       let groups = ref Id.Set.empty in
@@ -255,6 +260,8 @@ let load filename =
                   !groups
           | "name" ->
               name := JSON.as_id value
+          | "type" ->
+              mod_type := JSON.as_id value
           | "required_level" ->
               required_level := JSON.as_int value
           | "spawn_weights" ->
@@ -340,6 +347,7 @@ let load filename =
                     let modifier =
                       {
                         id;
+                        mod_type = !mod_type;
                         domain;
                         generation_type;
                         groups = !groups;
@@ -358,7 +366,8 @@ let load filename =
                       | None ->
                           pool := modifier :: !pool;
                           id_map := Id.Map.add id modifier !id_map;
-                          mod_groups := Id.Set.union modifier.groups !mod_groups
+                          mod_groups := Id.Set.union modifier.groups !mod_groups;
+                          mod_types := Id.Set.add modifier.mod_type !mod_types
   in
   List.iter add_entry JSON.(parse_file filename |> as_object)
 
@@ -375,6 +384,9 @@ let by_id id =
 let group_exists id =
   Id.Set.mem id !mod_groups
 
+let mod_type_exists id =
+  Id.Set.mem id !mod_types
+
 type show_mode =
   | With_placeholders
   | With_ranges


Reviews:
user: login: doomeer
state: COMMENTED
author_association: OWNER
submitted_at: 2025-04-21T09:36:31Z
user: login: Fuco1
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-22T15:18:39Z

Review comments:
diff_hunk: @@ -230,12 +230,12 @@ let full_mod_pool = memoize @@ fun (domain, item_tags, keep_weight_0_tag) ->
   in
   List.filter_map can_spawn_mod !Mod.pool
 
-let sort_mod_group = memoize @@ fun (domain, item_tags, mod_groups, keep_weight_0_tag) ->
+let sort_mod_group = memoize @@ fun (domain, item_tags, mod_type, keep_weight_0_tag) ->
path: src/item.ml
user: login: doomeer
body: Maybe the function should be renamed as well since it now sorts mods for a given mod type, instead of mods of a given group?
created_at: 2025-04-21T09:36:31Z
author_association: OWNER
diff_hunk: @@ -230,12 +230,12 @@ let full_mod_pool = memoize @@ fun (domain, item_tags, keep_weight_0_tag) ->
   in
   List.filter_map can_spawn_mod !Mod.pool
 
-let sort_mod_group = memoize @@ fun (domain, item_tags, mod_groups, keep_weight_0_tag) ->
+let sort_mod_group = memoize @@ fun (domain, item_tags, mod_type, keep_weight_0_tag) ->
path: src/item.ml
user: login: Fuco1
body: I renamed this function to `sort_tier_group`
created_at: 2025-04-22T15:18:38Z
author_association: CONTRIBUTOR

PR comments:
user: login: doomeer
created_at: 2025-04-19T16:37:11Z
body: The code to load the mod type looks good to me, however this changes the format of the cache, which will result in a confusing error for users who already have a cache. This is easy to fix, you just need to change this line at the top of the file:
```
let version = "KKASH006"
```
to this:
```
let version = "KKASH007"
```
author_association: OWNER
user: login: Fuco1
created_at: 2025-04-19T19:13:21Z
body: > The code to load the mod type looks good to me, however this changes the format of the cache, which will result in a confusing error for users who already have a cache. This is easy to fix, you just need to change this line at the top of the file:
> 
> ```
> let version = "KKASH006"
> ```
> 
> to this:
> 
> ```
> let version = "KKASH007"
> ```

Thanks, updated and force-pushed.
author_association: CONTRIBUTOR
user: login: Fuco1
created_at: 2025-04-20T19:39:06Z
body: I'm not quite sure about calling this "type".  That's what it's called in the data, but it doesn't really reflect the idea to me.  When I talk about the "+ to max mana" modifiers I say "group" for all of them.  What we call "groups" is called "family" on poedb.tw and in dat-schema schemas (IDK how GGG calls it).

Would it be worth to reword this?  The "family" is the family of mutually exclusive "groups", so that you can only have one mod on an item from all the groups of all the families it's part of.  Each mod can have multiple families effectively blocking all of them.  The "tier" would be the modifier tier within the group (what is now called "type" in the data).

![image](https://github.com/user-attachments/assets/567f43cd-a537-4994-9966-47d09da3ff3b)

author_association: CONTRIBUTOR
user: login: Fuco1
created_at: 2025-04-21T00:36:46Z
body: I've implemented the logic with the old nomenclature
author_association: CONTRIBUTOR
user: login: doomeer
created_at: 2025-04-21T09:53:16Z
body: > I'm not quite sure about calling this "type".

Also worth noting is that Craft of Exile calls the groups of exclusive mod "Modifier Group":
![image](https://github.com/user-attachments/assets/30bdefa9-1bde-478a-8646-a876334b1cd0)

I think that in the code, I would use the following names:
- `exclusivity_group` (or `exclusive_group`) for the one where an item can only have one mod of the group
- `tier_group` for the one that is used for tiers

I think this should disambiguate rather nicely. I would also use these names in the results of the `find` command.

In other places in the UI, I guess I would write "group" when there is no ambiguity, and specify "exclusivity group" or "tier group" when needed. For instance the error message that you added would become:
```
you are using exclusivity group: %S. Exclusivity groups were replaced by tier groups
```
author_association: OWNER
user: login: Fuco1
created_at: 2025-04-21T13:16:10Z
body: That sounds good actually.  If you don't mind I would finish this PR and then do the renaming in another one to keep the change sets reasonable
author_association: CONTRIBUTOR
user: login: doomeer
created_at: 2025-04-21T20:24:29Z
body: Sure, we can merge without the renaming :+1: 
author_association: OWNER
user: login: Fuco1
created_at: 2025-04-22T15:19:15Z
body: I rebased this on the latest and I think it's ready to merge.
author_association: CONTRIBUTOR
user: login: doomeer
created_at: 2025-04-22T19:29:52Z
body: Code looks good. Tested with the recipe from the issue:
```
buy "Metadata/Items/Armours/BodyArmours/BodyDexInt16"
  with "LocalBaseEnergyShieldAndLife1"
```
It shows T11 on `master` and `T4` on this branch, which is what we expect according to the screenshot in the issue. I also ran a couple other recipes to quickly check that it worked for some other mods. This is great :+1: 
author_association: OWNER