Repository: sharkdp/numbat (Rust)

Issue:
Struct field access fails if expression type is not yet concrete
I was playing around with the new type system and came across this issue:
```
>>> struct F { a: Current, b: Current }
>>> head([F { a: 1A, b: 2A }]).b
error: while type checking
  ┌─ <input:24>:1:28
  │
1 │ head([F { a: 1A, b: 2A }]).b
  │ -------------------------- ^ Can not access field 'b' of non struct type 'T185'
  │ │
  │ T185
```

My guess is there just needs to be a zonking step performed before looking up the struct type for field accesses?


Discussion:
Thank you for testing my implementation and reporting this!

This is something I completely missed. It might be a bit harder to fix. It's not related to lists. Here's an even simpler example that also fails:
```rs
fn id(x) = x
struct F { x: Scalar }
id(F { x: 1 }).x
```

The problem is with the polymorphic function call (`head` or `id`), because we create a fresh type variable for the instantiation of that function (e.g `T0 -> T0` for `id`). We then add a `T0 ~ F` constraint matching the parameter type with the argument type, but as you pointed out, that will only get solved *after* we look up the type for the `.x` field access (the return type `T0` in this case), leading to a *"Can not access field 'x' of non struct type 'T0'" error.*

> My guess is there just needs to be a zonking step performed before looking up the struct type for field accesses?

Hm. Are there general rules for when this is okay? We are currently solving constraints (and then zonking unification variables) only after we have looked at (and elaborated) the whole statement. We are not doing classical Hindley-Milner Algorithm W, where constraints are solved immediately when they are discovered. Instead, I tried to implement a strategy called "constraint typing", where constraints are first collected and only solved after looking at the full statement. I'm not a type theory expert, so I might have made some mistakes. But it is my understanding that this formulation is more suited (or even needed) for Numbat because (1) we need to support dimension types and (2) we want to support type class constraints.

From my understanding of the literature, the constraint typing approach is not strictly needed for (1), but Algorithm W needs some rather complex adaptations to support dimension types directly, while the formulation is more natural using constraint typing. And type class constraints (2), like the `Dim` constraint that we already have, also nicely fit into this formulation. And I think they can't be integrated into HM directly(?).

Something that *doesn't* fit well into the current type system are the nominally typed records that you implemented (because I asked you to). Looking back at this, I was recently thinking if structurally typed records would have been the right call after all 🙃. I think they fit better into functional programming languages, and they can fully support type inference (for example: PureScript has a powerful record system using row polymorphism). But nominally type structs still feel a bit more natural to me.

And I thought that we could get away with this by simply requiring type annotations for usage of structs. So something like `fn f(x) = x.foo` would simply not be supported. I thought this was a reasonable limitation. But this bug shows that I didn't think this through...

Maybe the solution would be an additional constraint? I quickly thought about this when I encountered a similar thing that you reported here... but then I found another solution for that case (whenever we have closed types, i.e. without any unification variables, we now try to pass them from the inside to the outside as far as possible. This didn't just solve my problem with struct field accesses, but also leads to much better error messages. Otherwise, we would only see constraint solver errors, which are horrible). The additional constraint could be something like `HasField(expression-type, field-name, field-type)`. The way this could work looks like this:

When we encounter a field access `expr.x`, we would not require direct knowledge about the type of `expr` (and force it to be of the right struct type). Instead, we would only generate a new `HasField` constraint that we could later solve. We wouldn't know the field-type just yet, so we would generate a fresh type variable, use that in the constraint, and return it as the type of the field-access expression. For the `id(...).x` example, we would generate the constraints
```
T0 ~ F
HasField(T0, "x", T1)
```
and return a type of `T1`. We would then solve the first constraint with a `T0 := F` substitution, turning the second constraint into
```
HasField(F, "x", T1)
```
Whenever the first argument in the `HasField` constraint has turned into a concrete type, we can attempt to solve it by looking up whether or not that type (`F`) has a field with name `"x"`. If it does, we solve that constraint by substituting `T1` with the looked-up field type (`Scalar` in this example).

If someone knows more about type systems then I do, I would really appreciate some help 😄. But right now, I'm thinking this might be a way out for this particular problem.
Oh, haha: https://ghc.gitlab.haskell.org/ghc/doc/users_guide/exts/hasfield.html

Need to read it in detail, but sounds very much related to the discussion here
HasField constraints are a good solution and I've used them often in haskell. But there's some considerations I can think of:

- If we allow users to use HasField then we will need to support dictionary passing or monomorphisation in order to allow typing:
  ```
  fn foo<Y, X: HasField<"y", Y>>(x: X) -> Y = x.y
  ```
  So that this function is able to access `y` it can either be passed an extra accessor function at runtime, monomorphised, or we can change the runtime representation of structs to `Map<String, Value>`.
- However if we don't expose the `HasField` constraints, then I think it should always be possible to solve the HasField constraint into the concrete struct type.

Rust I think is a good example to look into here, there is no exposed HasField constraint, yet a field access always has to resolve a concrete type.
#461 implements a simple version of HasField constraints.

> If we allow users to use HasField then [...]

Right. Since we don't have other types where field access would be useful (except for some of the builtin types like `DateTime` which are not customizable), I think I would keep that constraint internal for now.

> * However if we don't expose the `HasField` constraints, then I think it should always be possible to solve the HasField constraint into the concrete struct type.

:+1: 


> Rust I think is a good example to look into here, there is no exposed HasField constraint, yet a field access always has to resolve a concrete type.

I think this is what we have now with #461 as well.

---

This discussion here also made me realize that there is a similar problem with one another builtin type (`DateTime`), that sparks a whole discussion around multi-parameter type classes / traits with associated types, for which I opened a new ticket here: #462



Patch:
diff --git a/numbat/src/bytecode_interpreter.rs b/numbat/src/bytecode_interpreter.rs
index aa7b11df6..29b91e48a 100644
--- a/numbat/src/bytecode_interpreter.rs
+++ b/numbat/src/bytecode_interpreter.rs
@@ -17,7 +17,7 @@ use crate::unit::{CanonicalName, Unit};
 use crate::unit_registry::{UnitMetadata, UnitRegistry};
 use crate::value::FunctionReference;
 use crate::vm::{Constant, ExecutionContext, Op, Vm};
-use crate::{decorator, ffi};
+use crate::{decorator, ffi, Type};
 
 #[derive(Debug, Clone, Default)]
 pub struct LocalMetadata {
@@ -187,9 +187,15 @@ impl BytecodeInterpreter {
                 self.vm
                     .add_op2(Op::BuildStructInstance, struct_info_idx, exprs.len() as u16);
             }
-            Expression::AccessField(_span, _full_span, expr, attr, struct_info, _result_type) => {
+            Expression::AccessField(_span, _full_span, expr, attr, struct_type, _result_type) => {
                 self.compile_expression_with_simplify(expr)?;
 
+                let Type::Struct(ref struct_info) = struct_type.to_concrete_type() else {
+                    unreachable!(
+                        "Field access of non-struct type should be prevented by the type checker"
+                    );
+                };
+
                 let idx = struct_info.fields.get_index_of(attr).unwrap();
 
                 self.vm.add_op1(Op::AccessStructField, idx as u16);
diff --git a/numbat/src/traversal.rs b/numbat/src/traversal.rs
index 313d31c96..21656d8de 100644
--- a/numbat/src/traversal.rs
+++ b/numbat/src/traversal.rs
@@ -57,10 +57,10 @@ impl ForAllTypeSchemes for Expression {
                 }
                 info.for_all_type_schemes(f);
             }
-            Expression::AccessField(_, _, expr, _, info, type_) => {
+            Expression::AccessField(_, _, expr, _, struct_type, field_type) => {
                 expr.for_all_type_schemes(f);
-                info.for_all_type_schemes(f);
-                f(type_);
+                f(struct_type);
+                f(field_type);
             }
             Expression::List(_, elements, type_) => {
                 for element in elements {
diff --git a/numbat/src/typechecker/constraints.rs b/numbat/src/typechecker/constraints.rs
index b828e1161..e770c085a 100644
--- a/numbat/src/typechecker/constraints.rs
+++ b/numbat/src/typechecker/constraints.rs
@@ -180,6 +180,7 @@ pub enum Constraint {
     Equal(Type, Type),
     IsDType(Type),
     EqualScalar(DType),
+    HasField(Type, String, Type),
 }
 
 impl Constraint {
@@ -208,6 +209,10 @@ impl Constraint {
                 TrivialResultion::Violated
             }
             Constraint::EqualScalar(_) => TrivialResultion::Unknown,
+            Constraint::HasField(_, _, _) => {
+                // Trivial resolution handling for structs is done directly in the type checker
+                TrivialResultion::Unknown
+            }
         }
     }
 
@@ -293,6 +298,23 @@ impl Constraint {
                 }
                 _ => None,
             },
+            Constraint::HasField(struct_type, field_name, field_type)
+                if struct_type.is_closed() =>
+            {
+                if let Type::Struct(info) = struct_type {
+                    if let Some((_, actual_field_type)) = info.fields.get(field_name) {
+                        Some(Satisfied::with_new_constraints(vec![Constraint::Equal(
+                            actual_field_type.clone(),
+                            field_type.clone(),
+                        )]))
+                    } else {
+                        None
+                    }
+                } else {
+                    None
+                }
+            }
+            Constraint::HasField(_, _, _) => None,
         }
     }
 
@@ -303,6 +325,12 @@ impl Constraint {
             }
             Constraint::IsDType(t) => format!("  {}: DType", t),
             Constraint::EqualScalar(d) => format!("  {} = Scalar", d),
+            Constraint::HasField(struct_type, field_name, field_type) => {
+                format!(
+                    "HasField({}, \"{}\", {})",
+                    struct_type, field_name, field_type
+                )
+            }
         }
     }
 
@@ -327,6 +355,10 @@ impl ApplySubstitution for Constraint {
                 t.apply(substitution)?;
             }
             Constraint::EqualScalar(d) => d.apply(substitution)?,
+            Constraint::HasField(struct_type, _, field_type) => {
+                struct_type.apply(substitution)?;
+                field_type.apply(substitution)?;
+            }
         }
         Ok(())
     }
diff --git a/numbat/src/typechecker/mod.rs b/numbat/src/typechecker/mod.rs
index 9d4da924a..c0942d724 100644
--- a/numbat/src/typechecker/mod.rs
+++ b/numbat/src/typechecker/mod.rs
@@ -964,38 +964,52 @@ impl TypeChecker {
                     struct_info.clone(),
                 )
             }
-            ast::Expression::AccessField(full_span, ident_span, expr, attr) => {
+            ast::Expression::AccessField(full_span, ident_span, expr, field_name) => {
                 let expr_checked = self.elaborate_expression(expr)?;
 
                 let type_ = expr_checked.get_type();
 
-                let Type::Struct(struct_info) = type_.clone() else {
-                    return Err(TypeCheckError::FieldAccessOfNonStructType(
-                        *ident_span,
-                        expr.full_span(),
-                        attr.to_string(),
-                        type_.clone(),
-                    ));
-                };
+                let field_type = if type_.is_closed() {
+                    let Type::Struct(ref struct_info) = type_ else {
+                        return Err(TypeCheckError::FieldAccessOfNonStructType(
+                            *ident_span,
+                            expr.full_span(),
+                            field_name.to_string(),
+                            type_.clone(),
+                        ));
+                    };
 
-                let Some((_, result_type)) = struct_info.fields.get(attr) else {
-                    return Err(TypeCheckError::UnknownFieldAccess(
-                        *ident_span,
-                        expr.full_span(),
-                        attr.to_string(),
-                        type_.clone(),
-                    ));
-                };
+                    let Some((_, field_type)) = struct_info.fields.get(field_name) else {
+                        return Err(TypeCheckError::UnknownFieldAccess(
+                            *ident_span,
+                            expr.full_span(),
+                            field_name.to_string(),
+                            type_.clone(),
+                        ));
+                    };
+
+                    field_type.clone()
+                } else {
+                    let field_type = self.fresh_type_variable();
 
-                let result_type = result_type.to_owned();
+                    self.constraints
+                        .add(Constraint::HasField(
+                            type_.clone(),
+                            field_name.clone(),
+                            field_type.clone(),
+                        ))
+                        .ok();
+
+                    field_type
+                };
 
                 Expression::AccessField(
                     *ident_span,
                     *full_span,
                     Box::new(expr_checked),
-                    attr.to_owned(),
-                    struct_info,
-                    TypeScheme::concrete(result_type),
+                    field_name.to_owned(),
+                    TypeScheme::concrete(type_),
+                    TypeScheme::concrete(field_type),
                 )
             }
             ast::Expression::List(span, elements) => {
diff --git a/numbat/src/typechecker/substitutions.rs b/numbat/src/typechecker/substitutions.rs
index 4e4db48e3..353616e5f 100644
--- a/numbat/src/typechecker/substitutions.rs
+++ b/numbat/src/typechecker/substitutions.rs
@@ -194,10 +194,10 @@ impl ApplySubstitution for Expression {
                 }
                 info.apply(s)
             }
-            Expression::AccessField(_, _, instance, _, info, type_) => {
+            Expression::AccessField(_, _, instance, _, struct_type, field_type) => {
                 instance.apply(s)?;
-                info.apply(s)?;
-                type_.apply(s)
+                struct_type.apply(s)?;
+                field_type.apply(s)
             }
             Expression::List(_, elements, element_type) => {
                 for element in elements {
diff --git a/numbat/src/typed_ast.rs b/numbat/src/typed_ast.rs
index 890ee2518..bdaa4a3b1 100644
--- a/numbat/src/typed_ast.rs
+++ b/numbat/src/typed_ast.rs
@@ -518,7 +518,14 @@ pub enum Expression {
     Condition(Span, Box<Expression>, Box<Expression>, Box<Expression>),
     String(Span, Vec<StringPart>),
     InstantiateStruct(Span, Vec<(String, Expression)>, StructInfo),
-    AccessField(Span, Span, Box<Expression>, String, StructInfo, TypeScheme),
+    AccessField(
+        Span,
+        Span,
+        Box<Expression>,
+        String,     // field name
+        TypeScheme, // struct type
+        TypeScheme, // resulting field type
+    ),
     List(Span, Vec<Expression>, TypeScheme),
     TypedHole(Span, TypeScheme),
 }
@@ -661,7 +668,9 @@ impl Expression {
             Expression::Condition(_, _, then_, _) => then_.get_type(),
             Expression::String(_, _) => Type::String,
             Expression::InstantiateStruct(_, _, info_) => Type::Struct(info_.clone()),
-            Expression::AccessField(_, _, _, _, _, type_) => type_.unsafe_as_concrete(),
+            Expression::AccessField(_, _, _, _, _struct_type, field_type) => {
+                field_type.unsafe_as_concrete()
+            }
             Expression::List(_, _, element_type) => {
                 Type::List(Box::new(element_type.unsafe_as_concrete()))
             }
@@ -685,7 +694,7 @@ impl Expression {
             Expression::InstantiateStruct(_, _, info_) => {
                 TypeScheme::make_quantified(Type::Struct(info_.clone()))
             }
-            Expression::AccessField(_, _, _, _, _, type_) => type_.clone(),
+            Expression::AccessField(_, _, _, _, _struct_type, field_type) => field_type.clone(),
             Expression::List(_, _, inner) => match inner {
                 TypeScheme::Concrete(t) => TypeScheme::Concrete(Type::List(Box::new(t.clone()))),
                 TypeScheme::Quantified(ngen, qt) => TypeScheme::Quantified(


Test patch:
diff --git a/numbat/src/typechecker/tests/type_checking.rs b/numbat/src/typechecker/tests/type_checking.rs
index ed378590d..9cbf3a70f 100644
--- a/numbat/src/typechecker/tests/type_checking.rs
+++ b/numbat/src/typechecker/tests/type_checking.rs
@@ -613,6 +613,15 @@ fn structs() {
         get_typecheck_error("SomeStruct {}"),
         TypeCheckError::MissingFieldsInStructInstantiation(..)
     ));
+
+    // Regression test for https://github.com/sharkdp/numbat/issues/459
+    assert_successful_typecheck("id(SomeStruct { a: 1a, b: 1b }).a");
+
+    // Make sure that we still get nice error messages for concrete types (not some constraint solver error)
+    assert!(matches!(
+        get_typecheck_error("(SomeStruct {a: 1a, b: 1b}).a + 2b"),
+        TypeCheckError::IncompatibleDimensions(..)
+    ));
 }
 
 #[test]