Repository: dbt-labs/dbt-core

Issue:
Unit test fixture (csv) returns "" for empty value
### Current Behavior

I have fixture with empty value, the unit test will convert them to "" instead of null. It looks like seeding is working by converting them to null but not the case for unit test.

<img width="456" alt="image" src="https://github.com/dbt-labs/dbt-core/assets/324418/fec91b2b-1ace-40a1-970e-7e7f1760feb0">


### Expected Behavior

csv with empty value should be treating as null.

### Steps To Reproduce

1. create a unit test with fixture
2. in the fixture provide empty value
3. run dbt test
4. error - Database Error: invalid input syntax for integer: ""

### Relevant log output

_No response_

### Environment

```markdown
- OS:
- Python: 3.12.2
- dbt: 1.8.0-b2 + redshift 1.8.0b2
```


### Which database adapter are you using with dbt?

redshift

### Additional Context

_No response_


Discussion:
Thanks for trying out unit testing and reporting this @leesiongchan !

### How dbt seeds treat empty values

https://github.com/dbt-labs/docs.getdbt.com/issues/4867 describes how dbt [seeds](https://docs.getdbt.com/docs/build/seeds) handle empty strings by converting them to SQL `NULL` values.

> It also describes that there are two values that you can't load directly with CSV seeds:
> 
> 1. An empty / blank string (`""` / `''`)
> 1. A string with the value `"null"`

It totally makes sense that you'd expect CSV-formatted unit test fixtures to behave the same way as dbt seeds.

But we'll need to take another look at this and choose between a couple options for moving forward.

### Our options

What's your take on this @graciegoheen? We'd need to choose between one of these two options:

1. Update the behavior of blanks in CSV-formatted unit test fixtures to align with CSV-formatted seeds, accepting that it won't be possible to represent blank strings. Users that need to represent blank strings for unit tests will need to use `dict` rather than `csv` for the `format`.

1. Keep current behavior of blanks in CSV-formatted unit test fixtures, accepting that it is different than CSV-formatted seeds. We'd want to add appropriate caveats in our documentation.

To me the 1st option has more congruence and the 2nd option has more dissonance. What do you think, Grace?

<details>
<summary>

### Reprex
</summary>

`seeds/my_seed.csv`

```csv
animal,legs
duck,2
snake,""
```

`tests/fixtures/my_model_fixture.csv`

```csv
animal,legs
duck,2
snake,
```

`models/animals.sql`

```sql
select 'duck' as animal, 2 as legs union all
select 'snake' as animal, null as legs
```

`models/_unit.yml`

```yaml
unit_tests:

  - name: test_a_9881
    description: https://github.com/dbt-labs/dbt-core/issues/9881
    model: animals
    given: []
    expect:
      # This doesn't work as expected
      format: csv
      fixture: my_model_fixture

  - name: test_b_9881
    description: https://github.com/dbt-labs/dbt-core/issues/9881
    model: animals
    given: []
    expect:
      # But this works as expected
      rows:
        - {animal: duck, legs: 2}
        - {animal: snake, legs: null}
```

Run this command to see how seeds work with a blank value (a blank string `""` converts to a SQL `NULL`):

```shell
dbt show -s seeds/my_seed.csv --output json
```

Run this command and get an unexpected failure:

```shell
dbt build -s my_seed animals --full-refresh
```

</details>
Hello! @leesiongchan - thanks for opening up this request. I'd like to know more about your use case here. We've designed unit tests such that you only have to supply input mock data for the columns that are relevant to your unit test.

If we imagine a unit test that only needs `col_a` and `col_c`…

Instead of:
```
# tests/fixtures/my_fixture.csv

col_a,col_b,col_c
1,,doug
2,,grace
```

You should do:
```
# tests/fixtures/my_fixture.csv

col_a,col_c
1,doug
2,grace
```

Now, that won’t be relevant if you do need all columns but want to test 1 case where `col_b` is `null`. 

In that case, could you explicitly call out the `null` like so:
```
# tests/fixtures/my_fixture.csv

col_a,col_b,col_c
1,blue,doug
2,null,grace
```

Seeds behave differently than fixtures (though understand the comparison since they're both csv files). I am also hesitant to convert empty values to `null` for fixtures, if it would prevent someone from using an empty string in their mock input/output data for a unit test. 
@graciegoheen how does this behave for you when you add it as a fixture?

```csv
animal,legs
duck,2
snake,null
```

Here's the error message I get when I use the project files described in https://github.com/dbt-labs/dbt-core/issues/9881#issuecomment-2045286159:

```
16:15:10    Runtime Error in unit_test test_a_9881 (models/_unit.yml)
  An error occurred during execution of unit test 'test_a_9881'. There may be an error in the unit test definition: check the data types.
   Database Error
    invalid input syntax for type integer: "null"
    LINE 29:     cast('null' as integer)  
```

But maybe I have a bjorked local environment somehow?
> In that case, could you explicitly call out the `null` like so:
> 
> ```
> # tests/fixtures/my_fixture.csv
> 
> col_a,col_b,col_c
> 1,blue,doug
> 2,null,grace
> ```
> 

If this is supported perhaps it is the best solution for both worlds (seeding & unit test fixture)? In our case we need all columns because our update/delete event consists only partial of the data.
So I think this is going to be dependent on the warehouse you're using. For example, Snowflake is totally fine with `try_cast('null' as NUMBER(38,0))`, so this actually works fine for Snowflake (defining a numeric column as null in a fixture file). 

In postgres (which is what @dbeatty10 is using), `cast('null' as integer)` does not work but `cast(null as integer)` does. 

Things get funky as well when trying to cast to a string, because `try_cast('null' as text)` will result in just the actual word "null" rather than `NULL`.

@dbeatty10 did a deep dive on what you can and can't cast `NULL` to in Snowflake [here](https://github.com/dbt-labs/dbt-snowflake/issues/894#issuecomment-1935233603).

We've had a [tradeoff](https://github.com/dbt-labs/docs.getdbt.com/issues/4867) for seeds for awhile where:
- you cannot define an empty string in a seed
- you can define `NULL`

I understand it's confusing for us to decide the opposite for fixtures:
- you can define an empty string
- you cannot define `NULL`

That being said, you _can_ define `NULL` using the other formats available (currently `dict` and coming soon `sql`). I believe this is because we're wrapping the values inputted in the csv fixture in single quotes, but we don't do that for the other formats.

At minimum, I think we have a few things to do here:
- add a test to core to cover this case 
- update our [docs on data types for unit testing](https://docs.getdbt.com/reference/resource-properties/data-types) with an example of how you'd supply an intended null in each of the warehouses

In addition, we should decide how we want to handle this. A few options include:
- allowing seeds and fixtures to support both empty strings and explicit `NULL`s
```
col_a,col_b,col_c,col_d
1,blue,doug,hat
2,null,,shoe
```
- making fixtures consistent with seeds (not support empty strings, but support NULLs)
- documenting that fixtures _don't_ support NULLs (must use alternative `format` for this case)
- something else?

I'm going to move this over to technical refinement to get input from our engineers! Thanks for the discussion. 
https://github.com/dbt-labs/dbt-snowflake/issues/894#issuecomment-1936133733 has the deep dive of casting `NULL` (without quotes) across all known standard data types for the following databases:

- snowflake
- bigquery
- redshift
- postgres
- databricks
- spark
## Acceptance Criteria
- My seeds and fixtures should treat empty values the same
```
col_a,col_b,col_c,col_d
1,blue,doug,hat
2,red,,shoe
```
- `,,` becomes NULL
- there is no current way to specify an empty string using `csv` format

Resolving patch:
diff --git a/core/dbt/events/types.py b/core/dbt/events/types.py
--- a/core/dbt/events/types.py
+++ b/core/dbt/events/types.py
@@ -1822,7 +1822,7 @@ def code(self) -> str:
         return "Z026"
 
     def message(self) -> str:
-        return f"  compiled Code at {self.path}"
+        return f"  compiled code at {self.path}"
 
 
 class CheckNodeTestFailure(InfoLevel):


Test patch:
diff --git a/core/dbt/parser/unit_tests.py b/core/dbt/parser/unit_tests.py
--- a/core/dbt/parser/unit_tests.py
+++ b/core/dbt/parser/unit_tests.py
@@ -365,11 +365,17 @@ def _validate_and_normalize_rows(self, ut_fixture, unit_test_definition, fixture
                 )
 
             if ut_fixture.fixture:
-                ut_fixture.rows = self.get_fixture_file_rows(
+                csv_rows = self.get_fixture_file_rows(
                     ut_fixture.fixture, self.project.project_name, unit_test_definition.unique_id
                 )
             else:
-                ut_fixture.rows = self._convert_csv_to_list_of_dicts(ut_fixture.rows)
+                csv_rows = self._convert_csv_to_list_of_dicts(ut_fixture.rows)
+
+            # Empty values (e.g. ,,) in a csv fixture should default to null, not ""
+            ut_fixture.rows = [
+                {k: (None if v == "" else v) for k, v in row.items()} for row in csv_rows
+            ]
+
         elif ut_fixture.format == UnitTestFormat.SQL:
             if not (isinstance(ut_fixture.rows, str) or isinstance(ut_fixture.fixture, str)):
                 raise ParsingError(
diff --git a/tests/functional/unit_testing/fixtures.py b/tests/functional/unit_testing/fixtures.py
--- a/tests/functional/unit_testing/fixtures.py
+++ b/tests/functional/unit_testing/fixtures.py
@@ -41,6 +41,15 @@
 'b' as string_b
 """
 
+my_model_check_null_sql = """
+SELECT
+CASE
+  WHEN a IS null THEN True
+  ELSE False
+END a_is_null
+FROM {{ ref('my_model_a') }}
+"""
+
 test_my_model_yml = """
 unit_tests:
   - name: test_my_model
@@ -507,6 +516,11 @@
 1,a
 """
 
+test_my_model_a_with_null_fixture_csv = """id,a
+1,
+2,3
+"""
+
 test_my_model_a_empty_fixture_csv = """
 """
 
@@ -1060,3 +1074,39 @@ def external_package():
       rows:
         - {id: 2}
 """
+
+
+test_my_model_csv_null_yml = """
+unit_tests:
+  - name: test_my_model_check_null
+    model: my_model_check_null
+    given:
+      - input: ref('my_model_a')
+        format: csv
+        rows: |
+          id,a
+          1,
+          2,3
+    expect:
+      format: csv
+      rows: |
+        a_is_null
+        True
+        False
+"""
+
+test_my_model_file_csv_null_yml = """
+unit_tests:
+  - name: test_my_model_check_null
+    model: my_model_check_null
+    given:
+      - input: ref('my_model_a')
+        format: csv
+        fixture: test_my_model_a_with_null_fixture
+    expect:
+      format: csv
+      rows: |
+        a_is_null
+        True
+        False
+"""
diff --git a/tests/functional/unit_testing/test_csv_fixtures.py b/tests/functional/unit_testing/test_csv_fixtures.py
--- a/tests/functional/unit_testing/test_csv_fixtures.py
+++ b/tests/functional/unit_testing/test_csv_fixtures.py
@@ -5,15 +5,19 @@
     datetime_test_invalid_format_key,
     my_model_a_sql,
     my_model_b_sql,
+    my_model_check_null_sql,
     my_model_sql,
     test_my_model_a_empty_fixture_csv,
     test_my_model_a_fixture_csv,
     test_my_model_a_numeric_fixture_csv,
+    test_my_model_a_with_null_fixture_csv,
     test_my_model_b_fixture_csv,
     test_my_model_basic_fixture_csv,
     test_my_model_concat_fixture_csv,
+    test_my_model_csv_null_yml,
     test_my_model_csv_yml,
     test_my_model_duplicate_csv_yml,
+    test_my_model_file_csv_null_yml,
     test_my_model_file_csv_yml,
     test_my_model_fixture_csv,
     test_my_model_missing_csv_yml,
@@ -208,6 +212,50 @@ def test_unit_test(self, project):
             results = run_dbt(["test", "--select", "my_model"], expect_pass=False)
 
 
+class TestUnitTestsInlineCSVEmptyValueIsNull:
+    @pytest.fixture(scope="class")
+    def models(self):
+        return {
+            "my_model_a.sql": my_model_a_sql,
+            "my_model_check_null.sql": my_model_check_null_sql,
+            "test_my_model_csv_null.yml": test_my_model_csv_null_yml,
+        }
+
+    def test_unit_test(self, project):
+        results = run_dbt(["run"])
+        assert len(results) == 2
+
+        # Select by model name
+        results = run_dbt(["test", "--select", "my_model_check_null"], expect_pass=True)
+        assert len(results) == 1
+
+
+class TestUnitTestsFileCSVEmptyValueIsNull:
+    @pytest.fixture(scope="class")
+    def models(self):
+        return {
+            "my_model_a.sql": my_model_a_sql,
+            "my_model_check_null.sql": my_model_check_null_sql,
+            "test_my_model_file_csv_null.yml": test_my_model_file_csv_null_yml,
+        }
+
+    @pytest.fixture(scope="class")
+    def tests(self):
+        return {
+            "fixtures": {
+                "test_my_model_a_with_null_fixture.csv": test_my_model_a_with_null_fixture_csv,
+            }
+        }
+
+    def test_unit_test(self, project):
+        results = run_dbt(["run"])
+        assert len(results) == 2
+
+        # Select by model name
+        results = run_dbt(["test", "--select", "my_model_check_null"], expect_pass=True)
+        assert len(results) == 1
+
+
 class TestUnitTestsMissingCSVFile:
     @pytest.fixture(scope="class")
     def models(self):