Repository: ibis-project/ibis

Issue:
feat: Add Support for read_json with Snowflake backend
### Is your feature request related to a problem?

It would be nice to have the capability to read JSON files from a Snowflake backend in Ibis, similar to the added `read_csv` feature support in #6823 implemented in #6829. 

### Describe the solution you'd like

Use a Snowflake connection to read_json. This could be extended to other formats supported by Snowflake later, if desired, with Avro, ORC, Parquet, and XML remaining. These other formats aren't available for short-hand with any other non-Snowflake backends, but CSV and JSON certainly are. 

I mocked up a local function and was able to get this working, using a Snowflake backend and some of the other logic I found that was available. I've published a Hex Project app that demonstrates this working with the code to repro, similar to the `read_csv` example previously. I added two equality checks here between counts reading JSON with the DuckDB backend and Pandas and comparing them to the new function. 

https://app.hex.tech/99d4b06a-ff63-47a7-8092-ee0a51db9be9/app/04a6c277-acae-4caa-9cbe-4a29be629cc9/latest

One thing I noticed that differs from the behavior in Snowflake's `INFER_SCHEMA` function with JSON, it seems to lose the ordinal position. There might be another way to retain the positioning, but the functionality would still work. Here's what I mean:

<img width="1102" alt="image" src="https://github.com/ibis-project/ibis/assets/50381805/d6e96ae6-3cba-4bcd-aaff-99e63c8564cc">

The actual position of the columns in the file are 'species', 'island', 'bill_length_mm', 'bill_depth_mm', 'flipper_length_mm', 'body_mass_g', 'sex', and 'year'. Snowflake is ordering the columns alphabetically. 

I used the MATCH_BY_COLUMN_NAME argument with COPY INTO just in case there could be any issues with mixed-case columns with identifiers. This might be something to test with different source files. 

I also used the STRIP_OUTER_ARRAY file format specification for the JSON format; I think that might be required if the JSON file was created using  `orient = "records"` in [`pandas.DataFrame.to_json`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_json.html). 

I also created the file format this time, providing it an associated stage so that it wouldn't need to be referenced later with the `COPY INTO`: `self.con.execute(f"CREATE TEMP STAGE {stage} FILE_FORMAT = {file_format};")`. 

### What version of ibis are you running?

6.1.0

### What backend(s) are you using, if any?

Snowflake

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct


Discussion:

Resolving patch:
diff --git a/ibis/backends/snowflake/__init__.py b/ibis/backends/snowflake/__init__.py
--- a/ibis/backends/snowflake/__init__.py
+++ b/ibis/backends/snowflake/__init__.py
@@ -767,6 +767,78 @@ def read_csv(
 
         return self.table(table)
 
+    def read_json(
+        self, path: str | Path, table_name: str | None = None, **kwargs: Any
+    ) -> ir.Table:
+        """Read newline-delimited JSON into an ibis table, using Snowflake.
+
+        Parameters
+        ----------
+        path
+            File or list of files
+        table_name
+            Optional table name
+        **kwargs
+            Additional keyword arguments. See
+            https://docs.snowflake.com/en/sql-reference/sql/create-file-format#type-json
+            for the full list of options.
+
+        Returns
+        -------
+        Table
+            An ibis table expression
+        """
+        stage = util.gen_name("read_json_stage")
+        file_format = util.gen_name("read_json_format")
+        table = table_name or util.gen_name("read_json_snowflake")
+        qtable = self._quote(table)
+        threads = min((os.cpu_count() or 2) // 2, 99)
+
+        kwargs.setdefault("strip_outer_array", True)
+        match_by_column_name = kwargs.pop("match_by_column_name", "case_sensitive")
+
+        options = " " * bool(kwargs) + " ".join(
+            f"{name.upper()} = {value!r}" for name, value in kwargs.items()
+        )
+
+        with self.begin() as con:
+            con.exec_driver_sql(
+                f"CREATE TEMP FILE FORMAT {file_format} TYPE = JSON" + options
+            )
+
+            con.exec_driver_sql(
+                f"CREATE TEMP STAGE {stage} FILE_FORMAT = {file_format}"
+            )
+            con.exec_driver_sql(
+                f"PUT '{Path(path).absolute().as_uri()}' @{stage} PARALLEL = {threads:d}"
+            )
+
+            con.exec_driver_sql(
+                f"""
+                CREATE TEMP TABLE {qtable}
+                USING TEMPLATE (
+                    SELECT ARRAY_AGG(OBJECT_CONSTRUCT(*))
+                    FROM TABLE(
+                        INFER_SCHEMA(
+                            LOCATION => '@{stage}',
+                            FILE_FORMAT => '{file_format}'
+                        )
+                    )
+                )
+                """
+            )
+
+            # load the JSON file into the table
+            con.exec_driver_sql(
+                f"""
+                COPY INTO {qtable}
+                FROM @{stage}
+                MATCH_BY_COLUMN_NAME = {str(match_by_column_name).upper()}
+                """
+            )
+
+        return self.table(table)
+
 
 @compiles(sa.sql.Join, "snowflake")
 def compile_join(element, compiler, **kw):


Test patch:
diff --git a/ibis/backends/snowflake/tests/test_client.py b/ibis/backends/snowflake/tests/test_client.py
--- a/ibis/backends/snowflake/tests/test_client.py
+++ b/ibis/backends/snowflake/tests/test_client.py
@@ -1,9 +1,12 @@
 from __future__ import annotations
 
+import json
+
 import pandas as pd
 import pandas.testing as tm
 import pyarrow as pa
 import pytest
+from pytest import param
 
 import ibis
 import ibis.common.exceptions as com
@@ -182,3 +185,29 @@ def test_read_csv_options(con, tmp_path):
     t = con.read_csv(path, field_delimiter="|")
 
     assert t.schema() == ibis.schema(dict(a="int64", b="int64"))
+
+
+@pytest.fixture(scope="module")
+def json_data():
+    return [
+        {"a": 1, "b": "abc", "c": [{"d": 1}]},
+        {"a": 2, "b": "def", "c": [{"d": 2}]},
+        {"a": 3, "b": "ghi", "c": [{"d": 3}]},
+    ]
+
+
+@pytest.mark.parametrize(
+    "serialize",
+    [
+        param(lambda obj: "\n".join(map(json.dumps, obj)), id="ndjson"),
+        param(json.dumps, id="json"),
+    ],
+)
+def test_read_json(con, tmp_path, serialize, json_data):
+    path = tmp_path / "test.json"
+    path.write_text(serialize(json_data))
+
+    t = con.read_json(path)
+
+    assert t.schema() == ibis.schema(dict(a="int", b="string", c="array<json>"))
+    assert t.count().execute() == len(json_data)