Repository: scikit-hep/awkward

Issue:
non-PrimitiveType `what` should fail in `ak.with_field()`


```python
>>> import awkward as ak
>>> ak.__version__
1.2.3
>>> test = ak.Array({'': []})
# this should fail (or create a type `?unknown` ?)
>>> test = ak.with_field(
    base=test,
    what=None,
    where='non-PrimitiveType_field'
)
>>> test
python3.8/site-packages/awkward/_util.py in highlevel_type(layout, behavior, isarray)
   1412 def highlevel_type(layout, behavior, isarray):
   1413     if isarray:
-> 1414         return ak.types.ArrayType(layout.type(typestrs(behavior)), len(layout))
   1415     else:
   1416         return layout.type(typestrs(behavior))

ValueError: Numpy format "O" cannot be expressed as a PrimitiveType
(https://github.com/scikit-hep/awkward-1.0/blob/1.2.3/src/libawkward/array/NumpyArray.cpp#L81)
```

I think this should fail early and not only at ak._utils / repr(test).
A type check should be around [these (structure.py#L963)](https://github.com/scikit-hep/awkward-1.0/blob/22501ba218646dc24dc515c4394eb22f126d340d/src/awkward/operations/structure.py#L963) lines?

Sorry for bringing up all these null-typed issues... I can try to PR for this issue later this week.


Discussion:
I know that the following is broadcasted and is working as expected (to get `unknown` typed):
`what=[None]`
or
`what=[None for _ in range(len(test))]` (`if len(test) != 0`)

But the above should still raise a ValueError early (if not broadcasted to one of those alternatives).



Yes, if `what is None` on the line you linked to, then the appropriate thing to convert `what` into is

```python
>>> what = ak.layout.IndexedOptionArray64(
...     ak.layout.Index64(nplike.full(len(base), -1, np.int64)),
...     ak.layout.EmptyArray()
... )
>>> what
<IndexedOptionArray64>
    <index><Index64 i="[-1 -1 -1 -1 -1 -1 -1 -1 -1 -1]" offset="0" length="10"/></index>
    <content><EmptyArray/></content>
</IndexedOptionArray64>
>>> ak.Array(what)
<Array [None, None, None, ... None, None, None] type='10 * ?unknown'>
```

If you could make a PR, I'd really appreciate it. Thanks!
Will do, including test.

Resolving patch:
diff --git a/src/awkward/operations/structure.py b/src/awkward/operations/structure.py
--- a/src/awkward/operations/structure.py
+++ b/src/awkward/operations/structure.py
@@ -959,7 +959,12 @@ def getfunction(inputs):
                 nplike = ak.nplike.of(*inputs)
                 base, what = inputs
                 if isinstance(base, ak.layout.RecordArray):
-                    if not isinstance(what, ak.layout.Content):
+                    if what is None:
+                        what = ak.layout.IndexedOptionArray64(
+                            ak.layout.Index64(nplike.full(len(base), -1, np.int64)),
+                            ak.layout.EmptyArray(),
+                        )
+                    elif not isinstance(what, ak.layout.Content):
                         what = ak.layout.NumpyArray(nplike.repeat(what, len(base)))
                     if base.istuple and where is None:
                         recordlookup = None


Test patch:
diff --git a/tests/test_0879-non-primitive-with-field.py b/tests/test_0879-non-primitive-with-field.py
new file mode 100644
--- /dev/null
+++ b/tests/test_0879-non-primitive-with-field.py
@@ -0,0 +1,27 @@
+# BSD 3-Clause License; see https://github.com/scikit-hep/awkward-1.0/blob/main/LICENSE
+
+from __future__ import absolute_import
+
+import pytest  # noqa: F401
+import numpy as np  # noqa: F401
+import awkward as ak  # noqa: F401
+
+
+def test_unknown_type():
+    array = ak.Array({"x": np.arange(10)})
+    array = ak.with_field(base=array, what=None, where="unknown field1")
+    array = ak.with_field(base=array, what=[None], where="unknown field2")
+
+    # Try to access the type of a single element
+    # This raises a ValueError in #879
+    tpe1 = array["unknown field1"].type
+    tpe2 = array["unknown field2"].type
+    assert str(tpe1) == "10 * ?unknown"
+    assert str(tpe2) == "10 * ?unknown"
+
+
+def test_in_place_wrapper_broadcasting():
+    array = ak.Array({"x": np.arange(3)})
+    array["unknown field"] = None
+
+    assert array["unknown field"].tolist() == [None, None, None]