Repository: frictionlessdata/frictionless-py

Issue:
302 redirect
# Overview

Using frictionless (4.21.2) CLI to validate a CSV file available from a 302 redirect URL leads to strange behaviour.

The incriminated URL is <https://demo.data.gouv.fr/fr/datasets/r/d4d78089-f218-40b6-a86a-d7cc4e624b0c>. It redirects to <https://static.data.gouv.fr/resources/donnees-essentielles-de-conventions-de-subvention-a-antibes/20200325-153014/antibes-subventions-2017.csv> using `Location` http header.

Validating the primary URL using frictionless:

```bash
> frictionless validate --schema https://git.opendatafrance.net/scdl/subventions/-/raw/v2.0.1/schema.json https://demo.data.gouv.fr/fr/datasets/r/d4d78089-f218-40b6-a86a-d7cc4e624b0c

# -----
# valid: https://demo.data.gouv.fr/fr/datasets/r/d4d78089-f218-40b6-a86a-d7cc4e624b0c
# -----
```

Validating the target url:

```bash
frictionless validate --schema https://git.opendatafrance.net/scdl/subventions/-/raw/v2.0.1/schema.json https://static.data.gouv.fr/resources/donnees-essentielles-de-conventions-de-subvention-a-antibes/20200325-153014/antibes-subventions-2017.csv

# -------
# invalid: https://static.data.gouv.fr/resources/donnees-essentielles-de-conventions-de-subvention-a-antibes/20200325-153014/antibes-subventions-2017.csv
# -------
==========  ======================================
code        message
==========  ======================================
task-error  The task has an error: 'fieldPosition'
==========  ======================================
```

So I got some questions/remarks:
- it seems that frictionless doesn't support 302 redirect. Am I wrong? If not, do you plan to implement it or is there a workaround to deal with this type of URL?
- first frictionless command reports a valid result. It's strange as the URL body is HTML and not tabular data (?)
- second frictionless command reports an invalid result. How should the message be understood?

Thanks in advance for your help.

---

Please preserve this line to notify @roll (lead of this repository)


Discussion:

Patch:
diff --git a/frictionless/error.py b/frictionless/error.py
index b0a4a104..22d2ea5a 100644
--- a/frictionless/error.py
+++ b/frictionless/error.py
@@ -1,4 +1,5 @@
 from .metadata import Metadata
+from . import helpers
 
 
 # NOTE:
@@ -37,7 +38,7 @@ class Error(Metadata):
         self.setinitial("name", self.name)
         self.setinitial("tags", self.tags)
         self.setinitial("note", note)
-        self.setinitial("message", self.template.format(**self))
+        self.setinitial("message", helpers.safe_format(self.template, self))
         self.setinitial("description", self.description)
 
     @property
diff --git a/frictionless/helpers.py b/frictionless/helpers.py
index 679e56d3..b9c41303 100644
--- a/frictionless/helpers.py
+++ b/frictionless/helpers.py
@@ -55,6 +55,15 @@ def pass_through(iterator):
         pass
 
 
+def safe_format(text, data):
+    return text.format_map(SafeFormatDict(data))
+
+
+class SafeFormatDict(dict):
+    def __missing__(self, key):
+        return ""
+
+
 def remove_non_values(mapping):
     return {key: value for key, value in mapping.items() if value is not None}
 
diff --git a/frictionless/row.py b/frictionless/row.py
index ae2e6140..c5b18ac6 100644
--- a/frictionless/row.py
+++ b/frictionless/row.py
@@ -297,7 +297,7 @@ class Row(dict):
             if field_mapping is None:
                 break
             field, field_number, field_position = field_mapping
-            if not is_empty and not is_empty and super().__contains__(field.name):
+            if not is_empty and super().__contains__(field.name):
                 continue
 
             # Read cell
@@ -322,21 +322,26 @@ class Row(dict):
                     )
                 )
 
-            # Constraint errors
-            if notes:
-                for note in notes.values():
-                    self.__errors.append(
-                        errors.ConstraintError(
-                            note=note,
-                            cells=list(map(to_str, cells)),
-                            row_number=self.__row_number,
-                            row_position=self.__row_position,
-                            cell=str(source),
-                            field_name=field.name,
-                            field_number=field_number,
-                            field_position=field_position,
+            # NOTE: review this logic (why we can't skip reading also?)
+            # Check constriants if there is an existent cell
+            # Otherwise we emit only "missing-cell" which is enough
+            if field_position:
+
+                # Constraint errors
+                if notes:
+                    for note in notes.values():
+                        self.__errors.append(
+                            errors.ConstraintError(
+                                note=note,
+                                cells=list(map(to_str, cells)),
+                                row_number=self.__row_number,
+                                row_position=self.__row_position,
+                                cell=str(source),
+                                field_name=field.name,
+                                field_number=field_number,
+                                field_position=field_position,
+                            )
                         )
-                    )
 
             # Set/return value
             super().__setitem__(field.name, target)


Test patch:
diff --git a/tests/validate/test_main.py b/tests/validate/test_main.py
index 8fe31074..28e96021 100644
--- a/tests/validate/test_main.py
+++ b/tests/validate/test_main.py
@@ -1,4 +1,4 @@
-from frictionless import Resource, validate
+from frictionless import Resource, Field, validate, describe
 
 
 # Table
@@ -29,6 +29,20 @@ def test_validate_from_resource_instance():
     assert report.valid
 
 
+# Issues
+
+
 def test_validate_multiple_files_issue_850():
     report = validate("data/package/*.csv")
     assert report.stats["tasks"] == 2
+
+
+def test_validate_less_actual_fields_with_required_constraint_issue_950():
+    schema = describe("data/table.csv", type="schema")
+    schema.add_field(Field(name="bad", constraints={"required": True}))
+    report = validate("data/table.csv", schema=schema)
+    assert report.flatten(["rowPosition", "fieldPosition", "code"]) == [
+        [None, 3, "missing-label"],
+        [2, 3, "missing-cell"],
+        [3, 3, "missing-cell"],
+    ]