Repository: marshmallow-code/marshmallow

Issue:
DateTime ignores date formatting string
The documentation for the `marshmallow.fields.DateTime` says:

    Parameters:	
        * format (str) – Either "rfc" (for RFC822), "iso" (for ISO8601), or a date format string. If None, defaults to “iso”.

But the part `or a date format string` is not true. I would've expected this to accept date formatting strings as defined in the [`time` module](https://docs.python.org/2/library/time.html#time.strftime), however, if the `format` parameter is a date formatting string, it is ignored and the parsing is done using `dateutil` (I noticed this [here in the source code](https://github.com/marshmallow-code/marshmallow/blob/dev/marshmallow/fields.py#L771).

It looks like the documentation is not consistent with the code here. I think it would be very valuable if you could indeed provide a date formatting string and that marshmallow will use it with (`time.strptime`](https://docs.python.org/2/library/time.html#time.strptime) to parse it instead of letting `dateutil` do a guess about the format.

I am willing to edit this in the source code, but I would like to discuss the desired behaviour first.

Cheers

Discussion:

Patch:
diff --git a/AUTHORS.rst b/AUTHORS.rst
index b9a42126..26fc232f 100644
--- a/AUTHORS.rst
+++ b/AUTHORS.rst
@@ -40,3 +40,4 @@ Contributors (chronological)
 - Kelvin Hammond `@kelvinhammond <https://github.com/kelvinhammond>`_
 - Matt Stobo `@mwstobo <https://github.com/mwstobo>`_
 - Max Orhai `@max-orhai <https://github.com/max-orhai>`_
+- Praveen `@praveen-p <https://github.com/praveen-p>`_
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index 5090a42a..f1c43987 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -1,10 +1,7 @@
 Changelog
 ---------
 
-2.0.0x (unreleased)
-+++++++++++++++++++
-
-2.0.0b5 (2015-08-23)
+2.0.0b5 (unreleased)
 ++++++++++++++++++++
 
 Features:
@@ -17,11 +14,9 @@ Features:
 Bug fixes:
 
 - `make_object` is only called after all validators and postprocessors have finished (:issue:`253`). Thanks :user:`sunsongxp` for reporting.
-- If an invalid type is passed to ``Schema`` and ``strict=False``, store a ``_schema`` error in the errors dict rather than raise an exception (:issue:`261`). Thanks :user:`density` for reporting.
 
 Other changes:
 
-- ``make_object`` is only called when input data are completely valid (:issue:`243`). Thanks :user:`kissgyorgy` for reporting.
 - Change default error messages for ``URL`` and ``Email`` validators so that they don't include user input (:issue:`255`).
 - ``Email`` validator permits email addresses with non-ASCII characters, as per RFC 6530 (:issue:`221`). Thanks :user:`lextoumbourou` for reporting and :user:`mwstobo` for sending the patch.
 
diff --git a/MANIFEST.in b/MANIFEST.in
index f6857893..b7f36481 100644
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -1,11 +1,1 @@
-include *.rst LICENSE NOTICE
-recursive-include tests *
-recursive-include examples *
-recursive-include docs *
-recursive-exclude docs *.pyc
-recursive-exclude docs *.pyo
-recursive-exclude tests *.pyc
-recursive-exclude tests *.pyo
-recursive-exclude examples *.pyc
-recursive-exclude examples *.pyo
-prune docs/_build
+include *.rst LICENSE
\ No newline at end of file
diff --git a/docs/extending.rst b/docs/extending.rst
index 606c246c..fdae3460 100644
--- a/docs/extending.rst
+++ b/docs/extending.rst
@@ -184,7 +184,6 @@ Schema-level Validation
 You can register schema-level validation functions for a :class:`Schema` using the :meth:`marshmallow.validates_schema <marshmallow.decorators.validates_schema>` decorator. Schema-level validation errors will be stored on the ``_schema`` key of the errors dictonary.
 
 .. code-block:: python
-    :emphasize-lines: 7
 
     from marshmallow import Schema, fields, validates_schema, ValidationError
 
@@ -269,15 +268,16 @@ However, if you want to specify how values are accessed from an object, you can
         return obj.get(key, default)
 
 
-Error Handlers and Accessors as Class Members
----------------------------------------------
+Handler Functions as Class Members
+----------------------------------
 
-You can register a Schema's error handler and accessor as optional class members. This might be useful for defining an abstract `Schema` class.
+You can register a Schema's error handler, validators, and accessor as optional class members. This might be useful for defining an abstract `Schema` class.
 
 .. code-block:: python
 
     class BaseSchema(Schema):
         __error_handler__ = handle_errors  # A function
+        __validators__ = [validate_schema]  # List of functions
         __accessor__ = get_from_dict  # A function
 
 
@@ -362,15 +362,3 @@ Our application schemas can now inherit from our custom schema class.
     result = ser.dump(user)
     result.data  # {"user": {"name": "Keith", "email": "keith@stones.com"}}
 
-Using Context
--------------
-
-The ``context`` attribute of a `Schema` is a general-purpose store for extra information that may be needed for (de)serialization. It may be used in both ``Schema`` and ``Field`` methods.
-
-.. code-block:: python
-
-    schema = UserSchema()
-    # Make current HTTP request available to
-    # custom fields, schema methods, schema validators, etc.
-    schema.context['request'] = request
-    schema.dump(user)
diff --git a/marshmallow/__init__.py b/marshmallow/__init__.py
index fc41de39..b8581d21 100644
--- a/marshmallow/__init__.py
+++ b/marshmallow/__init__.py
@@ -13,7 +13,7 @@ from marshmallow.decorators import (
 from marshmallow.utils import pprint, missing
 from marshmallow.exceptions import MarshallingError, UnmarshallingError, ValidationError
 
-__version__ = '2.0.0.dev'
+__version__ = '2.0.0b5-dev'
 __author__ = 'Steven Loria'
 __license__ = 'MIT'
 
diff --git a/marshmallow/fields.py b/marshmallow/fields.py
index 0990f115..b45afb9b 100755
--- a/marshmallow/fields.py
+++ b/marshmallow/fields.py
@@ -810,6 +810,11 @@ class DateTime(Field):
                 return func(value)
             except (TypeError, AttributeError, ValueError):
                 raise err
+        elif self.dateformat:
+            try:
+                return dt.datetime.strptime(value, self.dateformat)
+            except (TypeError, AttributeError, ValueError):
+                raise err
         elif utils.dateutil_available:
             try:
                 return utils.from_datestring(value)
diff --git a/marshmallow/marshalling.py b/marshmallow/marshalling.py
index 8bbb42dd..630c51f5 100644
--- a/marshmallow/marshalling.py
+++ b/marshmallow/marshalling.py
@@ -248,20 +248,13 @@ class Unmarshaller(ErrorStore):
                 key = fields_dict[attr_name].attribute or attr_name
                 try:
                     raw_value = data.get(attr_name, missing)
-                except AttributeError:  # Input data is not a dict
+                except AttributeError:
                     msg = 'Data must be a dict, got a {0}'.format(data.__class__.__name__)
-                    errors = self.get_errors(index=index)
-                    if strict:
-                        raise ValidationError(
-                            msg,
-                            field_names=[SCHEMA],
-                            fields=[]
-                        )
-                    else:
-                        errors = self.get_errors()
-                        errors.setdefault(SCHEMA, []).append(msg)
-                        # Input data type is incorrect, so we can bail out early
-                        break
+                    raise ValidationError(
+                        msg,
+                        field_names=[attr_name],
+                        fields=[field_obj]
+                    )
                 field_name = attr_name
                 if raw_value is missing and field_obj.load_from:
                     field_name = field_obj.load_from
diff --git a/marshmallow/schema.py b/marshmallow/schema.py
index 950fce69..8b1e8f92 100644
--- a/marshmallow/schema.py
+++ b/marshmallow/schema.py
@@ -257,6 +257,8 @@ class BaseSchema(base.SchemaABC):
         instead of failing silently and storing the errors.
     :param bool many: Should be set to `True` if ``obj`` is a collection
         so that the object will be serialized to a list.
+    :param bool skip_missing: If `True`, don't include key:value pairs in
+        serialized results if ``value`` is `None`.
     :param dict context: Optional context passed to :class:`fields.Method` and
         :class:`fields.Function` fields.
     :param tuple load_only: A list or tuple of fields to skip during serialization
@@ -596,15 +598,9 @@ class BaseSchema(base.SchemaABC):
         """Override-able method that defines how to create the final deserialization
         output. Defaults to noop (i.e. just return ``data`` as is).
 
-        .. note::
-
-            This method will only be invoked if when the input data are completely valid.
-
         :param dict data: The deserialized data.
 
         .. versionadded:: 1.0.0
-        .. versionchanged:: 2.0.0
-            Only invoked when data are valid.
         """
         return data
 
@@ -656,7 +652,7 @@ class BaseSchema(base.SchemaABC):
 
         result = self._invoke_load_processors(POST_LOAD, result, many)
 
-        if not errors and postprocess:
+        if postprocess:
             if many:
                 result = [self.make_object(each) for each in result]
             else:
diff --git a/marshmallow/validate.py b/marshmallow/validate.py
index 4f8f7a26..680c9019 100644
--- a/marshmallow/validate.py
+++ b/marshmallow/validate.py
@@ -97,7 +97,7 @@ class Email(Validator):
     """
 
     USER_REGEX = re.compile(
-        r"(^[-!#$%&'*+/=?^`{}|~\w]+(\.[-!#$%&'*+/=?^`{}|~\w]+)*$"  # dot-atom
+        r"(^[-!#$%&'*+/=?^_`{}|~0-9\w]+(\.[-!#$%&'*+/=?^_`{}|~0-9\w]+)*$"  # dot-atom
         # quoted-string
         r'|^"([\001-\010\013\014\016-\037!#-\[\]-\177]'
         r'|\\[\001-\011\013\014\016-\177])*"$)', re.IGNORECASE | re.UNICODE)
diff --git a/tasks.py b/tasks.py
index b1b20780..04eab04a 100644
--- a/tasks.py
+++ b/tasks.py
@@ -12,7 +12,7 @@ build_dir = os.path.join(docs_dir, '_build')
 def test():
     """Run the tests."""
     flake()
-    run('python setup.py test', echo=True)
+    run('python setup.py test', echo=True, pty=True)
 
 @task
 def flake():
@@ -48,7 +48,7 @@ def docs(clean=False, browse=False, watch=False):
     """Build the docs."""
     if clean:
         clean_docs()
-    run("sphinx-build %s %s" % (docs_dir, build_dir), echo=True)
+    run("sphinx-build %s %s" % (docs_dir, build_dir), pty=True)
     if browse:
         browse_docs()
     if watch:
@@ -69,7 +69,7 @@ def watch_docs():
 
 @task
 def readme(browse=False):
-    run("rst2html.py README.rst > README.html")
+    run("rst2html.py README.rst > README.html", pty=True)
     if browse:
         webbrowser.open_new_tab('README.html')
 


Test patch:
diff --git a/tests/test_deserialization.py b/tests/test_deserialization.py
index 356f759d..d7c66630 100644
--- a/tests/test_deserialization.py
+++ b/tests/test_deserialization.py
@@ -288,6 +288,26 @@ class TestFieldDeserialization:
         msg = 'Could not deserialize {0!r} to a datetime object.'.format(in_value)
         assert msg in str(excinfo)
 
+    def test_custom_date_format_datetime_field_deserialization(self):
+
+        dtime = dt.datetime.now()
+        datestring = dtime.strftime('%H:%M:%S %Y-%m-%d')
+
+        field = fields.DateTime(format='%d-%m-%Y %H:%M:%S')
+        #deserialization should fail when datestring is not of same format
+        with pytest.raises(ValidationError) as excinfo:
+            field.deserialize(datestring)
+        msg = 'Could not deserialize {0!r} to a datetime object.'.format(datestring)
+        assert msg in str(excinfo)
+        field = fields.DateTime(format='%H:%M:%S %Y-%m-%d')
+        assert_datetime_equal(field.deserialize(datestring), dtime)
+
+        field = fields.DateTime()
+        if utils.dateutil_available:
+            assert_datetime_equal(field.deserialize(datestring), dtime)
+        else:
+            assert msg in str(excinfo)
+
     @pytest.mark.parametrize('fmt', ['rfc', 'rfc822'])
     def test_rfc_datetime_field_deserialization(self, fmt):
         dtime = dt.datetime.now()
@@ -741,16 +761,6 @@ class TestSchemaDeserialization:
         assert result.name == 'Monty'
         assert_almost_equal(result.age, 42.3)
 
-    # https://github.com/marshmallow-code/marshmallow/issues/243
-    def test_make_object_not_called_if_data_are_invalid(self):
-        class MySchema(Schema):
-            email = fields.Email()
-
-            def make_object(self, data):
-                assert False, 'make_object should not have been called'
-        result, errors = MySchema().load({'email': 'invalid'})
-        assert 'email' in errors
-
     # Regression test for https://github.com/marshmallow-code/marshmallow/issues/253
     def test_validators_run_before_make_object(self):
         class UserSchema(Schema):
@@ -1168,24 +1178,3 @@ def test_required_message_can_be_changed(message):
     expected = [message] if isinstance(message, basestring) else message
     assert expected == errs['age']
     assert data == {}
-
-# Regression test for https://github.com/marshmallow-code/marshmallow/issues/261
-def test_deserialize_doesnt_raise_exception_if_strict_is_false_and_input_type_is_incorrect():
-    class MySchema(Schema):
-        foo = fields.Field()
-        bar = fields.Field()
-    data, errs = MySchema().load([])
-    assert '_schema' in errs
-    assert errs['_schema'] == ['Data must be a dict, got a list']
-
-
-def test_deserialize_raises_exception_if_strict_is_true_and_input_type_is_incorrect():
-    class MySchema(Schema):
-        foo = fields.Field()
-        bar = fields.Field()
-    with pytest.raises(ValidationError) as excinfo:
-        MySchema(strict=True).load([])
-    assert 'Data must be a dict, got a list' in str(excinfo)
-    exc = excinfo.value
-    assert exc.field_names == ['_schema']
-    assert exc.fields == []
diff --git a/tests/test_schema.py b/tests/test_schema.py
index 0fda3b71..bfbf5dad 100644
--- a/tests/test_schema.py
+++ b/tests/test_schema.py
@@ -1706,7 +1706,7 @@ class TestNestedSchema:
 
         schema = OuterSchema()
         _, errors = schema.load({'inner': 1})
-        assert errors['inner']['_schema'] == ['Data must be a dict, got a int']
+        assert errors['inner'] == ['Data must be a dict, got a int']
 
     def test_missing_required_nested_field(self):
         class Inner(Schema):