Repository: MongoEngine/mongoengine

Issue:
ComplexDateTimeField should fallback to None if null=True
Hi,

I was playing with the new 'null' behavior, which I would like to use in a project I'm working on. While messing with it on master, I noticed some inconsistency in the behavior with regard to DateTimeField and ComplexDateTimeField.

Here's a demonstration:

``` python
def test_null_datetime_fields():
    class TestDoc(Document):
        dt_field = DateTimeField(null=True)
        cdt_field = ComplexDateTimeField(null=True)

    doc = TestDoc()
    doc.save()

    assert doc.dt_field is None
    assert doc.cdt_field is None
```

The cause seems to be related to some logic specific to ComplexDateTimeField that falls back to datetime.now on retrieval from db. I'm not sure what the motivation was for the different behavior between DateTimeField and ComplexDateTimeField -- it was introduced a long time ago and there's even a test to ensure it doesn't regress:

https://github.com/MongoEngine/mongoengine/commit/0187a0e1

Thoughts?

ComplexDateTimeField should fallback to None if null=True
Hi,

I was playing with the new 'null' behavior, which I would like to use in a project I'm working on. While messing with it on master, I noticed some inconsistency in the behavior with regard to DateTimeField and ComplexDateTimeField.

Here's a demonstration:

``` python
def test_null_datetime_fields():
    class TestDoc(Document):
        dt_field = DateTimeField(null=True)
        cdt_field = ComplexDateTimeField(null=True)

    doc = TestDoc()
    doc.save()

    assert doc.dt_field is None
    assert doc.cdt_field is None
```

The cause seems to be related to some logic specific to ComplexDateTimeField that falls back to datetime.now on retrieval from db. I'm not sure what the motivation was for the different behavior between DateTimeField and ComplexDateTimeField -- it was introduced a long time ago and there's even a test to ensure it doesn't regress:

https://github.com/MongoEngine/mongoengine/commit/0187a0e1

Thoughts?



Discussion:
[![Coverage Status](https://coveralls.io/builds/1872339/badge)](https://coveralls.io/builds/1872339)

Coverage increased (+13.0%) to 88.08% when pulling **af8b09017660f0338ece6a5fc78ec0be716b5b96 on jimmyshen:datetime-null-patch** into **1cc7f80109cdf630ae572aca0ad49c9e19226407 on MongoEngine:master**.

[![Coverage Status](https://coveralls.io/builds/1872339/badge)](https://coveralls.io/builds/1872339)

Coverage remained the same at 75.09% when pulling **af8b09017660f0338ece6a5fc78ec0be716b5b96 on jimmyshen:datetime-null-patch** into **1cc7f80109cdf630ae572aca0ad49c9e19226407 on MongoEngine:master**.

Could you please add your test to your commit and yourself to the AUTHORS file?

Done! I extended my test to cover a few new fields too since it didn't look like there were any tests for null=True yet

[![Coverage Status](https://coveralls.io/builds/1873509/badge)](https://coveralls.io/builds/1873509)

Coverage remained the same at 75.09% when pulling **af2f38cf1effcae75e277f2cf72a0add941b432f on jimmyshen:datetime-null-patch** into **1cc7f80109cdf630ae572aca0ad49c9e19226407 on MongoEngine:master**.

[![Code Health](https://landscape.io/badge/106295/landscape.svg?style=flat)](https://landscape.io/diff/91650)
Repository health increased by 4% when pulling **[af2f38c](https://github.com/jimmyshen/mongoengine/commit/af2f38cf1effcae75e277f2cf72a0add941b432f) on jimmyshen:datetime-null-patch** into **[1cc7f80](https://github.com/MongoEngine/mongoengine/commit/1cc7f80109cdf630ae572aca0ad49c9e19226407) on MongoEngine:master**.
- [24 new problems were found](https://landscape.io/diff/91650) (including 0 errors and 24 code smells).
- [160 problems were fixed](https://landscape.io/diff/91650/fixed) (including 112 errors and 36 code smells).

[![Code Health](https://landscape.io/badge/106295/landscape.svg?style=flat)](https://landscape.io/diff/91650)
Repository health increased by 4% when pulling **[af2f38c](https://github.com/jimmyshen/mongoengine/commit/af2f38cf1effcae75e277f2cf72a0add941b432f) on jimmyshen:datetime-null-patch** into **[1cc7f80](https://github.com/MongoEngine/mongoengine/commit/1cc7f80109cdf630ae572aca0ad49c9e19226407) on MongoEngine:master**.
- [24 new problems were found](https://landscape.io/diff/91650) (including 0 errors and 24 code smells).
- [160 problems were fixed](https://landscape.io/diff/91650/fixed) (including 112 errors and 36 code smells).

Whoops! That was a silly mistake. Pushed the suggested updates and refreshed from upstream.

[![Coverage Status](https://coveralls.io/builds/1878148/badge)](https://coveralls.io/builds/1878148)

Coverage remained the same at 75.06% when pulling **fff77ef564a7551b31d5a7aa9da15129bbdb44f7 on jimmyshen:datetime-null-patch** into **65f205bca8f65d9145003ae11f21eecaa44a919c on MongoEngine:master**.

It will be nicer if you would have added the extra fields to the existing class in the test instead of duplicating some of them to another class
In addition, please squash your commits 

Sorry, I was busy for a while. I updated the test per your comment; squashed my changes, had to force push them over the existing branch and then I re-merged upstream master.

[![Coverage Status](https://coveralls.io/builds/1945818/badge)](https://coveralls.io/builds/1945818)

Coverage remained the same at 74.8% when pulling **30d154811658b66375e337c923ae5dd9390601d5 on jimmyshen:datetime-null-patch** into **aca8899c4d6a60fdadf62fa1ab2dde90dd691dde on MongoEngine:master**.

Thanks
One last thing I forgot - please add the change to `changelog.rst`

[![Coverage Status](https://coveralls.io/builds/1872339/badge)](https://coveralls.io/builds/1872339)

Coverage increased (+13.0%) to 88.08% when pulling **af8b09017660f0338ece6a5fc78ec0be716b5b96 on jimmyshen:datetime-null-patch** into **1cc7f80109cdf630ae572aca0ad49c9e19226407 on MongoEngine:master**.

[![Coverage Status](https://coveralls.io/builds/1872339/badge)](https://coveralls.io/builds/1872339)

Coverage remained the same at 75.09% when pulling **af8b09017660f0338ece6a5fc78ec0be716b5b96 on jimmyshen:datetime-null-patch** into **1cc7f80109cdf630ae572aca0ad49c9e19226407 on MongoEngine:master**.

Could you please add your test to your commit and yourself to the AUTHORS file?

Done! I extended my test to cover a few new fields too since it didn't look like there were any tests for null=True yet

[![Coverage Status](https://coveralls.io/builds/1873509/badge)](https://coveralls.io/builds/1873509)

Coverage remained the same at 75.09% when pulling **af2f38cf1effcae75e277f2cf72a0add941b432f on jimmyshen:datetime-null-patch** into **1cc7f80109cdf630ae572aca0ad49c9e19226407 on MongoEngine:master**.

[![Code Health](https://landscape.io/badge/106295/landscape.svg?style=flat)](https://landscape.io/diff/91650)
Repository health increased by 4% when pulling **[af2f38c](https://github.com/jimmyshen/mongoengine/commit/af2f38cf1effcae75e277f2cf72a0add941b432f) on jimmyshen:datetime-null-patch** into **[1cc7f80](https://github.com/MongoEngine/mongoengine/commit/1cc7f80109cdf630ae572aca0ad49c9e19226407) on MongoEngine:master**.
- [24 new problems were found](https://landscape.io/diff/91650) (including 0 errors and 24 code smells).
- [160 problems were fixed](https://landscape.io/diff/91650/fixed) (including 112 errors and 36 code smells).

[![Code Health](https://landscape.io/badge/106295/landscape.svg?style=flat)](https://landscape.io/diff/91650)
Repository health increased by 4% when pulling **[af2f38c](https://github.com/jimmyshen/mongoengine/commit/af2f38cf1effcae75e277f2cf72a0add941b432f) on jimmyshen:datetime-null-patch** into **[1cc7f80](https://github.com/MongoEngine/mongoengine/commit/1cc7f80109cdf630ae572aca0ad49c9e19226407) on MongoEngine:master**.
- [24 new problems were found](https://landscape.io/diff/91650) (including 0 errors and 24 code smells).
- [160 problems were fixed](https://landscape.io/diff/91650/fixed) (including 112 errors and 36 code smells).

Whoops! That was a silly mistake. Pushed the suggested updates and refreshed from upstream.

[![Coverage Status](https://coveralls.io/builds/1878148/badge)](https://coveralls.io/builds/1878148)

Coverage remained the same at 75.06% when pulling **fff77ef564a7551b31d5a7aa9da15129bbdb44f7 on jimmyshen:datetime-null-patch** into **65f205bca8f65d9145003ae11f21eecaa44a919c on MongoEngine:master**.

It will be nicer if you would have added the extra fields to the existing class in the test instead of duplicating some of them to another class
In addition, please squash your commits 

Sorry, I was busy for a while. I updated the test per your comment; squashed my changes, had to force push them over the existing branch and then I re-merged upstream master.

[![Coverage Status](https://coveralls.io/builds/1945818/badge)](https://coveralls.io/builds/1945818)

Coverage remained the same at 74.8% when pulling **30d154811658b66375e337c923ae5dd9390601d5 on jimmyshen:datetime-null-patch** into **aca8899c4d6a60fdadf62fa1ab2dde90dd691dde on MongoEngine:master**.

Thanks
One last thing I forgot - please add the change to `changelog.rst`


Resolving patch:
diff --git a/mongoengine/fields.py b/mongoengine/fields.py
--- a/mongoengine/fields.py
+++ b/mongoengine/fields.py
@@ -510,7 +510,7 @@ def _convert_from_string(self, data):
     def __get__(self, instance, owner):
         data = super(ComplexDateTimeField, self).__get__(instance, owner)
         if data is None:
-            return datetime.datetime.now()
+            return None if self.null else datetime.datetime.now()
         if isinstance(data, datetime.datetime):
             return data
         return self._convert_from_string(data)


Test patch:
diff --git a/tests/document/instance.py b/tests/document/instance.py
--- a/tests/document/instance.py
+++ b/tests/document/instance.py
@@ -2760,6 +2760,12 @@ def test_null_field(self):
         class User(Document):
             name = StringField()
             height = IntField(default=184, null=True)
+            str_fld = StringField(null=True)
+            int_fld = IntField(null=True)
+            flt_fld = FloatField(null=True)
+            dt_fld = DateTimeField(null=True)
+            cdt_fld = ComplexDateTimeField(null=True)
+
         User.objects.delete()
         u = User(name='user')
         u.save()
@@ -2767,6 +2773,12 @@ class User(Document):
         u_from_db.height = None
         u_from_db.save()
         self.assertEquals(u_from_db.height, None)
+        # 864
+        self.assertEqual(u_from_db.str_fld, None)
+        self.assertEqual(u_from_db.int_fld, None)
+        self.assertEqual(u_from_db.flt_fld, None)
+        self.assertEqual(u_from_db.dt_fld, None)
+        self.assertEqual(u_from_db.cdt_fld, None)
 
         # 735
         User.objects.delete()