Repository: googleapis/python-spanner-django

Issue:
tests/get_or_create.tests.UpdateOrCreateTests: obscure failures with TransactionManagementError yet raises IntegrityError correctly
I ran test `get_or_create.tests.UpdateOrCreateTests` and it failed on 2 tests with obscure test failures  that I couldn't understand because they are supposed to raise IntegrityError which we do

[test_integrity](https://github.com/timgraham/django/blob/daa96e641301c3e6b794857dc2daf33d63b37c59/tests/get_or_create/tests.py#L322-L328)
![Screen Shot 2020-03-25 at 10 22 11 PM](https://user-images.githubusercontent.com/4898263/77613104-1c542a00-6ee7-11ea-9812-4d1777aa176b.png)

and [test_manual_primary_key_test](https://github.com/timgraham/django/blob/daa96e641301c3e6b794857dc2daf33d63b37c59/tests/get_or_create/tests.py#L330-L338)
![Screen Shot 2020-03-25 at 10 23 22 PM](https://user-images.githubusercontent.com/4898263/77613167-41489d00-6ee7-11ea-8b7a-71ec35d7450d.png)


### Test failures

#### test_integrity (get_or_create.tests.UpdateOrCreateTests)
<details>

```shell
Traceback (most recent call last):
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 554, in update_or_create
    obj = self.select_for_update().get(**kwargs)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 408, in get
    self.model._meta.object_name
get_or_create.models.Person.DoesNotExist: Person matching query does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 79, in next
    return six.next(self._wrapped)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/grpc/_channel.py", line 364, in __next__
    return self._next()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/grpc/_channel.py", line 358, in _next
    raise self
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
	status = StatusCode.FAILED_PRECONDITION
	details = "birthday must not be NULL in table get_or_create_person."
	debug_error_string = "{"created":"@1585199232.734670000","description":"Error received from peer ipv6:[2607:f8b0:4005:807::200a]:443","file":"src/core/lib/surface/call.cc","file_line":1046,"grpc_message":"birthday must not be NULL in table get_or_create_person.","grpc_status":9}"
>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/spanner/dbapi/autocommit_on_cursor.py", line 57, in execute
    self.__handle_insert(sql, args or None)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/spanner/dbapi/autocommit_on_cursor.py", line 113, in __handle_insert
    sql_params_list,
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/spanner/dbapi/base_connection.py", line 43, in in_transaction
    return self._dbhandle.run_in_transaction(fn, *args, **kwargs)
  File "/Users/emmanuelodeke/Library/Python/3.7/lib/python/site-packages/google/cloud/spanner_v1/database.py", line 444, in run_in_transaction
    return session.run_in_transaction(func, *args, **kw)
  File "/Users/emmanuelodeke/Library/Python/3.7/lib/python/site-packages/google/cloud/spanner_v1/session.py", line 300, in run_in_transaction
    return_value = func(txn, *args, **kw)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/spanner/dbapi/autocommit_on_cursor.py", line 123, in __do_execute_insert_heterogenous
    _ = list(res)
  File "/Users/emmanuelodeke/Library/Python/3.7/lib/python/site-packages/google/cloud/spanner_v1/streamed.py", line 143, in __iter__
    self._consume_next()
  File "/Users/emmanuelodeke/Library/Python/3.7/lib/python/site-packages/google/cloud/spanner_v1/streamed.py", line 116, in _consume_next
    response = six.next(self._response_iterator)
  File "/Users/emmanuelodeke/Library/Python/3.7/lib/python/site-packages/google/cloud/spanner_v1/snapshot.py", line 45, in _restart_on_unavailable
    for item in iterator:
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 81, in next
    six.raise_from(exceptions.from_grpc_error(exc), exc)
  File "<string>", line 3, in raise_from
google.api_core.exceptions.FailedPrecondition: 400 birthday must not be NULL in table get_or_create_person.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/spanner/dbapi/autocommit_on_cursor.py", line 61, in execute
    raise IntegrityError(e.details if hasattr(e, 'details') else e)
spanner.dbapi.exceptions.IntegrityError: 400 birthday must not be NULL in table get_or_create_person.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 575, in _create_object_from_params
    obj = self.create(**params)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 422, in create
    obj.save(force_insert=True, using=self.db)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/base.py", line 741, in save
    force_update=force_update, update_fields=update_fields)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/base.py", line 779, in save_base
    force_update, using, update_fields,
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/base.py", line 870, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/base.py", line 908, in _do_insert
    using=using, raw=raw)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 1186, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/sql/compiler.py", line 1375, in execute_sql
    cursor.execute(sql, params)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/spanner/dbapi/autocommit_on_cursor.py", line 61, in execute
    raise IntegrityError(e.details if hasattr(e, 'details') else e)
django.db.utils.IntegrityError: 400 birthday must not be NULL in table get_or_create_person.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/tests/get_or_create/tests.py", line 328, in test_integrity
    Person.objects.update_or_create(first_name="Tom", last_name="Smith")
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 559, in update_or_create
    obj, created = self._create_object_from_params(kwargs, params, lock=True)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 580, in _create_object_from_params
    return qs.get(**lookup), False
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 402, in get
    num = len(clone)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 256, in __len__
    self._fetch_all()
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 1242, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 55, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/sql/compiler.py", line 1140, in execute_sql
    cursor.execute(sql, params)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/backends/utils.py", line 79, in _execute
    self.db.validate_no_broken_transaction()
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/backends/base/base.py", line 438, in validate_no_broken_transaction
    "An error occurred in the current transaction. You can't "
django.db.transaction.TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.
```
</details>

#### test_manual_primary_key_test (get_or_create.tests.UpdateOrCreateTests)
<details>

```shell
======================================================================
ERROR: test_integrity (get_or_create.tests.UpdateOrCreateTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 554, in update_or_create
    obj = self.select_for_update().get(**kwargs)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 408, in get
    self.model._meta.object_name
get_or_create.models.Person.DoesNotExist: Person matching query does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 79, in next
    return six.next(self._wrapped)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/grpc/_channel.py", line 364, in __next__
    return self._next()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/grpc/_channel.py", line 358, in _next
    raise self
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
	status = StatusCode.FAILED_PRECONDITION
	details = "birthday must not be NULL in table get_or_create_person."
	debug_error_string = "{"created":"@1585199232.734670000","description":"Error received from peer ipv6:[2607:f8b0:4005:807::200a]:443","file":"src/core/lib/surface/call.cc","file_line":1046,"grpc_message":"birthday must not be NULL in table get_or_create_person.","grpc_status":9}"
>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/spanner/dbapi/autocommit_on_cursor.py", line 57, in execute
    self.__handle_insert(sql, args or None)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/spanner/dbapi/autocommit_on_cursor.py", line 113, in __handle_insert
    sql_params_list,
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/spanner/dbapi/base_connection.py", line 43, in in_transaction
    return self._dbhandle.run_in_transaction(fn, *args, **kwargs)
  File "/Users/emmanuelodeke/Library/Python/3.7/lib/python/site-packages/google/cloud/spanner_v1/database.py", line 444, in run_in_transaction
    return session.run_in_transaction(func, *args, **kw)
  File "/Users/emmanuelodeke/Library/Python/3.7/lib/python/site-packages/google/cloud/spanner_v1/session.py", line 300, in run_in_transaction
    return_value = func(txn, *args, **kw)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/spanner/dbapi/autocommit_on_cursor.py", line 123, in __do_execute_insert_heterogenous
    _ = list(res)
  File "/Users/emmanuelodeke/Library/Python/3.7/lib/python/site-packages/google/cloud/spanner_v1/streamed.py", line 143, in __iter__
    self._consume_next()
  File "/Users/emmanuelodeke/Library/Python/3.7/lib/python/site-packages/google/cloud/spanner_v1/streamed.py", line 116, in _consume_next
    response = six.next(self._response_iterator)
  File "/Users/emmanuelodeke/Library/Python/3.7/lib/python/site-packages/google/cloud/spanner_v1/snapshot.py", line 45, in _restart_on_unavailable
    for item in iterator:
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 81, in next
    six.raise_from(exceptions.from_grpc_error(exc), exc)
  File "<string>", line 3, in raise_from
google.api_core.exceptions.FailedPrecondition: 400 birthday must not be NULL in table get_or_create_person.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/spanner/dbapi/autocommit_on_cursor.py", line 61, in execute
    raise IntegrityError(e.details if hasattr(e, 'details') else e)
spanner.dbapi.exceptions.IntegrityError: 400 birthday must not be NULL in table get_or_create_person.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 575, in _create_object_from_params
    obj = self.create(**params)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 422, in create
    obj.save(force_insert=True, using=self.db)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/base.py", line 741, in save
    force_update=force_update, update_fields=update_fields)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/base.py", line 779, in save_base
    force_update, using, update_fields,
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/base.py", line 870, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/base.py", line 908, in _do_insert
    using=using, raw=raw)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 1186, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/sql/compiler.py", line 1375, in execute_sql
    cursor.execute(sql, params)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/spanner/dbapi/autocommit_on_cursor.py", line 61, in execute
    raise IntegrityError(e.details if hasattr(e, 'details') else e)
django.db.utils.IntegrityError: 400 birthday must not be NULL in table get_or_create_person.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/tests/get_or_create/tests.py", line 328, in test_integrity
    Person.objects.update_or_create(first_name="Tom", last_name="Smith")
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 559, in update_or_create
    obj, created = self._create_object_from_params(kwargs, params, lock=True)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 580, in _create_object_from_params
    return qs.get(**lookup), False
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 402, in get
    num = len(clone)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 256, in __len__
    self._fetch_all()
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 1242, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/query.py", line 55, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/models/sql/compiler.py", line 1140, in execute_sql
    cursor.execute(sql, params)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/backends/utils.py", line 79, in _execute
    self.db.validate_no_broken_transaction()
  File "/Users/emmanuelodeke/Desktop/django-spanner/django_tests/django/django/db/backends/base/base.py", line 438, in validate_no_broken_transaction
    "An error occurred in the current transaction. You can't "
django.db.transaction.TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.
```
</details>





Discussion:

Resolving patch:
diff --git a/django_spanner/features.py b/django_spanner/features.py
--- a/django_spanner/features.py
+++ b/django_spanner/features.py
@@ -198,8 +198,10 @@ class DatabaseFeatures(BaseDatabaseFeatures):
         'cache.tests.CreateCacheTableForDBCacheTests',
         'cache.tests.DBCacheTests',
         'cache.tests.DBCacheWithTimeZoneTests',
-        # A rollback failed and should be investigated:
-        # https://github.com/orijtech/django-spanner/issues/299
+        # Tests that require savepoints.
+        'get_or_create.tests.UpdateOrCreateTests.test_integrity',
+        'get_or_create.tests.UpdateOrCreateTests.test_manual_primary_key_test',
+        'get_or_create.tests.UpdateOrCreateTestsWithManualPKs.test_create_with_duplicate_primary_key',
         'test_utils.tests.TestBadSetUpTestData.test_failure_in_setUpTestData_should_rollback_transaction',
         # Spanner doesn't support views.
         'inspectdb.tests.InspectDBTransactionalTests.test_include_views',


Test patch: