internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: googleapis/python-spanner-django

Issue:
dbapi: remove classification for AUDIT, ANALYZE, EXPLAIN,SHOW since they aren't supported by Cloud Spanner
Coming here from the design document, where @bvandiver raised to me:

>AUDIT, ANALYZE are not keywords in spanner - EXPLAIN will always fail with an unsupported error. SHOW is part of the Connections API, but not part of Spanner proper. Clean up?


Discussion:

Resolving patch:
diff --git a/spanner/dbapi/parse_utils.py b/spanner/dbapi/parse_utils.py
--- a/spanner/dbapi/parse_utils.py
+++ b/spanner/dbapi/parse_utils.py
@@ -22,7 +22,9 @@
 STMT_INSERT = 'INSERT'
 
 # Heuristic for identifying statements that don't need to be run as updates.
-re_NON_UPDATE = re.compile(r'^\s*(SELECT|ANALYZE|AUDIT|EXPLAIN|SHOW)', re.IGNORECASE)
+re_NON_UPDATE = re.compile(r'^\s*(SELECT)', re.IGNORECASE)
+
+re_WITH = re.compile(r'^\s*(WITH)', re.IGNORECASE)
 
 # DDL statements follow https://cloud.google.com/spanner/docs/data-definition-language
 re_DDL = re.compile(r'^\s*(CREATE|ALTER|DROP)', re.IGNORECASE | re.DOTALL)
@@ -37,6 +39,11 @@ def classify_stmt(sql):
         return STMT_INSERT
     elif re_NON_UPDATE.match(sql):
         return STMT_NON_UPDATING
+    elif re_WITH.match(sql):
+        # As of Fri-13th-March-2020, Cloud Spanner only supports WITH for DQL
+        # statements and doesn't yet support WITH for DML statements.
+        # When WITH for DML is added, we'll need to update this classifier accordingly.
+        return STMT_NON_UPDATING
     else:
         return STMT_UPDATING
 


Test patch:
diff --git a/tests/spanner/dbapi/test_parse_utils.py b/tests/spanner/dbapi/test_parse_utils.py
--- a/tests/spanner/dbapi/test_parse_utils.py
+++ b/tests/spanner/dbapi/test_parse_utils.py
@@ -22,7 +22,7 @@ def test_classify_stmt(self):
         cases = [
                 ('SELECT 1', STMT_NON_UPDATING,),
                 ('SELECT s.SongName FROM Songs AS s', STMT_NON_UPDATING,),
-                ('EXPLAIN app', STMT_NON_UPDATING,),
+                ('WITH sq AS (SELECT SchoolID FROM Roster) SELECT * from sq', STMT_NON_UPDATING,),
                 (
                     'CREATE TABLE django_content_type (id STRING(64) NOT NULL, name STRING(100) '
                     'NOT NULL, app_label STRING(100) NOT NULL, model STRING(100) NOT NULL) PRIMARY KEY(id)',
Repository: googleapis/python-spanner-django

Issue:
django: fix Chr and Ord database functions
It looks like `Chr` could use [`CODE_POINTS_TO_STRING`](https://cloud.google.com/spanner/docs/functions-and-operators#code_points_to_string) and `Ord` could use [`TO_CODE_POINTS`](https://cloud.google.com/spanner/docs/functions-and-operators#to_code_points).


Discussion:

Resolving patch:
diff --git a/spanner/django/features.py b/spanner/django/features.py
--- a/spanner/django/features.py
+++ b/spanner/django/features.py
@@ -196,13 +196,6 @@ class DatabaseFeatures(BaseDatabaseFeatures):
         # FLOAT64. Supported signatures: MOD(INT64, INT64)
         'db_functions.math.test_mod.ModTests.test_decimal',
         'db_functions.math.test_mod.ModTests.test_float',
-        # The Chr and Ord database functions are not implemented:
-        # https://github.com/orijtech/spanner-orm/issues/243
-        'db_functions.text.test_chr.ChrTests.test_basic',
-        'db_functions.text.test_chr.ChrTests.test_non_ascii',
-        'db_functions.text.test_chr.ChrTests.test_transform',
-        'db_functions.text.test_ord.OrdTests.test_basic',
-        'db_functions.text.test_ord.OrdTests.test_transform',
         # Spanner's CAST function returns null if any of its arguments are
         # null: https://github.com/orijtech/spanner-orm/issues/242
         'db_functions.text.test_concat.ConcatTests.test_basic',
diff --git a/spanner/django/functions.py b/spanner/django/functions.py
--- a/spanner/django/functions.py
+++ b/spanner/django/functions.py
@@ -7,10 +7,14 @@
 import math
 
 from django.db.models.functions import (
-    Cot, Degrees, Left, Log, Pi, Radians, Right, StrIndex, Substr,
+    Chr, Cot, Degrees, Left, Log, Ord, Pi, Radians, Right, StrIndex, Substr,
 )
 
 
+def chr_(self, compiler, connection, **extra_context):
+    return self.as_sql(compiler, connection, template='CODE_POINTS_TO_STRING([%(expressions)s])', **extra_context)
+
+
 def cot(self, compiler, connection, **extra_context):
     return self.as_sql(compiler, connection, template='(1 / TAN(%(expressions)s))', **extra_context)
 
@@ -35,6 +39,10 @@ def log(self, compiler, connection, **extra_context):
     return clone.as_sql(compiler, connection, **extra_context)
 
 
+def ord_(self, compiler, connection, **extra_context):
+    return self.as_sql(compiler, connection, template='TO_CODE_POINTS(%(expressions)s)[OFFSET(0)]', **extra_context)
+
+
 def pi(self, compiler, connection, **extra_context):
     return self.as_sql(compiler, connection, template=str(math.pi), **extra_context)
 
@@ -56,10 +64,12 @@ def substr(self, compiler, connection, **extra_context):
 
 
 def register_functions():
+    Chr.as_spanner = chr_
     Cot.as_spanner = cot
     Degrees.as_spanner = degrees
     Left.as_spanner = left_and_right
     Log.as_spanner = log
+    Ord.as_spanner = ord_
     Pi.as_spanner = pi
     Radians.as_spanner = radians
     Right.as_spanner = left_and_right


Test patch:
Repository: googleapis/python-spanner-django

Issue:
django: Cast to CharField with max_length doesn't work
```
======================================================================
FAIL: test_cast_to_char_field_with_max_length (db_functions.comparison.test_cast.CastTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tim/code/django/django/test/utils.py", line 373, in inner
    return func(*args, **kwargs)
  File "/home/tim/code/django/django/test/testcases.py", line 1229, in skip_wrapper
    return test_func(*args, **kwargs)
  File "/home/tim/code/django/tests/db_functions/comparison/test_cast.py", line 38, in test_cast_to_char_field_with_max_length
    self.assertEqual(names.get().cast_string, 'B')
AssertionError: 'Bob' != 'B'
- Bob
+ B
```


Discussion:

Resolving patch:
diff --git a/spanner/django/features.py b/spanner/django/features.py
--- a/spanner/django/features.py
+++ b/spanner/django/features.py
@@ -196,9 +196,6 @@ class DatabaseFeatures(BaseDatabaseFeatures):
         # null: https://github.com/orijtech/spanner-orm/issues/242
         'db_functions.text.test_concat.ConcatTests.test_basic',
         'db_functions.text.test_concat.ConcatTests.test_many',
-        # Cast to CharField with max_length doesn't work:
-        # https://github.com/orijtech/spanner-orm/issues/258
-        'db_functions.comparison.test_cast.CastTests.test_cast_to_char_field_with_max_length',
         # casting DateField to DateTimeField adds an unexpected hour:
         # https://github.com/orijtech/spanner-orm/issues/260
         'db_functions.comparison.test_cast.CastTests.test_cast_from_db_date_to_datetime',
diff --git a/spanner/django/functions.py b/spanner/django/functions.py
--- a/spanner/django/functions.py
+++ b/spanner/django/functions.py
@@ -7,10 +7,21 @@
 import math
 
 from django.db.models.functions import (
-    Chr, Cot, Degrees, Left, Log, Ord, Pi, Radians, Right, StrIndex, Substr,
+    Cast, Chr, Cot, Degrees, Left, Log, Ord, Pi, Radians, Right, StrIndex,
+    Substr,
 )
 
 
+def cast(self, compiler, connection, **extra_context):
+    # Account for a field's max_length using SUBSTR.
+    max_length = getattr(self.output_field, 'max_length', None)
+    if max_length is not None:
+        template = 'SUBSTR(' + self.template + ', 0, %s)' % max_length
+    else:
+        template = self.template
+    return self.as_sql(compiler, connection, template=template, **extra_context)
+
+
 def chr_(self, compiler, connection, **extra_context):
     return self.as_sql(compiler, connection, template='CODE_POINTS_TO_STRING([%(expressions)s])', **extra_context)
 
@@ -64,6 +75,7 @@ def substr(self, compiler, connection, **extra_context):
 
 
 def register_functions():
+    Cast.as_spanner = cast
     Chr.as_spanner = chr_
     Cot.as_spanner = cot
     Degrees.as_spanner = degrees


Test patch:
Repository: googleapis/python-spanner-django

Issue:
dbapi: session pooling not working with multiple databases
Various `multiple_database` tests started failing after the global session pool was introduced in 2b5546aad99d77dfe179a57292f416f2d1c0a65f. It seems like queries are routed to the wrong database.

For example:
```
======================================================================
FAIL: test_basic_queries (multiple_database.tests.QueryTestCase)
Queries are constrained to a single database
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tim/code/django/tests/multiple_database/tests.py", line 122, in test_basic_queries
    Book.objects.using('default').get(published=datetime.date(2009, 5, 4))
AssertionError: DoesNotExist not raised
```
The multiple_database tests were running on CI but (inadvertently?) removed in cfbdb6ec586140ef5614e5a6d6bb9caee4f51d2a (where they were already failing).


Discussion:

Resolving patch:
diff --git a/spanner/dbapi/__init__.py b/spanner/dbapi/__init__.py
--- a/spanner/dbapi/__init__.py
+++ b/spanner/dbapi/__init__.py
@@ -30,8 +30,6 @@
 # are working properly, we'll update the threadsafety level.
 threadsafety = 1
 
-global_session_pool = spanner.pool.BurstyPool()
-
 
 def connect(project=None, instance=None, database=None, credentials_uri=None, user_agent=None, autocommit=True):
     """
@@ -70,11 +68,11 @@ def connect(project=None, instance=None, database=None, credentials_uri=None, us
     if not client_instance.exists():
         raise ProgrammingError("instance '%s' does not exist." % instance)
 
-    db = client_instance.database(database, pool=global_session_pool)
+    db = client_instance.database(database, pool=spanner.pool.BurstyPool())
     if not db.exists():
         raise ProgrammingError("database '%s' does not exist." % database)
 
-    # Correctly retrieve a session from the global session pool.
+    # Correctly retrieve a session from the session pool.
     # See:
     #   * https://github.com/orijtech/django-spanner/issues/291
     #   * https://github.com/googleapis/python-spanner/issues/10#issuecomment-585056760


Test patch:
Repository: googleapis/python-spanner-django

Issue:
all: discourage use of DecimalField until Spanner has NUMERIC
Coming here from the design doc, with support and advisory from @timgraham and @bvandiver, let's document in the README and also perhaps fail in code if users try to use DecimalField, up until Cloud Spanner announces support for NUMERIC.  


Discussion:
We can use the system check framework to raise an error if any models use DecimalField.
Sounds like a plan to me, thank you!

Resolving patch:
diff --git a/django_spanner/base.py b/django_spanner/base.py
--- a/django_spanner/base.py
+++ b/django_spanner/base.py
@@ -14,6 +14,7 @@
 from .introspection import DatabaseIntrospection
 from .operations import DatabaseOperations
 from .schema import DatabaseSchemaEditor
+from .validation import DatabaseValidation
 
 
 class DatabaseWrapper(BaseDatabaseWrapper):
@@ -87,6 +88,7 @@ class DatabaseWrapper(BaseDatabaseWrapper):
     introspection_class = DatabaseIntrospection
     ops_class = DatabaseOperations
     client_class = DatabaseClient
+    validation_class = DatabaseValidation
 
     @property
     def instance(self):
diff --git a/django_spanner/validation.py b/django_spanner/validation.py
new file mode 100644
--- /dev/null
+++ b/django_spanner/validation.py
@@ -0,0 +1,21 @@
+import os
+
+from django.core import checks
+from django.db.backends.base.validation import BaseDatabaseValidation
+from django.db.models import DecimalField
+
+
+class DatabaseValidation(BaseDatabaseValidation):
+
+    def check_field_type(self, field, field_type):
+        errors = []
+        # Disable the error when running the Django test suite.
+        if os.environ.get('RUNNING_SPANNER_BACKEND_TESTS') != '1' and isinstance(field, DecimalField):
+            errors.append(
+                checks.Error(
+                    'DecimalField is not yet supported by Spanner.',
+                    obj=field,
+                    id='spanner.E001',
+                )
+            )
+        return errors


Test patch:
Repository: googleapis/python-spanner-django

Issue:
django-test-suite: admin_changelist test_multiuser_edit failure
- [ ] test_multiuser_edit

fails currently with
```
======================================================================
FAIL: test_multiuser_edit (admin_changelist.tests.ChangeListTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/emmanuelodeke/Desktop/spanner-orm/django_tests/django/tests/admin_changelist/tests.py", line 760, in test_multiuser_edit
    self.assertContains(response, '1 swallow was changed successfully.')
  File "/Users/emmanuelodeke/Desktop/spanner-orm/django_tests/django/django/test/testcases.py", line 454, in assertContains
    self.assertTrue(real_count != 0, msg_prefix + "Couldn't find %s in response" % text_repr)
AssertionError: False is not true : Couldn't find '1 swallow was changed successfully.' in response
```


Discussion:

Resolving patch:
diff --git a/django_spanner/features.py b/django_spanner/features.py
--- a/django_spanner/features.py
+++ b/django_spanner/features.py
@@ -121,8 +121,6 @@ class DatabaseFeatures(BaseDatabaseFeatures):
         # extract() with timezone not working as expected:
         # https://github.com/orijtech/spanner-orm/issues/191
         'timezones.tests.NewDatabaseTests.test_query_datetimes',
-        # To be investigated: https://github.com/orijtech/spanner-orm/issues/135
-        'admin_changelist.tests.ChangeListTests.test_multiuser_edit',
         # using NULL with + crashes: https://github.com/orijtech/spanner-orm/issues/201
         'annotations.tests.NonAggregateAnnotationTestCase.test_combined_annotation_commutative',
         # Spanner loses DecimalField precision due to conversion to float:


Test patch:
Repository: googleapis/python-spanner-django

Issue:
django-test-suite: iendswith lookup isn't case-insensitive
Test failure:
```
======================================================================
FAIL: test_ticket8597 (queries.tests.ComparisonTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmpfs/src/github/python-spanner-django/django_tests_3/django/tests/queries/tests.py", line 1984, in test_ticket8597
    self.assertQuerysetEqual(
  File "/tmpfs/src/github/python-spanner-django/django_tests_3/django/django/test/testcases.py", line 1056, in assertQuerysetEqual
    return self.assertEqual(list(items), values, msg=msg)
AssertionError: Lists differ: [] != ['<Item: a_b>']

Second list contains 1 additional elements.
First extra element 0:
'<Item: a_b>'

- []
+ ['<Item: a_b>']
```

The generated query is missing `(?i)` in the parameter:

```sql
SELECT ...
FROM queries_item INNER JOIN queries_note ON (queries_item.note_id = queries_note.id) 
WHERE REGEXP_CONTAINS(CAST(queries_item.name AS STRING), %s) 

PARAMS = ('A_b$',);
```

Regression in ff3f43d897c54e3a678811d4f076956975ec721a.


Discussion:

Resolving patch:
diff --git a/django_spanner/lookups.py b/django_spanner/lookups.py
--- a/django_spanner/lookups.py
+++ b/django_spanner/lookups.py
@@ -84,9 +84,9 @@ def startswith_endswith(self, compiler, connection):
             params[0] = str(params[0][1:]) + '$'
         else:
             params[0] = '^' + str(params[0][:-1])
-            # Add the case insensitive flag for istartswith or iendswith.
-            if is_insensitive:
-                params[0] = '(?i)' + params[0]
+        # Add the case insensitive flag for istartswith or iendswith.
+        if is_insensitive:
+            params[0] = '(?i)' + params[0]
         # rhs_sql is REGEXP_CONTAINS(%s, %%s), and lhs_sql is the column name.
         return rhs_sql % lhs_sql, params
     else:


Test patch:
Repository: googleapis/python-spanner-django

Issue:
django: regex lookup with an F expression crashes
#251 fixed icontains with F expressions (e.g. `Employee.objects.filter(firstname__iexact=F('lastname')`. The same issue affects regex/iregex. It looks like we'll need to add such tests to Django's test suite.

(It may appears that  contains/icontains, startswith, endswith, istartswith, and iendswith are also affected but these are the subject of #178.)


Discussion:

Resolving patch:
diff --git a/django_spanner/lookups.py b/django_spanner/lookups.py
--- a/django_spanner/lookups.py
+++ b/django_spanner/lookups.py
@@ -60,12 +60,21 @@ def regex(self, compiler, connection):
     rhs_sql, rhs_params = self.process_rhs(compiler, connection)
     params.extend(rhs_params)
     rhs_sql = self.get_rhs_op(connection, rhs_sql)
-    if self.lookup_name.startswith('i'):
-        params[0] = '(?i)%s' % params[0]
+    is_iregex = self.lookup_name.startswith('i')
+    if self.rhs_is_direct_value() and params and not self.bilateral_transforms:
+        if is_iregex:
+            params[0] = '(?i)%s' % params[0]
+        else:
+            params[0] = str(params[0])
+        # rhs_sql is REGEXP_CONTAINS(%s, %%s), and lhs_sql is the column name.
+        return rhs_sql % lhs_sql, params
     else:
-        params[0] = str(params[0])
-    # rhs_sql is REGEXP_CONTAINS(%s, %%s), and lhs_sql is the column name.
-    return rhs_sql % lhs_sql, params
+        # rhs is the expression/column to use as the base of the regular
+        # expression.
+        rhs = compiler.compile(self.rhs)[0]
+        if is_iregex:
+            rhs = "CONCAT('(?i)', " + rhs + ")"
+        return 'REGEXP_CONTAINS(%s, %s)' % (lhs_sql, rhs), params
 
 
 def startswith_endswith(self, compiler, connection):


Test patch:
Repository: googleapis/python-spanner-django

Issue:
django-test-suite: migration_test_data_persistence.tests.MigrationDataNormalPersistenceTestCase failure
Just noticed as the sole failure after moving to Google Cloud Build:
[Full build listing](https://source.cloud.google.com/results/invocations/492627f5-7fa7-4506-978e-ddc4e9456ab4/targets/cloud-devrel%2Fclient-libraries%2Fpython%2Fgoogleapis%2Fpython-spanner-django%2Fpresubmit%2Fpresubmit/log)

and in particular the [test failure](https://source.cloud.google.com/results/invocations/81c963da-2ed6-4ce3-bd98-82594caf869f/targets/cloud-devrel%2Fclient-libraries%2Fpython%2Fgoogleapis%2Fpython-spanner-django%2Fpresubmit%2Fworker_1/log)
```shell
======================================================================
FAIL: test_persistence (migration_test_data_persistence.tests.MigrationDataNormalPersistenceTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmpfs/src/github/python-spanner-django/django_tests_1/django/tests/migration_test_data_persistence/tests.py", line 28, in test_persistence
    self.assertEqual(
AssertionError: 0 != 1

----------------------------------------------------------------------
```

To be investigated, as am currently prioritizing getting the tests moved and passing on Google Cloud Build.


Discussion:

Resolving patch:
diff --git a/django_spanner/features.py b/django_spanner/features.py
--- a/django_spanner/features.py
+++ b/django_spanner/features.py
@@ -278,13 +278,13 @@ class DatabaseFeatures(BaseDatabaseFeatures):
         # This test isn't isolated on databases like Spanner that don't
         # support transactions: https://code.djangoproject.com/ticket/31413
         'migrations.test_loader.LoaderTests.test_loading_squashed',
-        # "Permission errors are not swallowed":
-        # https://github.com/googleapis/python-spanner-django/issues/407
-        'file_uploads.tests.DirectoryCreationTests.test_readonly_root',
-        # Migrations data  persistance issue to be investigated:
-        # https://github.com/googleapis/python-spanner-django/issues/408
-        'migration_test_data_persistence.tests.MigrationDataNormalPersistenceTestCase.test_persistence',
     )
+    # Kokoro-specific skips.
+    if os.environ.get('KOKORO_JOB_NAME'):
+        skip_tests += (
+            # os.chmod() doesn't work on Kokoro?
+            'file_uploads.tests.DirectoryCreationTests.test_readonly_root',
+        )
 
     if os.environ.get('SPANNER_EMULATOR_HOST', None):
         # Some code isn't yet supported by the Spanner emulator.


Test patch:
Repository: googleapis/python-spanner-django

Issue:
django: implement SchemaEditor.quote_value()
For example:
```
======================================================================
ERROR: test_extra_args (schema.test_logging.SchemaLoggerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tim/code/django/tests/schema/test_logging.py", line 12, in test_extra_args
    editor.execute(sql, params)
  File "/home/tim/code/django/django/db/backends/base/schema.py", line 132, in execute
    self.collected_sql.append((sql % tuple(map(self.quote_value, params))) + ending)
  File "/home/tim/code/django/django/db/backends/base/schema.py", line 243, in quote_value
    raise NotImplementedError()
NotImplementedError
```


Discussion:
The docstring of this method says, "Return a quoted version of the value so it's safe to use in an SQL string. This is not safe against injection from user code; it is intended only for use in making SQL scripts or preparing default values for particularly tricky backends (defaults are not user-defined, though, so this is safe)."

Some implementations on other databases:

SQLite: `sqlite3.adapt()`
MySQL: `self.connection.connection.escape(value, self.connection.connection.encoders)`
PostgreSQL: `psycopg2.extensions.adapt(value).getquoted().decode()`

@odeke-em, do you know if Spanner's Python libraries provides anything similar? If not, we could consider an implementation in Python similar to what the [Oracle backend does](https://github.com/django/django/blob/828e3b1335e2614d338a630fd5b5f88d38a6b5d2/django/db/backends/oracle/schema.py#L22-L32).
Unfortunately, I don't think that the Python client libraries support quoting values, we might have to add that function ourselves. Looking at the Cloud Spanner advisory on string and byte literals, they have a section on quoting https://cloud.google.com/spanner/docs/lexical#string-and-bytes-literals
It looks like the usage of `quote_value()` is limited to conditional indexes and constraints which Spanner doesn't support, so I don't think this is high priority.
```
django/db/models/indexes.py:        return sql % tuple(schema_editor.quote_value(p) for p in params)
django/db/models/constraints.py:        return sql % tuple(schema_editor.quote_value(p) for p in params)
```

Resolving patch:
diff --git a/django_spanner/schema.py b/django_spanner/schema.py
--- a/django_spanner/schema.py
+++ b/django_spanner/schema.py
@@ -199,8 +199,7 @@ def add_index(self, model, index):
         super().add_index(model, index)
 
     def quote_value(self, value):
-        # TODO: a real implementation:
-        # https://github.com/orijtech/django-spanner/issues/227
+        # A more complete implementation isn't currently required.
         return str(value)
 
     def _alter_field(self, model, old_field, new_field, old_type, new_type,


Test patch: