Repository: apache/airflow

Issue:
Airflow database upgrade fails with "psycopg2.errors.NotNullViolation: column "map_index" of relation "task_instance" contains null value"s
### Apache Airflow version

main (development)

### What happened

I currently have Airflow 2.2.3 and due to this [issue](https://github.com/apache/airflow/issues/19699) I have tried to upgrade Airflow to this [commit](https://github.com/apache/airflow/commit/14ee831c7ad767e31a3aeccf3edbc519b3b8c923).

When I run `airflow db upgrade` I get the following error:
```
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade 587bdf053233 -> e655c0453f75, Add TaskMap and map_index on TaskInstance.
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.NotNullViolation: column "map_index" of relation "task_instance" contains null values
```

The map_index column was introduced with this [PR](https://github.com/apache/airflow/pull/20286).

Could you please advise? 

### What you expected to happen

_No response_

### How to reproduce

_No response_

### Operating System

Ubuntu 18.04.6 LTS

### Versions of Apache Airflow Providers

_No response_

### Deployment

Other

### Deployment details

Kubernetes

### Anything else

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)



Discussion:
Thanks for opening your first issue here! Be sure to follow the issue template!

Map Index hasn't been introduced in Airflow 2.2.3 and nor intended to, it is currently available in the main branch and will be released in Airflow 2.3. So it is very likely you somehow used some code from the main branch.

Check https://airflow.apache.org/docs/apache-airflow/stable/migrations-ref.html -- that migration isn't available in 2.2.3

Double verify by running the following:

```
AIRFLOW_VERSION=2.2.3
PYTHON_VERSION="$(python --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
pip install "apache-airflow[async,postgres,google]==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
```
Thank you for your answer.

Yes, I was using the code from the main branch.

Is there an estimate as to when Airflow 2.3 will be released?
@dcardinha Nothing firm, but our aim is "in about a month"

Resolving patch:
diff --git a/airflow/migrations/versions/e655c0453f75_add_taskmap_and_map_id_on_taskinstance.py b/airflow/migrations/versions/e655c0453f75_add_taskmap_and_map_id_on_taskinstance.py
--- a/airflow/migrations/versions/e655c0453f75_add_taskmap_and_map_id_on_taskinstance.py
+++ b/airflow/migrations/versions/e655c0453f75_add_taskmap_and_map_id_on_taskinstance.py
@@ -24,7 +24,7 @@
 """
 
 from alembic import op
-from sqlalchemy import Column, ForeignKeyConstraint, Integer
+from sqlalchemy import Column, ForeignKeyConstraint, Integer, text
 
 from airflow.models.base import StringID
 from airflow.utils.sqlalchemy import ExtendedJSON
@@ -47,12 +47,12 @@ def upgrade():
     with op.batch_alter_table("task_instance") as batch_op:
         # I think we always use this name for TaskInstance after 7b2661a43ba3?
         batch_op.drop_constraint("task_instance_pkey", type_="primary")
-        batch_op.add_column(Column("map_index", Integer, nullable=False, default=-1))
+        batch_op.add_column(Column("map_index", Integer, nullable=False, server_default=text("-1")))
         batch_op.create_primary_key("task_instance_pkey", ["dag_id", "task_id", "run_id", "map_index"])
 
     # Re-create task_reschedule's constraints.
     with op.batch_alter_table("task_reschedule") as batch_op:
-        batch_op.add_column(Column("map_index", Integer, nullable=False, default=-1))
+        batch_op.add_column(Column("map_index", Integer, nullable=False, server_default=text("-1")))
         batch_op.create_foreign_key(
             "task_reschedule_ti_fkey",
             "task_instance",
diff --git a/airflow/models/taskinstance.py b/airflow/models/taskinstance.py
--- a/airflow/models/taskinstance.py
+++ b/airflow/models/taskinstance.py
@@ -60,6 +60,7 @@
     func,
     inspect,
     or_,
+    text,
     tuple_,
 )
 from sqlalchemy.ext.associationproxy import association_proxy
@@ -343,7 +344,7 @@ class TaskInstance(Base, LoggingMixin):
     task_id = Column(String(ID_LEN, **COLLATION_ARGS), primary_key=True, nullable=False)
     dag_id = Column(String(ID_LEN, **COLLATION_ARGS), primary_key=True, nullable=False)
     run_id = Column(String(ID_LEN, **COLLATION_ARGS), primary_key=True, nullable=False)
-    map_index = Column(Integer, primary_key=True, nullable=False, default=-1)
+    map_index = Column(Integer, primary_key=True, nullable=False, server_default=text("-1"))
 
     start_date = Column(UtcDateTime)
     end_date = Column(UtcDateTime)
diff --git a/airflow/models/taskreschedule.py b/airflow/models/taskreschedule.py
--- a/airflow/models/taskreschedule.py
+++ b/airflow/models/taskreschedule.py
@@ -20,7 +20,7 @@
 import datetime
 from typing import TYPE_CHECKING
 
-from sqlalchemy import Column, ForeignKeyConstraint, Index, Integer, String, asc, desc
+from sqlalchemy import Column, ForeignKeyConstraint, Index, Integer, String, asc, desc, text
 from sqlalchemy.ext.associationproxy import association_proxy
 from sqlalchemy.orm import relationship
 
@@ -41,7 +41,7 @@ class TaskReschedule(Base):
     task_id = Column(String(ID_LEN, **COLLATION_ARGS), nullable=False)
     dag_id = Column(String(ID_LEN, **COLLATION_ARGS), nullable=False)
     run_id = Column(String(ID_LEN, **COLLATION_ARGS), nullable=False)
-    map_index = Column(Integer, nullable=False, default=-1)
+    map_index = Column(Integer, nullable=False, server_default=text("-1"))
     try_number = Column(Integer, nullable=False)
     start_date = Column(UtcDateTime, nullable=False)
     end_date = Column(UtcDateTime, nullable=False)


Test patch: