Repository: mlflow/mlflow

Issue:
[SETUP-BUG] Table schema in latest_schema.sql gives errors on Microsoft SQL Server
Thank you for submitting an issue. Please refer to our [issue policy](https://www.github.com/mlflow/mlflow/blob/master/ISSUE_POLICY.md)
for information on what types of issues we address.

Please fill in this template and do not delete it unless you are sure your issue is outside its scope.

### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**:  Ubuntu 18.4 / Microsoft Windows 10 
- **MLflow installed from (source or binary)**:	source
- **MLflow version (run ``mlflow --version``)**:	1.5.0
- **Python version**:					        Python 3.7.3
- **Database version**:				                Microsoft SQL Azure (RTM) - 12.0.2000.8		
- **Exact command to reproduce**:

Run the schema from tests/resources/db/latest_schema.sql on Microsoft SQL database.
For example:

•	Start Microsoft SQL Server Management Studio
•	Connect to the database where mlflow tables should be created
•	Start SQL editor (New Query)
•	Cut and paste code from latest_schema.sql (or just the code snippet below) into the SQL editor
•	Execute the SQL

CREATE TABLE metrics_test (
        key VARCHAR(250) NOT NULL,
        value FLOAT NOT NULL,
        timestamp BIGINT NOT NULL,
        run_uuid VARCHAR(32) NOT NULL,
        step BIGINT DEFAULT '0' NOT NULL,
        is_nan BOOLEAN DEFAULT '0' NOT NULL,
        CONSTRAINT metric_pk PRIMARY KEY (key, value, timestamp, run_uuid, step, is_nan),
        FOREIGN KEY(run_uuid) REFERENCES runs (run_uuid)
)

Errors encountered are displayed below. 

Issue 1:  Column names:

key VARCHAR(250) NOT NULL,
Msg 156, Level 15, State 1, Line 11
Incorrect syntax near the keyword 'key'.

Fix applied:	Use square brackets for keywords, for example:

                        [key] VARCHAR(250) NOT NULL,

Issue 2:  Boolean data type:

is_nan BOOLEAN DEFAULT '0' NOT NULL,
Msg 2715, Level 16, State 6, Line 90
Column, parameter, or variable #5: Cannot find data type BOOLEAN.

Fix applied:	Use data type BIT instead of BOOLEAN, for example:

                        is_nan BIT DEFAULT '0' NOT NULL,

Abstract

The table schema in tests/resources/db/latest_schema.sql gives errors when Microsoft SQL database is used as the backend store.
What database types are supported by mlflow? Is Microsoft SQL Server database supported?
Can we amend the mlflow database schema so it works with Microsoft SQL database?

Additional information

I have successfully installed mlflow version 0.91 in May 2019 but had to amend the table schema first as I was getting syntax errors. I am now trying to install the latest version and still have issues.

One of the issues I encountered last year was with 'lifecycle_stage' column being duplicated in two tables, experiments and runs. I can see this has now been corrected in version 1.0. Thank you.

I am still encountering syntax errors when the table column use SQL reserved words i.e. ‘key’.
In addition, Microsoft SQL Server does not support the BOOLEAN type.
Is Microsoft SQL database supported as the backend store in mlflow?



Discussion:
@arees88 Copying the `latest_schema.sql` file should not be necessary / is not the recommended approach for using the SQL Tracking backend. Can you try setting your MLflow tracking URI to point to a fresh SQLServer database via `mlflow.set_tracking_uri()` and then attempt to log to MLflow?
@dbczumar Many thanks for your prompt response.
May I confirm that if I set the MLFLOW_TRACKING_URI environment variable to point to a fresh SQLServer database, then all the required tables will be automatically created in the database when I try to log to MLflow for the first time, e.g. call mlflow.log_param() from the python code?
B.T.W, Which part of the source code is invoked to create the new tables in the DB?
@arees88 That's correct! For reference, please check out the constructor of the `SqlAlchemyStore` class (https://github.com/mlflow/mlflow/blob/07d2b4a1a13f9245d1519423ad775a2e064faecb/mlflow/store/tracking/sqlalchemy_store.py#L66). I'm going to go ahead and close this issue; feel free to re-open it if you're still running into issues.
@dbczumar That's what I did last year with 0.91 but did not get very far because of the  'lifecycle_stage' column being duplicated bug. Hence I created the tables directly on the DB and after a few other fixes in the code everything worked fine. 

Re: 1.5.0
I have run one of the example scripts to initialise the tables in the database:

python examples/sklearn_elasticnet_wine/train.py

I did not get very far this time either. The execution stopped with the NotImplementedError exception in:
lib\site-packages\sqlalchemy\engine\interfaces.py", line 413, in get_check_constraints
      raise NotImplementedError()

Looks like the implementation for SQLAlchemy interface methods is missing or not found.

Please note, I have initialised the MLFLOW_TRACKING_URI  to point to an empty DB, so not sure why all the migration steps were triggered, as it is seen in the trace below:

2020/01/23 11:28:25 INFO mlflow.store.db.utils: Creating initial MLflow database tables...
2020/01/23 11:28:25 INFO mlflow.store.db.utils: Updating database tables at mssql://<cut>
INFO  [alembic.runtime.migration] Context impl MSSQLImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> 451aebb31d03, add metric step
INFO  [alembic.runtime.migration] Running upgrade 451aebb31d03 -> 90e64c465722, migrate user column to tags
INFO  [alembic.runtime.migration] Running upgrade 90e64c465722 -> 181f10493468, allow nulls for metric values
INFO  [alembic.runtime.migration] Running upgrade 181f10493468 -> df50e92ffc5e, Add Experiment Tags Table
INFO  [alembic.runtime.migration] Running upgrade df50e92ffc5e -> 7ac759974ad8, Update run tags with larger limit
INFO  [alembic.runtime.migration] Running upgrade 7ac759974ad8 -> 89d4b8295536, create latest metrics table
INFO  [89d4b8295536_create_latest_metrics_table_py] Migration complete!
INFO  [alembic.runtime.migration] Running upgrade 89d4b8295536 -> 2b4d017a5e9b, add model registry tables to db
INFO  [2b4d017a5e9b_add_model_registry_tables_to_db_py] Adding registered_models and model_versions tables to database.
INFO  [2b4d017a5e9b_add_model_registry_tables_to_db_py] Migration complete!
INFO  [alembic.runtime.migration] Running upgrade 2b4d017a5e9b -> cfd24bdc0731, Update run status constraint with killed
Traceback (most recent call last):
  File "examples/sklearn_elasticnet_wine/train.py", line 56, in <module>
    with mlflow.start_run():
. . .
  File "C:\Users\AgnieszkaRees\Anaconda3\envs\mlflow_last\lib\site-packages\mlflow\store\db_migrations\versions\cfd24bdc0731_update_run_status_constraint_with_killed.py", line 38, in _has_check_constraints
    for constraint in inspector.get_check_constraints("runs")]
  File "C:\Users\AgnieszkaRees\Anaconda3\envs\mlflow_last\lib\site-packages\sqlalchemy\engine\reflection.py", line 569, in get_check_constraints
    self.bind, table_name, schema, info_cache=self.info_cache, **kw
  File "C:\Users\AgnieszkaRees\Anaconda3\envs\mlflow_last\lib\site-packages\sqlalchemy\engine\interfaces.py", line 413, in get_check_constraints
    raise NotImplementedError()
NotImplementedError

@arees88 Are you running the latest version of sqlalchemy? If not, can you try upgrading?
@dbczumar I installed sqlalchemy-1.3.12 today again
@arees88 I'll take a look at this as soon as possible.

Resolving patch:
diff --git a/mlflow/store/db_migrations/versions/0a8213491aaa_drop_duplicate_killed_constraint.py b/mlflow/store/db_migrations/versions/0a8213491aaa_drop_duplicate_killed_constraint.py
new file mode 100644
--- /dev/null
+++ b/mlflow/store/db_migrations/versions/0a8213491aaa_drop_duplicate_killed_constraint.py
@@ -0,0 +1,42 @@
+"""drop_duplicate_killed_constraint
+
+Revision ID: 0a8213491aaa
+Revises: cfd24bdc0731
+Create Date: 2020-01-28 15:26:14.757445
+
+This migration drops a duplicate constraint on the `runs.status` column that was left as a byproduct
+of an erroneous implementation of the `cfd24bdc0731_update_run_status_constraint_with_killed`
+migration in MLflow 1.5. The implementation of this migration has since been fixed.
+"""
+import logging
+from alembic import op
+
+_logger = logging.getLogger(__name__)
+
+# revision identifiers, used by Alembic.
+revision = '0a8213491aaa'
+down_revision = 'cfd24bdc0731'
+branch_labels = None
+depends_on = None
+
+
+def upgrade():
+    # Attempt to drop any existing `status` constraints on the `runs` table. This operation
+    # may fail against certain backends with different classes of Exception. For example,
+    # in MySQL <= 8.0.15, dropping constraints produces an invalid `ALTER TABLE` expression.
+    # Further, in certain versions of sqlite, `ALTER` (which is invoked by `drop_constraint`)
+    # is unsupported on `CHECK` constraints. Accordingly, we catch the generic `Exception`
+    # object because the failure modes are not well-enumerated or consistent across database
+    # backends. Because failures automatically stop batch operations and the `drop_constraint()`
+    # operation is expected to fail under certain circumstances, we execute `drop_constraint()`
+    # outside of the batch operation context.
+    try:
+        op.drop_constraint(constraint_name="status", table_name="runs", type_="check")
+    except Exception as e: # pylint: disable=broad-except
+        _logger.warning(
+            "Failed to drop check constraint. Dropping check constraints may not be supported"
+            " by your SQL database. Exception content: %s", e)
+
+
+def downgrade():
+    pass
diff --git a/mlflow/store/db_migrations/versions/cfd24bdc0731_update_run_status_constraint_with_killed.py b/mlflow/store/db_migrations/versions/cfd24bdc0731_update_run_status_constraint_with_killed.py
--- a/mlflow/store/db_migrations/versions/cfd24bdc0731_update_run_status_constraint_with_killed.py
+++ b/mlflow/store/db_migrations/versions/cfd24bdc0731_update_run_status_constraint_with_killed.py
@@ -5,10 +5,11 @@
 Create Date: 2019-10-11 15:55:10.853449
 
 """
-from alembic import op, context
-from mlflow.entities import RunStatus
+from alembic import op
+from mlflow.entities import RunStatus, ViewType
+from mlflow.entities.lifecycle_stage import LifecycleStage
+from mlflow.store.tracking.dbmodels.models import SqlRun, SourceTypes
 from sqlalchemy import CheckConstraint, Enum
-from sqlalchemy.engine.reflection import Inspector
 
 # revision identifiers, used by Alembic.
 revision = 'cfd24bdc0731'
@@ -16,7 +17,7 @@
 branch_labels = None
 depends_on = None
 
-new_run_status = [
+new_run_statuses = [
     RunStatus.to_string(RunStatus.SCHEDULED),
     RunStatus.to_string(RunStatus.FAILED),
     RunStatus.to_string(RunStatus.FINISHED),
@@ -24,51 +25,31 @@
     RunStatus.to_string(RunStatus.KILLED)
 ]
 
-previous_run_status = [
-    RunStatus.to_string(RunStatus.SCHEDULED),
-    RunStatus.to_string(RunStatus.FAILED),
-    RunStatus.to_string(RunStatus.FINISHED),
-    RunStatus.to_string(RunStatus.RUNNING)
+# Certain SQL backends (e.g., SQLite) do not preserve CHECK constraints during migrations.
+# For these backends, CHECK constraints must be specified as table arguments. Here, we define
+# the collection of CHECK constraints that should be preserved when performing the migration.
+# The "status" constraint is excluded from this set because it is explicitly modified
+# within the migration's `upgrade()` routine.
+check_constraint_table_args = [
+    CheckConstraint(SqlRun.source_type.in_(SourceTypes), name='source_type'),
+    CheckConstraint(SqlRun.lifecycle_stage.in_(LifecycleStage.view_type_to_stages(ViewType.ALL)),
+                    name='runs_lifecycle_stage'),
 ]
 
 
-def _has_check_constraints():
-    inspector = Inspector(context.get_bind().engine)
-    constraints = [constraint['name']
-                   for constraint in inspector.get_check_constraints("runs")]
-    return "status" in constraints
-
-
-def get_check_constraints():
-    """
-    Get all check constraint except status
-    """
-    inspector = Inspector(context.get_bind().engine)
-    constraints = [CheckConstraint(constraint['sqltext'], name=constraint['name'])
-                   for constraint in inspector.get_check_constraints("runs")
-                   if constraint['name'] != 'status']
-    return constraints
-
-
 def upgrade():
-    if _has_check_constraints():
-        with op.batch_alter_table("runs", table_args=get_check_constraints()) as batch_op:
-            # Update the "status" constraint via the SqlAlchemy `Enum` type. Specify
-            # `native_enum=False` to create a check constraint rather than a
-            # database-backend-dependent enum (see https://docs.sqlalchemy.org/en/13/core/
-            # type_basics.html#sqlalchemy.types.Enum.params.native_enum)
-            batch_op.alter_column("status",
-                                  type_=Enum(*new_run_status, create_constraint=True,
-                                             constraint_name="status", native_enum=False))
+    with op.batch_alter_table("runs", table_args=check_constraint_table_args) as batch_op:
+        # Transform the "status" column to an `Enum` and define a new check constraint. Specify
+        # `native_enum=False` to create a check constraint rather than a
+        # database-backend-dependent enum (see https://docs.sqlalchemy.org/en/13/core/
+        # type_basics.html#sqlalchemy.types.Enum.params.native_enum)
+        batch_op.alter_column("status",
+                              type_=Enum(*new_run_statuses, create_constraint=True,
+                                         native_enum=False))
 
 
 def downgrade():
-    if _has_check_constraints():
-        with op.batch_alter_table("runs", table_args=get_check_constraints()) as batch_op:
-            # Update the "status" constraint via the SqlAlchemy `Enum` construct. Specify
-            # `native_enum=False` to create a check constraint rather than a
-            # database-backend-dependent enum (see https://docs.sqlalchemy.org/en/13/core/
-            # type_basics.html#sqlalchemy.types.Enum.params.native_enum)
-            batch_op.alter_column("status",
-                                  type_=Enum(*previous_run_status, create_constraint=True,
-                                             constraint_name="status", native_enum=False))
+    # Omit downgrade logic for now - we don't currently provide users a command/API for
+    # reverting a database migration, instead recommending that they take a database backup
+    # before running the migration.
+    pass


Test patch: