Repository: mlflow/mlflow

Issue:
I cannot see the model signature and input_example in MLmodel file of a spark flavor model
Hi, I am wondering if I have miss something when I try to log model with signature and input example? Can anyone help me please. Thank you!

My tracking-uri and backend-store-uri are the same mysql database uri
and default-artifact-root is ./mlruns

I am trying to train an ALS model using pyspark2 ml package and log it by mlflow,
but on the mlflow ui http://<ip_address>:<port>, I cannot see the model signature and input_example in MLmodel file.

And I don't see the difference for the one I didn't log model with signature and the one I did, in the process of "train -> models serve -> send data in json file and get prediction ". Could someone tell me how can I see the difference.
The information of model signature and input_example should be there as I see for model of sklearn flavor.

<img width="902" alt="Screen Shot 2020-07-22 at 2 55 09 PM" src="https://user-images.githubusercontent.com/60844171/88144342-58945e80-cc2b-11ea-8e54-b685952d7ee2.png">

<img width="1021" alt="Screen Shot 2020-07-22 at 3 25 39 PM" src="https://user-images.githubusercontent.com/60844171/88147330-9c896280-cc2f-11ea-9693-aec10c2470b3.png">

<img width="1294" alt="Screen Shot 2020-07-22 at 2 56 45 PM" src="https://user-images.githubusercontent.com/60844171/88144486-91ccce80-cc2b-11ea-9966-a0561d2c2e45.png">




Discussion:
@Astonzzh Thanks for filing this issue. I was able to reproduce the issue.

```python
from pyspark.sql import SparkSession
from pyspark.ml.recommendation import ALS

import mlflow
import mlflow.spark
from mlflow.models.signature import infer_signature
from pyspark.ml import Pipeline

spark = (
    SparkSession
    .builder
    .getOrCreate()
)

# prepare train and test data
train = spark.createDataFrame(
    [(0, 0, 4.0),
     (0, 1, 2.0),
     (1, 1, 3.0)],
    ["user", "item", "rating"],
)

test = spark.createDataFrame(
    [(0, 2), (1, 0), (2, 0)],
    ["user", "item"]
)

# train model
als = ALS(rank=10, maxIter=10, regParam=0.1, userCol="user", itemCol="item", ratingCol="rating")
pipeline = Pipeline(stages=[als])
model = pipeline.fit(train)

# create signature and example
signature = infer_signature(test, model.transform(test))
example_dict = {'user': 0, 'item': 1}

# log model
mlflow.spark.log_model(
    model,
    'small_ALS',
    signature=signature,
    input_example=example_dict
)

```
I found `signature` and `input_example` are not passed to `Model.log`.

https://github.com/mlflow/mlflow/blob/c2708b5735354e10ec052164253eb74243c5603f/mlflow/spark.py#L174-L177
I have confirmed that adding missing `input_example` and `signature` solves this issue.

```diff
    if is_local_uri(run_root_artifact_uri):
        return Model.log(artifact_path=artifact_path, flavor=mlflow.spark, spark_model=spark_model,
                         conda_env=conda_env, dfs_tmpdir=dfs_tmpdir, sample_input=sample_input,
                         registered_model_name=registered_model_name,
+                        signature=signature,
+                        input_example=input_example)
```

![Screen Shot 2020-07-22 at 17 01 08](https://user-images.githubusercontent.com/17039389/88150972-0e1bdd80-cc3d-11ea-8070-7a490c32b67b.png)

![Screen Shot 2020-07-22 at 17 04 52](https://user-images.githubusercontent.com/17039389/88151371-939f8d80-cc3d-11ea-80a6-7aaa250dd3d6.png)


Resolving patch:
diff --git a/mlflow/spark.py b/mlflow/spark.py
--- a/mlflow/spark.py
+++ b/mlflow/spark.py
@@ -174,7 +174,8 @@ def log_model(spark_model, artifact_path, conda_env=None, dfs_tmpdir=None,
     if is_local_uri(run_root_artifact_uri):
         return Model.log(artifact_path=artifact_path, flavor=mlflow.spark, spark_model=spark_model,
                          conda_env=conda_env, dfs_tmpdir=dfs_tmpdir, sample_input=sample_input,
-                         registered_model_name=registered_model_name)
+                         registered_model_name=registered_model_name,
+                         signature=signature, input_example=input_example)
     # If Spark cannot write directly to the artifact repo, defer to Model.log() to persist the
     # model
     model_dir = os.path.join(run_root_artifact_uri, artifact_path)


Test patch:
diff --git a/tests/spark/test_spark_model_export.py b/tests/spark/test_spark_model_export.py
--- a/tests/spark/test_spark_model_export.py
+++ b/tests/spark/test_spark_model_export.py
@@ -215,6 +215,29 @@ def test_model_export_with_signature_and_examples(iris_df, spark_model_iris):
                     assert all((_read_example(mlflow_model, path) == example).all())
 
 
+@pytest.mark.large
+def test_log_model_with_signature_and_examples(iris_df, spark_model_iris):
+    _, _, iris_spark_df = iris_df
+    signature_ = infer_signature(iris_spark_df)
+    example_ = iris_spark_df.toPandas().head(3)
+    artifact_path = "model"
+    for signature in (None, signature_):
+        for example in (None, example_):
+            with mlflow.start_run():
+                sparkm.log_model(spark_model_iris.model,
+                                 artifact_path=artifact_path,
+                                 signature=signature,
+                                 input_example=example)
+                artifact_uri = mlflow.get_artifact_uri()
+                model_path = os.path.join(artifact_uri, artifact_path)
+                mlflow_model = Model.load(model_path)
+                assert signature == mlflow_model.signature
+                if example is None:
+                    assert mlflow_model.saved_input_example_info is None
+                else:
+                    assert all((_read_example(mlflow_model, model_path) == example).all())
+
+
 @pytest.mark.large
 def test_estimator_model_export(spark_model_estimator, model_path, spark_custom_env):
     sparkm.save_model(spark_model_estimator.model, path=model_path, conda_env=spark_custom_env)