Repository: keras-team/autokeras

Issue:
Public datasets list
This is a list of public datasets, that we need to ensure AutoKeras is having good performances on.
Welcome to post more.

| Name | API | Code | Metric | Results |
| - | - | - | - | - |
| [MNIST](http://yann.lecun.com/exdb/mnist/)  | ImageClassifier| [code](https://github.com/keras-team/autokeras/blob/master/benchmark/mnist.py) | Accuracy | 99.04% |
| [CIFAR10](https://www.cs.toronto.edu/~kriz/cifar.html)   | ImageClassifier|[code](https://github.com/keras-team/autokeras/blob/master/benchmark/cifar10.py) |Accuracy | 97.10% |
| [CIFAR100](https://www.cs.toronto.edu/~kriz/cifar.html)  | ImageClassifier | | Accuracy |
| [IMDB](https://ai.stanford.edu/~amaas/data/sentiment/)  | TextClassifier |[code](https://github.com/keras-team/autokeras/blob/master/benchmark/imdb.py) | Accuracy | 93.93% |
| [Titanic](https://www.tensorflow.org/datasets/catalog/titanic)  | StructuredDataClassifier |  [code](https://github.com/keras-team/autokeras/blob/master/benchmark/imdb.py)| Accuracy | 82.20% |
| [Wine](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_wine.html#sklearn.datasets.load_wine)  | StructuredDataClassifier | | Accuracy|
| [Iris](https://archive.ics.uci.edu/ml/datasets/iris) | StructuredDataClassifier | | Accuracy |
| [California Housing](https://scikit-learn.org/stable/datasets/index.html#california-housing-dataset)  | StructuredDataRegression |  [code](https://github.com/keras-team/autokeras/blob/master/benchmark/housing.py) | MSE | 0.23
| [Houses](https://github.com/emanhamed/Houses-dataset)  | IO API | | MSE| | 


Discussion:
@haifeng-jin Can I take up few of the datasets from the list and work on them? Let me know which ones are yet to be tested out.
@mandalbiswadip Would you help us test the Wine dataset? Thank you!
@haifeng-jin 
As you commented in the previous issue #1283, I created a pull request.
Would you review and merge it?
Thanks.
Can I try [`Reuters` dataset](https://keras.io/ja/datasets/#_5)?
Or are there any prioritized dataset? @haifeng-jin 
@toohsk Yes, please. Thank you! I think it is good to try these Keras dataset first.

Resolving patch:
diff --git a/examples/reuters.py b/examples/reuters.py
new file mode 100644
--- /dev/null
+++ b/examples/reuters.py
@@ -0,0 +1,73 @@
+"""shell
+!pip install -q -U pip
+!pip install -q -U autokeras==1.0.8
+!pip install -q git+https://github.com/keras-team/keras-tuner.git@1.0.2rc1
+"""
+
+"""
+Search for a good model for the
+[Reuters](https://keras.io/ja/datasets/#_5) dataset.
+"""
+
+import tensorflow as tf
+from tf.keras.datasets import reuters
+import numpy as np
+
+import autokeras as ak
+
+# Prepare the dataset.
+def reuters_raw(max_features=20000):
+
+    index_offset = 3  # word index offset
+
+    (x_train, y_train), (x_test, y_test) = tf.keras.datasets.reuters.load_data(
+        num_words=max_features,
+        index_from=index_offset)
+    x_train = x_train
+    y_train = y_train.reshape(-1, 1)
+    x_test = x_test
+    y_test = y_test.reshape(-1, 1)
+
+    word_to_id = tf.keras.datasets.reuters.get_word_index()
+    word_to_id = {k: (v + index_offset) for k, v in word_to_id.items()}
+    word_to_id["<PAD>"] = 0
+    word_to_id["<START>"] = 1
+    word_to_id["<UNK>"] = 2
+
+    id_to_word = {value: key for key, value in word_to_id.items()}
+    x_train = list(map(lambda sentence: ' '.join(
+        id_to_word[i] for i in sentence), x_train))
+    x_test = list(map(lambda sentence: ' '.join(
+        id_to_word[i] for i in sentence), x_test))
+    x_train = np.array(x_train, dtype=np.str)
+    x_test = np.array(x_test, dtype=np.str)
+    return (x_train, y_train), (x_test, y_test)
+
+
+# Prepare the data.
+(x_train, y_train), (x_test, y_test) = reuters_raw()
+print(x_train.shape)  # (8982,)
+print(y_train.shape)  # (8982, 1)
+print(x_train[0][:50])  # <START> <UNK> <UNK> said as a result of its decemb
+
+# Initialize the TextClassifier
+clf = ak.TextClassifier(
+    max_trials=5,
+    overwrite=True,
+)
+
+# Callback to avoid overfitting with the EarlyStopping.
+cbs = [
+    tf.keras.callbacks.EarlyStopping(patience=3),
+]
+
+# Search for the best model.
+clf.fit(
+    x_train,
+    y_train,
+    epochs=10,
+    callback=cbs
+)
+
+# Evaluate on the testing data.
+print('Accuracy: {accuracy}'.format(accuracy=clf.evaluate(x_test, y_test)))


Test patch: