internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: keras-team/autokeras

Issue:
PortableImageSupervised.predict is slow with many images
# Bug Description (maybe feature enhancement.)
PortableImageSupervised.predict method on the cifar10 test dataset is a bit slower when test dataset size is bigger. The number of images in the dataset is 10000.
Maybe predict should be run on gpu like the code in the url above?
https://github.com/jhfjhfj1/autokeras/compare/master...sammy-yusuke:0e71db8dab4841a577c7943daf4bfdb19acbfc0c

### Reproducing Steps
1. Place all files in the URL below into some dir
https://gist.github.com/sammy-yusuke/bc8aa440d2fdbe3908198fdcf82442e2
2. Train model
```
python train_cifar10.py --timelimit-h 0.5 --dataset cifar10
```
3. Test model
```
python test_cifar10.py --training-output-path ~/nas_try/${trained_date}
```
### Expected Behavior
Step 3 should finish in at least 30 min.

### Setup Details
Include the details about the versions of:
 - OS type and version: Description:    Ubuntu 16.04.5 LTS
 - Python: 3.6.6
 - autokeras: master
 - scikit-learn:0.19.1
 - numpy:1.14.5
 - keras:2.2.2
 - scipy:1.1.0
 - tensorflow:1.10.0
 - pytorch:0.4.1

### Additional context
I was just trying to see all the results of all searched models by autokeras.


Discussion:
@satyakesav Would you please fix this issue?

I think it can be solved by moving this line https://github.com/jhfjhfj1/autokeras/blob/master/autokeras/image/image_supervised.py#L279
to the predict() function.

Resolving patch:
diff --git a/autokeras/image/image_supervised.py b/autokeras/image/image_supervised.py
--- a/autokeras/image/image_supervised.py
+++ b/autokeras/image/image_supervised.py
@@ -259,6 +259,7 @@ def predict(self, x_test):
         if Constant.LIMIT_MEMORY:
             pass
 
+        x_test = resize_image_data(x_test, self.resize_shape)
         test_loader = self.data_transformer.transform_test(x_test)
         model = self.graph.produce_model()
         model.eval()
@@ -275,6 +276,5 @@ def inverse_transform_y(self, output):
 
     def evaluate(self, x_test, y_test):
         """Return the accuracy score between predict value and `y_test`."""
-        x_test = resize_image_data(x_test, self.resize_shape)
         y_predict = self.predict(x_test)
         return self.metric().evaluate(y_test, y_predict)


Test patch:
Repository: keras-team/autokeras

Issue:
improve code quality using Codacy
<!---
**If you are reporting a bug:**
*  Verify that your issue is not being currently addressed by other issues or pull requests.
*  Please note that Auto-Keras is only compatible with **Python 3.6**.
*  Tag the issue with the `bug report` tag.
-->

### Bug Description
<!---
A clear and concise description of what the bug is.
-->
We are now using Codacy and CodeClimate to improve our code quality.

Please try to solve anything reported by Codacy by following this [link](https://app.codacy.com/manual/jhfjhfj1/autokeras/issues/index).
On the page, you can see a list of **issues**.
You can click to see the details and suggestions to fix the issue.

Thanks.


Discussion:

Resolving patch:
diff --git a/autokeras/pretrained/voice_generator/deepvoice3_pytorch/conv.py b/autokeras/pretrained/voice_generator/deepvoice3_pytorch/conv.py
--- a/autokeras/pretrained/voice_generator/deepvoice3_pytorch/conv.py
+++ b/autokeras/pretrained/voice_generator/deepvoice3_pytorch/conv.py
@@ -12,29 +12,29 @@ def __init__(self, *args, **kwargs):
         self.clear_buffer()
         self._linearized_weight = None
 
-    def incremental_forward(self, input):
+    def incremental_forward(self, input_data):
 
         # reshape weight
         weight = self._get_linearized_weight()
         kw = self.kernel_size[0]
         dilation = self.dilation[0]
 
-        bsz = input.size(0)  # input: bsz x len x dim
+        bsz = input_data.size(0)  # conv_input: bsz x len x dim
         if kw > 1:
-            input = input.data
+            input_data = input_data.data
             if self.input_buffer is None:
-                self.input_buffer = input.new(bsz, kw + (kw - 1) * (dilation - 1), input.size(2))
+                self.input_buffer = input_data.new(bsz, kw + (kw - 1) * (dilation - 1), input_data.size(2))
                 self.input_buffer.zero_()
             else:
                 # shift buffer
                 self.input_buffer[:, :-1, :] = self.input_buffer[:, 1:, :].clone()
             # append next input
-            self.input_buffer[:, -1, :] = input[:, -1, :]
-            input = self.input_buffer
+            self.input_buffer[:, -1, :] = input_data[:, -1, :]
+            input_data = self.input_buffer
             if dilation > 1:
-                input = input[:, 0::dilation, :].contiguous()
-        output = F.linear(input.view(bsz, -1), weight, self.bias)
-        return output.view(bsz, 1, -1)
+                input_data = input_data[:, 0::dilation, :].contiguous()
+        input_data = F.linear(input_data.view(bsz, -1), weight, self.bias)
+        return input_data.view(bsz, 1, -1)
 
     def clear_buffer(self):
         self.input_buffer = None


Test patch:
Repository: keras-team/autokeras

Issue:
Evaluation criteria for MLP
Are there any evaluation criterias for MLP module in Autokeras? 



Discussion:
Same question. After final_fit the MLP module, I'm wondering how to evaluate it performace since it doess't support 'evaluate' or 'predict'. 
@droidadroit Would you please take a look into this issue? Thanks.

Resolving patch:
diff --git a/autokeras/net_module.py b/autokeras/net_module.py
--- a/autokeras/net_module.py
+++ b/autokeras/net_module.py
@@ -11,6 +11,7 @@
 
 from autokeras.utils import pickle_to_file, rand_temp_folder_generator, ensure_dir
 from autokeras.nn.generator import CnnGenerator, MlpGenerator, ResNetGenerator, DenseNetGenerator
+from autokeras.utils import get_device
 
 
 class NetworkModule:
@@ -123,6 +124,24 @@ def predict(self, test_loader):
         output = reduce(lambda x, y: np.concatenate((x, y)), outputs)
         return output
 
+    def evaluate(self, test_data):
+        """Evaluate the performance of the best architecture in terms of the loss.
+
+        Args:
+            test_data: A DataLoader instance representing the testing data.
+        """
+        model = self.best_model.produce_model()
+        model.eval()
+        device = get_device()
+        target, prediction = [], []
+
+        with torch.no_grad():
+            for _, (x, y) in enumerate(test_data):
+                x, y = x.to(device), y.to(device)
+                prediction.append(model(x))
+                target.append(y)
+        return self.metric().compute(prediction, target)
+
 
 class CnnModule(NetworkModule):
     """ Class to create a CNN module."""


Test patch:
diff --git a/tests/test_preprocessor.py b/tests/test_preprocessor.py
--- a/tests/test_preprocessor.py
+++ b/tests/test_preprocessor.py
@@ -24,4 +24,7 @@ def test_batch_dataset(_, _1):
     test_dataloader = DataLoader(test_dataset, batch_size=1, shuffle=False)
     cnn = CnnModule(classification_loss, Accuracy, {}, TEST_TEMP_DIR, True)
     cnn.fit(2, (4, 250, 250, 3), train_dataloader, test_dataloader, 12 * 60 * 60)
+    score = cnn.evaluate(test_dataloader)
+    if score < 0 or score > 1.0:
+        raise AssertionError()
     clean_dir(TEST_TEMP_DIR)
Repository: keras-team/autokeras

Issue:
improve code quality using Codacy
<!---
**If you are reporting a bug:**
*  Verify that your issue is not being currently addressed by other issues or pull requests.
*  Please note that Auto-Keras is only compatible with **Python 3.6**.
*  Tag the issue with the `bug report` tag.
-->

### Bug Description
<!---
A clear and concise description of what the bug is.
-->
We are now using Codacy and CodeClimate to improve our code quality.

Please try to solve anything reported by Codacy by following this [link](https://app.codacy.com/manual/jhfjhfj1/autokeras/issues/index).
On the page, you can see a list of **issues**.
You can click to see the details and suggestions to fix the issue.

Thanks.


Discussion:

Resolving patch:
diff --git a/autokeras/auto_model.py b/autokeras/auto_model.py
--- a/autokeras/auto_model.py
+++ b/autokeras/auto_model.py
@@ -5,7 +5,6 @@
 import tensorflow as tf
 from tensorflow.python.util import nest
 
-from autokeras import const
 from autokeras import meta_model
 from autokeras import tuner
 from autokeras import utils
diff --git a/autokeras/const.py b/autokeras/const.py
--- a/autokeras/const.py
+++ b/autokeras/const.py
@@ -1,5 +1,3 @@
-import tensorflow as tf
-
 
 class Constant(object):
     # Text


Test patch:
Repository: keras-team/autokeras

Issue:
Finish the TODO list of classes/functions to type.
We have a [big todo list of classes/functions to type](https://github.com/keras-team/autokeras/blob/dda2d9de0c602a1f47b45bb26783eacfd9572815/tests/autokeras/typed_api_test.py#L11) (add type hints, see [PEP484](https://www.python.org/dev/peps/pep-0484/) for more infos about type hints) and the docs: https://docs.python.org/3/library/typing.html

Functions of the list should be typed and for classes, only the contructor (`__init__()`) should be typed. No need to specify the return type of a `__init__()` because it's always `None`.

When you add types for a function/class, you can remove it from the TODO list in the same pull request.

Please make multiple small pull requests. Not a single big one :)

If you don't really understand what is asked:
We did the same procedure in tensorflow/addons here: https://github.com/tensorflow/addons/issues/989 See this issue and look into the pull requests made.

You can always ask if you're wondering what to do. It's fairly easy and a good way to start contributing to autokeras!

 


Discussion:

Resolving patch:
diff --git a/autokeras/hypermodels/basic.py b/autokeras/hypermodels/basic.py
--- a/autokeras/hypermodels/basic.py
+++ b/autokeras/hypermodels/basic.py
@@ -173,12 +173,12 @@ class ConvBlock(block_module.Block):
     """
 
     def __init__(self,
-                 kernel_size=None,
-                 num_blocks=None,
-                 num_layers=None,
-                 max_pooling=None,
-                 separable=None,
-                 dropout_rate=None,
+                 kernel_size: Optional[int] = None,
+                 num_blocks: Optional[int] = None,
+                 num_layers: Optional[int] = None,
+                 max_pooling: Optional[bool] = None,
+                 separable: Optional[bool] = None,
+                 dropout_rate: Optional[float] = None,
                  **kwargs):
         super().__init__(**kwargs)
         self.kernel_size = kernel_size


Test patch:
diff --git a/tests/autokeras/typed_api_test.py b/tests/autokeras/typed_api_test.py
--- a/tests/autokeras/typed_api_test.py
+++ b/tests/autokeras/typed_api_test.py
@@ -15,7 +15,6 @@
     autokeras.Node,
     autokeras.CategoricalToNumerical,
     autokeras.ClassificationHead,
-    autokeras.ConvBlock,
     autokeras.Embedding,
     autokeras.Flatten,
     autokeras.ImageAugmentation,
Repository: keras-team/autokeras

Issue:
Multi-label classification `predict` method return probabilities rather than calibrated labels.
### Bug Description
<!---
A clear and concise description of what the bug is.
-->

Multi-label classification `predict()` method return probabilities rather than calibrated labels.


### Bug Reproduction
https://github.com/datamllab/automl-in-action-notebooks/blob/master/3.3.4-Task-API-Multi-label.ipynb


### Setup Details
Include the details about the versions of:
 - OS type and version:
 - Python: 
 - autokeras: 1.0.2
 - keras-tuner:
 - scikit-learn:
 - numpy:
 - pandas:
 - tensorflow: 2.1.0




Discussion:

Resolving patch:
diff --git a/autokeras/adapters/output_adapter.py b/autokeras/adapters/output_adapter.py
--- a/autokeras/adapters/output_adapter.py
+++ b/autokeras/adapters/output_adapter.py
@@ -78,6 +78,7 @@ def fit_before_convert(self, dataset):
             dataset = dataset.values.reshape(-1, 1)
 
         # If encoded.
+        # TODO: support raw string labels for multi-label.
         if len(dataset.flatten()) != len(dataset):
             if self.num_classes:
                 self._check_data_shape(dataset.shape[1:])
@@ -126,6 +127,9 @@ def fit(self, dataset):
                                      actual=shape))
 
     def postprocess(self, y):
+        if self.multi_label:
+            y[y < 0.5] = 0
+            y[y > 0.5] = 1
         if self.label_encoder:
             y = self.label_encoder.decode(y)
         return y


Test patch:
diff --git a/tests/autokeras/adapters/output_adapter_test.py b/tests/autokeras/adapters/output_adapter_test.py
--- a/tests/autokeras/adapters/output_adapter_test.py
+++ b/tests/autokeras/adapters/output_adapter_test.py
@@ -68,3 +68,11 @@ def test_multi_label_two_classes():
     adapter = output_adapter.ClassificationHeadAdapter(name='a', multi_label=True)
     adapter.fit_transform(y)
     assert adapter.label_encoder is None
+
+
+def test_multi_label_postprocessing():
+    y = np.random.rand(10, 3)
+    adapter = output_adapter.ClassificationHeadAdapter(name='a', multi_label=True)
+    adapter.fit_transform(y)
+    y = adapter.postprocess(y)
+    assert set(y.flatten().tolist()) == set([1, 0])
Repository: keras-team/autokeras

Issue:
tuner use previous layer's output to adapt the next
<!-- STEP 1: Give the pull request a meaningful title. -->
### Which issue(s) does this Pull Request fix?
<!-- STEP 2: Replace the "000" with the issue ID this pull request resolves. -->
resolves #1312 

### Details of the Pull Request
<!-- STEP 3: Add details/comments on the pull request. -->

<!-- STEP 4: If the pull request is in progress, click the down green arrow to select "Create Draft Pull Request", and click the button. If the pull request is ready to be reviewed, click "Create Pull Request" button directly. -->



Discussion:

Resolving patch:
diff --git a/autokeras/engine/tuner.py b/autokeras/engine/tuner.py
--- a/autokeras/engine/tuner.py
+++ b/autokeras/engine/tuner.py
@@ -91,6 +91,7 @@ def get_output_layer(tensor):
             layer = get_output_layer(input_node)
             while isinstance(layer, preprocessing.PreprocessingLayer):
                 layer.adapt(temp_x)
+                temp_x = temp_x.map(layer)
                 layer = get_output_layer(layer.output)
         return model
 


Test patch:
diff --git a/tests/autokeras/engine/tuner_test.py b/tests/autokeras/engine/tuner_test.py
--- a/tests/autokeras/engine/tuner_test.py
+++ b/tests/autokeras/engine/tuner_test.py
@@ -18,6 +18,7 @@
 import tensorflow as tf
 from tensorflow.keras.layers.experimental import preprocessing
 
+from autokeras import keras_layers
 from autokeras.engine import tuner as tuner_module
 from autokeras.tuners import greedy
 from tests import utils
@@ -129,7 +130,19 @@ def test_tuner_does_not_crash_with_distribution_strategy(tmp_path):
     tuner.hypermodel.build(tuner.oracle.hyperparameters)
 
 
-def test_preprocessing_adapt():
+def test_preprocessing_adapt_with_cat_to_int_and_norm():
+    x = np.array([["a", 5], ["b", 6]]).astype(np.unicode)
+    y = np.array([[1, 2], [3, 4]]).astype(np.unicode)
+    dataset = tf.data.Dataset.from_tensor_slices((x, y)).batch(32)
+    model = tf.keras.models.Sequential()
+    model.add(tf.keras.Input(shape=(2,), dtype=tf.string))
+    model.add(keras_layers.MultiCategoryEncoding(["int", "none"]))
+    model.add(preprocessing.Normalization(axis=-1))
+
+    tuner_module.AutoTuner.adapt(model, dataset)
+
+
+def test_preprocessing_adapt_with_text_vec():
     class MockLayer(preprocessing.TextVectorization):
         def adapt(self, *args, **kwargs):
             super().adapt(*args, **kwargs)
Repository: keras-team/autokeras

Issue:
function _convert_to_dataset returned result is incorrect
### Bug Description
When validation_data is tf.data.Dataset, validation_data will be assigned to dataset (training data), the dataset returned is the same as validation_data.

The code below
[https://github.com/keras-team/autokeras/blob/68dffdb7abfc97696a256071374533e07007ca00/autokeras/auto_model.py#L379](url)，I think there's a problem here.



### Expected Behavior

the dataset should return training data。


### Setup Details
Include the details about the versions of:
 - OS type and version: windows 10
 - Python: 3.6.9
 - autokeras: 1.0.9
 - keras-tuner: 1.0.2RC2
 - scikit-learn: 0.23.2
 - numpy: 1.18.5
 - pandas: 1.1.2
 - tensorflow: 2.3.1




Discussion:

Resolving patch:
diff --git a/autokeras/auto_model.py b/autokeras/auto_model.py
--- a/autokeras/auto_model.py
+++ b/autokeras/auto_model.py
@@ -375,9 +375,8 @@ def _convert_to_dataset(self, x, y, validation_data, batch_size):
         if validation_data:
             self._check_data_format(validation_data, validation=True)
             if isinstance(validation_data, tf.data.Dataset):
-                dataset = validation_data
-                x = dataset.map(lambda x, y: x)
-                y = dataset.map(lambda x, y: y)
+                x = validation_data.map(lambda x, y: x)
+                y = validation_data.map(lambda x, y: y)
             else:
                 x, y = validation_data
             x = self._adapt(x, self.inputs, batch_size)


Test patch:
diff --git a/tests/integration_tests/task_api_test.py b/tests/integration_tests/task_api_test.py
--- a/tests/integration_tests/task_api_test.py
+++ b/tests/integration_tests/task_api_test.py
@@ -77,7 +77,7 @@ def test_structured_data_regressor(tmp_path):
     num_train = 400
     data = pd.read_csv(utils.TRAIN_CSV_PATH).to_numpy().astype(np.unicode)[:num_data]
     x_train, x_test = data[:num_train], data[num_train:]
-    y = utils.generate_data(num_instances=num_data, shape=(1,))
+    y = utils.generate_data(num_instances=num_data, shape=tuple())
     y_train, y_test = y[:num_train], y[num_train:]
     clf = ak.StructuredDataRegressor(
         directory=tmp_path, max_trials=2, seed=utils.SEED
Repository: keras-team/autokeras

Issue:
Adding types information on the API surface.
Adding types on the public API surface would allow us to do some runtime type checking later on and would allow user's IDE to have more info for static analysis. 

The functions/signatures to type are the ones listed here https://github.com/keras-team/autokeras/blob/master/autokeras/__init__.py

For the context, see #856 where I add some type information on a ImageClassifier.

This issue can be considered easy to solve (good first issue), it's just long to do because the public API surface is big.

I'll do it but some help is welcome :) make sure to make small pull requests.


Discussion:
We're going to have a nice TODO list with https://github.com/tensorflow/addons/pull/1051 . Then we'll be able to track easily what's left and we'll avoid regressions with it.

Resolving patch:
diff --git a/autokeras/utils/io_utils.py b/autokeras/utils/io_utils.py
--- a/autokeras/utils/io_utils.py
+++ b/autokeras/utils/io_utils.py
@@ -13,6 +13,7 @@
 # limitations under the License.
 
 import json
+from typing import Optional
 
 import numpy as np
 import tensorflow as tf
@@ -68,14 +69,14 @@ def get_training_or_validation_split(samples, labels, validation_split, subset):
 
 
 def text_dataset_from_directory(
-    directory,
-    batch_size=32,
-    max_length=None,
-    shuffle=True,
-    seed=None,
-    validation_split=None,
-    subset=None,
-):
+    directory: str,
+    batch_size: int = 32,
+    max_length: Optional[int] = None,
+    shuffle: bool = True,
+    seed: Optional[int] = None,
+    validation_split: Optional[float] = None,
+    subset: Optional[str] = None,
+) -> tf.data.Dataset:
     """Generates a `tf.data.Dataset` from text files in a directory.
 
     If your directory structure is:


Test patch:
diff --git a/tests/autokeras/typed_api_test.py b/tests/autokeras/typed_api_test.py
--- a/tests/autokeras/typed_api_test.py
+++ b/tests/autokeras/typed_api_test.py
@@ -25,7 +25,6 @@
 # TODO: add types and remove all elements from
 # the exception list.
 EXCEPTION_LIST = [
-    autokeras.text_dataset_from_directory,
     autokeras.image_dataset_from_directory,
     autokeras.MultiCategoryEncoding,
     autokeras.BertTokenizer,
Repository: keras-team/autokeras

Issue:
Inconsistency in the docstring for the fit-function. 
### Bug Description
Inconsistency in the docstring for the fit-function. 

### Bug Reproduction
Code for reproducing the bug:

https://github.com/keras-team/autokeras/blob/3b5f435a369747b9acf8fe404b89ddc32042bb27/autokeras/tasks/structured_data.py#L87

https://github.com/keras-team/autokeras/blob/3b5f435a369747b9acf8fe404b89ddc32042bb27/autokeras/tasks/image.py#L119


### Setup Details
Include the details about the versions of:
 - OS type and version:
 - Python: 
 - autokeras: <!--- e.g. 0.4.0, 1.0.2, master-->
 - keras-tuner:
 - scikit-learn:
 - numpy:
 - pandas:
 - tensorflow:





Discussion:

Resolving patch:
diff --git a/autokeras/tasks/structured_data.py b/autokeras/tasks/structured_data.py
--- a/autokeras/tasks/structured_data.py
+++ b/autokeras/tasks/structured_data.py
@@ -84,7 +84,7 @@ def fit(
         validation_data=None,
         **kwargs
     ):
-        """Search for the best model and hyperparameters for the task.
+        """Search for the best model and hyperparameters for the AutoModel.
 
         # Arguments
             x: String, numpy.ndarray, pandas.DataFrame or tensorflow.Dataset.
@@ -280,7 +280,7 @@ def fit(
         validation_data=None,
         **kwargs
     ):
-        """Search for the best model and hyperparameters for the task.
+        """Search for the best model and hyperparameters for the AutoModel.
 
         # Arguments
             x: String, numpy.ndarray, pandas.DataFrame or tensorflow.Dataset.
diff --git a/autokeras/tasks/time_series_forecaster.py b/autokeras/tasks/time_series_forecaster.py
--- a/autokeras/tasks/time_series_forecaster.py
+++ b/autokeras/tasks/time_series_forecaster.py
@@ -226,7 +226,7 @@ def __init__(
     def fit(
         self, x=None, y=None, validation_split=0.2, validation_data=None, **kwargs
     ):
-        """Search for the best model and hyperparameters for the task.
+        """Search for the best model and hyperparameters for the AutoModel.
 
         # Arguments
             x: String, numpy.ndarray, pandas.DataFrame or tensorflow.Dataset.
@@ -399,7 +399,7 @@ def __init__(
     def fit(
         self, x=None, y=None, validation_split=0.2, validation_data=None, **kwargs
     ):
-        """Search for the best model and hyperparameters for the task.
+        """Search for the best model and hyperparameters for the AutoModel.
 
         # Arguments
             x: String, numpy.ndarray, pandas.DataFrame or tensorflow.Dataset.


Test patch:
Repository: keras-team/autokeras

Issue:
Linking https://arxiv.org/abs/1610.02357 to arxiv.org
Would be nice to link the _arxiv article_ in XceptionBlock is linked to [https://arxiv.org/abs/1610.02357](https://arxiv.org/abs/1610.02357) for the docs.

https://github.com/keras-team/autokeras/blob/e925897775aee6dcdd84183e4ceced0b866302c5/autokeras/blocks/basic.py#L753

<img width="786" alt="image" src="https://user-images.githubusercontent.com/13209783/161061847-a2de7894-381a-46a6-82e8-e9ba7f76c249.png">



Discussion:

Resolving patch:
diff --git a/autokeras/blocks/basic.py b/autokeras/blocks/basic.py
--- a/autokeras/blocks/basic.py
+++ b/autokeras/blocks/basic.py
@@ -749,7 +749,8 @@ class XceptionBlock(KerasApplicationBlock):
 
     An Xception structure, used for specifying your model with specific datasets.
 
-    The original Xception architecture is from https://arxiv.org/abs/1610.02357.
+    The original Xception architecture is from
+    [https://arxiv.org/abs/1610.02357](https://arxiv.org/abs/1610.02357).
     The data first goes through the entry flow, then through the middle flow which
     is repeated eight times, and finally through the exit flow.
 


Test patch: