Repository: tensorflow/addons

Issue:
Package Addons for python3.7 
Using the current docker image, we're unable to install / build TensorFlow in python3.7 with the following error:
```
ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/miniconda2/envs/py37/lib/python3.7/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
```

It seems like we need to compile using Ubuntu 16.04 libs, which will require a bump in the custom-op docker image. @yifeif would it be possible to bump the base image in the container? If so we can rebuild our release as 0.1.1 and include py37.

Alternatively, we could create a new docker image for builds.


Discussion:
 @dynamicwebpaige Paige, tf-2.0-preview has supported python 3.7 in https://github.com/tensorflow/tensorflow/issues/25429 . Could you tell us how did you do it?
cc @gunan. Is ubuntu 14.04 being used for the build or some other docker image?
For python 3.7, we had to base our builders off of ubuntu 16.04.
Everything else uses ubuntu 14.04
We will definitely need a new image for python3.7.
Addons 0.2.0 has a py37 available on pypi
Bumping to ubuntu16+GCC5 caused issues issues as seen here in the most recent builds of TF:
https://github.com/tensorflow/tensorflow/issues/27067

This will remain unsolved until the manylinux2010 toolchain is finalized. See #119 

What's the ETA for tf addons python 3.7 support?
@perfinion is there an ETA for the updated build environment / ETA when TF pip packages will be built in it? Just for reference python 3.8 release is soon.
Looks like this is almost ready:
https://github.com/tensorflow/tensorflow/commit/ec83c8a8292aa881b701928a7ffbe2305cc0e4e5

@av8ramit Could you comment on the above question?
Ideally worst case within two weeks we will transition over. Waiting on one last issue with GPU custom ops.

Resolving patch:
diff --git a/build_deps/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010-nvcc-cuda10.0/windows/msvc_wrapper_for_nvcc.py b/build_deps/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010-nvcc-cuda10.0/windows/msvc_wrapper_for_nvcc.py
new file mode 100755
--- /dev/null
+++ b/build_deps/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010-nvcc-cuda10.0/windows/msvc_wrapper_for_nvcc.py
@@ -0,0 +1,210 @@
+#!/usr/bin/env python
+# Copyright 2015 The TensorFlow Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ==============================================================================
+"""Crosstool wrapper for compiling CUDA programs with nvcc on Windows.
+
+DESCRIPTION:
+  This script is the Windows version of crosstool_wrapper_is_not_gcc
+"""
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+from argparse import ArgumentParser
+import os
+import subprocess
+import re
+import sys
+import pipes
+
+# Template values set by cuda_autoconf.
+CPU_COMPILER = ('/dt7/usr/bin/gcc')
+GCC_HOST_COMPILER_PATH = ('/dt7/usr/bin/gcc')
+
+NVCC_PATH = '/usr/local/cuda-10.0/bin/nvcc'
+NVCC_VERSION = '10.0'
+NVCC_TEMP_DIR = "C:\\Windows\\Temp\\nvcc_inter_files_tmp_dir"
+supported_cuda_compute_capabilities = ["3.0", "6.0"]
+
+
+def Log(s):
+    print('gpus/crosstool: {0}'.format(s))
+
+
+def GetOptionValue(argv, option):
+    """Extract the list of values for option from options.
+
+    Args:
+      option: The option whose value to extract, without the leading '/'.
+
+    Returns:
+      1. A list of values, either directly following the option,
+      (eg., /opt val1 val2) or values collected from multiple occurrences of
+      the option (eg., /opt val1 /opt val2).
+      2. The leftover options.
+    """
+
+    parser = ArgumentParser(prefix_chars='/')
+    parser.add_argument('/' + option, nargs='*', action='append')
+    args, leftover = parser.parse_known_args(argv)
+    if args and vars(args)[option]:
+        return (sum(vars(args)[option], []), leftover)
+    return ([], leftover)
+
+
+def _update_options(nvcc_options):
+    if NVCC_VERSION in ("7.0",):
+        return nvcc_options
+
+    update_options = {"relaxed-constexpr": "expt-relaxed-constexpr"}
+    return [
+        update_options[opt] if opt in update_options else opt
+        for opt in nvcc_options
+    ]
+
+
+def GetNvccOptions(argv):
+    """Collect the -nvcc_options values from argv.
+
+    Args:
+      argv: A list of strings, possibly the argv passed to main().
+
+    Returns:
+      1. The string that can be passed directly to nvcc.
+      2. The leftover options.
+    """
+
+    parser = ArgumentParser()
+    parser.add_argument('-nvcc_options', nargs='*', action='append')
+
+    args, leftover = parser.parse_known_args(argv)
+
+    if args.nvcc_options:
+        options = _update_options(sum(args.nvcc_options, []))
+        return (['--' + a for a in options], leftover)
+    return ([], leftover)
+
+
+def InvokeNvcc(argv, log=False):
+    """Call nvcc with arguments assembled from argv.
+
+    Args:
+      argv: A list of strings, possibly the argv passed to main().
+      log: True if logging is requested.
+
+    Returns:
+      The return value of calling os.system('nvcc ' + args)
+    """
+
+    src_files = [
+        f for f in argv if re.search('\.cpp$|\.cc$|\.c$|\.cxx$|\.C$', f)
+    ]
+    if len(src_files) == 0:
+        raise ValueError('No source files found for cuda compilation.')
+
+    out_file = [f for f in argv if f.startswith('/Fo')]
+    if len(out_file) != 1:
+        raise ValueError(
+            'Please sepecify exactly one output file for cuda compilation.')
+    out = ['-o', out_file[0][len('/Fo'):]]
+
+    nvcc_compiler_options, argv = GetNvccOptions(argv)
+
+    opt_option, argv = GetOptionValue(argv, 'O')
+    opt = ['-g', '-G']
+    if (len(opt_option) > 0 and opt_option[0] != 'd'):
+        opt = ['-O2']
+
+    include_options, argv = GetOptionValue(argv, 'I')
+    includes = ["-I " + include for include in include_options]
+
+    defines, argv = GetOptionValue(argv, 'D')
+    defines = ['-D' + define for define in defines]
+
+    undefines, argv = GetOptionValue(argv, 'U')
+    undefines = ['-U' + define for define in undefines]
+
+    # The rest of the unrecongized options should be passed to host compiler
+    host_compiler_options = [
+        option for option in argv if option not in (src_files + out_file)
+    ]
+
+    m_options = ["-m64"]
+
+    nvccopts = ['-D_FORCE_INLINES']
+    for capability in supported_cuda_compute_capabilities:
+        capability = capability.replace('.', '')
+        nvccopts += [
+            r'-gencode=arch=compute_%s,"code=sm_%s,compute_%s"' %
+            (capability, capability, capability)
+        ]
+    nvccopts += nvcc_compiler_options
+    nvccopts += undefines
+    nvccopts += defines
+    nvccopts += m_options
+    nvccopts += [
+        '--compiler-options="' + " ".join(host_compiler_options) + '"'
+    ]
+    nvccopts += ['-x', 'cu'] + opt + includes + out + ['-c'] + src_files
+    # If we don't specify --keep-dir, nvcc will generate intermediate files under TEMP
+    # Put them under NVCC_TEMP_DIR instead, then Bazel can ignore files under NVCC_TEMP_DIR during dependency check
+    # http://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#options-for-guiding-compiler-driver
+    # Different actions are sharing NVCC_TEMP_DIR, so we cannot remove it if the directory already exists.
+    if os.path.isfile(NVCC_TEMP_DIR):
+        os.remove(NVCC_TEMP_DIR)
+    if not os.path.exists(NVCC_TEMP_DIR):
+        os.makedirs(NVCC_TEMP_DIR)
+    nvccopts += ['--keep', '--keep-dir', NVCC_TEMP_DIR]
+    cmd = [NVCC_PATH] + nvccopts
+    if log:
+        Log(cmd)
+    proc = subprocess.Popen(
+        cmd,
+        stdout=sys.stdout,
+        stderr=sys.stderr,
+        env=os.environ.copy(),
+        shell=True)
+    proc.wait()
+    return proc.returncode
+
+
+def main():
+    parser = ArgumentParser()
+    parser.add_argument('-x', nargs=1)
+    parser.add_argument('--cuda_log', action='store_true')
+    args, leftover = parser.parse_known_args(sys.argv[1:])
+
+    if args.x and args.x[0] == 'cuda':
+        if args.cuda_log: Log('-x cuda')
+        leftover = [pipes.quote(s) for s in leftover]
+        if args.cuda_log: Log('using nvcc')
+        return InvokeNvcc(leftover, log=args.cuda_log)
+
+    # Strip our flags before passing through to the CPU compiler for files which
+    # are not -x cuda. We can't just pass 'leftover' because it also strips -x.
+    # We not only want to pass -x to the CPU compiler, but also keep it in its
+    # relative location in the argv list (the compiler is actually sensitive to
+    # this).
+    cpu_compiler_flags = [
+        flag for flag in sys.argv[1:] if not flag.startswith(('--cuda_log'))
+        and not flag.startswith(('-nvcc_options'))
+    ]
+
+    return subprocess.call([CPU_COMPILER] + cpu_compiler_flags)
+
+
+if __name__ == '__main__':
+    sys.exit(main())
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -35,6 +35,7 @@
 from setuptools import find_packages
 from setuptools import setup
 from setuptools.dist import Distribution
+from setuptools import Extension
 
 DOCLINES = __doc__.split('\n')
 
@@ -73,6 +74,7 @@ def has_ext_modules(self):
     author='Google Inc.',
     author_email='opensource@google.com',
     packages=find_packages(),
+    ext_modules=[Extension('_foo', ['stub.cc'])],
     install_requires=REQUIRED_PACKAGES,
     include_package_data=True,
     zip_safe=False,


Test patch:
diff --git a/tensorflow_addons/layers/sparsemax_test.py b/tensorflow_addons/layers/sparsemax_test.py
--- a/tensorflow_addons/layers/sparsemax_test.py
+++ b/tensorflow_addons/layers/sparsemax_test.py
@@ -20,6 +20,7 @@
 import numpy as np
 import tensorflow as tf
 
+from absl.testing import parameterized
 from tensorflow_addons.layers import Sparsemax
 from tensorflow_addons.utils import test_utils
 
@@ -50,10 +51,10 @@ def _np_sparsemax(z):
     return np.maximum(0, z - tau_z)
 
 
-@test_utils.run_all_with_types(['float32', 'float64'])
+@parameterized.parameters([np.float32, np.float64])
 @test_utils.run_all_in_graph_and_eager_modes
 class SparsemaxTest(tf.test.TestCase):
-    def test_sparsemax_layer_against_numpy(self, dtype=None):
+    def test_sparsemax_layer_against_numpy(self, dtype):
         """check sparsemax kernel against numpy."""
         random = np.random.RandomState(1)
 
@@ -61,6 +62,7 @@ def test_sparsemax_layer_against_numpy(self, dtype=None):
 
         test_utils.layer_test(
             Sparsemax,
+            kwargs={'dtype': dtype},
             input_data=z,
             expected_output=_np_sparsemax(z).astype(dtype))
 
diff --git a/tensorflow_addons/rnn/cell_test.py b/tensorflow_addons/rnn/cell_test.py
--- a/tensorflow_addons/rnn/cell_test.py
+++ b/tensorflow_addons/rnn/cell_test.py
@@ -156,7 +156,7 @@ def test_config(self):
         cell = rnn_cell.NASCell(10, projection=5, use_bias=True)
 
         expected_config = {
-            "dtype": None,
+            "dtype": "float32",
             "name": "nas_cell",
             "trainable": True,
             "units": 10,
@@ -241,7 +241,7 @@ def test_config(self):
         cell = rnn_cell.LayerNormLSTMCell(10)
 
         expected_config = {
-            "dtype": None,
+            "dtype": "float32",
             "name": "layer_norm_lstm_cell",
             "trainable": True,
             "units": 10,
diff --git a/tensorflow_addons/seq2seq/attention_wrapper_test.py b/tensorflow_addons/seq2seq/attention_wrapper_test.py
--- a/tensorflow_addons/seq2seq/attention_wrapper_test.py
+++ b/tensorflow_addons/seq2seq/attention_wrapper_test.py
@@ -466,7 +466,8 @@ def _testWithMaybeMultiAttention(self,
                     expected_final_alignment_history,
                     final_alignment_history_info)
 
-    @parameterized.parameters([np.float32, np.float64])
+    # TODO: #407 Float64 test is failing
+    @parameterized.parameters([np.float32])
     def testBahdanauNormalizedDType(self, dtype):
         encoder_outputs = self.encoder_outputs.astype(dtype)
         decoder_inputs = self.decoder_inputs.astype(dtype)
@@ -492,7 +493,8 @@ def testBahdanauNormalizedDType(self, dtype):
         self.assertEqual(final_outputs.rnn_output.dtype, dtype)
         self.assertIsInstance(final_state, wrapper.AttentionWrapperState)
 
-    @parameterized.parameters([np.float32, np.float64])
+    # TODO: #407 Float64 test is failing
+    @parameterized.parameters([np.float32])
     def testLuongScaledDType(self, dtype):
         # Test case for GitHub issue 18099
         encoder_outputs = self.encoder_outputs.astype(dtype)
diff --git a/tools/ci_testing/install_py36.sh b/tools/ci_testing/install_py36.sh
deleted file mode 100755
--- a/tools/ci_testing/install_py36.sh
+++ /dev/null
@@ -1,31 +0,0 @@
-#!/usr/bin/env bash
-# Copyright 2019 The TensorFlow Authors. All Rights Reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# ==============================================================================
-# Needed until docker image defaults to at least py35.
-
-set -e -x
-
-curl -sSOL https://bootstrap.pypa.io/get-pip.py
-add-apt-repository -y ppa:deadsnakes/ppa
-
-apt-get -y -qq update && apt-get -y -qq install python3.6
-
-python3.6 get-pip.py -q
-python3.6 -m pip --version
-rm get-pip.py
-
-ln -sfn /usr/bin/python3.6 /usr/bin/python3
-pip3 install scipy  # Pre-installed in custom-op
\ No newline at end of file