internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: GPflow/GPflow

Issue:
Paramlist objects do not obey tf_mode
Parameterized objects within a ParamList do not get the tf_mode set correctly because the parameter class incorrectly uses this list comprehension:

``` python
[child._begin_tf_mode() for key, child in self.__dict__.items() if isinstance(child, Parameterized) and key is not '_parent']
```

Which duplicates the definition of `self.sorted_params`:

``` python
    @property
    def sorted_params(self):
        #snip
        params = [child for key, child in self.__dict__.items() if isinstance(child, (Param, Parameterized)) and key is not '_parent']
        return sorted(params, key=id)
```



Discussion:

Resolving patch:
diff --git a/GPflow/param.py b/GPflow/param.py
--- a/GPflow/param.py
+++ b/GPflow/param.py
@@ -347,11 +347,11 @@ class will appear as their tf-variables. Example
         self._end_tf_mode()
 
     def _begin_tf_mode(self):
-        [child._begin_tf_mode() for key, child in self.__dict__.items() if isinstance(child, Parameterized) and key is not '_parent']
+        [child._begin_tf_mode() for child in self.sorted_params if isinstance(child, Parameterized)]
         self._tf_mode = True
 
     def _end_tf_mode(self):
-        [child._end_tf_mode() for key, child in self.__dict__.items() if isinstance(child, Parameterized) and key is not '_parent']
+        [child._end_tf_mode() for child in self.sorted_params if isinstance(child, Parameterized)]
         self._tf_mode = False
 
     def build_prior(self):


Test patch:
diff --git a/testing/test_param.py b/testing/test_param.py
--- a/testing/test_param.py
+++ b/testing/test_param.py
@@ -243,6 +243,12 @@ def test_with_parameterized(self):
         l[0].p = 5
         self.failUnless( l.get_free_state()==5 )
 
+        #test to make sure tf_mode get turned on and off
+        self.failIf(pzd._tf_mode)
+        with l.tf_mode():
+            self.failUnless(pzd._tf_mode)
+        self.failIf(pzd._tf_mode)
+
 
     def test_in_model(self):
 
@@ -256,7 +262,7 @@ def build_likelihood(self):
 
         m = Foo()
         self.failUnless(m.get_free_state().size ==2)
-        m.optimize()
+        m.optimize(display=False)
         self.failUnless(np.allclose(m.get_free_state(), 0.))
Repository: GPflow/GPflow

Issue:
compute_log_likelihood for SVGP class results in unitialized variable error. 
The following code gives an error.

```
import numpy as np
import GPflow

rng = np.random.RandomState(1)
X = rng.randn( 10, 1)
Y = rng.randn( 10, 1 ) 
Z = rng.randn( 3,1 )
model = GPflow.svgp.SVGP( X=X, Y=Y, kern=GPflow.kernels.RBF(1) ,likelihood=GPflow.likelihoods.Gaussian() , Z=Z )
model.compute_log_likelihood()
```

> FailedPreconditionError: Attempting to use uninitialized value tfY

It seems this may be down to recent changes? 



Discussion:

Resolving patch:
diff --git a/GPflow/param.py b/GPflow/param.py
--- a/GPflow/param.py
+++ b/GPflow/param.py
@@ -297,6 +297,7 @@ def runnable(instance, *np_args):
                 with instance.tf_mode():
                     storage['tf_result'] = tf_method(instance, *storage['tf_args'])
                 storage['session'] = tf.Session()
+                storage['session'].run(tf.initialize_all_variables())
             feed_dict = dict(zip(storage['tf_args'], np_args))
             feed_dict[storage['free_vars']] = instance.get_free_state()
             return storage['session'].run(storage['tf_result'], feed_dict=feed_dict)


Test patch:
diff --git a/testing/test_autoflow.py b/testing/test_autoflow.py
--- a/testing/test_autoflow.py
+++ b/testing/test_autoflow.py
@@ -88,7 +88,7 @@ def test_predict_y(self):
         mu, var = self.m.predict_y(self.Xtest)
 
     def test_predict_density(self):
-        d = self.m.predict_density(self.Xtest, self.Ytest)
+        self.m.predict_density(self.Xtest, self.Ytest)
 
 
 class TestResetGraph(unittest.TestCase):
@@ -127,5 +127,18 @@ def test(self):
         _, _ = self.m.predict_f(self.m.X)
 
 
+class TestSVGP(unittest.TestCase):
+    """
+    This replicates Alex's code from bug #99
+    """
+    def test(self):
+        rng = np.random.RandomState(1)
+        X = rng.randn(10, 1)
+        Y = rng.randn(10, 1)
+        Z = rng.randn(3, 1)
+        model = GPflow.svgp.SVGP(X=X, Y=Y, kern=GPflow.kernels.RBF(1), likelihood=GPflow.likelihoods.Gaussian(), Z=Z)
+        model.compute_log_likelihood()
+
+
 if __name__ == "__main__":
     unittest.main()
Repository: GPflow/GPflow

Issue:
pickle and active_dims
Hi All,

Thanks for implementing the getters and setters for pickle, makes saving experiments much easier! However I noticed a small bug, if you have active_dims set in the kernel it stops pickle from working since the active dimensions are stored as tf.constants.

example code to reproduce bug:

``` python
X = np.random.randn(100).reshape((10,10))
y = np.random.randn(10)

k = GPflow.kernels.RBF(len(X[0]), active_dims=[0])
m = GPflow.gpr.GPR(X, y[:, np.newaxis], k)
```

Best,
Matt



Discussion:

Resolving patch:
diff --git a/GPflow/kernels.py b/GPflow/kernels.py
--- a/GPflow/kernels.py
+++ b/GPflow/kernels.py
@@ -1,3 +1,4 @@
+from __future__ import print_function
 from functools import reduce
 
 import tensorflow as tf
@@ -22,7 +23,8 @@ def __init__(self, input_dim, active_dims=None):
         if active_dims is None:
             self.active_dims = slice(input_dim)
         else:
-            self.active_dims = tf.constant(np.array(active_dims, dtype=np.int32), tf.int32)
+            self._active_dims_array = np.array(active_dims, dtype=np.int32)
+            self.active_dims = tf.constant(self._active_dims_array, tf.int32)
 
     def _slice(self, X, X2):
         if isinstance(self.active_dims, slice):
@@ -50,6 +52,17 @@ def compute_K(self, X, Z):
     def compute_K_symm(self, X):
         return self.K(X)
 
+    def __getstate__(self):
+        d = Parameterized.__getstate__(self)
+        if hasattr(self, '_active_dims_array'):
+            d.pop('active_dims')
+        return d
+
+    def __setstate__(self, d):
+        Parameterized.__setstate__(self, d)
+        if hasattr(self, '_active_dims_array'):
+            self.active_dims = tf.constant(self._active_dims_array, tf.int32)
+
 
 class Static(Kern):
     """
diff --git a/GPflow/param.py b/GPflow/param.py
--- a/GPflow/param.py
+++ b/GPflow/param.py
@@ -610,6 +610,14 @@ def _kill_autoflow(self):
                 delattr(self, key)
         [p._kill_autoflow() for p in self.sorted_params if isinstance(p, Parameterized)]
 
+    def __getstate__(self):
+        d = Parentable.__getstate__(self)
+        # do not pickle autoflow
+        for key in list(d.keys()):
+            if key[0] == '_' and key[-11:] == '_AF_storage':
+                d.pop(key)
+        return d
+
     def make_tf_array(self, X):
         """
         X is a tf. placeholder. It gets passed to all the children of


Test patch:
diff --git a/testing/test_pickle.py b/testing/test_pickle.py
--- a/testing/test_pickle.py
+++ b/testing/test_pickle.py
@@ -29,6 +29,16 @@ def test(self):
         self.assertTrue(m2.p2._parent is m2)
 
 
+class TestActiveDims(unittest.TestCase):
+    def test(self):
+        k = GPflow.kernels.RBF(2, active_dims=[0, 1])
+        X = np.random.randn(10, 2)
+        K = k.compute_K_symm(X)
+        k = pickle.loads(pickle.dumps(k))
+        K2 = k.compute_K_symm(X)
+        self.assertTrue(np.allclose(K, K2))
+
+
 class TestPickleGPR(unittest.TestCase):
     def setUp(self):
         tf.reset_default_graph()
Repository: GPflow/GPflow

Issue:
tests fail with tensorflow 10.0rc0
One test fails with the new tensorflow pre-release, it's related to the custom-op test:

``` python
ImportError: No module named 'tensorflow.python.kernel_tests'
```

In addition, there sees to be an issue with one of the notebooks ( #161 ), I'm looking into this.



Discussion:

Resolving patch:
diff --git a/GPflow/_version.py b/GPflow/_version.py
--- a/GPflow/_version.py
+++ b/GPflow/_version.py
@@ -13,4 +13,4 @@
 # limitations under the License.
 
 
-__version__ = "0.2.1"
+__version__ = "0.3.0"
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -33,12 +33,14 @@
       license="BSD 3-clause",
       keywords="machine-learning gaussian-processes kernels tensorflow",
       url="http://github.com/gpflow/gpflow",
+      package_data={'GPflow': ['GPflow/tfops/*.so']},
+      include_package_data=True,
       ext_modules=[],
       packages=["GPflow"],
       package_dir={'GPflow': 'GPflow'},
       py_modules=['GPflow.__init__'],
       test_suite='testing',
-      install_requires=['numpy>=1.9', 'scipy>=0.16', 'tensorflow>=0.7.1'],
+      install_requires=['numpy>=1.9', 'scipy>=0.16', 'tensorflow>=0.10.0rc0'],
       classifiers=['License :: OSI Approved :: BSD License',
                    'Natural Language :: English',
                    'Operating System :: MacOS :: MacOS X',


Test patch:
diff --git a/testing/test_custom_op.py b/testing/test_custom_op.py
--- a/testing/test_custom_op.py
+++ b/testing/test_custom_op.py
@@ -2,7 +2,7 @@
 import numpy as np
 import tensorflow as tf
 from GPflow.tf_hacks import vec_to_tri, tri_to_vec
-import tensorflow.python.kernel_tests.gradient_checker as gc
+from tensorflow.python.ops.gradient_checker import compute_gradient_error
 
 
 def np_vec_to_tri(vec):
@@ -43,7 +43,7 @@ def testGradient(self):
         v = tf.Variable(initval[None, :])
         with tf.Session(''):
             f = (vec_to_tri(v) * np.random.randn(N, N)) ** 2.0  # Some function involving vec_to_tri
-            self.assertLess(gc.compute_gradient_error(v, [1, len(initval)], f, [1, N, N]), 10 ** -10)
+            self.assertLess(compute_gradient_error(v, [1, len(initval)], f, [1, N, N]), 10 ** -10)
 
 
 class TestTriToVec(unittest.TestCase):
Repository: GPflow/GPflow

Issue:
print summary fails for sum kernel

System information:

* Python version: 3.7
* TensorFlow installed from pip tf2 nightly
* TensorFlow version (use command below):'2.0.0-dev20190930'
* GPflow installed from (source or binary): python setup.py develop
* GPflow version: awav/gpflow-2.0



... _Describe the current behavior_
MWE:
```
k = gpflow.kernels.SquaredExponential() + gpflow.kernels.SquaredExponential()
print_summary(k)

Output:
name    class    transform    trainable    shape    dtype    value
------  -------  -----------  -----------  -------  -------  -------
```
... _Describe the expected behavior_
should print the values of the summed kernel's parameters.



Discussion:

Resolving patch:
diff --git a/gpflow/utilities/utilities.py b/gpflow/utilities/utilities.py
--- a/gpflow/utilities/utilities.py
+++ b/gpflow/utilities/utilities.py
@@ -100,6 +100,7 @@ def get_transform(v):
         _str_tensor_value(variable.numpy())
     ] for path, variable in merged_leaf_components.items()]
 
+
     if fmt == "notebook":
         from IPython.core.display import display, HTML
         tab = tabulate(column_values, headers=column_names, tablefmt="html")
@@ -154,7 +155,7 @@ def _get_leaf_components(input: tf.Module, prefix: Optional[str] = None):
             submodule_name = input.__class__.__name__
             for term_idx, subterm in enumerate(submodule):
                 subterm_key = f"{submodule_name}_{key}[{term_idx}]"
-                if isinstance(subterm, Parameter):
+                if isinstance(subterm, tf.Module):
                     subterm_var = _get_leaf_components(subterm, prefix=f"{prefix}.{subterm_key}")
                     var_dict.update(subterm_var)
         elif isinstance(submodule, _DictWrapper):


Test patch:
diff --git a/tests/test_printing.py b/tests/test_printing.py
--- a/tests/test_printing.py
+++ b/tests/test_printing.py
@@ -253,3 +253,11 @@ def test_merge_leaf_components_merges_keys_with_same_values(dag_module, expected
         for sub_path in path.split('\n'):
             assert sub_path in leaf_components_dict
             assert leaf_components_dict[sub_path] is variable
+
+
+def test_leaf_components_combination_kernel():
+    """
+    Regression test for kernel compositions - output for printing should not be empty (issue #1066).
+    """
+    k = gpflow.kernels.SquaredExponential() + gpflow.kernels.SquaredExponential()
+    assert leaf_components(k), "Combination kernel should have non-empty leaf components"
Repository: GPflow/GPflow

Issue:
GPflow2 set_default_float and set_default_int accept tf dtypes but that breaks code
Some functions such as the Gauss-Hermite quadrature code call `numpy_array.astype(default_float())`, which fails if `default_float()` returns a *tensorflow* dtype such as `tf.float64` instead of `np.float64`. However, `gpflow.config.set_default_float()` explicitly allows both numpy and tensorflow dtypes as arguments.


Discussion:
Also, it'd be really nice to have a config context-manager to be able to have localised changes to the config...

Resolving patch:
diff --git a/gpflow/config/__config__.py b/gpflow/config/__config__.py
--- a/gpflow/config/__config__.py
+++ b/gpflow/config/__config__.py
@@ -114,7 +114,7 @@ def set_default_int(value_type):
     if not tf_dtype.is_integer:
         raise TypeError(f"{value_type} is not an integer dtype")
 
-    set_config(replace(config(), int=value_type))
+    set_config(replace(config(), int=tf_dtype.as_numpy_dtype))
 
 
 def set_default_float(value_type):
@@ -126,7 +126,7 @@ def set_default_float(value_type):
     if not tf_dtype.is_floating:
         raise TypeError(f"{value_type} is not a float dtype")
 
-    set_config(replace(config(), float=value_type))
+    set_config(replace(config(), float=tf_dtype.as_numpy_dtype))
 
 
 def set_default_jitter(value: float):


Test patch:
Repository: GPflow/GPflow

Issue:
tf2.2-rc1 gast requirement
Hi,

gpflow 2.0.0-rc1 has gast requirement 0.2.2.

TensorFlow has gast requirement 0.3.3 from 2.2-rc1, which is incompatible with gpflow requirement.

Best Regards,

Marco


Discussion:
TensorFlow has some issues with installing the right version of gast - we previously had gast unpinned, but that led to failures (on the CI if I recall correctly) due to pip simply installing gast 0.3.3, which is incompatible with tensorflow 2.0 and 2.1... Do you have any ideas how to make GPflow work with both TensorFlow 2.0/1 and 2.2(rc1)?

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -18,8 +18,7 @@
     'numpy>=1.10.0',
     'scipy>=0.18.0',
     'multipledispatch>=0.4.9',
-    'tabulate',
-    'gast==0.2.2',
+    'tabulate'
 ]
 
 if not is_py37:
@@ -32,6 +31,22 @@
 tf_cpu = 'tensorflow'
 tf_gpu = 'tensorflow-gpu'
 
+
+# for latest_version() [see https://github.com/GPflow/GPflow/issues/1348]:
+def latest_version(package_name):
+    import json
+    from urllib import request
+    import re
+
+    url = f"https://pypi.python.org/pypi/{package_name}/json"
+    data = json.load(request.urlopen(url))
+    # filter out rc and beta releases and, more generally, any releases that
+    # do not contain exclusively numbers and dots.
+    versions = [parse_version(v) for v in data["releases"].keys() if re.match("^[0-9.]+$", v)]  
+    versions.sort()
+    return versions[-1]  # return latest version
+
+
 # Only detect TF if not installed or outdated. If not, do not do not list as
 # requirement to avoid installing over e.g. tensorflow-gpu
 # To avoid this, rely on importing rather than the package name (like pip).
@@ -47,6 +62,9 @@
     if not on_rtd:
         # Do not add TF if we are installing GPflow on readthedocs
         requirements.append(tf_cpu)
+        gast_requirement = 'gast>=0.2.2,<0.3' if latest_version('tensorflow') < parse_version('2.2') else 'gast>=0.3.3'
+        requirements.append(gast_requirement)
+        
 
 with open(str(Path(".", "VERSION").absolute())) as version_file:
     version = version_file.read().strip()


Test patch:
Repository: GPflow/GPflow

Issue:
setup.py depends on external dataclasses package for python >= 3.8
Setup.py has a check
```python
is_py37 = sys.version_info.major == 3 and sys.version_info.minor == 7
```
and adds the PyPI `dataclasses` package to the requirements when `not is_py37`. (`dataclasses` has been incorporated in the stdlib in python 3.7.) With python 3.8 released, this check is inaccurate, as setup.py currently adds the dependency on dataclasses when the python version is 3.8 or later, not just when it's less than 3.7.


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -10,13 +10,13 @@
 from pkg_resources import parse_version
 from setuptools import find_packages, setup
 
-is_py37 = sys.version_info.major == 3 and sys.version_info.minor == 7
 on_rtd = os.environ.get("READTHEDOCS", None) == "True"  # copied from the docs
 
 # Dependencies of GPflow
 requirements = ["numpy>=1.10.0", "scipy>=0.18.0", "multipledispatch>=0.4.9", "tabulate"]
 
-if not is_py37:
+if sys.version_info < (3, 7):
+    # became part of stdlib in python 3.7
     requirements.append("dataclasses")
 
 if not on_rtd:


Test patch:
Repository: GPflow/GPflow

Issue:
Check deps on CI
`pip install gpflow` currently installs dependencies (setuptools, scipy) with versions that are incompatible with the tensorflow version installed.

This ticket isn't to fix the dependencies, per se, but suggests adding a `pip check -vvv` stage to CI, so that such problems are caught at PR stage.


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -12,11 +12,10 @@
 # Dependencies of GPflow
 requirements = [
     "numpy>=1.10.0",
-    "scipy>=0.18.0",
+    "scipy>=0.18.0,==1.4.1",  # pinned to ==1.4.1 to satisfy tensorflow requirements
     "multipledispatch>=0.6",
     "tabulate",
     "typing_extensions",
-    "cloudpickle==1.3.0",  # temporary workaround for tensorflow/probability#991
 ]
 
 if sys.version_info < (3, 7):
@@ -27,7 +26,18 @@
 # use autodoc_mock_imports instead. Hence we use this flag to decide whether or
 # not to append tensorflow and tensorflow_probability to the requirements:
 if os.environ.get("READTHEDOCS") != "True":
-    requirements.extend(["tensorflow>=2.1.0,<2.3", "tensorflow-probability>=0.9,<0.11"])
+    requirements.extend(
+        [
+            # tensorflow>=2.3 not compatible with tensorflow-probability<0.11
+            "tensorflow>=2.1.0,<2.3",
+            # tensorflow-probability==0.10.0 doesn't install correctly
+            # https://github.com/tensorflow/probability/issues/991
+            #
+            # gpflow uses private functionality not present in tensorflow-probability~=0.11
+            "tensorflow-probability>=0.9,<0.11,!=0.10.0",
+            "setuptools>=41.0.0",  # to satisfy dependency constraints
+        ]
+    )
 
 
 def read_file(filename):


Test patch:
Repository: GPflow/GPflow

Issue:
Bug: Predictions with full_cov=True
The functions `model.predict_y()` and `model.predict_density()` do not work nicely with `full_cov=True`. For example, take `GPR`, where the `Gaussian` likelihood is used. In `predict_y()` the likelihood function `predict_mean_and_var()` is called. Here a scalar value of the likelihood variance is added to the covariance matrix. For `full_cov=True`, this should be added only to the diagonal.

I'm not sure what the right resolution for this is. In the short term, maybe throwing an error for `full_cov=True` would be helpful to show that this is unsupported.
Bug: Predictions with full_cov=True
The functions `model.predict_y()` and `model.predict_density()` do not work nicely with `full_cov=True`. For example, take `GPR`, where the `Gaussian` likelihood is used. In `predict_y()` the likelihood function `predict_mean_and_var()` is called. Here a scalar value of the likelihood variance is added to the covariance matrix. For `full_cov=True`, this should be added only to the diagonal.

I'm not sure what the right resolution for this is. In the short term, maybe throwing an error for `full_cov=True` would be helpful to show that this is unsupported.


Discussion:

Resolving patch:
diff --git a/gpflow/models/model.py b/gpflow/models/model.py
--- a/gpflow/models/model.py
+++ b/gpflow/models/model.py
@@ -211,6 +211,12 @@ def predict_y(
         """
         Compute the mean and variance of the held-out data at the input points.
         """
+        if full_cov or full_output_cov:
+            # See https://github.com/GPflow/GPflow/issues/1461
+            raise NotImplementedError(
+                "The predict_y method currently supports only the argument values full_cov=False and full_output_cov=False"
+            )
+
         f_mean, f_var = self.predict_f(Xnew, full_cov=full_cov, full_output_cov=full_output_cov)
         return self.likelihood.predict_mean_and_var(f_mean, f_var)
 
@@ -220,6 +226,12 @@ def predict_log_density(
         """
         Compute the log density of the data at the new data points.
         """
+        if full_cov or full_output_cov:
+            # See https://github.com/GPflow/GPflow/issues/1461
+            raise NotImplementedError(
+                "The predict_log_density method currently supports only the argument values full_cov=False and full_output_cov=False"
+            )
+
         X, Y = data
         f_mean, f_var = self.predict_f(X, full_cov=full_cov, full_output_cov=full_output_cov)
         return self.likelihood.predict_log_density(f_mean, f_var, Y)


Test patch:
Repository: GPflow/GPflow

Issue:
update intro_to_gpflow2 notebook to account for saved_model now working
to finally close #1127


Discussion:
# [Codecov](https://codecov.io/gh/GPflow/GPflow/pull/1586?src=pr&el=h1) Report
> Merging [#1586](https://codecov.io/gh/GPflow/GPflow/pull/1586?src=pr&el=desc) into [develop](https://codecov.io/gh/GPflow/GPflow/commit/98f7997b362e142aefb45170152726063eb2aa42?el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/GPflow/GPflow/pull/1586/graphs/tree.svg?width=650&height=150&src=pr&token=ykbhyEEbld)](https://codecov.io/gh/GPflow/GPflow/pull/1586?src=pr&el=tree)

```diff
@@           Coverage Diff            @@
##           develop    #1586   +/-   ##
========================================
  Coverage    95.13%   95.13%           
========================================
  Files           86       86           
  Lines         3845     3845           
========================================
  Hits          3658     3658           
  Misses         187      187           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/GPflow/GPflow/pull/1586?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/GPflow/GPflow/pull/1586?src=pr&el=footer). Last update [98f7997...42faff8](https://codecov.io/gh/GPflow/GPflow/pull/1586?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Resolving patch:
diff --git a/doc/source/conf.py b/doc/source/conf.py
--- a/doc/source/conf.py
+++ b/doc/source/conf.py
@@ -94,7 +94,7 @@
 # The short X.Y version.
 version = "2.1"
 # The full version, including alpha/beta/rc tags.
-release = "2.1.2"
+release = "2.1.3"
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.


Test patch: