internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: replicate/cog

Issue:
Set python package version explicitly and expose in package
The cog python package sets version metadata but this has never been updated:

```python
In [1]: from importlib.metadata import version

In [2]: version('cog')
Out[2]: '0.0.1'
```

In addition, there's no `__version__` property on the package. This isn't essential but it would be nice to have this too:

```python
In [3]: import cog

In [4]: cog.__version__
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In [4], line 1
----> 1 cog.__version__

AttributeError: module 'cog' has no attribute '__version__'
```

It would be really nice to do this in a way that:

- returns the same version from both of the above
- returns the tagged version in tagged builds (e.g. `0.3.4`)
- appends git metadata when not on a tagged build (e.g. `0.3.4-dev+630e696`)




Discussion:
Some prior art for the fetching version from git bit of this (note that this is from a repo that doesn't have a need for tagged builds): https://github.com/hypothesis/h/blob/main/h/_version.py
Some other prior art for how this is done in the `replicate` Python client: https://github.com/replicate/replicate-python/commit/6ffed387d6616fecb5e6e439de671fe87d903c3f

Resolving patch:
diff --git a/python/cog/__init__.py b/python/cog/__init__.py
--- a/python/cog/__init__.py
+++ b/python/cog/__init__.py
@@ -3,7 +3,14 @@
 from .predictor import BasePredictor
 from .types import File, Input, Path
 
+try:
+    from ._version import __version__
+except ImportError:
+    __version__ = "0.0.0+unknown"
+
+
 __all__ = [
+    "__version__",
     "BaseModel",
     "BasePredictor",
     "File",
diff --git a/python/setup.py b/python/setup.py
deleted file mode 100644
--- a/python/setup.py
+++ /dev/null
@@ -1,32 +0,0 @@
-import setuptools
-
-with open("../README.md", "r", encoding="utf-8") as fh:
-    long_description = fh.read()
-
-
-setuptools.setup(
-    name="cog",
-    version="0.0.1",
-    author_email="team@replicate.com",
-    description="Containers for machine learning",
-    long_description=long_description,
-    long_description_content_type="text/markdown",
-    url="https://github.com/replicate/cog",
-    license="Apache License 2.0",
-    python_requires=">=3.6.0",
-    install_requires=[
-        # intentionally loose. perhaps these should be vendored to not collide with user code?
-        "attrs>=20.1,<23",
-        "fastapi>=0.75.2,<1",
-        "opentelemetry-exporter-otlp>=1.11.1,<2",
-        "opentelemetry-sdk>=1.11.1,<2",
-        "protobuf<=3.20.3",
-        "pydantic>=1,<2",
-        "PyYAML",
-        "redis>=4,<5",
-        "requests>=2,<3",
-        "typing_extensions>=4.1.0",
-        "uvicorn[standard]>=0.12,<1",
-    ],
-    packages=setuptools.find_packages(),
-)


Test patch:
Repository: microsoft/Qcodes

Issue:
version requirement of dependencies
When I try installing Qcodes with `python setup.py develop` I get an error:

```
Traceback (most recent call last):
  File ".\setup.py", line 81, in <module>
    if StrictVersion(module.__version__) < StrictVersion(min_version):
  File "C:\anaconda\lib\distutils\version.py", line 40, in __init__
    self.parse(vstring)
  File "C:\anaconda\lib\distutils\version.py", line 137, in parse
    raise ValueError("invalid version number '%s'" % vstring)
ValueError: invalid version number 'pyqtgraph-0.9.10-100-gf224936'
```

So I have a specific branch of pyqtgraph installed that has a somehow funny version number, but it is larger than the required `0.9.10`.

Is there any way to make this work without swiching the pyqtgraph branch and then switching it back?



Discussion:
That check happens after `setup()` has finished, so you're probably good to go already... anyway, it's not saying the version is too small, it's just barfing on the version string. After I sort out the more pressing issue you uncovered last night, I'll make a fix for this - unless you want to do it, it's a pretty easy `except ValueError:` at the very end with another template asking you to manually ensure that the version is OK.


Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -73,6 +73,15 @@ def readme():
 *****
 '''
 
+valueerror_template = '''
+*****
+***** package {0} version not understood
+***** Please make sure the installed version ({1})
+***** is compatible with the minimum required version ({2})
+***** in order to use {3}
+*****
+'''
+
 # now test the versions of extras
 for extra, (module_name, min_version) in extras.items():
     try:
@@ -81,3 +90,5 @@ def readme():
             print(version_template.format(module_name, min_version, extra))
     except ImportError:
         print(missing_template.format(module_name, extra))
+    except ValueError:
+        print(valueerror_template.format(module_name, module.__version__, min_version, extra))


Test patch:
Repository: microsoft/Qcodes

Issue:
DiskIO discards absolute path information
``` python
> my_io = qcodes.DiskIO('/home/eendebakpt/tmp')
> print(my_io)
<DiskIO, base_location=/mounts/d3/home/eendebakpt/svn/qtt/home/eendebakpt/tmp>
```

The DiskIO object converts my absolute path to a relative path. The problem is in `def _normalize_slashes(self, location)` from `qcodes/data/io.py`.  
I am not sure about what `_normalize_slashes` should do, so I am not sure how to fix this



Discussion:

Resolving patch:
diff --git a/qcodes/data/io.py b/qcodes/data/io.py
--- a/qcodes/data/io.py
+++ b/qcodes/data/io.py
@@ -84,7 +84,10 @@ def open(self, filename, mode):
             yield f
 
     def _normalize_slashes(self, location):
-        return os.path.join(*re.split('[\\\\/]', location))
+        # note that this is NOT os.path.join - the difference is os.path.join
+        # discards empty strings, so if you use it on a re.split absolute
+        # path you will get a relative path!
+        return os.sep.join(re.split('[\\\\/]', location))
 
     def _add_base(self, location):
         location = self._normalize_slashes(location)


Test patch:
Repository: microsoft/Qcodes

Issue:
PR #70 breaks parameter .get and .set functionality
I cannot debug the issue properly because all the objects are `multiprocessing` objects. A minimal example showing the issue:

``` python
%matplotlib nbagg
import matplotlib.pyplot as plt
import time
import numpy as np
import qcodes as qc

from toymodel import AModel, MockGates, MockSource, MockMeter, AverageGetter, AverageAndRaw

# now create this "experiment"
model = AModel()
gates = MockGates('gates', model=model)

c0, c1, c2 = gates.chan0, gates.chan1, gates.chan2
print('fine so far...')

print('error...')
c2.get()
print('no effect?')
c2.set(0.5)

```



Discussion:

Resolving patch:
diff --git a/docs/examples/toymodel.py b/docs/examples/toymodel.py
--- a/docs/examples/toymodel.py
+++ b/docs/examples/toymodel.py
@@ -40,7 +40,7 @@ def gates_set(self, parameter, value):
 
     def gates_get(self, parameter):
         if parameter[0] == 'c':
-            return self.fmt(self.gates[int(parameter[1:])])
+            return self.fmt(self._gates[int(parameter[1:])])
         else:
             raise ValueError
 


Test patch:
Repository: microsoft/Qcodes

Issue:
QtPlot colormap reset during live plotting
When live plotting a 2d plot, i tried to change the colormap from the pyqtgraph gui, but that change is not accepted by the plotting due to the updating of the colormap.
I guess it should only update the range.



Discussion:
Is this as simple as removing [this line](https://github.com/qdev-dk/Qcodes/blob/master/qcodes/plots/pyqtgraph.py#L191)?

Oh wow, yes!


Resolving patch:
diff --git a/qcodes/plots/pyqtgraph.py b/qcodes/plots/pyqtgraph.py
--- a/qcodes/plots/pyqtgraph.py
+++ b/qcodes/plots/pyqtgraph.py
@@ -46,9 +46,9 @@ def __init__(self, *args, figsize=(1000, 600), interval=0.25,
         if remote:
             if not self.__class__.proc:
                 self._init_qt()
-        else:            
+        else:
             # overrule the remote pyqtgraph class
-            self.rpg = pg 
+            self.rpg = pg
         self.win = self.rpg.GraphicsWindow(title=windowTitle)
         self.win.setBackground(theme[1])
         self.win.resize(*figsize)
@@ -163,6 +163,7 @@ def _draw_image(self, subplot_object, z, x=None, y=None, cmap='hot',
         }
 
         self._update_image(plot_object, {'x': x, 'y': y, 'z': z})
+        self._update_cmap(plot_object)
 
         return plot_object
 
@@ -188,8 +189,6 @@ def _update_image(self, plot_object, config):
                 return
         z[np.where(np.isnan(z))] = z_range[0]
 
-        self._update_cmap(plot_object)
-
         hist_range = hist.getLevels()
         if hist_range == plot_object['histlevels']:
             plot_object['histlevels'] = z_range


Test patch:
Repository: microsoft/Qcodes

Issue:
DataSet .complete and .fraction_complete do not work
A minimal example is included below. The `DataSet.complete` returns while the loop is still running.
```
import time
import numpy as np
import qcodes
from qcodes.tests.data_mocks import DataSet2D
from qcodes.tests.instrument_mocks import DummyInstrument

dataset = DataSet2D()
A = DummyInstrument(server_name='test')
sweepvalues=A.dac2[0:100:.05]

#%%
#%Loop in background
data=qcodes.Loop(sweepvalues, progress_interval=.2, delay=.001).each(A.dac1).run(background=True)
print('sync %s' % data.sync())
print('fraction %.3f' % data.fraction_complete())
data.complete() # should wait on data to be ready

for ii in range(20):    
    data.sync()
    print('sync %s' % data.sync())
    print('fraction %.3f' % data.fraction_complete())
    time.sleep(0.2)
``` 

@giulioungaretti @alexcjohnson 


Discussion:
Yeah, this is a confusing situation... you need to explicitly enable the `DataManager` too, then it works:
```
data=qcodes.Loop(sweepvalues, progress_interval=.2, delay=.001).each(A.dac1).run(
    background=True, data_manager=True)
```

Without the DataManager there's no way for the background process to sync data back to the main process.

I suppose we could change the `run` flags so you only need one of them to enable multiprocessing. I wrote it this way because there *are* reasons you might want background and no data manager or vice versa, though they're kind of edge cases. So perhaps a better way, if you're going to be running this way most of the time while the new architecture is still being developed, would be to set `config.core.legacy_mp = True` which turns them both on by default.

One thing we could do is throw a warning if you use `background=True` without `data_manager=True`. That would help any `legacy_mp` users to get to the right settings.

Resolving patch:
diff --git a/qcodes/loops.py b/qcodes/loops.py
--- a/qcodes/loops.py
+++ b/qcodes/loops.py
@@ -785,6 +785,12 @@ def run(self, background=USE_MP, use_threads=False, quiet=False,
 
         data_set = self.get_data_set(data_manager, *args, **kwargs)
 
+        if background and not getattr(data_set, 'data_manager', None):
+            warnings.warn(
+                'With background=True you must also set data_manager=True '
+                'or you will not be able to sync your DataSet.',
+                UserWarning)
+
         self.set_common_attrs(data_set=data_set, use_threads=use_threads,
                               signal_queue=self.signal_queue)
 


Test patch:
Repository: microsoft/Qcodes

Issue:
hdf5 formatter crashed for simple dataset
A minimal example:
```
import qcodes
import qcodes.tests.data_mocks
from qcodes.data.hdf5_format import HDF5Format
qcodes.DataSet.default_formatter = HDF5Format()

alldata=qcodes.tests.data_mocks.DataSet2D()
    
alldata.metadata={'a': ['hi', 'there'] }
alldata.write(write_metadata=True)
```


Discussion:

Resolving patch:
diff --git a/qcodes/data/hdf5_format.py b/qcodes/data/hdf5_format.py
--- a/qcodes/data/hdf5_format.py
+++ b/qcodes/data/hdf5_format.py
@@ -280,6 +280,7 @@ def write_dict_to_hdf5(self, data_dict, entry_point):
                         elif isinstance(item[0], str):
                             dt = h5py.special_dtype(vlen=str)
                             data = np.array(item)
+                            data = data.reshape( (-1,1))
                             ds = entry_point.create_dataset(
                                 key, (len(data), 1), dtype=dt)
                             ds[:] = data


Test patch:
diff --git a/qcodes/tests/test_hdf5formatter.py b/qcodes/tests/test_hdf5formatter.py
--- a/qcodes/tests/test_hdf5formatter.py
+++ b/qcodes/tests/test_hdf5formatter.py
@@ -331,3 +331,9 @@ def test_writing_unsupported_types_to_hdf5(self):
         F['weird_dict'].attrs['list_type'] = 'unsuported_list_type'
         with self.assertRaises(NotImplementedError):
             self.formatter.read_dict_from_hdf5(new_dict, F)
+
+    def test_writing_metadata(self):
+        # test for issue reported in 442
+        data = DataSet2D(location=self.loc_provider, name='MetaDataTest')
+        data.metadata = {'a': ['hi', 'there']}
+        self.formatter.write(data, write_metadata=True)
Repository: microsoft/Qcodes

Issue:
hdf5 formatter does not implement proper interface for write_metadata
The `write_metadata` function has an argument `read_first`:

https://github.com/QCoDeS/Qcodes/blob/master/qcodes/data/format.py#L107

The HDF5 formatter does not have this argument:

https://github.com/QCoDeS/Qcodes/blob/master/qcodes/data/hdf5_format.py#L238

This generates an error here: 

https://github.com/QCoDeS/Qcodes/blob/master/qcodes/data/data_set.py#L764

A one line fix would be to add the `read_first` argument to the hdf5 formatter, but ignore it.

@giulioungaretti @AdriaanRol 


Discussion:
@peendebak , I thought I took care to preserve the interface when making the hdf5 formatter. Maybe this got added later? (or maybe I just messed up). I approve of your immediate solution. 

Resolving patch:
diff --git a/qcodes/data/hdf5_format.py b/qcodes/data/hdf5_format.py
--- a/qcodes/data/hdf5_format.py
+++ b/qcodes/data/hdf5_format.py
@@ -235,7 +235,7 @@ def _create_dataarray_dset(self, array, group):
 
         return dset
 
-    def write_metadata(self, data_set, io_manager=None, location=None):
+    def write_metadata(self, data_set, io_manager=None, location=None, read_first=True):
         """
         Writes metadata of dataset to file using write_dict_to_hdf5 method
 
@@ -243,6 +243,7 @@ def write_metadata(self, data_set, io_manager=None, location=None):
         of backwards compatibility with the loop.
         This formatter uses io and location as specified for the main
         dataset.
+        The read_first argument is ignored.
         """
         if not hasattr(data_set, '_h5_base_group'):
             # added here because loop writes metadata before data itself


Test patch:
Repository: microsoft/Qcodes

Issue:
Unused class FileWrapper
In the io manager module there is a FileWrapper class that seems to be never used anywhere?
Aside from the dubious utility of such a class (maybe if documented one could figure out) , maybe we should just remove it ?

@QCoDeS/core  ? 



Discussion:
I forget why I made that... :hocho:

Resolving patch:
diff --git a/qcodes/data/io.py b/qcodes/data/io.py
--- a/qcodes/data/io.py
+++ b/qcodes/data/io.py
@@ -28,13 +28,9 @@
     - iter(): usually return self, but can be any iterator over lines
     - next(): assuming iter() returns self, this yields the next line.
 
-    (note: iter and next can be constructed automatically by FileWrapper
-    if you implement readline.)
-
   In write or append mode:
     - write(s): add string s to the end of the file.
-    - writelines(seq): add a sequence of strings (can be constructed
-      automatically if you use FileWrapper)
+    - writelines(seq): add a sequence of strings
 
 IO managers should also implement:
 
@@ -223,7 +219,7 @@ def list(self, location, maxdepth=1, include_dirs=False):
     def remove(self, filename):
         """Delete a file or folder and prune the directory tree."""
         path = self.to_path(filename)
-        if(os.path.isdir(path)):
+        if os.path.isdir(path):
             shutil.rmtree(path)
         else:
             os.remove(path)
@@ -243,28 +239,3 @@ def remove_all(self, location):
         """
         for fn in self.list(location):
             self.remove(fn)
-
-
-class FileWrapper:
-    def read(self, size=None):
-        raise NotImplementedError
-
-    def readline(self, size=None):
-        raise NotImplementedError
-
-    def __iter__(self):
-        return self
-
-    def __next__(self):
-        line = self.readline()
-        if line:
-            return line
-        else:
-            raise StopIteration
-
-    def write(self, s):
-        raise NotImplementedError
-
-    def writelines(self, seq):
-        for s in seq:
-            self.write(s)


Test patch:
Repository: microsoft/Qcodes

Issue:
Keithely 2600 "resolution"
@MerlinSmiles  right now  we are limiting the set to 8 digits (https://github.com/QCoDeS/Qcodes/blob/master/qcodes/instrument_drivers/tektronix/Keithley_2600.py#L23)
Afaik it can go to to 12 digits. Do you confirm ? 



Discussion:
@giulioungaretti no hardware around in this train, try it out? stupid mistake if it can go to 12 digits.
@MerlinSmiles  

![sketch 1](https://cloud.githubusercontent.com/assets/2694199/22370878/0eedc604-e494-11e6-9317-f1fe039678a8.png)

Let me know if you agree with the way I read the manual 📄  
@giulioungaretti looks very reasonable. Could you check if you can send 12 digits when in >1A range? And also how many digits do you get back when reading (in large and small range)?

Resolving patch:
diff --git a/qcodes/instrument_drivers/tektronix/Keithley_2600.py b/qcodes/instrument_drivers/tektronix/Keithley_2600.py
--- a/qcodes/instrument_drivers/tektronix/Keithley_2600.py
+++ b/qcodes/instrument_drivers/tektronix/Keithley_2600.py
@@ -20,11 +20,11 @@ def __init__(self, name, address, channel, **kwargs):
         self._channel = channel
 
         self.add_parameter('volt', get_cmd='measure.v()',
-                           get_parser=float, set_cmd='source.levelv={:.8f}',
+                           get_parser=float, set_cmd='source.levelv={:.12f}',
                            label='Voltage',
                            unit='V')
         self.add_parameter('curr', get_cmd='measure.i()',
-                           get_parser=float, set_cmd='source.leveli={:.8f}',
+                           get_parser=float, set_cmd='source.leveli={:.12f}',
                            label='Current',
                            unit='A')
         self.add_parameter('mode',


Test patch:
Repository: microsoft/Qcodes

Issue:
background task hides exceptions. 
If you encounter a bug use the following template.
If you have a feature request feel free to freestyle.
### Steps to reproduce
``` python 
import qcodes as qc
from qcodes.tests.instrument_mocks import DummyInstrument

d = DummyInstrument("dummy")

def fail():
    raise ValueError


loop = qc.Loop(d.dac1.sweep(0,20,0.1), delay=0.001).each(d.dac2)
data = loop.get_data_set(name='testsweep')


plot = qc.QtPlot()
plot.add(data.dummy_dac2)
_ = loop.with_bg_task(fail, plot.save).run()
``` 

### Expected behaviour
Raise exception and stop loop?

### Actual behaviour
Nothing happens 




Discussion:

Resolving patch:
diff --git a/qcodes/loops.py b/qcodes/loops.py
--- a/qcodes/loops.py
+++ b/qcodes/loops.py
@@ -47,6 +47,7 @@
 """
 
 from datetime import datetime
+import logging
 import multiprocessing as mp
 import time
 import numpy as np
@@ -64,6 +65,8 @@
 from .actions import (_actions_snapshot, Task, Wait, _Measure, _Nest,
                       BreakIf, _QcodesBreak)
 
+
+log = logging.getLogger(__name__)
 # Switches off multiprocessing by default, cant' be altered after module
 USE_MP = config.core.legacy_mp
 MP_NAME = 'Measurement'
@@ -870,7 +873,6 @@ def run(self, background=USE_MP, use_threads=False, quiet=False,
             # this one later.
             self.data_set = None
 
-
         return ds
 
     def _compile_actions(self, actions, action_indices=()):
@@ -939,8 +941,10 @@ def _run_loop(self, first_delay=0, action_indices=(),
 
         t0 = time.time()
         last_task = t0
-        last_task_failed = False
         imax = len(self.sweep_values)
+
+        self.last_task_failed = False
+
         for i, value in enumerate(self.sweep_values):
             if self.progress_interval is not None:
                 tprint('loop %s: %d/%d (%.1f [s])' % (
@@ -997,16 +1001,18 @@ def _run_loop(self, first_delay=0, action_indices=(),
                 if t - last_task >= self.bg_min_delay:
                     try:
                         self.bg_task()
-                        last_task_failed = False
                     except Exception:
-                        if last_task_failed:
+                        if self.last_task_failed:
                             self.bg_task = None
-                        last_task_failed = True
+                        self.last_task_failed = True
+                        log.exception("Failed to execute bg task")
+
                     last_task = t
 
         if self.progress_interval is not None:
             # final progress note: set dt=-1 so it *always* prints
             tprint('loop %s DONE: %d/%d (%.1f [s])' % (
+
                    self.sweep_values.name, i + 1, imax, time.time() - t0),
                    dt=-1, tag='outerloop')
 
@@ -1022,8 +1028,6 @@ def _run_loop(self, first_delay=0, action_indices=(),
         if self.bg_final_task is not None:
             self.bg_final_task()
 
-
-
     def _wait(self, delay):
         if delay:
             finish_clock = time.perf_counter() + delay


Test patch: