Repository: microsoft/Qcodes

Issue:
Bug: Saving metadata issue when dict contains both numeric and non-numeric keys
### Steps to reproduce
1. Create: `p = Parameter(name='test', val_mapping={1: 1, '2': 2})`
2. Perform measurement with p

### Expected behaviour
Measurement should run without problems

### Actual behaviour
Saving the metadata as a JSON fails because it tries to sort the keys.
Problem lies in lines 338-339 of qcodes/data/gnuplot_format.py:

```
json.dump(data_set.metadata, snap_file, sort_keys=True,
                 indent=4, ensure_ascii=False, cls=NumpyJSONEncoder)
```

### Proposed solution
Remove sort_keys, is it really necessary to sort them? Alternatively, force all dict keys to be of same type. Some instruments (e.g. SR830) will need modifications

@jenshnielsen @WilliamHPNielsen @sohailc 


Discussion:
I could be talked in to both solutions. 

On one hand I don't think the sorting is particularly important 
on the other hand all keys will be serialised to strings anyway when dumping to JSON so perhaps it makes sense to standardise? 

It seems to me that that is a bug in the python JSON library that sorting happens before conversion to string but its the case in both the python 3.5 and 3.6 releases that I tested so I guess we have to live with that. 




@jenshnielsen I guess pre-converting to strings would also work, though this can fail if the string representation would have a different sorting order, e.g. `'2' > '1e9'`
@jenshnielsen @nulinspiratie The same problem arises with the SR830 driver. A minimal example constructed from the driver:
```
import qcodes

class Dummy(qcodes.Instrument):
    
    def __init__(self, name, **kwargs):
        super().__init__(name,**kwargs)
        
        self.add_parameter('p', set_cmd=None)
        self.add_parameter('q', set_cmd=None)
        self.add_parameter('buffer_SR',
                           label='Buffer sample rate',
                           set_cmd=None,
                           initial_value=0,
                           unit='Hz',
                           val_mapping={0:0, 62.5e-3: 0,
                                        0.125: 1,
                                        0.250: 2,
                                        0.5: 3,
                                        1: 4, 2: 5,
                                        4: 6, 8: 7,
                                        16: 8, 32: 9,
                                        64: 10, 128: 11,
                                        256: 12, 512: 13,
                                        'Trigger': 14},
                           get_parser=int
                           )
        
d=Dummy('test6')
station=qcodes.Station(d)
ds=qcodes.Loop(d.p[0:10:1]).each(d.q ).run()
```


@nulinspiratie @jenshnielsen The code works if I use `import simplejson as json`. Would this be a workaround? 
@peendebak I guess that would be a solution, though it's not clear to me
why this is fixed in simplejson. Perhaps it was a recent bugfix?

On Fri, 27 Oct 2017 at 04:30 peendebak <notifications@github.com> wrote:

> @nulinspiratie <https://github.com/nulinspiratie> @jenshnielsen
> <https://github.com/jenshnielsen> The code works if I use import
> simplejson as json. Would this be a workaround?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/QCoDeS/Qcodes/issues/775#issuecomment-339740255>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/AGgLHVxruvpARYkCTZRZStohqoqzV81kks5swMG9gaJpZM4PrlMV>
> .
>

Not sure why simplejson did work

@jenshnielsen @WilliamHPNielsen @nulinspiratie We have multiple users who cannot use recent versions of qcodes together with the SR830 driver. Do you have any idea when you can look into this issue?
I think we can live without the sorting. I'll make a PR soon ( 😛 ) to remove the sorting.

Resolving patch:
diff --git a/qcodes/data/gnuplot_format.py b/qcodes/data/gnuplot_format.py
--- a/qcodes/data/gnuplot_format.py
+++ b/qcodes/data/gnuplot_format.py
@@ -350,7 +350,7 @@ def write_metadata(self, data_set, io_manager, location, read_first=True):
 
         fn = io_manager.join(location, self.metadata_file)
         with io_manager.open(fn, 'w', encoding='utf8') as snap_file:
-            json.dump(data_set.metadata, snap_file, sort_keys=True,
+            json.dump(data_set.metadata, snap_file, sort_keys=False,
                       indent=4, ensure_ascii=False, cls=NumpyJSONEncoder)
 
     def read_metadata(self, data_set):


Test patch:
diff --git a/qcodes/tests/test_generic_formatter.py b/qcodes/tests/test_generic_formatter.py
--- a/qcodes/tests/test_generic_formatter.py
+++ b/qcodes/tests/test_generic_formatter.py
@@ -31,3 +31,19 @@ def test_read_write(self):
             # strings should be read and written identically
             self.assertEqual(dataset.metadata['string'],
                              dataset2.metadata['string'])
+
+
+class TestNoSorting(TestCase):
+    """
+    (WilliamHPNielsen): I'm not too sure where this test belongs... It tests
+    that parameters with non-sortable keys can be saved using the gnuplot
+    formatter, so I guess it goes here.
+    """
+
+    param = qcodes.Parameter(name='mixed_val_mapping_param',
+                             get_cmd=lambda: np.random.randint(1, 3),
+                             val_mapping={1: 1, '2': 2}
+                             )
+
+    def test_can_measure(self):
+        qcodes.Measure(self.param).run()