Repository: microsoft/Qcodes

Issue:
Measurement of array like values 
Feature as discussed in V0.1 QCodes meeting 

``` python
parameter.setpoints = np.linspace() 
dataset = Measure(parameter)
```



Discussion:
links to [#243](https://github.com/qdev-dk/Qcodes/issues/243) as we can currently measure but only as part of a loop. I'll close that issue

@AdriaanRol  @nataliejpg .
For the sake of clarity,  #243 and this are two different things right ? 

In #274 there was a question whether we need separate setup and teardown actions also for `Measure`. I don't think so - you can just include your before and after actions in the same list, and if they don't generate any data we won't record any data from them, just like `Loop().each(...)`:

``` python
qc.Measure(Task(get_ready),
           vna.s11, vna.s21,
           Task(change_something),
           vna.s11, vna.s21,
           Task(close_up))
```

re: `Measure().run()` - it would be nice to be able to keep a pre-defined measurement around and reuse it, like you can with Loops (and also use the measurement inside a Loop!). Also it helps partition the arguments between actions/parameters and the settings to execute them and save the results. However `Measure().run()` is not grammatically correct... perhaps `Measurement().run()`? Then we would always be free to implement a quicker convenience function like:

``` python
def measure(*args, **kwargs):
    return Measurement(*args).run(**kwargs)
```

for when you don't care about saving the measurement for later.

@giulioungaretti  I think that this was part of #243 (in fact I think it's the first bullet point in alexcjohnson's comment on that issue). That issue was about having an arraylike harware measured parameter. That is now implemented however you can only do '2d' measurement in a Loop because it is currently not possible to create a dataset from a single parameter measurement which this issue addresses. I linked it because I would say that the functionality of an arraylike parameter is not complete if you can't do a 'single shot' measurement of it. 

@alexcjohnson Regarding #274 and having `first` and `then` actions, I view the concept of loops and measurement to be closely related and so it makes sense to try and keep the functionality of `Measurement` and `Loop` as similar as possible. There are situations where I want to test the first point of a long running loop, which can be done using a `Measurement` so I propose having as much syntax overlap as possible between the two actions.

In addition to that, conceptually I view actions that happen as part of setting up and tearing down not to be part of the measurement action, so there is an argument for separating it out of the body of the measurement for that reason as well.

The new name you propose for Measurement you propose makes a lot of sense, I agree that that is clearer.

Is this something which is hard to implement @giulioungaretti @alexcjohnson ? I don't have a good enough understanding of how the DataServer and DataSet work but since it is now possible to manually create a DataSet it superficially seems to me that this isn't much harder? Is anyone doing this sort of measurement currently by manually creating a DataSet? @peendebak I think you mentioned this? 


Resolving patch:
diff --git a/docs/examples/toymodel.py b/docs/examples/toymodel.py
--- a/docs/examples/toymodel.py
+++ b/docs/examples/toymodel.py
@@ -146,3 +146,37 @@ def get(self):
         data = loop.run_temp()
         array = data.arrays[self.measured_param.full_name]
         return (array, array.mean())
+
+
+class ArrayGetter(Parameter):
+    """
+    Example parameter that just returns a single array
+
+    TODO: in theory you can make this same Parameter with
+    name, label & shape (instead of names, labels & shapes) and altered
+    setpoints (not wrapped in an extra tuple) and this mostly works,
+    but when run in a loop it doesn't propagate setpoints to the
+    DataSet. We could track down this bug, but perhaps a better solution
+    would be to only support the simplest and the most complex Parameter
+    forms (ie cases 1 and 5 in the Parameter docstring) and do away with
+    the intermediate forms that make everything more confusing.
+    """
+    def __init__(self, measured_param, sweep_values, delay):
+        name = measured_param.name
+        super().__init__(names=(name,))
+        self._instrument = getattr(measured_param, '_instrument', None)
+        self.measured_param = measured_param
+        self.sweep_values = sweep_values
+        self.delay = delay
+        self.shapes = ((len(sweep_values),),)
+        set_array = DataArray(parameter=sweep_values.parameter,
+                              preset_data=sweep_values)
+        self.setpoints = ((set_array,),)
+        if hasattr(measured_param, 'label'):
+            self.labels = (measured_param.label,)
+
+    def get(self):
+        loop = Loop(self.sweep_values, self.delay).each(self.measured_param)
+        data = loop.run_temp()
+        array = data.arrays[self.measured_param.full_name]
+        return (array,)
diff --git a/qcodes/__init__.py b/qcodes/__init__.py
--- a/qcodes/__init__.py
+++ b/qcodes/__init__.py
@@ -32,6 +32,7 @@
 
 from qcodes.station import Station
 from qcodes.loops import get_bg, halt_bg, Loop
+from qcodes.measure import Measure
 from qcodes.actions import Task, Wait, BreakIf
 
 from qcodes.data.manager import get_data_manager
diff --git a/qcodes/measure.py b/qcodes/measure.py
new file mode 100644
--- /dev/null
+++ b/qcodes/measure.py
@@ -0,0 +1,155 @@
+from datetime import datetime
+
+from qcodes.instrument.parameter import ManualParameter
+from qcodes.loops import Loop, USE_MP
+from qcodes.actions import _actions_snapshot
+from qcodes.utils.helpers import full_class
+from qcodes.utils.metadata import Metadatable
+
+
+class Measure(Metadatable):
+    """
+    Create a DataSet from a single (non-looped) set of actions.
+
+    Args:
+        *actions (any): sequence of actions to perform. Any action that is
+            valid in a ``Loop`` can be used here. If an action is a gettable
+            ``Parameter``, its output will be included in the DataSet.
+            Scalars returned by an action will be saved as length-1 arrays,
+            with a dummy setpoint for consistency with other DataSets.
+    """
+    dummy_parameter = ManualParameter(name='single',
+                                      label='Single Measurement')
+
+    def __init__(self, *actions):
+        super().__init__()
+        self._dummyLoop = Loop(self.dummy_parameter[0]).each(*actions)
+
+    def run_temp(self, **kwargs):
+        """
+        Wrapper to run this measurement as a temporary data set
+        """
+        return self.run(quiet=True, data_manager=False, location=False,
+                        **kwargs)
+
+    def run(self, use_threads=False, quiet=False, data_manager=USE_MP,
+            station=None, **kwargs):
+        """
+        Run the actions in this measurement and return their data as a DataSet
+
+        Args:
+            quiet (Optional[bool]): Set True to not print anything except
+                errors. Default False.
+
+            station (Optional[Station]): the ``Station`` this measurement
+                pertains to. Defaults to ``Station.default`` if one is defined.
+                Only used to supply metadata.
+
+            use_threads (Optional[bool]): whether to parallelize ``get``
+                operations using threads. Default False.
+
+            Other kwargs are passed along to data_set.new_data. The key ones
+            are:
+
+            location (Optional[Union[str, False]]): the location of the
+                DataSet, a string whose meaning depends on formatter and io,
+                or False to only keep in memory. May be a callable to provide
+                automatic locations. If omitted, will use the default
+                DataSet.location_provider
+
+            name (Optional[str]): if location is default or another provider
+                function, name is a string to add to location to make it more
+                readable/meaningful to users
+
+            formatter (Optional[Formatter]): knows how to read and write the
+                file format. Default can be set in DataSet.default_formatter
+
+            io (Optional[io_manager]): knows how to connect to the storage
+                (disk vs cloud etc)
+
+        returns:
+            a DataSet object containing the results of the measurement
+        """
+
+        # background is not configurable, would be weird to run this in the bg
+        background = False
+
+        data_set = self._dummyLoop.get_data_set(data_manager=data_manager,
+                                                **kwargs)
+
+        # set the DataSet to local for now so we don't save it, since
+        # we're going to massage it afterward
+        original_location = data_set.location
+        data_set.location = False
+
+        # run the measurement as if it were a Loop
+        self._dummyLoop.run(background=background, use_threads=use_threads,
+                            station=station, quiet=True)
+
+        # look for arrays that are unnecessarily nested, and un-nest them
+        all_unnested = True
+        for array in data_set.arrays.values():
+            if array.ndim == 1:
+                if array.is_setpoint:
+                    dummy_setpoint = array
+                else:
+                    # we've found a scalar - so keep the dummy setpoint
+                    all_unnested = False
+            else:
+                # The original return was an array, so take off the extra dim.
+                # (This ensures the outer dim length was 1, otherwise this
+                # will raise a ValueError.)
+                array.ndarray.shape = array.ndarray.shape[1:]
+
+                # TODO: DataArray.shape masks ndarray.shape, and a user *could*
+                # change it, thinking they were reshaping the underlying array,
+                # but this would a) not actually reach the ndarray right now,
+                # and b) if it *did* and the array was reshaped, this array
+                # would be out of sync with its setpoint arrays, so bad things
+                # would happen. So we probably want some safeguards in place
+                # against this
+                array.shape = array.ndarray.shape
+
+                array.set_arrays = array.set_arrays[1:]
+
+                array.init_data()
+
+        # Do we still need the dummy setpoint array at all?
+        if all_unnested:
+            del data_set.arrays[dummy_setpoint.array_id]
+            if hasattr(data_set, 'action_id_map'):
+                del data_set.action_id_map[dummy_setpoint.action_indices]
+
+        # now put back in the DataSet location and save it
+        data_set.location = original_location
+        data_set.write()
+
+        # metadata: ActiveLoop already provides station snapshot, but also
+        # puts in a 'loop' section that we need to replace with 'measurement'
+        # but we use the info from 'loop' to ensure consistency and avoid
+        # duplication.
+        LOOP_SNAPSHOT_KEYS = ['background', 'ts_start', 'ts_end',
+                              'use_data_manager', 'use_threads']
+        data_set.add_metadata({'measurement': {
+            k: data_set.metadata['loop'][k] for k in LOOP_SNAPSHOT_KEYS
+        }})
+        del data_set.metadata['loop']
+
+        # actions are included in self.snapshot() rather than in
+        # LOOP_SNAPSHOT_KEYS because they are useful if someone just
+        # wants a local snapshot of the Measure object
+        data_set.add_metadata({'measurement': self.snapshot()})
+
+        data_set.save_metadata()
+
+        if not quiet:
+            print(repr(data_set))
+            print(datetime.now().strftime('acquired at %Y-%m-%d %H:%M:%S'))
+
+        return data_set
+
+    def snapshot_base(self, update=False):
+        return {
+            '__class__': full_class(self),
+            'actions': _actions_snapshot(self._dummyLoop.actions, update)
+        }


Test patch:
diff --git a/qcodes/tests/instrument_mocks.py b/qcodes/tests/instrument_mocks.py
--- a/qcodes/tests/instrument_mocks.py
+++ b/qcodes/tests/instrument_mocks.py
@@ -1,11 +1,10 @@
 import time
+import numpy as np
 
 from qcodes.instrument.base import Instrument
 from qcodes.instrument.mock import MockInstrument, MockModel
-from qcodes.instrument.base import Instrument
 from qcodes.utils.validators import Numbers
-from qcodes.instrument.parameter import ManualParameter
-import numpy as np
+from qcodes.instrument.parameter import Parameter, ManualParameter
 
 
 class AMockModel(MockModel):
@@ -49,8 +48,7 @@ def source_set(self, parameter, value):
         if parameter == 'ampl':
             try:
                 self._excitation = float(value)
-            # TODO(giulioungaretti)  fix bare-except
-            except:
+            except ValueError:
                 # "Off" as in the MultiType sweep step test
                 self._excitation = None
         else:
@@ -301,3 +299,29 @@ def __init__(self, name='dummy', gates=['dac1', 'dac2', 'dac3'], **kwargs):
                                initial_value=0,
                                label='Gate {} (arb. units)'.format(g),
                                vals=Numbers(-800, 400))
+
+
+class MultiGetter(Parameter):
+    """
+    Test parameters with complicated return values
+    instantiate with kwargs:
+        MultiGetter(name1=return_val1, name2=return_val2)
+    to set the names and (constant) return values of the
+    pieces returned. Each return_val can be any array-like
+    object
+    eg:
+        MultiGetter(one=1, onetwo=(1, 2))
+    """
+    def __init__(self, **kwargs):
+        if len(kwargs) == 1:
+            name, self._return = list(kwargs.items())[0]
+            super().__init__(name=name)
+            self.shape = np.shape(self._return)
+        else:
+            names = tuple(sorted(kwargs.keys()))
+            super().__init__(names=names)
+            self._return = tuple(kwargs[k] for k in names)
+            self.shapes = tuple(np.shape(v) for v in self._return)
+
+    def get(self):
+        return self._return
diff --git a/qcodes/tests/test_loop.py b/qcodes/tests/test_loop.py
--- a/qcodes/tests/test_loop.py
+++ b/qcodes/tests/test_loop.py
@@ -19,7 +19,8 @@
 from qcodes.utils.validators import Numbers
 from qcodes.utils.helpers import LogCapture
 
-from .instrument_mocks import AMockModel, MockGates, MockSource, MockMeter
+from .instrument_mocks import (AMockModel, MockGates, MockSource, MockMeter,
+                               MultiGetter)
 
 
 class TestMockInstLoop(TestCase):
@@ -256,22 +257,6 @@ def call(self, finish_by=None):
         self.delay_array.append(finish_by - time.perf_counter())
 
 
-class MultiGetter(Parameter):
-    def __init__(self, **kwargs):
-        if len(kwargs) == 1:
-            name, self._return = list(kwargs.items())[0]
-            super().__init__(name=name)
-            self.shape = np.shape(self._return)
-        else:
-            names = tuple(sorted(kwargs.keys()))
-            super().__init__(names=names)
-            self._return = tuple(kwargs[k] for k in names)
-            self.shapes = tuple(np.shape(v) for v in self._return)
-
-    def get(self):
-        return self._return
-
-
 class TestLoop(TestCase):
     @classmethod
     def setUpClass(cls):
diff --git a/qcodes/tests/test_measure.py b/qcodes/tests/test_measure.py
new file mode 100644
--- /dev/null
+++ b/qcodes/tests/test_measure.py
@@ -0,0 +1,49 @@
+from unittest import TestCase
+from datetime import datetime
+
+from qcodes.instrument.parameter import ManualParameter
+from qcodes.measure import Measure
+
+from .instrument_mocks import MultiGetter
+
+
+class TestMeasure(TestCase):
+    def setUp(self):
+        self.p1 = ManualParameter('P1', initial_value=1)
+
+    def test_simple_scalar(self):
+        data = Measure(self.p1).run_temp()
+
+        self.assertEqual(data.single_set.tolist(), [0])
+        self.assertEqual(data.P1.tolist(), [1])
+        self.assertEqual(len(data.arrays), 2, data.arrays)
+
+        self.assertNotIn('loop', data.metadata)
+
+        meta = data.metadata['measurement']
+        self.assertEqual(meta['__class__'], 'qcodes.measure.Measure')
+        self.assertEqual(len(meta['actions']), 1)
+        self.assertFalse(meta['background'])
+        self.assertFalse(meta['use_data_manager'])
+        self.assertFalse(meta['use_threads'])
+
+        ts_start = datetime.strptime(meta['ts_start'], '%Y-%m-%d %H:%M:%S')
+        ts_end = datetime.strptime(meta['ts_end'], '%Y-%m-%d %H:%M:%S')
+        self.assertGreaterEqual(ts_end, ts_start)
+
+    def test_simple_array(self):
+        data = Measure(MultiGetter(arr=(1.2, 3.4))).run_temp()
+
+        self.assertEqual(data.index0.tolist(), [0, 1])
+        self.assertEqual(data.arr.tolist(), [1.2, 3.4])
+        self.assertEqual(len(data.arrays), 2, data.arrays)
+
+    def test_array_and_scalar(self):
+        self.p1.set(42)
+        data = Measure(MultiGetter(arr=(5, 6)), self.p1).run_temp()
+
+        self.assertEqual(data.single_set.tolist(), [0])
+        self.assertEqual(data.P1.tolist(), [42])
+        self.assertEqual(data.index0.tolist(), [0, 1])
+        self.assertEqual(data.arr.tolist(), [5, 6])
+        self.assertEqual(len(data.arrays), 4, data.arrays)