Repository: scikit-hep/pyhf

Issue:
Parse normalization factor ranges from xml
# Description

The .xml workspace specification includes the ranges of normalization factors (NFs), such as 
```
<NormFactor Name="my_NF"  Val="1"  High="20"  Low="-20"  Const="False"   />
```
A simple example can be found [here](https://github.com/alexander-held/template_fit_workflows/blob/master/TRExFitter/minimal_example/RooStats/minimal_example_Signal_region.xml).
When the NF affects multiple samples or channels, the information is consistently specified in all instances.

Given that this information is specified in the xml, it should be picked up by `pyhf.readxml.parse()` and subsequently be available within pyhf.

## Is your feature request related to a problem? Please describe.
The aim of this is to reduce small differences between the results of `xml workspace -> fit within ROOT` and `xml workspace -> translate to pyhf -> fit` that can be hard to track down.

### Describe the solution you'd like
I would like the ranges to be picked up automatically by pyhf.

### Describe alternatives you've considered
Could alternatively specify ranges manually from within pyhf.

# Relevant Issues and Pull Requests
None that I am aware of.

# Additional context
None



Discussion:
Notes:
- does not change schema or API
- simply need to update readxml/writexml to handle reading/writing out the relevant information

Assumption:
- the high/low for NormFactor are set consistently everywhere for the given systematic
"the high/low for NormFactor are set consistently everywhere for the given systematic" that is just another one of these "reduction" reqs no?
> "the high/low for NormFactor are set consistently everywhere for the given systematic" that is just another one of these "reduction" reqs no?

I'm not sure. I'm thinking this doesn't need to be true. The correlation is the parameter, not the value. It seems plausible that one can have:

```
<Sample Name="ttbar" >
  <NormFactor Name="my_NF"  Val="1"  High="20"  Low="-20"  Const="False"   />
</Sample>
<Sample Name="vjets" >
  <NormFactor Name="my_NF"  Val="1"  High="10"  Low="-10"  Const="False"   />
</Sample>
```

so it doesn't need to be consistent... This does cut away a piece of the space that HiFa can do. Perhaps the parameter configuration should be on the modifier itself for normfactor.
I don't see how the above sample would work in the fit - `my_NF` is one parameter, so there can only be one lower bound for it. I guess you could argue that you take the larger boundary and cap it at -10 for `vjets`, but I cannot think of a case where that would be desired.
You're right. I'm thinking of other constrained systematics where this is more configurable. This doesn't make sense in the context of HiFa. So yes, I can write up something tonight in a branch and test it soon.

Resolving patch:
diff --git a/src/pyhf/readxml.py b/src/pyhf/readxml.py
--- a/src/pyhf/readxml.py
+++ b/src/pyhf/readxml.py
@@ -69,6 +69,7 @@ def process_sample(
 
     data, err = import_root_histogram(rootdir, inputfile, histopath, histoname)
 
+    parameter_configs = []
     modifiers = []
     # first check if we need to add lumi modifier for this sample
     if sample.attrib.get("NormalizeByTheory", "False") == 'True':
@@ -101,6 +102,15 @@ def process_sample(
             modifiers.append(
                 {'name': modtag.attrib['Name'], 'type': 'normfactor', 'data': None}
             )
+            parameter_config = {
+                'name': modtag.attrib['Name'],
+                'bounds': [[float(modtag.attrib['Low']), float(modtag.attrib['High'])]],
+                'inits': [float(modtag.attrib['Val'])],
+            }
+            if modtag.attrib.get('Const'):
+                parameter_config['fixed'] = modtag.attrib['Const'] == 'True'
+
+            parameter_configs.append(parameter_config)
         elif modtag.tag == 'HistoSys':
             lo, _ = import_root_histogram(
                 rootdir,
@@ -165,7 +175,12 @@ def process_sample(
         else:
             log.warning('not considering modifier tag %s', modtag)
 
-    return {'name': sample.attrib['Name'], 'data': data, 'modifiers': modifiers}
+    return {
+        'name': sample.attrib['Name'],
+        'data': data,
+        'modifiers': modifiers,
+        'parameter_configs': parameter_configs,
+    }
 
 
 def process_data(sample, rootdir, inputfile, histopath):
@@ -197,19 +212,24 @@ def process_channel(channelxml, rootdir, track_progress=False):
     channelname = channel.attrib['Name']
 
     results = []
+    channel_parameter_configs = []
     for sample in samples:
         samples.set_description('  - sample {}'.format(sample.attrib.get('Name')))
         result = process_sample(
             sample, rootdir, inputfile, histopath, channelname, track_progress
         )
+        channel_parameter_configs.extend(result.pop('parameter_configs'))
         results.append(result)
 
-    return channelname, parsed_data, results
+    return channelname, parsed_data, results, channel_parameter_configs
 
 
-def process_measurements(toplvl):
+def process_measurements(toplvl, other_parameter_configs=None):
     results = []
+    other_parameter_configs = other_parameter_configs if other_parameter_configs else []
+
     for x in toplvl.findall('Measurement'):
+        parameter_configs_map = {k['name']: dict(**k) for k in other_parameter_configs}
         lumi = float(x.attrib['Lumi'])
         lumierr = lumi * float(x.attrib['LumiRelErr'])
 
@@ -228,13 +248,14 @@ def process_measurements(toplvl):
                 ],
             },
         }
+
         for param in x.findall('ParamSetting'):
             # determine what all parameters in the paramsetting have in common
             overall_param_obj = {}
             if param.attrib.get('Const'):
                 overall_param_obj['fixed'] = param.attrib['Const'] == 'True'
             if param.attrib.get('Val'):
-                overall_param_obj['value'] = param.attrib['Val']
+                overall_param_obj['inits'] = [float(param.attrib['Val'])]
 
             # might be specifying multiple parameters in the same ParamSetting
             if param.text:
@@ -243,13 +264,40 @@ def process_measurements(toplvl):
                     if param_name == 'Lumi':
                         result['config']['parameters'][0].update(overall_param_obj)
                     else:
-                        param_obj = {'name': param_name}
+                        # pop from parameter_configs_map because we don't want to duplicate
+                        param_obj = parameter_configs_map.pop(
+                            param_name, {'name': param_name}
+                        )
+                        # ParamSetting will always take precedence
                         param_obj.update(overall_param_obj)
-                        result['config']['parameters'].append(param_obj)
+                        # add it back in to the parameter_configs_map
+                        parameter_configs_map[param_name] = param_obj
+        result['config']['parameters'].extend(parameter_configs_map.values())
         results.append(result)
+
     return results
 
 
+def dedupe_parameters(parameters):
+    duplicates = {}
+    for p in parameters:
+        duplicates.setdefault(p['name'], []).append(p)
+    for parname in duplicates.keys():
+        parameter_list = duplicates[parname]
+        if len(parameter_list) == 1:
+            continue
+        elif any(p != parameter_list[0] for p in parameter_list[1:]):
+            for p in parameter_list:
+                log.warning(p)
+            raise RuntimeError(
+                'cannot import workspace due to incompatible parameter configurations for {0:s}.'.format(
+                    parname
+                )
+            )
+    # no errors raised, de-dupe and return
+    return list({v['name']: v for v in parameters}.values())
+
+
 def parse(configfile, rootdir, track_progress=False):
     toplvl = ET.parse(configfile)
     inputs = tqdm.tqdm(
@@ -259,20 +307,24 @@ def parse(configfile, rootdir, track_progress=False):
     )
 
     channels = {}
+    parameter_configs = []
     for inp in inputs:
         inputs.set_description('Processing {}'.format(inp))
-        channel, data, samples = process_channel(
+        channel, data, samples, channel_parameter_configs = process_channel(
             ET.parse(os.path.join(rootdir, inp)), rootdir, track_progress
         )
         channels[channel] = {'data': data, 'samples': samples}
+        parameter_configs.extend(channel_parameter_configs)
 
+    parameter_configs = dedupe_parameters(parameter_configs)
     result = {
-        'measurements': process_measurements(toplvl),
+        'measurements': process_measurements(
+            toplvl, other_parameter_configs=parameter_configs
+        ),
         'channels': [{'name': k, 'samples': v['samples']} for k, v in channels.items()],
         'observations': [{'name': k, 'data': v['data']} for k, v in channels.items()],
         'version': utils.SCHEMA_VERSION,
     }
-
     utils.validate(result, 'workspace.json')
 
     return result
diff --git a/src/pyhf/writexml.py b/src/pyhf/writexml.py
--- a/src/pyhf/writexml.py
+++ b/src/pyhf/writexml.py
@@ -12,6 +12,17 @@
 
 log = logging.getLogger(__name__)
 
+# 'spec' gets passed through all functions as NormFactor is a unique case of having
+# parameter configurations stored at the modifier-definition-spec level. This means
+# that build_modifier() needs access to the measurements. The call stack is:
+#
+#      writexml
+#          ->build_channel
+#              ->build_sample
+#                  ->build_modifier
+#
+#  Therefore, 'spec' needs to be threaded through all these calls.
+
 
 def _make_hist_name(channel, sample, modifier='', prefix='hist', suffix=''):
     return "{prefix}{middle}{suffix}".format(
@@ -89,7 +100,7 @@ def build_measurement(measurementspec):
     return meas
 
 
-def build_modifier(modifierspec, channelname, samplename, sampledata):
+def build_modifier(spec, modifierspec, channelname, samplename, sampledata):
     if modifierspec['name'] == 'lumi':
         return None
     mod_map = {
@@ -115,9 +126,31 @@ def build_modifier(modifierspec, channelname, samplename, sampledata):
         attrs['High'] = str(modifierspec['data']['hi'])
         attrs['Low'] = str(modifierspec['data']['lo'])
     elif modifierspec['type'] == 'normfactor':
-        attrs['Val'] = '1'
-        attrs['High'] = '10'
-        attrs['Low'] = '0'
+        # NB: only look at first measurement for normfactor configs. In order
+        # to dump as HistFactory XML, this has to be the same for all
+        # measurements or it will not work correctly. Why?
+        #
+        # Unlike other modifiers, NormFactor has the unique circumstance of
+        # defining its parameter configurations at the modifier level inside
+        # the channel specification, instead of at the measurement level, like
+        # all of the other modifiers.
+        #
+        # However, since I strive for perfection, the "Const" attribute will
+        # never be set here, but at the per-measurement configuration instead
+        # like all other parameters. This is an acceptable compromise.
+        #
+        # Lastly, if a normfactor parameter configuration doesn't exist in the
+        # first measurement parameter configuration, then set defaults.
+        val = 1
+        low = 0
+        high = 10
+        for p in spec['measurements'][0]['config']['parameters']:
+            if p['name'] == modifierspec['name']:
+                val = p['inits'][0]
+                low, high = p['bounds'][0]
+        attrs['Val'] = str(val)
+        attrs['Low'] = str(low)
+        attrs['High'] = str(high)
     elif modifierspec['type'] == 'staterror':
         attrs['Activate'] = 'True'
         attrs['HistoName'] = _make_hist_name(
@@ -161,7 +194,7 @@ def build_modifier(modifierspec, channelname, samplename, sampledata):
     return modifier
 
 
-def build_sample(samplespec, channelname):
+def build_sample(spec, samplespec, channelname):
     histname = _make_hist_name(channelname, samplespec['name'])
     attrs = {
         'Name': samplespec['name'],
@@ -175,7 +208,7 @@ def build_sample(samplespec, channelname):
         if modspec['type'] == 'lumi':
             sample.attrib.update({'NormalizeByTheory': 'True'})
         modifier = build_modifier(
-            modspec, channelname, samplespec['name'], samplespec['data']
+            spec, modspec, channelname, samplespec['name'], samplespec['data']
         )
         if modifier is not None:
             sample.append(modifier)
@@ -192,7 +225,7 @@ def build_data(obsspec, channelname):
     return data
 
 
-def build_channel(channelspec, obsspec):
+def build_channel(spec, channelspec, obsspec):
     channel = ET.Element(
         'Channel', Name=channelspec['name'], InputFile=_ROOT_DATA_FILE._path
     )
@@ -200,7 +233,7 @@ def build_channel(channelspec, obsspec):
         data = build_data(obsspec, channelspec['name'])
         channel.append(data)
     for samplespec in channelspec['samples']:
-        channel.append(build_sample(samplespec, channelspec['name']))
+        channel.append(build_sample(spec, samplespec, channelspec['name']))
     return channel
 
 
@@ -221,7 +254,7 @@ def writexml(spec, specdir, data_rootdir, resultprefix):
                 specdir, '{0:s}_{1:s}.xml'.format(resultprefix, channelspec['name'])
             )
             with open(channelfilename, 'w') as channelfile:
-                channel = build_channel(channelspec, spec.get('observations'))
+                channel = build_channel(spec, channelspec, spec.get('observations'))
                 indent(channel)
                 channelfile.write(
                     "<!DOCTYPE Channel SYSTEM '../HistFactorySchema.dtd'>\n\n"


Test patch:
diff --git a/tests/test_export.py b/tests/test_export.py
--- a/tests/test_export.py
+++ b/tests/test_export.py
@@ -195,7 +195,11 @@ def test_export_modifier(mocker, spec, has_root_data, attrs):
 
     mocker.patch('pyhf.writexml._ROOT_DATA_FILE')
     modifier = pyhf.writexml.build_modifier(
-        modifierspec, channelname, samplename, sampledata
+        {'measurements': [{'config': {'parameters': []}}]},
+        modifierspec,
+        channelname,
+        samplename,
+        sampledata,
     )
     # if the modifier is a staterror, it has no Name
     if 'Name' in modifier.attrib:
@@ -204,6 +208,53 @@ def test_export_modifier(mocker, spec, has_root_data, attrs):
     assert pyhf.writexml._ROOT_DATA_FILE.__setitem__.called == has_root_data
 
 
+@pytest.mark.parametrize(
+    "spec, normfactor_config",
+    [
+        (spec_staterror(), dict(name='mu', inits=[1.0], bounds=[[0.0, 8.0]])),
+        (spec_histosys(), dict()),
+        (spec_normsys(), dict(name='mu', inits=[2.0], bounds=[[0.0, 10.0]])),
+        (spec_shapesys(), dict(name='mu', inits=[1.0], bounds=[[5.0, 10.0]])),
+    ],
+    ids=['upper-bound', 'empty-config', 'init', 'lower-bound'],
+)
+def test_export_modifier_normfactor(mocker, spec, normfactor_config):
+    channelspec = spec['channels'][0]
+    channelname = channelspec['name']
+    samplespec = channelspec['samples'][0]
+    samplename = samplespec['name']
+    sampledata = samplespec['data']
+    modifierspec = samplespec['modifiers'][0]
+
+    mocker.patch('pyhf.writexml._ROOT_DATA_FILE')
+    modifier = pyhf.writexml.build_modifier(
+        {
+            'measurements': [
+                {
+                    'config': {
+                        'parameters': [normfactor_config] if normfactor_config else []
+                    }
+                }
+            ]
+        },
+        modifierspec,
+        channelname,
+        samplename,
+        sampledata,
+    )
+
+    assert all(attr in modifier.attrib for attr in ['Name', 'Val', 'High', 'Low'])
+    assert float(modifier.attrib['Val']) == normfactor_config.get('inits', [1.0])[0]
+    assert (
+        float(modifier.attrib['Low'])
+        == normfactor_config.get('bounds', [[0.0, 10.0]])[0][0]
+    )
+    assert (
+        float(modifier.attrib['High'])
+        == normfactor_config.get('bounds', [[0.0, 10.0]])[0][1]
+    )
+
+
 @pytest.mark.parametrize(
     "spec",
     [spec_staterror(), spec_histosys(), spec_normsys(), spec_shapesys()],
@@ -218,7 +269,7 @@ def test_export_sample(mocker, spec):
 
     mocker.patch('pyhf.writexml.build_modifier', return_value=ET.Element("Modifier"))
     mocker.patch('pyhf.writexml._ROOT_DATA_FILE')
-    sample = pyhf.writexml.build_sample(samplespec, channelname)
+    sample = pyhf.writexml.build_sample({}, samplespec, channelname)
     assert sample.attrib['Name'] == samplespec['name']
     assert sample.attrib['HistoName']
     assert sample.attrib['InputFile']
@@ -242,7 +293,11 @@ def test_export_sample_zerodata(mocker, spec):
     with pytest.warns(None) as record:
         for modifierspec in samplespec['modifiers']:
             modifier = pyhf.writexml.build_modifier(
-                modifierspec, channelname, samplename, sampledata
+                {'measurements': [{'config': {'parameters': []}}]},
+                modifierspec,
+                channelname,
+                samplename,
+                sampledata,
             )
     assert not record.list
 
@@ -259,7 +314,7 @@ def test_export_channel(mocker, spec):
     mocker.patch('pyhf.writexml.build_data', return_value=ET.Element("Data"))
     mocker.patch('pyhf.writexml.build_sample', return_value=ET.Element("Sample"))
     mocker.patch('pyhf.writexml._ROOT_DATA_FILE')
-    channel = pyhf.writexml.build_channel(channelspec, {})
+    channel = pyhf.writexml.build_channel({}, channelspec, {})
     assert channel.attrib['Name'] == channelspec['name']
     assert channel.attrib['InputFile']
     assert pyhf.writexml.build_data.called is False
diff --git a/tests/test_import.py b/tests/test_import.py
--- a/tests/test_import.py
+++ b/tests/test_import.py
@@ -4,6 +4,7 @@
 import uproot
 import os
 import pytest
+import xml.etree.cElementTree as ET
 
 
 def assert_equal_dictionary(d1, d2):
@@ -16,6 +17,96 @@ def assert_equal_dictionary(d1, d2):
             assert d1[k] == d2[k]
 
 
+def test_dedupe_parameters():
+    parameters = [
+        {'name': 'SigXsecOverSM', 'bounds': [[0.0, 10.0]]},
+        {'name': 'SigXsecOverSM', 'bounds': [[0.0, 10.0]]},
+    ]
+    assert len(pyhf.readxml.dedupe_parameters(parameters)) == 1
+    parameters[1]['bounds'] = [[0.0, 2.0]]
+    with pytest.raises(RuntimeError) as excinfo:
+        pyhf.readxml.dedupe_parameters(parameters)
+        assert 'SigXsecOverSM' in str(excinfo.value)
+
+
+def test_process_normfactor_configs():
+    # Check to see if mu_ttbar NormFactor is overridden correctly
+    # - ParamSetting has a config for it
+    # - other_parameter_configs has a config for it
+    # Make sure that when two measurements exist, we're copying things across correctly
+    toplvl = ET.Element("Combination")
+    meas = ET.Element(
+        "Measurement",
+        Name='NormalMeasurement',
+        Lumi=str(1.0),
+        LumiRelErr=str(0.017),
+        ExportOnly=str(True),
+    )
+    poiel = ET.Element('POI')
+    poiel.text = 'mu_SIG'
+    meas.append(poiel)
+
+    setting = ET.Element('ParamSetting', Const='True')
+    setting.text = ' '.join(['Lumi', 'mu_both', 'mu_paramSettingOnly'])
+    meas.append(setting)
+
+    setting = ET.Element('ParamSetting', Val='2.0')
+    setting.text = ' '.join(['mu_both'])
+    meas.append(setting)
+
+    toplvl.append(meas)
+
+    meas = ET.Element(
+        "Measurement",
+        Name='ParallelMeasurement',
+        Lumi=str(1.0),
+        LumiRelErr=str(0.017),
+        ExportOnly=str(True),
+    )
+    poiel = ET.Element('POI')
+    poiel.text = 'mu_BKG'
+    meas.append(poiel)
+
+    setting = ET.Element('ParamSetting', Val='3.0')
+    setting.text = ' '.join(['mu_both'])
+    meas.append(setting)
+
+    toplvl.append(meas)
+
+    other_parameter_configs = [
+        dict(name='mu_both', inits=[1.0], bounds=[[1.0, 5.0]], fixed=False),
+        dict(name='mu_otherConfigOnly', inits=[1.0], bounds=[[0.0, 10.0]], fixed=False),
+    ]
+
+    result = pyhf.readxml.process_measurements(
+        toplvl, other_parameter_configs=other_parameter_configs
+    )
+    result = {
+        m['name']: {k['name']: k for k in m['config']['parameters']} for m in result
+    }
+    assert result
+
+    # make sure ParamSetting configs override NormFactor configs
+    assert result['NormalMeasurement']['mu_both']['fixed']
+    assert result['NormalMeasurement']['mu_both']['inits'] == [2.0]
+    assert result['NormalMeasurement']['mu_both']['bounds'] == [[1.0, 5.0]]
+
+    # make sure ParamSetting is doing the right thing
+    assert result['NormalMeasurement']['mu_paramSettingOnly']['fixed']
+    assert 'inits' not in result['NormalMeasurement']['mu_paramSettingOnly']
+    assert 'bounds' not in result['NormalMeasurement']['mu_paramSettingOnly']
+
+    # make sure our code doesn't accidentally override other parameter configs
+    assert not result['NormalMeasurement']['mu_otherConfigOnly']['fixed']
+    assert result['NormalMeasurement']['mu_otherConfigOnly']['inits'] == [1.0]
+    assert result['NormalMeasurement']['mu_otherConfigOnly']['bounds'] == [[0.0, 10.0]]
+
+    # make sure settings from one measurement don't leak to other
+    assert not result['ParallelMeasurement']['mu_both']['fixed']
+    assert result['ParallelMeasurement']['mu_both']['inits'] == [3.0]
+    assert result['ParallelMeasurement']['mu_both']['bounds'] == [[1.0, 5.0]]
+
+
 def test_import_measurements():
     parsed_xml = pyhf.readxml.parse(
         'validation/xmlimport_input/config/example.xml', 'validation/xmlimport_input/'
@@ -26,9 +117,9 @@ def test_import_measurements():
     measurement_configs = measurements[0]['config']
 
     assert 'parameters' in measurement_configs
-    assert len(measurement_configs['parameters']) == 2
-    assert measurement_configs['parameters'][0]['name'] == 'lumi'
-    assert measurement_configs['parameters'][1]['name'] == 'alpha_syst1'
+    assert len(measurement_configs['parameters']) == 3
+    parnames = [p['name'] for p in measurement_configs['parameters']]
+    assert sorted(parnames) == sorted(['lumi', 'SigXsecOverSM', 'alpha_syst1'])
 
     lumi_param_config = measurement_configs['parameters'][0]
     assert 'auxdata' in lumi_param_config
@@ -193,3 +284,22 @@ def test_import_shapesys():
     assert channels['channel1']['samples'][1]['modifiers'][1]['data'] == pytest.approx(
         [10.0, 1.5e-5]
     )
+
+
+def test_import_normfactor_bounds():
+    parsed_xml = pyhf.readxml.parse(
+        'validation/xmlimport_input2/config/example.xml', 'validation/xmlimport_input2'
+    )
+
+    ws = pyhf.Workspace(parsed_xml)
+    assert ('SigXsecOverSM', 'normfactor') in ws.modifiers
+    parameters = [
+        p
+        for p in ws.get_measurement(measurement_name='GaussExample')['config'][
+            'parameters'
+        ]
+        if p['name'] == 'SigXsecOverSM'
+    ]
+    assert len(parameters) == 1
+    parameter = parameters[0]
+    assert parameter['bounds'] == [[0, 10]]