Repository: sunpy/sunpy

Issue:
Fetching HMI data fails whenn no attribute vso.Physobs is provided
When trying to download HMI data without specifying a vso.Physobs attribute, the query fails with the following error: 
>WebFault: Server raised fault: 'Element '' can't be allowed in valid XML message. Died. at /opt/vso/lib/perl5/SOAP/Lite.pm line 1480.
'

Searching for the data is succesful, and when at least one vso.Physobs attribute is specified, the download is successful.

Example code:
```python
import sunpy
from sunpy.net import Fido, attrs as a

#this queryresult cannot be downloaded
qr = Fido.search(a.Time('2017-09-02 23:52:00', '2017-09-02 23:52:30') 
                 & a.Instrument('HMI'))
print(qr)
Fido.fetch(qr)

#this queryresult can be downloaded
qr = Fido.search(a.Time('2017-09-02 23:52:00', '2017-09-02 23:52:30') 
                 & a.Instrument('HMI')
                 & (a.vso.Physobs('intensity') | a.vso.Physobs('LOS_magnetic_field')))
print(qr)
```

Traceback:
[traceback.txt](https://github.com/sunpy/sunpy/files/1284485/traceback.txt)


Discussion:
Thanks for the bug report @Hackempluf. This looks like a VSO side issue to me, I will try and get some VSO peoples eyes on it.
This is believed to be the message that SunPy was sending to the VSO:

    <?xml version="1.0" encoding="UTF-8"?>
    <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns0="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns1="http://virtualsolar.org/VSO/VSOi" xmlns:ns2="http://schemas.xmlsoap.org/soap/envelope/" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
       <SOAP-ENV:Header/>
       <ns2:Body>
          <ns1:GetData>
             <body>
                <request>
                   <method>
                      <methodtype>URL-FILE_Rice</methodtype>
                      <methodtype>URL-FILE</methodtype>
                   </method>
                   <datacontainer>
                      <datarequestitem>
                         <provider>JSOC</provider>
                         <fileiditem>
                            <fileid>hmi__ic_45s:17301111:17301111</fileid>
                            <fileid>hmi__m_45s:17301111:17301111</fileid>
                            <fileid>hmi__v_45s:17301111:17301111</fileid>
                         </fileiditem>
                      </datarequestitem>
                   </datacontainer>
                </request>
             </body>
          </ns1:GetData>
       </ns2:Body>
    </SOAP-ENV:Envelope>

For whatever reason, it's not parsing the name of the element (e.g., '<fileid>'), as the code that's throwing the error is 

    die "Element '$tag' can't be allowed in valid XML message. Died."
        if $tag !~ /^$SOAP::Constants::NSMASK$/o;

    # note : $NSMASK = '[a-zA-Z_:][\w.\-:]*';

I should've mentioned -- $NSMASK likely doesn't matter, as attempting to print $tag gives an empty string.  But as $tag is passed into the function, this is going to require a trace to debug it.

And only having 3 HMI records for that time period is correct.  (IDL reports 2 series w/out matches for the time period requested): 

    Fido.search(a.Time('2017-09-02 23:52:00', '2017-09-02 23:52:30') & a.Instrument('HMI'))
@jhourcle Am I to interpret this as a VSO issue or a requirement to change our side?
@Cadair If this is an error related to sunpy (not a VSO fault), I would like to give it a try. Should I start?
@Cadair Is there a way to know how VSO reacts to our queries, I mean how does it parses our query?
(We should also add a test for this usecase, i.e. a search involving `Physobs`.
Okay @Cadair I will do it just when I will be done with my previous issue.

Resolving patch:
diff --git a/sunpy/net/vso/vso.py b/sunpy/net/vso/vso.py
--- a/sunpy/net/vso/vso.py
+++ b/sunpy/net/vso/vso.py
@@ -14,10 +14,12 @@
 import re
 import os
 import sys
+import copy
 import logging
 import requests
 import warnings
 import socket
+import itertools
 
 from datetime import datetime, timedelta
 from functools import partial
@@ -711,14 +713,37 @@ def create_getdatarequest(self, maps, methods, info=None):
         if info is None:
             info = {}
 
+        # For the JSOC provider we need to make a DataRequestItem for each
+        # series, not just one for the whole provider.
+
+        # Remove JSOC provider items from the map
+        jsoc = maps.pop('JSOC', [])
+
+        # Make DRIs for everything that's not JSOC one per provider
+        dris = [self.make('DataRequestItem', provider=k, fileiditem__fileid=[v])
+                for k, v in iteritems(maps)]
+
+        def series_func(x):
+            """ Extract the series from the fileid. """
+            return x.split(':')[0]
+
+        # Sort the JSOC fileids by series
+        # This is a precursor to groupby as recommended by the groupby docs
+        series_sorted = sorted(jsoc, key=series_func)
+
+        # Iterate over the series and make a DRI for each.
+        # groupby creates an iterator based on a key function, in this case
+        # based on the series (the part before the first ':')
+        for series, fileids in itertools.groupby(series_sorted, key=series_func):
+            dris.append(self.make('DataRequestItem',
+                                  provider='JSOC',
+                                  fileiditem__fileid=[list(fileids)]))
+
         return self.make(
             'VSOGetDataRequest',
             request__method__methodtype=methods,
             request__info=info,
-            request__datacontainer__datarequestitem=[
-                self.make('DataRequestItem', provider=k, fileiditem__fileid=[v])
-                for k, v in iteritems(maps)
-            ]
+            request__datacontainer__datarequestitem=dris
         )
 
     # pylint: disable=R0913,R0912


Test patch:
diff --git a/sunpy/net/tests/test_vso.py b/sunpy/net/tests/test_vso.py
--- a/sunpy/net/tests/test_vso.py
+++ b/sunpy/net/tests/test_vso.py
@@ -401,3 +401,26 @@ def test_QueryResponse_build_table_with_no_end_time():
     end_time_ = table['End Time']
     assert len(end_time_) == 1
     assert end_time_[0] == 'None'
+
+
+@pytest.mark.remote_data
+def test_vso_hmi(client, tmpdir):
+    """
+    This is a regression test for https://github.com/sunpy/sunpy/issues/2284
+    """
+    res = client.search(va.Time('2017-09-02 23:52:00', '2017-09-02 23:54:00'),
+                        va.Instrument('HMI') | va.Instrument('AIA'))
+
+    dr = client.make_getdatarequest(res)
+
+    # Extract the DRIs from the request
+    dris = dr.request.datacontainer.datarequestitem
+
+    # 3 HMI series and one AIA
+    assert len(dris) == 4
+
+    # For each DataRequestItem assert that there is only one series in it.
+    for dri in dris:
+        fileids = dri.fileiditem.fileid[0]
+        series = list(map(lambda x: x.split(':')[0], fileids))
+        assert all([s == series[0] for s in series])