Repository: yt-project/yt

Issue:
Extraheader with Gadget
<!--To help us understand and resolve your issue, please fill out the form to
the best of your ability.-->
<!--You can feel free to delete the sections that do not apply.-->

### Bug report

**Bug summary**

<!--A short 1-2 sentences that succinctly describes the bug-->
When reading a gadget file with an extra header (e.g. https://obswww.unige.ch/~lhausamm/swift/IC/AgoraDisk/Gear/snapshot_0000), you do not take it into account here https://github.com/yt-project/yt/blob/master/yt/frontends/gadget/data_structures.py#L396

**Code for reproduction**

<!--A minimum code snippet required to reproduce the bug, also minimizing the
number of dependencies required.-->
I am using the AGORA analysis script (https://bitbucket.org/mornkr/agora-analysis-script/src/8f12b112e19614d66274ece5a5bf21af23a76778/script-Paper4-disk-new.py?at=default&fileviewer=file-view-default) where they define the extra header here https://bitbucket.org/mornkr/agora-analysis-script/src/8f12b112e19614d66274ece5a5bf21af23a76778/script-Paper4-disk-new.py?at=default&fileviewer=file-view-default#script-Paper4-disk-new.py-150 and the code fails when adding a field here https://bitbucket.org/mornkr/agora-analysis-script/src/8f12b112e19614d66274ece5a5bf21af23a76778/script-Paper4-disk-new.py?at=default&fileviewer=file-view-default#script-Paper4-disk-new.py-501

<!-- If you need to use a data file to trigger the issue you're having, consider
using one of the datasets from the yt data hub (http://yt-project.org/data). If
your issue cannot be triggered using a public dataset, you can use the yt
curldrop (https://docs.hub.yt/services.html#curldrop) to share data
files. Please include a link to the dataset in the issue if you use the
curldrop.-->

**Actual outcome**

<!--The output produced by the above code, which may be a screenshot, console
output, etc.-->

```
Traceback (most recent call last):
  File "./plot_solution.py", line 519, in <module>
    pf.add_field(("gas", "density_squared"), function=_density_squared, units="g**2/cm**6")
  File "/usr/local/lib/python2.7/dist-packages/yt/data_objects/static_output.py", line 1191, in add_field
    self.index
  File "/usr/local/lib/python2.7/dist-packages/yt/data_objects/static_output.py", line 504, in index
    self, dataset_type=self.dataset_type)
  File "/usr/local/lib/python2.7/dist-packages/yt/geometry/particle_geometry_handler.py", line 39, in __init__
    super(ParticleIndex, self).__init__(ds, dataset_type)
  File "/usr/local/lib/python2.7/dist-packages/yt/geometry/geometry_handler.py", line 50, in __init__
    self._setup_geometry()
  File "/usr/local/lib/python2.7/dist-packages/yt/geometry/particle_geometry_handler.py", line 50, in _setup_geometry
    self._initialize_particle_handler()
  File "/usr/local/lib/python2.7/dist-packages/yt/geometry/particle_geometry_handler.py", line 104, in _initialize_particle_handler
    self._initialize_indices()
  File "/usr/local/lib/python2.7/dist-packages/yt/geometry/particle_geometry_handler.py", line 134, in _initialize_indices
    self.io._initialize_index(data_file, self.regions)
  File "/usr/local/lib/python2.7/dist-packages/yt/frontends/gadget/io.py", line 351, in _initialize_index
    data_file, count, 0, regions, DLE, DRE)
  File "/usr/local/lib/python2.7/dist-packages/yt/frontends/gadget/io.py", line 334, in _get_morton_from_position
    pp = np.fromfile(f, dtype=self._endian + self._float_type,
TypeError: cannot concatenate 'str' and 'int' objects

```

**Expected outcome**

<!--A description of the expected outcome from the code snippet-->
<!--If this used to work in an earlier version of yt, please note the
version it used to work on-->

**Version Information**
<!--Please specify your platform and versions of the relevant libraries you are
using:-->
  * Operating System: Linux Ubuntu 16.04
  * Python Version: 2.7.12
  * yt version: 3.4.1 (installed through pip)
  * Other Libraries (if applicable): 

<!--Please tell us how you installed yt and python e.g., from source,
pip, conda. If you installed from conda, please specify which channel you used
if not the default-->




Discussion:
Thanks for the report and steps to reproduce. I will try to take a look at this soon.
Hi @loikki, I ran into the same issue with another dataset. Basically, I can trigger the issue by

```python
from yt.frontends.gadget.api import GadgetDataset
ds = GadgetDataset('path/to/snapshot')
ds.field_list
```

But my dataset is too large (> 30G) to share. I'm wondering if you could share your dataset again (the first link is broken)? I'd like to take a deeper look and hopefully come up with a fix.
We do have a number of twiddles for specifying additional fields and whatnot (`gadget_header_spec` https://github.com/yt-project/yt/blob/master/yt/frontends/gadget/definitions.py#L17 ), but it looks like this falls into a corner case.
@MatthewTurk I guess the fix would be to check the code and use a calculated header size instead of 256, for example in this [line](https://github.com/yt-project/yt/blob/15c0691d9de2abdf320eaaccab5d2b8ac3af4aef/yt/frontends/gadget/data_structures.py#L396) as @loikki pointed out.
Sure, if real-world datasets don't have that exact amount of padding, we should inspect the data and calculate it.

FWIW I'm pretty sure that constant comes from the documentation for the the Gadget-2 output format in the user guide, I think it was fair of yt to expect real-world datasets to conform to the format documented there although if that's not the case we should definitely fix it :)

Presumably there's a way to do this either by looking very low-level at the bytes in the header or through creative use of the `struct` module.
@ngoldbaum Yeah, 256 is the Gadget standard. But since [`header_spec` is already configurable](https://github.com/yt-project/yt/blob/15c0691d9de2abdf320eaaccab5d2b8ac3af4aef/yt/frontends/gadget/data_structures.py#L107), to be self-consistent, I think we should use a caculated header size.

I'm willing to make the change if you think this is the right way to solve the issue.
Sorry for the late answer.

I have reduced in size my snapshot (less than 1Mo) and you can download it here https://obswww.unige.ch/~lhausamm/git/test

I am using the following script to read it:
```python
from yt.frontends.gadget.definitions import gadget_header_specs
from yt.frontends.gadget.definitions import gadget_field_specs
from yt.mods import *

filename = "test"

gadget_header_specs["chemistry"] = (('h1',  4, 'c'), ('h2',  4, 'c'),
                                    ('empty', 256, 'c'),)
gear_ptype_specs = ("Gas", "Stars", "Halo", "Disk", "Bulge", "Bndry")

gadget_default_unit_base = {
    'UnitLength_in_cm': 3.08568e+21,
    'UnitMass_in_g': 1.989e+43,
    'UnitVelocity_in_cm_per_s': 100000
}

pf = GadgetDataset(filename, unit_base=gadget_default_unit_base,
                   bounding_box=[[-1000.0, 1000.0], [-1000.0, 1000.0],
                                 [-1000.0, 1000.0]],
                   header_spec="default+chemistry",
                   ptype_spec=gear_ptype_specs)
```
Hmm maybe I merged Bili's PR prematurely, I didn't see this comment.

@qobilidop when I run @loikki's test script using the current master branch, I get:

```
Traceback (most recent call last):
  File "testg.py", line 21, in <module>
    ptype_spec=gear_ptype_specs)
  File "/Users/goldbaum/Documents/yt-git-fixes/yt/frontends/gadget/data_structures.py", line 171, in __init__
    kernel_name=kernel_name)
  File "/Users/goldbaum/Documents/yt-git-fixes/yt/frontends/sph/data_structures.py", line 36, in __init__
    n_ref=n_ref, over_refine_factor=over_refine_factor)
  File "/Users/goldbaum/Documents/yt-git-fixes/yt/data_objects/static_output.py", line 1502, in __init__
    units_override=units_override, unit_system=unit_system)
  File "/Users/goldbaum/Documents/yt-git-fixes/yt/data_objects/static_output.py", line 309, in __init__
    self._parse_parameter_file()
  File "/Users/goldbaum/Documents/yt-git-fixes/yt/frontends/gadget/data_structures.py", line 208, in _parse_parameter_file
    hvals = self._get_hvals()
  File "/Users/goldbaum/Documents/yt-git-fixes/yt/frontends/gadget/data_structures.py", line 196, in _get_hvals
    gformat = _get_gadget_format(self.parameter_filename, self._header_size)
  File "/Users/goldbaum/Documents/yt-git-fixes/yt/frontends/gadget/data_structures.py", line 75, in _get_gadget_format
    raise RuntimeError("Incorrect Gadget format %s!" % str(rhead))
RuntimeError: Incorrect Gadget format 256!
```

Is that what you'd expect? Is there some modification to the test script that needs to happen before the data loads?
@loikki Thanks for the test data!

@ngoldbaum I have taken a look. This happens because the header size is inferred to be 520 from the header spec (256 for default, and 4 + 4 + 256 for chemistry). But the first 4 bytes of the file gives 256, which is different. So in this sense, this is the intended behavior.

When I take a closer look, this file is actually comprised of two 256 header blocks. So I suspect the header spec is intended to tell yt to ignore the 4 + 4 block size bytes between the 2 blocks. I'm not sure if this was the intended behavior in yt previously. But I don't this is a good idea.

So the way I would suggest to fix the issue is to ensure there's only one header block in the snapshot file.
Sorry, to clarify, is the conclusion you came to that the file is corrupt somehow and Loic should fix it?

Given the real-world existence of this data, is there something we can do to detect this case, maybe print a warning, and load the data anyway? Or am I misunderstanding?
Yes, my conclusion is that there should be a single header block in the snapshot file. As long as that's the case, yt shall be able to load correctly.

But before that, there are several things I'm uncertain about, especially the interpretation of the header spec. Does it really mean to ignore the block size bytes between the two blocks? Maybe @loikki could clarify.

We could make this snapshot loadable by ignoring the block size bytes and rely on header spec solely. I just worry that without the checking of block size, there will be other issues.
We do have two headers. The decision was motivated to make the header more compatible with other softwares (in this way, we just need to skip the second header and not modify the first reading).

I am fine with any of your decision, I can easily trick yt to work on my old snapshot.
I guess that we can improve the error message if you wish to allow only a single header.
@loikki Thanks for the information!

So it comes down to how we want to support multiple headers. I have two proposals:

1. The cleaner way: For multiple headers, we pass a list to `header_spec`. `header_spec='default+chemistry'` means a single long header, while `header_spec=['default', 'chemistry']` means 2 headers.
2. The compatible way: We make it optional to turn off the header size check, and define some extra fields in `header_spec` (like the `h1`, `h2` in the example) to jump between headers. This would make the example script work out of the box. But it's a little bit hacky in my opinion.

Which one do people prefer? Are there other proposals? I'll make a new PR to support multiple headers once the design descision has been made. @ngoldbaum @MatthewTurk
Given the status quo is that things don't work, I don't think we're bound by backward compatibility guarantees (unless this used to work?). In that case I'd go with what you think is the cleaner option, #1.
I have no idea if it was working or not before, but I approve the first possibility, it is the cleanest way to do it.

Thank you very much for your help and time!

Resolving patch:
diff --git a/yt/frontends/gadget/data_structures.py b/yt/frontends/gadget/data_structures.py
--- a/yt/frontends/gadget/data_structures.py
+++ b/yt/frontends/gadget/data_structures.py
@@ -5,8 +5,6 @@
 
 
 """
-from __future__ import print_function
-
 #-----------------------------------------------------------------------------
 # Copyright (c) 2014, yt Development Team.
 #
@@ -54,39 +52,142 @@ def _byte_swap_32(x):
     return struct.unpack('>I', struct.pack('<I', x))[0]
 
 
-def _compute_header_size(header_spec):
-    return sum(field[1] * np.dtype(field[2]).itemsize for field in header_spec)
+class GadgetBinaryHeader(object):
+    """A convenient interface to Gadget binary header.
+    
+    This is a helper class to facilitate the main dataset and IO classes.
+    The main usage is through the GadgetDataset._header attribute. It is also
+    used stand-alone in GadgetDataset._is_valid method.
+    """
+    _placeholder_keys = ['unused', 'empty']
+
+    def __init__(self, filename, header_spec):
+        self.filename = filename
+        if isinstance(header_spec, string_types):
+            header_spec = [header_spec]
+        self.spec = [GadgetDataset._setup_binary_spec(hs, gadget_header_specs)
+                     for hs in header_spec]
+
+    @property
+    def float_type(self):
+        """Determine whether the floats are single or double precision."""
+        gformat, endianswap = self.gadget_format
+        # We do not currently support double precision snapshot format 2
+        if gformat == 2:
+            return 'f4'
+        else:
+            hvals = self.value
+            np0 = sum(hvals['Npart'])
+            with self.open() as f:
+                f.seek(self.position_offset)
+                # Calculate particle number assuming single precision
+                np1 = struct.unpack(endianswap + 'I', f.read(4))[0] / (4 * 3)
+            if np1 == np0:
+                return 'f4'
+            elif np1 == 2 * np0:
+                return 'f8'
+            else:
+                raise RuntimeError(
+                    "Gadget snapshot file is likely corrupted! "
+                    "Cannot determine float type."
+                )
+
+    @property
+    def gadget_format(self):
+        """Determine Gadget snapshot format and endianness.
+        
+        The difference between Gadget snapshot format 1 and 2 can be told from
+        the first 4 bytes of the file. For format 1, it's the header size. For
+        format 2, it's always 8.
+        """
+        first_header_size = self.size[0]
+        # Read the first 4 bytes assuming little endian int32
+        with self.open() as f:
+            (rhead,) = struct.unpack('<I', f.read(4))
+        # Foramt 1?
+        if rhead == first_header_size:
+            return 1, '<'
+        elif rhead == _byte_swap_32(first_header_size):
+            return 1, '>'
+        # Format 2?
+        elif rhead == 8:
+            return 2, '<'
+        elif rhead == _byte_swap_32(8):
+            return 2, '>'
+        else:
+            raise RuntimeError(
+                "Gadget snapshot file is likely corrupted! "
+                "The first 4 bytes represent %s (as little endian int32). "
+                "But we are looking for %s (for format 1) or 8 (for format 2)."
+                % (rhead, first_header_size)
+            )
+
+    @property
+    def position_offset(self):
+        """Offset to the position block."""
+        n_header = len(self.size)
+        offset = 8 * n_header + sum(self.size)
+        if self.gadget_format[0] == 2:
+            offset += SNAP_FORMAT_2_OFFSET * (n_header + 1)
+        return offset
+
+    @property
+    def size(self):
+        """Header size in bytes."""
+        def single_header_size(single_header_spec):
+            return sum(field[1] * np.dtype(field[2]).itemsize
+                       for field in single_header_spec)
+        return [single_header_size(spec) for spec in self.spec]
+
+    @property
+    def value(self):
+        """Header values as a dictionary."""
+        # The entries in this header are capitalized and named to match Table 4
+        # in the GADGET-2 user guide.
+        gformat, endianswap = self.gadget_format
+        # Read header
+        with self.open() as f:
+            hvals = {}
+            for spec in self.spec:
+                if gformat == 2:
+                    f.seek(f.tell() + SNAP_FORMAT_2_OFFSET)
+                hvals_new = read_record(f, spec, endian=endianswap)
+                hvals.update(hvals_new)
+        # Remove placeholder keys
+        for key in self._placeholder_keys:
+            if key in hvals:
+                del hvals[key]
+        # Convert length 1 list to scalar value
+        for i in hvals:
+            if len(hvals[i]) == 1:
+                hvals[i] = hvals[i][0]
+        return hvals
 
+    def open(self):
+        """Open snapshot file."""
+        for filename in [self.filename, self.filename + '.0']:
+            if os.path.exists(filename):
+                return open(filename, 'rb')
+        raise RuntimeError("Snapshot file %s does not exist." % self.filename)
 
-def _get_gadget_format(filename, header_size):
-    # check and return gadget binary format with file endianness
-    ff = open(filename, 'rb')
-    (rhead,) = struct.unpack('<I', ff.read(4))
-    ff.close()
-    if (rhead == _byte_swap_32(8)):
-        return 2, '>'
-    elif (rhead == 8):
-        return 2, '<'
-    elif (rhead == _byte_swap_32(header_size)):
-        return 1, '>'
-    elif (rhead == header_size):
-        return 1, '<'
-    else:
-        raise RuntimeError("Incorrect Gadget format %s!" % str(rhead))
+    def validate(self):
+        """Validate data integrity."""
+        try:
+            self.open().close()
+            self.gadget_format
+            self.float_type
+        except:
+            return False
+        return True
 
 
 class GadgetBinaryFile(ParticleFile):
     def __init__(self, ds, io, filename, file_id):
-        gformat = _get_gadget_format(filename, ds._header_size)
-        with open(filename, "rb") as f:
-            if gformat[0] == 2:
-                f.seek(f.tell() + SNAP_FORMAT_2_OFFSET)
-            self.header = read_record(f, ds._header_spec, endian=gformat[1])
-            if gformat[0] == 2:
-                f.seek(f.tell() + SNAP_FORMAT_2_OFFSET)
-            self._position_offset = f.tell()
-            f.seek(0, os.SEEK_END)
-            self._file_size = f.tell()
+        header = ds._header
+        self.header = header.value
+        self._position_offset = header.position_offset
+        with header.open() as f:
+            self._file_size = f.seek(0, os.SEEK_END)
 
         super(GadgetBinaryFile, self).__init__(ds, io, filename, file_id)
 
@@ -122,10 +223,9 @@ def __init__(self, filename, dataset_type="gadget_binary",
                  w_a = 0.0):
         if self._instantiated:
             return
-        self._header_spec = self._setup_binary_spec(
-            header_spec, gadget_header_specs)
-        self._header_size = _compute_header_size(self._header_spec)
-        if self._header_size != 256:
+        self._header = GadgetBinaryHeader(filename, header_spec)
+        header_size = self._header.size
+        if header_size != [256]:
             only_on_root(
                 mylog.warn,
                 "Non-standard header size is detected! "
@@ -135,7 +235,7 @@ def __init__(self, filename, dataset_type="gadget_binary",
                 "and you might want to check how the dataset is loaded. "
                 "Futher information about header specification can be found in "
                 "https://yt-project.org/docs/dev/examining/loading_data.html#header-specification.",
-                self._header_size
+                header_size
             )
         self._field_spec = self._setup_binary_spec(
             field_spec, gadget_field_specs)
@@ -180,7 +280,7 @@ def __init__(self, filename, dataset_type="gadget_binary",
 
     @classmethod
     def _setup_binary_spec(cls, spec, spec_dict):
-        if isinstance(spec, str):
+        if isinstance(spec, string_types):
             _hs = ()
             for hs in spec.split("+"):
                 _hs += spec_dict[hs]
@@ -191,17 +291,7 @@ def __repr__(self):
         return os.path.basename(self.parameter_filename).split(".")[0]
 
     def _get_hvals(self):
-        # The entries in this header are capitalized and named to match Table 4
-        # in the GADGET-2 user guide.
-        gformat = _get_gadget_format(self.parameter_filename, self._header_size)
-        f = open(self.parameter_filename, 'rb')
-        if gformat[0] == 2:
-            f.seek(f.tell() + SNAP_FORMAT_2_OFFSET)
-        hvals = read_record(f, self._header_spec, endian=gformat[1])
-        for i in hvals:
-            if len(hvals[i]) == 1:
-                hvals[i] = hvals[i][0]
-        return hvals
+        return self._header.value
 
     def _parse_parameter_file(self):
 
@@ -371,74 +461,14 @@ def _set_code_unit_attributes(self):
         specific_energy_unit = _fix_unit_ordering(specific_energy_unit)
         self.specific_energy_unit = self.quan(*specific_energy_unit)
 
-    @staticmethod
-    def _validate_header(filename, header_size):
-        '''
-        This method automatically detects whether the Gadget file is big/little endian
-        and is not corrupt/invalid using the first 4 bytes in the file.  It returns a
-        tuple of (Valid, endianswap) where Valid is a boolean that is true if the file
-        is a Gadget binary file, and endianswap is the endianness character '>' or '<'.
-        '''
-        try:
-            f = open(filename, 'rb')
-        except IOError:
-            try:
-                f = open(filename + ".0")
-            except IOError:
-                return False, 1
-
-        # First int32 is 256 for a Gadget2 binary file with SnapFormat=1,
-        # 8 for a Gadget2 binary file with SnapFormat=2 file,
-        # or the byte swapped equivalents.
-        # The int32 following the header (first 4+256 bytes) must equal this
-        # number.
-        # Note that 256 is the Gadget2 standard value, but other value could be
-        # set using the header_size argument.
-        try:
-            (rhead,) = struct.unpack('<I', f.read(4))
-        except struct.error:
-            f.close()
-            return False, 1
-        # Use value to check endianness
-        if rhead == header_size:
-            endianswap = '<'
-        elif rhead == _byte_swap_32(header_size):
-            endianswap = '>'
-        elif rhead in (8, _byte_swap_32(8)):
-            # This is only true for snapshot format 2
-            # we do not currently support double precision
-            # snap format 2 data
-            f.close()
-            return True, 'f4'
-        else:
-            f.close()
-            return False, 1
-        # Read in particle number from header
-        np0 = sum(struct.unpack(endianswap + 'IIIIII', f.read(6 * 4)))
-        # Read in size of position block. It should be 4 bytes per float,
-        # with 3 coordinates (x,y,z) per particle. (12 bytes per particle)
-        f.seek(4 + header_size + 4, 0)
-        np1 = struct.unpack(endianswap + 'I', f.read(4))[0] / (4 * 3)
-        f.close()
-        # Compare
-        if np0 == np1:
-            return True, 'f4'
-        elif np1 == 2*np0:
-            return True, 'f8'
-        else:
-            return False, 1
-
     @classmethod
     def _is_valid(cls, *args, **kwargs):
         if 'header_spec' in kwargs:
-            # Compute header size if header is customized
-            header_spec = cls._setup_binary_spec(
-                kwargs['header_spec'], gadget_header_specs)
-            header_size = _compute_header_size(header_spec)
+            header_spec = kwargs['header_spec']
         else:
-            header_size = 256
-        # First 4 bytes used to check load
-        return GadgetDataset._validate_header(args[0], header_size)[0]
+            header_spec = 'default'
+        header = GadgetBinaryHeader(args[0], header_spec)
+        return header.validate()
 
 
 class GadgetHDF5Dataset(GadgetDataset):
diff --git a/yt/frontends/gadget/io.py b/yt/frontends/gadget/io.py
--- a/yt/frontends/gadget/io.py
+++ b/yt/frontends/gadget/io.py
@@ -26,9 +26,6 @@
 from yt.utilities.logger import ytLogger as mylog
 from yt.utilities.on_demand_imports import _h5py as h5py
 
-from .data_structures import \
-    _get_gadget_format
-
 from .definitions import \
     gadget_hdf5_ptypes, \
     SNAP_FORMAT_2_OFFSET
@@ -242,10 +239,10 @@ def __init__(self, ds, *args, **kwargs):
         self._fields = ds._field_spec
         self._ptypes = ds._ptype_spec
         self.data_files = set([])
-        gformat = _get_gadget_format(ds.parameter_filename, ds._header_size)
+        gformat, endianswap = ds._header.gadget_format
         # gadget format 1 original, 2 with block name
-        self._format = gformat[0]
-        self._endian = gformat[1]
+        self._format = gformat
+        self._endian = endianswap
         super(IOHandlerGadgetBinary, self).__init__(ds, *args, **kwargs)
 
     @property
@@ -344,8 +341,7 @@ def _get_morton_from_position(self, data_file, count, offset_count,
     def _initialize_index(self, data_file, regions):
         DLE = data_file.ds.domain_left_edge
         DRE = data_file.ds.domain_right_edge
-        self._float_type = data_file.ds._validate_header(
-            data_file.filename, data_file.ds._header_size)[1]
+        self._float_type = data_file.ds._header.float_type
         if self.index_ptype == "all":
             count = sum(data_file.total_particles.values())
             return self._get_morton_from_position(


Test patch:
diff --git a/yt/frontends/gadget/testing.py b/yt/frontends/gadget/testing.py
--- a/yt/frontends/gadget/testing.py
+++ b/yt/frontends/gadget/testing.py
@@ -2,10 +2,12 @@
 
 from .definitions import (
     gadget_field_specs,
-    gadget_header_specs,
     gadget_ptype_specs,
 )
-from .data_structures import GadgetDataset
+from .data_structures import (
+    GadgetBinaryHeader,
+    GadgetDataset,
+)
 from .io import IOHandlerGadgetBinary
 
 
@@ -53,9 +55,7 @@ def fake_gadget_binary(
         endian='', fmt=2
     ):
     """Generate a fake Gadget binary snapshot."""
-    header_spec = GadgetDataset._setup_binary_spec(
-        header_spec, gadget_header_specs
-    )
+    header = GadgetBinaryHeader(filename, header_spec)
     field_spec = GadgetDataset._setup_binary_spec(
         field_spec, gadget_field_specs
     )
@@ -63,17 +63,19 @@ def fake_gadget_binary(
         ptype_spec, gadget_ptype_specs
     )
     with open(filename, 'wb') as fp:
-        # Generate and write header block
-        header_dtype = np.dtype(
-            [(name, endian + dtype, dim) for name, dim, dtype in header_spec]
-        )
-        header = np.zeros(1, dtype=header_dtype)
-        header['Npart'] = npart
-        header['Nall'] = npart
-        header['NumFiles'] = 1
-        header['BoxSize'] = 1
-        header['HubbleParam'] = 1
-        write_block(fp, header, endian, fmt, 'HEAD')
+        # Generate and write header blocks
+        for i_header, header_spec in enumerate(header.spec):
+            header_dtype = np.dtype(
+                [(name, endian + dtype, dim) for name, dim, dtype in header_spec]
+            )
+            header = np.zeros(1, dtype=header_dtype)
+            if i_header == 0:
+                header['Npart'] = npart
+                header['Nall'] = npart
+                header['NumFiles'] = 1
+                header['BoxSize'] = 1
+                header['HubbleParam'] = 1
+            write_block(fp, header, endian, fmt, 'HEAD')
 
         npart = dict(zip(ptype_spec, npart))
         for fs in field_spec:
diff --git a/yt/frontends/gadget/tests/test_outputs.py b/yt/frontends/gadget/tests/test_outputs.py
--- a/yt/frontends/gadget/tests/test_outputs.py
+++ b/yt/frontends/gadget/tests/test_outputs.py
@@ -50,7 +50,7 @@
 
 
 def test_gadget_binary():
-    header_specs = ['default', 'default+pad32']
+    header_specs = ['default', 'default+pad32', ['default', 'pad32']]
     for header_spec, endian, fmt in product(header_specs, '<>', [1, 2]):
         fake_snap = fake_gadget_binary(
             header_spec=header_spec,