Repository: pydicom/pydicom

Issue:
Generators in encaps don't handle single fragment per frame correctly with no BOT value
#### Description
Generators in `encaps.py` handling of encapsulated pixel data incorrect when the Basic Offset Table has no value and each frame is a single fragment.

#### Steps/Code to Reproduce
```python
from pydicom import dcmread
from pydicom.encaps import generate_pixel_data_frame

fpath = 'pydicom/data/test_files/emri_small_jpeg_2k_lossless.dcm'
ds = dcmread(fpath)
ds.NumberOfFrames  # 10
frame_generator = generate_pixel_data_frame(ds.PixelData)
next(frame_generator)
next(frame_generator) # StopIteration raised
```

#### Expected Results
All 10 frames of the pixel data should be accessible.

#### Actual Results
Only the first frame is accessible.
[MRG] Some pixel handlers will not decode multiple fragments per frame


Added test cases to demonstrate failures for jpeg ls with multiple fragments per frame.  The test files were created with dcmtk 3.6.1 using dcmcjpls +fs 1. One file has an offset table, the other does not.

#### Reference Issue
See #685 


#### What does this implement/fix? Explain your changes.
These test cases show that the pixel decoders (jpeg and jpeg_ls most likely) will not handle multiple fragments per frame.

No fix yet...

Any suggestions?


Discussion:
The problem is that as far as I can tell there's no unambiguous way to determine the difference between one frame with multiple fragments and multiple frames of 1 fragment each when the Basic Offset Table (Part 5, Annex A.4 and Section 8.2) has no value. You can look at the NumberOfFrames value but that won't help if the number of fragments is larger than the number of frames (indicating one or more frames has multiple fragments).

I could add a parser that determines the number of Items in the Pixel Data, and if the BOT has no value then compare that against the number of frames and use that to determine the generator yields. It seems clunky though.

Anyone have a better idea?


As long as you are not speaking about arbitrary access to single frames, it shall be possible to ignore the offset table completely and just decode fragments sequentially, checking for the decoded frame size as you go along, and yield the next frame as soon as it has the correct frame size (or larger in case of padding). 

This is in case I didn't miss something here, and only valid for the generator as opposed to arbitrary frame access.
Ok, I did miss something here - the frame is not decoded by the generator, but only later by the pixel data handler, so we won't know the decoded fragment size at that point. 
We could change the logic respectively (hand over each fragment to the pixel handler for decoding), but that would change the behavior, and also may be slower (or not, not sure).

You could at least check if the number of frames is the same as the number of fragments (which is probably the most common case) and handle that case, and also the case where the number of fragment is a multiple of the number of frames (which is less sure), but for other cases, you really need to decode the data to be on the sure side. 
As I recall when I first did this some time ago,  I had an awful time trying to deal with frames and fragments.  I ended up just focusing on getting it to work with the EMRI images I had for test cases and not really worrying about individual frame boundaries.  I am pretty sure where the pixel handler code says "for frame in CompressedPixelDataSeq", it is actually looping over each fragment...

I am pretty sure the handlers will fail when there is more than one fragment per frame...

> I am pretty sure the handlers will fail when there is more than one fragment per frame..

In this case maybe it makes sense to restructure the code to use the handlers to decode each fragment. Not sure if this would be a compatibility issue for existing code, though. 
The test cases I added for jpeg_ls have a different number of fragments per frame in the sequence.  I have to imagine that if the the offset table is empty for a muiltframe image, then the best you can do is assume one fragment per frame (that is what the code currently does).  Reading the standard, it is unclear to me that having an empty offset table with more than one fragment per frame is valid.  

I would welcome a more expert opinion on this.  


As far as I can see, the standard does not enforce a value for the offset table in any case, so this would be valid. I have no idea if this is used in the real world, but I wouldn't exclude the possibility.
I just added some changes to #688 that change the jpeg_ls handler to more properly use generate_pixel_data_frame rather than decode_data_sequence.  This fixes the test case of multiple fragments per frame with an offset table.  

For the case of no offset table, then maybe we can just go fragment by fragment with:

```
for fragment in data:
    frame.append(fragment)
    try:
        decompress(frame)
        print "yea! this is a valid frame"
    except:
        print "not quite done yet"

```
Looks promising! I'm off to my day job now, may have another look in the evening.
I was thinking of modifying the frame generator so the following logic applies:
```
if no BOT value:
    if NumberOfFrames == (number of pixel data items):
        # Multiple fragments, one frame each
    elif NumberOfFrames == 1:
        # One or more fragments, one frame
    elif (number of pixel data items) == 1:
        # One fragment, one frame
    else:
        # Multiple fragments, but no way to tell what fragments are what frame
        raise exception, tell user they should use the 
            fragment generator and their own judgement?
```
In the last case I think the only way to generate the frame correctly is if the BOT has a value.

Edit - Or to try to decompress? But not all encapsulated data has to be compressed...
Is it guaranteed that decompressing a frame will fail if the frame isn't complete? What's the performance penalty for a failed decompress?
> I was thinking of modifying the frame generator so the following logic applies

I think this sounds good. The last case is the real problem, of course. I have been thinking about decompressing it, too, but I'm not sure how decompressing/decoding separate fragments behaves. 
A working but probably slow solution would be to try to decompress a fragment as proposed, and if that fails, or if the resulting image is too small for a frame, try again with another fragment appended (the decoder itself could do this incrementally, but we don't have control over that). I don't know if there is the possibility to decode fragments independently - that would be easier (e.g. no need to append fragments), but I doubt that.
Anyway, that would basically mean either to transfer a part of the generator logic to the pixel handlers, or make the generator dependend on the pixel handlers.
It would be good if we had some test data with multiple fragments per frame to check this... I may have to think a bit more about this, having no real experience in this field.
For all the JPEG syntaxes (Jpeg200, Jpeg-LS, jpeg), a single frame always ends with a End-of-Image (EOI in JPEG/JPEG-LS, EOC in JPEG2K) that is "FF D9".  If there are N frames, there should always be exactly N of those markers in the fragments.   

So, for JPEG family, we could loop over the fragments:

```

frame = bytearray()
for fragment in data:
    frame.extend(fragment)
    if "FFD9" in frame[-10:]:
        print "Hooray! The frame is done"
        break
    else:
        print "Still more to do"

```

Although the try decompress ... except should work with jpeg syntaxes, there is no requirement that a decoder fail on an incomplete frame (it could just return decompressed data and wait for more input) (the current api we use will fail on incomplete images - but no guarantees).  

For the RLE syntaxes, there is only one fragment per frame.

For uncompressed syntaxes, the size of each frame may be determined by BitsStored x Rows x Cols.


That is much better! As we don't support any other compressed syntaxes apart from JPEG and RLE syntaxes, this should be sufficient to handle these cases. This also makes it possible to get some kind of indexed access for the worst case by building up the missing offset table ourselves if needed.
That logic has to be in the handlers though, or at some intermediate level, as it is the same for all JPEG syntaxes.
Is it possible to hit FF D9 because that's the value at that offset, not because we're at the end of the frame? Or is this only in the JPEG pixel data handlers?

I keep getting confused whether were talking about the pixel data handlers or the generator functions in encaps.
This is only for JPEG handlers.  FF D9 is guaranteed to not be in the encoded data.  It is a reserved JPEG marker.
> I keep getting confused whether were talking about the pixel data handlers or the generator functions in encaps.

This is currently a bit mixed up. The current implementation of the generator functions does not know about specific transfer syntaxes - with the proposed PR this would change. Maybe the generator could ask the pixel handler about a fragment being the last in a frame in case it doesn't know that, and the JPEG pixel handlers implement the logic (which should be in some common place for all JPEG handlers)?

@rhaxton @scaramallion - this would be a good candidate for 1.2, in my opinion. Is this doable? Do you need help for the conflict resolving?
@rhaxton, @scaramallion, pinging on this issue again ... was this ever resolved completely? I'll assign to v1.4 milestone for now, perhaps we can try to look at this in the coming months?
Hello @rhaxton! Thanks for updating the PR.











Cheers ! There are no PEP8 issues in this Pull Request. :beers: 

##### Comment last updated on August 06, 2018 at 18:28 Hours UTC
@mrbean-bremen If you could take a quick look at this particularly why there are issues with python 3, that would be a great help...

This will still fail for uncompressed frames with no BoT value.

Also the docstrings need updating.
I've been thinking it might be better to change `get_frame_offsets` to return an empty list if the BoT has no value rather than trying to be clever about it since the two cases (BoT has no value and BoT has single 0 offset) may represent different things. Then the functions that rely on it should be updated accordingly.
Can someone come up with a test case for an encapsulated uncompressed PixelData image?  I can't figure out how to create one with dcmtk...
Also, the standard explicitly calls out RLE syntax as having one fragment per frame, and calls out JPEG as possibly having more than one fragment per frame, but I couldn't find any statement about the fragments per frame for uncompressed syntax.  Any ideas on that one?
I have some code I'm working on that should at least return mixed fragments/frame native data in a single blob while still following your approach for JPEG data.

```python
# N fragments, M frames; without BOT values there's no generic
#   way to determine where a frame ends so we try our best
frame = []
frame_number = 0
for fragment in generate_pixel_data_fragment(fp):
    # For JPEG transfer syntaxes try to locate the EOC marker
    if b'\xFF\xD9' in fragment[-10:]:
        yield tuple(frame)
        frame_number += 1
        frame = []
    else:
        frame.append(fragment)

# If we failed to locate the EOC marker then either nothing will
#   have been yielded and/or one or more frames will have been
#   skipped. This will be the case with native transfer syntaxes
if frame_number != no_frames:
    yield tuple(frame)
```

[Here](https://github.com/pydicom/pydicom/files/2252088/encap_jpeg.zip) is some JPEG 2x2 11 frame encapsulated data with and without BOT (1 fragment per frame and 2 fragments per frame).

[Here](https://github.com/pydicom/pydicom/files/2252147/encap_native.zip) is some native 2x2 11 frame encapsulated data with and without BOT (1 fragment per frame and 2 fragments per frame).

Image pattern is black upper left, white upper right, white lower left, black lower right and the black pixels shade to white over the frames. Maybe just double check the encapsulation has been performed correctly. 

With the code above the no BOT/native/2 fragments per frame returns a blob of the entire pixel data, but I'd expect that in that case the numpy pixel handler should be able to rearrange the data into frames correctly, provided the blob is the right length. The no BOT/jpeg/2 fragments per frame returns the frames correctly.
I'm a bit confused by this.  When I read [this section](http://dicom.nema.org/dicom/2013/output/chtml/part05/sect_8.2.html) of DICOM,  it seems to imply that Uncompressed and Encapsulated are mutually exclusive?  I don't know if I have ever seen a DICOM uncompressed image with encapsulated pixel data.  Has anyone else?

Whoops, obviously I've gotten confused about this somewhere. Nevermind me then...

[This is what I've done.](https://github.com/scaramallion/pydicom/blob/dev-encaps/pydicom/encaps.py#L247)
@rhaxton , @scaramallion - what is the state of this PR (apart from having conflicts) - can it make it into the 1.4 release?

Patch:
diff --git a/pydicom/encaps.py b/pydicom/encaps.py
--- a/pydicom/encaps.py
+++ b/pydicom/encaps.py
@@ -2,6 +2,7 @@
 """Functions for working with encapsulated (compressed) pixel data."""
 
 from struct import pack
+import warnings
 
 import pydicom.config
 from pydicom.filebase import DicomBytesIO
@@ -50,9 +51,10 @@ def get_frame_offsets(fp):
 
     Returns
     -------
-    list of int
-        The byte offsets to the first fragment of each frame, as measured from
-        the start of the first item following the Basic Offset Table item.
+    bool, list of int
+        Whether or not the BOT is empty, and a list of the byte offsets
+        to the first fragment of each frame, as measured from the start of the
+        first item following the Basic Offset Table item.
 
     Raises
     ------
@@ -86,7 +88,41 @@ def get_frame_offsets(fp):
     for ii in range(length // 4):
         offsets.append(fp.read_UL())
 
-    return offsets
+    return bool(length), offsets
+
+
+def get_nr_fragments(fp):
+    """Return the number of fragments in `fp`."""
+    if not fp.is_little_endian:
+        raise ValueError("'fp.is_little_endian' must be True")
+
+    nr_fragments = 0
+    start = fp.tell()
+    while True:
+        try:
+            tag = Tag(fp.read_tag())
+        except EOFError:
+            break
+
+        if tag == 0xFFFEE000:
+            # Item
+            length = fp.read_UL()
+            if length == 0xFFFFFFFF:
+                raise ValueError("Undefined item length at offset {} when "
+                                 "parsing the encapsulated pixel data "
+                                 "fragments.".format(fp.tell() - 4))
+            fp.seek(length, 1)
+            nr_fragments += 1
+        elif tag == 0xFFFEE0DD:
+            # Sequence Delimiter
+            break
+        else:
+            raise ValueError("Unexpected tag '{}' at offset {} when parsing "
+                             "the encapsulated pixel data fragment items."
+                             .format(tag, fp.tell() - 4))
+
+    fp.seek(start)
+    return nr_fragments
 
 
 def generate_pixel_data_fragment(fp):
@@ -175,7 +211,7 @@ def generate_pixel_data_fragment(fp):
                              .format(tag, fp.tell() - 4))
 
 
-def generate_pixel_data_frame(bytestream):
+def generate_pixel_data_frame(bytestream, nr_frames=None):
     """Yield an encapsulated pixel data frame.
 
     Parameters
@@ -184,6 +220,10 @@ def generate_pixel_data_frame(bytestream):
         The value of the (7fe0, 0010) *Pixel Data* element from an encapsulated
         dataset. The Basic Offset Table item should be present and the
         Sequence Delimiter item may or may not be present.
+    nr_frames : int, optional
+        Required for multi-frame data when the Basic Offset Table is empty
+        and there are multiple frames. This should be the value of (0028,0008)
+        *Number of Frames*.
 
     Yields
     ------
@@ -194,11 +234,11 @@ def generate_pixel_data_frame(bytestream):
     ----------
     DICOM Standard Part 5, :dcm:`Annex A <part05/chapter_A.html>`
     """
-    for fragmented_frame in generate_pixel_data(bytestream):
+    for fragmented_frame in generate_pixel_data(bytestream, nr_frames):
         yield b''.join(fragmented_frame)
 
 
-def generate_pixel_data(bytestream):
+def generate_pixel_data(bytestream, nr_frames=None):
     """Yield an encapsulated pixel data frame.
 
     For the following transfer syntaxes, a fragment may not contain encoded
@@ -230,6 +270,10 @@ def generate_pixel_data(bytestream):
         The value of the (7fe0, 0010) *Pixel Data* element from an encapsulated
         dataset. The Basic Offset Table item should be present and the
         Sequence Delimiter item may or may not be present.
+    nr_frames : int, optional
+        Required for multi-frame data when the Basic Offset Table is empty
+        and there are multiple frames. This should be the value of (0028,0008)
+        *Number of Frames*.
 
     Yields
     -------
@@ -237,6 +281,14 @@ def generate_pixel_data(bytestream):
         An encapsulated pixel data frame, with the contents of the
         :class:`tuple` the frame's fragmented data.
 
+    Notes
+    -----
+    If the Basic Offset Table is empty and there are multiple fragments per
+    frame then an attempt will be made to locate the frame boundaries by
+    searching for the JPEG/JPEG-LS/JPEG2000 EOI/EOC marker (``0xFFD9``). If the
+    marker is not present or the pixel data hasn't been compressed using one of
+    the JPEG standards then the generated pixel data may be incorrect.
+
     References
     ----------
     DICOM Standard Part 5, :dcm:`Annex A <part05/chapter_A.html>`
@@ -245,28 +297,90 @@ def generate_pixel_data(bytestream):
     fp.is_little_endian = True
 
     # `offsets` is a list of the offsets to the first fragment in each frame
-    offsets = get_frame_offsets(fp)
+    has_bot, offsets = get_frame_offsets(fp)
     # Doesn't actually matter what the last offset value is, as long as its
     # greater than the total number of bytes in the fragments
     offsets.append(len(bytestream))
 
-    frame = []
-    frame_length = 0
-    frame_number = 0
-    for fragment in generate_pixel_data_fragment(fp):
-        if frame_length < offsets[frame_number + 1]:
-            frame.append(fragment)
+    if has_bot:
+        # Use the BOT to determine the frame boundaries
+        frame = []
+        frame_length = 0
+        frame_number = 0
+        for fragment in generate_pixel_data_fragment(fp):
+            if frame_length < offsets[frame_number + 1]:
+                frame.append(fragment)
+            else:
+                yield tuple(frame)
+                frame = [fragment]
+                frame_number += 1
+
+            frame_length += len(fragment) + 8
+
+        # Yield the final frame - required here because the frame_length will
+        # never be greater than offsets[-1] and thus never trigger the final
+        # yield within the for block
+        yield tuple(frame)
+    else:
+        nr_fragments = get_nr_fragments(fp)
+        if nr_fragments == 1:
+            # Single fragment: 1 frame
+            for fragment in generate_pixel_data_fragment(fp):
+                yield tuple([fragment])
+        elif nr_frames:
+            # Multiple fragments: 1 or more frames
+            if nr_fragments == nr_frames:
+                # 1 fragment per frame
+                # Covers RLE and others if 1:1 ratio
+                for fragment in generate_pixel_data_fragment(fp):
+                    yield tuple([fragment])
+            elif nr_frames == 1:
+                # Multiple fragments: 1 frame
+                frame = []
+                for fragment in generate_pixel_data_fragment(fp):
+                    frame.append(fragment)
+                yield tuple(frame)
+            elif nr_fragments > nr_frames:
+                # More fragments then frames
+                # Search for JPEG/JPEG-LS/JPEG2K EOI/EOC marker
+                # Should be the last two bytes of a frame
+                # May fail if no EOI/EOC marker or not JPEG
+                eoi_marker = b'\xff\xd9'
+                frame = []
+                frame_nr = 0
+                for fragment in generate_pixel_data_fragment(fp):
+                    frame.append(fragment)
+                    if eoi_marker in fragment[-10:]:
+                        yield tuple(frame)
+                        frame_nr += 1
+                        frame = []
+
+                if frame or frame_nr != nr_frames:
+                    # If data in `frame` or fewer frames yielded then we
+                    #   must've missed a frame boundary
+                    warnings.warn(
+                        "The end of the encapsulated pixel data has been "
+                        "reached but one or more frame boundaries may have "
+                        "been missed; please confirm that the generated frame "
+                        "data is correct"
+                    )
+                    if frame:
+                        yield tuple(frame)
+
+            else:
+                # Fewer fragments than frames
+                raise ValueError(
+                    "Unable to parse encapsulated pixel data as the Basic "
+                    "Offset Table is empty and there are fewer fragments then "
+                    "frames; the dataset may be corrupt"
+                )
         else:
-            yield tuple(frame)
-            frame = [fragment]
-            frame_number += 1
-
-        frame_length += len(fragment) + 8
-
-    # Yield the final frame - required here because the frame_length will
-    # never be greater than offsets[-1] and thus never trigger the final yield
-    # within the for block
-    yield tuple(frame)
+            # Multiple fragments but unknown number of frames
+            raise ValueError(
+                "Unable to determine the frame boundaries for the "
+                "encapsulated pixel data as the Basic Offset Table is empty "
+                "and `nr_frames` parameter is None"
+            )
 
 
 def decode_data_sequence(data):


Test patch:
diff --git a/pydicom/tests/test_encaps.py b/pydicom/tests/test_encaps.py
--- a/pydicom/tests/test_encaps.py
+++ b/pydicom/tests/test_encaps.py
@@ -4,10 +4,11 @@
 import pytest
 
 from pydicom import dcmread
-from pydicom.data import get_testdata_files
+from pydicom.data import get_testdata_file
 from pydicom.encaps import (
     generate_pixel_data_fragment,
     get_frame_offsets,
+    get_nr_fragments,
     generate_pixel_data_frame,
     generate_pixel_data,
     decode_data_sequence,
@@ -20,7 +21,7 @@
 from pydicom.filebase import DicomBytesIO
 
 
-JP2K_10FRAME_NOBOT = get_testdata_files('emri_small_jpeg_2k_lossless.dcm')[0]
+JP2K_10FRAME_NOBOT = get_testdata_file('emri_small_jpeg_2k_lossless.dcm')
 
 
 class TestGetFrameOffsets(object):
@@ -57,7 +58,7 @@ def test_zero_length(self):
                      b'\x00\x00\x00\x00'
         fp = DicomBytesIO(bytestream)
         fp.is_little_endian = True
-        assert [0] == get_frame_offsets(fp)
+        assert (False, [0]) == get_frame_offsets(fp)
 
     def test_multi_frame(self):
         """Test reading multi-frame BOT item"""
@@ -69,7 +70,7 @@ def test_multi_frame(self):
                      b'\xFE\x37\x00\x00'
         fp = DicomBytesIO(bytestream)
         fp.is_little_endian = True
-        assert [0, 4966, 9716, 14334] == get_frame_offsets(fp)
+        assert (True, [0, 4966, 9716, 14334]) == get_frame_offsets(fp)
 
     def test_single_frame(self):
         """Test reading single-frame BOT item"""
@@ -78,7 +79,7 @@ def test_single_frame(self):
                      b'\x00\x00\x00\x00'
         fp = DicomBytesIO(bytestream)
         fp.is_little_endian = True
-        assert [0] == get_frame_offsets(fp)
+        assert (True, [0]) == get_frame_offsets(fp)
 
     def test_not_little_endian(self):
         """Test reading big endian raises exception"""
@@ -91,6 +92,113 @@ def test_not_little_endian(self):
             get_frame_offsets(fp)
 
 
+class TestGetNrFragments(object):
+    """Test encaps.get_nr_fragments"""
+    def test_item_undefined_length(self):
+        """Test exception raised if item length undefined."""
+        bytestream = (
+            b'\xFE\xFF\x00\xE0'
+            b'\xFF\xFF\xFF\xFF'
+            b'\x00\x00\x00\x01'
+        )
+        fp = DicomBytesIO(bytestream)
+        fp.is_little_endian = True
+        with pytest.raises(ValueError):
+            get_nr_fragments(fp)
+
+    def test_item_sequence_delimiter(self):
+        """Test that the fragments are returned if seq delimiter hit."""
+        bytestream = (
+            b'\xFE\xFF\x00\xE0'
+            b'\x04\x00\x00\x00'
+            b'\x01\x00\x00\x00'
+            b'\xFE\xFF\xDD\xE0'
+            b'\x00\x00\x00\x00'
+            b'\xFE\xFF\x00\xE0'
+            b'\x04\x00\x00\x00'
+            b'\x02\x00\x00\x00'
+        )
+        fp = DicomBytesIO(bytestream)
+        fp.is_little_endian = True
+        assert 1 == get_nr_fragments(fp)
+
+    def test_item_bad_tag(self):
+        """Test exception raised if item has unexpected tag"""
+        bytestream = (
+            b'\xFE\xFF\x00\xE0'
+            b'\x04\x00\x00\x00'
+            b'\x01\x00\x00\x00'
+            b'\x10\x00\x10\x00'
+            b'\x00\x00\x00\x00'
+            b'\xFE\xFF\x00\xE0'
+            b'\x04\x00\x00\x00'
+            b'\x02\x00\x00\x00'
+        )
+        fp = DicomBytesIO(bytestream)
+        fp.is_little_endian = True
+        msg = (
+            r"Unexpected tag '\(0010, 0010\)' at offset 12 when parsing the "
+            r"encapsulated pixel data fragment items."
+        )
+        with pytest.raises(ValueError, match=msg):
+            get_nr_fragments(fp)
+
+    def test_single_fragment_no_delimiter(self):
+        """Test single fragment is returned OK"""
+        bytestream = b'\xFE\xFF\x00\xE0' \
+                     b'\x04\x00\x00\x00' \
+                     b'\x01\x00\x00\x00'
+        fp = DicomBytesIO(bytestream)
+        fp.is_little_endian = True
+        assert 1 == get_nr_fragments(fp)
+
+    def test_multi_fragments_no_delimiter(self):
+        """Test multi fragments are returned OK"""
+        bytestream = b'\xFE\xFF\x00\xE0' \
+                     b'\x04\x00\x00\x00' \
+                     b'\x01\x00\x00\x00' \
+                     b'\xFE\xFF\x00\xE0' \
+                     b'\x06\x00\x00\x00' \
+                     b'\x01\x02\x03\x04\x05\x06'
+        fp = DicomBytesIO(bytestream)
+        fp.is_little_endian = True
+        assert 2 == get_nr_fragments(fp)
+
+    def test_single_fragment_delimiter(self):
+        """Test single fragment is returned OK with sequence delimiter item"""
+        bytestream = b'\xFE\xFF\x00\xE0' \
+                     b'\x04\x00\x00\x00' \
+                     b'\x01\x00\x00\x00' \
+                     b'\xFE\xFF\xDD\xE0'
+        fp = DicomBytesIO(bytestream)
+        fp.is_little_endian = True
+        assert 1 == get_nr_fragments(fp)
+
+    def test_multi_fragments_delimiter(self):
+        """Test multi fragments are returned OK with sequence delimiter item"""
+        bytestream = b'\xFE\xFF\x00\xE0' \
+                     b'\x04\x00\x00\x00' \
+                     b'\x01\x00\x00\x00' \
+                     b'\xFE\xFF\x00\xE0' \
+                     b'\x06\x00\x00\x00' \
+                     b'\x01\x02\x03\x04\x05\x06' \
+                     b'\xFE\xFF\xDD\xE0'
+        fp = DicomBytesIO(bytestream)
+        fp.is_little_endian = True
+        assert 2 == get_nr_fragments(fp)
+
+    def test_not_little_endian(self):
+        """Test reading big endian raises exception"""
+        bytestream = b'\xFE\xFF\x00\xE0' \
+                     b'\x04\x00\x00\x00' \
+                     b'\x01\x00\x00\x00'
+        fp = DicomBytesIO(bytestream)
+        fp.is_little_endian = False
+        with pytest.raises(ValueError,
+                           match="'fp.is_little_endian' must be True"):
+            get_nr_fragments(fp)
+
+
 class TestGeneratePixelDataFragment(object):
     """Test encaps.generate_pixel_data_fragment"""
     def test_item_undefined_length(self):
@@ -242,7 +350,7 @@ def test_empty_bot_triple_fragment_single_frame(self):
                      b'\xFE\xFF\x00\xE0' \
                      b'\x04\x00\x00\x00' \
                      b'\x03\x00\x00\x00'
-        frames = generate_pixel_data_frame(bytestream)
+        frames = generate_pixel_data_frame(bytestream, 1)
         assert next(frames) == (
             b'\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00'
         )
@@ -362,6 +470,18 @@ def test_multi_frame_varied_ratio(self):
         assert next(frames) == b'\x03\x00\x00\x00\x02\x04'
         pytest.raises(StopIteration, next, frames)
 
+    def test_empty_bot_multi_fragments_per_frame(self):
+        """Test multi-frame where multiple frags per frame and no BOT."""
+        # Regression test for #685
+        ds = dcmread(JP2K_10FRAME_NOBOT)
+        assert 10 == ds.NumberOfFrames
+        frame_gen = generate_pixel_data_frame(ds.PixelData, ds.NumberOfFrames)
+        for ii in range(10):
+            next(frame_gen)
+
+        with pytest.raises(StopIteration):
+            next(frame_gen)
+
 
 class TestGeneratePixelData(object):
     """Test encaps.generate_pixel_data"""
@@ -391,12 +511,122 @@ def test_empty_bot_triple_fragment_single_frame(self):
                      b'\xFE\xFF\x00\xE0' \
                      b'\x04\x00\x00\x00' \
                      b'\x03\x00\x00\x00'
-        frames = generate_pixel_data(bytestream)
+        frames = generate_pixel_data(bytestream, 1)
         assert next(frames) == (b'\x01\x00\x00\x00',
                                 b'\x02\x00\x00\x00',
                                 b'\x03\x00\x00\x00')
         pytest.raises(StopIteration, next, frames)
 
+    def test_empty_bot_no_nr_frames_raises(self):
+        """Test parsing raises if not BOT and no nr_frames."""
+        # 1 frame, 3 fragments long
+        bytestream = b'\xFE\xFF\x00\xE0' \
+                     b'\x00\x00\x00\x00' \
+                     b'\xFE\xFF\x00\xE0' \
+                     b'\x04\x00\x00\x00' \
+                     b'\x01\x00\x00\x00' \
+                     b'\xFE\xFF\x00\xE0' \
+                     b'\x04\x00\x00\x00' \
+                     b'\x02\x00\x00\x00' \
+                     b'\xFE\xFF\x00\xE0' \
+                     b'\x04\x00\x00\x00' \
+                     b'\x03\x00\x00\x00'
+        msg = (
+            r"Unable to determine the frame boundaries for the "
+            r"encapsulated pixel data as the Basic Offset Table is empty "
+            r"and `nr_frames` parameter is None"
+        )
+        with pytest.raises(ValueError, match=msg):
+            next(generate_pixel_data(bytestream))
+
+    def test_empty_bot_too_few_fragments(self):
+        """Test parsing with too few fragments."""
+        ds = dcmread(JP2K_10FRAME_NOBOT)
+        assert 10 == ds.NumberOfFrames
+
+        msg = (
+            r"Unable to parse encapsulated pixel data as the Basic "
+            r"Offset Table is empty and there are fewer fragments then "
+            r"frames; the dataset may be corrupt"
+        )
+        with pytest.raises(ValueError, match=msg):
+            next(generate_pixel_data_frame(ds.PixelData, 20))
+
+    def test_empty_bot_multi_fragments_per_frame(self):
+        """Test parsing with multiple fragments per frame."""
+        # 4 frames in 6 fragments with JPEG EOI marker
+        bytestream = (
+            b'\xFE\xFF\x00\xE0\x00\x00\x00\x00'
+            b'\xFE\xFF\x00\xE0\x04\x00\x00\x00\x01\x00\x00\x00'
+            b'\xFE\xFF\x00\xE0\x04\x00\x00\x00\x01\xFF\xD9\x00'
+            b'\xFE\xFF\x00\xE0\x04\x00\x00\x00\x01\x00\xFF\xD9'
+            b'\xFE\xFF\x00\xE0\x04\x00\x00\x00\x01\xFF\xD9\x00'
+            b'\xFE\xFF\x00\xE0\x04\x00\x00\x00\x01\x00\x00\x00'
+            b'\xFE\xFF\x00\xE0\x04\x00\x00\x00\x01\xFF\xD9\x00'
+        )
+
+        frames = generate_pixel_data(bytestream, 4)
+        for ii in range(4):
+            next(frames)
+
+        with pytest.raises(StopIteration):
+            next(frames)
+
+    def test_empty_bot_no_marker(self):
+        """Test parsing not BOT and no final marker with multi fragments."""
+        # 4 frames in 6 fragments with JPEG EOI marker (1 missing EOI)
+        bytestream = (
+            b'\xFE\xFF\x00\xE0\x00\x00\x00\x00'
+            b'\xFE\xFF\x00\xE0\x04\x00\x00\x00\x01\x00\x00\x00'
+            b'\xFE\xFF\x00\xE0\x04\x00\x00\x00\x01\xFF\xD9\x00'
+            b'\xFE\xFF\x00\xE0\x04\x00\x00\x00\x01\x00\x00\x00'
+            b'\xFE\xFF\x00\xE0\x04\x00\x00\x00\x01\xFF\xD9\x00'
+            b'\xFE\xFF\x00\xE0\x04\x00\x00\x00\x01\xFF\xFF\xD9'
+            b'\xFE\xFF\x00\xE0\x04\x00\x00\x00\x01\xFF\x00\x00'
+        )
+
+        frames = generate_pixel_data(bytestream, 4)
+        for ii in range(3):
+            next(frames)
+
+        msg = (
+            r"The end of the encapsulated pixel data has been "
+            r"reached but one or more frame boundaries may have "
+            r"been missed; please confirm that the generated frame "
+            r"data is correct"
+        )
+        with pytest.warns(UserWarning, match=msg):
+            next(frames)
+
+        with pytest.raises(StopIteration):
+            next(frames)
+
+    def test_empty_bot_missing_marker(self):
+        """Test parsing not BOT and missing marker with multi fragments."""
+        # 4 frames in 6 fragments with JPEG EOI marker (1 missing EOI)
+        bytestream = (
+            b'\xFE\xFF\x00\xE0\x00\x00\x00\x00'
+            b'\xFE\xFF\x00\xE0\x04\x00\x00\x00\x01\x00\x00\x00'
+            b'\xFE\xFF\x00\xE0\x04\x00\x00\x00\x01\xFF\xD9\x00'
+            b'\xFE\xFF\x00\xE0\x04\x00\x00\x00\x01\x00\x00\x00'
+            b'\xFE\xFF\x00\xE0\x04\x00\x00\x00\x01\xFF\x00\x00'
+            b'\xFE\xFF\x00\xE0\x04\x00\x00\x00\x01\xFF\xFF\xD9'
+            b'\xFE\xFF\x00\xE0\x04\x00\x00\x00\x01\xFF\xD9\x00'
+        )
+
+        msg = (
+            r"The end of the encapsulated pixel data has been "
+            r"reached but one or more frame boundaries may have "
+            r"been missed; please confirm that the generated frame "
+            r"data is correct"
+        )
+        with pytest.warns(UserWarning, match=msg):
+            ii = 0
+            for frames in generate_pixel_data(bytestream, 4):
+                ii += 1
+
+        assert 3 == ii
+
     def test_bot_single_fragment(self):
         """Test a single-frame image where the frame is one fragment"""
         # 1 frame, 1 fragment long
@@ -932,7 +1162,7 @@ def test_encapsulate_single_fragment_per_frame_bot(self):
 
         fp = DicomBytesIO(data)
         fp.is_little_endian = True
-        offsets = get_frame_offsets(fp)
+        length, offsets = get_frame_offsets(fp)
         assert offsets == [
             0x0000,  # 0
             0x0eee,  # 3822