Repository: amzn/ion-java
Pull request #408 (2022-01-27 20:31:24)

Enables the incremental reader to iterate annotations lazily.
*Description of changes:*
This PR includes two optimizations:
1. (Minor) Uses `IntList` introduced in #404 for storing annotation SIDs without upgrading to `Integer`.
2. Lazily parsing and resolving annotation SIDs. Previously, annotation SIDs were eagerly parsed (in `IonReaderLookaheadBuffer`) at the top-level, and parsed on-demand but in their entirety below the top-level (in `IonReaderBinaryIncremental.getAnnotationSids`). Now, as long as annotation iterator reuse is enabled, annotations at any level are both parsed and resolved lazily (when accessed via the iterator provided by `IonReader.iterateTypeAnnotations`). This enables performance improvements in cases where the user does not access some or all annotations on a value, as is the case when skip-scanning.

*Performance:*

I used ion-java-benchmark-cli to verify that there is no observed impact to performance for streams that don't make heavy use of annotations.

I also used ion-java-benchmark-cli to do a comparison targeted to this change. The test data is a stream of top-level floats (chosen to minimize the value parsing time), each with a variable number of annotations.

The first trial involved iterating all annotations on every value.

Before:

```
Benchmark                                               (input)                                              (options)  Mode  Cnt          Score       Error   Units
Bench.run                                   annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4        632.625 ±     6.560   ms/op
Bench.run:Heap usage                        annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4         28.486 ±     0.305      MB
Bench.run:Serialized size                   annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4        100.350                  MB
Bench.run:·gc.alloc.rate                    annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4        188.526 ±     1.849  MB/sec
Bench.run:·gc.alloc.rate.norm               annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4  131332260.000 ±     7.462    B/op
Bench.run:·gc.churn.PS_Eden_Space           annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4        186.506 ±     1.829  MB/sec
Bench.run:·gc.churn.PS_Eden_Space.norm      annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4  129925120.000 ±     0.001    B/op
Bench.run:·gc.churn.PS_Survivor_Space       annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4          0.314 ±     0.066  MB/sec
Bench.run:·gc.churn.PS_Survivor_Space.norm  annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4     218647.250 ± 45039.772    B/op
Bench.run:·gc.count                         annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4        260.000              counts
Bench.run:·gc.time                          annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4         96.000                  ms
```

After:

```
Benchmark                                  (input)                                              (options)  Mode  Cnt      Score    Error   Units
Bench.run                      annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4    564.878 ±  4.420   ms/op
Bench.run:Heap usage           annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4      5.666 ±  4.339      MB
Bench.run:Serialized size      annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4    100.350               MB
Bench.run:·gc.alloc.rate       annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4      0.135 ±  0.001  MB/sec
Bench.run:·gc.alloc.rate.norm  annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4  84128.889 ± 11.488    B/op
Bench.run:·gc.count            annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4        ≈ 0           counts
```

The second trial involved retrieving only the first annotation on each value.

Before:

```
Benchmark                                               (input)                                              (options)  Mode  Cnt          Score         Error   Units
Bench.run                                   annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4        595.577 ±      24.995   ms/op
Bench.run:Heap usage                        annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4         19.235 ±      94.244      MB
Bench.run:Serialized size                   annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4        100.350                    MB
Bench.run:·gc.alloc.rate                    annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4        200.281 ±       8.033  MB/sec
Bench.run:·gc.alloc.rate.norm               annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4  131332258.353 ±       7.023    B/op
Bench.run:·gc.churn.PS_Eden_Space           annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4        199.079 ±       3.125  MB/sec
Bench.run:·gc.churn.PS_Eden_Space.norm      annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4  130547711.882 ± 5544439.327    B/op
Bench.run:·gc.churn.PS_Survivor_Space       annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4          0.303 ±       0.077  MB/sec
Bench.run:·gc.churn.PS_Survivor_Space.norm  annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4     199042.118 ±   56985.986    B/op
Bench.run:·gc.count                         annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4        258.000                counts
Bench.run:·gc.time                          annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4        118.000                    ms
```

After:

```
Benchmark                                  (input)                                              (options)  Mode  Cnt      Score   Error   Units
Bench.run                      annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4    497.792 ± 8.578   ms/op
Bench.run:Heap usage           annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4      5.330 ± 0.028      MB
Bench.run:Serialized size      annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4    100.350              MB
Bench.run:·gc.alloc.rate       annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4      0.154 ± 0.003  MB/sec
Bench.run:·gc.alloc.rate.norm  annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4  84125.333 ± 5.685    B/op
Bench.run:·gc.count            annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:INCREMENTAL}  avgt    4        ≈ 0          counts
```

For reference, here are the results for the incremental reader (which had no performance difference when reading a single annotation vs all annotations because annotations are parsed and resolved eagerly):

```
Benchmark                                               (input)                                                  (options)  Mode  Cnt          Score         Error   Units
Bench.run                                   annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:NON_INCREMENTAL}  avgt    4        832.241 ±      19.998   ms/op
Bench.run:Heap usage                        annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:NON_INCREMENTAL}  avgt    4         27.072 ±      44.461      MB
Bench.run:Serialized size                   annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:NON_INCREMENTAL}  avgt    4        100.350                    MB
Bench.run:·gc.alloc.rate                    annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:NON_INCREMENTAL}  avgt    4        199.936 ±       7.366  MB/sec
Bench.run:·gc.alloc.rate.norm               annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:NON_INCREMENTAL}  avgt    4  182990436.615 ±      26.216    B/op
Bench.run:·gc.churn.PS_Eden_Space           annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:NON_INCREMENTAL}  avgt    4        198.118 ±       8.860  MB/sec
Bench.run:·gc.churn.PS_Eden_Space.norm      annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:NON_INCREMENTAL}  avgt    4  181326347.987 ± 3377440.465    B/op
Bench.run:·gc.churn.PS_Survivor_Space       annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:NON_INCREMENTAL}  avgt    4          0.232 ±       0.043  MB/sec
Bench.run:·gc.churn.PS_Survivor_Space.norm  annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:NON_INCREMENTAL}  avgt    4     212396.051 ±   39487.424    B/op
Bench.run:·gc.count                         annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:NON_INCREMENTAL}  avgt    4        282.000                counts
Bench.run:·gc.time                          annotatedFloats.10n  read::{f:ION_BINARY,t:FILE,a:STREAMING,R:NON_INCREMENTAL}  avgt    4        113.000                    ms
```

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.


codecov[bot]: # [Codecov](https://codecov.io/gh/amzn/ion-java/pull/408?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) Report
> Merging [#408](https://codecov.io/gh/amzn/ion-java/pull/408?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) (178cdac) into [master](https://codecov.io/gh/amzn/ion-java/commit/1baad70fa6d6ce6b0f20b3f2f92297465b00adf6?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) (1baad70) will **increase** coverage by `0.06%`.
> The diff coverage is `94.94%`.

[![Impacted file tree graph](https://codecov.io/gh/amzn/ion-java/pull/408/graphs/tree.svg?width=650&height=150&src=pr&token=M93Y3e3kKn&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None)](https://codecov.io/gh/amzn/ion-java/pull/408?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None)

```diff
@@             Coverage Diff              @@
##             master     #408      +/-   ##
============================================
+ Coverage     66.39%   66.45%   +0.06%     
- Complexity     5387     5398      +11     
============================================
  Files           156      156              
  Lines         22686    22718      +32     
  Branches       4091     4094       +3     
============================================
+ Hits          15062    15097      +35     
+ Misses         6264     6262       -2     
+ Partials       1360     1359       -1     
```


| [Impacted Files](https://codecov.io/gh/amzn/ion-java/pull/408?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | Coverage Δ | |
|---|---|---|
| [...om/amazon/ion/impl/IonReaderBinaryIncremental.java](https://codecov.io/gh/amzn/ion-java/pull/408/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c3JjL2NvbS9hbWF6b24vaW9uL2ltcGwvSW9uUmVhZGVyQmluYXJ5SW5jcmVtZW50YWwuamF2YQ==) | `96.08% <94.33%> (-0.19%)` | :arrow_down: |
| [.../com/amazon/ion/impl/IonReaderLookaheadBuffer.java](https://codecov.io/gh/amzn/ion-java/pull/408/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c3JjL2NvbS9hbWF6b24vaW9uL2ltcGwvSW9uUmVhZGVyTG9va2FoZWFkQnVmZmVyLmphdmE=) | `97.97% <95.12%> (-0.63%)` | :arrow_down: |
| [src/com/amazon/ion/impl/bin/IntList.java](https://codecov.io/gh/amzn/ion-java/pull/408/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c3JjL2NvbS9hbWF6b24vaW9uL2ltcGwvYmluL0ludExpc3QuamF2YQ==) | `73.52% <100.00%> (+4.56%)` | :arrow_up: |
| [src/com/amazon/ion/impl/BlockedBuffer.java](https://codecov.io/gh/amzn/ion-java/pull/408/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c3JjL2NvbS9hbWF6b24vaW9uL2ltcGwvQmxvY2tlZEJ1ZmZlci5qYXZh) | `51.70% <0.00%> (+0.85%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/amzn/ion-java/pull/408?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/amzn/ion-java/pull/408?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None). Last update [1baad70...178cdac](https://codecov.io/gh/amzn/ion-java/pull/408?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None).


Patch:
diff --git a/src/com/amazon/ion/impl/IonReaderBinaryIncremental.java b/src/com/amazon/ion/impl/IonReaderBinaryIncremental.java
index 7ff31070..22d77417 100644
--- a/src/com/amazon/ion/impl/IonReaderBinaryIncremental.java
+++ b/src/com/amazon/ion/impl/IonReaderBinaryIncremental.java
@@ -15,6 +15,7 @@ import com.amazon.ion.SymbolToken;
 import com.amazon.ion.Timestamp;
 import com.amazon.ion.UnknownSymbolException;
 import com.amazon.ion.ValueFactory;
+import com.amazon.ion.impl.bin.IntList;
 import com.amazon.ion.system.IonReaderBuilder;
 import com.amazon.ion.impl.bin.utf8.Utf8StringDecoder;
 import com.amazon.ion.impl.bin.utf8.Utf8StringDecoderPool;
@@ -27,7 +28,6 @@ import java.math.BigInteger;
 import java.nio.ByteBuffer;
 
 import java.util.ArrayList;
-import java.util.Collections;
 import java.util.Date;
 import java.util.HashMap;
 import java.util.Iterator;
@@ -216,7 +216,7 @@ class IonReaderBinaryIncremental implements IonReader, _Private_ReaderWriter, _P
     private final Utf8StringDecoder utf8Decoder = Utf8StringDecoderPool.getInstance().getOrCreate();
 
     // The symbol IDs for the annotations on the current value.
-    private final List<Integer> annotationSids;
+    private final IntList annotationSids;
 
     // True if the annotation iterator will be reused across values; otherwise, false.
     private final boolean isAnnotationIteratorReuseEnabled;
@@ -285,8 +285,8 @@ class IonReaderBinaryIncremental implements IonReader, _Private_ReaderWriter, _P
     // The buffer position of the first byte of the annotation wrapper for the current value.
     private int annotationStartPosition = -1;
 
-    // The number of bytes occupied by the annotation SIDs in the annotation wrapper for the current value.
-    private int annotationsLength = -1;
+    // The buffer position of the byte after the last byte in the annotation wrapper for the current value.
+    private int annotationEndPosition = -1;
 
     // The index of the next byte to peek from the underlying buffer.
     private int peekIndex = -1;
@@ -318,7 +318,7 @@ class IonReaderBinaryIncremental implements IonReader, _Private_ReaderWriter, _P
             CONTAINER_STACK_INITIAL_CAPACITY,
             CONTAINER_INFO_FACTORY
         );
-        annotationSids = new ArrayList<Integer>(ANNOTATIONS_LIST_INITIAL_CAPACITY);
+        annotationSids = new IntList(ANNOTATIONS_LIST_INITIAL_CAPACITY);
         symbols = new ArrayList<String>(SYMBOLS_LIST_INITIAL_CAPACITY);
         scalarConverter = new _Private_ScalarConversions.ValueVariant();
         resetImports();
@@ -329,24 +329,25 @@ class IonReaderBinaryIncremental implements IonReader, _Private_ReaderWriter, _P
      */
     private class AnnotationIterator implements Iterator<String> {
 
-        // All of the annotation SIDs on the current value.
-        protected List<Integer> annotationSids = Collections.emptyList();
-        // The index into `annotationSids` containing the next annotation to be returned.
-        protected int index = 0;
+        // The byte position of the annotation to return from the next call to next().
+        private int nextAnnotationPeekIndex;
 
         @Override
         public boolean hasNext() {
-            return index < annotationSids.size();
+            return nextAnnotationPeekIndex < annotationEndPosition;
         }
 
         @Override
         public String next() {
-            int sid = annotationSids.get(index);
+            int savedPeekIndex = peekIndex;
+            peekIndex = nextAnnotationPeekIndex;
+            int sid = readVarUInt();
+            nextAnnotationPeekIndex = peekIndex;
+            peekIndex = savedPeekIndex;
             String annotation = getSymbol(sid);
             if (annotation == null) {
                 throw new UnknownSymbolException(sid);
             }
-            index++;
             return annotation;
         }
 
@@ -356,11 +357,18 @@ class IonReaderBinaryIncremental implements IonReader, _Private_ReaderWriter, _P
         }
 
         /**
-         * Reset the iterator so that it may be reused.
+         * Prepare the iterator to iterate over the annotations on the current value.
+         */
+        void ready() {
+            nextAnnotationPeekIndex = annotationStartPosition;
+        }
+
+        /**
+         * Invalidate the iterator so that all future calls to {@link #hasNext()} will return false until the
+         * next call to {@link #ready()}.
          */
-        public void reset() {
-            index = 0;
-            annotationSids = getAnnotationSids();
+        void invalidate() {
+            nextAnnotationPeekIndex = Integer.MAX_VALUE;
         }
     }
 
@@ -368,16 +376,36 @@ class IonReaderBinaryIncremental implements IonReader, _Private_ReaderWriter, _P
      * Non-reusable iterator over the annotations on the current value. May be iterated even if the reader advances
      * past the current value.
      */
-    private class SingleUseAnnotationIterator extends AnnotationIterator {
+    private class SingleUseAnnotationIterator implements Iterator<String> {
+
+        // All of the annotation SIDs on the current value.
+        private final IntList annotationSids;
+        // The index into `annotationSids` containing the next annotation to be returned.
+        private int index = 0;
 
         SingleUseAnnotationIterator() {
-            index = 0;
-            annotationSids = new ArrayList<Integer>(getAnnotationSids());
+            annotationSids = new IntList(getAnnotationSids());
         }
 
         @Override
-        public void reset() {
-            throw new IllegalStateException("Single-use annotation iterators cannot be reset.");
+        public boolean hasNext() {
+            return index < annotationSids.size();
+        }
+
+        @Override
+        public String next() {
+            int sid = annotationSids.get(index);
+            String annotation = getSymbol(sid);
+            if (annotation == null) {
+                throw new UnknownSymbolException(sid);
+            }
+            index++;
+            return annotation;
+        }
+
+        @Override
+        public void remove() {
+            throw new UnsupportedOperationException("This iterator does not support element removal.");
         }
     }
 
@@ -694,6 +722,16 @@ class IonReaderBinaryIncremental implements IonReader, _Private_ReaderWriter, _P
         }
     }
 
+    /**
+     * Resets the value's annotations.
+     */
+    private void resetAnnotations() {
+        hasAnnotations = false;
+        if (isAnnotationIteratorReuseEnabled) {
+            annotationIterator.invalidate();
+        }
+    }
+
     /**
      * Clear the list of imported shared symbol tables.
      */
@@ -801,7 +839,7 @@ class IonReaderBinaryIncremental implements IonReader, _Private_ReaderWriter, _P
      * Reads a local symbol table from the buffer.
      * @param marker marker for the start and end positions of the local symbol table in the buffer.
      */
-    private void readSymbolTable(IonReaderLookaheadBuffer.SymbolTableMarker marker) {
+    private void readSymbolTable(IonReaderLookaheadBuffer.Marker marker) {
         peekIndex = marker.startIndex;
         boolean isAppend = false;
         boolean hasSeenImports = false;
@@ -937,22 +975,24 @@ class IonReaderBinaryIncremental implements IonReader, _Private_ReaderWriter, _P
             // not represent valid binary Ion, a quick failure is necessary.
             throw new IonException("Binary Ion must start with an Ion version marker.");
         }
-        List<IonReaderLookaheadBuffer.SymbolTableMarker> symbolTableMarkers = lookahead.getSymbolTableMarkers();
+        List<IonReaderLookaheadBuffer.Marker> symbolTableMarkers = lookahead.getSymbolTableMarkers();
         if (!symbolTableMarkers.isEmpty()) {
             // The cached SymbolTable (if any) is a snapshot in time, so it must be cleared whenever a new symbol
             // table is read regardless of whether the new LST is an append or a reset.
             cachedReadOnlySymbolTable = null;
-            for (IonReaderLookaheadBuffer.SymbolTableMarker symbolTableMarker : symbolTableMarkers) {
+            for (IonReaderLookaheadBuffer.Marker symbolTableMarker : symbolTableMarkers) {
                 readSymbolTable(symbolTableMarker);
             }
             lookahead.resetSymbolTableMarkers();
         }
         peekIndex = lookahead.getValueStart();
-        if (lookahead.getAnnotationSids().isEmpty()) {
-            valueTypeID = lookahead.getValueTid();
-            valueType = valueTypeID.type;
-        } else {
-            hasAnnotations = true;
+        hasAnnotations = lookahead.hasAnnotations();
+        if (hasAnnotations) {
+            annotationSids.clear();
+            IonReaderLookaheadBuffer.Marker annotationSidsMarker = lookahead.getAnnotationSidsMarker();
+            annotationStartPosition = annotationSidsMarker.startIndex;
+            annotationEndPosition = annotationSidsMarker.endIndex;
+            peekIndex = annotationEndPosition;
             valueTypeID = IonTypeID.TYPE_IDS[buffer.peek(peekIndex++)];
             int wrappedValueLength = valueTypeID.length;
             if (valueTypeID.variableLength) {
@@ -965,6 +1005,9 @@ class IonReaderBinaryIncremental implements IonReader, _Private_ReaderWriter, _P
             if (peekIndex + wrappedValueLength != lookahead.getValueEnd()) {
                 throw new IonException("Mismatched annotation wrapper length.");
             }
+        } else {
+            valueTypeID = lookahead.getValueTid();
+            valueType = valueTypeID.type;
         }
         valueStartPosition = peekIndex;
         valueEndPosition = lookahead.getValueEnd();
@@ -1005,7 +1048,7 @@ class IonReaderBinaryIncremental implements IonReader, _Private_ReaderWriter, _P
         valueType = null;
         valueTypeID = null;
         annotationStartPosition = -1;
-        annotationsLength = -1;
+        annotationEndPosition = -1;
         hasAnnotations = false;
     }
 
@@ -1040,9 +1083,10 @@ class IonReaderBinaryIncremental implements IonReader, _Private_ReaderWriter, _P
             if (valueType == IonTypeID.ION_TYPE_ANNOTATION_WRAPPER) {
                 hasAnnotations = true;
                 annotationSids.clear();
-                annotationsLength = readVarUInt();
+                int annotationsLength = readVarUInt();
                 annotationStartPosition = peekIndex;
-                peekIndex = annotationStartPosition + annotationsLength;
+                annotationEndPosition = annotationStartPosition + annotationsLength;
+                peekIndex = annotationEndPosition;
                 typeID = readTypeId();
                 if (typeID.isNopPad) {
                     throw new IonException(
@@ -1061,7 +1105,7 @@ class IonReaderBinaryIncremental implements IonReader, _Private_ReaderWriter, _P
                 }
             } else {
                 annotationStartPosition = -1;
-                annotationsLength = -1;
+                annotationEndPosition = -1;
                 hasAnnotations = false;
                 if (valueEndPosition > containerStack.peek().endPosition) {
                     throw new IonException("Value overflowed its container.");
@@ -1079,7 +1123,7 @@ class IonReaderBinaryIncremental implements IonReader, _Private_ReaderWriter, _P
         fieldNameSid = -1;
         lobBytesRead = 0;
         valueStartPosition = -1;
-        hasAnnotations = false;
+        resetAnnotations();
         if (containerStack.isEmpty()) {
             nextAtTopLevel();
         } else {
@@ -1701,27 +1745,22 @@ class IonReaderBinaryIncremental implements IonReader, _Private_ReaderWriter, _P
      * Gets the annotation symbol IDs for the current value, reading them from the buffer first if necessary.
      * @return the annotation symbol IDs, or an empty list if the current value is not annotated.
      */
-    private List<Integer> getAnnotationSids() {
-        if (containerStack.isEmpty()) {
-            return lookahead.getAnnotationSids();
-        } else {
-            if (annotationSids.isEmpty()) {
-                int savedPeekIndex = peekIndex;
-                peekIndex = annotationStartPosition;
-                long annotationsEndPosition = peekIndex + annotationsLength;
-                while (peekIndex < annotationsEndPosition) {
-                    annotationSids.add(readVarUInt());
-                }
-                peekIndex = savedPeekIndex;
+    private IntList getAnnotationSids() {
+        if (annotationSids.isEmpty()) {
+            int savedPeekIndex = peekIndex;
+            peekIndex = annotationStartPosition;
+            while (peekIndex < annotationEndPosition) {
+                annotationSids.add(readVarUInt());
             }
-            return annotationSids;
+            peekIndex = savedPeekIndex;
         }
+        return annotationSids;
     }
 
     @Override
     public String[] getTypeAnnotations() {
         if (hasAnnotations) {
-            List<Integer> annotationSids = getAnnotationSids();
+            IntList annotationSids = getAnnotationSids();
             String[] annotationArray = new String[annotationSids.size()];
             for (int i = 0; i < annotationArray.length; i++) {
                 String symbol = getSymbol(annotationSids.get(i));
@@ -1738,7 +1777,7 @@ class IonReaderBinaryIncremental implements IonReader, _Private_ReaderWriter, _P
     @Override
     public SymbolToken[] getTypeAnnotationSymbols() {
         if (hasAnnotations) {
-            List<Integer> annotationSids = getAnnotationSids();
+            IntList annotationSids = getAnnotationSids();
             SymbolToken[] annotationArray = new SymbolToken[annotationSids.size()];
             for (int i = 0; i < annotationArray.length; i++) {
                 annotationArray[i] = getSymbolToken(annotationSids.get(i));
@@ -1770,7 +1809,7 @@ class IonReaderBinaryIncremental implements IonReader, _Private_ReaderWriter, _P
     public Iterator<String> iterateTypeAnnotations() {
         if (hasAnnotations) {
             if (isAnnotationIteratorReuseEnabled) {
-                annotationIterator.reset();
+                annotationIterator.ready();
                 return annotationIterator;
             } else {
                 return new SingleUseAnnotationIterator();
diff --git a/src/com/amazon/ion/impl/IonReaderLookaheadBuffer.java b/src/com/amazon/ion/impl/IonReaderLookaheadBuffer.java
index 325daa99..c510b3af 100644
--- a/src/com/amazon/ion/impl/IonReaderLookaheadBuffer.java
+++ b/src/com/amazon/ion/impl/IonReaderLookaheadBuffer.java
@@ -142,24 +142,24 @@ public final class IonReaderLookaheadBuffer extends ReaderLookaheadBufferBase {
     }
 
     /**
-     * Holds the start and end indices of a buffered symbol table.
+     * Holds the start and end indices of a slice of the buffer.
      */
-    static class SymbolTableMarker {
+    static class Marker {
         /**
-         * Index of the first byte of the symbol table struct's contents.
+         * Index of the first byte in the slice.
          */
         int startIndex;
 
         /**
-         * Index of the first byte after the end of the symbol table.
+         * Index of the first byte after the end of the slice.
          */
         int endIndex;
 
         /**
-         * @param startIndex index of the first byte of the symbol table struct's contents.
-         * @param length the number of bytes that remain in the symbol table struct after 'startIndex'.
+         * @param startIndex index of the first byte in the slice.
+         * @param length the number of bytes in the slice.
          */
-        private SymbolTableMarker(final int startIndex, final int length) {
+        private Marker(final int startIndex, final int length) {
             this.startIndex = startIndex;
             this.endIndex = startIndex + length;
         }
@@ -178,12 +178,13 @@ public final class IonReaderLookaheadBuffer extends ReaderLookaheadBufferBase {
     /**
      * Markers for any symbol tables that occurred in the stream between the last value and the current value.
      */
-    private final List<SymbolTableMarker> symbolTableMarkers = new ArrayList<SymbolTableMarker>(2);
+    private final List<Marker> symbolTableMarkers = new ArrayList<Marker>(2);
 
     /**
-     * The symbol IDs of any annotations on the current value.
+     * Marker for the sequence of annotation symbol IDs on the current value. If there are no annotations on the
+     * current value, the startIndex will be negative.
      */
-    private final List<Integer> annotationSids = new ArrayList<Integer>(3);
+    private final Marker annotationSidsMarker = new Marker(-1, 0);
 
     /**
      * The handler that will be notified when a symbol table exceeds the maximum buffer size.
@@ -204,12 +205,6 @@ public final class IonReaderLookaheadBuffer extends ReaderLookaheadBufferBase {
      */
     private boolean isSystemValue;
 
-    /**
-     * True if the current value has an annotation wrapper whose first annotation is `$ion_symbol_table`.
-     * The value will be deemed a system value if it is later determined to be a struct.
-     */
-    private boolean isSymbolTableAnnotationFirst;
-
     /**
      * The number of bytes of annotation SIDs left to read from the value's annotation wrapper.
      */
@@ -284,14 +279,13 @@ public final class IonReaderLookaheadBuffer extends ReaderLookaheadBufferBase {
     private void reset() {
         additionalBytesNeeded = 0;
         isSystemValue = false;
-        isSymbolTableAnnotationFirst = false;
         numberOfAnnotationSidBytesRemaining = 0;
         currentNumberOfAnnotations = 0;
         valuePreHeaderIndex = -1;
         valuePostHeaderIndex = -1;
         valueTid = null;
         valueEndIndex = -1;
-        annotationSids.clear();
+        annotationSidsMarker.startIndex = -1;
         valueStartAvailable = pipe.available();
         startNewValue();
     }
@@ -490,29 +484,21 @@ public final class IonReaderLookaheadBuffer extends ReaderLookaheadBufferBase {
             additionalBytesNeeded -= inProgressVarUInt.numberOfBytesRead;
             numberOfAnnotationSidBytesRemaining = inProgressVarUInt.value;
             initializeVarUInt(VarUInt.Location.ANNOTATION_WRAPPER_SID);
+            annotationSidsMarker.startIndex = peekIndex;
+            annotationSidsMarker.endIndex = annotationSidsMarker.startIndex + (int) numberOfAnnotationSidBytesRemaining;
         }
         if (inProgressVarUInt.location == VarUInt.Location.ANNOTATION_WRAPPER_SID) {
-            while (true) {
-                readVarUInt();
-                if (inProgressVarUInt.isComplete) {
-                    currentNumberOfAnnotations++;
-                    if (currentNumberOfAnnotations == 1 && inProgressVarUInt.value == ION_SYMBOL_TABLE_SID) {
-                        isSymbolTableAnnotationFirst = true;
-                    }
-                    annotationSids.add((int) inProgressVarUInt.value);
-                    numberOfAnnotationSidBytesRemaining -= inProgressVarUInt.numberOfBytesRead;
-                    additionalBytesNeeded -= inProgressVarUInt.numberOfBytesRead;
-                    if (numberOfAnnotationSidBytesRemaining <= 0) {
-                        state = State.SKIPPING_VALUE;
-                    } else {
-                        initializeVarUInt(VarUInt.Location.ANNOTATION_WRAPPER_SID);
-                        continue;
-                    }
-                    if (isSymbolTableAnnotationFirst) {
-                        state = State.READING_VALUE_WITH_SYMBOL_TABLE_ANNOTATION;
-                    }
+            // Read the first annotation SID, which is all that is required to determine whether the value is a
+            // symbol table.
+            readVarUInt();
+            if (inProgressVarUInt.isComplete) {
+                numberOfAnnotationSidBytesRemaining -= inProgressVarUInt.numberOfBytesRead;
+                additionalBytesNeeded -= inProgressVarUInt.numberOfBytesRead;
+                if (inProgressVarUInt.value == ION_SYMBOL_TABLE_SID) {
+                    state = State.READING_VALUE_WITH_SYMBOL_TABLE_ANNOTATION;
+                } else {
+                    state = State.SKIPPING_VALUE;
                 }
-                break;
             }
         }
     }
@@ -528,12 +514,16 @@ public final class IonReaderLookaheadBuffer extends ReaderLookaheadBufferBase {
         peekIndex = Math.max(peekIndex - shiftAmount, 0);
         valuePreHeaderIndex -= shiftAmount;
         valuePostHeaderIndex -= shiftAmount;
-        for (SymbolTableMarker symbolTableMarker : symbolTableMarkers) {
+        for (Marker symbolTableMarker : symbolTableMarkers) {
             if (symbolTableMarker.startIndex > afterIndex) {
                 symbolTableMarker.startIndex -= shiftAmount;
                 symbolTableMarker.endIndex -= shiftAmount;
             }
         }
+        if (annotationSidsMarker.startIndex > afterIndex) {
+            annotationSidsMarker.startIndex -= shiftAmount;
+            annotationSidsMarker.endIndex -= shiftAmount;
+        }
         if (ivmSecondByteIndex > afterIndex) {
             ivmSecondByteIndex -= shiftAmount;
         }
@@ -660,6 +650,27 @@ public final class IonReaderLookaheadBuffer extends ReaderLookaheadBufferBase {
         return bytesFilled;
     }
 
+    /**
+     * Attempts to skip the requested number of bytes.
+     * @param numberOfBytesToSkip the number of bytes to attempt to skip.
+     * @return the number of bytes actually skipped.
+     * @throws Exception if thrown by the event handler.
+     */
+    private long skip(long numberOfBytesToSkip) throws Exception {
+        int numberOfBytesSkipped;
+        if (pipe.availableBeyondBoundary() >= numberOfBytesToSkip) {
+            numberOfBytesSkipped = (int) numberOfBytesToSkip;
+            pipe.extendBoundary(numberOfBytesSkipped);
+            peekIndex += numberOfBytesSkipped;
+        } else {
+            numberOfBytesSkipped = fillOrSkip();
+        }
+        if (numberOfBytesSkipped > 0) {
+            dataHandler.onData(numberOfBytesSkipped);
+        }
+        return numberOfBytesSkipped;
+    }
+
     /*
      * The state transitions of the fillInput() method are summarized by the following diagram.
      *
@@ -729,6 +740,15 @@ public final class IonReaderLookaheadBuffer extends ReaderLookaheadBufferBase {
                 valuePostHeaderIndex = peekIndex;
             }
             if (state == State.READING_VALUE_WITH_SYMBOL_TABLE_ANNOTATION) {
+                // Skip annotations until positioned on the value's type ID.
+                while (numberOfAnnotationSidBytesRemaining > 0) {
+                    long numberOfBytesSkipped = skip(numberOfAnnotationSidBytesRemaining);
+                    if (numberOfBytesSkipped < 1) {
+                        return;
+                    }
+                    numberOfAnnotationSidBytesRemaining -= numberOfBytesSkipped;
+                    additionalBytesNeeded -= numberOfBytesSkipped;
+                }
                 ReadTypeIdResult result = readTypeID(false);
                 if (result == ReadTypeIdResult.NO_DATA) {
                     return;
@@ -750,7 +770,7 @@ public final class IonReaderLookaheadBuffer extends ReaderLookaheadBufferBase {
                     }
                     additionalBytesNeeded = inProgressVarUInt.value;
                 }
-                symbolTableMarkers.add(new SymbolTableMarker(peekIndex, (int) additionalBytesNeeded));
+                symbolTableMarkers.add(new Marker(peekIndex, (int) additionalBytesNeeded));
                 state = State.SKIPPING_VALUE;
             }
             if (state == State.SKIPPING_VALUE) {
@@ -768,19 +788,11 @@ public final class IonReaderLookaheadBuffer extends ReaderLookaheadBufferBase {
                     // large enough that it doesn't fit within the buffer's configured maximum size.
                 }
                 while (additionalBytesNeeded > 0) {
-                    int numberOfBytesToRead;
-                    if (pipe.availableBeyondBoundary() >= additionalBytesNeeded) {
-                        numberOfBytesToRead = (int) additionalBytesNeeded;
-                        pipe.extendBoundary(numberOfBytesToRead);
-                        peekIndex += numberOfBytesToRead;
-                    } else {
-                        numberOfBytesToRead = fillOrSkip();
-                        if (numberOfBytesToRead < 1) {
-                            return;
-                        }
+                    long numberOfBytesSkipped = skip(additionalBytesNeeded);
+                    if (numberOfBytesSkipped < 1) {
+                        return;
                     }
-                    dataHandler.onData(numberOfBytesToRead);
-                    additionalBytesNeeded -= numberOfBytesToRead;
+                    additionalBytesNeeded -= numberOfBytesSkipped;
                 }
                 state = State.BEFORE_TYPE_ID;
             }
@@ -880,6 +892,9 @@ public final class IonReaderLookaheadBuffer extends ReaderLookaheadBufferBase {
      * @return the index of the first byte of the value representation (past the type ID and the optional length field).
      */
     int getValueStart() {
+        if (hasAnnotations()) {
+            return annotationSidsMarker.endIndex;
+        }
         return valuePostHeaderIndex;
     }
 
@@ -898,9 +913,12 @@ public final class IonReaderLookaheadBuffer extends ReaderLookaheadBufferBase {
     }
 
     /**
-     * @return markers for any symbol tables that occurred in the stream between the last value and the current value.
+     * Returns markers for any symbol tables that occurred in the stream between the last value and the current value.
+     * The startIndex of the returned markers is the index of the first byte of the symbol table struct's contents.
+     * The endIndex of the returned markers is the index of the first byte after the end of the symbol table.
+     * @return the markers.
      */
-    List<SymbolTableMarker> getSymbolTableMarkers() {
+    List<Marker> getSymbolTableMarkers() {
         return symbolTableMarkers;
     }
 
@@ -912,10 +930,19 @@ public final class IonReaderLookaheadBuffer extends ReaderLookaheadBufferBase {
     }
 
     /**
-     * @return the symbol IDs of any annotations on the current value.
+     * @return true if the current value has annotations; otherwise, false.
+     */
+    boolean hasAnnotations() {
+        return annotationSidsMarker.startIndex >= 0;
+    }
+    /**
+     * Returns the marker for the sequence of annotation symbol IDs on the current value. The startIndex of the
+     * returned marker is the index of the first byte of the first annotation symbol ID in the sequence. The endIndex
+     * of the returned marker is the index of the type ID byte of the value to which the annotations are applied.
+     * @return  the marker.
      */
-    List<Integer> getAnnotationSids() {
-        return annotationSids;
+    Marker getAnnotationSidsMarker() {
+        return annotationSidsMarker;
     }
 
 }
diff --git a/src/com/amazon/ion/impl/bin/IntList.java b/src/com/amazon/ion/impl/bin/IntList.java
index 8c0ba5c2..3ff039cc 100644
--- a/src/com/amazon/ion/impl/bin/IntList.java
+++ b/src/com/amazon/ion/impl/bin/IntList.java
@@ -29,6 +29,16 @@ public class IntList {
         numberOfValues = 0;
     }
 
+    /**
+     * Constructs a new IntList that contains all the elements of the given IntList.
+     * @param other the IntList to copy.
+     */
+    public IntList(final IntList other) {
+        this.numberOfValues = other.numberOfValues;
+        this.data = new int[other.data.length];
+        System.arraycopy(other.data, 0, this.data, 0, numberOfValues);
+    }
+
     /**
      * Accessor.
      * @return  The number of ints currently stored in the list.


Test patch:
diff --git a/test/com/amazon/ion/impl/IonReaderLookaheadBufferTest.java b/test/com/amazon/ion/impl/IonReaderLookaheadBufferTest.java
index d5602750..bac28697 100644
--- a/test/com/amazon/ion/impl/IonReaderLookaheadBufferTest.java
+++ b/test/com/amazon/ion/impl/IonReaderLookaheadBufferTest.java
@@ -1000,7 +1000,7 @@ public class IonReaderLookaheadBufferTest
 
         lookahead.fillInput();
         assertEquals(1, lookahead.getIvmIndex());
-        assertTrue(lookahead.getAnnotationSids().isEmpty());
+        assertFalse(lookahead.hasAnnotations());
         assertTrue(lookahead.getSymbolTableMarkers().isEmpty());
         assertEquals(5, lookahead.getValueStart());
         assertIonTypeId(IonType.INT, 0x1, lookahead.getValueTid());
@@ -1014,7 +1014,7 @@ public class IonReaderLookaheadBufferTest
         assertEquals(indexForInitialBufferSize(1, 1, 7), lookahead.getIvmIndex());
         lookahead.resetIvmIndex();
         assertEquals(-1, lookahead.getIvmIndex());
-        assertTrue(lookahead.getAnnotationSids().isEmpty());
+        assertFalse(lookahead.hasAnnotations());
         assertTrue(lookahead.getSymbolTableMarkers().isEmpty());
         assertEquals(indexForInitialBufferSize(6, 6, 12), lookahead.getValueStart());
         assertIonTypeId(IonType.STRUCT, 0x1, lookahead.getValueTid());
@@ -1023,7 +1023,9 @@ public class IonReaderLookaheadBufferTest
         lookahead.fillInput();
         assertEquals(-1, lookahead.getIvmIndex());
         assertTrue(lookahead.getSymbolTableMarkers().isEmpty());
-        assertEquals(Arrays.asList(4, 5), lookahead.getAnnotationSids());
+        assertTrue(lookahead.hasAnnotations());
+        assertEquals(indexForInitialBufferSize(10, 10, 16), lookahead.getAnnotationSidsMarker().startIndex);
+        assertEquals(indexForInitialBufferSize(12, 12, 18), lookahead.getAnnotationSidsMarker().endIndex);
         assertEquals(indexForInitialBufferSize(12, 12, 18), lookahead.getValueStart());
         assertIonTypeId(IonTypeID.ION_TYPE_ANNOTATION_WRAPPER, 0x5, lookahead.getValueTid());
         assertEquals(indexForInitialBufferSize(14, 14, 20), lookahead.getValueEnd());
@@ -1031,7 +1033,7 @@ public class IonReaderLookaheadBufferTest
         lookahead.fillInput();
         assertEquals(-1, lookahead.getIvmIndex());
         assertTrue(lookahead.getSymbolTableMarkers().isEmpty());
-        assertTrue(lookahead.getAnnotationSids().isEmpty());
+        assertFalse(lookahead.hasAnnotations());
         assertEquals(indexForInitialBufferSize(15, 15, 21), lookahead.getValueStart());
         assertIonTypeId(IonType.INT, 0x0, lookahead.getValueTid());
         assertEquals(indexForInitialBufferSize(15, 15, 21), lookahead.getValueEnd());
@@ -1060,8 +1062,8 @@ public class IonReaderLookaheadBufferTest
             builder.withInitialBufferSize(initialBufferSize);
         }
         lookahead = bufferFor(
-            // Symbol table declaring symbol 'x'.
-            0xE7, 0x81, 0x83, 0xD4, 0x87, 0xB2, 0x81, 'x',
+            // Symbol table with extra annotation 'name' declaring symbol 'x'.
+            0xE8, 0x82, 0x83, 0x84, 0xD4, 0x87, 0xB2, 0x81, 'x',
             // Symbol value with SID 10 ('x').
             0x71, 0x0A,
             // The value will not be consumed, so the indices continue.
@@ -1087,25 +1089,25 @@ public class IonReaderLookaheadBufferTest
         );
         lookahead.fillInput();
         assertEquals(1, lookahead.getIvmIndex());
-        assertTrue(lookahead.getAnnotationSids().isEmpty());
-        List<IonReaderLookaheadBuffer.SymbolTableMarker> symbolTableMarkers = lookahead.getSymbolTableMarkers();
+        assertFalse(lookahead.hasAnnotations());
+        List<IonReaderLookaheadBuffer.Marker> symbolTableMarkers = lookahead.getSymbolTableMarkers();
         assertEquals(1, symbolTableMarkers.size());
-        assertEquals(8, symbolTableMarkers.get(0).startIndex);
-        assertEquals(12, symbolTableMarkers.get(0).endIndex);
-        assertEquals(13, lookahead.getValueStart());
+        assertEquals(9, symbolTableMarkers.get(0).startIndex);
+        assertEquals(13, symbolTableMarkers.get(0).endIndex);
+        assertEquals(14, lookahead.getValueStart());
         assertIonTypeId(IonType.SYMBOL, 0x1, lookahead.getValueTid());
-        assertEquals(14, lookahead.getValueEnd());
+        assertEquals(15, lookahead.getValueEnd());
         lookahead.resetSymbolTableMarkers();
         assertTrue(lookahead.getSymbolTableMarkers().isEmpty());
 
         lookahead.fillInput();
         symbolTableMarkers = lookahead.getSymbolTableMarkers();
         assertEquals(1, symbolTableMarkers.size());
-        assertEquals(18, symbolTableMarkers.get(0).startIndex);
-        assertEquals(25, symbolTableMarkers.get(0).endIndex);
-        assertEquals(26, lookahead.getValueStart());
+        assertEquals(19, symbolTableMarkers.get(0).startIndex);
+        assertEquals(26, symbolTableMarkers.get(0).endIndex);
+        assertEquals(27, lookahead.getValueStart());
         assertIonTypeId(IonType.SYMBOL, 0x1, lookahead.getValueTid());
-        assertEquals(27, lookahead.getValueEnd());
+        assertEquals(28, lookahead.getValueEnd());
 
         IonReader reader = lookahead.newIonReader(IonReaderBuilder.standard());
         assertEquals(IonType.SYMBOL, reader.next());
@@ -1116,13 +1118,13 @@ public class IonReaderLookaheadBufferTest
         lookahead.fillInput();
         symbolTableMarkers = lookahead.getSymbolTableMarkers();
         assertEquals(2, symbolTableMarkers.size());
-        assertEquals(indexForInitialBufferSize(28, 28, 55), symbolTableMarkers.get(0).startIndex);
-        assertEquals(indexForInitialBufferSize(32, 32, 59), symbolTableMarkers.get(0).endIndex);
-        assertEquals(indexForInitialBufferSize(36, 36, 63), symbolTableMarkers.get(1).startIndex);
-        assertEquals(indexForInitialBufferSize(40, 40, 67), symbolTableMarkers.get(1).endIndex);
-        assertEquals(indexForInitialBufferSize(41, 41, 68), lookahead.getValueStart());
+        assertEquals(indexForInitialBufferSize(28, 28, 56), symbolTableMarkers.get(0).startIndex);
+        assertEquals(indexForInitialBufferSize(32, 32, 60), symbolTableMarkers.get(0).endIndex);
+        assertEquals(indexForInitialBufferSize(36, 36, 64), symbolTableMarkers.get(1).startIndex);
+        assertEquals(indexForInitialBufferSize(40, 40, 68), symbolTableMarkers.get(1).endIndex);
+        assertEquals(indexForInitialBufferSize(41, 41, 69), lookahead.getValueStart());
         assertIonTypeId(IonType.SYMBOL, 0x1, lookahead.getValueTid());
-        assertEquals(indexForInitialBufferSize(42, 42, 69), lookahead.getValueEnd());
+        assertEquals(indexForInitialBufferSize(42, 42, 70), lookahead.getValueEnd());
 
         assertEquals(IonType.SYMBOL, reader.next());
         assertEquals("d", reader.stringValue());
@@ -1194,4 +1196,42 @@ public class IonReaderLookaheadBufferTest
         assertNull(reader.next());
         reader.close();
     }
+
+    @Test
+    public void annotationMarkersAreCorrectlyShifted() throws Exception {
+        if (initialBufferSize == null || initialBufferSize != 1) {
+            return;
+        }
+        // Set the maximum size at 1 IVM (4 bytes) + the symbol table (12 bytes) + the value (2 bytes) + part of the
+        // next value (3 bytes).
+        builder = IonBufferConfiguration.Builder.standard()
+            .withInitialBufferSize(21)
+            .withMaximumBufferSize(21);
+        createCountingEventHandler(builder, new AtomicLong());
+        lookahead = bufferFor(
+            // Symbol table with the symbol 'hello'.
+            0xEB, 0x81, 0x83, 0xD8, 0x87, 0xB6, 0x85, 'h', 'e', 'l', 'l', 'o',
+            // Symbol 10 (hello)
+            0x71, 0x0A,
+            // name::hello
+            0xE4, 0x81, 0x84, 0x71, 0x0A
+        );
+
+        lookahead.fillInput();
+        assertFalse(lookahead.moreDataRequired());
+        IonReader reader = lookahead.newIonReader(IonReaderBuilder.standard());
+        assertEquals(IonType.SYMBOL, reader.next());
+        assertFalse(lookahead.hasAnnotations());
+        // The buffer will reach its maximum size after recording the annotation marker. The marker's indices will
+        // be shifted left when space is reclaimed to fit the value.
+        lookahead.fillInput();
+        assertFalse(lookahead.moreDataRequired());
+        assertEquals("hello", reader.stringValue());
+        assertEquals(IonType.SYMBOL, reader.next());
+        assertTrue(lookahead.hasAnnotations());
+        assertEquals(2, lookahead.getAnnotationSidsMarker().startIndex);
+        assertEquals(3, lookahead.getAnnotationSidsMarker().endIndex);
+        assertNull(reader.next());
+        reader.close();
+    }
 }