Repository: opensearch-project/neural-search
Pull request #1217: implement batch document optimization for text embedding processor
Author: login: will-hwang
Created: 2025-03-07T22:42:52Z | Merged: 2025-03-18T21:13:36Z

### Description
This PR includes changes for:

1. refactor of InferenceFilter to simplify logic for copying embeddings
2. implementation for batch document update optimization by overriding InferenceProcessor's `subBatchExecute` to include filter when `skip_existing` flag is on

### Proposed State [Batch Document Update]

![proposed-batch-flow](https://github.com/user-attachments/assets/b9d4c06a-d9e4-40af-8792-85f9a35b9f2d)

Steps:

1. Process Maps are generated for each IngestDocument based on the Field Map defined. 
2. If skip_existing feature is set to true, filter the process map for each IngestDocument. 
    1. Existing documents are fetched via OpenSearch client’s Multi-Get Action to compare each of the existing inference text against its corresponding IngestDocument 
        1. if documents do not exist, or any exception is thrown, fallback to calling model inference
    2. Locate the embedding fields in each existing document
        1. Recursively traverse the process map to find embedding fields. Keeping track of the traversal path is required for look up in existing document. 
        2. Once embedding fields are found, attempt to copy embeddings from existing document to its corresponding ingest document. 
    3. If eligible, copy over the vector embeddings from existing document to its corresponding ingest document
        1. It is eligible for copy if inference text in ingest document and its corresponding existing document is the same, and embeddings for the inference text exist in its existing document. 
        2. Note, if in case of values in list, the fields in the same index are compared to determine text equality
    4. If eligible fields have been copied, remove the entry in process map 
3. Inference List is generated based on entries in Filtered Process Map. 
4. ML Commons InferenceSentence API is invoked with filtered inference list.
5. Embeddings for filtered inference list are generated in ML Commons. 
6. Embeddings for filtered inference list are mapped to target fields via entries defined in process map.
7. Embeddings for filtered inference list are populated to target fields in IngestDocument.

### Related Issues
HLD: https://github.com/opensearch-project/neural-search/issues/1138
### Check List
- [x] New functionality includes testing.
- [x] New functionality has been documented.
- [ ] API changes companion pull request [created](https://github.com/opensearch-project/opensearch-api-specification/blob/main/DEVELOPER_GUIDE.md).
- [x] Commits are signed per the DCO using `--signoff`.
- [ ] Public documentation issue/PR [created](https://github.com/opensearch-project/documentation-website/issues/new/choose).

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check [here](https://github.com/opensearch-project/neural-search/blob/main/CONTRIBUTING.md#developer-certificate-of-origin).



Diff:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c84cf6830e..7098561cf6 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,7 +5,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 
 ## [Unreleased 3.0](https://github.com/opensearch-project/neural-search/compare/2.x...HEAD)
 ### Features
-- Add Optimized Text Embedding Processor ([#1191](https://github.com/opensearch-project/neural-search/pull/1191))
+- Optimizing embedding generation in text embedding processor ([#1191](https://github.com/opensearch-project/neural-search/pull/1191))
 ### Enhancements
 - Set neural-search plugin 3.0.0 baseline JDK version to JDK-21 ([#838](https://github.com/opensearch-project/neural-search/pull/838))
 - Support different embedding types in model's response ([#1007](https://github.com/opensearch-project/neural-search/pull/1007))
diff --git a/src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java b/src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
index 7d0b683b41..efb1d7694a 100644
--- a/src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
+++ b/src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
@@ -26,6 +26,8 @@
 import org.apache.commons.lang3.StringUtils;
 import org.apache.commons.lang3.tuple.ImmutablePair;
 import org.apache.commons.lang3.tuple.Pair;
+import org.opensearch.action.get.MultiGetItemResponse;
+import org.opensearch.action.get.MultiGetRequest;
 import org.opensearch.common.collect.Tuple;
 import org.opensearch.core.action.ActionListener;
 import org.opensearch.core.common.util.CollectionUtils;
@@ -54,6 +56,8 @@ public abstract class InferenceProcessor extends AbstractBatchingProcessor {
 
     public static final String MODEL_ID_FIELD = "model_id";
     public static final String FIELD_MAP_FIELD = "field_map";
+    public static final String INDEX_FIELD = "_index";
+    public static final String ID_FIELD = "_id";
     private static final BiFunction<Object, Object, Object> REMAPPING_FUNCTION = (v1, v2) -> {
         if (v1 instanceof Collection && v2 instanceof Collection) {
             ((Collection) v1).addAll((Collection) v2);
@@ -169,49 +173,91 @@ void preprocessIngestDocument(IngestDocument ingestDocument) {
      */
     abstract void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler, Consumer<Exception> onException);
 
+    /**
+     * This is the function which does actual inference work for subBatchExecute interface.
+     * @param ingestDocumentWrappers a list of IngestDocuments in a batch.
+     * @param handler a callback handler to handle inference results which is a list of objects.
+     */
     @Override
     public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
-        if (CollectionUtils.isEmpty(ingestDocumentWrappers)) {
-            handler.accept(Collections.emptyList());
-            return;
-        }
+        try {
+            if (CollectionUtils.isEmpty(ingestDocumentWrappers)) {
+                handler.accept(Collections.emptyList());
+                return;
+            }
 
-        List<DataForInference> dataForInferences = getDataForInference(ingestDocumentWrappers);
-        List<String> inferenceList = constructInferenceTexts(dataForInferences);
-        if (inferenceList.isEmpty()) {
+            List<DataForInference> dataForInferences = getDataForInference(ingestDocumentWrappers);
+            List<String> inferenceList = constructInferenceTexts(dataForInferences);
+            if (inferenceList.isEmpty()) {
+                handler.accept(ingestDocumentWrappers);
+                return;
+            }
+            doSubBatchExecute(ingestDocumentWrappers, inferenceList, dataForInferences, handler);
+        } catch (Exception e) {
+            updateWithExceptions(ingestDocumentWrappers, e);
             handler.accept(ingestDocumentWrappers);
-            return;
         }
-        Tuple<List<String>, Map<Integer, Integer>> sortedResult = sortByLengthAndReturnOriginalOrder(inferenceList);
-        inferenceList = sortedResult.v1();
-        Map<Integer, Integer> originalOrder = sortedResult.v2();
-        doBatchExecute(inferenceList, results -> {
-            int startIndex = 0;
-            results = restoreToOriginalOrder(results, originalOrder);
-            for (DataForInference dataForInference : dataForInferences) {
-                if (dataForInference.getIngestDocumentWrapper().getException() != null
-                    || CollectionUtils.isEmpty(dataForInference.getInferenceList())) {
-                    continue;
+    }
+
+    /**
+     * This is a helper function for subBatchExecute, which invokes doBatchExecute for given inference list.
+     * @param ingestDocumentWrappers a list of IngestDocuments in a batch.
+     * @param inferenceList a list of String for inference.
+     * @param dataForInferences a list of data for inference, which includes ingestDocumentWrapper, processMap, inferenceList.
+     * @param handler a callback handler to handle inference results which is a list of objects.
+     */
+    protected void doSubBatchExecute(
+        List<IngestDocumentWrapper> ingestDocumentWrappers,
+        List<String> inferenceList,
+        List<DataForInference> dataForInferences,
+        Consumer<List<IngestDocumentWrapper>> handler
+    ) {
+        try {
+            Tuple<List<String>, Map<Integer, Integer>> sortedResult = sortByLengthAndReturnOriginalOrder(inferenceList);
+            inferenceList = sortedResult.v1();
+            Map<Integer, Integer> originalOrder = sortedResult.v2();
+            doBatchExecute(inferenceList, results -> {
+                try {
+                    int startIndex = 0;
+                    results = restoreToOriginalOrder(results, originalOrder);
+                    for (DataForInference dataForInference : dataForInferences) {
+                        if (dataForInference.getIngestDocumentWrapper().getException() != null
+                            || CollectionUtils.isEmpty(dataForInference.getInferenceList())) {
+                            continue;
+                        }
+                        List<?> inferenceResults = results.subList(startIndex, startIndex + dataForInference.getInferenceList().size());
+                        startIndex += dataForInference.getInferenceList().size();
+                        setVectorFieldsToDocument(
+                            dataForInference.getIngestDocumentWrapper().getIngestDocument(),
+                            dataForInference.getProcessMap(),
+                            inferenceResults
+                        );
+                    }
+                    handler.accept(ingestDocumentWrappers);
+                } catch (Exception e) {
+                    updateWithExceptions(ingestDocumentWrappers, e);
+                    handler.accept(ingestDocumentWrappers);
                 }
-                List<?> inferenceResults = results.subList(startIndex, startIndex + dataForInference.getInferenceList().size());
-                startIndex += dataForInference.getInferenceList().size();
-                setVectorFieldsToDocument(
-                    dataForInference.getIngestDocumentWrapper().getIngestDocument(),
-                    dataForInference.getProcessMap(),
-                    inferenceResults
-                );
-            }
-            handler.accept(ingestDocumentWrappers);
-        }, exception -> {
-            for (IngestDocumentWrapper ingestDocumentWrapper : ingestDocumentWrappers) {
-                // The IngestDocumentWrapper might already run into exception and not sent for inference. So here we only
-                // set exception to IngestDocumentWrapper which doesn't have exception before.
-                if (ingestDocumentWrapper.getException() == null) {
-                    ingestDocumentWrapper.update(ingestDocumentWrapper.getIngestDocument(), exception);
+            }, exception -> {
+                try {
+                    for (IngestDocumentWrapper ingestDocumentWrapper : ingestDocumentWrappers) {
+                        // The IngestDocumentWrapper might already run into exception and not sent for inference. So here we only
+                        // set exception to IngestDocumentWrapper which doesn't have exception before.
+                        if (ingestDocumentWrapper.getException() == null) {
+                            ingestDocumentWrapper.update(ingestDocumentWrapper.getIngestDocument(), exception);
+                        }
+                    }
+                    handler.accept(ingestDocumentWrappers);
+                } catch (Exception e) {
+                    updateWithExceptions(ingestDocumentWrappers, e);
+                    handler.accept(ingestDocumentWrappers);
                 }
-            }
+
+            });
+        } catch (Exception e) {
+            updateWithExceptions(ingestDocumentWrappers, e);
             handler.accept(ingestDocumentWrappers);
-        });
+        }
     }
 
     private Tuple<List<String>, Map<Integer, Integer>> sortByLengthAndReturnOriginalOrder(List<String> inferenceList) {
@@ -238,7 +284,7 @@ private List<?> restoreToOriginalOrder(List<?> results, Map<Integer, Integer> or
         return sortedResults;
     }
 
-    private List<String> constructInferenceTexts(List<DataForInference> dataForInferences) {
+    protected List<String> constructInferenceTexts(List<DataForInference> dataForInferences) {
         List<String> inferenceTexts = new ArrayList<>();
         for (DataForInference dataForInference : dataForInferences) {
             if (dataForInference.getIngestDocumentWrapper().getException() != null
@@ -250,7 +296,7 @@ private List<String> constructInferenceTexts(List<DataForInference> dataForInfer
         return inferenceTexts;
     }
 
-    private List<DataForInference> getDataForInference(List<IngestDocumentWrapper> ingestDocumentWrappers) {
+    protected List<DataForInference> getDataForInference(List<IngestDocumentWrapper> ingestDocumentWrappers) {
         List<DataForInference> dataForInferences = new ArrayList<>();
         for (IngestDocumentWrapper ingestDocumentWrapper : ingestDocumentWrappers) {
             Map<String, Object> processMap = null;
@@ -272,7 +318,7 @@ private List<DataForInference> getDataForInference(List<IngestDocumentWrapper> i
 
     @Getter
     @AllArgsConstructor
-    private static class DataForInference {
+    protected static class DataForInference {
         private final IngestDocumentWrapper ingestDocumentWrapper;
         private final Map<String, Object> processMap;
         private final List<String> inferenceList;
@@ -415,6 +461,36 @@ protected void setVectorFieldsToDocument(IngestDocument ingestDocument, Map<Stri
         nlpResult.forEach(ingestDocument::setFieldValue);
     }
 
+    /**
+     * This method creates a MultiGetRequest from a list of ingest documents to be fetched for comparison
+     * @param ingestDocumentWrappers, list of ingest documents
+     * */
+    protected MultiGetRequest buildMultiGetRequest(List<IngestDocumentWrapper> ingestDocumentWrappers) {
+        MultiGetRequest multiGetRequest = new MultiGetRequest();
+        for (IngestDocumentWrapper ingestDocumentWrapper : ingestDocumentWrappers) {
+            Object index = ingestDocumentWrapper.getIngestDocument().getSourceAndMetadata().get(INDEX_FIELD);
+            Object id = ingestDocumentWrapper.getIngestDocument().getSourceAndMetadata().get(ID_FIELD);
+            if (Objects.nonNull(index) && Objects.nonNull(id)) {
+                multiGetRequest.add(index.toString(), id.toString());
+            }
+        }
+        return multiGetRequest;
+    }
+
+    /**
+     * This method creates a map of documents from MultiGetItemResponse where the key is document ID and value is corresponding document
+     * @param multiGetItemResponses, array of responses from Multi Get Request
+     * */
+    protected Map<String, Map<String, Object>> createDocumentMap(MultiGetItemResponse[] multiGetItemResponses) {
+        Map<String, Map<String, Object>> existingDocuments = new HashMap<>();
+        for (MultiGetItemResponse item : multiGetItemResponses) {
+            String id = item.getId();
+            Map<String, Object> existingDocument = item.getResponse().getSourceAsMap();
+            existingDocuments.put(id, existingDocument);
+        }
+        return existingDocuments;
+    }
+
     @SuppressWarnings({ "unchecked" })
     @VisibleForTesting
     Map<String, Object> buildNLPResult(Map<String, Object> processorMap, List<?> results, Map<String, Object> sourceAndMetadataMap) {
@@ -504,6 +580,13 @@ private void processMapEntryValue(
         }
     }
 
+    // This method updates each ingestDocument with exceptions
+    protected void updateWithExceptions(List<IngestDocumentWrapper> ingestDocumentWrappers, Exception e) {
+        for (IngestDocumentWrapper ingestDocumentWrapper : ingestDocumentWrappers) {
+            ingestDocumentWrapper.update(ingestDocumentWrapper.getIngestDocument(), e);
+        }
+    }
+
     private void processMapEntryValue(
         List<?> results,
         IndexWrapper indexWrapper,
@@ -582,7 +665,7 @@ private List<Map<String, Object>> buildNLPResultForListType(List<String> sourceV
         List<Map<String, Object>> keyToResult = new ArrayList<>();
         sourceValue.stream()
             .filter(Objects::nonNull) // explicit null check is required since sourceValue can contain null values in cases where
-                                      // sourceValue has been filtered
+            // sourceValue has been filtered
             .forEachOrdered(x -> keyToResult.add(ImmutableMap.of(listTypeNestedMapKey, results.get(indexWrapper.index++))));
         return keyToResult;
     }
diff --git a/src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java b/src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
index 74dfb08fec..01210e2507 100644
--- a/src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
+++ b/src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
@@ -4,6 +4,8 @@
  */
 package org.opensearch.neuralsearch.processor;
 
+import java.util.ArrayList;
+import java.util.Collections;
 import java.util.List;
 import java.util.Map;
 import java.util.Objects;
@@ -13,10 +15,14 @@
 
 import org.opensearch.action.get.GetAction;
 import org.opensearch.action.get.GetRequest;
+import org.opensearch.action.get.MultiGetAction;
+import org.opensearch.action.get.MultiGetItemResponse;
 import org.opensearch.cluster.service.ClusterService;
 import org.opensearch.core.action.ActionListener;
+import org.opensearch.core.common.util.CollectionUtils;
 import org.opensearch.env.Environment;
 import org.opensearch.ingest.IngestDocument;
+import org.opensearch.ingest.IngestDocumentWrapper;
 import org.opensearch.neuralsearch.ml.MLCommonsClientAccessor;
 
 import lombok.extern.log4j.Log4j2;
@@ -106,4 +112,90 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        try {
+            if (CollectionUtils.isEmpty(ingestDocumentWrappers)) {
+                handler.accept(Collections.emptyList());
+                return;
+            }
+            List<DataForInference> dataForInferences = getDataForInference(ingestDocumentWrappers);
+            if (dataForInferences.isEmpty()) {
+                handler.accept(ingestDocumentWrappers);
+                return;
+            }
+            List<String> inferenceList = constructInferenceTexts(dataForInferences);
+            if (inferenceList.isEmpty()) {
+                handler.accept(ingestDocumentWrappers);
+                return;
+            }
+            // skip existing flag is turned off. Call doSubBatchExecute without filtering
+            if (skipExisting == false) {
+                doSubBatchExecute(ingestDocumentWrappers, inferenceList, dataForInferences, handler);
+                return;
+            }
+            // skipExisting flag is turned on, eligible inference texts in dataForInferences will be compared and filtered after embeddings
+            // are copied
+            openSearchClient.execute(
+                MultiGetAction.INSTANCE,
+                buildMultiGetRequest(ingestDocumentWrappers),
+                ActionListener.wrap(response -> {
+                    try {
+                        MultiGetItemResponse[] multiGetItemResponses = response.getResponses();
+                        if (multiGetItemResponses == null || multiGetItemResponses.length == 0) {
+                            doSubBatchExecute(ingestDocumentWrappers, inferenceList, dataForInferences, handler);
+                            return;
+                        }
+                        // create a map of documents with key: doc_id and value: doc
+                        Map<String, Map<String, Object>> existingDocuments = createDocumentMap(multiGetItemResponses);
+                        List<DataForInference> filteredDataForInference = filterDataForInference(dataForInferences, existingDocuments);
+                        List<String> filteredInferenceList = constructInferenceTexts(filteredDataForInference);
+                        if (filteredInferenceList.isEmpty()) {
+                            handler.accept(ingestDocumentWrappers);
+                        } else {
+                            doSubBatchExecute(ingestDocumentWrappers, filteredInferenceList, filteredDataForInference, handler);
+                        }
+                    } catch (Exception e) {
+                        updateWithExceptions(ingestDocumentWrappers, e);
+                        handler.accept(ingestDocumentWrappers);
+                    }
+                }, e -> {
+                    // When exception is thrown in for MultiGetAction, set exception to all ingestDocumentWrappers
+                    updateWithExceptions(ingestDocumentWrappers, e);
+                    handler.accept(ingestDocumentWrappers);
+                })
+            );
+        } catch (Exception e) {
+            updateWithExceptions(ingestDocumentWrappers, e);
+            handler.accept(ingestDocumentWrappers);
+        }
+    }
+
+    // This is a helper method to filter the given list of dataForInferences by comparing its documents with existingDocuments with
+    // textEmbeddingInferenceFilter
+    private List<DataForInference> filterDataForInference(
+        List<DataForInference> dataForInferences,
+        Map<String, Map<String, Object>> existingDocuments
+    ) {
+        List<DataForInference> filteredDataForInference = new ArrayList<>();
+        for (DataForInference dataForInference : dataForInferences) {
+            IngestDocumentWrapper ingestDocumentWrapper = dataForInference.getIngestDocumentWrapper();
+            Map<String, Object> processMap = dataForInference.getProcessMap();
+            Map<String, Object> document = ingestDocumentWrapper.getIngestDocument().getSourceAndMetadata();
+            Object id = document.get(ID_FIELD);
+            // insert non-filtered dataForInference if existing document does not exist
+            if (Objects.isNull(id) || existingDocuments.containsKey(id.toString()) == false) {
+                filteredDataForInference.add(dataForInference);
+                continue;
+            }
+            // filter dataForInference when existing document exists
+            String docId = id.toString();
+            Map<String, Object> existingDocument = existingDocuments.get(docId);
+            Map<String, Object> filteredProcessMap = textEmbeddingInferenceFilter.filter(existingDocument, document, processMap);
+            List<String> filteredInferenceList = createInferenceList(filteredProcessMap);
+            filteredDataForInference.add(new DataForInference(ingestDocumentWrapper, filteredProcessMap, filteredInferenceList));
+        }
+        return filteredDataForInference;
+    }
 }
diff --git a/src/main/java/org/opensearch/neuralsearch/processor/optimization/InferenceFilter.java b/src/main/java/org/opensearch/neuralsearch/processor/optimization/InferenceFilter.java
index fe5d58b025..01a1615167 100644
--- a/src/main/java/org/opensearch/neuralsearch/processor/optimization/InferenceFilter.java
+++ b/src/main/java/org/opensearch/neuralsearch/processor/optimization/InferenceFilter.java
@@ -13,6 +13,7 @@
 import java.util.List;
 import java.util.ListIterator;
 import java.util.Map;
+import java.util.Objects;
 import java.util.Optional;
 
 /**
@@ -59,8 +60,8 @@ public abstract Object filterInferenceValue(
     );
 
     /**
-     * Abstract helper method to filter individual values based on the existing and new metadata maps.
-     * Implementations should provide logic to compare values and determine if embeddings can be reused.
+     * Abstract helper method to filter individual objects based on the existing and new metadata maps.
+     * Implementations should provide logic to compare objects and determine if embeddings can be reused.
      *
      * @param embeddingKey The dot-notation path for the embedding field
      * @param processValue The value to be checked for potential embedding reuse
@@ -71,7 +72,7 @@ public abstract Object filterInferenceValue(
      * @return The processed value or null if embeddings are reused
      */
 
-    public abstract Object copyEmbeddingForSingleValue(
+    public abstract Object copyEmbeddingForSingleObject(
         String embeddingKey,
         Object processValue,
         Object existingValue,
@@ -81,9 +82,9 @@ public abstract Object copyEmbeddingForSingleValue(
     );
 
     /**
-     * Abstract method to filter and compare lists of values.
-     * If all elements in the list are identical between the new and existing metadata maps, embeddings are copied,
-     * and an empty list is returned to indicate no further processing is required.
+     * Abstract method to filter and compare lists of objects.
+     * If all objects in the list are identical between the new and existing metadata maps, embeddings are copied,
+     * and null is returned to indicate no further processing is required.
      *
      * @param embeddingKey The dot-notation path for the embedding field
      * @param processList The list of values to be checked for potential embedding reuse
@@ -93,7 +94,7 @@ public abstract Object copyEmbeddingForSingleValue(
      * @return A processed list or an empty list if embeddings are reused.
      */
 
-    public abstract List<Object> copyEmbeddingForMultipleValues(
+    public abstract List<Object> copyEmbeddingForListObject(
         String embeddingKey,
         List<Object> processList,
         List<Object> existingList,
@@ -194,7 +195,7 @@ protected List<Object> filterListValue(
         List<Object> existingListValue = ProcessorUtils.unsafeCastToObjectList(existingListOptional.get());
         if (existingListValue.getFirst() instanceof List) {
             // in case of nested list, compare and copy by list comparison
-            return copyEmbeddingForMultipleValues(
+            return copyEmbeddingForListObject(
                 embeddingKey,
                 processList,
                 ProcessorUtils.unsafeCastToObjectList(existingListValue.getFirst()),
@@ -235,16 +236,19 @@ public List<Object> filterMapValuesInList(
         ListIterator<Object> existingListIterator = existingList.listIterator();
         ListIterator<Object> embeddingListIterator = embeddingList.listIterator();
         int index = 0;
-        while (processListIterator.hasNext() && existingListIterator.hasNext() && embeddingListIterator.hasNext()) {
-            Object processedItem = copyEmbeddingForSingleValue(
-                embeddingKey,
-                processListIterator.next(),
-                existingListIterator.next(),
-                embeddingListIterator.next(),
-                sourceAndMetadataMap,
-                index++
-            );
-            filteredList.add(processedItem);
+        for (Object processValue : processList) {
+            if (Objects.nonNull(processValue) && existingListIterator.hasNext() && embeddingListIterator.hasNext()) {
+                Object processedItem = copyEmbeddingForSingleObject(
+                    embeddingKey,
+                    processValue,
+                    existingListIterator.next(),
+                    embeddingListIterator.next(),
+                    sourceAndMetadataMap,
+                    index
+                );
+                filteredList.add(processedItem);
+            }
+            index++;
         }
         return filteredList;
     }
diff --git a/src/main/java/org/opensearch/neuralsearch/processor/optimization/TextEmbeddingInferenceFilter.java b/src/main/java/org/opensearch/neuralsearch/processor/optimization/TextEmbeddingInferenceFilter.java
index 114777ab88..59754610ba 100644
--- a/src/main/java/org/opensearch/neuralsearch/processor/optimization/TextEmbeddingInferenceFilter.java
+++ b/src/main/java/org/opensearch/neuralsearch/processor/optimization/TextEmbeddingInferenceFilter.java
@@ -7,7 +7,6 @@
 import lombok.extern.log4j.Log4j2;
 import org.opensearch.neuralsearch.processor.util.ProcessorUtils;
 
-import java.util.Collections;
 import java.util.List;
 import java.util.Map;
 import java.util.Objects;
@@ -47,7 +46,7 @@ public Object filterInferenceValue(
         Optional<Object> existingValueOptional = ProcessorUtils.getValueFromSource(existingSourceAndMetadataMap, textPath);
         Optional<Object> embeddingValueOptional = ProcessorUtils.getValueFromSource(existingSourceAndMetadataMap, embeddingKey);
         if (existingValueOptional.isPresent() && embeddingValueOptional.isPresent()) {
-            return copyEmbeddingForSingleValue(
+            return copyEmbeddingForSingleObject(
                 embeddingKey,
                 processValue,
                 existingValueOptional.get(),
@@ -67,7 +66,7 @@ public Object filterInferenceValue(
      * @return null if embeddings are reused; the processValue otherwise.
      */
     @Override
-    public Object copyEmbeddingForSingleValue(
+    public Object copyEmbeddingForSingleObject(
         String embeddingKey,
         Object processValue,
         Object existingValue,
@@ -90,7 +89,7 @@ public Object copyEmbeddingForSingleValue(
      * @return empty list if embeddings are reused; processList otherwise.
      */
     @Override
-    public List<Object> copyEmbeddingForMultipleValues(
+    public List<Object> copyEmbeddingForListObject(
         String embeddingKey,
         List<Object> processList,
         List<Object> existingList,
@@ -99,8 +98,8 @@ public List<Object> copyEmbeddingForMultipleValues(
     ) {
         if (Objects.equals(processList, existingList)) {
             ProcessorUtils.setValueToSource(sourceAndMetadataMap, embeddingKey, embeddingList);
-            // if successfully copied, return empty list to be filtered out from process map
-            return Collections.emptyList();
+            // if successfully copied, return null to be filtered out from process map
+            return null;
         }
         // source list and existing list are different, return processList to be included in process map
         return processList;
diff --git a/src/test/java/org/opensearch/neuralsearch/processor/InferenceProcessorTestCase.java b/src/test/java/org/opensearch/neuralsearch/processor/InferenceProcessorTestCase.java
index 1a7276aa06..a7911d3ad6 100644
--- a/src/test/java/org/opensearch/neuralsearch/processor/InferenceProcessorTestCase.java
+++ b/src/test/java/org/opensearch/neuralsearch/processor/InferenceProcessorTestCase.java
@@ -7,6 +7,8 @@
 import com.google.common.collect.ImmutableList;
 import org.apache.commons.lang.math.RandomUtils;
 import org.opensearch.action.get.GetResponse;
+import org.opensearch.action.get.MultiGetItemResponse;
+import org.opensearch.action.get.MultiGetResponse;
 import org.opensearch.common.xcontent.XContentFactory;
 import org.opensearch.core.common.bytes.BytesReference;
 import org.opensearch.core.xcontent.XContentBuilder;
@@ -24,18 +26,22 @@
 import java.util.Map;
 
 public class InferenceProcessorTestCase extends OpenSearchTestCase {
-
-    protected List<IngestDocumentWrapper> createIngestDocumentWrappers(int count) {
+    protected List<IngestDocumentWrapper> createIngestDocumentWrappers(int count, String value) {
         List<IngestDocumentWrapper> wrapperList = new ArrayList<>();
-        for (int i = 0; i < count; ++i) {
+        for (int i = 1; i <= count; ++i) {
             Map<String, Object> sourceAndMetadata = new HashMap<>();
-            sourceAndMetadata.put("key1", "value1");
+            sourceAndMetadata.put("key1", value);
             sourceAndMetadata.put(IndexFieldMapper.NAME, "my_index");
+            sourceAndMetadata.put("_id", String.valueOf(i));
             wrapperList.add(new IngestDocumentWrapper(i, new IngestDocument(sourceAndMetadata, new HashMap<>()), null));
         }
         return wrapperList;
     }
 
+    protected List<IngestDocumentWrapper> createIngestDocumentWrappers(int count) {
+        return createIngestDocumentWrappers(count, "value1");
+    }
+
     protected List<List<Float>> createMockVectorWithLength(int size) {
         float suffix = .234f;
         List<List<Float>> result = new ArrayList<>();
@@ -120,6 +126,10 @@ protected GetResponse mockEmptyGetResponse() throws IOException {
         return GetResponse.fromXContent(contentParser);
     }
 
+    protected MultiGetResponse mockEmptyMultiGetItemResponse() throws IOException {
+        return new MultiGetResponse(new MultiGetItemResponse[0]);
+    }
+
     protected GetResponse convertToGetResponse(IngestDocument ingestDocument) throws IOException {
         String index = ingestDocument.getSourceAndMetadata().get("_index").toString();
         String id = ingestDocument.getSourceAndMetadata().get("_id").toString();
@@ -130,4 +140,12 @@ protected GetResponse convertToGetResponse(IngestDocument ingestDocument) throws
         GetResult result = new GetResult(index, id, 0, 1, 1, true, bytes, null, null);
         return new GetResponse(result);
     }
+
+    protected MultiGetResponse convertToMultiGetItemResponse(List<IngestDocumentWrapper> ingestDocuments) throws IOException {
+        MultiGetItemResponse[] multiGetItems = new MultiGetItemResponse[ingestDocuments.size()];
+        for (int i = 0; i < ingestDocuments.size(); i++) {
+            multiGetItems[i] = new MultiGetItemResponse(convertToGetResponse(ingestDocuments.get(i).getIngestDocument()), null);
+        }
+        return new MultiGetResponse(multiGetItems);
+    }
 }
diff --git a/src/test/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessorIT.java b/src/test/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessorIT.java
index 71c65d3e1f..846c4c73c8 100644
--- a/src/test/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessorIT.java
+++ b/src/test/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessorIT.java
@@ -55,6 +55,7 @@ public class TextEmbeddingProcessorIT extends BaseNeuralSearchIT {
     private final String INGEST_DOC4 = Files.readString(Path.of(classLoader.getResource("processor/ingest_doc4.json").toURI()));
     private final String INGEST_DOC5 = Files.readString(Path.of(classLoader.getResource("processor/ingest_doc5.json").toURI()));
     private final String UPDATE_DOC1 = Files.readString(Path.of(classLoader.getResource("processor/update_doc1.json").toURI()));
+    private final String UPDATE_DOC2 = Files.readString(Path.of(classLoader.getResource("processor/update_doc2.json").toURI()));
     private final String UPDATE_DOC3 = Files.readString(Path.of(classLoader.getResource("processor/update_doc3.json").toURI()));
     private final String UPDATE_DOC4 = Files.readString(Path.of(classLoader.getResource("processor/update_doc4.json").toURI()));
     private final String UPDATE_DOC5 = Files.readString(Path.of(classLoader.getResource("processor/update_doc5.json").toURI()));
@@ -96,7 +97,7 @@ public void testTextEmbeddingProcessor_batch() throws Exception {
         loadModel(modelId);
         createPipelineProcessor(modelId, PIPELINE_NAME, ProcessorType.TEXT_EMBEDDING, 2);
         createIndexWithPipeline(INDEX_NAME, "IndexMappings.json", PIPELINE_NAME);
-        ingestBatchDocumentWithBulk("batch_", 2, Collections.emptySet(), Collections.emptySet());
+        ingestBatchDocumentWithBulk("batch_", 2, Collections.emptySet(), Collections.emptySet(), List.of(INGEST_DOC1, INGEST_DOC2));
         assertEquals(2, getDocCount(INDEX_NAME));
 
         ingestDocument(INDEX_NAME, String.format(LOCALE, INGEST_DOC1, "success"), "1");
@@ -106,6 +107,25 @@ public void testTextEmbeddingProcessor_batch() throws Exception {
         assertEquals(getDocById(INDEX_NAME, "2").get("_source"), getDocById(INDEX_NAME, "batch_2").get("_source"));
     }
 
+    public void testTextEmbeddingProcessor_WithSkipExisting_batchUpdate() throws Exception {
+        String modelId = uploadTextEmbeddingModel();
+        loadModel(modelId);
+        createPipelineProcessor(modelId, PIPELINE_NAME, ProcessorType.TEXT_EMBEDDING_WITH_SKIP_EXISTING, 2);
+        createIndexWithPipeline(INDEX_NAME, "IndexMappings.json", PIPELINE_NAME);
+        ingestBatchDocumentWithBulk("batch_", 2, Collections.emptySet(), Collections.emptySet(), List.of(INGEST_DOC1, INGEST_DOC2));
+        ingestBatchDocumentWithBulk("batch_", 2, Collections.emptySet(), Collections.emptySet(), List.of(UPDATE_DOC1, UPDATE_DOC2));
+
+        assertEquals(2, getDocCount(INDEX_NAME));
+
+        ingestDocument(INDEX_NAME, String.format(LOCALE, INGEST_DOC1, "success"), "1");
+        ingestDocument(INDEX_NAME, String.format(LOCALE, INGEST_DOC2, "success"), "2");
+        updateDocument(INDEX_NAME, String.format(LOCALE, UPDATE_DOC1, "success"), "1");
+        updateDocument(INDEX_NAME, String.format(LOCALE, UPDATE_DOC2, "success"), "2");
+
+        assertEquals(getDocById(INDEX_NAME, "1").get("_source"), getDocById(INDEX_NAME, "batch_1").get("_source"));
+        assertEquals(getDocById(INDEX_NAME, "2").get("_source"), getDocById(INDEX_NAME, "batch_2").get("_source"));
+    }
+
     public void testNestedFieldMapping_whenDocumentsIngested_thenSuccessful() throws Exception {
         String modelId = uploadTextEmbeddingModel();
         loadModel(modelId);
@@ -266,6 +286,41 @@ private void assertDocWithLevel2AsList(Map<String, Object> sourceMap) {
         });
     }
 
+    public void testTextEmbeddingProcessor_updateWithSkipExisting_withBatchSizeInProcessor() throws Exception {
+        String modelId = uploadTextEmbeddingModel();
+        loadModel(modelId);
+        URL pipelineURLPath = classLoader.getResource("processor/PipelineConfigurationWithBatchSizeWithSkipExisting.json");
+        Objects.requireNonNull(pipelineURLPath);
+        String requestBody = Files.readString(Path.of(pipelineURLPath.toURI()));
+        createPipelineProcessor(requestBody, PIPELINE_NAME, modelId, null);
+        createIndexWithPipeline(INDEX_NAME, "IndexMappings.json", PIPELINE_NAME);
+        int docCount = 5;
+        ingestBatchDocumentWithBulk("batch_", docCount, Collections.emptySet(), Collections.emptySet(), List.of(INGEST_DOC1, INGEST_DOC2));
+        ingestBatchDocumentWithBulk("batch_", docCount, Collections.emptySet(), Collections.emptySet(), List.of(UPDATE_DOC1, UPDATE_DOC2));
+
+        assertEquals(5, getDocCount(INDEX_NAME));
+
+        for (int i = 0; i < docCount; ++i) {
+            String template = List.of(INGEST_DOC1, INGEST_DOC2).get(i % 2);
+            String payload = String.format(LOCALE, template, "success");
+            ingestDocument(INDEX_NAME, payload, String.valueOf(i + 1));
+        }
+
+        for (int i = 0; i < docCount; ++i) {
+            String template = List.of(UPDATE_DOC1, UPDATE_DOC2).get(i % 2);
+            String payload = String.format(LOCALE, template, "success");
+            updateDocument(INDEX_NAME, payload, String.valueOf(i + 1));
+        }
+
+        for (int i = 0; i < docCount; ++i) {
+            assertEquals(
+                getDocById(INDEX_NAME, String.valueOf(i + 1)).get("_source"),
+                getDocById(INDEX_NAME, "batch_" + (i + 1)).get("_source")
+            );
+
+        }
+    }
+
     public void testTextEmbeddingProcessor_withBatchSizeInProcessor() throws Exception {
         String modelId = uploadTextEmbeddingModel();
         loadModel(modelId);
@@ -275,7 +330,7 @@ public void testTextEmbeddingProcessor_withBatchSizeInProcessor() throws Excepti
         createPipelineProcessor(requestBody, PIPELINE_NAME, modelId, null);
         createIndexWithPipeline(INDEX_NAME, "IndexMappings.json", PIPELINE_NAME);
         int docCount = 5;
-        ingestBatchDocumentWithBulk("batch_", docCount, Collections.emptySet(), Collections.emptySet());
+        ingestBatchDocumentWithBulk("batch_", docCount, Collections.emptySet(), Collections.emptySet(), List.of(INGEST_DOC1, INGEST_DOC2));
         assertEquals(5, getDocCount(INDEX_NAME));
 
         for (int i = 0; i < docCount; ++i) {
@@ -302,7 +357,7 @@ public void testTextEmbeddingProcessor_withFailureAndSkip() throws Exception {
         createPipelineProcessor(requestBody, PIPELINE_NAME, modelId, null);
         createIndexWithPipeline(INDEX_NAME, "IndexMappings.json", PIPELINE_NAME);
         int docCount = 5;
-        ingestBatchDocumentWithBulk("batch_", docCount, Set.of(0), Set.of(1));
+        ingestBatchDocumentWithBulk("batch_", docCount, Set.of(0), Set.of(1), List.of(INGEST_DOC1, INGEST_DOC2));
         assertEquals(3, getDocCount(INDEX_NAME));
 
         for (int i = 2; i < docCount; ++i) {
@@ -358,11 +413,16 @@ private String uploadTextEmbeddingModel() throws Exception {
         return registerModelGroupAndUploadModel(requestBody);
     }
 
-    private void ingestBatchDocumentWithBulk(String idPrefix, int docCount, Set<Integer> failedIds, Set<Integer> droppedIds)
-        throws Exception {
+    private void ingestBatchDocumentWithBulk(
+        String idPrefix,
+        int docCount,
+        Set<Integer> failedIds,
+        Set<Integer> droppedIds,
+        List<String> docs
+    ) throws Exception {
         StringBuilder payloadBuilder = new StringBuilder();
         for (int i = 0; i < docCount; ++i) {
-            String docTemplate = List.of(INGEST_DOC1, INGEST_DOC2).get(i % 2);
+            String docTemplate = docs.get(i % 2);
             if (failedIds.contains(i)) {
                 docTemplate = String.format(LOCALE, docTemplate, "fail");
             } else if (droppedIds.contains(i)) {
diff --git a/src/test/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessorTests.java b/src/test/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessorTests.java
index e8f2d2a0b9..11bd1a596c 100644
--- a/src/test/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessorTests.java
+++ b/src/test/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessorTests.java
@@ -41,6 +41,9 @@
 import org.opensearch.action.get.GetAction;
 import org.opensearch.action.get.GetRequest;
 import org.opensearch.action.get.GetResponse;
+import org.opensearch.action.get.MultiGetAction;
+import org.opensearch.action.get.MultiGetRequest;
+import org.opensearch.action.get.MultiGetResponse;
 import org.opensearch.cluster.service.ClusterService;
 import org.opensearch.common.settings.Settings;
 import org.opensearch.core.action.ActionListener;
@@ -125,12 +128,13 @@ private TextEmbeddingProcessor createInstanceWithLevel1MapConfig(boolean skipExi
     }
 
     @SneakyThrows
-    private TextEmbeddingProcessor createInstanceWithLevel1MapConfig(int batchSize) {
+    private TextEmbeddingProcessor createInstanceWithLevel1MapConfig(int batchSize, boolean skipExisting) {
         Map<String, Processor.Factory> registry = new HashMap<>();
         Map<String, Object> config = new HashMap<>();
         config.put(TextEmbeddingProcessor.MODEL_ID_FIELD, "mockModelId");
         config.put(TextEmbeddingProcessor.FIELD_MAP_FIELD, ImmutableMap.of("key1", "key1_knn", "key2", "key2_knn"));
         config.put(AbstractBatchingProcessor.BATCH_SIZE_FIELD, batchSize);
+        config.put(TextEmbeddingProcessor.SKIP_EXISTING, skipExisting);
         return (TextEmbeddingProcessor) textEmbeddingProcessorFactory.create(registry, PROCESSOR_TAG, DESCRIPTION, config);
     }
 
@@ -1099,7 +1103,7 @@ public void test_doublyNestedList_withMapType_successful() {
     public void test_batchExecute_successful() {
         final int docCount = 5;
         List<IngestDocumentWrapper> ingestDocumentWrappers = createIngestDocumentWrappers(docCount);
-        TextEmbeddingProcessor processor = createInstanceWithLevel1MapConfig(docCount);
+        TextEmbeddingProcessor processor = createInstanceWithLevel1MapConfig(docCount, false);
 
         List<List<Float>> modelTensorList = createMockVectorWithLength(10);
         doAnswer(invocation -> {
@@ -1122,7 +1126,7 @@ public void test_batchExecute_successful() {
     public void test_batchExecute_exception() {
         final int docCount = 5;
         List<IngestDocumentWrapper> ingestDocumentWrappers = createIngestDocumentWrappers(docCount);
-        TextEmbeddingProcessor processor = createInstanceWithLevel1MapConfig(docCount);
+        TextEmbeddingProcessor processor = createInstanceWithLevel1MapConfig(docCount, false);
         doAnswer(invocation -> {
             ActionListener<List<List<Float>>> listener = invocation.getArgument(1);
             listener.onFailure(new RuntimeException());
@@ -1140,6 +1144,25 @@ public void test_batchExecute_exception() {
         }
     }
 
+    public void test_batchExecute_withSkipExisting_exception() {
+        final int docCount = 5;
+        List<IngestDocumentWrapper> ingestDocumentWrappers = createIngestDocumentWrappers(docCount);
+        List<IngestDocumentWrapper> updateDocumentWrappers = createIngestDocumentWrappers(docCount);
+        TextEmbeddingProcessor processor = createInstanceWithLevel1MapConfig(docCount, true);
+        Consumer resultHandler = mock(Consumer.class);
+        TextInferenceRequest ingestRequest = TextInferenceRequest.builder()
+            .modelId("mockModelID")
+            .inputTexts(List.of("value1", "value1", "value1", "value1", "value1"))
+            .build();
+        mockVectorCreation(ingestRequest, ingestRequest);
+        mockFailedUpdateMultipleDocuments(ingestDocumentWrappers);
+        processor.batchExecute(ingestDocumentWrappers, resultHandler);
+        processor.batchExecute(updateDocumentWrappers, resultHandler);
+        for (IngestDocumentWrapper updateDocumentWrapper : updateDocumentWrappers) {
+            assertNotNull(updateDocumentWrapper.getException());
+        }
+    }
+
     @SneakyThrows
     public void testExecute_when_initial_ingest_with_skip_existing_flag_successful() {
         Map<String, Object> sourceAndMetadata = new HashMap<>();
@@ -2010,6 +2033,71 @@ public void testNestedFieldInMappingMixedSyntax_withMapTypeInput_with_update_suc
         assertEquals(ingestVectors.size(), updateVectors.size());
     }
 
+    public void test_batchExecute_no_update_successful() {
+        final int docCount = 5;
+        List<IngestDocumentWrapper> ingestDocumentWrappers = createIngestDocumentWrappers(docCount);
+        List<IngestDocumentWrapper> updateDocumentWrappers = createIngestDocumentWrappers(docCount);
+        TextEmbeddingProcessor processor = createInstanceWithLevel1MapConfig(docCount, true);
+        Consumer resultHandler = mock(Consumer.class);
+        TextInferenceRequest ingestRequest = TextInferenceRequest.builder()
+            .modelId("mockModelID")
+            .inputTexts(List.of("value1", "value1", "value1", "value1", "value1"))
+            .build();
+        mockVectorCreation(ingestRequest, null);
+        mockUpdateMultipleDocuments(ingestDocumentWrappers);
+        processor.batchExecute(ingestDocumentWrappers, resultHandler);
+        processor.batchExecute(updateDocumentWrappers, resultHandler);
+        for (int i = 0; i < docCount; ++i) {
+            assertEquals(
+                ingestDocumentWrappers.get(i).getIngestDocument().getSourceAndMetadata().get("key1"),
+                updateDocumentWrappers.get(i).getIngestDocument().getSourceAndMetadata().get("key1")
+            );
+            verifyEqualEmbedding(
+                (List<Number>) ingestDocumentWrappers.get(i).getIngestDocument().getSourceAndMetadata().get("key1_knn"),
+                (List<Number>) updateDocumentWrappers.get(i).getIngestDocument().getSourceAndMetadata().get("key1_knn")
+            );
+        }
+        verify(openSearchClient, times(2)).execute(isA(MultiGetAction.class), isA(MultiGetRequest.class), isA(ActionListener.class));
+        verify(mlCommonsClientAccessor, times(1)).inferenceSentences(inferenceRequestCaptor.capture(), isA(ActionListener.class));
+        assertEquals(ingestRequest.getInputTexts(), inferenceRequestCaptor.getValue().getInputTexts());
+    }
+
+    public void test_batchExecute_with_update_successful() {
+        final int docCount = 5;
+        List<IngestDocumentWrapper> ingestDocumentWrappers = createIngestDocumentWrappers(docCount);
+        List<IngestDocumentWrapper> updateDocumentWrappers = createIngestDocumentWrappers(docCount, "newValue");
+        TextEmbeddingProcessor processor = createInstanceWithLevel1MapConfig(docCount, true);
+        Consumer resultHandler = mock(Consumer.class);
+        TextInferenceRequest ingestRequest = TextInferenceRequest.builder()
+            .modelId("mockModelID")
+            .inputTexts(List.of("value1", "value1", "value1", "value1", "value1"))
+            .build();
+        TextInferenceRequest updateRequest = TextInferenceRequest.builder()
+            .modelId("mockModelID")
+            .inputTexts(List.of("newValue", "newValue", "newValue", "newValue", "newValue"))
+            .build();
+        mockVectorCreation(ingestRequest, updateRequest);
+        mockUpdateMultipleDocuments(ingestDocumentWrappers);
+        processor.batchExecute(ingestDocumentWrappers, resultHandler);
+        processor.batchExecute(updateDocumentWrappers, resultHandler);
+        for (int i = 0; i < docCount; ++i) {
+            List<Number> ingestVectors = (List<Number>) ingestDocumentWrappers.get(i)
+                .getIngestDocument()
+                .getSourceAndMetadata()
+                .get("key1_knn");
+            List<Number> updateVectors = (List<Number>) updateDocumentWrappers.get(i)
+                .getIngestDocument()
+                .getSourceAndMetadata()
+                .get("key1_knn");
+            assertEquals(ingestVectors.size(), updateVectors.size());
+        }
+        verify(openSearchClient, times(2)).execute(isA(MultiGetAction.class), isA(MultiGetRequest.class), isA(ActionListener.class));
+        verify(mlCommonsClientAccessor, times(2)).inferenceSentences(inferenceRequestCaptor.capture(), isA(ActionListener.class));
+        List<TextInferenceRequest> requests = inferenceRequestCaptor.getAllValues();
+        assertEquals(ingestRequest.getInputTexts(), requests.get(0).getInputTexts());
+        assertEquals(updateRequest.getInputTexts(), requests.get(1).getInputTexts());
+    }
+
     public void testParsingNestedField_whenNestedFieldsConfigured_thenSuccessful() {
         Map<String, Object> config = createNestedMapConfiguration();
         TextEmbeddingProcessor processor = createInstanceWithNestedMapConfiguration(config);
@@ -2312,6 +2400,30 @@ private void mockUpdateDocument(IngestDocument ingestDocument) {
         }).when(openSearchClient).execute(isA(GetAction.class), isA(GetRequest.class), isA(ActionListener.class));
     }
 
+    private void mockUpdateMultipleDocuments(List<IngestDocumentWrapper> ingestDocuments) {
+        doAnswer(invocation -> {
+            ActionListener<MultiGetResponse> listener = invocation.getArgument(2);
+            listener.onResponse(mockEmptyMultiGetItemResponse()); // returns empty result for ingest action
+            return null;
+        }).doAnswer(invocation -> {
+            ActionListener<MultiGetResponse> listener = invocation.getArgument(2);
+            listener.onResponse(convertToMultiGetItemResponse(ingestDocuments)); // returns previously ingested document for update action
+            return null;
+        }).when(openSearchClient).execute(isA(MultiGetAction.class), isA(MultiGetRequest.class), isA(ActionListener.class));
+    }
+
+    private void mockFailedUpdateMultipleDocuments(List<IngestDocumentWrapper> ingestDocuments) {
+        doAnswer(invocation -> {
+            ActionListener<MultiGetResponse> listener = invocation.getArgument(2);
+            listener.onResponse(mockEmptyMultiGetItemResponse()); // returns empty result for ingest action
+            return null;
+        }).doAnswer(invocation -> {
+            ActionListener<MultiGetResponse> listener = invocation.getArgument(2);
+            listener.onFailure(new RuntimeException()); // throw exception on update
+            return null;
+        }).when(openSearchClient).execute(isA(MultiGetAction.class), isA(MultiGetRequest.class), isA(ActionListener.class));
+    }
+
     private void mockVectorCreation(TextInferenceRequest ingestRequest, TextInferenceRequest updateRequest) {
         doAnswer(invocation -> {
             int numVectors = ingestRequest.getInputTexts().size();
diff --git a/src/test/java/org/opensearch/neuralsearch/processor/optimization/TextEmbeddingInferenceFilterTests.java b/src/test/java/org/opensearch/neuralsearch/processor/optimization/TextEmbeddingInferenceFilterTests.java
index a1de94226c..574bebd496 100644
--- a/src/test/java/org/opensearch/neuralsearch/processor/optimization/TextEmbeddingInferenceFilterTests.java
+++ b/src/test/java/org/opensearch/neuralsearch/processor/optimization/TextEmbeddingInferenceFilterTests.java
@@ -104,7 +104,7 @@ public void test_filterInferenceValuesInList_ListUnchanged_ShouldCopyAllEmbeddin
 
         String fullEmbeddingKey = "embeddingField";
 
-        List<Object> result = textEmbeddingInferenceFilter.copyEmbeddingForMultipleValues(
+        List<Object> result = textEmbeddingInferenceFilter.copyEmbeddingForListObject(
             fullEmbeddingKey,
             processList,
             existingList,
@@ -112,7 +112,7 @@ public void test_filterInferenceValuesInList_ListUnchanged_ShouldCopyAllEmbeddin
             sourceAndMetadataMap
         );
 
-        assertTrue(result.isEmpty());
+        assertNull(result);
         assertEquals(embeddingList, sourceAndMetadataMap.get(fullEmbeddingKey));
     }
 
@@ -123,7 +123,7 @@ public void test_filterInferenceValuesInList_ListPartiallyChanged_ShouldNotCopyE
 
         String fullEmbeddingKey = "embeddingField";
 
-        List<Object> result = textEmbeddingInferenceFilter.copyEmbeddingForMultipleValues(
+        List<Object> result = textEmbeddingInferenceFilter.copyEmbeddingForListObject(
             fullEmbeddingKey,
             processList,
             existingList,
@@ -131,7 +131,7 @@ public void test_filterInferenceValuesInList_ListPartiallyChanged_ShouldNotCopyE
             sourceAndMetadataMap
         );
 
-        assertEquals(processList.size(), result.size());
+        assertEquals(processList, result);
         assertNull(sourceAndMetadataMap.get(fullEmbeddingKey));
     }
 
@@ -139,7 +139,7 @@ public void test_filterInferenceValuesInList_NoMatchingField_ShouldNotCopyEmbedd
         List<Object> processList = Arrays.asList("Text A", "Text B");
         String fullEmbeddingKey = "embeddingField";
 
-        List<Object> result = textEmbeddingInferenceFilter.copyEmbeddingForMultipleValues(
+        List<Object> result = textEmbeddingInferenceFilter.copyEmbeddingForListObject(
             fullEmbeddingKey,
             processList,
             Collections.emptyList(),
@@ -204,7 +204,7 @@ public void test_filter_nestedListValue_Unchanged_ShouldCopyEmbeddings() {
 
         Map<String, Object> result = nestedTextEmbeddingInferenceFilter.filter(existingMap, sourceAndMetadataMap, processMap);
 
-        assertEquals(0, ((List) ((Map) result.get("outerField")).get("embeddingField")).size());
+        assertNull(((Map) result.get("outerField")).get("embeddingField"));
         assertEquals(Arrays.asList(0.1, 0.2, 0.3), ((List) ((Map) sourceAndMetadataMap.get("outerField")).get("embeddingField")).get(0));
         assertEquals(Arrays.asList(0.4, 0.5, 0.6), ((List) ((Map) sourceAndMetadataMap.get("outerField")).get("embeddingField")).get(1));
     }
diff --git a/src/test/resources/processor/PipelineConfigurationWithBatchSizeWithSkipExisting.json b/src/test/resources/processor/PipelineConfigurationWithBatchSizeWithSkipExisting.json
new file mode 100644
index 0000000000..5a89d71e89
--- /dev/null
+++ b/src/test/resources/processor/PipelineConfigurationWithBatchSizeWithSkipExisting.json
@@ -0,0 +1,34 @@
+{
+  "description": "text embedding pipeline for hybrid",
+  "processors": [
+    {
+      "drop": {
+        "if": "ctx.text.contains('drop')"
+      }
+    },
+    {
+      "fail": {
+        "if": "ctx.text.contains('fail')",
+        "message": "fail"
+      }
+    },
+    {
+      "text_embedding": {
+        "model_id": "%s",
+        "batch_size": 2,
+        "field_map": {
+          "title": "title_knn",
+          "favor_list": "favor_list_knn",
+          "favorites": {
+            "game": "game_knn",
+            "movie": "movie_knn"
+          },
+          "nested_passages": {
+            "text": "embedding"
+          }
+        },
+        "skip_existing": true
+      }
+    }
+  ]
+}


Reviews:
user: login: heemin32
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T16:47:45Z
user: login: will-hwang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T17:56:25Z
user: login: will-hwang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T17:58:06Z
user: login: will-hwang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T17:59:25Z
user: login: heemin32
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T18:08:34Z
user: login: heemin32
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T18:10:41Z
user: login: will-hwang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T18:14:49Z
user: login: heemin32
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T18:20:48Z
user: login: will-hwang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T19:41:11Z
user: login: will-hwang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T19:46:57Z
user: login: heemin32
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-11T20:13:04Z
user: login: will-hwang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T22:01:52Z
user: login: will-hwang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-11T22:27:01Z
user: login: vibrantvarun
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-12T16:15:07Z
user: login: will-hwang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-12T18:54:33Z
user: login: will-hwang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-12T20:28:11Z
user: login: will-hwang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-12T21:58:43Z
user: login: heemin32
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-13T23:24:19Z
user: login: will-hwang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-13T23:48:10Z
user: login: heemin32
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-14T00:56:27Z
user: login: heemin32
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-14T00:58:37Z
user: login: heemin32
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-14T00:58:48Z
user: login: vibrantvarun
body: @will-hwang could you add tests in failure cases of this feature? Like when will the functionality fail and user will not see 200 status from api
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-14T17:28:07Z
user: login: will-hwang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T19:18:26Z
user: login: heemin32
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-14T19:28:10Z
user: login: will-hwang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T04:46:42Z
user: login: heemin32
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-17T23:16:51Z
user: login: heemin32
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-17T23:17:19Z
user: login: heemin32
body: LGTM
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-03-18T00:00:18Z
user: login: junqiu-lei
body: LGTM, thanks.
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-18T21:10:30Z

Review comments:
diff_hunk: @@ -214,6 +263,19 @@ public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers,
         });
     }
 
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        super.batchExecute(ingestDocumentWrappers, handler);
+    }
+
+    protected List<List<IngestDocumentWrapper>> cutBatches(List<IngestDocumentWrapper> ingestDocumentWrappers) {
+        List<List<IngestDocumentWrapper>> batches = new ArrayList();
+        for (int i = 0; i < ingestDocumentWrappers.size(); i += this.batchSize) {
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: heemin32
body: This document mention that `batch_size` parameter is deprecated. Should we keep supporting it for 3.0?
created_at: 2025-03-11T16:22:58Z
author_association: COLLABORATOR
diff_hunk: @@ -214,6 +263,19 @@ public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers,
         });
     }
 
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        super.batchExecute(ingestDocumentWrappers, handler);
+    }
+
+    protected List<List<IngestDocumentWrapper>> cutBatches(List<IngestDocumentWrapper> ingestDocumentWrappers) {
+        List<List<IngestDocumentWrapper>> batches = new ArrayList();
+        for (int i = 0; i < ingestDocumentWrappers.size(); i += this.batchSize) {
+            batches.add(ingestDocumentWrappers.subList(i, Math.min(i + this.batchSize, ingestDocumentWrappers.size())));
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: heemin32
body: Using `subList` may not be efficient for certain data structures. For instance, with a `LinkedList`, if `batchSize` is 1, making `n` calls to `subList` would result in an O(n²) time complexity.
created_at: 2025-03-11T16:27:27Z
author_association: COLLABORATOR
diff_hunk: @@ -214,6 +263,19 @@ public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers,
         });
     }
 
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        super.batchExecute(ingestDocumentWrappers, handler);
+    }
+
+    protected List<List<IngestDocumentWrapper>> cutBatches(List<IngestDocumentWrapper> ingestDocumentWrappers) {
+        List<List<IngestDocumentWrapper>> batches = new ArrayList();
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: heemin32
body: If we know the size of this list upfront, better to specify it.
```suggestion
        List<List<IngestDocumentWrapper>> batches = new ArrayList((ingestDocumentWrappers.size() + this.batchSize) / this.batchSize - 1);
```
created_at: 2025-03-11T16:29:01Z
author_association: COLLABORATOR
diff_hunk: @@ -106,4 +111,44 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        // skip existing flag is turned off. Call existing batchExecute without filtering
+        if (skipExisting == false) {
+            super.batchExecute(ingestDocumentWrappers, handler);
+            return;
+        }
+        if (ingestDocumentWrappers.isEmpty()) {
+            handler.accept(Collections.emptyList());
+            return;
+        }
+        // if skipExisting flag is turned on, inference texts in each document will be compared with existing documents fetched via
+        // MultiGet. TextEmbeddingInferenceFilter will be used to filter each inference texts
+        openSearchClient.execute(MultiGetAction.INSTANCE, buildMultiGetRequest(ingestDocumentWrappers), ActionListener.wrap(response -> {
+            MultiGetItemResponse[] multiGetItemResponses = response.getResponses();
+            if (multiGetItemResponses == null || multiGetItemResponses.length == 0) {
+                super.batchExecute(ingestDocumentWrappers, handler);
+                return;
+            }
+            Map<String, Map<String, Object>> existingDocuments = createDocumentMap(multiGetItemResponses);
+            if (this.batchSize >= ingestDocumentWrappers.size()) {
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: heemin32
body: I don't think we need this line of code. This case will be handled by the rest of the code anyway.
created_at: 2025-03-11T16:35:16Z
author_association: COLLABORATOR
diff_hunk: @@ -106,4 +111,44 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        // skip existing flag is turned off. Call existing batchExecute without filtering
+        if (skipExisting == false) {
+            super.batchExecute(ingestDocumentWrappers, handler);
+            return;
+        }
+        if (ingestDocumentWrappers.isEmpty()) {
+            handler.accept(Collections.emptyList());
+            return;
+        }
+        // if skipExisting flag is turned on, inference texts in each document will be compared with existing documents fetched via
+        // MultiGet. TextEmbeddingInferenceFilter will be used to filter each inference texts
+        openSearchClient.execute(MultiGetAction.INSTANCE, buildMultiGetRequest(ingestDocumentWrappers), ActionListener.wrap(response -> {
+            MultiGetItemResponse[] multiGetItemResponses = response.getResponses();
+            if (multiGetItemResponses == null || multiGetItemResponses.length == 0) {
+                super.batchExecute(ingestDocumentWrappers, handler);
+                return;
+            }
+            Map<String, Map<String, Object>> existingDocuments = createDocumentMap(multiGetItemResponses);
+            if (this.batchSize >= ingestDocumentWrappers.size()) {
+                subBatchExecuteWithFilter(ingestDocumentWrappers, existingDocuments, textEmbeddingInferenceFilter, handler);
+                return;
+            }
+            List<List<IngestDocumentWrapper>> batches = cutBatches(ingestDocumentWrappers);
+            int size = ingestDocumentWrappers.size();
+            AtomicInteger counter = new AtomicInteger(size);
+            List<IngestDocumentWrapper> allResults = Collections.synchronizedList(new ArrayList());
+            for (List<IngestDocumentWrapper> batch : batches) {
+                this.subBatchExecuteWithFilter(batch, existingDocuments, textEmbeddingInferenceFilter, (batchResults) -> {
+                    allResults.addAll(batchResults);
+                    if (counter.addAndGet(-batchResults.size()) == 0) {
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: heemin32
body: Can we just compare the size of allResult without using the counter?
```suggestion
                    if (allResults.size == ingestDocumentWrappers.size()) {
```
created_at: 2025-03-11T16:39:58Z
author_association: COLLABORATOR
diff_hunk: @@ -106,4 +111,44 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        // skip existing flag is turned off. Call existing batchExecute without filtering
+        if (skipExisting == false) {
+            super.batchExecute(ingestDocumentWrappers, handler);
+            return;
+        }
+        if (ingestDocumentWrappers.isEmpty()) {
+            handler.accept(Collections.emptyList());
+            return;
+        }
+        // if skipExisting flag is turned on, inference texts in each document will be compared with existing documents fetched via
+        // MultiGet. TextEmbeddingInferenceFilter will be used to filter each inference texts
+        openSearchClient.execute(MultiGetAction.INSTANCE, buildMultiGetRequest(ingestDocumentWrappers), ActionListener.wrap(response -> {
+            MultiGetItemResponse[] multiGetItemResponses = response.getResponses();
+            if (multiGetItemResponses == null || multiGetItemResponses.length == 0) {
+                super.batchExecute(ingestDocumentWrappers, handler);
+                return;
+            }
+            Map<String, Map<String, Object>> existingDocuments = createDocumentMap(multiGetItemResponses);
+            if (this.batchSize >= ingestDocumentWrappers.size()) {
+                subBatchExecuteWithFilter(ingestDocumentWrappers, existingDocuments, textEmbeddingInferenceFilter, handler);
+                return;
+            }
+            List<List<IngestDocumentWrapper>> batches = cutBatches(ingestDocumentWrappers);
+            int size = ingestDocumentWrappers.size();
+            AtomicInteger counter = new AtomicInteger(size);
+            List<IngestDocumentWrapper> allResults = Collections.synchronizedList(new ArrayList());
+            for (List<IngestDocumentWrapper> batch : batches) {
+                this.subBatchExecuteWithFilter(batch, existingDocuments, textEmbeddingInferenceFilter, (batchResults) -> {
+                    allResults.addAll(batchResults);
+                    if (counter.addAndGet(-batchResults.size()) == 0) {
+                        handler.accept(allResults);
+                    }
+                    assert counter.get() >= 0 : "counter is negative";
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: heemin32
body: This assert won't help much. If counter is below zero, the bug should be caught by unit test or integ test. 
created_at: 2025-03-11T16:44:59Z
author_association: COLLABORATOR
diff_hunk: @@ -106,4 +111,44 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        // skip existing flag is turned off. Call existing batchExecute without filtering
+        if (skipExisting == false) {
+            super.batchExecute(ingestDocumentWrappers, handler);
+            return;
+        }
+        if (ingestDocumentWrappers.isEmpty()) {
+            handler.accept(Collections.emptyList());
+            return;
+        }
+        // if skipExisting flag is turned on, inference texts in each document will be compared with existing documents fetched via
+        // MultiGet. TextEmbeddingInferenceFilter will be used to filter each inference texts
+        openSearchClient.execute(MultiGetAction.INSTANCE, buildMultiGetRequest(ingestDocumentWrappers), ActionListener.wrap(response -> {
+            MultiGetItemResponse[] multiGetItemResponses = response.getResponses();
+            if (multiGetItemResponses == null || multiGetItemResponses.length == 0) {
+                super.batchExecute(ingestDocumentWrappers, handler);
+                return;
+            }
+            Map<String, Map<String, Object>> existingDocuments = createDocumentMap(multiGetItemResponses);
+            if (this.batchSize >= ingestDocumentWrappers.size()) {
+                subBatchExecuteWithFilter(ingestDocumentWrappers, existingDocuments, textEmbeddingInferenceFilter, handler);
+                return;
+            }
+            List<List<IngestDocumentWrapper>> batches = cutBatches(ingestDocumentWrappers);
+            int size = ingestDocumentWrappers.size();
+            AtomicInteger counter = new AtomicInteger(size);
+            List<IngestDocumentWrapper> allResults = Collections.synchronizedList(new ArrayList());
+            for (List<IngestDocumentWrapper> batch : batches) {
+                this.subBatchExecuteWithFilter(batch, existingDocuments, textEmbeddingInferenceFilter, (batchResults) -> {
+                    allResults.addAll(batchResults);
+                    if (counter.addAndGet(-batchResults.size()) == 0) {
+                        handler.accept(allResults);
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: heemin32
body: My concern is the case when this handler.accept is not being called because some of thread does not call handler due to some error. Then, we should call exception somewhere here.
created_at: 2025-03-11T16:47:06Z
author_association: COLLABORATOR
diff_hunk: @@ -106,4 +111,44 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        // skip existing flag is turned off. Call existing batchExecute without filtering
+        if (skipExisting == false) {
+            super.batchExecute(ingestDocumentWrappers, handler);
+            return;
+        }
+        if (ingestDocumentWrappers.isEmpty()) {
+            handler.accept(Collections.emptyList());
+            return;
+        }
+        // if skipExisting flag is turned on, inference texts in each document will be compared with existing documents fetched via
+        // MultiGet. TextEmbeddingInferenceFilter will be used to filter each inference texts
+        openSearchClient.execute(MultiGetAction.INSTANCE, buildMultiGetRequest(ingestDocumentWrappers), ActionListener.wrap(response -> {
+            MultiGetItemResponse[] multiGetItemResponses = response.getResponses();
+            if (multiGetItemResponses == null || multiGetItemResponses.length == 0) {
+                super.batchExecute(ingestDocumentWrappers, handler);
+                return;
+            }
+            Map<String, Map<String, Object>> existingDocuments = createDocumentMap(multiGetItemResponses);
+            if (this.batchSize >= ingestDocumentWrappers.size()) {
+                subBatchExecuteWithFilter(ingestDocumentWrappers, existingDocuments, textEmbeddingInferenceFilter, handler);
+                return;
+            }
+            List<List<IngestDocumentWrapper>> batches = cutBatches(ingestDocumentWrappers);
+            int size = ingestDocumentWrappers.size();
+            AtomicInteger counter = new AtomicInteger(size);
+            List<IngestDocumentWrapper> allResults = Collections.synchronizedList(new ArrayList());
+            for (List<IngestDocumentWrapper> batch : batches) {
+                this.subBatchExecuteWithFilter(batch, existingDocuments, textEmbeddingInferenceFilter, (batchResults) -> {
+                    allResults.addAll(batchResults);
+                    if (counter.addAndGet(-batchResults.size()) == 0) {
+                        handler.accept(allResults);
+                    }
+                    assert counter.get() >= 0 : "counter is negative";
+                });
+            }
+        }, e -> { handler.accept(null); }));
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: heemin32
body: If we call handler.accept(null), will the cx see the error?
created_at: 2025-03-11T16:47:40Z
author_association: COLLABORATOR
diff_hunk: @@ -214,6 +263,19 @@ public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers,
         });
     }
 
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        super.batchExecute(ingestDocumentWrappers, handler);
+    }
+
+    protected List<List<IngestDocumentWrapper>> cutBatches(List<IngestDocumentWrapper> ingestDocumentWrappers) {
+        List<List<IngestDocumentWrapper>> batches = new ArrayList();
+        for (int i = 0; i < ingestDocumentWrappers.size(); i += this.batchSize) {
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: will-hwang
body: wanted to keep the changes minimal to the existing implementation of batchExecute in core here: https://github.com/opensearch-project/OpenSearch/blob/main/server/src/main/java/org/opensearch/ingest/AbstractBatchingProcessor.java#L48-L74

Do you think we should completely rewrite the function? 
created_at: 2025-03-11T17:56:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -214,6 +263,19 @@ public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers,
         });
     }
 
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        super.batchExecute(ingestDocumentWrappers, handler);
+    }
+
+    protected List<List<IngestDocumentWrapper>> cutBatches(List<IngestDocumentWrapper> ingestDocumentWrappers) {
+        List<List<IngestDocumentWrapper>> batches = new ArrayList();
+        for (int i = 0; i < ingestDocumentWrappers.size(); i += this.batchSize) {
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: will-hwang
body: since there are conflicting explanation for `batch_size`, i think it's safe to leave it for now
created_at: 2025-03-11T17:58:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -214,6 +263,19 @@ public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers,
         });
     }
 
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        super.batchExecute(ingestDocumentWrappers, handler);
+    }
+
+    protected List<List<IngestDocumentWrapper>> cutBatches(List<IngestDocumentWrapper> ingestDocumentWrappers) {
+        List<List<IngestDocumentWrapper>> batches = new ArrayList();
+        for (int i = 0; i < ingestDocumentWrappers.size(); i += this.batchSize) {
+            batches.add(ingestDocumentWrappers.subList(i, Math.min(i + this.batchSize, ingestDocumentWrappers.size())));
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: will-hwang
body: this is used as a default behavior for batchExecute (https://github.com/opensearch-project/OpenSearch/blob/main/server/src/main/java/org/opensearch/ingest/AbstractBatchingProcessor.java#L48-L74), which we extend in our inferenceProcessor. 
I think it's better to leave the existing logic as is, and make changes to what is needed only. let me know your thoughts. 
created_at: 2025-03-11T17:59:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -214,6 +263,19 @@ public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers,
         });
     }
 
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        super.batchExecute(ingestDocumentWrappers, handler);
+    }
+
+    protected List<List<IngestDocumentWrapper>> cutBatches(List<IngestDocumentWrapper> ingestDocumentWrappers) {
+        List<List<IngestDocumentWrapper>> batches = new ArrayList();
+        for (int i = 0; i < ingestDocumentWrappers.size(); i += this.batchSize) {
+            batches.add(ingestDocumentWrappers.subList(i, Math.min(i + this.batchSize, ingestDocumentWrappers.size())));
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: heemin32
body: Got it. Can't we just override `subBatchExecute` instead of repeating the core code here?
created_at: 2025-03-11T18:08:34Z
author_association: COLLABORATOR
diff_hunk: @@ -106,4 +111,44 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        // skip existing flag is turned off. Call existing batchExecute without filtering
+        if (skipExisting == false) {
+            super.batchExecute(ingestDocumentWrappers, handler);
+            return;
+        }
+        if (ingestDocumentWrappers.isEmpty()) {
+            handler.accept(Collections.emptyList());
+            return;
+        }
+        // if skipExisting flag is turned on, inference texts in each document will be compared with existing documents fetched via
+        // MultiGet. TextEmbeddingInferenceFilter will be used to filter each inference texts
+        openSearchClient.execute(MultiGetAction.INSTANCE, buildMultiGetRequest(ingestDocumentWrappers), ActionListener.wrap(response -> {
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: heemin32
body: I believe `mget` should be performed with a `batch_size`, allowing customers to adjust it if the request fails due to requesting too many documents at once.
created_at: 2025-03-11T18:10:41Z
author_association: COLLABORATOR
diff_hunk: @@ -214,6 +263,19 @@ public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers,
         });
     }
 
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        super.batchExecute(ingestDocumentWrappers, handler);
+    }
+
+    protected List<List<IngestDocumentWrapper>> cutBatches(List<IngestDocumentWrapper> ingestDocumentWrappers) {
+        List<List<IngestDocumentWrapper>> batches = new ArrayList();
+        for (int i = 0; i < ingestDocumentWrappers.size(); i += this.batchSize) {
+            batches.add(ingestDocumentWrappers.subList(i, Math.min(i + this.batchSize, ingestDocumentWrappers.size())));
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: will-hwang
body: if we just override `subBatchExecute`, we won't be able to use `multiGet` operation to fetch all documents at once, since `subBatchExecute` only contains a subset (mostly just one) of the documents. 
created_at: 2025-03-11T18:14:49Z
author_association: CONTRIBUTOR
diff_hunk: @@ -214,6 +263,19 @@ public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers,
         });
     }
 
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        super.batchExecute(ingestDocumentWrappers, handler);
+    }
+
+    protected List<List<IngestDocumentWrapper>> cutBatches(List<IngestDocumentWrapper> ingestDocumentWrappers) {
+        List<List<IngestDocumentWrapper>> batches = new ArrayList();
+        for (int i = 0; i < ingestDocumentWrappers.size(); i += this.batchSize) {
+            batches.add(ingestDocumentWrappers.subList(i, Math.min(i + this.batchSize, ingestDocumentWrappers.size())));
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: heemin32
body: That is what we want. We should not get all documents at once. It might fail if the number of document is too big. Customer can increase batch_size if they want.
created_at: 2025-03-11T18:20:48Z
author_association: COLLABORATOR
diff_hunk: @@ -214,6 +263,19 @@ public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers,
         });
     }
 
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        super.batchExecute(ingestDocumentWrappers, handler);
+    }
+
+    protected List<List<IngestDocumentWrapper>> cutBatches(List<IngestDocumentWrapper> ingestDocumentWrappers) {
+        List<List<IngestDocumentWrapper>> batches = new ArrayList();
+        for (int i = 0; i < ingestDocumentWrappers.size(); i += this.batchSize) {
+            batches.add(ingestDocumentWrappers.subList(i, Math.min(i + this.batchSize, ingestDocumentWrappers.size())));
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: will-hwang
body: okay, with this change we would be calling multiple `multiGet` operation, since we are calling it at subBatch level. 

For example, If user ingests 6 documents, with a batch size of 2,

there will be 3 multiGet calls with each batch size of 2. 
created_at: 2025-03-11T19:41:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -106,4 +111,44 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        // skip existing flag is turned off. Call existing batchExecute without filtering
+        if (skipExisting == false) {
+            super.batchExecute(ingestDocumentWrappers, handler);
+            return;
+        }
+        if (ingestDocumentWrappers.isEmpty()) {
+            handler.accept(Collections.emptyList());
+            return;
+        }
+        // if skipExisting flag is turned on, inference texts in each document will be compared with existing documents fetched via
+        // MultiGet. TextEmbeddingInferenceFilter will be used to filter each inference texts
+        openSearchClient.execute(MultiGetAction.INSTANCE, buildMultiGetRequest(ingestDocumentWrappers), ActionListener.wrap(response -> {
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: will-hwang
body: understood, thanks
created_at: 2025-03-11T19:46:57Z
author_association: CONTRIBUTOR
diff_hunk: @@ -214,6 +263,19 @@ public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers,
         });
     }
 
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        super.batchExecute(ingestDocumentWrappers, handler);
+    }
+
+    protected List<List<IngestDocumentWrapper>> cutBatches(List<IngestDocumentWrapper> ingestDocumentWrappers) {
+        List<List<IngestDocumentWrapper>> batches = new ArrayList();
+        for (int i = 0; i < ingestDocumentWrappers.size(); i += this.batchSize) {
+            batches.add(ingestDocumentWrappers.subList(i, Math.min(i + this.batchSize, ingestDocumentWrappers.size())));
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: heemin32
body: Yes.
created_at: 2025-03-11T20:13:04Z
author_association: COLLABORATOR
diff_hunk: @@ -106,4 +111,44 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        // skip existing flag is turned off. Call existing batchExecute without filtering
+        if (skipExisting == false) {
+            super.batchExecute(ingestDocumentWrappers, handler);
+            return;
+        }
+        if (ingestDocumentWrappers.isEmpty()) {
+            handler.accept(Collections.emptyList());
+            return;
+        }
+        // if skipExisting flag is turned on, inference texts in each document will be compared with existing documents fetched via
+        // MultiGet. TextEmbeddingInferenceFilter will be used to filter each inference texts
+        openSearchClient.execute(MultiGetAction.INSTANCE, buildMultiGetRequest(ingestDocumentWrappers), ActionListener.wrap(response -> {
+            MultiGetItemResponse[] multiGetItemResponses = response.getResponses();
+            if (multiGetItemResponses == null || multiGetItemResponses.length == 0) {
+                super.batchExecute(ingestDocumentWrappers, handler);
+                return;
+            }
+            Map<String, Map<String, Object>> existingDocuments = createDocumentMap(multiGetItemResponses);
+            if (this.batchSize >= ingestDocumentWrappers.size()) {
+                subBatchExecuteWithFilter(ingestDocumentWrappers, existingDocuments, textEmbeddingInferenceFilter, handler);
+                return;
+            }
+            List<List<IngestDocumentWrapper>> batches = cutBatches(ingestDocumentWrappers);
+            int size = ingestDocumentWrappers.size();
+            AtomicInteger counter = new AtomicInteger(size);
+            List<IngestDocumentWrapper> allResults = Collections.synchronizedList(new ArrayList());
+            for (List<IngestDocumentWrapper> batch : batches) {
+                this.subBatchExecuteWithFilter(batch, existingDocuments, textEmbeddingInferenceFilter, (batchResults) -> {
+                    allResults.addAll(batchResults);
+                    if (counter.addAndGet(-batchResults.size()) == 0) {
+                        handler.accept(allResults);
+                    }
+                    assert counter.get() >= 0 : "counter is negative";
+                });
+            }
+        }, e -> { handler.accept(null); }));
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: will-hwang
body: no, will update the PR to include exception in each document
created_at: 2025-03-11T22:01:52Z
author_association: CONTRIBUTOR
diff_hunk: @@ -106,4 +111,44 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        // skip existing flag is turned off. Call existing batchExecute without filtering
+        if (skipExisting == false) {
+            super.batchExecute(ingestDocumentWrappers, handler);
+            return;
+        }
+        if (ingestDocumentWrappers.isEmpty()) {
+            handler.accept(Collections.emptyList());
+            return;
+        }
+        // if skipExisting flag is turned on, inference texts in each document will be compared with existing documents fetched via
+        // MultiGet. TextEmbeddingInferenceFilter will be used to filter each inference texts
+        openSearchClient.execute(MultiGetAction.INSTANCE, buildMultiGetRequest(ingestDocumentWrappers), ActionListener.wrap(response -> {
+            MultiGetItemResponse[] multiGetItemResponses = response.getResponses();
+            if (multiGetItemResponses == null || multiGetItemResponses.length == 0) {
+                super.batchExecute(ingestDocumentWrappers, handler);
+                return;
+            }
+            Map<String, Map<String, Object>> existingDocuments = createDocumentMap(multiGetItemResponses);
+            if (this.batchSize >= ingestDocumentWrappers.size()) {
+                subBatchExecuteWithFilter(ingestDocumentWrappers, existingDocuments, textEmbeddingInferenceFilter, handler);
+                return;
+            }
+            List<List<IngestDocumentWrapper>> batches = cutBatches(ingestDocumentWrappers);
+            int size = ingestDocumentWrappers.size();
+            AtomicInteger counter = new AtomicInteger(size);
+            List<IngestDocumentWrapper> allResults = Collections.synchronizedList(new ArrayList());
+            for (List<IngestDocumentWrapper> batch : batches) {
+                this.subBatchExecuteWithFilter(batch, existingDocuments, textEmbeddingInferenceFilter, (batchResults) -> {
+                    allResults.addAll(batchResults);
+                    if (counter.addAndGet(-batchResults.size()) == 0) {
+                        handler.accept(allResults);
+                    }
+                    assert counter.get() >= 0 : "counter is negative";
+                });
+            }
+        }, e -> { handler.accept(null); }));
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: will-hwang
body: with the latest change, when `mget` throws an exception, handler will still accept the ingested document, with exception message populated in each document
created_at: 2025-03-11T22:27:01Z
author_association: CONTRIBUTOR
diff_hunk: @@ -106,4 +112,60 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: vibrantvarun
body: Could you divide this method into three parts? 
1. Validations
2. MGet call
3. Checking for embeddings
created_at: 2025-03-12T16:12:55Z
author_association: MEMBER
diff_hunk: @@ -182,6 +186,15 @@ public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers,
             handler.accept(ingestDocumentWrappers);
             return;
         }
+        doSubBatchExecute(ingestDocumentWrappers, inferenceList, dataForInferences, handler);
+    }
+
+    protected void doSubBatchExecute(
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: vibrantvarun
body: Add javadoc to all the methods in this class.
created_at: 2025-03-12T16:13:32Z
author_association: MEMBER
diff_hunk: @@ -106,4 +111,44 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        // skip existing flag is turned off. Call existing batchExecute without filtering
+        if (skipExisting == false) {
+            super.batchExecute(ingestDocumentWrappers, handler);
+            return;
+        }
+        if (ingestDocumentWrappers.isEmpty()) {
+            handler.accept(Collections.emptyList());
+            return;
+        }
+        // if skipExisting flag is turned on, inference texts in each document will be compared with existing documents fetched via
+        // MultiGet. TextEmbeddingInferenceFilter will be used to filter each inference texts
+        openSearchClient.execute(MultiGetAction.INSTANCE, buildMultiGetRequest(ingestDocumentWrappers), ActionListener.wrap(response -> {
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: vibrantvarun
body: +1 to @heemin32 
created_at: 2025-03-12T16:14:02Z
author_association: MEMBER
diff_hunk: @@ -47,7 +45,7 @@ public Object filterInferenceValue(
         Optional<Object> existingValueOptional = ProcessorUtils.getValueFromSource(existingSourceAndMetadataMap, textPath);
         Optional<Object> embeddingValueOptional = ProcessorUtils.getValueFromSource(existingSourceAndMetadataMap, embeddingKey);
         if (existingValueOptional.isPresent() && embeddingValueOptional.isPresent()) {
-            return copyEmbeddingForSingleValue(
path: src/main/java/org/opensearch/neuralsearch/processor/optimization/TextEmbeddingInferenceFilter.java
user: login: vibrantvarun
body: QQ: why we change this method here?
created_at: 2025-03-12T16:15:01Z
author_association: MEMBER
diff_hunk: @@ -47,7 +45,7 @@ public Object filterInferenceValue(
         Optional<Object> existingValueOptional = ProcessorUtils.getValueFromSource(existingSourceAndMetadataMap, textPath);
         Optional<Object> embeddingValueOptional = ProcessorUtils.getValueFromSource(existingSourceAndMetadataMap, embeddingKey);
         if (existingValueOptional.isPresent() && embeddingValueOptional.isPresent()) {
-            return copyEmbeddingForSingleValue(
path: src/main/java/org/opensearch/neuralsearch/processor/optimization/TextEmbeddingInferenceFilter.java
user: login: will-hwang
body: this was a change from previous PR. Since we're comparing single value and list values as object values, we can have just one method for comparison and copying embeddings
created_at: 2025-03-12T18:54:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -106,4 +111,44 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void batchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        // skip existing flag is turned off. Call existing batchExecute without filtering
+        if (skipExisting == false) {
+            super.batchExecute(ingestDocumentWrappers, handler);
+            return;
+        }
+        if (ingestDocumentWrappers.isEmpty()) {
+            handler.accept(Collections.emptyList());
+            return;
+        }
+        // if skipExisting flag is turned on, inference texts in each document will be compared with existing documents fetched via
+        // MultiGet. TextEmbeddingInferenceFilter will be used to filter each inference texts
+        openSearchClient.execute(MultiGetAction.INSTANCE, buildMultiGetRequest(ingestDocumentWrappers), ActionListener.wrap(response -> {
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: will-hwang
body: the latest change includes this change as mget is performed under `subBatchExecute` method, which has a size of `batch_size`
created_at: 2025-03-12T20:28:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -106,4 +112,60 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: will-hwang
body: i refactored the method and added comments to make it more readable. Left the validations at the beginning of the method for cleaner implementation
created_at: 2025-03-12T21:58:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -415,6 +440,36 @@ protected void setVectorFieldsToDocument(IngestDocument ingestDocument, Map<Stri
         nlpResult.forEach(ingestDocument::setFieldValue);
     }
 
+    /**
+     * This method creates a MultiGetRequest from a list of ingest documents to be fetched for comparison
+     * @param ingestDocumentWrappers, list of ingest documents
+     * */
+    protected MultiGetRequest buildMultiGetRequest(List<IngestDocumentWrapper> ingestDocumentWrappers) {
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: heemin32
body: Better to move these method out of this class then it will be much easier to write unit test.
created_at: 2025-03-13T23:15:15Z
author_association: COLLABORATOR
diff_hunk: @@ -106,4 +112,74 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        if (CollectionUtils.isEmpty(ingestDocumentWrappers)) {
+            handler.accept(Collections.emptyList());
+            return;
+        }
+        List<DataForInference> dataForInferences = getDataForInference(ingestDocumentWrappers);
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: heemin32
body: What happen if this line throw exception? Shouldn't we catch it and set it in handler or will it be handled already?
created_at: 2025-03-13T23:19:18Z
author_association: COLLABORATOR
diff_hunk: @@ -106,4 +112,74 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        if (CollectionUtils.isEmpty(ingestDocumentWrappers)) {
+            handler.accept(Collections.emptyList());
+            return;
+        }
+        List<DataForInference> dataForInferences = getDataForInference(ingestDocumentWrappers);
+        List<String> inferenceList = constructInferenceTexts(dataForInferences);
+        if (inferenceList.isEmpty()) {
+            handler.accept(ingestDocumentWrappers);
+            return;
+        }
+        // skip existing flag is turned off. Call doSubBatchExecute without filtering
+        if (skipExisting == false) {
+            doSubBatchExecute(ingestDocumentWrappers, inferenceList, dataForInferences, handler);
+            return;
+        }
+        // skipExisting flag is turned on, eligible inference texts in dataForInferences will be compared and filtered after embeddings are
+        // copied
+        openSearchClient.execute(MultiGetAction.INSTANCE, buildMultiGetRequest(ingestDocumentWrappers), ActionListener.wrap(response -> {
+            MultiGetItemResponse[] multiGetItemResponses = response.getResponses();
+            if (multiGetItemResponses == null || multiGetItemResponses.length == 0) {
+                doSubBatchExecute(ingestDocumentWrappers, inferenceList, dataForInferences, handler);
+                return;
+            }
+            // create a map of documents with key: doc_id and value: doc
+            Map<String, Map<String, Object>> existingDocuments = createDocumentMap(multiGetItemResponses);
+            List<DataForInference> filteredDataForInference = filterDataForInference(dataForInferences, existingDocuments);
+            List<String> filteredInferenceList = constructInferenceTexts(filteredDataForInference);
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: heemin32
body: What happen if exception is thrown here. Will that be handled gracefully?
created_at: 2025-03-13T23:19:38Z
author_association: COLLABORATOR
diff_hunk: @@ -194,13 +174,15 @@ protected List<Object> filterListValue(
         List<Object> existingListValue = ProcessorUtils.unsafeCastToObjectList(existingListOptional.get());
         if (existingListValue.getFirst() instanceof List) {
             // in case of nested list, compare and copy by list comparison
-            return copyEmbeddingForMultipleValues(
+            Object processedList = copyEmbedding(
                 embeddingKey,
                 processList,
-                ProcessorUtils.unsafeCastToObjectList(existingListValue.getFirst()),
-                ProcessorUtils.unsafeCastToObjectList(embeddingListOptional.get()),
-                sourceAndMetadataMap
+                existingListValue.getFirst(),
+                embeddingListOptional.get(),
+                sourceAndMetadataMap,
+                -1
path: src/main/java/org/opensearch/neuralsearch/processor/optimization/InferenceFilter.java
user: login: heemin32
body: This is a bit unclear—I’m not sure when to pass -1 versus another value. It would be helpful to have a separate method for clarity and include a detailed Javadoc explaining when to use each method.
```
// Use when xys 
public Object copyEmbeddingXYZ(
        String embeddingKey,
        Object processValue,
        Object existingValue,
        Object embeddingValue,
        Map<String, Object> sourceAndMetadataMap
    ) {
  return copyEmbedding(embeddingKey, processValue, existingValue, embeddingValue, sourceAndMetadataMap, -1);
}
```

created_at: 2025-03-13T23:24:15Z
author_association: COLLABORATOR
diff_hunk: @@ -415,6 +440,36 @@ protected void setVectorFieldsToDocument(IngestDocument ingestDocument, Map<Stri
         nlpResult.forEach(ingestDocument::setFieldValue);
     }
 
+    /**
+     * This method creates a MultiGetRequest from a list of ingest documents to be fetched for comparison
+     * @param ingestDocumentWrappers, list of ingest documents
+     * */
+    protected MultiGetRequest buildMultiGetRequest(List<IngestDocumentWrapper> ingestDocumentWrappers) {
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: will-hwang
body: okay, i can move these methods to [ProcessorUtil](https://github.com/opensearch-project/neural-search/blob/main/src/main/java/org/opensearch/neuralsearch/processor/util/ProcessorUtils.java) class. Let me know if you have better suggestions. 
created_at: 2025-03-13T23:43:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -106,4 +112,74 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        if (CollectionUtils.isEmpty(ingestDocumentWrappers)) {
+            handler.accept(Collections.emptyList());
+            return;
+        }
+        List<DataForInference> dataForInferences = getDataForInference(ingestDocumentWrappers);
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: will-hwang
body: it is already handled in the method: [here](https://github.com/opensearch-project/neural-search/blob/main/src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java#L264)
created_at: 2025-03-13T23:44:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -106,4 +112,74 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        if (CollectionUtils.isEmpty(ingestDocumentWrappers)) {
+            handler.accept(Collections.emptyList());
+            return;
+        }
+        List<DataForInference> dataForInferences = getDataForInference(ingestDocumentWrappers);
+        List<String> inferenceList = constructInferenceTexts(dataForInferences);
+        if (inferenceList.isEmpty()) {
+            handler.accept(ingestDocumentWrappers);
+            return;
+        }
+        // skip existing flag is turned off. Call doSubBatchExecute without filtering
+        if (skipExisting == false) {
+            doSubBatchExecute(ingestDocumentWrappers, inferenceList, dataForInferences, handler);
+            return;
+        }
+        // skipExisting flag is turned on, eligible inference texts in dataForInferences will be compared and filtered after embeddings are
+        // copied
+        openSearchClient.execute(MultiGetAction.INSTANCE, buildMultiGetRequest(ingestDocumentWrappers), ActionListener.wrap(response -> {
+            MultiGetItemResponse[] multiGetItemResponses = response.getResponses();
+            if (multiGetItemResponses == null || multiGetItemResponses.length == 0) {
+                doSubBatchExecute(ingestDocumentWrappers, inferenceList, dataForInferences, handler);
+                return;
+            }
+            // create a map of documents with key: doc_id and value: doc
+            Map<String, Map<String, Object>> existingDocuments = createDocumentMap(multiGetItemResponses);
+            List<DataForInference> filteredDataForInference = filterDataForInference(dataForInferences, existingDocuments);
+            List<String> filteredInferenceList = constructInferenceTexts(filteredDataForInference);
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: will-hwang
body: it checks for whether exception was thrown and only create inference text for valid data inferences [here](https://github.com/opensearch-project/neural-search/blob/main/src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java#L240-L249)
created_at: 2025-03-13T23:46:35Z
author_association: CONTRIBUTOR
diff_hunk: @@ -194,13 +174,15 @@ protected List<Object> filterListValue(
         List<Object> existingListValue = ProcessorUtils.unsafeCastToObjectList(existingListOptional.get());
         if (existingListValue.getFirst() instanceof List) {
             // in case of nested list, compare and copy by list comparison
-            return copyEmbeddingForMultipleValues(
+            Object processedList = copyEmbedding(
                 embeddingKey,
                 processList,
-                ProcessorUtils.unsafeCastToObjectList(existingListValue.getFirst()),
-                ProcessorUtils.unsafeCastToObjectList(embeddingListOptional.get()),
-                sourceAndMetadataMap
+                existingListValue.getFirst(),
+                embeddingListOptional.get(),
+                sourceAndMetadataMap,
+                -1
path: src/main/java/org/opensearch/neuralsearch/processor/optimization/InferenceFilter.java
user: login: will-hwang
body: okay, will create a separate method
created_at: 2025-03-13T23:47:50Z
author_association: CONTRIBUTOR
diff_hunk: @@ -415,6 +440,36 @@ protected void setVectorFieldsToDocument(IngestDocument ingestDocument, Map<Stri
         nlpResult.forEach(ingestDocument::setFieldValue);
     }
 
+    /**
+     * This method creates a MultiGetRequest from a list of ingest documents to be fetched for comparison
+     * @param ingestDocumentWrappers, list of ingest documents
+     * */
+    protected MultiGetRequest buildMultiGetRequest(List<IngestDocumentWrapper> ingestDocumentWrappers) {
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: heemin32
body: Be careful not to make is as static. If it become static method then no difference with current one.
created_at: 2025-03-14T00:56:27Z
author_association: COLLABORATOR
diff_hunk: @@ -106,4 +112,74 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        if (CollectionUtils.isEmpty(ingestDocumentWrappers)) {
+            handler.accept(Collections.emptyList());
+            return;
+        }
+        List<DataForInference> dataForInferences = getDataForInference(ingestDocumentWrappers);
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: heemin32
body: That's the issue—you need to check every method to ensure exceptions are handled properly, which isn't a future-proof approach. It would be better to handle the exception here as well, ensuring that any exceptions thrown by underlying methods are caught.
created_at: 2025-03-14T00:58:37Z
author_association: COLLABORATOR
diff_hunk: @@ -106,4 +112,74 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        if (CollectionUtils.isEmpty(ingestDocumentWrappers)) {
+            handler.accept(Collections.emptyList());
+            return;
+        }
+        List<DataForInference> dataForInferences = getDataForInference(ingestDocumentWrappers);
+        List<String> inferenceList = constructInferenceTexts(dataForInferences);
+        if (inferenceList.isEmpty()) {
+            handler.accept(ingestDocumentWrappers);
+            return;
+        }
+        // skip existing flag is turned off. Call doSubBatchExecute without filtering
+        if (skipExisting == false) {
+            doSubBatchExecute(ingestDocumentWrappers, inferenceList, dataForInferences, handler);
+            return;
+        }
+        // skipExisting flag is turned on, eligible inference texts in dataForInferences will be compared and filtered after embeddings are
+        // copied
+        openSearchClient.execute(MultiGetAction.INSTANCE, buildMultiGetRequest(ingestDocumentWrappers), ActionListener.wrap(response -> {
+            MultiGetItemResponse[] multiGetItemResponses = response.getResponses();
+            if (multiGetItemResponses == null || multiGetItemResponses.length == 0) {
+                doSubBatchExecute(ingestDocumentWrappers, inferenceList, dataForInferences, handler);
+                return;
+            }
+            // create a map of documents with key: doc_id and value: doc
+            Map<String, Map<String, Object>> existingDocuments = createDocumentMap(multiGetItemResponses);
+            List<DataForInference> filteredDataForInference = filterDataForInference(dataForInferences, existingDocuments);
+            List<String> filteredInferenceList = constructInferenceTexts(filteredDataForInference);
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: heemin32
body: Same as above comment.
created_at: 2025-03-14T00:58:48Z
author_association: COLLABORATOR
diff_hunk: @@ -415,6 +440,36 @@ protected void setVectorFieldsToDocument(IngestDocument ingestDocument, Map<Stri
         nlpResult.forEach(ingestDocument::setFieldValue);
     }
 
+    /**
+     * This method creates a MultiGetRequest from a list of ingest documents to be fetched for comparison
+     * @param ingestDocumentWrappers, list of ingest documents
+     * */
+    protected MultiGetRequest buildMultiGetRequest(List<IngestDocumentWrapper> ingestDocumentWrappers) {
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: will-hwang
body: If we need this method separated from this class without using static, we would need to instantiate or inject a class into `TextEmbeddingProcessor`. Do we prefer this approach over defining the methods here?
created_at: 2025-03-14T19:18:26Z
author_association: CONTRIBUTOR
diff_hunk: @@ -415,6 +440,36 @@ protected void setVectorFieldsToDocument(IngestDocument ingestDocument, Map<Stri
         nlpResult.forEach(ingestDocument::setFieldValue);
     }
 
+    /**
+     * This method creates a MultiGetRequest from a list of ingest documents to be fetched for comparison
+     * @param ingestDocumentWrappers, list of ingest documents
+     * */
+    protected MultiGetRequest buildMultiGetRequest(List<IngestDocumentWrapper> ingestDocumentWrappers) {
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: heemin32
body: Yes, that is generally the preferred approach, but it's important to ensure that an instance is not created for every request. This approach will also significantly simplify writing unit tests. If you think that is too much then current approach is fine though.
created_at: 2025-03-14T19:28:10Z
author_association: COLLABORATOR
diff_hunk: @@ -415,6 +440,36 @@ protected void setVectorFieldsToDocument(IngestDocument ingestDocument, Map<Stri
         nlpResult.forEach(ingestDocument::setFieldValue);
     }
 
+    /**
+     * This method creates a MultiGetRequest from a list of ingest documents to be fetched for comparison
+     * @param ingestDocumentWrappers, list of ingest documents
+     * */
+    protected MultiGetRequest buildMultiGetRequest(List<IngestDocumentWrapper> ingestDocumentWrappers) {
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: will-hwang
body: If it's fine, will leave the methods in InferenceProcessor since it already has similar helper methods for processing inferences, and will make child class extension simpler. 
created_at: 2025-03-17T04:46:42Z
author_association: CONTRIBUTOR
diff_hunk: @@ -169,49 +173,80 @@ void preprocessIngestDocument(IngestDocument ingestDocument) {
      */
     abstract void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler, Consumer<Exception> onException);
 
+    /**
+     * This is the function which does actual inference work for subBatchExecute interface.
+     * @param ingestDocumentWrappers a list of IngestDocuments in a batch.
+     * @param handler a callback handler to handle inference results which is a list of objects.
+     */
     @Override
     public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
-        if (CollectionUtils.isEmpty(ingestDocumentWrappers)) {
-            handler.accept(Collections.emptyList());
-            return;
-        }
+        try {
+            if (CollectionUtils.isEmpty(ingestDocumentWrappers)) {
+                handler.accept(Collections.emptyList());
+                return;
+            }
 
-        List<DataForInference> dataForInferences = getDataForInference(ingestDocumentWrappers);
-        List<String> inferenceList = constructInferenceTexts(dataForInferences);
-        if (inferenceList.isEmpty()) {
+            List<DataForInference> dataForInferences = getDataForInference(ingestDocumentWrappers);
+            List<String> inferenceList = constructInferenceTexts(dataForInferences);
+            if (inferenceList.isEmpty()) {
+                handler.accept(ingestDocumentWrappers);
+                return;
+            }
+            doSubBatchExecute(ingestDocumentWrappers, inferenceList, dataForInferences, handler);
+        } catch (Exception e) {
+            updateWithExceptions(ingestDocumentWrappers, e);
             handler.accept(ingestDocumentWrappers);
-            return;
         }
-        Tuple<List<String>, Map<Integer, Integer>> sortedResult = sortByLengthAndReturnOriginalOrder(inferenceList);
-        inferenceList = sortedResult.v1();
-        Map<Integer, Integer> originalOrder = sortedResult.v2();
-        doBatchExecute(inferenceList, results -> {
-            int startIndex = 0;
-            results = restoreToOriginalOrder(results, originalOrder);
-            for (DataForInference dataForInference : dataForInferences) {
-                if (dataForInference.getIngestDocumentWrapper().getException() != null
-                    || CollectionUtils.isEmpty(dataForInference.getInferenceList())) {
-                    continue;
+    }
+
+    /**
+     * This is a helper function for subBatchExecute, which invokes doBatchExecute for given inference list.
+     * @param ingestDocumentWrappers a list of IngestDocuments in a batch.
+     * @param inferenceList a list of String for inference.
+     * @param dataForInferences a list of data for inference, which includes ingestDocumentWrapper, processMap, inferenceList.
+     * @param handler a callback handler to handle inference results which is a list of objects.
+     */
+    protected void doSubBatchExecute(
+        List<IngestDocumentWrapper> ingestDocumentWrappers,
+        List<String> inferenceList,
+        List<DataForInference> dataForInferences,
+        Consumer<List<IngestDocumentWrapper>> handler
+    ) {
+        try {
+            Tuple<List<String>, Map<Integer, Integer>> sortedResult = sortByLengthAndReturnOriginalOrder(inferenceList);
+            inferenceList = sortedResult.v1();
+            Map<Integer, Integer> originalOrder = sortedResult.v2();
+            doBatchExecute(inferenceList, results -> {
+                int startIndex = 0;
path: src/main/java/org/opensearch/neuralsearch/processor/InferenceProcessor.java
user: login: heemin32
body: Missing try catch here.
created_at: 2025-03-17T23:16:51Z
author_association: COLLABORATOR
diff_hunk: @@ -106,4 +112,85 @@ public void doBatchExecute(List<String> inferenceList, Consumer<List<?>> handler
             ActionListener.wrap(handler::accept, onException)
         );
     }
+
+    @Override
+    public void subBatchExecute(List<IngestDocumentWrapper> ingestDocumentWrappers, Consumer<List<IngestDocumentWrapper>> handler) {
+        try {
+            if (CollectionUtils.isEmpty(ingestDocumentWrappers)) {
+                handler.accept(Collections.emptyList());
+                return;
+            }
+            List<DataForInference> dataForInferences = getDataForInference(ingestDocumentWrappers);
+            if (dataForInferences.isEmpty()) {
+                handler.accept(ingestDocumentWrappers);
+                return;
+            }
+            List<String> inferenceList = constructInferenceTexts(dataForInferences);
+            if (inferenceList.isEmpty()) {
+                handler.accept(ingestDocumentWrappers);
+                return;
+            }
+            // skip existing flag is turned off. Call doSubBatchExecute without filtering
+            if (skipExisting == false) {
+                doSubBatchExecute(ingestDocumentWrappers, inferenceList, dataForInferences, handler);
+                return;
+            }
+            // skipExisting flag is turned on, eligible inference texts in dataForInferences will be compared and filtered after embeddings
+            // are copied
+            openSearchClient.execute(
+                MultiGetAction.INSTANCE,
+                buildMultiGetRequest(ingestDocumentWrappers),
+                ActionListener.wrap(response -> {
+                    MultiGetItemResponse[] multiGetItemResponses = response.getResponses();
path: src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
user: login: heemin32
body: missing try catch here.
created_at: 2025-03-17T23:17:19Z
author_association: COLLABORATOR

PR comments:
user: login: codecov[bot]
created_at: 2025-03-08T04:53:38Z
body: ## [Codecov](https://app.codecov.io/gh/opensearch-project/neural-search/pull/1217?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 81.97%. Comparing base [(`1a6e58e`)](https://app.codecov.io/gh/opensearch-project/neural-search/commit/1a6e58e95699d6101b328473cc342acf74e10fe3?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) to head [(`3bb0a19`)](https://app.codecov.io/gh/opensearch-project/neural-search/commit/3bb0a19bcccd441a7b0b7f3d8955912b46977b47?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project).

<details><summary>Additional details and impacted files</summary>


```diff
@@                    Coverage Diff                    @@
##             optimized-processor    #1217      +/-   ##
=========================================================
+ Coverage                  81.94%   81.97%   +0.02%     
+ Complexity                  2604     1315    -1289     
=========================================================
  Files                        194       97      -97     
  Lines                       8858     4487    -4371     
  Branches                    1498      760     -738     
=========================================================
- Hits                        7259     3678    -3581     
+ Misses                      1016      513     -503     
+ Partials                     583      296     -287     
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/opensearch-project/neural-search/pull/1217?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project).

<details><summary>🚀 New features to boost your workflow: </summary>

- ❄ [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
author_association: NONE
user: login: vibrantvarun
created_at: 2025-03-14T17:29:07Z
body: Overall I have similar comments like @heemin32 so will wait for once it gets addressed before doing final round of review
author_association: MEMBER
user: login: will-hwang
created_at: 2025-03-14T22:43:49Z
body: > @will-hwang could you add tests in failure cases of this feature? Like when will the functionality fail and user will not see 200 status from api

i've added a failure test case [here](https://github.com/opensearch-project/neural-search/blob/d69a21901810a79d2dbac694cf27065406437a91/src/test/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessorTests.java#L2415-L2425) where `mget` fails with runtime exception. In summary, the error handling is the same as our existing implementation, where if dependent service (opensearch client or mlcommonclient) fails with exception, ingestDocument is updated with exception, instead of failing ingestion. 
author_association: CONTRIBUTOR
user: login: heemin32
created_at: 2025-03-18T21:13:28Z
body: Build failures are coming from changes in knn and it is resolved by https://github.com/opensearch-project/neural-search/pull/1233
author_association: COLLABORATOR