Repository: opensearch-project/query-insights
Pull request #295: Inflight Queries API
Author: login: ansjcy
Labels: name: backport 3.0
Created: 2025-04-10T04:40:13Z | Merged: 2025-04-22T23:45:43Z

### Description
Add Inflight Queries API to fetch the ongoing search queries.
- Fetch running SearchTasks from nodes on demand and construct SearchQueryRecord from it.
- Extensive Unit tests coverage
- Integration test coverage for the API.

### Test
- Run an OpenSearch cluster with 2 nodes.
- Create document and perform 2 search requests on 2 nodes. Using Thread.sleep to simulate long running queries
- Get inflight queries:
```
➜  ~ curl -X GET "localhost:9200/_insights/live_queries?pretty"
{
  "live_queries" : [
    {
      "timestamp" : 1745359226777,
      "id" : "troGHNGUShqDj3wK_K5ZIw:512",
      "description" : "indices[my-index-*], search_type[QUERY_THEN_FETCH], source[{\"size\":20,\"query\":{\"term\":{\"user.id\":{\"value\":\"userId\",\"boost\":1.0}}}}]",
      "node_id" : "troGHNGUShqDj3wK_K5ZIw",
      "measurements" : {
        "latency" : {
          "number" : 13959364458,
          "count" : 1,
          "aggregationType" : "NONE"
        },
        "memory" : {
          "number" : 3104,
          "count" : 1,
          "aggregationType" : "NONE"
        },
        "cpu" : {
          "number" : 405000,
          "count" : 1,
          "aggregationType" : "NONE"
        }
      }
    },
    {
      "timestamp" : 1745359229158,
      "id" : "Y6eBnbdISPO6XaVfxCBRgg:454",
      "description" : "indices[my-index-*], search_type[QUERY_THEN_FETCH], source[{\"size\":20,\"query\":{\"term\":{\"user.id\":{\"value\":\"userId\",\"boost\":1.0}}}}]",
      "node_id" : "Y6eBnbdISPO6XaVfxCBRgg",
      "measurements" : {
        "latency" : {
          "number" : 11579097209,
          "count" : 1,
          "aggregationType" : "NONE"
        },
        "memory" : {
          "number" : 3104,
          "count" : 1,
          "aggregationType" : "NONE"
        },
        "cpu" : {
          "number" : 511000,
          "count" : 1,
          "aggregationType" : "NONE"
        }
      }
    }
  ]
}
```

- Get in flight queries on specific node
```
curl -X GET "localhost:9200/_insights/live_queries?nodeId=integTest-0&pretty"
{
  "live_queries" : [
    {
      "timestamp" : 1745019842751,
      "id" : "faQoGlu2T8iAcJtIP3vSDQ:303",
      "description" : "indices[my-index-*], search_type[QUERY_THEN_FETCH], source[{\"size\":20,\"query\":{\"term\":{\"user.id\":{\"value\":\"cyji\",\"boost\":1.0}}}}]",
      "node_id" : "faQoGlu2T8iAcJtIP3vSDQ",
      "measurements" : {
        "memory" : {
          "number" : 29560,
          "count" : 1,
          "aggregationType" : "NONE"
        },
        "latency" : {
          "number" : 7924553291,
          "count" : 1,
          "aggregationType" : "NONE"
        },
        "cpu" : {
          "number" : 1971000,
          "count" : 1,
          "aggregationType" : "NONE"
        }
      }
    }
  ]
}
```

- sort by
```
curl -X GET "localhost:9200/_insights/live_queries?sort=cpu&pretty"
{
  "live_queries" : [
    {
      "timestamp" : 1745358380622,
      "id" : "RrMW2BWRRfyrWQ47lPoGWg:89",
      "description" : "indices[my-index-*], search_type[QUERY_THEN_FETCH], source[{\"size\":20,\"query\":{\"term\":{\"user.id\":{\"value\":\"userId\",\"boost\":1.0}}}}]",
      "node_id" : "RrMW2BWRRfyrWQ47lPoGWg",
      "measurements" : {
        "latency" : {
          "number" : 11261929250,
          "count" : 1,
          "aggregationType" : "NONE"
        },
        "cpu" : {
          "number" : 4862000,
          "count" : 1,
          "aggregationType" : "NONE"
        },
        "memory" : {
          "number" : 547072,
          "count" : 1,
          "aggregationType" : "NONE"
        }
      }
    },
    {
      "timestamp" : 1745358378450,
      "id" : "fits7pNATQ6ZI6dbLJfn4Q:74",
      "description" : "indices[my-index-*], search_type[QUERY_THEN_FETCH], source[{\"size\":20,\"query\":{\"term\":{\"user.id\":{\"value\":\"userId\",\"boost\":1.0}}}}]",
      "node_id" : "fits7pNATQ6ZI6dbLJfn4Q",
      "measurements" : {
        "latency" : {
          "number" : 13432647416,
          "count" : 1,
          "aggregationType" : "NONE"
        },
        "cpu" : {
          "number" : 2359000,
          "count" : 1,
          "aggregationType" : "NONE"
        },
        "memory" : {
          "number" : 614720,
          "count" : 1,
          "aggregationType" : "NONE"
        }
      }
    }
  ]
}
```

- size
```
curl -X GET "localhost:9200/_insights/live_queries?sort=cpu&size=1&pretty"
{
  "live_queries" : [
    {
      "timestamp" : 1745358437352,
      "id" : "fits7pNATQ6ZI6dbLJfn4Q:152",
      "description" : "indices[my-index-*], search_type[QUERY_THEN_FETCH], source[{\"size\":20,\"query\":{\"term\":{\"user.id\":{\"value\":\"userId\",\"boost\":1.0}}}}]",
      "node_id" : "fits7pNATQ6ZI6dbLJfn4Q",
      "measurements" : {
        "latency" : {
          "number" : 10969900667,
          "count" : 1,
          "aggregationType" : "NONE"
        },
        "cpu" : {
          "number" : 394000,
          "count" : 1,
          "aggregationType" : "NONE"
        },
        "memory" : {
          "number" : 3168,
          "count" : 1,
          "aggregationType" : "NONE"
        }
      }
    }
  ]
}
```

- After search query finishes
```
curl -X GET "localhost:9200/_insights/live_queries?nodeId=integTest-0&pretty"
{
  "live_queries" : [ ]
}
```

### Issues Resolved
Related to https://github.com/opensearch-project/query-insights/issues/285

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check [here](https://github.com/opensearch-project/OpenSearch/blob/main/CONTRIBUTING.md#developer-certificate-of-origin).



Diff:
diff --git a/src/main/java/org/opensearch/plugin/insights/QueryInsightsPlugin.java b/src/main/java/org/opensearch/plugin/insights/QueryInsightsPlugin.java
index 8f5817df..fecf52ae 100644
--- a/src/main/java/org/opensearch/plugin/insights/QueryInsightsPlugin.java
+++ b/src/main/java/org/opensearch/plugin/insights/QueryInsightsPlugin.java
@@ -33,10 +33,13 @@
 import org.opensearch.plugin.insights.core.reader.QueryInsightsReaderFactory;
 import org.opensearch.plugin.insights.core.service.QueryInsightsService;
 import org.opensearch.plugin.insights.rules.action.health_stats.HealthStatsAction;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesAction;
 import org.opensearch.plugin.insights.rules.action.top_queries.TopQueriesAction;
 import org.opensearch.plugin.insights.rules.resthandler.health_stats.RestHealthStatsAction;
+import org.opensearch.plugin.insights.rules.resthandler.live_queries.RestLiveQueriesAction;
 import org.opensearch.plugin.insights.rules.resthandler.top_queries.RestTopQueriesAction;
 import org.opensearch.plugin.insights.rules.transport.health_stats.TransportHealthStatsAction;
+import org.opensearch.plugin.insights.rules.transport.live_queries.TransportLiveQueriesAction;
 import org.opensearch.plugin.insights.rules.transport.top_queries.TransportTopQueriesAction;
 import org.opensearch.plugin.insights.settings.QueryCategorizationSettings;
 import org.opensearch.plugin.insights.settings.QueryInsightsSettings;
@@ -117,14 +120,15 @@ public List<RestHandler> getRestHandlers(
         final IndexNameExpressionResolver indexNameExpressionResolver,
         final Supplier<DiscoveryNodes> nodesInCluster
     ) {
-        return List.of(new RestTopQueriesAction(), new RestHealthStatsAction());
+        return List.of(new RestTopQueriesAction(), new RestHealthStatsAction(), new RestLiveQueriesAction());
     }
 
     @Override
     public List<ActionHandler<? extends ActionRequest, ? extends ActionResponse>> getActions() {
         return List.of(
             new ActionPlugin.ActionHandler<>(TopQueriesAction.INSTANCE, TransportTopQueriesAction.class),
-            new ActionPlugin.ActionHandler<>(HealthStatsAction.INSTANCE, TransportHealthStatsAction.class)
+            new ActionPlugin.ActionHandler<>(HealthStatsAction.INSTANCE, TransportHealthStatsAction.class),
+            new ActionPlugin.ActionHandler<>(LiveQueriesAction.INSTANCE, TransportLiveQueriesAction.class)
         );
     }
 
diff --git a/src/main/java/org/opensearch/plugin/insights/rules/action/live_queries/LiveQueriesAction.java b/src/main/java/org/opensearch/plugin/insights/rules/action/live_queries/LiveQueriesAction.java
new file mode 100644
index 00000000..ae8129a5
--- /dev/null
+++ b/src/main/java/org/opensearch/plugin/insights/rules/action/live_queries/LiveQueriesAction.java
@@ -0,0 +1,30 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.action.live_queries;
+
+import org.opensearch.action.ActionType;
+
+/**
+ * Transport action for cluster/node level ongoing live queries information.
+ */
+public class LiveQueriesAction extends ActionType<LiveQueriesResponse> {
+
+    /**
+     * The LiveQueriesAction Instance.
+     */
+    public static final LiveQueriesAction INSTANCE = new LiveQueriesAction();
+    /**
+     * The name of this Action
+     */
+    public static final String NAME = "cluster:admin/opensearch/insights/live_queries";
+
+    private LiveQueriesAction() {
+        super(NAME, LiveQueriesResponse::new);
+    }
+}
diff --git a/src/main/java/org/opensearch/plugin/insights/rules/action/live_queries/LiveQueriesRequest.java b/src/main/java/org/opensearch/plugin/insights/rules/action/live_queries/LiveQueriesRequest.java
new file mode 100644
index 00000000..1f10f108
--- /dev/null
+++ b/src/main/java/org/opensearch/plugin/insights/rules/action/live_queries/LiveQueriesRequest.java
@@ -0,0 +1,113 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.action.live_queries;
+
+import java.io.IOException;
+import org.opensearch.action.ActionRequest;
+import org.opensearch.action.ActionRequestValidationException;
+import org.opensearch.core.common.io.stream.StreamInput;
+import org.opensearch.core.common.io.stream.StreamOutput;
+import org.opensearch.plugin.insights.rules.model.MetricType;
+import org.opensearch.plugin.insights.settings.QueryInsightsSettings;
+
+/**
+ * A request to get cluster/node level ongoing live queries information.
+ */
+public class LiveQueriesRequest extends ActionRequest {
+
+    private final boolean verbose;
+    private final MetricType sortBy;
+    // Maximum number of results to return
+    private final int size;
+    // Node IDs to filter queries by
+    private final String[] nodeIds;
+
+    /**
+     * Constructor for LiveQueriesRequest
+     *
+     * @param in A {@link StreamInput} object.
+     * @throws IOException if the stream cannot be deserialized.
+     */
+    public LiveQueriesRequest(final StreamInput in) throws IOException {
+        super(in);
+        this.verbose = in.readBoolean();
+        this.sortBy = MetricType.readFromStream(in);
+        this.size = in.readInt();
+        this.nodeIds = in.readStringArray();
+    }
+
+    /**
+     * Get live queries from nodes based on the nodes ids specified.
+     * If none are passed, cluster level live queries will be returned.
+     *
+     * @param verbose Whether to include verbose information about live queries (defaults to true)
+     * @param sortBy the metric to sort by (latency, cpu, memory)
+     * @param size maximum number of results
+     * @param nodeIds The node IDs specified in the request
+     */
+    public LiveQueriesRequest(final boolean verbose, final MetricType sortBy, final int size, final String... nodeIds) {
+        this.verbose = verbose;
+        this.sortBy = sortBy;
+        this.size = size;
+        this.nodeIds = nodeIds;
+    }
+
+    /**
+     * Convenience constructor using default sortBy=LATENCY and no size limit.
+     * @param verbose whether to include verbose information about live queries
+     * @param nodeIds the node IDs specified in the request
+     */
+    public LiveQueriesRequest(final boolean verbose, final String... nodeIds) {
+        this(verbose, MetricType.LATENCY, QueryInsightsSettings.DEFAULT_LIVE_QUERIES_SIZE, nodeIds);
+    }
+
+    /**
+     * Get whether verbose information is requested
+     * @return boolean indicating whether verbose information is requested
+     */
+    public boolean isVerbose() {
+        return verbose;
+    }
+
+    /**
+     * Get metric type to sort by
+     */
+    public MetricType getSortBy() {
+        return sortBy;
+    }
+
+    /**
+     * Get maximum result size
+     */
+    public int getSize() {
+        return size;
+    }
+
+    /**
+     * Get node IDs to filter by
+     * @return array of node IDs
+     */
+    public String[] nodesIds() {
+        return nodeIds;
+    }
+
+    @Override
+    public void writeTo(final StreamOutput out) throws IOException {
+        super.writeTo(out);
+        out.writeBoolean(verbose);
+        MetricType.writeTo(out, sortBy);
+        out.writeInt(size);
+        out.writeStringArray(nodeIds);
+    }
+
+    @Override
+    public ActionRequestValidationException validate() {
+        return null;
+    }
+}
diff --git a/src/main/java/org/opensearch/plugin/insights/rules/action/live_queries/LiveQueriesResponse.java b/src/main/java/org/opensearch/plugin/insights/rules/action/live_queries/LiveQueriesResponse.java
new file mode 100644
index 00000000..ebe0b00c
--- /dev/null
+++ b/src/main/java/org/opensearch/plugin/insights/rules/action/live_queries/LiveQueriesResponse.java
@@ -0,0 +1,72 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.action.live_queries;
+
+import java.io.IOException;
+import java.util.List;
+import org.opensearch.core.action.ActionResponse;
+import org.opensearch.core.common.io.stream.StreamInput;
+import org.opensearch.core.common.io.stream.StreamOutput;
+import org.opensearch.core.xcontent.ToXContentObject;
+import org.opensearch.core.xcontent.XContentBuilder;
+import org.opensearch.plugin.insights.rules.model.SearchQueryRecord;
+
+/**
+ * Transport response for cluster/node level live queries information.
+ */
+public class LiveQueriesResponse extends ActionResponse implements ToXContentObject {
+
+    private static final String CLUSTER_LEVEL_RESULTS_KEY = "live_queries";
+    private final List<SearchQueryRecord> liveQueries;
+
+    /**
+     * Constructor for LiveQueriesResponse.
+     *
+     * @param in A {@link StreamInput} object.
+     * @throws IOException if the stream cannot be deserialized.
+     */
+    public LiveQueriesResponse(final StreamInput in) throws IOException {
+        this.liveQueries = in.readList(SearchQueryRecord::new);
+    }
+
+    /**
+     * Constructor for LiveQueriesResponse
+     *
+     * @param liveQueries A flat list containing live queries results from relevant nodes
+     */
+    public LiveQueriesResponse(final List<SearchQueryRecord> liveQueries) {
+        this.liveQueries = liveQueries;
+    }
+
+    /**
+     * Get the live queries list
+     * @return the list of live query records
+     */
+    public List<SearchQueryRecord> getLiveQueries() {
+        return liveQueries;
+    }
+
+    @Override
+    public void writeTo(final StreamOutput out) throws IOException {
+        out.writeList(liveQueries);
+    }
+
+    @Override
+    public XContentBuilder toXContent(final XContentBuilder builder, final Params params) throws IOException {
+        builder.startObject();
+        builder.startArray(CLUSTER_LEVEL_RESULTS_KEY);
+
+        for (SearchQueryRecord query : liveQueries) {
+            query.toXContent(builder, params);
+        }
+        builder.endArray();
+        builder.endObject();
+        return builder;
+    }
+}
diff --git a/src/main/java/org/opensearch/plugin/insights/rules/action/live_queries/package-info.java b/src/main/java/org/opensearch/plugin/insights/rules/action/live_queries/package-info.java
new file mode 100644
index 00000000..6c8099bf
--- /dev/null
+++ b/src/main/java/org/opensearch/plugin/insights/rules/action/live_queries/package-info.java
@@ -0,0 +1,12 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+/**
+ * Transport Actions, Requests and Responses for live queries
+ */
+package org.opensearch.plugin.insights.rules.action.live_queries;
diff --git a/src/main/java/org/opensearch/plugin/insights/rules/model/Attribute.java b/src/main/java/org/opensearch/plugin/insights/rules/model/Attribute.java
index 5cf233a3..e684aa6a 100644
--- a/src/main/java/org/opensearch/plugin/insights/rules/model/Attribute.java
+++ b/src/main/java/org/opensearch/plugin/insights/rules/model/Attribute.java
@@ -64,7 +64,11 @@ public enum Attribute {
     /**
      * Grouping type of the query record (none, similarity)
      */
-    GROUP_BY;
+    GROUP_BY,
+    /**
+     * The description of the search query, often used in live queries.
+     */
+    DESCRIPTION;
 
     /**
      * Read an Attribute from a StreamInput
diff --git a/src/main/java/org/opensearch/plugin/insights/rules/resthandler/live_queries/RestLiveQueriesAction.java b/src/main/java/org/opensearch/plugin/insights/rules/resthandler/live_queries/RestLiveQueriesAction.java
new file mode 100644
index 00000000..7d13d3a4
--- /dev/null
+++ b/src/main/java/org/opensearch/plugin/insights/rules/resthandler/live_queries/RestLiveQueriesAction.java
@@ -0,0 +1,95 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.resthandler.live_queries;
+
+import static org.opensearch.plugin.insights.settings.QueryInsightsSettings.LIVE_QUERIES_BASE_URI;
+import static org.opensearch.rest.RestRequest.Method.GET;
+
+import java.util.List;
+import java.util.Locale;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.opensearch.common.settings.Settings;
+import org.opensearch.core.common.Strings;
+import org.opensearch.core.rest.RestStatus;
+import org.opensearch.core.xcontent.ToXContent;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesAction;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesRequest;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesResponse;
+import org.opensearch.plugin.insights.rules.model.MetricType;
+import org.opensearch.plugin.insights.settings.QueryInsightsSettings;
+import org.opensearch.rest.BaseRestHandler;
+import org.opensearch.rest.BytesRestResponse;
+import org.opensearch.rest.RestChannel;
+import org.opensearch.rest.RestRequest;
+import org.opensearch.rest.RestResponse;
+import org.opensearch.rest.action.RestResponseListener;
+import org.opensearch.transport.client.node.NodeClient;
+
+/**
+ * Rest action to get ongoing live queries
+ */
+public class RestLiveQueriesAction extends BaseRestHandler {
+    static final Set<String> ALLOWED_METRICS = MetricType.allMetricTypes().stream().map(MetricType::toString).collect(Collectors.toSet());
+
+    /**
+     * Constructor for RestLiveQueriesAction
+     */
+    public RestLiveQueriesAction() {}
+
+    @Override
+    public List<Route> routes() {
+        return List.of(new Route(GET, LIVE_QUERIES_BASE_URI));
+    }
+
+    @Override
+    public String getName() {
+        return "query_insights_live_queries_action";
+    }
+
+    @Override
+    public RestChannelConsumer prepareRequest(final RestRequest request, final NodeClient client) {
+        final LiveQueriesRequest liveQueriesRequest = prepareRequest(request);
+
+        return channel -> client.execute(LiveQueriesAction.INSTANCE, liveQueriesRequest, liveQueriesResponse(channel));
+    }
+
+    static LiveQueriesRequest prepareRequest(final RestRequest request) {
+        final String[] nodesIds = Strings.splitStringByCommaToArray(request.param("nodeId"));
+        final boolean verbose = request.paramAsBoolean("verbose", true);
+        final String sortParam = request.param("sort", MetricType.LATENCY.toString());
+        if (!ALLOWED_METRICS.contains(sortParam)) {
+            throw new IllegalArgumentException(
+                String.format(Locale.ROOT, "request [%s] contains invalid sort metric type [%s]", request.path(), sortParam)
+            );
+        }
+        final MetricType sortBy = MetricType.fromString(sortParam);
+        final int size = request.paramAsInt("size", QueryInsightsSettings.DEFAULT_LIVE_QUERIES_SIZE);
+        return new LiveQueriesRequest(verbose, sortBy, size, nodesIds);
+    }
+
+    @Override
+    protected Set<String> responseParams() {
+        return Settings.FORMAT_PARAMS;
+    }
+
+    @Override
+    public boolean canTripCircuitBreaker() {
+        return false;
+    }
+
+    private RestResponseListener<LiveQueriesResponse> liveQueriesResponse(final RestChannel channel) {
+        return new RestResponseListener<>(channel) {
+            @Override
+            public RestResponse buildResponse(final LiveQueriesResponse response) throws Exception {
+                return new BytesRestResponse(RestStatus.OK, response.toXContent(channel.newBuilder(), ToXContent.EMPTY_PARAMS));
+            }
+        };
+    }
+}
diff --git a/src/main/java/org/opensearch/plugin/insights/rules/resthandler/live_queries/package-info.java b/src/main/java/org/opensearch/plugin/insights/rules/resthandler/live_queries/package-info.java
new file mode 100644
index 00000000..be9517f3
--- /dev/null
+++ b/src/main/java/org/opensearch/plugin/insights/rules/resthandler/live_queries/package-info.java
@@ -0,0 +1,12 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+/**
+ * Rest Handlers for Query Insights live queries
+ */
+package org.opensearch.plugin.insights.rules.resthandler.live_queries;
diff --git a/src/main/java/org/opensearch/plugin/insights/rules/transport/live_queries/TransportLiveQueriesAction.java b/src/main/java/org/opensearch/plugin/insights/rules/transport/live_queries/TransportLiveQueriesAction.java
new file mode 100644
index 00000000..1be24b5b
--- /dev/null
+++ b/src/main/java/org/opensearch/plugin/insights/rules/transport/live_queries/TransportLiveQueriesAction.java
@@ -0,0 +1,131 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.transport.live_queries;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.opensearch.action.admin.cluster.node.tasks.list.ListTasksRequest;
+import org.opensearch.action.admin.cluster.node.tasks.list.ListTasksResponse;
+import org.opensearch.action.support.ActionFilters;
+import org.opensearch.action.support.HandledTransportAction;
+import org.opensearch.common.inject.Inject;
+import org.opensearch.core.action.ActionListener;
+import org.opensearch.core.tasks.resourcetracker.TaskResourceStats;
+import org.opensearch.core.tasks.resourcetracker.TaskResourceUsage;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesAction;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesRequest;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesResponse;
+import org.opensearch.plugin.insights.rules.model.Attribute;
+import org.opensearch.plugin.insights.rules.model.Measurement;
+import org.opensearch.plugin.insights.rules.model.MetricType;
+import org.opensearch.plugin.insights.rules.model.SearchQueryRecord;
+import org.opensearch.tasks.Task;
+import org.opensearch.tasks.TaskInfo;
+import org.opensearch.threadpool.ThreadPool;
+import org.opensearch.transport.TransportService;
+import org.opensearch.transport.client.Client;
+
+/**
+ * Transport action for fetching ongoing live queries
+ */
+public class TransportLiveQueriesAction extends HandledTransportAction<LiveQueriesRequest, LiveQueriesResponse> {
+
+    private static final Logger logger = LogManager.getLogger(TransportLiveQueriesAction.class);
+    private static final String TOTAL = "total";
+
+    private final Client client;
+
+    @Inject
+    public TransportLiveQueriesAction(final TransportService transportService, final Client client, final ActionFilters actionFilters) {
+        super(LiveQueriesAction.NAME, transportService, actionFilters, LiveQueriesRequest::new, ThreadPool.Names.GENERIC);
+        this.client = client;
+    }
+
+    @Override
+    protected void doExecute(final Task task, final LiveQueriesRequest request, final ActionListener<LiveQueriesResponse> listener) {
+        ListTasksRequest listTasksRequest = new ListTasksRequest().setDetailed(request.isVerbose()).setActions("indices:data/read/search*");
+
+        // Set nodes filter if provided in the request
+        String[] requestedNodeIds = request.nodesIds();
+        if (requestedNodeIds != null && requestedNodeIds.length > 0) {
+            listTasksRequest.setNodes(requestedNodeIds);
+        }
+
+        // Execute tasks request asynchronously to avoid blocking
+        client.admin().cluster().listTasks(listTasksRequest, new ActionListener<ListTasksResponse>() {
+            @Override
+            public void onResponse(ListTasksResponse taskResponse) {
+                try {
+                    List<SearchQueryRecord> allFilteredRecords = new ArrayList<>();
+                    for (TaskInfo taskInfo : taskResponse.getTasks()) {
+                        if (!taskInfo.getAction().startsWith("indices:data/read/search")) {
+                            continue;
+                        }
+                        long timestamp = taskInfo.getStartTime();
+                        String nodeId = taskInfo.getTaskId().getNodeId();
+                        long runningNanos = taskInfo.getRunningTimeNanos();
+
+                        Map<MetricType, Measurement> measurements = new HashMap<>();
+                        measurements.put(MetricType.LATENCY, new Measurement(runningNanos));
+
+                        long cpuNanos = 0L;
+                        long memBytes = 0L;
+                        TaskResourceStats stats = taskInfo.getResourceStats();
+                        if (stats != null) {
+                            Map<String, TaskResourceUsage> usageInfo = stats.getResourceUsageInfo();
+                            if (usageInfo != null) {
+                                TaskResourceUsage totalUsage = usageInfo.get(TOTAL);
+                                if (totalUsage != null) {
+                                    cpuNanos = totalUsage.getCpuTimeInNanos();
+                                    memBytes = totalUsage.getMemoryInBytes();
+                                }
+                            }
+                        }
+                        measurements.put(MetricType.CPU, new Measurement(cpuNanos));
+                        measurements.put(MetricType.MEMORY, new Measurement(memBytes));
+
+                        Map<Attribute, Object> attributes = new HashMap<>();
+                        attributes.put(Attribute.NODE_ID, nodeId);
+                        if (request.isVerbose()) {
+                            attributes.put(Attribute.DESCRIPTION, taskInfo.getDescription());
+                        }
+
+                        SearchQueryRecord record = new SearchQueryRecord(
+                            timestamp,
+                            measurements,
+                            attributes,
+                            taskInfo.getTaskId().toString()
+                        );
+                        allFilteredRecords.add(record);
+                    }
+
+                    // Sort descending by the requested metric and apply size limit in one pass
+                    List<SearchQueryRecord> finalRecords = allFilteredRecords.stream()
+                        .sorted((a, b) -> SearchQueryRecord.compare(b, a, request.getSortBy()))
+                        .limit(request.getSize() < 0 ? Long.MAX_VALUE : request.getSize())
+                        .toList();
+                    listener.onResponse(new LiveQueriesResponse(finalRecords));
+                } catch (Exception ex) {
+                    logger.error("Failed to process live queries response", ex);
+                    listener.onFailure(ex);
+                }
+            }
+
+            @Override
+            public void onFailure(Exception e) {
+                logger.error("Failed to retrieve live queries", e);
+                listener.onFailure(e);
+            }
+        });
+    }
+}
diff --git a/src/main/java/org/opensearch/plugin/insights/rules/transport/live_queries/package-info.java b/src/main/java/org/opensearch/plugin/insights/rules/transport/live_queries/package-info.java
new file mode 100644
index 00000000..37467d62
--- /dev/null
+++ b/src/main/java/org/opensearch/plugin/insights/rules/transport/live_queries/package-info.java
@@ -0,0 +1,12 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+/**
+ * Transport Actions for live queries.
+ */
+package org.opensearch.plugin.insights.rules.transport.live_queries;
diff --git a/src/main/java/org/opensearch/plugin/insights/settings/QueryInsightsSettings.java b/src/main/java/org/opensearch/plugin/insights/settings/QueryInsightsSettings.java
index e1c79754..ce5eb40b 100644
--- a/src/main/java/org/opensearch/plugin/insights/settings/QueryInsightsSettings.java
+++ b/src/main/java/org/opensearch/plugin/insights/settings/QueryInsightsSettings.java
@@ -80,6 +80,17 @@ public class QueryInsightsSettings {
      *
      */
     public static final String TOP_QUERIES_BASE_URI = PLUGINS_BASE_URI + "/top_queries";
+
+    /**
+     * Settings for Live Queries
+     */
+    public static final String LIVE_QUERIES_BASE_URI = PLUGINS_BASE_URI + "/live_queries";
+
+    /**
+     * Default number of results to return for live queries
+     */
+    public static final int DEFAULT_LIVE_QUERIES_SIZE = 100;
+
     /** Default prefix for top N queries feature */
     public static final String TOP_N_QUERIES_SETTING_PREFIX = "search.insights.top_queries";
     /** Default prefix for top N queries grouping feature */
diff --git a/src/test/java/org/opensearch/plugin/insights/QueryInsightsPluginTests.java b/src/test/java/org/opensearch/plugin/insights/QueryInsightsPluginTests.java
index e79329b9..1e1c1f9c 100644
--- a/src/test/java/org/opensearch/plugin/insights/QueryInsightsPluginTests.java
+++ b/src/test/java/org/opensearch/plugin/insights/QueryInsightsPluginTests.java
@@ -23,9 +23,12 @@
 import org.opensearch.plugin.insights.core.listener.QueryInsightsListener;
 import org.opensearch.plugin.insights.core.service.QueryInsightsService;
 import org.opensearch.plugin.insights.rules.action.health_stats.HealthStatsAction;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesAction;
 import org.opensearch.plugin.insights.rules.action.top_queries.TopQueriesAction;
 import org.opensearch.plugin.insights.rules.resthandler.health_stats.RestHealthStatsAction;
+import org.opensearch.plugin.insights.rules.resthandler.live_queries.RestLiveQueriesAction;
 import org.opensearch.plugin.insights.rules.resthandler.top_queries.RestTopQueriesAction;
+import org.opensearch.plugin.insights.rules.transport.live_queries.TransportLiveQueriesAction;
 import org.opensearch.plugin.insights.settings.QueryCategorizationSettings;
 import org.opensearch.plugin.insights.settings.QueryInsightsSettings;
 import org.opensearch.plugins.ActionPlugin;
@@ -119,16 +122,35 @@ public void testGetExecutorBuilders() {
 
     public void testGetRestHandlers() {
         List<RestHandler> components = queryInsightsPlugin.getRestHandlers(Settings.EMPTY, null, null, null, null, null, null);
-        assertEquals(2, components.size());
+        assertEquals(3, components.size());
         assertTrue(components.get(0) instanceof RestTopQueriesAction);
         assertTrue(components.get(1) instanceof RestHealthStatsAction);
+        assertTrue(components.get(2) instanceof RestLiveQueriesAction);
     }
 
     public void testGetActions() {
         List<ActionPlugin.ActionHandler<? extends ActionRequest, ? extends ActionResponse>> components = queryInsightsPlugin.getActions();
-        assertEquals(2, components.size());
+        assertEquals(3, components.size());
         assertTrue(components.get(0).getAction() instanceof TopQueriesAction);
         assertTrue(components.get(1).getAction() instanceof HealthStatsAction);
+        assertTrue(components.get(2).getAction() instanceof LiveQueriesAction);
+    }
+
+    public void testLiveQueriesActionRegistration() {
+        List<ActionPlugin.ActionHandler<? extends ActionRequest, ? extends ActionResponse>> actions = queryInsightsPlugin.getActions();
+        boolean hasLiveQueriesAction = actions.stream().anyMatch(handler -> handler.getAction().name().equals(LiveQueriesAction.NAME));
+        assertTrue("Plugin should register LiveQueriesAction", hasLiveQueriesAction);
+
+        boolean hasLiveQueriesTransport = actions.stream()
+            .filter(handler -> handler.getAction().name().equals(LiveQueriesAction.NAME))
+            .anyMatch(handler -> handler.getTransportAction().equals(TransportLiveQueriesAction.class));
+        assertTrue("Plugin should register TransportLiveQueriesAction", hasLiveQueriesTransport);
+    }
+
+    public void testLiveQueriesRestActionRegistration() {
+        List<RestHandler> restHandlers = queryInsightsPlugin.getRestHandlers(Settings.EMPTY, null, null, null, null, null, null);
+        boolean hasLiveQueriesRestHandler = restHandlers.stream().anyMatch(handler -> handler instanceof RestLiveQueriesAction);
+        assertTrue("Plugin should register RestLiveQueriesAction", hasLiveQueriesRestHandler);
     }
 
 }
diff --git a/src/test/java/org/opensearch/plugin/insights/QueryInsightsRestTestCase.java b/src/test/java/org/opensearch/plugin/insights/QueryInsightsRestTestCase.java
index c4e6cd8c..267ef773 100644
--- a/src/test/java/org/opensearch/plugin/insights/QueryInsightsRestTestCase.java
+++ b/src/test/java/org/opensearch/plugin/insights/QueryInsightsRestTestCase.java
@@ -195,6 +195,16 @@ public void wipeAllQueryInsightsIndices() throws Exception {
         }
     }
 
+    protected String disableTopQueriesSettings() {
+        return "{\n"
+            + "    \"persistent\" : {\n"
+            + "        \"search.insights.top_queries.latency.enabled\" : \"false\",\n"
+            + "        \"search.insights.top_queries.memory.enabled\" : \"false\",\n"
+            + "        \"search.insights.top_queries.cpu.enabled\" : \"false\"\n"
+            + "    }\n"
+            + "}";
+    }
+
     protected String defaultTopQueriesSettings() {
         return "{\n"
             + "    \"persistent\" : {\n"
diff --git a/src/test/java/org/opensearch/plugin/insights/rules/action/live_queries/LiveQueriesRequestTests.java b/src/test/java/org/opensearch/plugin/insights/rules/action/live_queries/LiveQueriesRequestTests.java
new file mode 100644
index 00000000..c77ad926
--- /dev/null
+++ b/src/test/java/org/opensearch/plugin/insights/rules/action/live_queries/LiveQueriesRequestTests.java
@@ -0,0 +1,106 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.action.live_queries;
+
+import java.io.IOException;
+import org.opensearch.common.io.stream.BytesStreamOutput;
+import org.opensearch.core.common.io.stream.StreamInput;
+import org.opensearch.plugin.insights.rules.model.MetricType;
+import org.opensearch.plugin.insights.settings.QueryInsightsSettings;
+import org.opensearch.test.OpenSearchTestCase;
+
+/**
+ * Unit tests for the {@link LiveQueriesRequest} class.
+ */
+public class LiveQueriesRequestTests extends OpenSearchTestCase {
+
+    public void testSerialization() throws IOException {
+        boolean verbose = true;
+        MetricType sortBy = MetricType.CPU;
+        int size = 5;
+        String[] nodeIds = new String[] { "nodeA", "nodeB", "nodeC" };
+
+        LiveQueriesRequest originalRequest = new LiveQueriesRequest(verbose, sortBy, size, nodeIds);
+
+        BytesStreamOutput out = new BytesStreamOutput();
+        originalRequest.writeTo(out);
+        StreamInput in = StreamInput.wrap(out.bytes().toBytesRef().bytes);
+        LiveQueriesRequest deserializedRequest = new LiveQueriesRequest(in);
+
+        assertEquals(originalRequest.isVerbose(), deserializedRequest.isVerbose());
+        assertEquals(originalRequest.getSortBy(), deserializedRequest.getSortBy());
+        assertEquals(originalRequest.getSize(), deserializedRequest.getSize());
+        assertArrayEquals(originalRequest.nodesIds(), deserializedRequest.nodesIds());
+    }
+
+    public void testSerializationWithEmptyNodes() throws IOException {
+        boolean verbose = false;
+        MetricType sortBy = MetricType.MEMORY;
+        int size = 3;
+        String[] nodeIds = new String[0];
+
+        LiveQueriesRequest originalRequest = new LiveQueriesRequest(verbose, sortBy, size, nodeIds);
+
+        BytesStreamOutput out = new BytesStreamOutput();
+        originalRequest.writeTo(out);
+        StreamInput in = StreamInput.wrap(out.bytes().toBytesRef().bytes);
+        LiveQueriesRequest deserializedRequest = new LiveQueriesRequest(in);
+
+        assertEquals(originalRequest.isVerbose(), deserializedRequest.isVerbose());
+        assertEquals(originalRequest.getSortBy(), deserializedRequest.getSortBy());
+        assertEquals(originalRequest.getSize(), deserializedRequest.getSize());
+        assertArrayEquals(originalRequest.nodesIds(), deserializedRequest.nodesIds());
+        assertEquals(0, deserializedRequest.nodesIds().length);
+    }
+
+    public void testMainConstructor() {
+        boolean verbose = true;
+        MetricType sortBy = MetricType.CPU;
+        int size = 50;
+        String[] nodeIds = { "node1", "node2" };
+
+        LiveQueriesRequest request = new LiveQueriesRequest(verbose, sortBy, size, nodeIds);
+
+        assertTrue(request.isVerbose());
+        assertEquals(MetricType.CPU, request.getSortBy());
+        assertEquals(50, request.getSize());
+        assertArrayEquals(new String[] { "node1", "node2" }, request.nodesIds());
+    }
+
+    public void testConvenienceConstructor() {
+        boolean verbose = false;
+        String[] nodeIds = { "node3" };
+
+        LiveQueriesRequest request = new LiveQueriesRequest(verbose, nodeIds);
+
+        assertFalse(request.isVerbose());
+        assertEquals(MetricType.LATENCY, request.getSortBy());
+        assertEquals(QueryInsightsSettings.DEFAULT_LIVE_QUERIES_SIZE, request.getSize());
+        assertArrayEquals(new String[] { "node3" }, request.nodesIds());
+    }
+
+    public void testConvenienceConstructorNoNodes() {
+        boolean verbose = true;
+        // No node IDs specified
+        LiveQueriesRequest request = new LiveQueriesRequest(verbose);
+
+        assertTrue(request.isVerbose());
+        assertEquals(MetricType.LATENCY, request.getSortBy());
+        assertEquals(QueryInsightsSettings.DEFAULT_LIVE_QUERIES_SIZE, request.getSize());
+        assertEquals(0, request.nodesIds().length);
+    }
+
+    public void testGetters() {
+        LiveQueriesRequest request = new LiveQueriesRequest(false, MetricType.MEMORY, 10, "nodeA");
+        assertFalse(request.isVerbose());
+        assertEquals(MetricType.MEMORY, request.getSortBy());
+        assertEquals(10, request.getSize());
+        assertArrayEquals(new String[] { "nodeA" }, request.nodesIds());
+    }
+}
diff --git a/src/test/java/org/opensearch/plugin/insights/rules/action/live_queries/LiveQueriesResponseTests.java b/src/test/java/org/opensearch/plugin/insights/rules/action/live_queries/LiveQueriesResponseTests.java
new file mode 100644
index 00000000..5b841bf2
--- /dev/null
+++ b/src/test/java/org/opensearch/plugin/insights/rules/action/live_queries/LiveQueriesResponseTests.java
@@ -0,0 +1,98 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.action.live_queries;
+
+import static java.util.Collections.emptyMap;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+import org.opensearch.common.io.stream.BytesStreamOutput;
+import org.opensearch.common.xcontent.XContentFactory;
+import org.opensearch.common.xcontent.XContentHelper;
+import org.opensearch.common.xcontent.json.JsonXContent;
+import org.opensearch.core.common.io.stream.StreamInput;
+import org.opensearch.core.xcontent.ToXContent;
+import org.opensearch.core.xcontent.XContentBuilder;
+import org.opensearch.plugin.insights.rules.model.Attribute;
+import org.opensearch.plugin.insights.rules.model.Measurement;
+import org.opensearch.plugin.insights.rules.model.MetricType;
+import org.opensearch.plugin.insights.rules.model.SearchQueryRecord;
+import org.opensearch.test.OpenSearchTestCase;
+
+/**
+ * Unit tests for the {@link LiveQueriesResponse} class.
+ */
+public class LiveQueriesResponseTests extends OpenSearchTestCase {
+
+    private List<SearchQueryRecord> createLiveQueriesList(int count, long baseLatency) {
+        return IntStream.range(0, count).mapToObj(i -> {
+            Map<MetricType, Measurement> measurements = new HashMap<>();
+            measurements.put(MetricType.LATENCY, new Measurement(baseLatency + i * 100));
+            measurements.put(MetricType.CPU, new Measurement(randomLongBetween(10, 1000)));
+            measurements.put(MetricType.MEMORY, new Measurement(randomLongBetween(1024, 10240)));
+            Map<Attribute, Object> attributes = new HashMap<>();
+            if (randomBoolean()) {
+                attributes.put(Attribute.DESCRIPTION, "desc_" + baseLatency + "_" + i);
+            }
+
+            return new SearchQueryRecord(System.currentTimeMillis(), measurements, attributes, "query_" + baseLatency + "_" + i);
+        }).collect(Collectors.toList());
+    }
+
+    public void testSerialization() throws IOException {
+        List<SearchQueryRecord> queries = createLiveQueriesList(3, 1000);
+        LiveQueriesResponse originalResponse = new LiveQueriesResponse(queries);
+        BytesStreamOutput out = new BytesStreamOutput();
+        originalResponse.writeTo(out);
+        StreamInput in = StreamInput.wrap(out.bytes().toBytesRef().bytes);
+        LiveQueriesResponse deserializedResponse = new LiveQueriesResponse(in);
+
+        assertEquals(originalResponse.getLiveQueries().size(), deserializedResponse.getLiveQueries().size());
+        assertEquals(originalResponse.getLiveQueries(), deserializedResponse.getLiveQueries());
+    }
+
+    public void testToXContent() throws IOException {
+        Map<MetricType, Measurement> measurements = Map.of(
+            MetricType.LATENCY,
+            new Measurement(10L),
+            MetricType.CPU,
+            new Measurement(20L),
+            MetricType.MEMORY,
+            new Measurement(30L)
+        );
+        SearchQueryRecord rec1 = new SearchQueryRecord(1L, measurements, emptyMap(), "id1");
+        SearchQueryRecord rec2 = new SearchQueryRecord(2L, measurements, emptyMap(), "id2");
+        SearchQueryRecord rec3 = new SearchQueryRecord(3L, measurements, emptyMap(), "id3");
+        List<SearchQueryRecord> records = List.of(rec2, rec1, rec3);
+        LiveQueriesResponse response = new LiveQueriesResponse(records);
+        XContentBuilder builder = XContentFactory.jsonBuilder();
+        response.toXContent(builder, ToXContent.EMPTY_PARAMS);
+        String json = builder.toString();
+        Map<String, Object> parsed = XContentHelper.convertToMap(JsonXContent.jsonXContent, json, false);
+        @SuppressWarnings("unchecked")
+        List<Map<String, Object>> liveQueriesList = (List<Map<String, Object>>) parsed.get("live_queries");
+        assertEquals(records.size(), liveQueriesList.size());
+        assertEquals("id2", liveQueriesList.get(0).get("id"));
+        assertEquals("id1", liveQueriesList.get(1).get("id"));
+        assertEquals("id3", liveQueriesList.get(2).get("id"));
+    }
+
+    public void testToXContentEmptyList() throws IOException {
+        LiveQueriesResponse response = new LiveQueriesResponse(Collections.emptyList());
+        XContentBuilder builder = XContentFactory.jsonBuilder();
+        response.toXContent(builder, ToXContent.EMPTY_PARAMS);
+        String json = builder.toString();
+        assertEquals("{\"live_queries\":[]}", json);
+    }
+}
diff --git a/src/test/java/org/opensearch/plugin/insights/rules/resthandler/live_queries/LiveQueriesRestIT.java b/src/test/java/org/opensearch/plugin/insights/rules/resthandler/live_queries/LiveQueriesRestIT.java
new file mode 100644
index 00000000..8bd83bc8
--- /dev/null
+++ b/src/test/java/org/opensearch/plugin/insights/rules/resthandler/live_queries/LiveQueriesRestIT.java
@@ -0,0 +1,384 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.resthandler.live_queries;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.junit.Assert;
+import org.opensearch.client.Request;
+import org.opensearch.client.Response;
+import org.opensearch.common.xcontent.LoggingDeprecationHandler;
+import org.opensearch.common.xcontent.json.JsonXContent;
+import org.opensearch.core.xcontent.NamedXContentRegistry;
+import org.opensearch.plugin.insights.QueryInsightsRestTestCase;
+import org.opensearch.plugin.insights.settings.QueryInsightsSettings;
+
+/**
+ * Integration tests for Live Queries API
+ */
+public class LiveQueriesRestIT extends QueryInsightsRestTestCase {
+
+    private static final Logger logger = LogManager.getLogger(LiveQueriesRestIT.class);
+    private static final String TEST_INDEX = "test-index";
+    private static final int MAX_POLL_ATTEMPTS = 1000;  // Maximum number of times to poll the live queries API
+    private static final int POLL_INTERVAL_MS = 5;  // Time between polling attempts
+    private static final int CONCURRENT_QUERIES = 5;  // Number of concurrent queries to run
+    private static final int MAX_QUERY_ITERATION = 100; // Max number of times to run the queries
+    private static final int QUERY_DURATION_MS = 10000; // Time each query should run
+
+    /**
+     * Verify Query Insights plugin is installed
+     */
+    @SuppressWarnings("unchecked")
+    public void ensureQueryInsightsPluginInstalled() throws IOException {
+        Request request = new Request("GET", "/_cat/plugins?s=component&h=name,component,version,description&format=json");
+        Response response = client().performRequest(request);
+        List<Object> pluginsList = JsonXContent.jsonXContent.createParser(
+            NamedXContentRegistry.EMPTY,
+            LoggingDeprecationHandler.INSTANCE,
+            response.getEntity().getContent()
+        ).list();
+        Assert.assertTrue(
+            pluginsList.stream().map(o -> (Map<String, Object>) o).anyMatch(plugin -> plugin.get("component").equals("query-insights"))
+        );
+    }
+
+    /**
+     * Try to detect live queries by running multiple concurrent search operations and polling the API.
+     */
+    @SuppressWarnings("unchecked")
+    public void testLiveQueriesWithConcurrentSearches() throws Exception {
+        // Create index and add documents with some data
+        createIndexWithData(100);
+
+        // Set up a coordinator for the search threads
+        CountDownLatch startLatch = new CountDownLatch(1);
+        CountDownLatch completionLatch = new CountDownLatch(CONCURRENT_QUERIES);
+        AtomicBoolean shouldStop = new AtomicBoolean(false);
+        AtomicInteger completedQueries = new AtomicInteger(0);
+        ExecutorService threadPool = Executors.newFixedThreadPool(CONCURRENT_QUERIES);
+
+        // Create and submit search tasks that will run for a while
+        for (int i = 0; i < CONCURRENT_QUERIES; i++) {
+            final int queryNum = i;
+            threadPool.submit(() -> {
+                try {
+                    logger.info("Search thread {} ready and waiting to start", queryNum);
+                    startLatch.await(); // Wait for all threads to be ready
+                    String searchJson = generateComplexQuery(queryNum);
+                    Request searchRequest = new Request("GET", "/" + TEST_INDEX + "/_search");
+                    searchRequest.setJsonEntity(searchJson);
+
+                    // Set longer timeout to let the query run for a while
+                    searchRequest.addParameter("timeout", QUERY_DURATION_MS + "ms");
+
+                    try {
+                        logger.info("Search thread {} starting search", queryNum);
+                        for (int j = 0; j < MAX_QUERY_ITERATION; j++) {
+                            client().performRequest(searchRequest);
+                        }
+                        logger.info("Search thread {} completed successfully", queryNum);
+                    } catch (Exception e) {
+                        // We expect this might timeout or be cancelled
+                        logger.info("Search thread {} ended with: {}", queryNum, e.getMessage());
+                    }
+                } catch (Exception e) {
+                    logger.error("Error in search thread {}: {}", queryNum, e.getMessage());
+                } finally {
+                    completedQueries.incrementAndGet();
+                    completionLatch.countDown();
+                }
+            });
+        }
+
+        // Start all the search threads..
+        logger.info("Starting all search threads");
+        startLatch.countDown();
+
+        // Poll the Live Queries API repeatedly to try to catch the searches
+        logger.info("Beginning to poll the live queries API");
+        boolean foundLiveQueries = false;
+        List<Map<String, Object>> liveQueries = new ArrayList<>();
+        Response nodesRes = client().performRequest(new Request("GET", "/_nodes"));
+        Map<String, Object> nodesMap = entityAsMap(nodesRes);
+        Map<String, Object> nodes = (Map<String, Object>) nodesMap.get("nodes");
+        String nodeId = nodes.keySet().iterator().next();
+
+        String[] params = new String[] { "?size=1", "", "?size=0", "?sort=cpu", "?verbose=false", "?nodeId=" + nodeId };
+        Map<String, Boolean> foundParams = new java.util.HashMap<>();
+        for (String param : params) {
+            foundParams.put(param, false);
+        }
+
+        // Create a separate thread to poll the API while search queries are running
+        for (int attempt = 0; attempt < MAX_POLL_ATTEMPTS && !shouldStop.get(); attempt++) {
+            Thread.sleep(POLL_INTERVAL_MS);
+
+            // Skip further checking if all queries are done already
+            if (completedQueries.get() >= CONCURRENT_QUERIES) {
+                logger.info("All queries completed, stopping poll attempts");
+                break;
+            }
+
+            try {
+                // Call the Live Queries API
+                Request liveQueriesRequest = new Request("GET", QueryInsightsSettings.LIVE_QUERIES_BASE_URI + "?verbose=true");
+                Response liveQueriesResponse = client().performRequest(liveQueriesRequest);
+
+                // Parse response
+                Map<String, Object> responseMap = entityAsMap(liveQueriesResponse);
+                assertTrue(responseMap.containsKey("live_queries"));
+                liveQueries = (List<Map<String, Object>>) responseMap.get("live_queries");
+
+                logger.info("Polling attempt {}: Found {} live queries", attempt, liveQueries.size());
+
+                if (!liveQueries.isEmpty()) {
+                    foundLiveQueries = true;
+                }
+                // Run parameter tests on each polling cycle
+                for (String param : params) {
+                    if (!foundParams.get(param)) {
+                        Request pReq = new Request("GET", QueryInsightsSettings.LIVE_QUERIES_BASE_URI + param);
+                        Response pRes = client().performRequest(pReq);
+                        assertEquals(200, pRes.getStatusLine().getStatusCode());
+                        Map<String, Object> pMap = entityAsMap(pRes);
+                        assertTrue(pMap.containsKey("live_queries"));
+                        List<?> pList = (List<?>) pMap.get("live_queries");
+                        boolean ok;
+                        if ("?size=0".equals(param)) {
+                            ok = pList.isEmpty();
+                        } else if ("?verbose=false".equals(param)) {
+                            ok = pList.stream().allMatch(q -> !((Map<String, Object>) q).containsKey("description"));
+                        } else if (param.startsWith("?nodeId=")) {
+                            String filterNode = param.substring("?nodeId=".length());
+                            ok = pList.stream().allMatch(q -> filterNode.equals(((Map<String, Object>) q).get("node_id")));
+                        } else {
+                            ok = !pList.isEmpty();
+                        }
+                        if (ok) {
+                            foundParams.put(param, true);
+                        }
+                    }
+                }
+                // Break only when main and all param tests have passed
+                if (foundLiveQueries && !foundParams.containsValue(false)) {
+                    logger.info("All checks succeeded by attempt {}", attempt);
+                    break;
+                }
+            } catch (Exception e) {
+                logger.error("Error polling live queries API: {}", e.getMessage());
+            }
+        }
+
+        // Signal all threads to stop
+        shouldStop.set(true);
+
+        // Wait for all search threads to complete with timeout
+        boolean allThreadsCompleted = completionLatch.await(QUERY_DURATION_MS * 2, TimeUnit.MILLISECONDS);
+        logger.info("All threads completed? {}", allThreadsCompleted);
+
+        // Shut down executor
+        threadPool.shutdownNow();
+
+        // We either found live queries or exhausted our polling attempts.
+        if (foundLiveQueries) {
+            logger.info("Test detected live queries, total: {}", liveQueries.size());
+            assertTrue("Should have found at least one live query", !liveQueries.isEmpty());
+
+            // Validate the format of live queries based on LiveQueries.java and LiveQueriesResponse.java
+            for (Map<String, Object> query : liveQueries) {
+                // Verify required fields are present
+                assertTrue("Query should have timestamp", query.containsKey("timestamp"));
+                assertTrue("Query should have id", query.containsKey("id"));
+                assertTrue("Query should have node_id", query.containsKey("node_id"));
+                assertTrue("Query should have measurements", query.containsKey("measurements"));
+                assertTrue("Query should have description", query.containsKey("description"));
+
+                // Validate timestamp is a number
+                assertTrue("Timestamp should be a number", query.get("timestamp") instanceof Number);
+
+                // Validate id is a string
+                assertTrue("ID should be a string", query.get("id") instanceof String);
+
+                // Validate node_id is a string
+                assertTrue("Node ID should be a string", query.get("node_id") instanceof String);
+
+                // Validate measurements structure
+                Map<String, Object> measurements = (Map<String, Object>) query.get("measurements");
+                assertTrue("Measurements should include latency", measurements.containsKey("latency"));
+                assertTrue("Measurements should include cpu", measurements.containsKey("cpu"));
+                assertTrue("Measurements should include memory", measurements.containsKey("memory"));
+
+                // Validate each measurement's structure
+                for (String metricType : new String[] { "latency", "cpu", "memory" }) {
+                    Map<String, Object> metric = (Map<String, Object>) measurements.get(metricType);
+                    assertTrue("Metric should have number", metric.containsKey("number"));
+                    assertTrue("Metric should have count", metric.containsKey("count"));
+                    assertTrue("Metric should have aggregationType", metric.containsKey("aggregationType"));
+
+                    // Validate number is a number
+                    assertTrue("Number should be a number", metric.get("number") instanceof Number);
+
+                    // Validate count is a number
+                    assertTrue("Count should be a number", metric.get("count") instanceof Number);
+
+                    // Validate aggregationType is a string
+                    assertTrue("AggregationType should be a string", metric.get("aggregationType") instanceof String);
+                }
+                assertTrue("Description should be a string", query.get("description") instanceof String);
+            }
+        } else {
+            fail("No live queries found.");
+        }
+        for (String param : params) {
+            assertTrue("Parameter test for '" + param + "' did not pass", foundParams.get(param));
+        }
+    }
+
+    /**
+     * Fallback tests: Basic test for all parameters including verbose, node filtering
+     */
+    @SuppressWarnings("unchecked")
+    public void testAllParameters() throws IOException {
+        // Retrieve one node ID for filtering tests
+        Request nodesRequest = new Request("GET", "/_nodes");
+        Response nodesResponse = client().performRequest(nodesRequest);
+        Map<String, Object> nodesMap = entityAsMap(nodesResponse);
+        Map<String, Object> nodes = (Map<String, Object>) nodesMap.get("nodes");
+        assertNotNull(nodes);
+        assertFalse(nodes.isEmpty());
+        String nodeId = nodes.keySet().iterator().next();
+
+        // Define parameter combinations to test
+        String[] params = new String[] { "", "?verbose=false", "?sort=cpu", "?size=1", "?size=0", "?nodeId=" + nodeId };
+        for (String param : params) {
+            String uri = QueryInsightsSettings.LIVE_QUERIES_BASE_URI + param;
+            Request req = new Request("GET", uri);
+            Response res = client().performRequest(req);
+            assertEquals("Status for param '" + param + "'", 200, res.getStatusLine().getStatusCode());
+            Map<String, Object> map = entityAsMap(res);
+            assertTrue("Response should contain live_queries for param '" + param + "'", map.containsKey("live_queries"));
+        }
+    }
+
+    /**
+     * Create a test index with the specified number of documents
+     */
+    private void createIndexWithData(int numDocs) throws IOException {
+        // Create test index
+        createDocument();
+
+        // Add more documents
+        for (int i = 1; i < numDocs; i++) {
+            Request indexRequest = new Request("POST", "/" + TEST_INDEX + "/_doc");
+            String docJson = String.format(
+                Locale.ROOT,
+                "{\"title\":\"Document %d\",\"value\":%d,\"tags\":[\"tag1\",\"tag2\",\"tag%d\"],\"nested\":{\"field1\":\"value%d\",\"field2\":%d}}",
+                i,
+                i % 100,
+                i % 10,
+                i,
+                i * 2
+            );
+            indexRequest.setJsonEntity(docJson);
+            client().performRequest(indexRequest);
+
+            // Occasionally refresh to make documents searchable
+            if (i % 1000 == 0) {
+                Request refreshRequest = new Request("POST", "/" + TEST_INDEX + "/_refresh");
+                client().performRequest(refreshRequest);
+            }
+        }
+
+        // Final refresh to ensure all documents are searchable
+        Request refreshRequest = new Request("POST", "/" + TEST_INDEX + "/_refresh");
+        client().performRequest(refreshRequest);
+        logger.info("Created index with {} documents", numDocs);
+    }
+
+    /**
+     * Generate a complex search query that should take a while to execute
+     */
+    private String generateComplexQuery(int queryNum) {
+        // Use different query patterns for more diversity
+        switch (queryNum % 3) {
+            case 0:
+                // Complex aggregation query
+                return "{\n"
+                    + "  \"size\": 1000,\n"
+                    + "  \"query\": { \"match_all\": {} },\n"
+                    + "  \"aggs\": {\n"
+                    + "    \"value_ranges\": {\n"
+                    + "      \"range\": {\n"
+                    + "        \"field\": \"value\",\n"
+                    + "        \"ranges\": [\n"
+                    + "          { \"to\": 20 },\n"
+                    + "          { \"from\": 20, \"to\": 40 },\n"
+                    + "          { \"from\": 40, \"to\": 60 },\n"
+                    + "          { \"from\": 60, \"to\": 80 },\n"
+                    + "          { \"from\": 80 }\n"
+                    + "        ]\n"
+                    + "      },\n"
+                    + "      \"aggs\": {\n"
+                    + "        \"tag_counts\": {\n"
+                    + "          \"terms\": { \"field\": \"tags\", \"size\": 100 }\n"
+                    + "        }\n"
+                    + "      }\n"
+                    + "    }\n"
+                    + "  },\n"
+                    + "  \"sort\": [\"_doc\"]\n"
+                    + "}";
+            case 1:
+                // Query with complex filtering
+                return "{\n"
+                    + "  \"size\": 1000,\n"
+                    + "  \"query\": {\n"
+                    + "    \"bool\": {\n"
+                    + "      \"must\": [\n"
+                    + "        { \"wildcard\": { \"title\": \"*Document*\" } }\n"
+                    + "      ],\n"
+                    + "      \"filter\": [\n"
+                    + "        { \"range\": { \"value\": { \"gte\": 10, \"lte\": 90 } } },\n"
+                    + "        { \"term\": { \"tags\": \"tag2\" } }\n"
+                    + "      ],\n"
+                    + "      \"should\": [\n"
+                    + "        { \"term\": { \"nested.field1\": \"value5\" } },\n"
+                    + "        { \"term\": { \"nested.field1\": \"value10\" } },\n"
+                    + "        { \"term\": { \"nested.field1\": \"value15\" } }\n"
+                    + "      ],\n"
+                    + "      \"minimum_should_match\": 1\n"
+                    + "    }\n"
+                    + "  },\n"
+                    + "  \"sort\": [\"_doc\"]\n"
+                    + "}";
+            default:
+                // Query with complex sorting
+                return "{\n"
+                    + "  \"size\": 1000,\n"
+                    + "  \"query\": { \"match_all\": {} },\n"
+                    + "  \"sort\": [\n"
+                    + "    { \"value\": { \"order\": \"desc\" } },\n"
+                    + "    { \"_score\": { \"order\": \"desc\" } },\n"
+                    + "    \"_doc\"\n"
+                    + "  ]\n"
+                    + "}";
+        }
+    }
+}
diff --git a/src/test/java/org/opensearch/plugin/insights/rules/resthandler/live_queries/RestLiveQueriesActionTests.java b/src/test/java/org/opensearch/plugin/insights/rules/resthandler/live_queries/RestLiveQueriesActionTests.java
new file mode 100644
index 00000000..f9593561
--- /dev/null
+++ b/src/test/java/org/opensearch/plugin/insights/rules/resthandler/live_queries/RestLiveQueriesActionTests.java
@@ -0,0 +1,167 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.resthandler.live_queries;
+
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import org.junit.Before;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesRequest;
+import org.opensearch.plugin.insights.rules.model.MetricType;
+import org.opensearch.plugin.insights.settings.QueryInsightsSettings;
+import org.opensearch.rest.BaseRestHandler;
+import org.opensearch.rest.RestChannel;
+import org.opensearch.rest.RestRequest;
+import org.opensearch.test.OpenSearchTestCase;
+import org.opensearch.test.rest.FakeRestRequest;
+
+/**
+ * Unit tests for the {@link RestLiveQueriesAction} class.
+ */
+public class RestLiveQueriesActionTests extends OpenSearchTestCase {
+
+    private RestLiveQueriesAction restLiveQueriesAction;
+
+    @Before
+    public void setUp() throws Exception {
+        super.setUp();
+        restLiveQueriesAction = new RestLiveQueriesAction();
+        RestChannel channel = mock(RestChannel.class);
+        when(channel.newBuilder()).thenThrow(new AssertionError("Should not be called in prepareRequest test"));
+    }
+
+    public void testRoutes() {
+        List<BaseRestHandler.Route> routes = restLiveQueriesAction.routes();
+        assertEquals(1, routes.size());
+        assertEquals(RestRequest.Method.GET, routes.getFirst().getMethod());
+        assertEquals(QueryInsightsSettings.LIVE_QUERIES_BASE_URI, routes.getFirst().getPath());
+    }
+
+    public void testGetName() {
+        assertEquals("query_insights_live_queries_action", restLiveQueriesAction.getName());
+    }
+
+    public void testPrepareRequestWithNodeIds() {
+        Map<String, String> params = Map.of("nodeId", "node1,node2", "verbose", "false");
+        RestRequest request = new FakeRestRequest.Builder(xContentRegistry()).withPath("/_insights/live_queries/node1,node2")
+            .withParams(params)
+            .withMethod(RestRequest.Method.GET)
+            .build();
+
+        LiveQueriesRequest liveQueriesRequest = RestLiveQueriesAction.prepareRequest(request);
+        assertEquals(2, liveQueriesRequest.nodesIds().length);
+        assertEquals("node1", liveQueriesRequest.nodesIds()[0]);
+        assertEquals("node2", liveQueriesRequest.nodesIds()[1]);
+        assertFalse(liveQueriesRequest.isVerbose());
+    }
+
+    public void testPrepareRequestWithoutNodeIds() {
+        Map<String, String> params = Map.of("verbose", "true");
+        RestRequest request = new FakeRestRequest.Builder(xContentRegistry()).withPath("/_insights/live_queries")
+            .withParams(params)
+            .withMethod(RestRequest.Method.GET)
+            .build();
+
+        LiveQueriesRequest liveQueriesRequest = RestLiveQueriesAction.prepareRequest(request);
+        assertEquals(0, liveQueriesRequest.nodesIds().length); // Expect cluster-wide
+        assertTrue(liveQueriesRequest.isVerbose());
+    }
+
+    public void testPrepareRequestDefaults() {
+        RestRequest request = new FakeRestRequest.Builder(xContentRegistry()).withPath("/_insights/live_queries").build();
+
+        LiveQueriesRequest liveQueriesRequest = RestLiveQueriesAction.prepareRequest(request);
+        assertEquals(0, liveQueriesRequest.nodesIds().length); // Expect cluster-wide
+        assertTrue(liveQueriesRequest.isVerbose()); // Default verbose is true
+    }
+
+    public void testPrepareRequestWithCustomParams() {
+        Map<String, String> params = Map.of("nodeId", "node1", "verbose", "false", "sort", "cpu", "size", "3");
+        RestRequest request = new FakeRestRequest.Builder(xContentRegistry()).withPath(
+            QueryInsightsSettings.LIVE_QUERIES_BASE_URI + "/node1"
+        ).withParams(params).withMethod(RestRequest.Method.GET).build();
+        LiveQueriesRequest req = RestLiveQueriesAction.prepareRequest(request);
+        assertArrayEquals(new String[] { "node1" }, req.nodesIds());
+        assertFalse(req.isVerbose());
+        assertEquals(MetricType.CPU, req.getSortBy());
+        assertEquals(3, req.getSize());
+    }
+
+    public void testDefaultSortAndSize() {
+        RestRequest request = new FakeRestRequest.Builder(xContentRegistry()).withPath(QueryInsightsSettings.LIVE_QUERIES_BASE_URI)
+            .withMethod(RestRequest.Method.GET)
+            .build();
+        LiveQueriesRequest req = RestLiveQueriesAction.prepareRequest(request);
+        assertEquals(MetricType.LATENCY, req.getSortBy());
+        assertEquals(QueryInsightsSettings.DEFAULT_LIVE_QUERIES_SIZE, req.getSize());
+    }
+
+    public void testPrepareRequestInvalidSort() {
+        Map<String, String> params = Map.of("sort", "invalid_metric");
+        RestRequest request = new FakeRestRequest.Builder(xContentRegistry()).withPath(QueryInsightsSettings.LIVE_QUERIES_BASE_URI)
+            .withParams(params)
+            .withMethod(RestRequest.Method.GET)
+            .build();
+        assertThrows(IllegalArgumentException.class, () -> RestLiveQueriesAction.prepareRequest(request));
+    }
+
+    public void testPrepareRequestWithSortOnly() {
+        Map<String, String> params = Map.of("sort", "memory");
+        RestRequest request = new FakeRestRequest.Builder(xContentRegistry()).withPath(QueryInsightsSettings.LIVE_QUERIES_BASE_URI)
+            .withParams(params)
+            .withMethod(RestRequest.Method.GET)
+            .build();
+        LiveQueriesRequest req = RestLiveQueriesAction.prepareRequest(request);
+        assertEquals(MetricType.MEMORY, req.getSortBy());
+        assertEquals(100, req.getSize());
+    }
+
+    public void testPrepareRequestWithSizeOnly() {
+        Map<String, String> params = Map.of("size", "5");
+        RestRequest request = new FakeRestRequest.Builder(xContentRegistry()).withPath(QueryInsightsSettings.LIVE_QUERIES_BASE_URI)
+            .withParams(params)
+            .withMethod(RestRequest.Method.GET)
+            .build();
+        LiveQueriesRequest req = RestLiveQueriesAction.prepareRequest(request);
+        assertEquals(MetricType.LATENCY, req.getSortBy());
+        assertEquals(5, req.getSize());
+    }
+
+    public void testPrepareRequestWithZeroSize() {
+        Map<String, String> params = Map.of("size", "0");
+        RestRequest request = new FakeRestRequest.Builder(xContentRegistry()).withPath(QueryInsightsSettings.LIVE_QUERIES_BASE_URI)
+            .withParams(params)
+            .withMethod(RestRequest.Method.GET)
+            .build();
+        LiveQueriesRequest req = RestLiveQueriesAction.prepareRequest(request);
+        assertEquals(0, req.getSize());
+    }
+
+    public void testPrepareRequestInvalidVerbose() {
+        // non-boolean verbose parameter
+        Map<String, String> params = Map.of("verbose", "notABoolean");
+        RestRequest request = new FakeRestRequest.Builder(xContentRegistry()).withPath(QueryInsightsSettings.LIVE_QUERIES_BASE_URI)
+            .withParams(params)
+            .withMethod(RestRequest.Method.GET)
+            .build();
+        assertThrows(IllegalArgumentException.class, () -> RestLiveQueriesAction.prepareRequest(request));
+    }
+
+    public void testPrepareRequestInvalidSize() {
+        // non-numeric size parameter
+        Map<String, String> params = Map.of("size", "notANumber");
+        RestRequest request = new FakeRestRequest.Builder(xContentRegistry()).withPath(QueryInsightsSettings.LIVE_QUERIES_BASE_URI)
+            .withParams(params)
+            .withMethod(RestRequest.Method.GET)
+            .build();
+        assertThrows(IllegalArgumentException.class, () -> RestLiveQueriesAction.prepareRequest(request));
+    }
+}
diff --git a/src/test/java/org/opensearch/plugin/insights/rules/resthandler/top_queries/TopQueriesRestIT.java b/src/test/java/org/opensearch/plugin/insights/rules/resthandler/top_queries/TopQueriesRestIT.java
index 726c0684..68c1b407 100644
--- a/src/test/java/org/opensearch/plugin/insights/rules/resthandler/top_queries/TopQueriesRestIT.java
+++ b/src/test/java/org/opensearch/plugin/insights/rules/resthandler/top_queries/TopQueriesRestIT.java
@@ -47,22 +47,20 @@ public void testQueryInsightsPluginInstalled() throws IOException {
      * @throws IOException IOException
      */
     public void testTopQueriesResponses() throws IOException, InterruptedException {
-        waitForEmptyTopQueriesResponse();
-
         // Enable only Top N Queries by latency feature
         updateClusterSettings(this::defaultTopQueriesSettings);
 
-        doSearch(2);
+        doSearch(10);
 
-        assertTopQueriesCount(2, "latency");
+        assertTopQueriesCount(5, "latency");
 
         // Enable Top N Queries by resource usage
         updateClusterSettings(this::topQueriesByResourceUsagesSettings);
 
         // Do Search
-        doSearch(2);
+        doSearch(10);
 
-        assertTopQueriesCount(2, "cpu");
+        assertTopQueriesCount(5, "cpu");
     }
 
     /**
diff --git a/src/test/java/org/opensearch/plugin/insights/rules/transport/live_queries/TransportLiveQueriesActionTests.java b/src/test/java/org/opensearch/plugin/insights/rules/transport/live_queries/TransportLiveQueriesActionTests.java
new file mode 100644
index 00000000..1ed6a388
--- /dev/null
+++ b/src/test/java/org/opensearch/plugin/insights/rules/transport/live_queries/TransportLiveQueriesActionTests.java
@@ -0,0 +1,417 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.transport.live_queries;
+
+import static java.util.Collections.emptyList;
+import static java.util.Collections.emptyMap;
+import static java.util.Collections.emptySet;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.Mockito.doAnswer;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import org.junit.Before;
+import org.mockito.ArgumentCaptor;
+import org.opensearch.action.admin.cluster.node.tasks.list.ListTasksRequest;
+import org.opensearch.action.admin.cluster.node.tasks.list.ListTasksResponse;
+import org.opensearch.action.support.ActionFilters;
+import org.opensearch.action.support.PlainActionFuture;
+import org.opensearch.cluster.ClusterName;
+import org.opensearch.cluster.ClusterState;
+import org.opensearch.cluster.node.DiscoveryNode;
+import org.opensearch.cluster.node.DiscoveryNodes;
+import org.opensearch.cluster.service.ClusterService;
+import org.opensearch.common.settings.Settings;
+import org.opensearch.core.action.ActionListener;
+import org.opensearch.core.tasks.TaskId;
+import org.opensearch.core.tasks.resourcetracker.TaskResourceStats;
+import org.opensearch.core.tasks.resourcetracker.TaskResourceUsage;
+import org.opensearch.core.tasks.resourcetracker.TaskThreadUsage;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesRequest;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesResponse;
+import org.opensearch.plugin.insights.rules.model.Attribute;
+import org.opensearch.plugin.insights.rules.model.MetricType;
+import org.opensearch.plugin.insights.rules.model.SearchQueryRecord;
+import org.opensearch.tasks.TaskInfo;
+import org.opensearch.test.OpenSearchTestCase;
+import org.opensearch.test.VersionUtils;
+import org.opensearch.test.tasks.MockTaskManager;
+import org.opensearch.threadpool.ThreadPool;
+import org.opensearch.transport.TransportService;
+import org.opensearch.transport.client.AdminClient;
+import org.opensearch.transport.client.Client;
+import org.opensearch.transport.client.ClusterAdminClient;
+
+/**
+ * Unit tests for the {@link TransportLiveQueriesAction} class.
+ */
+@SuppressWarnings("unchecked")
+public class TransportLiveQueriesActionTests extends OpenSearchTestCase {
+
+    private TransportLiveQueriesAction transportLiveQueriesAction;
+    private Client client;
+    private ClusterService clusterService;
+    private DiscoveryNode node1;
+    private DiscoveryNode node2;
+    private ThreadPool threadPool;
+    private TransportService transportService;
+    private ActionFilters actionFilters;
+    private AdminClient adminClient;
+    private ClusterAdminClient clusterAdminClient;
+
+    @Before
+    public void setup() {
+        threadPool = mock(ThreadPool.class);
+        clusterService = mock(ClusterService.class);
+        transportService = mock(TransportService.class);
+        MockTaskManager taskManager = new MockTaskManager(Settings.EMPTY, threadPool, Collections.emptySet());
+        when(transportService.getTaskManager()).thenReturn(taskManager);
+        client = mock(Client.class, org.mockito.Answers.RETURNS_DEEP_STUBS); // Use deep stubs for client.admin().cluster()
+        actionFilters = mock(ActionFilters.class);
+        when(actionFilters.filters()).thenReturn(new org.opensearch.action.support.ActionFilter[0]);
+
+        node1 = new DiscoveryNode("node1", buildNewFakeTransportAddress(), emptyMap(), emptySet(), VersionUtils.randomVersion(random()));
+        node2 = new DiscoveryNode("node2", buildNewFakeTransportAddress(), emptyMap(), emptySet(), VersionUtils.randomVersion(random()));
+
+        when(clusterService.localNode()).thenReturn(node1); // Assume node1 is local for simplicity
+        when(clusterService.getClusterName()).thenReturn(new ClusterName("test-cluster"));
+        // Stub cluster state to include the nodes
+        DiscoveryNodes discoveryNodes = DiscoveryNodes.builder().add(node1).add(node2).localNodeId(node1.getId()).build();
+        ClusterState clusterState = ClusterState.builder(clusterService.getClusterName()).nodes(discoveryNodes).build();
+        when(clusterService.state()).thenReturn(clusterState);
+
+        // Mock the client administrative calls
+        adminClient = mock(AdminClient.class);
+        clusterAdminClient = mock(ClusterAdminClient.class);
+        when(client.admin()).thenReturn(adminClient);
+        when(adminClient.cluster()).thenReturn(clusterAdminClient);
+
+        transportLiveQueriesAction = new TransportLiveQueriesAction(transportService, client, actionFilters);
+    }
+
+    private TaskInfo createTaskInfo(
+        DiscoveryNode node,
+        String action,
+        long startTime,
+        long runningTimeNanos,
+        String description,
+        long cpuNanos,
+        long memoryBytes
+    ) throws IOException {
+        TaskId taskId = new TaskId(node.getId(), randomLong());
+
+        Map<String, TaskResourceUsage> resourceUsageInfoMap = new HashMap<>();
+        TaskResourceUsage totalUsage = new TaskResourceUsage(cpuNanos, memoryBytes);
+        resourceUsageInfoMap.put("total", totalUsage);
+
+        TaskResourceStats resourceStats = new TaskResourceStats(resourceUsageInfoMap, new TaskThreadUsage(0, 0));
+
+        return new TaskInfo(
+            taskId,
+            "test_type",
+            action,
+            description,
+            null,
+            startTime,
+            runningTimeNanos,
+            true,
+            false,
+            TaskId.EMPTY_TASK_ID,
+            Collections.singletonMap("foo", "bar"),
+            resourceStats
+        );
+    }
+
+    public void testNodeOperationVerbose_AllNodes() throws IOException {
+        // Request all nodes (no specific nodeIds)
+        LiveQueriesRequest request = new LiveQueriesRequest(true);
+        TaskInfo task1 = createTaskInfo(node1, "indices:data/read/search", System.currentTimeMillis(), 1000000L, "desc1", 500L, 1024L);
+        TaskInfo task2 = createTaskInfo(node2, "indices:data/read/search", System.currentTimeMillis(), 2000000L, "desc2", 600L, 2048L);
+        TaskInfo nonSearchTask = createTaskInfo(
+            node1,
+            "cluster:monitor/nodes/tasks/list",
+            System.currentTimeMillis(),
+            3000000L,
+            "desc3",
+            700L,
+            4096L
+        );
+        List<TaskInfo> tasks = List.of(task1, task2, nonSearchTask);
+        ListTasksResponse listTasksResponse = new ListTasksResponse(tasks, emptyList(), emptyList());
+
+        // Mock the listTasks asynchronous call
+        doAnswer(invocation -> {
+            ActionListener<ListTasksResponse> listener = invocation.getArgument(1);
+            listener.onResponse(listTasksResponse);
+            return null;
+        }).when(clusterAdminClient).listTasks(any(ListTasksRequest.class), any(ActionListener.class));
+
+        // Execute
+        PlainActionFuture<LiveQueriesResponse> futureResponse = PlainActionFuture.newFuture();
+        transportLiveQueriesAction.execute(request, futureResponse);
+        LiveQueriesResponse clusterResponse = futureResponse.actionGet();
+        List<SearchQueryRecord> records = clusterResponse.getLiveQueries();
+
+        // Verify listTasks was called and DID NOT have node filter set
+        ArgumentCaptor<ListTasksRequest> captor = ArgumentCaptor.forClass(ListTasksRequest.class);
+        verify(clusterAdminClient).listTasks(captor.capture(), any(ActionListener.class));
+        assertEquals(0, captor.getValue().getNodes().length);
+
+        // Verify results - should have 2 search tasks in the flat list
+        assertEquals(2, records.size());
+        Map<String, SearchQueryRecord> resultsById = records.stream().collect(Collectors.toMap(SearchQueryRecord::getId, r -> r));
+
+        assertTrue(resultsById.containsKey(task1.getTaskId().toString()));
+        SearchQueryRecord record1 = resultsById.get(task1.getTaskId().toString());
+        assertEquals(task1.getStartTime(), record1.getTimestamp());
+        assertEquals("desc1", record1.getAttributes().get(Attribute.DESCRIPTION));
+        assertEquals(1000000L, record1.getMeasurements().get(MetricType.LATENCY).getMeasurement().longValue());
+        assertEquals(500L, record1.getMeasurements().get(MetricType.CPU).getMeasurement().longValue());
+        assertEquals(1024L, record1.getMeasurements().get(MetricType.MEMORY).getMeasurement().longValue());
+
+        assertTrue(resultsById.containsKey(task2.getTaskId().toString()));
+        SearchQueryRecord record2 = resultsById.get(task2.getTaskId().toString());
+        assertEquals(task2.getStartTime(), record2.getTimestamp());
+        assertEquals("desc2", record2.getAttributes().get(Attribute.DESCRIPTION));
+    }
+
+    public void testNodeFiltering_SpecificNode() throws IOException {
+        // Request only node2
+        LiveQueriesRequest request = new LiveQueriesRequest(true, "node2");
+        // Prepare response as if listTasks only returned tasks for node2
+        TaskInfo task2 = createTaskInfo(node2, "indices:data/read/search", System.currentTimeMillis(), 2000000L, "desc2", 600L, 2048L);
+        List<TaskInfo> tasks = List.of(task2);
+        ListTasksResponse listTasksResponse = new ListTasksResponse(tasks, emptyList(), emptyList());
+
+        // Mock the listTasks asynchronous call
+        doAnswer(invocation -> {
+            ListTasksRequest listRequest = invocation.getArgument(0);
+            // Simulate upstream filtering: only return tasks if node matches
+            if (listRequest.getNodes() != null && listRequest.getNodes().length == 1 && listRequest.getNodes()[0].equals("node2")) {
+                ActionListener<ListTasksResponse> listener = invocation.getArgument(1);
+                listener.onResponse(listTasksResponse);
+            } else {
+                ActionListener<ListTasksResponse> listener = invocation.getArgument(1);
+                // Return empty if node doesn't match
+                listener.onResponse(new ListTasksResponse(Collections.emptyList(), Collections.emptyList(), Collections.emptyList()));
+            }
+            return null;
+        }).when(clusterAdminClient).listTasks(any(ListTasksRequest.class), any(ActionListener.class));
+
+        // Execute
+        PlainActionFuture<LiveQueriesResponse> futureResponse = PlainActionFuture.newFuture();
+        transportLiveQueriesAction.execute(request, futureResponse);
+        LiveQueriesResponse clusterResponse = futureResponse.actionGet();
+        List<SearchQueryRecord> records = clusterResponse.getLiveQueries();
+
+        // Verify listTasks was called and HAD node filter set
+        ArgumentCaptor<ListTasksRequest> captor = ArgumentCaptor.forClass(ListTasksRequest.class);
+        verify(clusterAdminClient).listTasks(captor.capture(), any(ActionListener.class));
+        assertArrayEquals(new String[] { "node2" }, captor.getValue().getNodes());
+
+        // Verify results - should only have task2 from node2
+        assertEquals(1, records.size());
+        SearchQueryRecord record2 = records.get(0);
+        assertEquals(task2.getTaskId().toString(), record2.getId());
+        assertEquals("desc2", record2.getAttributes().get(Attribute.DESCRIPTION));
+    }
+
+    public void testNodeOperationNonVerbose() throws IOException {
+        // Request non-verbose for node1
+        LiveQueriesRequest request = new LiveQueriesRequest(false, "node1");
+        TaskInfo task1 = createTaskInfo(node1, "indices:data/read/search", System.currentTimeMillis(), 1000000L, "desc1", 500L, 1024L);
+        List<TaskInfo> tasks = List.of(task1);
+        ListTasksResponse listTasksResponse = new ListTasksResponse(tasks, emptyList(), emptyList());
+
+        doAnswer(invocation -> {
+            ListTasksRequest listRequest = invocation.getArgument(0);
+            // Simulate upstream filtering: only return tasks if node matches
+            if (listRequest.getNodes() != null && listRequest.getNodes().length == 1 && listRequest.getNodes()[0].equals("node1")) {
+                ActionListener<ListTasksResponse> listener = invocation.getArgument(1);
+                listener.onResponse(listTasksResponse);
+            } else {
+                ActionListener<ListTasksResponse> listener = invocation.getArgument(1);
+                // Return empty if node doesn't match
+                listener.onResponse(new ListTasksResponse(Collections.emptyList(), Collections.emptyList(), Collections.emptyList()));
+            }
+            return null;
+        }).when(clusterAdminClient).listTasks(any(ListTasksRequest.class), any(ActionListener.class));
+
+        PlainActionFuture<LiveQueriesResponse> futureResponse = PlainActionFuture.newFuture();
+        transportLiveQueriesAction.execute(request, futureResponse);
+        LiveQueriesResponse clusterResponse = futureResponse.actionGet();
+        List<SearchQueryRecord> records = clusterResponse.getLiveQueries();
+
+        // Verify listTasks was called and HAD node filter set
+        ArgumentCaptor<ListTasksRequest> captor = ArgumentCaptor.forClass(ListTasksRequest.class);
+        verify(clusterAdminClient).listTasks(captor.capture(), any(ActionListener.class));
+        assertArrayEquals(new String[] { "node1" }, captor.getValue().getNodes());
+
+        assertEquals(1, records.size());
+        SearchQueryRecord record = records.get(0);
+        assertEquals(task1.getTaskId().toString(), record.getId());
+        assertNull(record.getAttributes().get(Attribute.DESCRIPTION)); // Description should be null
+        assertEquals(1000000L, record.getMeasurements().get(MetricType.LATENCY).getMeasurement().longValue());
+    }
+
+    public void testNodeOperationHandlesException() {
+        LiveQueriesRequest request = new LiveQueriesRequest(true);
+
+        doAnswer(invocation -> {
+            ActionListener<ListTasksResponse> listener = invocation.getArgument(1);
+            listener.onFailure(new RuntimeException("Simulated task fetch error"));
+            return null;
+        }).when(clusterAdminClient).listTasks(any(ListTasksRequest.class), any(ActionListener.class));
+
+        PlainActionFuture<LiveQueriesResponse> futureResponse = PlainActionFuture.newFuture();
+        transportLiveQueriesAction.execute(request, futureResponse);
+        expectThrows(RuntimeException.class, futureResponse::actionGet);
+    }
+
+    public void testResponseConstructor() {
+        List<SearchQueryRecord> records = Collections.emptyList();
+        LiveQueriesResponse response = new LiveQueriesResponse(records);
+        assertNotNull(response);
+        assertEquals(records, response.getLiveQueries());
+    }
+
+    public void testTransportActionSortsByCpuAndLimitsSize() throws IOException {
+        // Prepare a request to sort by CPU and limit to 1 result
+        LiveQueriesRequest request = new LiveQueriesRequest(true, MetricType.CPU, 1);
+        // Create tasks with different CPU values
+        TaskInfo lowCpu = createTaskInfo(node1, "indices:data/read/search", System.currentTimeMillis(), 1000L, "low", 100L, 100L);
+        TaskInfo highCpu = createTaskInfo(node1, "indices:data/read/search", System.currentTimeMillis(), 2000L, "high", 200L, 200L);
+        TaskInfo midCpu = createTaskInfo(node1, "indices:data/read/search", System.currentTimeMillis(), 1500L, "mid", 150L, 150L);
+        List<TaskInfo> tasks = List.of(lowCpu, highCpu, midCpu);
+        ListTasksResponse listTasksResponse = new ListTasksResponse(tasks, emptyList(), emptyList());
+        // Mock the async call
+        doAnswer(invocation -> {
+            ActionListener<ListTasksResponse> listener = invocation.getArgument(1);
+            listener.onResponse(listTasksResponse);
+            return null;
+        }).when(clusterAdminClient).listTasks(any(ListTasksRequest.class), any(ActionListener.class));
+
+        // Execute transport action
+        PlainActionFuture<LiveQueriesResponse> future = PlainActionFuture.newFuture();
+        transportLiveQueriesAction.execute(request, future);
+        LiveQueriesResponse response = future.actionGet();
+        List<SearchQueryRecord> records = response.getLiveQueries();
+
+        // Should only have the highCpu task
+        assertEquals(1, records.size());
+        assertEquals(highCpu.getTaskId().toString(), records.get(0).getId());
+    }
+
+    // Safeguard tests: missing or incomplete resource stats should default CPU/memory to 0
+    public void testNullResourceStatsDefaultsToZero() throws Exception {
+        LiveQueriesRequest request = new LiveQueriesRequest(true);
+        // Create a TaskInfo with null resourceStats
+        TaskInfo info = new TaskInfo(
+            new TaskId(node1.getId(), randomLong()),
+            "test_type",
+            "indices:data/read/search",
+            "desc",
+            null,
+            123L,
+            456L,
+            true,
+            false,
+            TaskId.EMPTY_TASK_ID,
+            Collections.emptyMap(),
+            null // resourceStats is null
+        );
+        ListTasksResponse listTasksResponse = new ListTasksResponse(List.of(info), emptyList(), emptyList());
+        doAnswer(invocation -> {
+            ActionListener<ListTasksResponse> listener = invocation.getArgument(1);
+            listener.onResponse(listTasksResponse);
+            return null;
+        }).when(clusterAdminClient).listTasks(any(ListTasksRequest.class), any(ActionListener.class));
+
+        PlainActionFuture<LiveQueriesResponse> future = PlainActionFuture.newFuture();
+        transportLiveQueriesAction.execute(request, future);
+        LiveQueriesResponse response = future.actionGet();
+        SearchQueryRecord rec = response.getLiveQueries().get(0);
+        assertEquals(0L, rec.getMeasurements().get(MetricType.CPU).getMeasurement().longValue());
+        assertEquals(0L, rec.getMeasurements().get(MetricType.MEMORY).getMeasurement().longValue());
+    }
+
+    public void testEmptyUsageInfoDefaultsToZero() throws Exception {
+        LiveQueriesRequest request = new LiveQueriesRequest(true);
+        // Create TaskResourceStats with empty usageInfo map
+        TaskResourceStats stats = new TaskResourceStats(Collections.emptyMap(), new TaskThreadUsage(0, 0));
+        TaskInfo info = new TaskInfo(
+            new TaskId(node1.getId(), randomLong()),
+            "test_type",
+            "indices:data/read/search",
+            "desc",
+            null,
+            123L,
+            456L,
+            true,
+            false,
+            TaskId.EMPTY_TASK_ID,
+            Collections.emptyMap(),
+            stats
+        );
+        ListTasksResponse listTasksResponse = new ListTasksResponse(List.of(info), emptyList(), emptyList());
+        doAnswer(invocation -> {
+            ActionListener<ListTasksResponse> listener = invocation.getArgument(1);
+            listener.onResponse(listTasksResponse);
+            return null;
+        }).when(clusterAdminClient).listTasks(any(ListTasksRequest.class), any(ActionListener.class));
+
+        PlainActionFuture<LiveQueriesResponse> future = PlainActionFuture.newFuture();
+        transportLiveQueriesAction.execute(request, future);
+        LiveQueriesResponse response = future.actionGet();
+        SearchQueryRecord rec = response.getLiveQueries().get(0);
+        assertEquals(0L, rec.getMeasurements().get(MetricType.CPU).getMeasurement().longValue());
+        assertEquals(0L, rec.getMeasurements().get(MetricType.MEMORY).getMeasurement().longValue());
+    }
+
+    public void testMissingTotalUsageDefaultsToZero() throws Exception {
+        LiveQueriesRequest request = new LiveQueriesRequest(true);
+        // Create usageInfo map without 'total' key
+        Map<String, TaskResourceUsage> usageInfo = Map.of("other", new TaskResourceUsage(111L, 222L));
+        TaskResourceStats stats = new TaskResourceStats(usageInfo, new TaskThreadUsage(0, 0));
+        TaskInfo info = new TaskInfo(
+            new TaskId(node1.getId(), randomLong()),
+            "test_type",
+            "indices:data/read/search",
+            "desc",
+            null,
+            123L,
+            456L,
+            true,
+            false,
+            TaskId.EMPTY_TASK_ID,
+            Collections.emptyMap(),
+            stats
+        );
+        ListTasksResponse listTasksResponse = new ListTasksResponse(List.of(info), emptyList(), emptyList());
+        doAnswer(invocation -> {
+            ActionListener<ListTasksResponse> listener = invocation.getArgument(1);
+            listener.onResponse(listTasksResponse);
+            return null;
+        }).when(clusterAdminClient).listTasks(any(ListTasksRequest.class), any(ActionListener.class));
+
+        PlainActionFuture<LiveQueriesResponse> future = PlainActionFuture.newFuture();
+        transportLiveQueriesAction.execute(request, future);
+        LiveQueriesResponse response = future.actionGet();
+        SearchQueryRecord rec = response.getLiveQueries().get(0);
+        assertEquals(0L, rec.getMeasurements().get(MetricType.CPU).getMeasurement().longValue());
+        assertEquals(0L, rec.getMeasurements().get(MetricType.MEMORY).getMeasurement().longValue());
+    }
+}


Reviews:
user: login: deshsidd
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-21T20:42:05Z
user: login: ansjcy
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-22T00:23:59Z
user: login: ansjcy
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-22T00:25:15Z
user: login: ansjcy
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-22T00:25:44Z
user: login: ansjcy
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-22T00:27:20Z
user: login: ansjcy
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-22T05:54:49Z
user: login: dzane17
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-22T22:25:01Z
user: login: ansjcy
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-22T22:30:18Z
user: login: ansjcy
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-22T22:33:25Z
user: login: ansjcy
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-22T22:42:10Z
user: login: dzane17
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-22T22:52:33Z

Review comments:
diff_hunk: @@ -0,0 +1,98 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.action.live_queries;
+
+import java.io.IOException;
+import java.util.Collection;
+import java.util.List;
+import org.opensearch.action.FailedNodeException;
+import org.opensearch.action.support.nodes.BaseNodesResponse;
+import org.opensearch.cluster.ClusterName;
+import org.opensearch.core.common.io.stream.StreamInput;
+import org.opensearch.core.common.io.stream.StreamOutput;
+import org.opensearch.core.xcontent.ToXContentFragment;
+import org.opensearch.core.xcontent.XContentBuilder;
+import org.opensearch.plugin.insights.rules.model.MetricType;
+import org.opensearch.plugin.insights.rules.model.SearchQueryRecord;
+
+/**
+ * Transport response for cluster/node level live queries information.
+ */
+public class LiveQueriesResponse extends BaseNodesResponse<LiveQueries> implements ToXContentFragment {
+
+    private static final String CLUSTER_LEVEL_RESULTS_KEY = "live_queries";
+
+    /**
+     * Constructor for LiveQueriesResponse.
+     *
+     * @param in A {@link StreamInput} object.
+     * @throws IOException if the stream cannot be deserialized.
+     */
+    public LiveQueriesResponse(final StreamInput in) throws IOException {
+        super(in);
+    }
+
+    /**
+     * Constructor for LiveQueriesResponse
+     *
+     * @param clusterName The current cluster name
+     * @param nodes A list that contains live queries results from all nodes
+     * @param failures A list that contains FailedNodeException
+     * @param verbose Whether verbose query information was requested (used by constructor, but not stored)
+     */
+    public LiveQueriesResponse(
+        final ClusterName clusterName,
+        final List<LiveQueries> nodes,
+        final List<FailedNodeException> failures,
+        final boolean verbose
+    ) {
+        super(clusterName, nodes, failures);
+    }
+
+    @Override
+    protected List<LiveQueries> readNodesFrom(final StreamInput in) throws IOException {
+        return in.readList(LiveQueries::new);
+    }
+
+    @Override
+    protected void writeNodesTo(final StreamOutput out, final List<LiveQueries> nodes) throws IOException {
+        out.writeList(nodes);
+    }
+
+    @Override
+    public XContentBuilder toXContent(final XContentBuilder builder, final Params params) throws IOException {
+        final List<LiveQueries> results = getNodes();
+        builder.startObject();
+        toClusterLevelResult(builder, params, results);
+        return builder.endObject();
+    }
+
+    /**
+     * Merge live queries results from nodes into cluster level results in XContent format.
+     *
+     * @param builder XContent builder
+     * @param params serialization parameters
+     * @param results live queries results from all nodes
+     * @throws IOException if an error occurs
+     */
+    private void toClusterLevelResult(final XContentBuilder builder, final Params params, final List<LiveQueries> results)
+        throws IOException {
+        final List<SearchQueryRecord> allQueries = results.stream()
+            .map(LiveQueries::getLiveQueries)
+            .flatMap(Collection::stream)
+            .sorted((a, b) -> SearchQueryRecord.compare(a, b, MetricType.LATENCY) * -1)
path: src/main/java/org/opensearch/plugin/insights/rules/action/live_queries/LiveQueriesResponse.java
user: login: deshsidd
body: Are we planning to add sort param in this PR?
created_at: 2025-04-21T20:27:42Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,98 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.action.live_queries;
+
+import java.io.IOException;
+import java.util.Collection;
+import java.util.List;
+import org.opensearch.action.FailedNodeException;
+import org.opensearch.action.support.nodes.BaseNodesResponse;
+import org.opensearch.cluster.ClusterName;
+import org.opensearch.core.common.io.stream.StreamInput;
+import org.opensearch.core.common.io.stream.StreamOutput;
+import org.opensearch.core.xcontent.ToXContentFragment;
+import org.opensearch.core.xcontent.XContentBuilder;
+import org.opensearch.plugin.insights.rules.model.MetricType;
+import org.opensearch.plugin.insights.rules.model.SearchQueryRecord;
+
+/**
+ * Transport response for cluster/node level live queries information.
+ */
+public class LiveQueriesResponse extends BaseNodesResponse<LiveQueries> implements ToXContentFragment {
+
+    private static final String CLUSTER_LEVEL_RESULTS_KEY = "live_queries";
+
+    /**
+     * Constructor for LiveQueriesResponse.
+     *
+     * @param in A {@link StreamInput} object.
+     * @throws IOException if the stream cannot be deserialized.
+     */
+    public LiveQueriesResponse(final StreamInput in) throws IOException {
+        super(in);
+    }
+
+    /**
+     * Constructor for LiveQueriesResponse
+     *
+     * @param clusterName The current cluster name
+     * @param nodes A list that contains live queries results from all nodes
+     * @param failures A list that contains FailedNodeException
+     * @param verbose Whether verbose query information was requested (used by constructor, but not stored)
+     */
+    public LiveQueriesResponse(
+        final ClusterName clusterName,
+        final List<LiveQueries> nodes,
+        final List<FailedNodeException> failures,
+        final boolean verbose
+    ) {
+        super(clusterName, nodes, failures);
+    }
+
+    @Override
+    protected List<LiveQueries> readNodesFrom(final StreamInput in) throws IOException {
+        return in.readList(LiveQueries::new);
+    }
+
+    @Override
+    protected void writeNodesTo(final StreamOutput out, final List<LiveQueries> nodes) throws IOException {
+        out.writeList(nodes);
+    }
+
+    @Override
+    public XContentBuilder toXContent(final XContentBuilder builder, final Params params) throws IOException {
+        final List<LiveQueries> results = getNodes();
+        builder.startObject();
+        toClusterLevelResult(builder, params, results);
+        return builder.endObject();
+    }
+
+    /**
+     * Merge live queries results from nodes into cluster level results in XContent format.
+     *
+     * @param builder XContent builder
+     * @param params serialization parameters
+     * @param results live queries results from all nodes
+     * @throws IOException if an error occurs
+     */
+    private void toClusterLevelResult(final XContentBuilder builder, final Params params, final List<LiveQueries> results)
+        throws IOException {
+        final List<SearchQueryRecord> allQueries = results.stream()
+            .map(LiveQueries::getLiveQueries)
+            .flatMap(Collection::stream)
+            .sorted((a, b) -> SearchQueryRecord.compare(a, b, MetricType.LATENCY) * -1)
path: src/main/java/org/opensearch/plugin/insights/rules/action/live_queries/LiveQueriesResponse.java
user: login: deshsidd
body: Also what are your thoughts about including size and duration as suggestion by @dzane17 ?
created_at: 2025-04-21T20:29:54Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,144 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.transport.live_queries;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.opensearch.action.admin.cluster.node.tasks.list.ListTasksRequest;
+import org.opensearch.action.admin.cluster.node.tasks.list.ListTasksResponse;
+import org.opensearch.action.support.ActionFilters;
+import org.opensearch.action.support.HandledTransportAction;
+import org.opensearch.cluster.node.DiscoveryNode;
+import org.opensearch.cluster.service.ClusterService;
+import org.opensearch.common.inject.Inject;
+import org.opensearch.core.action.ActionListener;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueries;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesAction;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesRequest;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesResponse;
+import org.opensearch.plugin.insights.rules.model.Attribute;
+import org.opensearch.plugin.insights.rules.model.Measurement;
+import org.opensearch.plugin.insights.rules.model.MetricType;
+import org.opensearch.plugin.insights.rules.model.SearchQueryRecord;
+import org.opensearch.tasks.Task;
+import org.opensearch.tasks.TaskInfo;
+import org.opensearch.threadpool.ThreadPool;
+import org.opensearch.transport.TransportService;
+import org.opensearch.transport.client.Client;
+
+/**
+ * Transport action for fetching ongoing live queries
+ */
+public class TransportLiveQueriesAction extends HandledTransportAction<LiveQueriesRequest, LiveQueriesResponse> {
+
+    private static final Logger logger = LogManager.getLogger(TransportLiveQueriesAction.class);
+    private static final String TOTAL = "total";
+
+    private final ClusterService clusterService;
+    private final Client client;
+
+    @Inject
+    public TransportLiveQueriesAction(
+        final ThreadPool threadPool,
+        final ClusterService clusterService,
+        final TransportService transportService,
+        final Client client,
+        final ActionFilters actionFilters
+    ) {
+        super(LiveQueriesAction.NAME, transportService, actionFilters, LiveQueriesRequest::new, ThreadPool.Names.GENERIC);
+        this.clusterService = clusterService;
+        this.client = client;
+    }
+
+    @Override
+    protected void doExecute(final Task task, final LiveQueriesRequest request, final ActionListener<LiveQueriesResponse> listener) {
+        // Prepare list-tasks request
+        ListTasksRequest listTasksRequest = new ListTasksRequest().setDetailed(request.isVerbose()).setActions("indices:data/read/search*");
+        if (request.timeout() != null) {
+            listTasksRequest.setTimeout(request.timeout());
+        }
+        String[] nodesIds = request.nodesIds();
+        if (nodesIds != null && nodesIds.length > 0) {
+            listTasksRequest.setNodes(nodesIds);
+        }
+
+        // Execute tasks request asynchronously to avoid blocking transport threads
+        client.admin().cluster().listTasks(listTasksRequest, new ActionListener<ListTasksResponse>() {
+            @Override
+            public void onResponse(ListTasksResponse taskResponse) {
+                try {
+                    // Group tasks by node and build records
+                    Map<String, List<SearchQueryRecord>> perNode = new HashMap<>();
+                    for (TaskInfo taskInfo : taskResponse.getTasks()) {
+                        if (!taskInfo.getAction().startsWith("indices:data/read/search")) {
path: src/main/java/org/opensearch/plugin/insights/rules/transport/live_queries/TransportLiveQueriesAction.java
user: login: deshsidd
body: We could also capture the current phase/type of search in the future?

```
indices:data/read/search[phase/query]

indices:data/read/search[phase/fetch]

indices:data/read/search/shard

indices:data/read/search/scroll

indices:data/read/search/template
```
created_at: 2025-04-21T20:33:20Z
author_association: COLLABORATOR
diff_hunk: @@ -80,6 +80,12 @@ public class QueryInsightsSettings {
      *
      */
     public static final String TOP_QUERIES_BASE_URI = PLUGINS_BASE_URI + "/top_queries";
+
+    /**
+     * Settings for Live Queries
+     */
+    public static final String LIVE_QUERIES_BASE_URI = PLUGINS_BASE_URI + "/live_queries";
path: src/main/java/org/opensearch/plugin/insights/settings/QueryInsightsSettings.java
user: login: deshsidd
body: No settings needed for this to enable/disable this feature?
created_at: 2025-04-21T20:34:47Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,371 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.resthandler.live_queries;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.junit.Assert;
+import org.opensearch.client.Request;
+import org.opensearch.client.Response;
+import org.opensearch.common.xcontent.LoggingDeprecationHandler;
+import org.opensearch.common.xcontent.json.JsonXContent;
+import org.opensearch.core.xcontent.NamedXContentRegistry;
+import org.opensearch.plugin.insights.QueryInsightsRestTestCase;
+import org.opensearch.plugin.insights.settings.QueryInsightsSettings;
+
+/**
+ * Integration tests for Live Queries API
+ */
+public class LiveQueriesRestIT extends QueryInsightsRestTestCase {
+
+    private static final Logger logger = LogManager.getLogger(LiveQueriesRestIT.class);
+    private static final String TEST_INDEX = "test-index";
+    private static final int MAX_POLL_ATTEMPTS = 1000;  // Maximum number of times to poll the live queries API
+    private static final int POLL_INTERVAL_MS = 5;  // Time between polling attempts
+    private static final int CONCURRENT_QUERIES = 5;  // Number of concurrent queries to run
+    private static final int MAX_QUERY_ITERATION = 100; // Max number of times to run the queries
+    private static final int QUERY_DURATION_MS = 10000; // Time each query should run
+
+    /**
+     * Verify Query Insights plugin is installed
+     */
+    @SuppressWarnings("unchecked")
+    public void ensureQueryInsightsPluginInstalled() throws IOException {
+        Request request = new Request("GET", "/_cat/plugins?s=component&h=name,component,version,description&format=json");
+        Response response = client().performRequest(request);
+        List<Object> pluginsList = JsonXContent.jsonXContent.createParser(
+            NamedXContentRegistry.EMPTY,
+            LoggingDeprecationHandler.INSTANCE,
+            response.getEntity().getContent()
+        ).list();
+        Assert.assertTrue(
+            pluginsList.stream().map(o -> (Map<String, Object>) o).anyMatch(plugin -> plugin.get("component").equals("query-insights"))
+        );
+    }
+
+    /**
+     * Try to detect live queries by running multiple concurrent search operations and polling the API.
+     *  TODO: Limitation of this integ test: It's hard to simulate a long running query without being flaky.
+     *   We can try using a painless script to simulate with sleep, but by default QueryInsightsRestTestCase disables painless.
+     * This test tries its best to simulate long running queries.
+     * It will pass if:
+     * 1. We find at least one live query, or
+     * 2. We hit our polling limit without finding any (due to test environment limitations)
+     */
+    @SuppressWarnings("unchecked")
+    public void testLiveQueriesWithConcurrentSearches() throws Exception {
+        // Create index and add documents with some data
+        createIndexWithData(100);
+
+        // Set up a coordinator for the search threads
+        CountDownLatch startLatch = new CountDownLatch(1);
+        CountDownLatch completionLatch = new CountDownLatch(CONCURRENT_QUERIES);
+        AtomicBoolean shouldStop = new AtomicBoolean(false);
+        AtomicInteger completedQueries = new AtomicInteger(0);
+        ExecutorService threadPool = Executors.newFixedThreadPool(CONCURRENT_QUERIES);
+
+        // Create and submit search tasks that will run for a while
+        for (int i = 0; i < CONCURRENT_QUERIES; i++) {
+            final int queryNum = i;
+            threadPool.submit(() -> {
+                try {
+                    logger.info("Search thread {} ready and waiting to start", queryNum);
+                    startLatch.await(); // Wait for all threads to be ready
+                    String searchJson = generateComplexQuery(queryNum);
+                    Request searchRequest = new Request("GET", "/" + TEST_INDEX + "/_search");
+                    searchRequest.setJsonEntity(searchJson);
+
+                    // Set longer timeout to let the query run for a while
+                    searchRequest.addParameter("timeout", QUERY_DURATION_MS + "ms");
+
+                    try {
+                        logger.info("Search thread {} starting search", queryNum);
+                        for (int j = 0; j < MAX_QUERY_ITERATION; j++) {
+                            client().performRequest(searchRequest);
+                        }
+                        logger.info("Search thread {} completed successfully", queryNum);
+                    } catch (Exception e) {
+                        // We expect this might timeout or be cancelled
+                        logger.info("Search thread {} ended with: {}", queryNum, e.getMessage());
+                    }
+                } catch (Exception e) {
+                    logger.error("Error in search thread {}: {}", queryNum, e.getMessage());
+                } finally {
+                    completedQueries.incrementAndGet();
+                    completionLatch.countDown();
+                }
+            });
+        }
+
+        // Start all the search threads..
+        logger.info("Starting all search threads");
+        startLatch.countDown();
+
+        // Poll the Live Queries API repeatedly to try to catch the searches
+        logger.info("Beginning to poll the live queries API");
+        boolean foundLiveQueries = false;
+        List<Map<String, Object>> liveQueries = new ArrayList<>();
+
+        // Create a separate thread to poll the API while search queries are running
+        for (int attempt = 0; attempt < MAX_POLL_ATTEMPTS && !shouldStop.get(); attempt++) {
+            Thread.sleep(POLL_INTERVAL_MS);
+
+            // Skip further checking if all queries are done already
+            if (completedQueries.get() >= CONCURRENT_QUERIES) {
+                logger.info("All queries completed, stopping poll attempts");
+                break;
+            }
+
+            try {
+                // Call the Live Queries API
+                Request liveQueriesRequest = new Request("GET", QueryInsightsSettings.LIVE_QUERIES_BASE_URI + "?verbose=true");
+                Response liveQueriesResponse = client().performRequest(liveQueriesRequest);
+
+                // Parse response
+                Map<String, Object> responseMap = entityAsMap(liveQueriesResponse);
+                assertTrue(responseMap.containsKey("live_queries"));
+                liveQueries = (List<Map<String, Object>>) responseMap.get("live_queries");
+
+                logger.info("Polling attempt {}: Found {} live queries", attempt, liveQueries.size());
+
+                if (liveQueries != null && !liveQueries.isEmpty()) {
+                    logger.info("Successfully found live queries on attempt {}", attempt);
+                    foundLiveQueries = true;
+                    break;
+                }
+            } catch (Exception e) {
+                logger.error("Error polling live queries API: {}", e.getMessage());
+            }
+        }
+
+        // Signal all threads to stop
+        shouldStop.set(true);
+
+        // Wait for all search threads to complete with timeout
+        boolean allThreadsCompleted = completionLatch.await(QUERY_DURATION_MS * 2, TimeUnit.MILLISECONDS);
+        logger.info("All threads completed? {}", allThreadsCompleted);
+
+        // Shut down executor
+        threadPool.shutdownNow();
+
+        // We either found live queries or exhausted our polling attempts.
+        if (foundLiveQueries) {
+            logger.info("Test detected live queries: {}", liveQueries);
+            assertTrue("Should have found at least one live query", !liveQueries.isEmpty());
+
+            // Validate the format of live queries based on LiveQueries.java and LiveQueriesResponse.java
+            for (Map<String, Object> query : liveQueries) {
+                // Verify required fields are present
+                assertTrue("Query should have timestamp", query.containsKey("timestamp"));
+                assertTrue("Query should have id", query.containsKey("id"));
+                assertTrue("Query should have node_id", query.containsKey("node_id"));
+                assertTrue("Query should have measurements", query.containsKey("measurements"));
+                assertTrue("Query should have description", query.containsKey("description"));
+
+                // Validate timestamp is a number
+                assertTrue("Timestamp should be a number", query.get("timestamp") instanceof Number);
+
+                // Validate id is a string
+                assertTrue("ID should be a string", query.get("id") instanceof String);
+
+                // Validate node_id is a string
+                assertTrue("Node ID should be a string", query.get("node_id") instanceof String);
+
+                // Validate measurements structure
+                Map<String, Object> measurements = (Map<String, Object>) query.get("measurements");
+                assertTrue("Measurements should include latency", measurements.containsKey("latency"));
+                assertTrue("Measurements should include cpu", measurements.containsKey("cpu"));
+                assertTrue("Measurements should include memory", measurements.containsKey("memory"));
+
+                // Validate each measurement's structure
+                for (String metricType : new String[] { "latency", "cpu", "memory" }) {
+                    Map<String, Object> metric = (Map<String, Object>) measurements.get(metricType);
+                    assertTrue("Metric should have number", metric.containsKey("number"));
+                    assertTrue("Metric should have count", metric.containsKey("count"));
+                    assertTrue("Metric should have aggregationType", metric.containsKey("aggregationType"));
+
+                    // Validate number is a number
+                    assertTrue("Number should be a number", metric.get("number") instanceof Number);
+
+                    // Validate count is a number
+                    assertTrue("Count should be a number", metric.get("count") instanceof Number);
+
+                    // Validate aggregationType is a string
+                    assertTrue("AggregationType should be a string", metric.get("aggregationType") instanceof String);
+                }
+                assertTrue("Description should be a string", query.get("description") instanceof String);
+            }
+        } else {
+            logger.info(
+                "No live queries found after {} attempts - this is acceptable due to test environment limitations",
+                MAX_POLL_ATTEMPTS
path: src/test/java/org/opensearch/plugin/insights/rules/resthandler/live_queries/LiveQueriesRestIT.java
user: login: deshsidd
body: Hope this does not cause the tests to silently pass in scenarios where it should fail ? Eg: cases where some code regressions cause the live queries api to return empty each time.
created_at: 2025-04-21T20:41:28Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,98 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.action.live_queries;
+
+import java.io.IOException;
+import java.util.Collection;
+import java.util.List;
+import org.opensearch.action.FailedNodeException;
+import org.opensearch.action.support.nodes.BaseNodesResponse;
+import org.opensearch.cluster.ClusterName;
+import org.opensearch.core.common.io.stream.StreamInput;
+import org.opensearch.core.common.io.stream.StreamOutput;
+import org.opensearch.core.xcontent.ToXContentFragment;
+import org.opensearch.core.xcontent.XContentBuilder;
+import org.opensearch.plugin.insights.rules.model.MetricType;
+import org.opensearch.plugin.insights.rules.model.SearchQueryRecord;
+
+/**
+ * Transport response for cluster/node level live queries information.
+ */
+public class LiveQueriesResponse extends BaseNodesResponse<LiveQueries> implements ToXContentFragment {
+
+    private static final String CLUSTER_LEVEL_RESULTS_KEY = "live_queries";
+
+    /**
+     * Constructor for LiveQueriesResponse.
+     *
+     * @param in A {@link StreamInput} object.
+     * @throws IOException if the stream cannot be deserialized.
+     */
+    public LiveQueriesResponse(final StreamInput in) throws IOException {
+        super(in);
+    }
+
+    /**
+     * Constructor for LiveQueriesResponse
+     *
+     * @param clusterName The current cluster name
+     * @param nodes A list that contains live queries results from all nodes
+     * @param failures A list that contains FailedNodeException
+     * @param verbose Whether verbose query information was requested (used by constructor, but not stored)
+     */
+    public LiveQueriesResponse(
+        final ClusterName clusterName,
+        final List<LiveQueries> nodes,
+        final List<FailedNodeException> failures,
+        final boolean verbose
+    ) {
+        super(clusterName, nodes, failures);
+    }
+
+    @Override
+    protected List<LiveQueries> readNodesFrom(final StreamInput in) throws IOException {
+        return in.readList(LiveQueries::new);
+    }
+
+    @Override
+    protected void writeNodesTo(final StreamOutput out, final List<LiveQueries> nodes) throws IOException {
+        out.writeList(nodes);
+    }
+
+    @Override
+    public XContentBuilder toXContent(final XContentBuilder builder, final Params params) throws IOException {
+        final List<LiveQueries> results = getNodes();
+        builder.startObject();
+        toClusterLevelResult(builder, params, results);
+        return builder.endObject();
+    }
+
+    /**
+     * Merge live queries results from nodes into cluster level results in XContent format.
+     *
+     * @param builder XContent builder
+     * @param params serialization parameters
+     * @param results live queries results from all nodes
+     * @throws IOException if an error occurs
+     */
+    private void toClusterLevelResult(final XContentBuilder builder, final Params params, final List<LiveQueries> results)
+        throws IOException {
+        final List<SearchQueryRecord> allQueries = results.stream()
+            .map(LiveQueries::getLiveQueries)
+            .flatMap(Collection::stream)
+            .sorted((a, b) -> SearchQueryRecord.compare(a, b, MetricType.LATENCY) * -1)
path: src/main/java/org/opensearch/plugin/insights/rules/action/live_queries/LiveQueriesResponse.java
user: login: ansjcy
body: Yes as I mentioned here: https://github.com/opensearch-project/query-insights/pull/295#issuecomment-2816394705
created_at: 2025-04-22T00:23:59Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,144 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.transport.live_queries;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.opensearch.action.admin.cluster.node.tasks.list.ListTasksRequest;
+import org.opensearch.action.admin.cluster.node.tasks.list.ListTasksResponse;
+import org.opensearch.action.support.ActionFilters;
+import org.opensearch.action.support.HandledTransportAction;
+import org.opensearch.cluster.node.DiscoveryNode;
+import org.opensearch.cluster.service.ClusterService;
+import org.opensearch.common.inject.Inject;
+import org.opensearch.core.action.ActionListener;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueries;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesAction;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesRequest;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesResponse;
+import org.opensearch.plugin.insights.rules.model.Attribute;
+import org.opensearch.plugin.insights.rules.model.Measurement;
+import org.opensearch.plugin.insights.rules.model.MetricType;
+import org.opensearch.plugin.insights.rules.model.SearchQueryRecord;
+import org.opensearch.tasks.Task;
+import org.opensearch.tasks.TaskInfo;
+import org.opensearch.threadpool.ThreadPool;
+import org.opensearch.transport.TransportService;
+import org.opensearch.transport.client.Client;
+
+/**
+ * Transport action for fetching ongoing live queries
+ */
+public class TransportLiveQueriesAction extends HandledTransportAction<LiveQueriesRequest, LiveQueriesResponse> {
+
+    private static final Logger logger = LogManager.getLogger(TransportLiveQueriesAction.class);
+    private static final String TOTAL = "total";
+
+    private final ClusterService clusterService;
+    private final Client client;
+
+    @Inject
+    public TransportLiveQueriesAction(
+        final ThreadPool threadPool,
+        final ClusterService clusterService,
+        final TransportService transportService,
+        final Client client,
+        final ActionFilters actionFilters
+    ) {
+        super(LiveQueriesAction.NAME, transportService, actionFilters, LiveQueriesRequest::new, ThreadPool.Names.GENERIC);
+        this.clusterService = clusterService;
+        this.client = client;
+    }
+
+    @Override
+    protected void doExecute(final Task task, final LiveQueriesRequest request, final ActionListener<LiveQueriesResponse> listener) {
+        // Prepare list-tasks request
+        ListTasksRequest listTasksRequest = new ListTasksRequest().setDetailed(request.isVerbose()).setActions("indices:data/read/search*");
+        if (request.timeout() != null) {
+            listTasksRequest.setTimeout(request.timeout());
+        }
+        String[] nodesIds = request.nodesIds();
+        if (nodesIds != null && nodesIds.length > 0) {
+            listTasksRequest.setNodes(nodesIds);
+        }
+
+        // Execute tasks request asynchronously to avoid blocking transport threads
+        client.admin().cluster().listTasks(listTasksRequest, new ActionListener<ListTasksResponse>() {
+            @Override
+            public void onResponse(ListTasksResponse taskResponse) {
+                try {
+                    // Group tasks by node and build records
+                    Map<String, List<SearchQueryRecord>> perNode = new HashMap<>();
+                    for (TaskInfo taskInfo : taskResponse.getTasks()) {
+                        if (!taskInfo.getAction().startsWith("indices:data/read/search")) {
path: src/main/java/org/opensearch/plugin/insights/rules/transport/live_queries/TransportLiveQueriesAction.java
user: login: ansjcy
body: No, the purpose of this PR is to add coordinator level search task only. All tasks you mentioned should belong to the data node or unrelated to search task. Even if we add it here, we won't be able to correlate to the actual search task.
created_at: 2025-04-22T00:25:15Z
author_association: MEMBER
diff_hunk: @@ -80,6 +80,12 @@ public class QueryInsightsSettings {
      *
      */
     public static final String TOP_QUERIES_BASE_URI = PLUGINS_BASE_URI + "/top_queries";
+
+    /**
+     * Settings for Live Queries
+     */
+    public static final String LIVE_QUERIES_BASE_URI = PLUGINS_BASE_URI + "/live_queries";
path: src/main/java/org/opensearch/plugin/insights/settings/QueryInsightsSettings.java
user: login: ansjcy
body: No, this would be an on-demand API. I checked with PM/UX, there's no need to add settings for on-demand APIs.
created_at: 2025-04-22T00:25:44Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,371 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.resthandler.live_queries;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.junit.Assert;
+import org.opensearch.client.Request;
+import org.opensearch.client.Response;
+import org.opensearch.common.xcontent.LoggingDeprecationHandler;
+import org.opensearch.common.xcontent.json.JsonXContent;
+import org.opensearch.core.xcontent.NamedXContentRegistry;
+import org.opensearch.plugin.insights.QueryInsightsRestTestCase;
+import org.opensearch.plugin.insights.settings.QueryInsightsSettings;
+
+/**
+ * Integration tests for Live Queries API
+ */
+public class LiveQueriesRestIT extends QueryInsightsRestTestCase {
+
+    private static final Logger logger = LogManager.getLogger(LiveQueriesRestIT.class);
+    private static final String TEST_INDEX = "test-index";
+    private static final int MAX_POLL_ATTEMPTS = 1000;  // Maximum number of times to poll the live queries API
+    private static final int POLL_INTERVAL_MS = 5;  // Time between polling attempts
+    private static final int CONCURRENT_QUERIES = 5;  // Number of concurrent queries to run
+    private static final int MAX_QUERY_ITERATION = 100; // Max number of times to run the queries
+    private static final int QUERY_DURATION_MS = 10000; // Time each query should run
+
+    /**
+     * Verify Query Insights plugin is installed
+     */
+    @SuppressWarnings("unchecked")
+    public void ensureQueryInsightsPluginInstalled() throws IOException {
+        Request request = new Request("GET", "/_cat/plugins?s=component&h=name,component,version,description&format=json");
+        Response response = client().performRequest(request);
+        List<Object> pluginsList = JsonXContent.jsonXContent.createParser(
+            NamedXContentRegistry.EMPTY,
+            LoggingDeprecationHandler.INSTANCE,
+            response.getEntity().getContent()
+        ).list();
+        Assert.assertTrue(
+            pluginsList.stream().map(o -> (Map<String, Object>) o).anyMatch(plugin -> plugin.get("component").equals("query-insights"))
+        );
+    }
+
+    /**
+     * Try to detect live queries by running multiple concurrent search operations and polling the API.
+     *  TODO: Limitation of this integ test: It's hard to simulate a long running query without being flaky.
+     *   We can try using a painless script to simulate with sleep, but by default QueryInsightsRestTestCase disables painless.
+     * This test tries its best to simulate long running queries.
+     * It will pass if:
+     * 1. We find at least one live query, or
+     * 2. We hit our polling limit without finding any (due to test environment limitations)
+     */
+    @SuppressWarnings("unchecked")
+    public void testLiveQueriesWithConcurrentSearches() throws Exception {
+        // Create index and add documents with some data
+        createIndexWithData(100);
+
+        // Set up a coordinator for the search threads
+        CountDownLatch startLatch = new CountDownLatch(1);
+        CountDownLatch completionLatch = new CountDownLatch(CONCURRENT_QUERIES);
+        AtomicBoolean shouldStop = new AtomicBoolean(false);
+        AtomicInteger completedQueries = new AtomicInteger(0);
+        ExecutorService threadPool = Executors.newFixedThreadPool(CONCURRENT_QUERIES);
+
+        // Create and submit search tasks that will run for a while
+        for (int i = 0; i < CONCURRENT_QUERIES; i++) {
+            final int queryNum = i;
+            threadPool.submit(() -> {
+                try {
+                    logger.info("Search thread {} ready and waiting to start", queryNum);
+                    startLatch.await(); // Wait for all threads to be ready
+                    String searchJson = generateComplexQuery(queryNum);
+                    Request searchRequest = new Request("GET", "/" + TEST_INDEX + "/_search");
+                    searchRequest.setJsonEntity(searchJson);
+
+                    // Set longer timeout to let the query run for a while
+                    searchRequest.addParameter("timeout", QUERY_DURATION_MS + "ms");
+
+                    try {
+                        logger.info("Search thread {} starting search", queryNum);
+                        for (int j = 0; j < MAX_QUERY_ITERATION; j++) {
+                            client().performRequest(searchRequest);
+                        }
+                        logger.info("Search thread {} completed successfully", queryNum);
+                    } catch (Exception e) {
+                        // We expect this might timeout or be cancelled
+                        logger.info("Search thread {} ended with: {}", queryNum, e.getMessage());
+                    }
+                } catch (Exception e) {
+                    logger.error("Error in search thread {}: {}", queryNum, e.getMessage());
+                } finally {
+                    completedQueries.incrementAndGet();
+                    completionLatch.countDown();
+                }
+            });
+        }
+
+        // Start all the search threads..
+        logger.info("Starting all search threads");
+        startLatch.countDown();
+
+        // Poll the Live Queries API repeatedly to try to catch the searches
+        logger.info("Beginning to poll the live queries API");
+        boolean foundLiveQueries = false;
+        List<Map<String, Object>> liveQueries = new ArrayList<>();
+
+        // Create a separate thread to poll the API while search queries are running
+        for (int attempt = 0; attempt < MAX_POLL_ATTEMPTS && !shouldStop.get(); attempt++) {
+            Thread.sleep(POLL_INTERVAL_MS);
+
+            // Skip further checking if all queries are done already
+            if (completedQueries.get() >= CONCURRENT_QUERIES) {
+                logger.info("All queries completed, stopping poll attempts");
+                break;
+            }
+
+            try {
+                // Call the Live Queries API
+                Request liveQueriesRequest = new Request("GET", QueryInsightsSettings.LIVE_QUERIES_BASE_URI + "?verbose=true");
+                Response liveQueriesResponse = client().performRequest(liveQueriesRequest);
+
+                // Parse response
+                Map<String, Object> responseMap = entityAsMap(liveQueriesResponse);
+                assertTrue(responseMap.containsKey("live_queries"));
+                liveQueries = (List<Map<String, Object>>) responseMap.get("live_queries");
+
+                logger.info("Polling attempt {}: Found {} live queries", attempt, liveQueries.size());
+
+                if (liveQueries != null && !liveQueries.isEmpty()) {
+                    logger.info("Successfully found live queries on attempt {}", attempt);
+                    foundLiveQueries = true;
+                    break;
+                }
+            } catch (Exception e) {
+                logger.error("Error polling live queries API: {}", e.getMessage());
+            }
+        }
+
+        // Signal all threads to stop
+        shouldStop.set(true);
+
+        // Wait for all search threads to complete with timeout
+        boolean allThreadsCompleted = completionLatch.await(QUERY_DURATION_MS * 2, TimeUnit.MILLISECONDS);
+        logger.info("All threads completed? {}", allThreadsCompleted);
+
+        // Shut down executor
+        threadPool.shutdownNow();
+
+        // We either found live queries or exhausted our polling attempts.
+        if (foundLiveQueries) {
+            logger.info("Test detected live queries: {}", liveQueries);
+            assertTrue("Should have found at least one live query", !liveQueries.isEmpty());
+
+            // Validate the format of live queries based on LiveQueries.java and LiveQueriesResponse.java
+            for (Map<String, Object> query : liveQueries) {
+                // Verify required fields are present
+                assertTrue("Query should have timestamp", query.containsKey("timestamp"));
+                assertTrue("Query should have id", query.containsKey("id"));
+                assertTrue("Query should have node_id", query.containsKey("node_id"));
+                assertTrue("Query should have measurements", query.containsKey("measurements"));
+                assertTrue("Query should have description", query.containsKey("description"));
+
+                // Validate timestamp is a number
+                assertTrue("Timestamp should be a number", query.get("timestamp") instanceof Number);
+
+                // Validate id is a string
+                assertTrue("ID should be a string", query.get("id") instanceof String);
+
+                // Validate node_id is a string
+                assertTrue("Node ID should be a string", query.get("node_id") instanceof String);
+
+                // Validate measurements structure
+                Map<String, Object> measurements = (Map<String, Object>) query.get("measurements");
+                assertTrue("Measurements should include latency", measurements.containsKey("latency"));
+                assertTrue("Measurements should include cpu", measurements.containsKey("cpu"));
+                assertTrue("Measurements should include memory", measurements.containsKey("memory"));
+
+                // Validate each measurement's structure
+                for (String metricType : new String[] { "latency", "cpu", "memory" }) {
+                    Map<String, Object> metric = (Map<String, Object>) measurements.get(metricType);
+                    assertTrue("Metric should have number", metric.containsKey("number"));
+                    assertTrue("Metric should have count", metric.containsKey("count"));
+                    assertTrue("Metric should have aggregationType", metric.containsKey("aggregationType"));
+
+                    // Validate number is a number
+                    assertTrue("Number should be a number", metric.get("number") instanceof Number);
+
+                    // Validate count is a number
+                    assertTrue("Count should be a number", metric.get("count") instanceof Number);
+
+                    // Validate aggregationType is a string
+                    assertTrue("AggregationType should be a string", metric.get("aggregationType") instanceof String);
+                }
+                assertTrue("Description should be a string", query.get("description") instanceof String);
+            }
+        } else {
+            logger.info(
+                "No live queries found after {} attempts - this is acceptable due to test environment limitations",
+                MAX_POLL_ATTEMPTS
path: src/test/java/org/opensearch/plugin/insights/rules/resthandler/live_queries/LiveQueriesRestIT.java
user: login: ansjcy
body: yeah potentially it will. The actual issue is we won't be able to simulate slow queries in a reliable way. I can try commenting out/ removing this part for now and see if this test would be flaky though.
created_at: 2025-04-22T00:27:20Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,371 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.resthandler.live_queries;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.junit.Assert;
+import org.opensearch.client.Request;
+import org.opensearch.client.Response;
+import org.opensearch.common.xcontent.LoggingDeprecationHandler;
+import org.opensearch.common.xcontent.json.JsonXContent;
+import org.opensearch.core.xcontent.NamedXContentRegistry;
+import org.opensearch.plugin.insights.QueryInsightsRestTestCase;
+import org.opensearch.plugin.insights.settings.QueryInsightsSettings;
+
+/**
+ * Integration tests for Live Queries API
+ */
+public class LiveQueriesRestIT extends QueryInsightsRestTestCase {
+
+    private static final Logger logger = LogManager.getLogger(LiveQueriesRestIT.class);
+    private static final String TEST_INDEX = "test-index";
+    private static final int MAX_POLL_ATTEMPTS = 1000;  // Maximum number of times to poll the live queries API
+    private static final int POLL_INTERVAL_MS = 5;  // Time between polling attempts
+    private static final int CONCURRENT_QUERIES = 5;  // Number of concurrent queries to run
+    private static final int MAX_QUERY_ITERATION = 100; // Max number of times to run the queries
+    private static final int QUERY_DURATION_MS = 10000; // Time each query should run
+
+    /**
+     * Verify Query Insights plugin is installed
+     */
+    @SuppressWarnings("unchecked")
+    public void ensureQueryInsightsPluginInstalled() throws IOException {
+        Request request = new Request("GET", "/_cat/plugins?s=component&h=name,component,version,description&format=json");
+        Response response = client().performRequest(request);
+        List<Object> pluginsList = JsonXContent.jsonXContent.createParser(
+            NamedXContentRegistry.EMPTY,
+            LoggingDeprecationHandler.INSTANCE,
+            response.getEntity().getContent()
+        ).list();
+        Assert.assertTrue(
+            pluginsList.stream().map(o -> (Map<String, Object>) o).anyMatch(plugin -> plugin.get("component").equals("query-insights"))
+        );
+    }
+
+    /**
+     * Try to detect live queries by running multiple concurrent search operations and polling the API.
+     *  TODO: Limitation of this integ test: It's hard to simulate a long running query without being flaky.
+     *   We can try using a painless script to simulate with sleep, but by default QueryInsightsRestTestCase disables painless.
+     * This test tries its best to simulate long running queries.
+     * It will pass if:
+     * 1. We find at least one live query, or
+     * 2. We hit our polling limit without finding any (due to test environment limitations)
+     */
+    @SuppressWarnings("unchecked")
+    public void testLiveQueriesWithConcurrentSearches() throws Exception {
+        // Create index and add documents with some data
+        createIndexWithData(100);
+
+        // Set up a coordinator for the search threads
+        CountDownLatch startLatch = new CountDownLatch(1);
+        CountDownLatch completionLatch = new CountDownLatch(CONCURRENT_QUERIES);
+        AtomicBoolean shouldStop = new AtomicBoolean(false);
+        AtomicInteger completedQueries = new AtomicInteger(0);
+        ExecutorService threadPool = Executors.newFixedThreadPool(CONCURRENT_QUERIES);
+
+        // Create and submit search tasks that will run for a while
+        for (int i = 0; i < CONCURRENT_QUERIES; i++) {
+            final int queryNum = i;
+            threadPool.submit(() -> {
+                try {
+                    logger.info("Search thread {} ready and waiting to start", queryNum);
+                    startLatch.await(); // Wait for all threads to be ready
+                    String searchJson = generateComplexQuery(queryNum);
+                    Request searchRequest = new Request("GET", "/" + TEST_INDEX + "/_search");
+                    searchRequest.setJsonEntity(searchJson);
+
+                    // Set longer timeout to let the query run for a while
+                    searchRequest.addParameter("timeout", QUERY_DURATION_MS + "ms");
+
+                    try {
+                        logger.info("Search thread {} starting search", queryNum);
+                        for (int j = 0; j < MAX_QUERY_ITERATION; j++) {
+                            client().performRequest(searchRequest);
+                        }
+                        logger.info("Search thread {} completed successfully", queryNum);
+                    } catch (Exception e) {
+                        // We expect this might timeout or be cancelled
+                        logger.info("Search thread {} ended with: {}", queryNum, e.getMessage());
+                    }
+                } catch (Exception e) {
+                    logger.error("Error in search thread {}: {}", queryNum, e.getMessage());
+                } finally {
+                    completedQueries.incrementAndGet();
+                    completionLatch.countDown();
+                }
+            });
+        }
+
+        // Start all the search threads..
+        logger.info("Starting all search threads");
+        startLatch.countDown();
+
+        // Poll the Live Queries API repeatedly to try to catch the searches
+        logger.info("Beginning to poll the live queries API");
+        boolean foundLiveQueries = false;
+        List<Map<String, Object>> liveQueries = new ArrayList<>();
+
+        // Create a separate thread to poll the API while search queries are running
+        for (int attempt = 0; attempt < MAX_POLL_ATTEMPTS && !shouldStop.get(); attempt++) {
+            Thread.sleep(POLL_INTERVAL_MS);
+
+            // Skip further checking if all queries are done already
+            if (completedQueries.get() >= CONCURRENT_QUERIES) {
+                logger.info("All queries completed, stopping poll attempts");
+                break;
+            }
+
+            try {
+                // Call the Live Queries API
+                Request liveQueriesRequest = new Request("GET", QueryInsightsSettings.LIVE_QUERIES_BASE_URI + "?verbose=true");
+                Response liveQueriesResponse = client().performRequest(liveQueriesRequest);
+
+                // Parse response
+                Map<String, Object> responseMap = entityAsMap(liveQueriesResponse);
+                assertTrue(responseMap.containsKey("live_queries"));
+                liveQueries = (List<Map<String, Object>>) responseMap.get("live_queries");
+
+                logger.info("Polling attempt {}: Found {} live queries", attempt, liveQueries.size());
+
+                if (liveQueries != null && !liveQueries.isEmpty()) {
+                    logger.info("Successfully found live queries on attempt {}", attempt);
+                    foundLiveQueries = true;
+                    break;
+                }
+            } catch (Exception e) {
+                logger.error("Error polling live queries API: {}", e.getMessage());
+            }
+        }
+
+        // Signal all threads to stop
+        shouldStop.set(true);
+
+        // Wait for all search threads to complete with timeout
+        boolean allThreadsCompleted = completionLatch.await(QUERY_DURATION_MS * 2, TimeUnit.MILLISECONDS);
+        logger.info("All threads completed? {}", allThreadsCompleted);
+
+        // Shut down executor
+        threadPool.shutdownNow();
+
+        // We either found live queries or exhausted our polling attempts.
+        if (foundLiveQueries) {
+            logger.info("Test detected live queries: {}", liveQueries);
+            assertTrue("Should have found at least one live query", !liveQueries.isEmpty());
+
+            // Validate the format of live queries based on LiveQueries.java and LiveQueriesResponse.java
+            for (Map<String, Object> query : liveQueries) {
+                // Verify required fields are present
+                assertTrue("Query should have timestamp", query.containsKey("timestamp"));
+                assertTrue("Query should have id", query.containsKey("id"));
+                assertTrue("Query should have node_id", query.containsKey("node_id"));
+                assertTrue("Query should have measurements", query.containsKey("measurements"));
+                assertTrue("Query should have description", query.containsKey("description"));
+
+                // Validate timestamp is a number
+                assertTrue("Timestamp should be a number", query.get("timestamp") instanceof Number);
+
+                // Validate id is a string
+                assertTrue("ID should be a string", query.get("id") instanceof String);
+
+                // Validate node_id is a string
+                assertTrue("Node ID should be a string", query.get("node_id") instanceof String);
+
+                // Validate measurements structure
+                Map<String, Object> measurements = (Map<String, Object>) query.get("measurements");
+                assertTrue("Measurements should include latency", measurements.containsKey("latency"));
+                assertTrue("Measurements should include cpu", measurements.containsKey("cpu"));
+                assertTrue("Measurements should include memory", measurements.containsKey("memory"));
+
+                // Validate each measurement's structure
+                for (String metricType : new String[] { "latency", "cpu", "memory" }) {
+                    Map<String, Object> metric = (Map<String, Object>) measurements.get(metricType);
+                    assertTrue("Metric should have number", metric.containsKey("number"));
+                    assertTrue("Metric should have count", metric.containsKey("count"));
+                    assertTrue("Metric should have aggregationType", metric.containsKey("aggregationType"));
+
+                    // Validate number is a number
+                    assertTrue("Number should be a number", metric.get("number") instanceof Number);
+
+                    // Validate count is a number
+                    assertTrue("Count should be a number", metric.get("count") instanceof Number);
+
+                    // Validate aggregationType is a string
+                    assertTrue("AggregationType should be a string", metric.get("aggregationType") instanceof String);
+                }
+                assertTrue("Description should be a string", query.get("description") instanceof String);
+            }
+        } else {
+            logger.info(
+                "No live queries found after {} attempts - this is acceptable due to test environment limitations",
+                MAX_POLL_ATTEMPTS
path: src/test/java/org/opensearch/plugin/insights/rules/resthandler/live_queries/LiveQueriesRestIT.java
user: login: ansjcy
body: Just updated the PR and removed this fallback. I also ran these tests without this fallback for 100 times and it is always passing. So my previous assumption that this case can be flaky is wrong.
```
TOTAL_RUNS=100
failures=0

for ((i=1; i<=TOTAL_RUNS; i++)); do
  echo "=== Run $i/$TOTAL_RUNS ==="
  if ! ./gradlew integTest --tests "org.opensearch.plugin.insights.rules.resthandler.live_queries.LiveQueriesRestIT"; then
    echo "Test failed on run $i"
    ((failures++))
  fi
  echo
done
successes=$((TOTAL_RUNS - failures))
echo "=== Summary ==="
echo "Total runs   : $TOTAL_RUNS"
echo "Successes    : $successes"
echo "Failures     : $failures"
```
Results:
```
=== Summary ===
Total runs   : 100
Successes    : 100
Failures     : 0
```
created_at: 2025-04-22T05:54:48Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,87 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.resthandler.live_queries;
+
+import static org.opensearch.plugin.insights.settings.QueryInsightsSettings.LIVE_QUERIES_BASE_URI;
+import static org.opensearch.rest.RestRequest.Method.GET;
+
+import java.util.List;
+import java.util.Set;
+import org.opensearch.common.settings.Settings;
+import org.opensearch.core.common.Strings;
+import org.opensearch.core.rest.RestStatus;
+import org.opensearch.core.xcontent.ToXContent;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesAction;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesRequest;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesResponse;
+import org.opensearch.plugin.insights.rules.model.MetricType;
+import org.opensearch.plugin.insights.settings.QueryInsightsSettings;
+import org.opensearch.rest.BaseRestHandler;
+import org.opensearch.rest.BytesRestResponse;
+import org.opensearch.rest.RestChannel;
+import org.opensearch.rest.RestRequest;
+import org.opensearch.rest.RestResponse;
+import org.opensearch.rest.action.RestResponseListener;
+import org.opensearch.transport.client.node.NodeClient;
+
+/**
+ * Rest action to get ongoing live queries
+ */
+public class RestLiveQueriesAction extends BaseRestHandler {
+
+    /**
+     * Constructor for RestLiveQueriesAction
+     */
+    public RestLiveQueriesAction() {}
+
+    @Override
+    public List<Route> routes() {
+        return List.of(new Route(GET, LIVE_QUERIES_BASE_URI));
+    }
+
+    @Override
+    public String getName() {
+        return "query_insights_live_queries_action";
+    }
+
+    @Override
+    public RestChannelConsumer prepareRequest(final RestRequest request, final NodeClient client) {
+        final LiveQueriesRequest liveQueriesRequest = prepareRequest(request);
+
+        return channel -> client.execute(LiveQueriesAction.INSTANCE, liveQueriesRequest, liveQueriesResponse(channel));
+    }
+
+    static LiveQueriesRequest prepareRequest(final RestRequest request) {
+        final String[] nodesIds = Strings.splitStringByCommaToArray(request.param("nodeId"));
+        final boolean verbose = request.paramAsBoolean("verbose", true);
+        final String sortParam = request.param("sort", MetricType.LATENCY.toString());
+        final MetricType sortBy = MetricType.fromString(sortParam);
path: src/main/java/org/opensearch/plugin/insights/rules/resthandler/live_queries/RestLiveQueriesAction.java
user: login: dzane17
body: Looks like `fromString` is validating the user input. Can you verify it returns a meaningful error message? 

If not, this is the message we emit for MetricType args in RestTopQueriesAction:
```
        if (!ALLOWED_METRICS.contains(metricType)) {
            throw new IllegalArgumentException(
                String.format(Locale.ROOT, "request [%s] contains invalid metric type [%s]", request.path(), metricType)
            );
        }
```


created_at: 2025-04-22T22:08:00Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,131 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.transport.live_queries;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.opensearch.action.admin.cluster.node.tasks.list.ListTasksRequest;
+import org.opensearch.action.admin.cluster.node.tasks.list.ListTasksResponse;
+import org.opensearch.action.support.ActionFilters;
+import org.opensearch.action.support.HandledTransportAction;
+import org.opensearch.common.inject.Inject;
+import org.opensearch.core.action.ActionListener;
+import org.opensearch.core.tasks.resourcetracker.TaskResourceStats;
+import org.opensearch.core.tasks.resourcetracker.TaskResourceUsage;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesAction;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesRequest;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesResponse;
+import org.opensearch.plugin.insights.rules.model.Attribute;
+import org.opensearch.plugin.insights.rules.model.Measurement;
+import org.opensearch.plugin.insights.rules.model.MetricType;
+import org.opensearch.plugin.insights.rules.model.SearchQueryRecord;
+import org.opensearch.tasks.Task;
+import org.opensearch.tasks.TaskInfo;
+import org.opensearch.threadpool.ThreadPool;
+import org.opensearch.transport.TransportService;
+import org.opensearch.transport.client.Client;
+
+/**
+ * Transport action for fetching ongoing live queries
+ */
+public class TransportLiveQueriesAction extends HandledTransportAction<LiveQueriesRequest, LiveQueriesResponse> {
+
+    private static final Logger logger = LogManager.getLogger(TransportLiveQueriesAction.class);
+    private static final String TOTAL = "total";
+
+    private final Client client;
+
+    @Inject
+    public TransportLiveQueriesAction(final TransportService transportService, final Client client, final ActionFilters actionFilters) {
+        super(LiveQueriesAction.NAME, transportService, actionFilters, LiveQueriesRequest::new, ThreadPool.Names.GENERIC);
+        this.client = client;
+    }
+
+    @Override
+    protected void doExecute(final Task task, final LiveQueriesRequest request, final ActionListener<LiveQueriesResponse> listener) {
+        ListTasksRequest listTasksRequest = new ListTasksRequest().setDetailed(request.isVerbose()).setActions("indices:data/read/search*");
+
+        // Set nodes filter if provided in the request
+        String[] requestedNodeIds = request.nodesIds();
+        if (requestedNodeIds != null && requestedNodeIds.length > 0) {
+            listTasksRequest.setNodes(requestedNodeIds);
+        }
+
+        // Execute tasks request asynchronously to avoid blocking
+        client.admin().cluster().listTasks(listTasksRequest, new ActionListener<ListTasksResponse>() {
+            @Override
+            public void onResponse(ListTasksResponse taskResponse) {
+                try {
+                    List<SearchQueryRecord> allFilteredRecords = new ArrayList<>();
+                    for (TaskInfo taskInfo : taskResponse.getTasks()) {
+                        if (!taskInfo.getAction().startsWith("indices:data/read/search")) {
+                            continue;
+                        }
+                        long timestamp = taskInfo.getStartTime();
+                        String nodeId = taskInfo.getTaskId().getNodeId();
+                        long runningNanos = taskInfo.getRunningTimeNanos();
+
+                        Map<MetricType, Measurement> measurements = new HashMap<>();
+                        measurements.put(MetricType.LATENCY, new Measurement(runningNanos));
+
+                        long cpuNanos = 0L;
+                        long memBytes = 0L;
+                        TaskResourceStats stats = taskInfo.getResourceStats();
+                        if (stats != null) {
+                            Map<String, TaskResourceUsage> usageInfo = stats.getResourceUsageInfo();
+                            if (usageInfo != null) {
+                                TaskResourceUsage totalUsage = usageInfo.get(TOTAL);
+                                if (totalUsage != null) {
+                                    cpuNanos = totalUsage.getCpuTimeInNanos();
+                                    memBytes = totalUsage.getMemoryInBytes();
+                                }
+                            }
+                        }
+                        measurements.put(MetricType.CPU, new Measurement(cpuNanos));
+                        measurements.put(MetricType.MEMORY, new Measurement(memBytes));
+
+                        Map<Attribute, Object> attributes = new HashMap<>();
+                        attributes.put(Attribute.NODE_ID, nodeId);
+                        if (request.isVerbose()) {
+                            attributes.put(Attribute.DESCRIPTION, taskInfo.getDescription());
+                        }
+
+                        SearchQueryRecord record = new SearchQueryRecord(
+                            timestamp,
+                            measurements,
+                            attributes,
+                            taskInfo.getTaskId().toString()
+                        );
+                        allFilteredRecords.add(record);
+                    }
+
+                    // Sort descending by the requested metric and apply size limit in one pass
+                    List<SearchQueryRecord> finalRecords = allFilteredRecords.stream()
+                        .sorted((a, b) -> SearchQueryRecord.compare(b, a, request.getSortBy()))
+                        .limit(request.getSize() < 0 ? Long.MAX_VALUE : request.getSize())
+                        .toList();
path: src/main/java/org/opensearch/plugin/insights/rules/transport/live_queries/TransportLiveQueriesAction.java
user: login: dzane17
body: I believe this syntax `toList()` will break on 2.x Java versions. You can use `.collect(Collectors.toList())`
created_at: 2025-04-22T22:23:31Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,131 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.transport.live_queries;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.opensearch.action.admin.cluster.node.tasks.list.ListTasksRequest;
+import org.opensearch.action.admin.cluster.node.tasks.list.ListTasksResponse;
+import org.opensearch.action.support.ActionFilters;
+import org.opensearch.action.support.HandledTransportAction;
+import org.opensearch.common.inject.Inject;
+import org.opensearch.core.action.ActionListener;
+import org.opensearch.core.tasks.resourcetracker.TaskResourceStats;
+import org.opensearch.core.tasks.resourcetracker.TaskResourceUsage;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesAction;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesRequest;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesResponse;
+import org.opensearch.plugin.insights.rules.model.Attribute;
+import org.opensearch.plugin.insights.rules.model.Measurement;
+import org.opensearch.plugin.insights.rules.model.MetricType;
+import org.opensearch.plugin.insights.rules.model.SearchQueryRecord;
+import org.opensearch.tasks.Task;
+import org.opensearch.tasks.TaskInfo;
+import org.opensearch.threadpool.ThreadPool;
+import org.opensearch.transport.TransportService;
+import org.opensearch.transport.client.Client;
+
+/**
+ * Transport action for fetching ongoing live queries
+ */
+public class TransportLiveQueriesAction extends HandledTransportAction<LiveQueriesRequest, LiveQueriesResponse> {
+
+    private static final Logger logger = LogManager.getLogger(TransportLiveQueriesAction.class);
+    private static final String TOTAL = "total";
+
+    private final Client client;
+
+    @Inject
+    public TransportLiveQueriesAction(final TransportService transportService, final Client client, final ActionFilters actionFilters) {
+        super(LiveQueriesAction.NAME, transportService, actionFilters, LiveQueriesRequest::new, ThreadPool.Names.GENERIC);
+        this.client = client;
+    }
+
+    @Override
+    protected void doExecute(final Task task, final LiveQueriesRequest request, final ActionListener<LiveQueriesResponse> listener) {
+        ListTasksRequest listTasksRequest = new ListTasksRequest().setDetailed(request.isVerbose()).setActions("indices:data/read/search*");
+
+        // Set nodes filter if provided in the request
+        String[] requestedNodeIds = request.nodesIds();
+        if (requestedNodeIds != null && requestedNodeIds.length > 0) {
+            listTasksRequest.setNodes(requestedNodeIds);
+        }
+
+        // Execute tasks request asynchronously to avoid blocking
+        client.admin().cluster().listTasks(listTasksRequest, new ActionListener<ListTasksResponse>() {
+            @Override
+            public void onResponse(ListTasksResponse taskResponse) {
+                try {
+                    List<SearchQueryRecord> allFilteredRecords = new ArrayList<>();
+                    for (TaskInfo taskInfo : taskResponse.getTasks()) {
+                        if (!taskInfo.getAction().startsWith("indices:data/read/search")) {
+                            continue;
+                        }
+                        long timestamp = taskInfo.getStartTime();
+                        String nodeId = taskInfo.getTaskId().getNodeId();
+                        long runningNanos = taskInfo.getRunningTimeNanos();
+
+                        Map<MetricType, Measurement> measurements = new HashMap<>();
+                        measurements.put(MetricType.LATENCY, new Measurement(runningNanos));
+
+                        long cpuNanos = 0L;
+                        long memBytes = 0L;
+                        TaskResourceStats stats = taskInfo.getResourceStats();
+                        if (stats != null) {
+                            Map<String, TaskResourceUsage> usageInfo = stats.getResourceUsageInfo();
+                            if (usageInfo != null) {
+                                TaskResourceUsage totalUsage = usageInfo.get(TOTAL);
+                                if (totalUsage != null) {
+                                    cpuNanos = totalUsage.getCpuTimeInNanos();
+                                    memBytes = totalUsage.getMemoryInBytes();
+                                }
+                            }
+                        }
+                        measurements.put(MetricType.CPU, new Measurement(cpuNanos));
+                        measurements.put(MetricType.MEMORY, new Measurement(memBytes));
+
+                        Map<Attribute, Object> attributes = new HashMap<>();
+                        attributes.put(Attribute.NODE_ID, nodeId);
+                        if (request.isVerbose()) {
+                            attributes.put(Attribute.DESCRIPTION, taskInfo.getDescription());
+                        }
+
+                        SearchQueryRecord record = new SearchQueryRecord(
+                            timestamp,
+                            measurements,
+                            attributes,
+                            taskInfo.getTaskId().toString()
+                        );
+                        allFilteredRecords.add(record);
+                    }
+
+                    // Sort descending by the requested metric and apply size limit in one pass
+                    List<SearchQueryRecord> finalRecords = allFilteredRecords.stream()
+                        .sorted((a, b) -> SearchQueryRecord.compare(b, a, request.getSortBy()))
+                        .limit(request.getSize() < 0 ? Long.MAX_VALUE : request.getSize())
+                        .toList();
path: src/main/java/org/opensearch/plugin/insights/rules/transport/live_queries/TransportLiveQueriesAction.java
user: login: ansjcy
body: I don't think we want to backport the 3.0 features to 2.x anymore since they will be in maintenance mode, so should this be fine?
created_at: 2025-04-22T22:30:17Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,87 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.resthandler.live_queries;
+
+import static org.opensearch.plugin.insights.settings.QueryInsightsSettings.LIVE_QUERIES_BASE_URI;
+import static org.opensearch.rest.RestRequest.Method.GET;
+
+import java.util.List;
+import java.util.Set;
+import org.opensearch.common.settings.Settings;
+import org.opensearch.core.common.Strings;
+import org.opensearch.core.rest.RestStatus;
+import org.opensearch.core.xcontent.ToXContent;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesAction;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesRequest;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesResponse;
+import org.opensearch.plugin.insights.rules.model.MetricType;
+import org.opensearch.plugin.insights.settings.QueryInsightsSettings;
+import org.opensearch.rest.BaseRestHandler;
+import org.opensearch.rest.BytesRestResponse;
+import org.opensearch.rest.RestChannel;
+import org.opensearch.rest.RestRequest;
+import org.opensearch.rest.RestResponse;
+import org.opensearch.rest.action.RestResponseListener;
+import org.opensearch.transport.client.node.NodeClient;
+
+/**
+ * Rest action to get ongoing live queries
+ */
+public class RestLiveQueriesAction extends BaseRestHandler {
+
+    /**
+     * Constructor for RestLiveQueriesAction
+     */
+    public RestLiveQueriesAction() {}
+
+    @Override
+    public List<Route> routes() {
+        return List.of(new Route(GET, LIVE_QUERIES_BASE_URI));
+    }
+
+    @Override
+    public String getName() {
+        return "query_insights_live_queries_action";
+    }
+
+    @Override
+    public RestChannelConsumer prepareRequest(final RestRequest request, final NodeClient client) {
+        final LiveQueriesRequest liveQueriesRequest = prepareRequest(request);
+
+        return channel -> client.execute(LiveQueriesAction.INSTANCE, liveQueriesRequest, liveQueriesResponse(channel));
+    }
+
+    static LiveQueriesRequest prepareRequest(final RestRequest request) {
+        final String[] nodesIds = Strings.splitStringByCommaToArray(request.param("nodeId"));
+        final boolean verbose = request.paramAsBoolean("verbose", true);
+        final String sortParam = request.param("sort", MetricType.LATENCY.toString());
+        final MetricType sortBy = MetricType.fromString(sortParam);
path: src/main/java/org/opensearch/plugin/insights/rules/resthandler/live_queries/RestLiveQueriesAction.java
user: login: ansjcy
body: Good point, now the error message is like
```
{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "No enum constant org.opensearch.plugin.insights.rules.model.MetricType.ANY"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "No enum constant org.opensearch.plugin.insights.rules.model.MetricType.ANY"
  },
  "status" : 400
}
```
Let me refactor this part.
created_at: 2025-04-22T22:33:25Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,87 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.plugin.insights.rules.resthandler.live_queries;
+
+import static org.opensearch.plugin.insights.settings.QueryInsightsSettings.LIVE_QUERIES_BASE_URI;
+import static org.opensearch.rest.RestRequest.Method.GET;
+
+import java.util.List;
+import java.util.Set;
+import org.opensearch.common.settings.Settings;
+import org.opensearch.core.common.Strings;
+import org.opensearch.core.rest.RestStatus;
+import org.opensearch.core.xcontent.ToXContent;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesAction;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesRequest;
+import org.opensearch.plugin.insights.rules.action.live_queries.LiveQueriesResponse;
+import org.opensearch.plugin.insights.rules.model.MetricType;
+import org.opensearch.plugin.insights.settings.QueryInsightsSettings;
+import org.opensearch.rest.BaseRestHandler;
+import org.opensearch.rest.BytesRestResponse;
+import org.opensearch.rest.RestChannel;
+import org.opensearch.rest.RestRequest;
+import org.opensearch.rest.RestResponse;
+import org.opensearch.rest.action.RestResponseListener;
+import org.opensearch.transport.client.node.NodeClient;
+
+/**
+ * Rest action to get ongoing live queries
+ */
+public class RestLiveQueriesAction extends BaseRestHandler {
+
+    /**
+     * Constructor for RestLiveQueriesAction
+     */
+    public RestLiveQueriesAction() {}
+
+    @Override
+    public List<Route> routes() {
+        return List.of(new Route(GET, LIVE_QUERIES_BASE_URI));
+    }
+
+    @Override
+    public String getName() {
+        return "query_insights_live_queries_action";
+    }
+
+    @Override
+    public RestChannelConsumer prepareRequest(final RestRequest request, final NodeClient client) {
+        final LiveQueriesRequest liveQueriesRequest = prepareRequest(request);
+
+        return channel -> client.execute(LiveQueriesAction.INSTANCE, liveQueriesRequest, liveQueriesResponse(channel));
+    }
+
+    static LiveQueriesRequest prepareRequest(final RestRequest request) {
+        final String[] nodesIds = Strings.splitStringByCommaToArray(request.param("nodeId"));
+        final boolean verbose = request.paramAsBoolean("verbose", true);
+        final String sortParam = request.param("sort", MetricType.LATENCY.toString());
+        final MetricType sortBy = MetricType.fromString(sortParam);
path: src/main/java/org/opensearch/plugin/insights/rules/resthandler/live_queries/RestLiveQueriesAction.java
user: login: ansjcy
body: Added validation. 
created_at: 2025-04-22T22:42:09Z
author_association: MEMBER

PR comments:
user: login: dzane17
created_at: 2025-04-18T23:48:35Z
body: @ansjcy Looks good. Do we need any of these features or are they planned for the future?

1. Sort – Add an option to sort by timestamp, latency, CPU, memory, node id. Or are we planning to handle this entirely in the dashboard?
2. Size/Limit – Limit the number of results returned
3. Duration Threshold – Filter for tasks running longer than a specified duration
author_association: MEMBER
user: login: ansjcy
created_at: 2025-04-19T00:25:28Z
body: @dzane17 Those are all good points, by default I'm returning records sorted by latency.
We should do the sort/filter on backend. For now since I'm only considering the SearchTask, the resource usages won't make a lot of sense. But you are right, let me at least add those API parameters.
author_association: MEMBER
user: login: ansjcy
created_at: 2025-04-22T00:23:45Z
body: There are issues with the current implementation: we are fetching task level information from coordinator level, thus we don't need Node level request and response objects - we can consider adding it later because we might need to correlate shard level information, but doing it now seems over engineering. I'll refactor this now.
author_association: MEMBER
user: login: ansjcy
created_at: 2025-04-22T05:57:24Z
body: Updated to address the issues mentioned in my above comments and added parameters
https://github.com/opensearch-project/query-insights/pull/295#issuecomment-2819762552

I'm adding `sort`, `size`, `nodeId` for now. I won't add the threshold for now because it can be a little bit tricky when used together with sort and size. I think we need further discussions on `threshold` and this cannot be done in the 3.0 timeframe. cc @dzane17 
author_association: MEMBER