Repository: opensearch-project/OpenSearch-Dashboards
Pull request #9501: Initial logic flow for building visualization expressions in discover
Author: login: abbyhu2000
Labels: name: distinguished-contributor
name: failed changeset
Created: 2025-03-06T19:33:53Z | Merged: 2025-03-18T22:24:14Z

### Description
Basic discover visualization for metrics using vega expressions


https://github.com/user-attachments/assets/e4cc5c99-fb6c-4ecb-9472-0d270be36c0b






### Issues Resolved

<!-- List any issues this PR will resolve. Prefix the issue with the keyword closes, fixes, fix -->
<!-- Example: closes #1234 or fixes <Issue_URL> -->

## Screenshot

<!-- Attach any relevant screenshots. Any change to the UI requires an attached screenshot in the PR Description -->

## Testing the changes

<!--
  Please provide detailed steps for validating your changes. This could involve specific commands to run,
  pages to visit, scenarios to try or any other information that would help reviewers verify
  the functionality of your change
-->

## Changelog
<!--
Add a short but concise sentence about the impact of this pull request. Prefix an entry with the type of change they correspond to: breaking, chore, deprecate, doc, feat, fix, infra, refactor, test.
- fix: Update the graph
- feat: Add a new feature

If this change does not need to added to the changelog, just add a single `skip` line e.g.
- skip

Descriptions following the prefixes must be 100 characters long or less
-->

### Check List

- [ ] All tests pass
  - [ ] `yarn test:jest`
  - [ ] `yarn test:jest_integration`
- [ ] New functionality includes testing.
- [ ] New functionality has been documented.
- [ ] Update [CHANGELOG.md](./../CHANGELOG.md)
- [ ] Commits are signed per the DCO using --signoff



Diff:
diff --git a/.github/workflows/build_plugin.yml b/.github/workflows/build_plugin.yml
new file mode 100644
index 000000000000..d21ec61be851
--- /dev/null
+++ b/.github/workflows/build_plugin.yml
@@ -0,0 +1,195 @@
+name: Build Release & Plugins
+
+on:
+  push:
+    branches:
+      - feature/node-2x
+    paths-ignore:
+      - '**.md'
+      - 'docs/**'
+      - '.github/**'
+      - '!.github/workflows/**'
+  workflow_dispatch:
+
+env:
+  VERSION: '3.0.0'
+
+jobs:
+  build_release:
+    name: Build OpenSearch Dashboards Release
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout OpenSearch Dashboards
+        uses: actions/checkout@v3
+
+      - name: Setup Node
+        uses: actions/setup-node@v4
+        with:
+          node-version-file: '.nvmrc'
+          registry-url: 'https://registry.npmjs.org'
+
+      - name: Install Yarn
+        run: npm install -g yarn@1.22.10
+
+      - name: Bootstrap OSD Core
+        run: yarn osd bootstrap --single-version=loose
+
+      - name: Build Release Artifact
+        run: yarn build-platform --linux --skip-os-packages --release
+
+      - name: Upload Release Artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: opensearch-dashboards-${{ env.VERSION }}
+          path: target/opensearch-dashboards-*.tar.gz
+          retention-days: 1
+
+  build_plugins:
+    name: Build Plugin ${{ matrix.plugin.name }}
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        plugin:
+          - name: reportsDashboards
+            repo: opensearch-project/dashboards-reporting
+            ref: main
+          - name: securityDashboards
+            repo: opensearch-project/security-dashboards-plugin
+            ref: main
+          - name: queryWorkbenchDashboards
+            repo: opensearch-project/dashboards-query-workbench
+            ref: main
+          - name: searchRelevanceDashboards
+            repo: opensearch-project/dashboards-search-relevance
+            ref: main
+          - name: observabilityDashboards
+            repo: opensearch-project/dashboards-observability
+            ref: main
+          - name: ganttChartDashboards
+            repo: opensearch-project/dashboards-visualizations
+            ref: main
+          - name: assistantDashboards
+            repo: opensearch-project/dashboards-assistant
+            ref: main
+          - name: flowFrameworkDashboards
+            repo: opensearch-project/dashboards-flow-framework
+            ref: main
+          - name: notificationsDashboards
+            repo: opensearch-project/dashboards-notifications
+            ref: main
+          - name: customImportMapDashboards
+            repo: opensearch-project/dashboards-maps
+            ref: main
+          - name: anomalyDetectionDashboards
+            repo: opensearch-project/anomaly-detection-dashboards-plugin
+            ref: main
+          - name: mlCommonsDashboards
+            repo: opensearch-project/ml-commons-dashboards
+            ref: main
+          - name: indexManagementDashboards
+            repo: opensearch-project/index-management-dashboards-plugin
+            ref: main
+          - name: alertingDashboards
+            repo: opensearch-project/alerting-dashboards-plugin
+            ref: main
+          - name: queryInsightsDashboards
+            repo: opensearch-project/query-insights-dashboards
+            ref: main
+    steps:
+      - name: Checkout OpenSearch Dashboards
+        uses: actions/checkout@v3
+
+      - name: Setup Node
+        uses: actions/setup-node@v4
+        with:
+          node-version-file: '.nvmrc'
+          registry-url: 'https://registry.npmjs.org'
+
+      - name: Install Yarn
+        run: npm install -g yarn@1.22.10
+
+      # Check out the plugin repository into the main repo's plugins folder.
+      - name: Checkout Plugin
+        uses: actions/checkout@v3
+        with:
+          repository: ${{ matrix.plugin.repo }}
+          ref: ${{ matrix.plugin.ref }}
+          path: plugins/${{ matrix.plugin.name }}
+
+      - name: Bootstrap with Plugins
+        run: yarn osd bootstrap --single-version=loose
+
+      - name: Build Plugin
+        working-directory: plugins/${{ matrix.plugin.name }}
+        run: node ../../scripts/plugin_helpers build --opensearch-dashboards-version=${{ env.VERSION }} 2>&1 | tee build.log
+
+      - name: Upload Plugin Artifact
+        if: success()
+        uses: actions/upload-artifact@v4
+        with:
+          name: plugin-${{ matrix.plugin.name }}
+          path: plugins/${{ matrix.plugin.name }}/build/*.zip
+          retention-days: 1
+
+      - name: Upload Build Log
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: build-log-${{ matrix.plugin.name }}
+          path: plugins/${{ matrix.plugin.name }}/build.log
+          retention-days: 1
+
+  assemble_release:
+    name: Assemble Final Release
+    needs: [build_release, build_plugins]
+    runs-on: ubuntu-latest
+    steps:
+      - name: Download Release Artifact
+        uses: actions/download-artifact@v4
+        with:
+          name: opensearch-dashboards-${{ env.VERSION }}
+          path: ./
+
+      - name: Extract Release
+        run: |
+          unzip target/opensearch-dashboards-${{ env.VERSION }}.zip -d target/
+          mkdir -p opensearch-dashboards-${{ env.VERSION }}
+          tar -xzf target/opensearch-dashboards-*.tar.gz -C opensearch-dashboards-${{ env.VERSION }}
+
+      - name: Download All Plugin Artifacts
+        uses: actions/download-artifact@v4
+        with:
+          path: plugins
+          pattern: plugin-*
+          merge-multiple: true
+
+      - name: Install Plugins
+        run: |
+          mkdir -p install_logs
+          for plugin in plugins/*.zip; do
+            if [ -f "$plugin" ]; then
+              plugin_name=$(basename "$plugin" .zip)
+              echo "Installing plugin: $plugin_name"
+              if ./opensearch-dashboards-${{ env.VERSION }}/bin/opensearch-dashboards-plugin install "file://$plugin" 2>&1 | tee install_logs/${plugin_name}_install.log; then
+                echo "✅ Successfully installed $plugin_name"
+              else
+                echo "⚠️ Failed to install $plugin_name"
+              fi
+            fi
+          done
+
+      - name: Upload Install Logs
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: plugin-install-logs
+          path: install_logs/
+          retention-days: 1
+
+      - name: Upload Final Release
+        uses: actions/upload-artifact@v4
+        with:
+          name: opensearch-dashboards-${{ env.VERSION }}-with-plugins
+          path: opensearch-dashboards-${{ env.VERSION }}/
+          retention-days: 1
diff --git a/src/plugins/discover/opensearch_dashboards.json b/src/plugins/discover/opensearch_dashboards.json
index c49528d2e227..4cadfdeaec9e 100644
--- a/src/plugins/discover/opensearch_dashboards.json
+++ b/src/plugins/discover/opensearch_dashboards.json
@@ -14,7 +14,8 @@
     "navigation",
     "uiActions",
     "visualizations",
-    "usageCollection"
+    "usageCollection",
+    "expressions"
   ],
   "optionalPlugins": ["home", "share"],
   "requiredBundles": [
diff --git a/src/plugins/discover/public/application/components/chart/utils/create_visualization_configs.ts b/src/plugins/discover/public/application/components/chart/utils/create_visualization_configs.ts
new file mode 100644
index 000000000000..87844fe367ba
--- /dev/null
+++ b/src/plugins/discover/public/application/components/chart/utils/create_visualization_configs.ts
@@ -0,0 +1,48 @@
+/*
+ * Copyright OpenSearch Contributors
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+import { OpenSearchSearchHit } from '../../../doc_views/doc_views_types';
+import { DataPublicPluginStart, IndexPattern } from '../../../../../../data/public';
+
+export function createVisualizationConfigs(
+  indexPattern: IndexPattern,
+  histogramInterval: string,
+  data: DataPublicPluginStart,
+  row: OpenSearchSearchHit
+) {
+  const timeField = indexPattern.timeFieldName!;
+  const visStateAggs = [
+    {
+      type: 'date_histogram',
+      schema: 'segment',
+      params: {
+        field: timeField,
+        interval: histogramInterval,
+        timeRange: data.query.timefilter.timefilter.getTime(),
+      },
+    },
+  ];
+
+  row._source.keys().forEach((key) => {
+    // We want to get all the fields except the time field
+    if (key === timeField) {
+      return;
+    }
+    visStateAggs.push({
+      type: 'count',
+      schema: 'metric',
+      params: {},
+    });
+  });
+
+  // If index pattern is created before the index, this function will fail since the required fields for the histogram agg will be missing.
+  try {
+    return data.search.aggs.createAggConfigs(indexPattern, visStateAggs);
+  } catch (error) {
+    // Just display the error to the user but continue to render the rest of the page
+    data.search.showError(error as Error);
+    return;
+  }
+}
diff --git a/src/plugins/discover/public/application/components/visualizations/vislib/line/line_vis_type.ts b/src/plugins/discover/public/application/components/visualizations/vislib/line/line_vis_type.ts
new file mode 100644
index 000000000000..742cf7abae24
--- /dev/null
+++ b/src/plugins/discover/public/application/components/visualizations/vislib/line/line_vis_type.ts
@@ -0,0 +1,20 @@
+/*
+ * Copyright OpenSearch Contributors
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+import { VisualizationType } from '../../../../view_components/utils/use_visualization_types';
+import { toExpression } from './to_expression';
+
+export interface LineOptionsDefaults {
+  type: 'line';
+}
+
+export const createLineConfig = (): VisualizationType<LineOptionsDefaults> => ({
+  name: 'line',
+  title: 'Line',
+  icon: 'visLine',
+  description: 'Display line chart',
+  toExpression,
+  ui: {},
+});
diff --git a/src/plugins/discover/public/application/components/visualizations/vislib/line/to_expression.ts b/src/plugins/discover/public/application/components/visualizations/vislib/line/to_expression.ts
new file mode 100644
index 000000000000..8cd06b55b881
--- /dev/null
+++ b/src/plugins/discover/public/application/components/visualizations/vislib/line/to_expression.ts
@@ -0,0 +1,164 @@
+/*
+ * Copyright OpenSearch Contributors
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+import {
+  buildExpression,
+  buildExpressionFunction,
+  ExpressionFunctionOpenSearchDashboards,
+  IExpressionLoaderParams,
+} from '../../../../../../../expressions/public';
+import { OpenSearchSearchHit } from '../../../../doc_views/doc_views_types';
+import { IndexPattern } from '../../../../../../../data/public';
+import { DiscoverViewServices } from '../../../../../build_services';
+
+export const toExpression = async (
+  services: DiscoverViewServices,
+  searchContext: IExpressionLoaderParams['searchContext'],
+  rows: OpenSearchSearchHit[],
+  indexPattern: IndexPattern
+) => {
+  const opensearchDashboards = buildExpressionFunction<ExpressionFunctionOpenSearchDashboards>(
+    'opensearchDashboards',
+    {}
+  );
+  const opensearchDashboardsContext = buildExpressionFunction('opensearch_dashboards_context', {
+    timeRange: JSON.stringify(searchContext!.timeRange || {}),
+    filters: JSON.stringify(searchContext!.filters || []),
+    query: JSON.stringify(searchContext!.query || []),
+  });
+
+  const vegaSpec = createVegaSpec(rows, indexPattern);
+
+  const vega = buildExpressionFunction<any>('vega', {
+    spec: JSON.stringify(vegaSpec),
+  });
+
+  return buildExpression([opensearchDashboards, opensearchDashboardsContext, vega]).toString();
+};
+
+const createVegaSpec = (rows: OpenSearchSearchHit[], indexPattern: IndexPattern) => {
+  const columns = Object.keys(rows[0]._source).map((column, index) => {
+    return {
+      field: column,
+      id: index,
+      name: index === 0 ? column : `series-${index}`,
+    };
+  });
+
+  const data = rows.map((row: OpenSearchSearchHit) => {
+    const transformedRow: Record<string, any> = {};
+    for (const column of columns) {
+      transformedRow[column.name] = row._source[column.field];
+    }
+    return transformedRow;
+  });
+
+  console.log('columns', columns);
+  console.log('data', data);
+
+  const xAxisParam = {
+    field: 'timestamp_ms',
+    type: 'temporal',
+    axis: { format: '%b %-d, %Y', title: columns[0].name },
+    scale: {
+      padding: 10,
+      nice: true, // Round the domain to nice values
+    },
+  };
+
+  const layer = [];
+
+  const colorPalette = [
+    '#4C78A8', // blue
+    '#E45756', // red
+    '#72B7B2', // teal
+    '#F58518', // orange
+    '#54A24B', // green
+    '#B279A2', // purple
+    '#D6A5C9', // pink
+    '#9D755D', // brown
+    '#BAB0AC', // gray
+  ];
+
+  for (let i = 1; i < columns.length; i++) {
+    const colorIndex = (i - 1) % colorPalette.length;
+    const color = colorPalette[colorIndex];
+
+    const yAxisParam = {
+      field: columns[i].name,
+      type: 'quantitative',
+      scale: { zero: false },
+      axis: { titleColor: color, title: columns[i].name },
+    };
+
+    const tooltipParam = [
+      {
+        field: 'timestamp_ms',
+        type: 'temporal',
+        title: columns[0].name,
+        format: '%b %-d, %Y @ %H:%M:%S.%L',
+      },
+      { field: columns[i].name, type: 'quantitative', title: columns[i].name },
+    ];
+
+    const markParam = {
+      type: 'line',
+      point: true,
+      strokeWidth: 2,
+      interpolate: 'monotone',
+    };
+
+    const encodingParam = {
+      x: xAxisParam,
+      y: yAxisParam,
+      tooltip: tooltipParam,
+      color: {
+        datum: columns[i].name,
+        type: 'nominal',
+        scale: {
+          domain: columns.slice(1).map((col) => col.name),
+          range: colorPalette.slice(0, columns.length - 1),
+        },
+        legend: {
+          title: 'Series',
+          labelExpr: "datum.label + ': ' + '" + columns[i].field + "'",
+        },
+      },
+    };
+
+    const layerParam = {
+      mark: markParam,
+      encoding: encodingParam,
+    };
+
+    layer.push(layerParam);
+  }
+
+  const spec = {
+    $schema: 'https://vega.github.io/schema/vega-lite/v5.json',
+    autosize: { type: 'fit', contains: 'padding' },
+    data: { values: data },
+    transform: [{ calculate: `datum['${columns[0].field}']`, as: 'timestamp_ms' }],
+    layer,
+    resolve: {
+      scale: {
+        y: 'independent', // This is critical for multiple y-axes
+      },
+    },
+    config: {
+      axis: {
+        labelFont: 'sans-serif',
+        titleFont: 'sans-serif',
+      },
+      legend: {
+        labelFont: 'sans-serif',
+        titleFont: 'sans-serif',
+      },
+    },
+  };
+
+  console.log('vega spec REAL', spec);
+  return spec;
+};
diff --git a/src/plugins/discover/public/application/view_components/canvas/discover_visualization.scss b/src/plugins/discover/public/application/view_components/canvas/discover_visualization.scss
new file mode 100644
index 000000000000..de79db6b595e
--- /dev/null
+++ b/src/plugins/discover/public/application/view_components/canvas/discover_visualization.scss
@@ -0,0 +1,6 @@
+.discoverVisualization {
+  display: flex;
+  min-height: 600px;
+  min-width: 600px;
+  padding: $euiSizeS;
+}
diff --git a/src/plugins/discover/public/application/view_components/canvas/discover_visualization.tsx b/src/plugins/discover/public/application/view_components/canvas/discover_visualization.tsx
new file mode 100644
index 000000000000..4f646beeab31
--- /dev/null
+++ b/src/plugins/discover/public/application/view_components/canvas/discover_visualization.tsx
@@ -0,0 +1,79 @@
+/*
+ * Copyright OpenSearch Contributors
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+import './discover_visualization.scss';
+
+import { EuiFlexItem, EuiPanel } from '@elastic/eui';
+import React, { useCallback, useEffect, useLayoutEffect, useState } from 'react';
+import { DiscoverViewServices } from '../../../build_services';
+import { useOpenSearchDashboards } from '../../../../../opensearch_dashboards_react/public';
+import { useDiscoverContext } from '../context';
+
+import { SearchData } from '../utils';
+import { IExpressionLoaderParams } from '../../../../../expressions/public';
+import { useVisualizationType } from '../utils/use_visualization_types';
+
+export const DiscoverVisualization = ({ hits, bucketInterval, chartData, rows }: SearchData) => {
+  const { services } = useOpenSearchDashboards<DiscoverViewServices>();
+  const {
+    data: {
+      query: { filterManager, queryString, timefilter },
+    },
+    expressions: { ReactExpressionRenderer },
+  } = services;
+  const { indexPattern } = useDiscoverContext();
+
+  // Get configs and expression utils from a specific visualization type
+  const { toExpression } = useVisualizationType();
+  //   const { aggConfigs, indexPattern } = useAggs();
+  const [expression, setExpression] = useState<string>();
+  const [searchContext, setSearchContext] = useState<IExpressionLoaderParams['searchContext']>({
+    query: queryString.getQuery(),
+    filters: filterManager.getFilters(),
+    timeRange: timefilter.timefilter.getTime(),
+  });
+
+  useEffect(() => {
+    async function loadExpression() {
+      if (!rows || !indexPattern) {
+        return;
+      }
+      const exp = await toExpression(services, searchContext, rows, indexPattern);
+      console.log('expression for vis in discover', exp);
+      setExpression(exp);
+    }
+
+    loadExpression();
+  }, [toExpression, searchContext, rows, indexPattern, services]);
+
+  useLayoutEffect(() => {
+    const subscription = services.data.query.state$.subscribe(({ state }) => {
+      setSearchContext({
+        query: state.query,
+        timeRange: state.time,
+        filters: state.filters,
+      });
+    });
+
+    return () => {
+      subscription.unsubscribe();
+    };
+  }, [services.data.query.state$]);
+
+  return (
+    <EuiPanel className="discoverVisualization" data-test-subj="visualizationLoader">
+      {expression ? (
+        <ReactExpressionRenderer expression={expression} searchContext={searchContext} />
+      ) : (
+        <EuiFlexItem
+          className="discoverVisualization__empty"
+          data-test-subj="emptyDiscoverVisualization"
+        >
+          <>No data to display</>
+        </EuiFlexItem>
+      )}
+    </EuiPanel>
+  );
+};
diff --git a/src/plugins/discover/public/application/view_components/canvas/index.tsx b/src/plugins/discover/public/application/view_components/canvas/index.tsx
index f216fc706ee6..4416aa0a8ae9 100644
--- a/src/plugins/discover/public/application/view_components/canvas/index.tsx
+++ b/src/plugins/discover/public/application/view_components/canvas/index.tsx
@@ -28,6 +28,7 @@ import { OpenSearchSearchHit } from '../../../application/doc_views/doc_views_ty
 import { buildColumns } from '../../utils/columns';
 import './discover_canvas.scss';
 import { HeaderVariant } from '../../../../../../core/public';
+import { DiscoverVisualization } from './discover_visualization';
 
 // eslint-disable-next-line import/no-default-export
 export default function DiscoverCanvas({ setHeaderActionMenu, history, optionalRef }: ViewProps) {
@@ -171,6 +172,7 @@ export default function DiscoverCanvas({ setHeaderActionMenu, history, optionalR
             (isEnhancementsEnabled ? (
               <>
                 <MemoizedDiscoverChartContainer {...fetchState} />
+                <MemoizedDiscoverVisualization {...fetchState} />
                 <MemoizedDiscoverTable rows={rows} scrollToTop={scrollToTop} />
               </>
             ) : (
@@ -197,3 +199,4 @@ export default function DiscoverCanvas({ setHeaderActionMenu, history, optionalR
 
 const MemoizedDiscoverTable = React.memo(DiscoverTable);
 const MemoizedDiscoverChartContainer = React.memo(DiscoverChartContainer);
+const MemoizedDiscoverVisualization = React.memo(DiscoverVisualization);
diff --git a/src/plugins/discover/public/application/view_components/utils/use_visualization_types.ts b/src/plugins/discover/public/application/view_components/utils/use_visualization_types.ts
new file mode 100644
index 000000000000..3d9c33dbd239
--- /dev/null
+++ b/src/plugins/discover/public/application/view_components/utils/use_visualization_types.ts
@@ -0,0 +1,32 @@
+/*
+ * Copyright OpenSearch Contributors
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+import { IconType } from '@elastic/eui';
+import { IExpressionLoaderParams } from '../../../../../expressions/public';
+import { createLineConfig } from '../../components/visualizations/vislib/line/line_vis_type';
+import { DiscoverViewServices } from '../../../build_services';
+import { IndexPattern } from '../../../opensearch_dashboards_services';
+import { OpenSearchSearchHit } from '../../../application/doc_views/doc_views_types';
+
+export interface VisualizationType<T = any> {
+  readonly name: string;
+  readonly title: string;
+  readonly description?: string;
+  readonly icon: IconType;
+  readonly stage?: 'production';
+  readonly ui: {};
+  readonly toExpression: (
+    services: DiscoverViewServices,
+    searchContext: IExpressionLoaderParams['searchContext'],
+    rows: OpenSearchSearchHit[],
+    indexPattern: IndexPattern
+  ) => Promise<string | undefined>;
+  readonly hierarchicalData?: boolean | ((vis: { params: T }) => boolean);
+}
+
+// TODO: Implement this function to return the visualization type based on the query
+export const useVisualizationType = (): VisualizationType => {
+  return createLineConfig();
+};
diff --git a/src/plugins/discover/public/plugin.ts b/src/plugins/discover/public/plugin.ts
index 2b17d334283b..f0a5bd3e5cd3 100644
--- a/src/plugins/discover/public/plugin.ts
+++ b/src/plugins/discover/public/plugin.ts
@@ -77,6 +77,7 @@ declare module '../../share/public' {
   }
 }
 import { UsageCollectionSetup } from '../../usage_collection/public';
+import { ExpressionsPublicPlugin, ExpressionsStart } from '../../expressions/public';
 
 /**
  * @public
@@ -131,6 +132,7 @@ export interface DiscoverSetupPlugins {
   data: DataPublicPluginSetup;
   dataExplorer: DataExplorerPluginSetup;
   usageCollection: UsageCollectionSetup;
+  expressions: ReturnType<ExpressionsPublicPlugin['setup']>;
 }
 
 /**
@@ -147,6 +149,7 @@ export interface DiscoverStartPlugins {
   urlForwarding: UrlForwardingStart;
   inspector: InspectorPublicPluginStart;
   visualizations: VisualizationsStart;
+  expressions: ExpressionsStart;
 }
 
 /**


Reviews:
user: login: abbyhu2000
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-06T22:13:28Z
user: login: abbyhu2000
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-06T22:15:29Z
user: login: kavilla
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-07T11:17:55Z
user: login: kavilla
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-07T11:24:39Z
user: login: kavilla
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-07T11:31:38Z
user: login: kavilla
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-07T11:32:08Z
user: login: abbyhu2000
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-07T20:10:07Z
user: login: abbyhu2000
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-07T20:10:44Z
user: login: abbyhu2000
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-08T00:11:40Z
user: login: abbyhu2000
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-08T00:15:09Z
user: login: abbyhu2000
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-08T00:19:23Z
user: login: d-buckner
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-18T21:06:55Z
user: login: abbyhu2000
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-18T22:12:03Z

Review comments:
diff_hunk: @@ -171,6 +172,7 @@ export default function DiscoverCanvas({ setHeaderActionMenu, history, optionalR
             (isEnhancementsEnabled ? (
               <>
                 <MemoizedDiscoverChartContainer {...fetchState} />
+                <MemoizedDiscoverVisualization {...fetchState} />
path: src/plugins/discover/public/application/view_components/canvas/index.tsx
user: login: abbyhu2000
body: @d-buckner i can expect that the fetchState contain all the information for a aggregated PromQL returned data just like how PPL returns it, right? 
especially `rows: OpenSearchHit[]` of the `fetchState` here is the data i will use to feed into the visualization render logics. so for promQL it will prob look something like: 
[
{timestamp: date, label1: number, label2: number},
{timestamp: date, label1: number, label2: number},
{timestamp: date, label1: number, label2: number},
]

<img width="1722" alt="Screenshot 2025-03-06 at 2 11 32 PM" src="https://github.com/user-attachments/assets/da0e6a15-9a87-4d40-bda8-bcd36972f7d3" />

created_at: 2025-03-06T22:13:28Z
author_association: MEMBER
diff_hunk: @@ -171,6 +172,7 @@ export default function DiscoverCanvas({ setHeaderActionMenu, history, optionalR
             (isEnhancementsEnabled ? (
               <>
                 <MemoizedDiscoverChartContainer {...fetchState} />
+                <MemoizedDiscoverVisualization {...fetchState} />
path: src/plugins/discover/public/application/view_components/canvas/index.tsx
user: login: abbyhu2000
body: also i can expect to be able to find the Prometheus data source information(id, schemas of the fields) through the data frame services, correct?  i think i will need the schemas of the fields info for the visualizations as well

```
/**
 * A data frame is a two-dimensional labeled data structure with columns of potentially different types.
 */
export interface IDataFrame {
  type?: DATA_FRAME_TYPES.DEFAULT;
  name?: string;
  schema?: Array<Partial<IFieldType>>;
  meta?: Record<string, any>;
  fields: IFieldType[];
  size: number;
}
```
created_at: 2025-03-06T22:15:29Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,60 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
path: src/plugins/data/public/search/expressions/discoverSearch.ts
user: login: kavilla
body: nit: wrong license
created_at: 2025-03-07T11:17:55Z
author_association: MEMBER
diff_hunk: @@ -171,6 +172,7 @@ export default function DiscoverCanvas({ setHeaderActionMenu, history, optionalR
             (isEnhancementsEnabled ? (
               <>
                 <MemoizedDiscoverChartContainer {...fetchState} />
+                <MemoizedDiscoverVisualization {...fetchState} />
path: src/plugins/discover/public/application/view_components/canvas/index.tsx
user: login: kavilla
body: that schema isn't really used anymore. depends on the prom said if they will returning back the schema with the row/col where the schema is the information about the source regardless of the query or resource.

i would propose still relying on the index pattern fields. the temporary index pattern should fetch the fields in the dataset config. 

that index pattern could point to a metric it's abstracted out enough
created_at: 2025-03-07T11:24:29Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,22 @@
+/*
+ * Copyright OpenSearch Contributors
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+import { i18n } from '@osd/i18n';
+
+import { VisualizationType } from '../../../../view_components/utils/use_visualization_type';
+import { toExpression } from './to_expression';
+
+export interface LineOptionsDefaults {
+  type: 'line';
+}
+
+export const createLineConfig = (): VisualizationType<LineOptionsDefaults> => ({
path: src/plugins/discover/public/application/components/visualizations/vislib/line/line_vis_type.ts
user: login: kavilla
body: we have create_histogram_configs  chart/utils/ would it make more sense to add it to that folder?
created_at: 2025-03-07T11:31:38Z
author_association: MEMBER
diff_hunk: @@ -22,7 +22,7 @@ import { DiscoverChart } from '../../components/chart/chart';
 import { QUERY_ENHANCEMENT_ENABLED_SETTING } from '../../../../common';
 import { getNewDiscoverSetting, setNewDiscoverSetting } from '../../components/utils/local_storage';
 
-export const DiscoverChartContainer = ({ hits, bucketInterval, chartData }: SearchData) => {
path: src/plugins/discover/public/application/view_components/canvas/discover_chart_container.tsx
user: login: kavilla
body: is rows and chartData different?
created_at: 2025-03-07T11:32:08Z
author_association: MEMBER
diff_hunk: @@ -171,6 +172,7 @@ export default function DiscoverCanvas({ setHeaderActionMenu, history, optionalR
             (isEnhancementsEnabled ? (
               <>
                 <MemoizedDiscoverChartContainer {...fetchState} />
+                <MemoizedDiscoverVisualization {...fetchState} />
path: src/plugins/discover/public/application/view_components/canvas/index.tsx
user: login: abbyhu2000
body: make sense. ideally the discover visualizations logic should work for both PPL and promQL @d-buckner can we make sure that the metrics are created and mapped using the temporary index pattern flow like PPL?
created_at: 2025-03-07T20:10:07Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,22 @@
+/*
+ * Copyright OpenSearch Contributors
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+import { i18n } from '@osd/i18n';
+
+import { VisualizationType } from '../../../../view_components/utils/use_visualization_type';
+import { toExpression } from './to_expression';
+
+export interface LineOptionsDefaults {
+  type: 'line';
+}
+
+export const createLineConfig = (): VisualizationType<LineOptionsDefaults> => ({
path: src/plugins/discover/public/application/components/visualizations/vislib/line/line_vis_type.ts
user: login: abbyhu2000
body: sure good idea
created_at: 2025-03-07T20:10:44Z
author_association: MEMBER
diff_hunk: @@ -22,7 +22,7 @@ import { DiscoverChart } from '../../components/chart/chart';
 import { QUERY_ENHANCEMENT_ENABLED_SETTING } from '../../../../common';
 import { getNewDiscoverSetting, setNewDiscoverSetting } from '../../components/utils/local_storage';
 
-export const DiscoverChartContainer = ({ hits, bucketInterval, chartData }: SearchData) => {
path: src/plugins/discover/public/application/view_components/canvas/discover_chart_container.tsx
user: login: abbyhu2000
body: This is an example of chart data, which is used to render the discover histogram with x being the timestamp, y being the sum of how many hits we got within that time bucket. it got produced in the [use_Search](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/2a9cb948bf42e7424bdd68e83aca28e7a49e8b2a/src/plugins/discover/public/application/view_components/utils/use_search.ts#L276) hook

![Screenshot 2025-03-07 at 4 08 31 PM](https://github.com/user-attachments/assets/60b8a360-3b92-4905-a61d-6ce32434bc72)

created_at: 2025-03-08T00:11:39Z
author_association: MEMBER
diff_hunk: @@ -22,7 +22,7 @@ import { DiscoverChart } from '../../components/chart/chart';
 import { QUERY_ENHANCEMENT_ENABLED_SETTING } from '../../../../common';
 import { getNewDiscoverSetting, setNewDiscoverSetting } from '../../components/utils/local_storage';
 
-export const DiscoverChartContainer = ({ hits, bucketInterval, chartData }: SearchData) => {
path: src/plugins/discover/public/application/view_components/canvas/discover_chart_container.tsx
user: login: abbyhu2000
body: chart data will only be produced if discover histogram exists/time field exists --> which means chart data will not be produced if the returned search data does not include the time field

rows will always contain information of the search response data. Therefore, i think it makes sense to use the rows data here to render visualizations in discover.

Show below:

![Screenshot 2025-03-07 at 4 22 12 PM](https://github.com/user-attachments/assets/cd0659f0-c957-4938-9160-cb18e83de602)

created_at: 2025-03-08T00:15:09Z
author_association: MEMBER
diff_hunk: @@ -22,7 +22,7 @@ import { DiscoverChart } from '../../components/chart/chart';
 import { QUERY_ENHANCEMENT_ENABLED_SETTING } from '../../../../common';
 import { getNewDiscoverSetting, setNewDiscoverSetting } from '../../components/utils/local_storage';
 
-export const DiscoverChartContainer = ({ hits, bucketInterval, chartData }: SearchData) => {
path: src/plugins/discover/public/application/view_components/canvas/discover_chart_container.tsx
user: login: abbyhu2000
body: I also see histogram in discover is rendered by directing using [Chart](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/2a9cb948bf42e7424bdd68e83aca28e7a49e8b2a/src/plugins/discover/public/application/components/chart/histogram/histogram.tsx) instead of using expression renderer. But since for discover visualization we might wish to onboard more visualization type in the future, it might be more beneficial to take the expressions renderer approach.
created_at: 2025-03-08T00:19:23Z
author_association: MEMBER
diff_hunk: @@ -16,7 +16,7 @@ import {
 } from '../../../data/common';
 
 // Query 500 samples by default
-const TARGET_SAMPLES = 500;
+const TARGET_SAMPLES = 9;
path: src/plugins/query_enhancements/server/search/promql_search_strategy.ts
user: login: d-buckner
body: Are we safe to revert this line now?
created_at: 2025-03-18T21:06:51Z
author_association: MEMBER
diff_hunk: @@ -16,7 +16,7 @@ import {
 } from '../../../data/common';
 
 // Query 500 samples by default
-const TARGET_SAMPLES = 500;
+const TARGET_SAMPLES = 9;
path: src/plugins/query_enhancements/server/search/promql_search_strategy.ts
user: login: abbyhu2000
body: yes
created_at: 2025-03-18T22:12:02Z
author_association: MEMBER

PR comments:
user: login: github-actions[bot]
created_at: 2025-03-06T19:34:32Z
body: ### ❌ Empty Changelog Section

The Changelog section in your PR description is empty. Please add a valid changelog entry or entries. If you did add a changelog entry, check to make sure that it was not accidentally included inside the comment block in the Changelog section.

author_association: CONTRIBUTOR
user: login: codecov[bot]
created_at: 2025-03-06T19:49:14Z
body: ## [Codecov](https://app.codecov.io/gh/opensearch-project/OpenSearch-Dashboards/pull/9501?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) Report
Attention: Patch coverage is `9.83607%` with `55 lines` in your changes missing coverage. Please review.
> Please [upload](https://docs.codecov.com/docs/codecov-uploader) report for BASE (`feature/prometheus@026a5fc`). [Learn more](https://docs.codecov.io/docs/error-reference?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project#section-missing-base-commit) about missing BASE report.

| [Files with missing lines](https://app.codecov.io/gh/opensearch-project/OpenSearch-Dashboards/pull/9501?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) | Patch % | Lines |
|---|---|---|
| [...onents/visualizations/vislib/line/to\_expression.ts](https://app.codecov.io/gh/opensearch-project/OpenSearch-Dashboards/pull/9501?src=pr&el=tree&filepath=src%2Fplugins%2Fdiscover%2Fpublic%2Fapplication%2Fcomponents%2Fvisualizations%2Fvislib%2Fline%2Fto_expression.ts&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project#diff-c3JjL3BsdWdpbnMvZGlzY292ZXIvcHVibGljL2FwcGxpY2F0aW9uL2NvbXBvbmVudHMvdmlzdWFsaXphdGlvbnMvdmlzbGliL2xpbmUvdG9fZXhwcmVzc2lvbi50cw==) | 5.55% | [34 Missing :warning: ](https://app.codecov.io/gh/opensearch-project/OpenSearch-Dashboards/pull/9501?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) |
| [.../view\_components/canvas/discover\_visualization.tsx](https://app.codecov.io/gh/opensearch-project/OpenSearch-Dashboards/pull/9501?src=pr&el=tree&filepath=src%2Fplugins%2Fdiscover%2Fpublic%2Fapplication%2Fview_components%2Fcanvas%2Fdiscover_visualization.tsx&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project#diff-c3JjL3BsdWdpbnMvZGlzY292ZXIvcHVibGljL2FwcGxpY2F0aW9uL3ZpZXdfY29tcG9uZW50cy9jYW52YXMvZGlzY292ZXJfdmlzdWFsaXphdGlvbi50c3g=) | 4.76% | [20 Missing :warning: ](https://app.codecov.io/gh/opensearch-project/OpenSearch-Dashboards/pull/9501?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) |
| [...n/view\_components/utils/use\_visualization\_types.ts](https://app.codecov.io/gh/opensearch-project/OpenSearch-Dashboards/pull/9501?src=pr&el=tree&filepath=src%2Fplugins%2Fdiscover%2Fpublic%2Fapplication%2Fview_components%2Futils%2Fuse_visualization_types.ts&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project#diff-c3JjL3BsdWdpbnMvZGlzY292ZXIvcHVibGljL2FwcGxpY2F0aW9uL3ZpZXdfY29tcG9uZW50cy91dGlscy91c2VfdmlzdWFsaXphdGlvbl90eXBlcy50cw==) | 50.00% | [1 Missing :warning: ](https://app.codecov.io/gh/opensearch-project/OpenSearch-Dashboards/pull/9501?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) |

<details><summary>Additional details and impacted files</summary>


```diff
@@                  Coverage Diff                  @@
##             feature/prometheus    #9501   +/-   ##
=====================================================
  Coverage                      ?   61.66%           
=====================================================
  Files                         ?     3826           
  Lines                         ?    92032           
  Branches                      ?    14584           
=====================================================
  Hits                          ?    56751           
  Misses                        ?    31668           
  Partials                      ?     3613           
```

| [Flag](https://app.codecov.io/gh/opensearch-project/OpenSearch-Dashboards/pull/9501/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) | Coverage Δ | |
|---|---|---|
| [Linux_1](https://app.codecov.io/gh/opensearch-project/OpenSearch-Dashboards/pull/9501/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) | `28.95% <9.83%> (?)` | |
| [Linux_3](https://app.codecov.io/gh/opensearch-project/OpenSearch-Dashboards/pull/9501/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) | `39.25% <9.83%> (?)` | |
| [Linux_4](https://app.codecov.io/gh/opensearch-project/OpenSearch-Dashboards/pull/9501/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) | `28.82% <9.83%> (?)` | |
| [Windows_1](https://app.codecov.io/gh/opensearch-project/OpenSearch-Dashboards/pull/9501/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) | `28.96% <9.83%> (?)` | |
| [Windows_2](https://app.codecov.io/gh/opensearch-project/OpenSearch-Dashboards/pull/9501/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) | `56.34% <ø> (?)` | |
| [Windows_3](https://app.codecov.io/gh/opensearch-project/OpenSearch-Dashboards/pull/9501/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) | `39.25% <9.83%> (?)` | |
| [Windows_4](https://app.codecov.io/gh/opensearch-project/OpenSearch-Dashboards/pull/9501/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) | `28.82% <9.83%> (?)` | |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project#carryforward-flags-in-the-pull-request-comment) to find out more.

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/opensearch-project/OpenSearch-Dashboards/pull/9501?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project).

<details><summary>🚀 New features to boost your workflow: </summary>

- ❄ [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- 📦 [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
author_association: NONE
user: login: github-actions[bot]
created_at: 2025-03-17T22:47:06Z
body: ### ❌ Empty Changelog Section

The Changelog section in your PR description is empty. Please add a valid changelog entry or entries. If you did add a changelog entry, check to make sure that it was not accidentally included inside the comment block in the Changelog section.

author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-03-18T22:44:38Z
body: ### ❌ Empty Changelog Section

The Changelog section in your PR description is empty. Please add a valid changelog entry or entries. If you did add a changelog entry, check to make sure that it was not accidentally included inside the comment block in the Changelog section.

author_association: CONTRIBUTOR