Repository: aws/graph-explorer
Pull request #813: Update schema sync to use TanStack Query
Author: login: kmcginnes
Created: 2025-03-03T19:00:31Z | Merged: 2025-03-04T22:36:07Z

<!--
Please read the [Code of Conduct](https://github.com/aws/graph-explorer/blob/main/CODE_OF_CONDUCT.md) and the [Contributing Guidelines](https://github.com/aws/graph-explorer/blob/main/CONTRIBUTING.md) before opening a pull request.
-->

## Description

Rewrite the schema sync react layer to use TanStack Query and show proper error messages in the UI. This also opts in to the default retry and caching policies for all of our queries.

- Add `schemaSyncQuery` to queries
- Update `searchQuery` to explicitly update the schema from results
  - This ensures we only do the update when the query succeeds
- Only call update schema from search when there are new things to add
  - This reduces state changes in the react logic
- Always run the update schema prefixes logic when the schema changes
  - Does nothing for non-RDF schemas
- Remove the toast notifications since the UI has all the same information

## Validation

- Ran multiple tests on working connections and broken connections
- Tested with broken internet
- Updated unit tests

## Related Issues

- Resolves #814
- Resolves #336 
- Resolves #269

### Check List

<!--
  ATTENTION
  Please follow this check list to ensure that you've followed all items before opening this PR
  You can check the items by adding an `x` between the brackets, like this: `[x]`
-->

- [x] I confirm that my contribution is made under the terms of the Apache 2.0
      license.
- [x] I have run `pnpm checks` to ensure code compiles and meets standards.
- [x] I have run `pnpm test` to check if all tests are passing.
- [x] I have covered new added functionality with unit tests if necessary.
- [x] I have added an entry in the `Changelog.md`.


Linked issue #814: Show error responses in UI
When an error happens in Graph Explorer we try to show a friendly error message to the user and we log out the full error in the console log.

However, it can be a bit tedious to ask a user to open their console log, reproduce the error and screenshot or export HAR file.

Instead, we should show the errors in the UI. We already handle some common errors in a more friendly way. This should just be a fallback for when we don't have a canned error message ready.

This will allow screenshots of the error to include error details useful for diagnosing the issue.

Diff:
diff --git a/Changelog.md b/Changelog.md
index 76bf6b67b..abc5f8011 100644
--- a/Changelog.md
+++ b/Changelog.md
@@ -2,6 +2,8 @@
 
 ## Upcoming Release
 
+- **Improved** reliability of schema syncing, retrying on failure
+  ([#813](https://github.com/aws/graph-explorer/pull/813))
 - **Improved** handling of errors when fetching data from the database
   ([#812](https://github.com/aws/graph-explorer/pull/812))
 - **Changed** to allow editing and deleting default connections
diff --git a/packages/graph-explorer/src/components/NotificationProvider/index.ts b/packages/graph-explorer/src/components/NotificationProvider/index.ts
index 9805216f7..fa2e73c68 100644
--- a/packages/graph-explorer/src/components/NotificationProvider/index.ts
+++ b/packages/graph-explorer/src/components/NotificationProvider/index.ts
@@ -1,3 +1,4 @@
 export { default as NotificationProvider } from "./NotificationProvider";
 export { default as useNotification } from "./useNotification";
 export type { NotificationComponentProps } from "./NotificationProvider";
+export type { Notification } from "./reducer";
diff --git a/packages/graph-explorer/src/connector/queries.ts b/packages/graph-explorer/src/connector/queries.ts
index 0ef1c3b1e..51314489b 100644
--- a/packages/graph-explorer/src/connector/queries.ts
+++ b/packages/graph-explorer/src/connector/queries.ts
@@ -5,10 +5,37 @@ import {
   Explorer,
   KeywordSearchRequest,
   KeywordSearchResponse,
+  SchemaResponse,
   VertexDetailsRequest,
   VertexDetailsResponse,
 } from "./useGEFetchTypes";
 import { Edge, Vertex, VertexId } from "@/core";
+import { updateSchemaPrefixes } from "@/core/StateProvider/schema";
+
+/**
+ * Fetches the schema from the given explorer and updates the local cache with the new schema.
+ * @param updateLocalCache The function to replace the schema in the cache.
+ * @param explorer The explorer to use for fetching the schema.
+ */
+export function schemaSyncQuery(
+  updateLocalCache: (schema: SchemaResponse) => void,
+  explorer: Explorer
+) {
+  return queryOptions({
+    queryKey: ["schema", explorer],
+    queryFn: async () => {
+      let schema = await explorer.fetchSchema();
+
+      // Update the prefixes for sparql connections
+      schema = updateSchemaPrefixes(schema);
+
+      // Update the schema in the cache
+      updateLocalCache(schema);
+
+      return schema;
+    },
+  });
+}
 
 /**
  * Performs a search with the provided parameters.
@@ -19,6 +46,10 @@ import { Edge, Vertex, VertexId } from "@/core";
  */
 export function searchQuery(
   request: KeywordSearchRequest,
+  updateSchema: (entities: {
+    vertices: Vertex[];
+    edges: Edge[];
+  }) => Promise<void>,
   explorer: Explorer,
   queryClient: QueryClient
 ) {
@@ -31,6 +62,7 @@ export function searchQuery(
       const results = await explorer.keywordSearch(request, { signal });
 
       updateVertexDetailsCache(explorer, queryClient, results.vertices);
+      await updateSchema(results);
 
       return results;
     },
diff --git a/packages/graph-explorer/src/core/ConfigurationProvider/useConfiguration.ts b/packages/graph-explorer/src/core/ConfigurationProvider/useConfiguration.ts
index c8bfdbfb9..5460dd884 100644
--- a/packages/graph-explorer/src/core/ConfigurationProvider/useConfiguration.ts
+++ b/packages/graph-explorer/src/core/ConfigurationProvider/useConfiguration.ts
@@ -144,3 +144,12 @@ export function useEdgeTypeConfigs(edgeTypes?: string[]) {
 export default function useConfiguration() {
   return useRecoilValue(assembledConfigSelector);
 }
+
+/** Gets the fully merged and augmented configuration & schema, and throws if no active configuration. */
+export function useResolvedConfig() {
+  const config = useConfiguration();
+  if (!config) {
+    throw new Error("Must have an active configuration");
+  }
+  return config;
+}
diff --git a/packages/graph-explorer/src/core/StateProvider/index.ts b/packages/graph-explorer/src/core/StateProvider/index.ts
index 3b2929fe9..5313cb750 100644
--- a/packages/graph-explorer/src/core/StateProvider/index.ts
+++ b/packages/graph-explorer/src/core/StateProvider/index.ts
@@ -7,3 +7,4 @@ export * from "./edges";
 export * from "./neighbors";
 export * from "./nodes";
 export * from "./userPreferences";
+export * from "./schema";
diff --git a/packages/graph-explorer/src/core/StateProvider/schema.test.ts b/packages/graph-explorer/src/core/StateProvider/schema.test.ts
index 9be17fbb4..5212422fb 100644
--- a/packages/graph-explorer/src/core/StateProvider/schema.test.ts
+++ b/packages/graph-explorer/src/core/StateProvider/schema.test.ts
@@ -1,12 +1,19 @@
 import {
   createRandomEdge,
+  createRandomEntities,
   createRandomSchema,
   createRandomVertex,
 } from "@/utils/testing";
-import { extractConfigFromEntity, updateSchemaFromEntities } from "./schema";
-import { createArray } from "@shared/utils/testing";
+import {
+  extractConfigFromEntity,
+  shouldUpdateSchemaFromEntities,
+  updateSchemaFromEntities,
+  updateSchemaPrefixes,
+} from "./schema";
+import { createArray, createRandomName } from "@shared/utils/testing";
 import { toNodeMap } from "./nodes";
 import { toEdgeMap } from "./edges";
+import { PrefixTypeConfig } from "../ConfigurationProvider";
 
 describe("schema", () => {
   describe("extractConfigFromEntity", () => {
@@ -140,4 +147,127 @@ describe("schema", () => {
       expect(result.edges[0].type).toBe("");
     });
   });
+
+  describe("updateSchemaPrefixes", () => {
+    it("should do nothing when there are no URIs to process", () => {
+      const schema = createRandomSchema();
+      delete schema.prefixes;
+      schema.vertices = [];
+      schema.edges = [];
+
+      const result = updateSchemaPrefixes(schema);
+
+      expect(result.prefixes).toBeUndefined();
+    });
+
+    it("should generate prefixes for a single URI", () => {
+      const schema = createRandomSchema();
+      schema.vertices.forEach(v => {
+        v.type = "http://abcdefg.com/vertex#" + encodeURIComponent(v.type);
+      });
+      schema.edges.forEach(e => {
+        e.type = "http://abcdefg.com/edge#" + encodeURIComponent(e.type);
+      });
+      const result = updateSchemaPrefixes(schema);
+
+      expect(result.prefixes).toBeDefined();
+      expect(result.prefixes).toEqual([
+        {
+          prefix: "ver",
+          uri: "http://abcdefg.com/vertex#",
+          __inferred: true,
+          __matches: new Set(schema.vertices.map(v => v.type)),
+        },
+        {
+          prefix: "edg",
+          uri: "http://abcdefg.com/edge#",
+          __inferred: true,
+          __matches: new Set(schema.edges.map(e => e.type)),
+        },
+      ] satisfies PrefixTypeConfig[]);
+    });
+  });
+
+  describe("shouldUpdateSchemaFromEntities", () => {
+    it("should return false when no entities are provided", () => {
+      const result = shouldUpdateSchemaFromEntities(
+        { vertices: [], edges: [] },
+        createRandomSchema()
+      );
+      expect(result).toBeFalsy();
+    });
+
+    it("should return true when entities are provided", () => {
+      const entities = createRandomEntities();
+      const result = shouldUpdateSchemaFromEntities(
+        {
+          vertices: entities.nodes.values().toArray(),
+          edges: entities.edges.values().toArray(),
+        },
+        createRandomSchema()
+      );
+      expect(result).toBeTruthy();
+    });
+
+    it("should return false when the vertex has an existing type", () => {
+      const schema = createRandomSchema();
+      const vertex = createRandomVertex();
+      vertex.type = schema.vertices[0].type;
+      vertex.attributes = schema.vertices[0].attributes.reduce(
+        (acc, attr) => {
+          acc[attr.name] = createRandomName("value");
+          return acc;
+        },
+        {} as Record<string, string | number>
+      );
+      const result = shouldUpdateSchemaFromEntities(
+        {
+          vertices: [vertex],
+          edges: [],
+        },
+        schema
+      );
+      expect(result).toBeFalsy();
+    });
+
+    it("should return false when the edge is an existing type", () => {
+      const schema = createRandomSchema();
+      const source = createRandomVertex();
+      const target = createRandomVertex();
+      source.type = schema.vertices[0].type;
+      source.attributes = schema.vertices[0].attributes.reduce(
+        (acc, attr) => {
+          acc[attr.name] = createRandomName("value");
+          return acc;
+        },
+        {} as Record<string, string | number>
+      );
+      target.type = schema.vertices[1].type;
+      target.attributes = schema.vertices[1].attributes.reduce(
+        (acc, attr) => {
+          acc[attr.name] = createRandomName("value");
+          return acc;
+        },
+        {} as Record<string, string | number>
+      );
+      const edge = createRandomEdge(source, target);
+      edge.type = schema.edges[0].type;
+      edge.attributes = schema.edges[0].attributes.reduce(
+        (acc, attr) => {
+          acc[attr.name] = createRandomName("value");
+          return acc;
+        },
+        {} as Record<string, string | number>
+      );
+
+      const result = shouldUpdateSchemaFromEntities(
+        {
+          vertices: [source, target],
+          edges: [edge],
+        },
+        schema
+      );
+      expect(result).toBeFalsy();
+    });
+  });
 });
diff --git a/packages/graph-explorer/src/core/StateProvider/schema.ts b/packages/graph-explorer/src/core/StateProvider/schema.ts
index 444328d21..8d9082b5e 100644
--- a/packages/graph-explorer/src/core/StateProvider/schema.ts
+++ b/packages/graph-explorer/src/core/StateProvider/schema.ts
@@ -1,14 +1,16 @@
-import { atom, DefaultValue, selector } from "recoil";
+import { atom, DefaultValue, selector, useRecoilCallback } from "recoil";
 import {
+  AttributeConfig,
   EdgeTypeConfig,
   PrefixTypeConfig,
   VertexTypeConfig,
 } from "@/core/ConfigurationProvider";
 import localForageEffect from "./localForageEffect";
 import { activeConfigurationAtom } from "./configuration";
-import isDefaultValue from "./isDefaultValue";
-import { Edge, Entities, Vertex } from "@/core";
-import { sanitizeText } from "@/utils";
+import { Edge, Entities, toEdgeMap, toNodeMap, Vertex } from "@/core";
+import { logger, sanitizeText } from "@/utils";
+import generatePrefixes from "@/utils/generatePrefixes";
+import { startTransition } from "react";
 
 export type SchemaInference = {
   vertices: VertexTypeConfig[];
@@ -59,20 +61,6 @@ export const activeSchemaSelector = selector({
 
 type SchemaEntities = Pick<Entities, "nodes" | "edges">;
 
-/** Write only atom that updates the schema based on the given nodes and edges. */
-export const updateSchemaFromEntitiesAtom = selector<SchemaEntities>({
-  key: "update-schema-from-entities",
-  get: () => ({ nodes: new Map(), edges: new Map() }),
-  set({ set }, newValue) {
-    set(activeSchemaSelector, prev => {
-      if (!prev || isDefaultValue(newValue)) {
-        return prev;
-      }
-      return updateSchemaFromEntities(newValue, prev);
-    });
-  },
-});
-
 /** Updates the schema based on the given nodes and edges. */
 export function updateSchemaFromEntities(
   entities: SchemaEntities,
@@ -87,11 +75,17 @@ export function updateSchemaFromEntities(
     .map(extractConfigFromEntity)
     .toArray();
 
-  return {
+  let newSchema = {
     ...schema,
     vertices: merge(schema.vertices, newVertexConfigs),
     edges: merge(schema.edges, newEdgeConfigs),
   } satisfies SchemaInference;
+
+  // Update the generated prefixes in the schema
+  newSchema = updateSchemaPrefixes(newSchema);
+
+  logger.debug("Updated schema:", newSchema);
+  return newSchema;
 }
 
 /** Merges new node or edge configs in to a set of existing node or edge configs. */
@@ -109,8 +103,20 @@ function merge<T extends VertexTypeConfig | EdgeTypeConfig>(
     }
 
     // Find missing attributes
+    const existingAttributes = new Set(
+      config.attributes.map(attr => attr.name)
+    );
     const missingAttributes = newConfig.attributes.filter(
-      newAttr => !config.attributes.find(attr => attr.name === newAttr.name)
+      newAttr => !existingAttributes.has(newAttr.name)
+    );
+
+    if (missingAttributes.length === 0) {
+      return config;
+    }
+
+    logger.debug(
+      `Adding missing attributes to ${config.type}:`,
+      missingAttributes
     );
 
     return {
@@ -123,6 +129,12 @@ function merge<T extends VertexTypeConfig | EdgeTypeConfig>(
   const existingTypes = new Set(updated.map(vt => vt.type));
   const missing = newConfigs.filter(vt => !existingTypes.has(vt.type));
 
+  if (missing.length === 0) {
+    return updated;
+  }
+
+  logger.debug(`Adding missing types:`, missing);
+
   // Combine all together
   return [...updated, ...missing];
 }
@@ -141,3 +153,130 @@ export function extractConfigFromEntity<Entity extends Vertex | Edge>(
     })),
   };
 }
+
+/** Generate RDF prefixes for all the resource URIs in the schema. */
+export function updateSchemaPrefixes(schema: SchemaInference): SchemaInference {
+  const existingPrefixes = schema.prefixes ?? [];
+
+  // Get all the resource URIs from the vertex and edge type configs
+  const resourceUris = schema.vertices
+    .flatMap(v => [v.type, ...v.attributes.map(attr => attr.name)])
+    .concat(schema.edges.map(e => e.type));
+
+  if (resourceUris.length === 0) {
+    return schema;
+  }
+
+  const genPrefixes = generatePrefixes(resourceUris, existingPrefixes);
+  if (!genPrefixes?.length) {
+    return schema;
+  }
+
+  logger.debug("Updating schema with prefixes:", genPrefixes);
+
+  return {
+    ...schema,
+    prefixes: genPrefixes,
+  };
+}
+
+/** Updates the schema with any new vertex or edge types, any new attributes, and updates the generated prefixes for sparql connections. */
+export function useUpdateSchemaFromEntities() {
+  return useRecoilCallback(
+    ({ snapshot, set }) =>
+      async (entities: { vertices: Vertex[]; edges: Edge[] }) => {
+        const activeSchema = await snapshot.getPromise(activeSchemaSelector);
+        if (entities.vertices.length === 0 && entities.edges.length === 0) {
+          return;
+        }
+        if (!activeSchema) {
+          return;
+        }
+        if (!shouldUpdateSchemaFromEntities(entities, activeSchema)) {
+          logger.debug("Schema is already up to date with the given entities");
+          return;
+        }
+        startTransition(() => {
+          logger.debug("Updating schema from entities");
+          set(activeSchemaSelector, prev => {
+            if (!prev) {
+              return prev;
+            }
+            return updateSchemaFromEntities(
+              {
+                nodes: toNodeMap(entities.vertices),
+                edges: toEdgeMap(entities.edges),
+              },
+              prev
+            );
+          });
+        });
+      },
+    []
+  );
+}
+
+/** Attempts to efficiently detect if the schema should be updated. */
+export function shouldUpdateSchemaFromEntities(
+  entities: {
+    vertices: Vertex[];
+    edges: Edge[];
+  },
+  schema: SchemaInference
+) {
+  if (entities.vertices.length > 0) {
+    // Check if the vertex types and attributes are the same
+    const fromEntities = getUniqueTypesAndAttributes(entities.vertices);
+    const fromSchema = getUniqueTypesAndAttributes(schema.vertices);
+
+    if (!fromSchema.isSupersetOf(fromEntities)) {
+      logger.debug(
+        "Found new vertex types or attributes:",
+        fromEntities.difference(fromSchema)
+      );
+      return true;
+    }
+  }
+
+  if (entities.edges.length > 0) {
+    // Check if the edge types and attributes are the same
+    const fromEntities = getUniqueTypesAndAttributes(entities.edges);
+    const fromSchema = getUniqueTypesAndAttributes(schema.edges);
+
+    if (!fromSchema.isSupersetOf(fromEntities)) {
+      logger.debug(
+        "Found new edge types or attributes:",
+        fromEntities.difference(fromSchema)
+      );
+      return true;
+    }
+  }
+
+  return false;
+}
+
+/**
+ * Creates a set of unique types and attribute names as a set of strings in order to be used for comparisons.
+ *
+ * The entries in the set will be in the format of `vertexType.attributeName` or `edgeType.attributeName`.
+ */
+function getUniqueTypesAndAttributes(
+  entities: (Vertex | Edge | VertexTypeConfig | EdgeTypeConfig)[]
+) {
+  return new Set(
+    entities.flatMap(e => {
+      return [
+        e.type,
+        ...getAttributeNames(e.attributes).map(a => `${e.type}.${a}`),
+      ];
+    })
+  );
+}
+
+function getAttributeNames(
+  attributes: Record<string, string | number> | AttributeConfig[]
+) {
+  return Array.isArray(attributes)
+    ? attributes.map(a => a.name)
+    : Object.keys(attributes);
+}
diff --git a/packages/graph-explorer/src/hooks/usePrefixesUpdater.ts b/packages/graph-explorer/src/hooks/usePrefixesUpdater.ts
deleted file mode 100644
index dfe27f161..000000000
--- a/packages/graph-explorer/src/hooks/usePrefixesUpdater.ts
+++ /dev/null
@@ -1,60 +0,0 @@
-import { useCallback } from "react";
-import { useSetRecoilState } from "recoil";
-import { useNotification } from "@/components/NotificationProvider";
-import { useConfiguration } from "@/core";
-import { activeSchemaSelector } from "@/core/StateProvider/schema";
-import generatePrefixes from "@/utils/generatePrefixes";
-
-const usePrefixesUpdater = () => {
-  const config = useConfiguration();
-  const setActiveSchema = useSetRecoilState(activeSchemaSelector);
-  const { enqueueNotification } = useNotification();
-
-  const supportsPrefixes = config?.connection?.queryEngine === "sparql";
-  const existingPrefixes = config?.schema?.prefixes;
-
-  return useCallback(
-    (ids: Array<string>) => {
-      if (supportsPrefixes || ids.length === 0) {
-        return;
-      }
-
-      const genPrefixes = generatePrefixes(ids, existingPrefixes);
-      if (!genPrefixes?.length) {
-        return;
-      }
-
-      setActiveSchema(prev => {
-        if (!prev) {
-          return prev;
-        }
-
-        return {
-          ...prev,
-          prefixes: genPrefixes,
-        };
-      });
-
-      const oldPrefixesSize = existingPrefixes?.length ?? 0;
-      const newPrefixesSize = genPrefixes.length;
-      if (
-        genPrefixes.length &&
-        existingPrefixes?.length !== genPrefixes.length
-      ) {
-        const addedCount = newPrefixesSize - oldPrefixesSize;
-        enqueueNotification({
-          title: "Namespaces updated",
-          message:
-            addedCount === 1
-              ? "1 new namespace has been generated"
-              : `${addedCount} new namespaces have been generated`,
-          type: "success",
-          stackable: true,
-        });
-      }
-    },
-    [supportsPrefixes, existingPrefixes, setActiveSchema, enqueueNotification]
-  );
-};
-
-export default usePrefixesUpdater;
diff --git a/packages/graph-explorer/src/hooks/useSchemaSync.ts b/packages/graph-explorer/src/hooks/useSchemaSync.ts
index c26446859..9e8cb7c7f 100644
--- a/packages/graph-explorer/src/hooks/useSchemaSync.ts
+++ b/packages/graph-explorer/src/hooks/useSchemaSync.ts
@@ -1,113 +1,45 @@
-import { useCallback, useRef } from "react";
-import { useNotification } from "@/components/NotificationProvider";
-import { useConfiguration } from "@/core/ConfigurationProvider";
-import { loggerSelector, useExplorer } from "@/core/connector";
-import usePrefixesUpdater from "./usePrefixesUpdater";
+import { useEffect, useMemo } from "react";
+import { useResolvedConfig } from "@/core/ConfigurationProvider";
+import { useExplorer } from "@/core/connector";
 import useUpdateSchema from "./useUpdateSchema";
-import { createDisplayError } from "@/utils/createDisplayError";
-import { useRecoilValue } from "recoil";
+import { useIsFetching, useQuery } from "@tanstack/react-query";
+import { schemaSyncQuery } from "@/connector";
 
-const useSchemaSync = (onSyncChange?: (isSyncing: boolean) => void) => {
-  const config = useConfiguration();
-  const explorer = useExplorer();
-  const remoteLogger = useRecoilValue(loggerSelector);
+export function useIsSyncing() {
+  return useIsFetching({ queryKey: ["schema"] }) > 0;
+}
 
-  const updatePrefixes = usePrefixesUpdater();
-  const { enqueueNotification, clearNotification } = useNotification();
-  const notificationId = useRef<string | null>(null);
+export function useSchemaSync() {
+  const config = useResolvedConfig();
+  const explorer = useExplorer();
 
   const { replaceSchema, setSyncFailure } = useUpdateSchema();
-  return useCallback(async () => {
-    if (!config) {
-      return;
-    }
-
-    onSyncChange?.(true);
-    try {
-      notificationId.current = enqueueNotification({
-        title: config.displayLabel || config.id,
-        message: "Updating the Database schema",
-        type: "info",
-      });
-
-      const schema = await explorer.fetchSchema();
-
-      if (!schema.vertices.length) {
-        notificationId.current && clearNotification(notificationId.current);
-        enqueueNotification({
-          title: config.displayLabel || config.id,
-          message: "This connection has no data available",
-          type: "info",
-          stackable: true,
-        });
-        remoteLogger.info(
-          `[${
-            config.displayLabel || config.id
-          }] This connection has no data available: ${JSON.stringify(
-            config.connection
-          )}`
-        );
-      }
-
-      replaceSchema(schema);
-      onSyncChange?.(false);
-
-      notificationId.current && clearNotification(notificationId.current);
-      enqueueNotification({
-        title: config.displayLabel || config.id,
-        message: "Connection successfully synchronized",
-        type: "success",
-        stackable: true,
-      });
-      remoteLogger.info(
-        `[${
-          config.displayLabel || config.id
-        }] Connection successfully synchronized: ${JSON.stringify(
-          config.connection
-        )}`
-      );
 
-      const ids = schema.vertices.flatMap(v => [
-        v.type,
-        ...v.attributes.map(attr => attr.name),
-      ]);
-      ids.push(...schema.edges.map(e => e.type));
-      updatePrefixes(ids);
-    } catch (e) {
-      notificationId.current && clearNotification(notificationId.current);
-      const displayError = createDisplayError(e);
-      enqueueNotification({
-        ...displayError,
-        type: "error",
-        stackable: true,
-      });
-      if (e instanceof Error && e.name === "AbortError") {
-        remoteLogger.error(
-          `[${
-            config.displayLabel || config.id
-          }] Fetch aborted, reached max time out ${config.connection?.fetchTimeoutMs} MS`
-        );
-      } else {
-        remoteLogger.error(
-          `[${
-            config.displayLabel || config.id
-          }] Error while fetching schema: ${e instanceof Error ? e.message : "Unexpected error"}`
-        );
-      }
+  // Check if the schema has ever been properly synced before providing initial data
+  const initialData = useMemo(() => {
+    return config.schema &&
+      config.schema.lastUpdate &&
+      config.schema.triedToSync
+      ? config.schema
+      : undefined;
+  }, [config]);
+
+  const query = useQuery({
+    ...schemaSyncQuery(replaceSchema, explorer),
+    initialData: initialData,
+    enabled: !initialData || config.schema?.lastSyncFail === true,
+  });
+  const { data, isFetching, status, error, refetch } = query;
+
+  // If the schema sync fails, set the schema to a failed state
+  useEffect(() => {
+    if (status === "error") {
       setSyncFailure();
-      onSyncChange?.(false);
     }
-  }, [
-    config,
-    explorer,
-    onSyncChange,
-    enqueueNotification,
-    replaceSchema,
-    clearNotification,
-    remoteLogger,
-    updatePrefixes,
-    setSyncFailure,
-  ]);
-};
+  }, [setSyncFailure, status]);
 
-export default useSchemaSync;
+  return useMemo(
+    () => ({ refetch, data, error, isFetching }),
+    [data, isFetching, refetch, error]
+  );
+}
diff --git a/packages/graph-explorer/src/hooks/useUpdateSchema.ts b/packages/graph-explorer/src/hooks/useUpdateSchema.ts
index 6152aecdd..19808c8a9 100644
--- a/packages/graph-explorer/src/hooks/useUpdateSchema.ts
+++ b/packages/graph-explorer/src/hooks/useUpdateSchema.ts
@@ -1,30 +1,34 @@
 import { useSetRecoilState } from "recoil";
 import type { SchemaResponse } from "@/connector";
 import { activeSchemaSelector } from "@/core/StateProvider/schema";
-import { useCallback } from "react";
+import { startTransition, useCallback } from "react";
 
 export default function useUpdateSchema() {
   const setSchema = useSetRecoilState(activeSchemaSelector);
 
   // Keeps previous data, but sets the status to sync failure
   const setSyncFailure = useCallback(() => {
-    setSchema(prev => ({
-      ...prev,
-      vertices: prev?.vertices ?? [],
-      edges: prev?.edges ?? [],
-      triedToSync: true,
-      lastSyncFail: true,
-    }));
+    startTransition(() => {
+      setSchema(prev => ({
+        ...prev,
+        vertices: prev?.vertices ?? [],
+        edges: prev?.edges ?? [],
+        triedToSync: true,
+        lastSyncFail: true,
+      }));
+    });
   }, [setSchema]);
 
   // Updates the stored schema with the given schema
   const replaceSchema = useCallback(
     (schema: SchemaResponse) => {
-      setSchema({
-        ...schema,
-        triedToSync: true,
-        lastUpdate: new Date(),
-        lastSyncFail: false,
+      startTransition(() => {
+        setSchema({
+          ...schema,
+          triedToSync: true,
+          lastUpdate: new Date(),
+          lastSyncFail: false,
+        });
       });
     },
     [setSchema]
@@ -33,17 +37,21 @@ export default function useUpdateSchema() {
   // Update the vertex totals (used by Data Explorer)
   const updateVertexTotal = useCallback(
     (vertexType: string, newTotal: number) => {
-      setSchema(prev => {
-        if (!prev) {
-          return prev;
-        }
+      startTransition(() => {
+        setSchema(prev => {
+          if (!prev) {
+            return prev;
+          }
 
-        return {
-          ...prev,
-          vertices: prev.vertices.map(vertex =>
-            vertex.type === vertexType ? { ...vertex, total: newTotal } : vertex
-          ),
-        };
+          return {
+            ...prev,
+            vertices: prev.vertices.map(vertex =>
+              vertex.type === vertexType
+                ? { ...vertex, total: newTotal }
+                : vertex
+            ),
+          };
+        });
       });
     },
     [setSchema]
diff --git a/packages/graph-explorer/src/modules/ConnectionDetail/ConnectionDetail.tsx b/packages/graph-explorer/src/modules/ConnectionDetail/ConnectionDetail.tsx
index 7885f01e3..40df6f50c 100644
--- a/packages/graph-explorer/src/modules/ConnectionDetail/ConnectionDetail.tsx
+++ b/packages/graph-explorer/src/modules/ConnectionDetail/ConnectionDetail.tsx
@@ -15,6 +15,7 @@ import {
   Panel,
   PanelContent,
   PanelEmptyState,
+  PanelError,
   PanelHeader,
   PanelHeaderActionButton,
   PanelHeaderActions,
@@ -23,40 +24,39 @@ import {
   SyncIcon,
   TrayArrowIcon,
 } from "@/components";
-import { showDebugActionsAtom, useConfiguration, useWithTheme } from "@/core";
+import {
+  ConfigurationContextProps,
+  showDebugActionsAtom,
+  useWithTheme,
+} from "@/core";
 import {
   activeConfigurationAtom,
   configurationAtom,
 } from "@/core/StateProvider/configuration";
 import { activeSchemaSelector, schemaAtom } from "@/core/StateProvider/schema";
-import useSchemaSync from "@/hooks/useSchemaSync";
+import { useSchemaSync } from "@/hooks/useSchemaSync";
 import useTranslations from "@/hooks/useTranslations";
 import { cn, formatDate, logger } from "@/utils";
 import saveConfigurationToFile from "@/utils/saveConfigurationToFile";
 import CreateConnection from "@/modules/CreateConnection";
 import ConnectionData from "./ConnectionData";
 import defaultStyles from "./ConnectionDetail.styles";
+import { useQueryClient } from "@tanstack/react-query";
 
 export type ConnectionDetailProps = {
-  isSync: boolean;
-  onSyncChange(isSync: boolean): void;
+  config: ConfigurationContextProps;
 };
 
-const ConnectionDetail = ({ isSync, onSyncChange }: ConnectionDetailProps) => {
+function ConnectionDetail({ config }: ConnectionDetailProps) {
   const styleWithTheme = useWithTheme();
-  const config = useConfiguration();
   const t = useTranslations();
   const [edit, setEdit] = useState(false);
 
-  const updateSchema = useSchemaSync();
-  const onConfigSync = useCallback(async () => {
-    onSyncChange(true);
-    try {
-      await updateSchema();
-    } finally {
-      onSyncChange(false);
-    }
-  }, [onSyncChange, updateSchema]);
+  const {
+    refetch: syncSchema,
+    isFetching: isSync,
+    error: syncError,
+  } = useSchemaSync();
 
   const onConfigExport = useCallback(() => {
     if (!config?.id) {
@@ -90,12 +90,8 @@ const ConnectionDetail = ({ isSync, onSyncChange }: ConnectionDetailProps) => {
     [config?.id]
   );
 
-  if (!config) {
-    return null;
-  }
-
-  const lastSyncUpdate = config?.schema?.lastUpdate;
-  const lastSyncFail = config?.schema?.lastSyncFail === true;
+  const lastSyncUpdate = config.schema?.lastUpdate;
+  const lastSyncFail = config.schema?.lastSyncFail === true;
 
   return (
     <Panel className={cn(styleWithTheme(defaultStyles))}>
@@ -109,7 +105,7 @@ const ConnectionDetail = ({ isSync, onSyncChange }: ConnectionDetailProps) => {
             label="Synchronize Database"
             icon={<SyncIcon className={isSync ? "animate-spin" : ""} />}
             isDisabled={isSync}
-            onActionClick={onConfigSync}
+            onActionClick={syncSchema}
           />
           <PanelHeaderActionButton
             label="Export Connection"
@@ -164,14 +160,17 @@ const ConnectionDetail = ({ isSync, onSyncChange }: ConnectionDetailProps) => {
           </NotInProduction>
         </div>
         {!isSync && !!lastSyncUpdate && <ConnectionData />}
-        {!lastSyncUpdate && !isSync && (
+        {!lastSyncUpdate && !isSync && syncError && (
+          <PanelError error={syncError} onRetry={syncSchema} className="p-6" />
+        )}
+        {!lastSyncUpdate && !isSync && !syncError && (
           <PanelEmptyState
             variant="error"
             icon={<SyncIcon />}
             title="Synchronization Required"
             subtitle="It is necessary to synchronize the connection to be able to work with the database."
             actionLabel="Start synchronization"
-            onAction={onConfigSync}
+            onAction={syncSchema}
             actionVariant="text"
             className="p-6"
           />
@@ -199,15 +198,19 @@ const ConnectionDetail = ({ isSync, onSyncChange }: ConnectionDetailProps) => {
       </PanelContent>
     </Panel>
   );
-};
+}
 
 function DebugActions() {
   const setActiveSchema = useSetRecoilState(activeSchemaSelector);
   const resetActiveSchema = useResetRecoilState(activeSchemaSelector);
+  const queryClient = useQueryClient();
 
   const deleteSchema = () => {
     logger.log("Deleting schema");
     resetActiveSchema();
+    queryClient.invalidateQueries({
+      queryKey: ["schema"],
+    });
   };
   const resetSchemaLastUpdated = () => {
     logger.log("Resetting schema last updated");
diff --git a/packages/graph-explorer/src/modules/SearchSidebar/useKeywordSearchQuery.ts b/packages/graph-explorer/src/modules/SearchSidebar/useKeywordSearchQuery.ts
index aea2cb235..85f2f87d5 100644
--- a/packages/graph-explorer/src/modules/SearchSidebar/useKeywordSearchQuery.ts
+++ b/packages/graph-explorer/src/modules/SearchSidebar/useKeywordSearchQuery.ts
@@ -1,8 +1,8 @@
 import { useQuery, useQueryClient } from "@tanstack/react-query";
 import { useExplorer } from "@/core/connector";
-import usePrefixesUpdater from "@/hooks/usePrefixesUpdater";
-import { useCallback, useEffect } from "react";
+import { useCallback } from "react";
 import { KeywordSearchRequest, searchQuery } from "@/connector";
+import { useUpdateSchemaFromEntities } from "@/core";
 
 export type SearchQueryRequest = {
   debouncedSearchTerm: string;
@@ -19,7 +19,7 @@ export function useKeywordSearchQuery({
 }: SearchQueryRequest) {
   const explorer = useExplorer();
   const queryClient = useQueryClient();
-  const updatePrefixes = usePrefixesUpdater();
+  const updateSchema = useUpdateSchemaFromEntities();
 
   const request: KeywordSearchRequest = {
     searchTerm: debouncedSearchTerm,
@@ -29,18 +29,9 @@ export function useKeywordSearchQuery({
     searchByAttributes: debouncedSearchTerm ? searchByAttributes : undefined,
     exactMatch: debouncedSearchTerm ? exactMatch : undefined,
   };
-  const query = useQuery(searchQuery(request, explorer, queryClient));
-
-  // Sync sparql prefixes
-  useEffect(() => {
-    if (!query.data) {
-      return;
-    }
-    const vertexIds = query.data.vertices
-      .map(v => v.id)
-      .filter(id => typeof id === "string");
-    updatePrefixes(vertexIds);
-  }, [query.data, updatePrefixes]);
+  const query = useQuery(
+    searchQuery(request, updateSchema, explorer, queryClient)
+  );
 
   return query;
 }
diff --git a/packages/graph-explorer/src/utils/createDisplayError.test.ts b/packages/graph-explorer/src/utils/createDisplayError.test.ts
index b4f168f13..3c15e1721 100644
--- a/packages/graph-explorer/src/utils/createDisplayError.test.ts
+++ b/packages/graph-explorer/src/utils/createDisplayError.test.ts
@@ -44,7 +44,7 @@ describe("createDisplayError", () => {
   });
 
   it("Should handle connection refused as inner error", () => {
-    const error = new Error("Some error message string", {
+    const error = new NetworkError("Some error message string", 500, {
       cause: { code: "ECONNREFUSED" },
     });
     const result = createDisplayError(error);
@@ -55,7 +55,7 @@ describe("createDisplayError", () => {
   });
 
   it("Should handle connection reset as inner error", () => {
-    const error = new Error("Some error message string", {
+    const error = new NetworkError("Some error message string", 500, {
       cause: { code: "ECONNRESET" },
     });
     const result = createDisplayError(error);
@@ -130,6 +130,16 @@ describe("createDisplayError", () => {
       message: "Some error",
     });
   });
+
+  it("Should handle network error with no data", () => {
+    const result = createDisplayError(
+      new NetworkError("Network error", 500, undefined)
+    );
+    expect(result).toStrictEqual({
+      title: "Network Response 500",
+      message: "An error occurred. Please try again.",
+    });
+  });
 });
 
 /** Used to create errors for test code. */
diff --git a/packages/graph-explorer/src/utils/createDisplayError.ts b/packages/graph-explorer/src/utils/createDisplayError.ts
index fc4abd127..8303423e8 100644
--- a/packages/graph-explorer/src/utils/createDisplayError.ts
+++ b/packages/graph-explorer/src/utils/createDisplayError.ts
@@ -18,26 +18,29 @@ const defaultDisplayError: DisplayError = {
  * @returns A `DisplayError` that contains a title and message.
  */
 export function createDisplayError(error: any): DisplayError {
-  if (typeof error === "object") {
+  const data =
+    error instanceof NetworkError
+      ? error.data
+      : typeof error === "object"
+        ? error
+        : null;
+  if (data != null) {
     // Bad connection configuration
-    if (
-      error?.code === "ECONNREFUSED" ||
-      error?.cause?.code === "ECONNREFUSED"
-    ) {
+    if (data.code === "ECONNREFUSED" || data.cause?.code === "ECONNREFUSED") {
       return {
         title: "Connection refused",
         message: "Please check your connection and try again.",
       };
     }
-    if (error?.code === "ECONNRESET" || error?.cause?.code === "ECONNRESET") {
+    if (data.code === "ECONNRESET" || data.cause?.code === "ECONNRESET") {
       return {
         title: "Connection reset",
         message: "Please check your connection and try again.",
       };
     }
     if (
-      error?.code === "ERR_INVALID_URL" ||
-      error?.cause?.code === "ERR_INVALID_URL"
+      data.code === "ERR_INVALID_URL" ||
+      data.cause?.code === "ERR_INVALID_URL"
     ) {
       return {
         title: "Invalid URL",
@@ -46,8 +49,8 @@ export function createDisplayError(error: any): DisplayError {
       };
     }
     if (
-      error?.code === "TimeLimitExceededException" ||
-      error?.cause?.code === "TimeLimitExceededException"
+      data.code === "TimeLimitExceededException" ||
+      data.cause?.code === "TimeLimitExceededException"
     ) {
       // Server timeout
       return {
@@ -59,8 +62,8 @@ export function createDisplayError(error: any): DisplayError {
 
     // Malformed query
     if (
-      error?.code === "MalformedQueryException" ||
-      error?.cause?.code === "MalformedQueryException"
+      data.code === "MalformedQueryException" ||
+      data.cause?.code === "MalformedQueryException"
     ) {
       return {
         title: "Malformed Query",
@@ -68,46 +71,46 @@ export function createDisplayError(error: any): DisplayError {
           "The executed query was rejected by the database. It is possible the query structure is not supported by your database.",
       };
     }
+  }
 
-    if (error instanceof Error) {
-      // Fetch timeout
-      if (error.name === "AbortError") {
-        return {
-          title: "Request cancelled",
-          message: "The request exceeded the configured timeout length.",
-        };
-      }
-      if (error.name === "TimeoutError") {
-        return {
-          title: "Fetch Timeout Exceeded",
-          message: "The request exceeded the configured fetch timeout.",
-        };
-      }
-
-      // Internet issues
-      if (error.name === "TypeError" && error.message === "Failed to fetch") {
-        return {
-          title: "Connection Error",
-          message: "Please check your connection and try again.",
-        };
-      }
+  if (error instanceof Error) {
+    // Fetch timeout
+    if (error.name === "AbortError") {
+      return {
+        title: "Request cancelled",
+        message: "The request exceeded the configured timeout length.",
+      };
+    }
+    if (error.name === "TimeoutError") {
+      return {
+        title: "Fetch Timeout Exceeded",
+        message: "The request exceeded the configured fetch timeout.",
+      };
     }
 
-    if (error instanceof NetworkError) {
-      if (error.statusCode === 429) {
-        return {
-          title: "Too Many Requests",
-          message:
-            "The database is currently overloaded. Please try again later.",
-        };
-      }
+    // Internet issues
+    if (error.name === "TypeError" && error.message === "Failed to fetch") {
+      return {
+        title: "Connection Error",
+        message: "Please check your connection and try again.",
+      };
+    }
+  }
 
+  if (error instanceof NetworkError) {
+    if (error.statusCode === 429) {
       return {
-        title: `Network Response ${error.statusCode}`,
+        title: "Too Many Requests",
         message:
-          extractMessageFromData(error.data) ?? defaultDisplayError.message,
+          "The database is currently overloaded. Please try again later.",
       };
     }
+
+    return {
+      title: `Network Response ${error.statusCode}`,
+      message:
+        extractMessageFromData(error.data) ?? defaultDisplayError.message,
+    };
   }
   return defaultDisplayError;
 }
diff --git a/packages/graph-explorer/src/utils/testing/randomData.ts b/packages/graph-explorer/src/utils/testing/randomData.ts
index 38f890208..b8677cc63 100644
--- a/packages/graph-explorer/src/utils/testing/randomData.ts
+++ b/packages/graph-explorer/src/utils/testing/randomData.ts
@@ -12,6 +12,7 @@ import {
   Entities,
   FeatureFlags,
   LineStyle,
+  PrefixTypeConfig,
   RawConfiguration,
   Schema,
   UserStyling,
@@ -116,6 +117,13 @@ export function createRandomVertexTypeConfig(): VertexTypeConfig {
   };
 }
 
+export function createRandomPrefixTypeConfig(): PrefixTypeConfig {
+  return {
+    prefix: createRandomName("prefix"),
+    uri: createRandomUrlString(),
+  };
+}
+
 /**
  * Creates a random schema object.
  * @returns A random Schema object.
diff --git a/packages/graph-explorer/src/workspaces/Connections/Connections.tsx b/packages/graph-explorer/src/workspaces/Connections/Connections.tsx
index d808cfb12..a3ebf003c 100644
--- a/packages/graph-explorer/src/workspaces/Connections/Connections.tsx
+++ b/packages/graph-explorer/src/workspaces/Connections/Connections.tsx
@@ -1,15 +1,12 @@
-import { useEffect, useState } from "react";
+import { useState } from "react";
 import { Link } from "react-router";
 import { useRecoilValue } from "recoil";
 import Button from "@/components/Button";
 import { ExplorerIcon, GearIcon } from "@/components/icons";
 import Workspace from "@/components/Workspace/Workspace";
 import { useConfiguration } from "@/core";
-import {
-  activeConfigurationAtom,
-  configurationAtom,
-} from "@/core/StateProvider/configuration";
-import useSchemaSync from "@/hooks/useSchemaSync";
+import { configurationAtom } from "@/core/StateProvider/configuration";
+import { useIsSyncing } from "@/hooks/useSchemaSync";
 import AvailableConnections from "@/modules/AvailableConnections";
 import ConnectionDetail from "@/modules/ConnectionDetail";
 import { APP_NAME } from "@/utils/constants";
@@ -18,21 +15,9 @@ import GraphExplorerIcon from "@/components/icons/GraphExplorerIcon";
 
 export default function Connections() {
   const config = useConfiguration();
-  const activeConfig = useRecoilValue(activeConfigurationAtom);
   const configuration = useRecoilValue(configurationAtom);
   const [isModalOpen, setModal] = useState(configuration.size === 0);
-  const [isSyncing, setSyncing] = useState(false);
-
-  // Every time that the active connection changes,
-  // if it was not synchronized yet, try to sync it
-  const updateSchema = useSchemaSync(setSyncing);
-  useEffect(() => {
-    if (config?.schema?.triedToSync === true) {
-      return;
-    }
-
-    updateSchema();
-  }, [activeConfig, config?.schema?.triedToSync, updateSchema]);
+  const isSyncing = useIsSyncing();
 
   return (
     <Workspace>
@@ -52,13 +37,11 @@ export default function Connections() {
 
             <Link
               to={
-                !activeConfig || !config?.schema?.lastUpdate
-                  ? "/connections"
-                  : "/graph-explorer"
+                !config?.schema?.lastUpdate ? "/connections" : "/graph-explorer"
               }
             >
               <Button
-                isDisabled={!activeConfig || !config?.schema?.lastUpdate}
+                isDisabled={!config?.schema?.lastUpdate}
                 className="button"
                 icon={<ExplorerIcon />}
                 variant="filled"
@@ -78,9 +61,9 @@ export default function Connections() {
               onModalChange={setModal}
             />
           </div>
-          {activeConfig ? (
+          {config ? (
             <div className="h-full grow">
-              <ConnectionDetail isSync={isSyncing} onSyncChange={setSyncing} />
+              <ConnectionDetail config={config} />
             </div>
           ) : (
             <NoActiveConnectionPanel />
diff --git a/packages/graph-explorer/src/workspaces/DataExplorer/DataExplorer.tsx b/packages/graph-explorer/src/workspaces/DataExplorer/DataExplorer.tsx
index 0ce9acda5..0238f164b 100644
--- a/packages/graph-explorer/src/workspaces/DataExplorer/DataExplorer.tsx
+++ b/packages/graph-explorer/src/workspaces/DataExplorer/DataExplorer.tsx
@@ -1,6 +1,6 @@
 import { cn } from "@/utils";
 import clone from "lodash/clone";
-import { useCallback, useEffect, useMemo, useRef } from "react";
+import { useCallback, useMemo, useRef } from "react";
 import {
   keepPreviousData,
   useQuery,
@@ -8,7 +8,7 @@ import {
 } from "@tanstack/react-query";
 import { Link, useNavigate, useParams, useSearchParams } from "react-router";
 import { useSetRecoilState } from "recoil";
-import { Vertex } from "@/core";
+import { useUpdateSchemaFromEntities, Vertex } from "@/core";
 import {
   CheckIcon,
   ChevronLeftIcon,
@@ -46,7 +46,6 @@ import {
   VertexPreferences,
 } from "@/core/StateProvider/userPreferences";
 import { useAddVertexToGraph, useHasVertexBeenAddedToGraph } from "@/hooks";
-import usePrefixesUpdater from "@/hooks/usePrefixesUpdater";
 import useTranslations from "@/hooks/useTranslations";
 import useUpdateVertexTypeCounts from "@/hooks/useUpdateVertexTypeCounts";
 import defaultStyles from "./DataExplorer.styles";
@@ -371,8 +370,7 @@ function useDataExplorerQuery(
 ) {
   const explorer = useExplorer();
   const queryClient = useQueryClient();
-
-  const updatePrefixes = usePrefixesUpdater();
+  const updateSchema = useUpdateSchemaFromEntities();
 
   const searchRequest: KeywordSearchRequest = {
     vertexTypes: [vertexType],
@@ -380,20 +378,9 @@ function useDataExplorerQuery(
     offset: pageIndex * pageSize,
   };
   const query = useQuery({
-    ...searchQuery(searchRequest, explorer, queryClient),
+    ...searchQuery(searchRequest, updateSchema, explorer, queryClient),
     placeholderData: keepPreviousData,
   });
 
-  useEffect(() => {
-    if (!query.data) {
-      return;
-    }
-
-    const vertexIds = query.data.vertices
-      .map(v => v.id)
-      .filter(id => typeof id === "string");
-    updatePrefixes(vertexIds);
-  }, [query.data, updatePrefixes]);
-
   return query;
 }


Reviews:
user: login: andreachild
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-04T17:54:02Z
user: login: andreachild
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-04T18:07:58Z
user: login: andreachild
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-04T18:08:47Z
user: login: kmcginnes
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-04T18:09:21Z
user: login: kmcginnes
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-04T18:09:31Z
user: login: andreachild
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-04T18:29:13Z
user: login: andreachild
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-04T19:15:11Z
user: login: andreachild
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-04T19:27:02Z
user: login: kmcginnes
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-04T21:25:40Z

Review comments:
diff_hunk: @@ -5,10 +5,38 @@ import {
   Explorer,
   KeywordSearchRequest,
   KeywordSearchResponse,
+  SchemaResponse,
   VertexDetailsRequest,
   VertexDetailsResponse,
 } from "./useGEFetchTypes";
 import { Edge, Vertex, VertexId } from "@/core";
+import { updateSchemaPrefixes } from "@/core/StateProvider/schema";
+
+/**
+ * Fetches the schema from the given explorer and updates the local cache with the new schema.
+ * @param updateLocalCache The function to replace the schema in the cache.
+ * @param updatePrefixes The function to update the prefixes in the cache.
path: packages/graph-explorer/src/connector/queries.ts
user: login: andreachild
body: This param is not in the function signature
created_at: 2025-03-04T17:54:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -109,8 +102,20 @@ function merge<T extends VertexTypeConfig | EdgeTypeConfig>(
     }
 
     // Find missing attributes
+    const existingAttributes = new Set(
+      config.attributes.map(attr => attr.name)
+    );
     const missingAttributes = newConfig.attributes.filter(
-      newAttr => !config.attributes.find(attr => attr.name === newAttr.name)
+      newAttr => !existingAttributes.has(newAttr.name)
path: packages/graph-explorer/src/core/StateProvider/schema.ts
user: login: andreachild
body: Can a new config remove attributes from existing config?
created_at: 2025-03-04T18:07:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -123,6 +128,12 @@ function merge<T extends VertexTypeConfig | EdgeTypeConfig>(
   const existingTypes = new Set(updated.map(vt => vt.type));
   const missing = newConfigs.filter(vt => !existingTypes.has(vt.type));
 
+  if (missing.length === 0) {
+    return updated;
+  }
+
+  logger.debug(`Adding missing types:`, missing);
path: packages/graph-explorer/src/core/StateProvider/schema.ts
user: login: andreachild
body: Can types be removed? Or only added?
created_at: 2025-03-04T18:08:47Z
author_association: CONTRIBUTOR
diff_hunk: @@ -109,8 +102,20 @@ function merge<T extends VertexTypeConfig | EdgeTypeConfig>(
     }
 
     // Find missing attributes
+    const existingAttributes = new Set(
+      config.attributes.map(attr => attr.name)
+    );
     const missingAttributes = newConfig.attributes.filter(
-      newAttr => !config.attributes.find(attr => attr.name === newAttr.name)
+      newAttr => !existingAttributes.has(newAttr.name)
path: packages/graph-explorer/src/core/StateProvider/schema.ts
user: login: kmcginnes
body: No. It is only additive.
created_at: 2025-03-04T18:09:21Z
author_association: COLLABORATOR
diff_hunk: @@ -123,6 +128,12 @@ function merge<T extends VertexTypeConfig | EdgeTypeConfig>(
   const existingTypes = new Set(updated.map(vt => vt.type));
   const missing = newConfigs.filter(vt => !existingTypes.has(vt.type));
 
+  if (missing.length === 0) {
+    return updated;
+  }
+
+  logger.debug(`Adding missing types:`, missing);
path: packages/graph-explorer/src/core/StateProvider/schema.ts
user: login: kmcginnes
body: Only added.
created_at: 2025-03-04T18:09:31Z
author_association: COLLABORATOR
diff_hunk: @@ -141,3 +152,142 @@ export function extractConfigFromEntity<Entity extends Vertex | Edge>(
     })),
   };
 }
+
+/** Generate RDF prefixes for all the resource URIs in the schema. */
+export function updateSchemaPrefixes(schema: SchemaInference): SchemaInference {
+  const existingPrefixes = schema.prefixes ?? [];
+
+  // Get all the resource URIs from the vertex and edge type configs
+  const resourceUris = schema.vertices
+    .flatMap(v => [v.type, ...v.attributes.map(attr => attr.name)])
+    .concat(schema.edges.map(e => e.type));
+
+  if (resourceUris.length === 0) {
+    return schema;
path: packages/graph-explorer/src/core/StateProvider/schema.ts
user: login: andreachild
body: If there are no `resourceUris` but there are `existingPrefixes` should the `existingPrefixes` be removed?
created_at: 2025-03-04T18:29:13Z
author_association: CONTRIBUTOR
diff_hunk: @@ -141,3 +152,142 @@ export function extractConfigFromEntity<Entity extends Vertex | Edge>(
     })),
   };
 }
+
+/** Generate RDF prefixes for all the resource URIs in the schema. */
+export function updateSchemaPrefixes(schema: SchemaInference): SchemaInference {
+  const existingPrefixes = schema.prefixes ?? [];
+
+  // Get all the resource URIs from the vertex and edge type configs
+  const resourceUris = schema.vertices
+    .flatMap(v => [v.type, ...v.attributes.map(attr => attr.name)])
+    .concat(schema.edges.map(e => e.type));
+
+  if (resourceUris.length === 0) {
+    return schema;
path: packages/graph-explorer/src/core/StateProvider/schema.ts
user: login: andreachild
body: After chatting with you I understand now schema changes are only additive as it only affects the search and is not affected by removal of nodes or edges from the graph itself.
created_at: 2025-03-04T19:15:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -141,3 +152,142 @@ export function extractConfigFromEntity<Entity extends Vertex | Edge>(
     })),
   };
 }
+
+/** Generate RDF prefixes for all the resource URIs in the schema. */
+export function updateSchemaPrefixes(schema: SchemaInference): SchemaInference {
+  const existingPrefixes = schema.prefixes ?? [];
+
+  // Get all the resource URIs from the vertex and edge type configs
+  const resourceUris = schema.vertices
+    .flatMap(v => [v.type, ...v.attributes.map(attr => attr.name)])
+    .concat(schema.edges.map(e => e.type));
+
+  if (resourceUris.length === 0) {
+    return schema;
+  }
+
+  const genPrefixes = generatePrefixes(resourceUris, existingPrefixes);
+  if (!genPrefixes?.length) {
+    return schema;
+  }
+
+  logger.debug("Updating schema with prefixes:", genPrefixes);
+
+  return {
+    ...schema,
+    prefixes: genPrefixes,
+  };
+}
+
+/** Updates the schema with any new vertex or edge types, any new attributes, and updates the generated prefixes for sparql connections. */
+export function useUpdateSchemaFromEntities() {
+  return useRecoilCallback(
+    ({ snapshot, set }) =>
+      async (entities: { vertices: Vertex[]; edges: Edge[] }) => {
+        const activeSchema = await snapshot.getPromise(activeSchemaSelector);
+        if (entities.vertices.length === 0 && entities.edges.length === 0) {
+          return;
+        }
+        if (!activeSchema) {
+          return;
+        }
+        if (!shouldUpdateSchemaFromEntities(entities, activeSchema)) {
+          logger.debug("Schema is already up to date with the given entities");
+          return;
+        }
+        startTransition(() => {
+          logger.debug("Updating schema from entities");
+          set(activeSchemaSelector, prev => {
+            if (!prev) {
+              return prev;
+            }
+            return updateSchemaFromEntities(
+              {
+                nodes: toNodeMap(entities.vertices),
+                edges: toEdgeMap(entities.edges),
+              },
+              prev
+            );
+          });
+        });
+      },
+    []
+  );
+}
+
+/** Attempts to efficiently detect if the schema should be updated. */
+export function shouldUpdateSchemaFromEntities(
+  entities: {
+    vertices: Vertex[];
+    edges: Edge[];
+  },
+  schema: SchemaInference
+) {
+  if (entities.vertices.length > 0) {
+    // Check if the vertex types are the same
+    const newVertexTypes = new Set(entities.vertices.map(n => n.type));
+    const existingVertexTypes = new Set(schema.vertices.map(v => v.type));
+
+    if (!existingVertexTypes.isSupersetOf(newVertexTypes)) {
+      logger.debug(
+        "Found new vertex types:",
+        newVertexTypes.difference(existingVertexTypes)
+      );
+      return true;
+    }
+
+    // Check if the vertex attributes are the same
+    const newVertexAttributes = new Set(
+      entities.vertices.flatMap(n =>
+        Object.keys(n.attributes).map(a => `${n.type}.${a}`)
path: packages/graph-explorer/src/core/StateProvider/schema.ts
user: login: andreachild
body: Instead of first checking type and then attributes can they be checked together since the attribute check contains the type? I think you can do this by adding additional set value for the type itself.

Example (T=type, A=attribute, added types/attributes in bold):

Set of existing vertex types and attributes:   T1, T1.A1, T1.A2, T2, T3, T3.A1
Set of new vertex types and attributes:         T1, T1.A1, T1.A2, **T1.A3**, T2, **T2.A1**, T3, T3.A1, **T3.A2**, **T4**
created_at: 2025-03-04T19:27:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -141,3 +152,142 @@ export function extractConfigFromEntity<Entity extends Vertex | Edge>(
     })),
   };
 }
+
+/** Generate RDF prefixes for all the resource URIs in the schema. */
+export function updateSchemaPrefixes(schema: SchemaInference): SchemaInference {
+  const existingPrefixes = schema.prefixes ?? [];
+
+  // Get all the resource URIs from the vertex and edge type configs
+  const resourceUris = schema.vertices
+    .flatMap(v => [v.type, ...v.attributes.map(attr => attr.name)])
+    .concat(schema.edges.map(e => e.type));
+
+  if (resourceUris.length === 0) {
+    return schema;
+  }
+
+  const genPrefixes = generatePrefixes(resourceUris, existingPrefixes);
+  if (!genPrefixes?.length) {
+    return schema;
+  }
+
+  logger.debug("Updating schema with prefixes:", genPrefixes);
+
+  return {
+    ...schema,
+    prefixes: genPrefixes,
+  };
+}
+
+/** Updates the schema with any new vertex or edge types, any new attributes, and updates the generated prefixes for sparql connections. */
+export function useUpdateSchemaFromEntities() {
+  return useRecoilCallback(
+    ({ snapshot, set }) =>
+      async (entities: { vertices: Vertex[]; edges: Edge[] }) => {
+        const activeSchema = await snapshot.getPromise(activeSchemaSelector);
+        if (entities.vertices.length === 0 && entities.edges.length === 0) {
+          return;
+        }
+        if (!activeSchema) {
+          return;
+        }
+        if (!shouldUpdateSchemaFromEntities(entities, activeSchema)) {
+          logger.debug("Schema is already up to date with the given entities");
+          return;
+        }
+        startTransition(() => {
+          logger.debug("Updating schema from entities");
+          set(activeSchemaSelector, prev => {
+            if (!prev) {
+              return prev;
+            }
+            return updateSchemaFromEntities(
+              {
+                nodes: toNodeMap(entities.vertices),
+                edges: toEdgeMap(entities.edges),
+              },
+              prev
+            );
+          });
+        });
+      },
+    []
+  );
+}
+
+/** Attempts to efficiently detect if the schema should be updated. */
+export function shouldUpdateSchemaFromEntities(
+  entities: {
+    vertices: Vertex[];
+    edges: Edge[];
+  },
+  schema: SchemaInference
+) {
+  if (entities.vertices.length > 0) {
+    // Check if the vertex types are the same
+    const newVertexTypes = new Set(entities.vertices.map(n => n.type));
+    const existingVertexTypes = new Set(schema.vertices.map(v => v.type));
+
+    if (!existingVertexTypes.isSupersetOf(newVertexTypes)) {
+      logger.debug(
+        "Found new vertex types:",
+        newVertexTypes.difference(existingVertexTypes)
+      );
+      return true;
+    }
+
+    // Check if the vertex attributes are the same
+    const newVertexAttributes = new Set(
+      entities.vertices.flatMap(n =>
+        Object.keys(n.attributes).map(a => `${n.type}.${a}`)
path: packages/graph-explorer/src/core/StateProvider/schema.ts
user: login: kmcginnes
body: I made a change to this effect. I'm not sure if it improves efficiency at all since it must still do the same number of iterations (maybe more), but I think it did simplify the logic a bit. Let me know what you think.
created_at: 2025-03-04T21:25:40Z
author_association: COLLABORATOR

PR comments:
user: login: kmcginnes
created_at: 2025-03-04T21:26:33Z
body: I also fixed a big in the `createDisplayError` logic where I used a `!== null` but I should've used `!= null`.
author_association: COLLABORATOR
user: login: andreachild
created_at: 2025-03-04T21:33:47Z
body: LGTM
author_association: CONTRIBUTOR