Repository: dust-tt/dust
Pull request #12691: [front] mcp(remote): auto sync remote servers and persist errors
Author: login: adrsimon
Labels: name: migration-ack
description: 📁 Label to acknowledge that a migration is required.
Created: 2025-05-15T13:43:56Z | Merged: 2025-05-22T09:52:58Z

## Description

<!-- Briefly describe the changes you've made and link any relevant issues (e.g., "Fixes #123"). -->
<!-- If the PR includes UI changes, please attach a screenshot or GIF to illustrate the modifications. -->

This PR adds:
- a worker that runs on sunday at noon, to re-sync all remote servers.
- tracking the error in the remote servers, and displaying them in the server details.

## Tests

<!-- Explain how you tested your changes, did you do it manually, did you add / update some existing tests ? See [here](https://www.notion.so/dust-tt/Guide-Testing-18428599d94180e09250ff256d6ac46e) -->

Tested the error sync by hand, making the sync crash.
Haven't tested the workflow yet

## Risk

<!-- Discuss potential risks and how they will be mitigated. Consider the impact and whether the changes are safe to rollback. -->

Low risks, safe to rollback

## Deploy Plan

<!-- Outline the deployment steps. Specify the order of operations and any considerations that should be made before, during, and after deployment/ -->

- block front
- run db migrations
- unblock front
- deploy front



Diff:
diff --git a/front/components/actions/mcp/RemoteMCPForm.tsx b/front/components/actions/mcp/RemoteMCPForm.tsx
index 91b63711b71d..5805a5e54ca5 100644
--- a/front/components/actions/mcp/RemoteMCPForm.tsx
+++ b/front/components/actions/mcp/RemoteMCPForm.tsx
@@ -28,7 +28,7 @@ import {
   useUpdateRemoteMCPServer,
 } from "@app/lib/swr/mcp_servers";
 import type { LightWorkspaceType } from "@app/types";
-import { asDisplayName } from "@app/types";
+import { asDisplayName, normalizeError } from "@app/types";
 
 interface RemoteMCPFormProps {
   owner: LightWorkspaceType;
@@ -48,7 +48,6 @@ export function RemoteMCPForm({ owner, mcpServer }: RemoteMCPFormProps) {
   const sendNotification = useSendNotification();
 
   const [isSynchronizing, setIsSynchronizing] = useState(false);
-  const [syncError, setSyncError] = useState<string | null>(null);
   const [isPopoverOpen, setIsPopoverOpen] = useState(false);
 
   const form = useForm<MCPFormType>({
@@ -61,7 +60,7 @@ export function RemoteMCPForm({ owner, mcpServer }: RemoteMCPFormProps) {
     },
   });
 
-  const { url } = mcpServer;
+  const { url, lastError, lastSyncAt } = mcpServer;
 
   const { mutateMCPServers } = useMCPServers({
     owner,
@@ -106,12 +105,6 @@ export function RemoteMCPForm({ owner, mcpServer }: RemoteMCPFormProps) {
   );
 
   const handleSynchronize = useCallback(async () => {
-    if (!url) {
-      setSyncError("Please enter a valid URL before synchronizing.");
-      return;
-    }
-
-    setSyncError(null);
     setIsSynchronizing(true);
 
     try {
@@ -133,18 +126,12 @@ export function RemoteMCPForm({ owner, mcpServer }: RemoteMCPFormProps) {
       sendNotification({
         title: "Error synchronizing MCP server",
         type: "error",
-        description:
-          error instanceof Error ? error.message : "An error occurred",
+        description: normalizeError(error ?? "An error occured").message,
       });
-      setSyncError(
-        error instanceof Error
-          ? error.message
-          : "Failed to synchronize with MCP server"
-      );
     } finally {
       setIsSynchronizing(false);
     }
-  }, [url, syncServer, mutateMCPServers, sendNotification]);
+  }, [syncServer, mutateMCPServers, sendNotification]);
 
   const closePopover = () => {
     setIsPopoverOpen(false);
@@ -152,30 +139,17 @@ export function RemoteMCPForm({ owner, mcpServer }: RemoteMCPFormProps) {
 
   return (
     <div className="space-y-5 text-foreground dark:text-foreground-night">
-      {syncError && (
+      {lastError && (
         <ContentMessage
           variant="warning"
           icon={ExclamationCircleIcon}
           size="sm"
           title="Synchronization Error"
         >
-          {syncError}
+          Server could not synchronize successfully. Last attempt{" "}
+          {lastSyncAt ? "on " + new Date(lastSyncAt).toLocaleString() : ""} :{" "}
+          {lastError}
         </ContentMessage>
-        // <div className="rounded-md bg-warning-50 p-4">
-        //   <div className="flex">
-        //     <div className="flex-shrink-0">
-        //       <Icon size="sm" icon={XMarkIcon} className="text-warning" />
-        //     </div>
-        //     <div className="ml-3">
-        //       <h3 className="text-sm font-medium text-warning-800">
-        //         Synchronization Error
-        //       </h3>
-        //       <div className="mt-2 text-sm text-warning-700">
-        //         <p>{syncError}</p>
-        //       </div>
-        //     </div>
-        //   </div>
-        // </div>
       )}
 
       <div className="heading-lg">Server Settings</div>
diff --git a/front/lib/api/mcp.ts b/front/lib/api/mcp.ts
index 53b8f9b4294b..f5d9cd42c259 100644
--- a/front/lib/api/mcp.ts
+++ b/front/lib/api/mcp.ts
@@ -55,6 +55,7 @@ export type RemoteMCPServerType = MCPServerType & {
   cachedDescription?: string | null;
   sharedSecret?: string | null;
   lastSyncAt?: Date | null;
+  lastError?: string | null;
   icon: RemoteAllowedIconType; // We enforce that we pass an icon here (among the ones we allow).
 };
 
diff --git a/front/lib/models/assistant/actions/remote_mcp_server.ts b/front/lib/models/assistant/actions/remote_mcp_server.ts
index e1ea90884639..292996dbf480 100644
--- a/front/lib/models/assistant/actions/remote_mcp_server.ts
+++ b/front/lib/models/assistant/actions/remote_mcp_server.ts
@@ -23,6 +23,8 @@ export class RemoteMCPServerModel extends WorkspaceAwareModel<RemoteMCPServerMod
   declare cachedTools: MCPToolType[];
 
   declare lastSyncAt: Date | null;
+  declare lastError: string | null;
+
   declare sharedSecret: string | null;
   declare authorization: AuthorizationInfo | null;
 }
@@ -77,6 +79,11 @@ RemoteMCPServerModel.init(
       type: DataTypes.DATE,
       allowNull: true,
     },
+    lastError: {
+      type: DataTypes.TEXT,
+      allowNull: true,
+      defaultValue: null,
+    },
     sharedSecret: {
       type: DataTypes.STRING,
       allowNull: true,
diff --git a/front/lib/resources/remote_mcp_servers_resource.ts b/front/lib/resources/remote_mcp_servers_resource.ts
index d384f02d4941..0296e2e7711d 100644
--- a/front/lib/resources/remote_mcp_servers_resource.ts
+++ b/front/lib/resources/remote_mcp_servers_resource.ts
@@ -5,6 +5,7 @@ import type {
   ModelStatic,
   Transaction,
 } from "sequelize";
+import { Op } from "sequelize";
 
 import {
   DEFAULT_MCP_ACTION_DESCRIPTION,
@@ -156,6 +157,27 @@ export class RemoteMCPServerResource extends BaseResource<RemoteMCPServerModel>
     return servers.length > 0 ? servers[0] : null;
   }
 
+  // Admin operations - don't use in non-temporal code.
+  static async dangerouslyListAllServersIds({
+    firstId,
+    limit = 100,
+  }: {
+    firstId?: number;
+    limit?: number;
+  }) {
+    const servers = await RemoteMCPServerModel.findAll({
+      where: {
+        id: {
+          [Op.gte]: firstId,
+        },
+      },
+      limit,
+      order: [["id", "ASC"]],
+    });
+
+    return servers.map((server) => server.id);
+  }
+
   // sId
   get sId(): string {
     return remoteMCPServerNameToSId({
@@ -235,6 +257,7 @@ export class RemoteMCPServerResource extends BaseResource<RemoteMCPServerModel>
       cachedDescription,
       cachedTools,
       lastSyncAt,
+      clearError,
     }: {
       name?: string;
       description?: string;
@@ -244,6 +267,7 @@ export class RemoteMCPServerResource extends BaseResource<RemoteMCPServerModel>
       cachedDescription?: string;
       cachedTools?: MCPToolType[];
       lastSyncAt: Date;
+      clearError?: boolean;
     }
   ) {
     // If we update the cachedName or cachedDescription, and the name is currently the default one,
@@ -264,6 +288,23 @@ export class RemoteMCPServerResource extends BaseResource<RemoteMCPServerModel>
       cachedDescription,
       cachedTools,
       lastSyncAt,
+      lastError: clearError ? null : this.lastError,
+    });
+  }
+
+  async markAsErrored(
+    auth: Authenticator,
+    {
+      lastError,
+      lastSyncAt,
+    }: {
+      lastError: string;
+      lastSyncAt: Date;
+    }
+  ) {
+    await this.update({
+      lastError,
+      lastSyncAt,
     });
   }
 
@@ -274,6 +315,7 @@ export class RemoteMCPServerResource extends BaseResource<RemoteMCPServerModel>
     cachedDescription: string | null;
     url: string;
     lastSyncAt: number | null;
+    lastError: string | null;
     sharedSecret: string | null;
   } {
     const currentTime = new Date();
@@ -306,6 +348,7 @@ export class RemoteMCPServerResource extends BaseResource<RemoteMCPServerModel>
       // Remote MCP Server specifics
       url: this.url,
       lastSyncAt: this.lastSyncAt?.getTime() ?? null,
+      lastError: this.lastError,
       sharedSecret: secret,
     };
   }
diff --git a/front/lib/swr/mcp_servers.ts b/front/lib/swr/mcp_servers.ts
index ff2553b6d550..70bf2ee4e42e 100644
--- a/front/lib/swr/mcp_servers.ts
+++ b/front/lib/swr/mcp_servers.ts
@@ -266,6 +266,7 @@ export function useSyncRemoteMCPServer(
 
     if (!response.ok) {
       const error = await response.json();
+      await mutateMCPServer();
       throw new Error(
         error.api_error?.message || "Failed to synchronize server"
       );
diff --git a/front/migrations/db/migration_269.sql b/front/migrations/db/migration_269.sql
new file mode 100644
index 000000000000..b5edea155730
--- /dev/null
+++ b/front/migrations/db/migration_269.sql
@@ -0,0 +1,2 @@
+-- Migration created on May 15, 2025
+ALTER TABLE "public"."remote_mcp_servers" ADD COLUMN "lastError" TEXT DEFAULT NULL;
diff --git a/front/pages/api/w/[wId]/mcp/[serverId]/sync.ts b/front/pages/api/w/[wId]/mcp/[serverId]/sync.ts
index 60fd95da401e..5526a9850bd1 100644
--- a/front/pages/api/w/[wId]/mcp/[serverId]/sync.ts
+++ b/front/pages/api/w/[wId]/mcp/[serverId]/sync.ts
@@ -67,6 +67,11 @@ async function handler(
 
   const r = await fetchRemoteServerMetaDataByURL(auth, server.url);
   if (r.isErr()) {
+    await server.markAsErrored(auth, {
+      lastError: r.error.message,
+      lastSyncAt: new Date(),
+    });
+
     return apiError(req, res, {
       status_code: 400,
       api_error: {
@@ -83,6 +88,7 @@ async function handler(
     cachedDescription: metadata.description,
     cachedTools: metadata.tools,
     lastSyncAt: new Date(),
+    clearError: true,
   });
 
   return res.status(200).json({
diff --git a/front/start_worker.ts b/front/start_worker.ts
index a45d8b7a47ca..fb4e6eb508c3 100644
--- a/front/start_worker.ts
+++ b/front/start_worker.ts
@@ -11,6 +11,7 @@ import { runMentionsCountWorker } from "@app/temporal/mentions_count_queue/worke
 import { runPermissionsWorker } from "@app/temporal/permissions_queue/worker";
 import { runProductionChecksWorker } from "@app/temporal/production_checks/worker";
 import { runRelocationWorker } from "@app/temporal/relocation/worker";
+import { runRemoteToolsSyncWorker } from "@app/temporal/remote_tools/worker";
 import { runScrubWorkspaceQueueWorker } from "@app/temporal/scrub_workspace/worker";
 import {
   runTrackerNotificationWorker,
@@ -34,6 +35,7 @@ type WorkerName =
   | "poke"
   | "production_checks"
   | "relocation"
+  | "remote_tools_sync"
   | "scrub_workspace_queue"
   | "tracker_notification"
   | "update_workspace_usage"
@@ -51,6 +53,7 @@ const workerFunctions: Record<WorkerName, () => Promise<void>> = {
   poke: runPokeWorker,
   production_checks: runProductionChecksWorker,
   relocation: runRelocationWorker,
+  remote_tools_sync: runRemoteToolsSyncWorker,
   scrub_workspace_queue: runScrubWorkspaceQueueWorker,
   tracker_notification: runTrackerNotificationWorker,
   update_workspace_usage: runUpdateWorkspaceUsageWorker,
diff --git a/front/temporal/remote_tools/activities.ts b/front/temporal/remote_tools/activities.ts
new file mode 100644
index 000000000000..45fa9551efd0
--- /dev/null
+++ b/front/temporal/remote_tools/activities.ts
@@ -0,0 +1,96 @@
+import { fetchRemoteServerMetaDataByURL } from "@app/lib/actions/mcp_metadata";
+import { Authenticator } from "@app/lib/auth";
+import { RemoteMCPServerResource } from "@app/lib/resources/remote_mcp_servers_resource";
+import { getWorkspaceByModelId } from "@app/lib/workspace";
+import logger from "@app/logger/logger";
+
+export async function syncRemoteMCPServers(ids: number[]): Promise<void> {
+  logger.info({ msg: "Starting sync of remote_mcp_servers" });
+
+  try {
+    for (const id of ids) {
+      // Retrieve the remote MCP server
+      const server = await RemoteMCPServerResource.fetchByModelId(id);
+      if (!server) {
+        logger.error({
+          msg: "Remote MCP server not found",
+          serverId: id,
+        });
+        continue;
+      }
+
+      // Retrieve the workspace
+      const workspace = await getWorkspaceByModelId(server.workspaceId);
+      if (!workspace) {
+        logger.error({
+          msg: "Workspace not found for remote MCP server",
+          workspaceId: server.workspaceId,
+          serverId: server.sId,
+        });
+        continue;
+      }
+      const workspaceId = workspace.sId;
+      const auth = await Authenticator.internalAdminForWorkspace(workspaceId);
+
+      // Fetch the remote server metadata
+      const r = await fetchRemoteServerMetaDataByURL(auth, server.url);
+
+      if (r.isErr()) {
+        logger.error(
+          {
+            workspaceId,
+            serverId: server.sId,
+            url: server.url,
+            error: r.error.message,
+          },
+          "Error fetching remote server metadata"
+        );
+        await server.markAsErrored(auth, {
+          lastError: r.error.message,
+          lastSyncAt: new Date(),
+        });
+        continue;
+      }
+
+      const metadata = r.value;
+
+      // Update the server metadata
+      await server.updateMetadata(auth, {
+        cachedName: metadata.name,
+        cachedDescription: metadata.description,
+        cachedTools: metadata.tools,
+        lastSyncAt: new Date(),
+        clearError: true,
+      });
+
+      logger.info({
+        msg: "Successfully synced remote MCP server",
+        workspaceId,
+        serverId: server.sId,
+        url: server.url,
+      });
+    }
+  } catch (error) {
+    logger.error({
+      msg: "Failed to resync remote_mcp_servers",
+      error,
+    });
+    throw error;
+  }
+}
+
+/**
+ * Returns a batch of up to 100 RemoteMCPServerResource servers and a function to get the next batch.
+ */
+export async function getBatchRemoteMCPServers({
+  firstId = 0,
+  limit = 100,
+}: {
+  firstId?: number;
+  limit?: number;
+}): Promise<number[]> {
+  return await RemoteMCPServerResource.dangerouslyListAllServersIds({
+    firstId,
+    limit,
+  });
+}
diff --git a/front/temporal/remote_tools/admin/cli.ts b/front/temporal/remote_tools/admin/cli.ts
new file mode 100644
index 000000000000..a58a04671044
--- /dev/null
+++ b/front/temporal/remote_tools/admin/cli.ts
@@ -0,0 +1,31 @@
+import parseArgs from "minimist";
+
+import { launchRemoteMCPServersSyncWorkflow } from "@app/temporal/remote_tools/client";
+
+const main = async () => {
+  const argv = parseArgs(process.argv.slice(2));
+
+  const [command] = argv._;
+
+  console.log(`Running command: ${command}`);
+
+  switch (command) {
+    case "start":
+      await launchRemoteMCPServersSyncWorkflow();
+      return;
+    default:
+      console.log("Unknown command, possible values: `start`");
+      return;
+  }
+};
+
+main()
+  .then(() => {
+    console.error("\x1b[32m%s\x1b[0m", `Done`);
+    process.exit(0);
+  })
+  .catch((err) => {
+    console.error("\x1b[31m%s\x1b[0m", `Error: ${err.message}`);
+    console.log(err);
+    process.exit(1);
+  });
diff --git a/front/temporal/remote_tools/client.ts b/front/temporal/remote_tools/client.ts
new file mode 100644
index 000000000000..04aa7fd3bef1
--- /dev/null
+++ b/front/temporal/remote_tools/client.ts
@@ -0,0 +1,55 @@
+import { WorkflowNotFoundError } from "@temporalio/client";
+
+import { getTemporalClient } from "@app/lib/temporal";
+import logger from "@app/logger/logger";
+import type { Result } from "@app/types";
+import { Err, normalizeError, Ok } from "@app/types";
+
+import { QUEUE_NAME } from "@app/temporal/remote_tools/config";
+import { syncRemoteMCPServersWorkflow } from "@app/temporal/remote_tools/workflows";
+
+export async function launchRemoteMCPServersSyncWorkflow(): Promise<
+  Result<string, Error>
+> {
+  const client = await getTemporalClient();
+  const workflowId = "remote-mcp-servers-sync";
+
+  try {
+    try {
+      const handle = client.workflow.getHandle(workflowId);
+      await handle.terminate();
+    } catch (e) {
+      if (!(e instanceof WorkflowNotFoundError)) {
+        throw e;
+      }
+    }
+
+    await client.workflow.start(syncRemoteMCPServersWorkflow, {
+      args: [],
+      taskQueue: QUEUE_NAME,
+      workflowId: workflowId,
+      memo: {
+        workflowId,
+      },
+    });
+
+    logger.info(
+      {
+        workflowId,
+      },
+      "Started weekly remote MCP servers sync workflow."
+    );
+
+    return new Ok(workflowId);
+  } catch (e) {
+    logger.error(
+      {
+        workflowId,
+        error: e,
+      },
+      "Failed to start remote MCP servers sync workflow."
+    );
+
+    return new Err(normalizeError(e));
+  }
+}
diff --git a/front/temporal/remote_tools/config.ts b/front/temporal/remote_tools/config.ts
new file mode 100644
index 000000000000..43860a16c5c5
--- /dev/null
+++ b/front/temporal/remote_tools/config.ts
@@ -0,0 +1,2 @@
+const QUEUE_VERSION = 1;
+export const QUEUE_NAME = `remote_tools_sync-v${QUEUE_VERSION}`;
diff --git a/front/temporal/remote_tools/worker.ts b/front/temporal/remote_tools/worker.ts
new file mode 100644
index 000000000000..d99e3730d4cd
--- /dev/null
+++ b/front/temporal/remote_tools/worker.ts
@@ -0,0 +1,39 @@
+import type { Context } from "@temporalio/activity";
+import { Worker } from "@temporalio/worker";
+
+import { getTemporalWorkerConnection } from "@app/lib/temporal";
+import { ActivityInboundLogInterceptor } from "@app/lib/temporal_monitoring";
+import logger from "@app/logger/logger";
+import * as activities from "@app/temporal/remote_tools/activities";
+
+import { QUEUE_NAME } from "@app/temporal/remote_tools/config";
+import TsconfigPathsPlugin from "tsconfig-paths-webpack-plugin";
+
+export async function runRemoteToolsSyncWorker() {
+  const { connection, namespace } = await getTemporalWorkerConnection();
+  const worker = await Worker.create({
+    workflowsPath: require.resolve("./workflows"),
+    activities,
+    taskQueue: QUEUE_NAME,
+    connection,
+    namespace,
+    interceptors: {
+      activityInbound: [
+        (ctx: Context) => {
+          return new ActivityInboundLogInterceptor(ctx, logger);
+        },
+      ],
+    },
+    bundlerOptions: {
+      // Update the webpack config to use aliases from our tsconfig.json.
+      webpackConfigHook: (config) => {
+        const plugins = config.resolve?.plugins ?? [];
+        // eslint-disable-next-line @typescript-eslint/no-non-null-assertion
+        config.resolve!.plugins = [...plugins, new TsconfigPathsPlugin({})];
+        return config;
+      },
+    },
+  });
+
+  await worker.run();
+}
diff --git a/front/temporal/remote_tools/workflows.ts b/front/temporal/remote_tools/workflows.ts
new file mode 100644
index 000000000000..a0b1e2462723
--- /dev/null
+++ b/front/temporal/remote_tools/workflows.ts
@@ -0,0 +1,23 @@
+import { proxyActivities } from "@temporalio/workflow";
+
+import type * as activities from "@app/temporal/remote_tools/activities";
+
+const { syncRemoteMCPServers, getBatchRemoteMCPServers } = proxyActivities<
+  typeof activities
+>({
+  startToCloseTimeout: "10 minutes",
+});
+
+export async function syncRemoteMCPServersWorkflow() {
+  let ids = await getBatchRemoteMCPServers({
+    firstId: 0,
+    limit: 100,
+  });
+  do {
+    await syncRemoteMCPServers(ids);
+    ids = await getBatchRemoteMCPServers({
+      firstId: ids[ids.length - 1] + 1,
+      limit: 100,
+    });
+  } while (ids.length > 0);
+}


Reviews:
user: login: flvndvd
body: You'll also need a PR on `dust-infra` to ensure this worker runs. 
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-15T14:36:14Z
user: login: adrsimon
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-15T15:18:27Z
user: login: adrsimon
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-15T15:18:45Z
user: login: dust-agent[bot]
body: I've reviewed the code for compliance with our coding rules. Here are some issues that need to be addressed:
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-15T15:46:36Z
user: login: adrsimon
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-15T16:06:56Z
user: login: dust-agent[bot]
body: I found one coding rule violation that needs to be addressed.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-16T07:07:54Z
user: login: dust-agent[bot]
body: This PR has a few areas that don't align with our coding rules. Please see the in-line comments for details.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-21T09:20:30Z
user: login: adrsimon
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-21T09:24:46Z
user: login: flvndvd
body: Left a few comments but LGTM
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T07:40:50Z
user: login: flvndvd
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T07:41:27Z

Review comments:
diff_hunk: @@ -139,15 +132,10 @@ export function RemoteMCPForm({ owner, mcpServer }: RemoteMCPFormProps) {
         description:
           error instanceof Error ? error.message : "An error occurred",
path: front/components/actions/mcp/RemoteMCPForm.tsx
user: login: flvndvd
body: While you're at it, let's use `normalizeError` here.
created_at: 2025-05-15T14:27:31Z
author_association: CONTRIBUTOR
diff_hunk: @@ -67,6 +67,11 @@ async function handler(
 
   const r = await fetchRemoteServerMetaDataByURL(auth, server.url);
   if (r.isErr()) {
+    await server.markAsErrored(auth, {
+      lastError: r.error.message,
path: front/pages/api/w/[wId]/mcp/[serverId]/sync.ts
user: login: flvndvd
body: Do we have an idea of this error message could be? 
created_at: 2025-05-15T14:28:40Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,81 @@
+import { fetchRemoteServerMetaDataByURL } from "@app/lib/actions/mcp_metadata";
+import { Authenticator } from "@app/lib/auth";
+import { RemoteMCPServerModel } from "@app/lib/models/assistant/actions/remote_mcp_server";
+import { RemoteMCPServerResource } from "@app/lib/resources/remote_mcp_servers_resource";
+import { getWorkspaceByModelId } from "@app/lib/workspace";
+import logger from "@app/logger/logger";
+
+export async function syncRemoteMCPServers(): Promise<void> {
+  logger.info({ msg: "Starting sync of remote_mcp_servers" });
+
+  try {
+    const servers = await RemoteMCPServerModel.findAll();
path: front/temporal/remote_tools/activities.ts
user: login: flvndvd
body: This goes agains the resource pattern, all models abstracted by a resource, should not be queried directly!
created_at: 2025-05-15T14:29:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,81 @@
+import { fetchRemoteServerMetaDataByURL } from "@app/lib/actions/mcp_metadata";
+import { Authenticator } from "@app/lib/auth";
+import { RemoteMCPServerModel } from "@app/lib/models/assistant/actions/remote_mcp_server";
+import { RemoteMCPServerResource } from "@app/lib/resources/remote_mcp_servers_resource";
+import { getWorkspaceByModelId } from "@app/lib/workspace";
+import logger from "@app/logger/logger";
+
+export async function syncRemoteMCPServers(): Promise<void> {
+  logger.info({ msg: "Starting sync of remote_mcp_servers" });
+
+  try {
+    const servers = await RemoteMCPServerModel.findAll();
+
+    for (const s of servers) {
+      // Retrieve the resource
+      const server = await RemoteMCPServerResource.fetchByModelId(s.id);
+      if (!server) {
+        logger.error({
+          msg: "Remote MCP server not found",
+          serverId: s.id,
+        });
+        continue;
+      }
+
+      // Retrieve the authentication
+      const workspace = await getWorkspaceByModelId(server.workspaceId);
+      if (!workspace) {
+        logger.error({
+          msg: "Workspace not found for remote MCP server",
+          workspaceId: server.workspaceId,
+          serverId: server.sId,
+        });
+        continue;
+      }
+      const workspaceId = workspace.sId;
+      const auth = await Authenticator.internalAdminForWorkspace(workspaceId);
+
+      // Fetch the remote server metadata
+      const r = await fetchRemoteServerMetaDataByURL(auth, server.url);
+
+      if (r.isErr()) {
+        logger.error({
+          msg: "Error fetching remote server metadata",
+          workspaceId,
+          serverId: server.sId,
+          url: server.url,
+          error: r.error.message,
+        });
path: front/temporal/remote_tools/activities.ts
user: login: flvndvd
body: `nit`
```suggestion
        logger.error({
          workspaceId,
          serverId: server.sId,
          url: server.url,
          error: r.error.message,
        }, "Error fetching remote server metadata");
```
created_at: 2025-05-15T14:30:40Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,81 @@
+import { fetchRemoteServerMetaDataByURL } from "@app/lib/actions/mcp_metadata";
+import { Authenticator } from "@app/lib/auth";
+import { RemoteMCPServerModel } from "@app/lib/models/assistant/actions/remote_mcp_server";
+import { RemoteMCPServerResource } from "@app/lib/resources/remote_mcp_servers_resource";
+import { getWorkspaceByModelId } from "@app/lib/workspace";
+import logger from "@app/logger/logger";
+
+export async function syncRemoteMCPServers(): Promise<void> {
+  logger.info({ msg: "Starting sync of remote_mcp_servers" });
+
+  try {
+    const servers = await RemoteMCPServerModel.findAll();
path: front/temporal/remote_tools/activities.ts
user: login: flvndvd
body: Is it expected that you include servers that potentially are in an error state?
created_at: 2025-05-15T14:31:26Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,81 @@
+import { fetchRemoteServerMetaDataByURL } from "@app/lib/actions/mcp_metadata";
+import { Authenticator } from "@app/lib/auth";
+import { RemoteMCPServerModel } from "@app/lib/models/assistant/actions/remote_mcp_server";
+import { RemoteMCPServerResource } from "@app/lib/resources/remote_mcp_servers_resource";
+import { getWorkspaceByModelId } from "@app/lib/workspace";
+import logger from "@app/logger/logger";
+
+export async function syncRemoteMCPServers(): Promise<void> {
path: front/temporal/remote_tools/activities.ts
user: login: flvndvd
body: If the number of MCP server we have grows quickly, it's not ideal to have one single activity to refresh metadata for all of them. Ideally we do batch fetch. When we deploy `front`, Temporal ongoing activities have 2 minutes to stop, if they don't they are being killed, here if one is being killed, we restart from the beginning. Even though you plan to run it on Sunday, I'd advise splitting into batches. 
created_at: 2025-05-15T14:33:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,53 @@
+import { WorkflowNotFoundError } from "@temporalio/client";
+
+import { getTemporalClient } from "@app/lib/temporal";
+import logger from "@app/logger/logger";
+import type { Result } from "@app/types";
+import { Err, Ok } from "@app/types";
+
+import { QUEUE_NAME } from "./config";
+import { syncRemoteMCPServersWorkflow } from "./workflows";
+
+export async function launchRemoteMCPServersSyncWorkflow(): Promise<
+  Result<string, Error>
+> {
+  const client = await getTemporalClient();
+  const workflowId = "remote-mcp-servers-sync";
+
+  try {
+    try {
+      const handle = client.workflow.getHandle(workflowId);
+      await handle.terminate();
+    } catch (e) {
+      if (!(e instanceof WorkflowNotFoundError)) {
+        throw e;
+      }
+    }
+
+    await client.workflow.start(syncRemoteMCPServersWorkflow, {
path: front/temporal/remote_tools/client.ts
user: login: flvndvd
body: We've been consolidating around the Temporal schedule for this type of need. Several reasons, including reliability. We also have a production checks that ensure that schedules are properly scheduled. See `launchLabsConnectionWorkflow` for inspiration. 
created_at: 2025-05-15T14:35:40Z
author_association: CONTRIBUTOR
diff_hunk: @@ -67,6 +67,11 @@ async function handler(
 
   const r = await fetchRemoteServerMetaDataByURL(auth, server.url);
   if (r.isErr()) {
+    await server.markAsErrored(auth, {
+      lastError: r.error.message,
path: front/pages/api/w/[wId]/mcp/[serverId]/sync.ts
user: login: adrsimon
body: Will be `Error getting metadata from the remote MCP server.` for now, as per `/Users/adrien/git/dust/front/lib/actions/mcp_metadata.ts`
created_at: 2025-05-15T15:18:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,81 @@
+import { fetchRemoteServerMetaDataByURL } from "@app/lib/actions/mcp_metadata";
+import { Authenticator } from "@app/lib/auth";
+import { RemoteMCPServerModel } from "@app/lib/models/assistant/actions/remote_mcp_server";
+import { RemoteMCPServerResource } from "@app/lib/resources/remote_mcp_servers_resource";
+import { getWorkspaceByModelId } from "@app/lib/workspace";
+import logger from "@app/logger/logger";
+
+export async function syncRemoteMCPServers(): Promise<void> {
+  logger.info({ msg: "Starting sync of remote_mcp_servers" });
+
+  try {
+    const servers = await RemoteMCPServerModel.findAll();
path: front/temporal/remote_tools/activities.ts
user: login: adrsimon
body: 1. Will refacto to use a resource
2. Yes, expected
created_at: 2025-05-15T15:18:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,81 @@
+import { fetchRemoteServerMetaDataByURL } from "@app/lib/actions/mcp_metadata";
+import { Authenticator } from "@app/lib/auth";
+import { RemoteMCPServerModel } from "@app/lib/models/assistant/actions/remote_mcp_server";
+import { RemoteMCPServerResource } from "@app/lib/resources/remote_mcp_servers_resource";
+import { getWorkspaceByModelId } from "@app/lib/workspace";
+import logger from "@app/logger/logger";
+
+export async function syncRemoteMCPServers(): Promise<void> {
+  logger.info({ msg: "Starting sync of remote_mcp_servers" });
+
+  try {
+    const servers = await RemoteMCPServerModel.findAll();
path: front/temporal/remote_tools/activities.ts
user: login: dust-agent[bot]
body: **[BACK3]** _Resource invariant: no sequelize models outside of resources_

Direct usage of `RemoteMCPServerModel.findAll()` violates our coding rule. Any model should be abstracted through a Resource.
created_at: 2025-05-15T15:46:35Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,81 @@
+import { fetchRemoteServerMetaDataByURL } from "@app/lib/actions/mcp_metadata";
+import { Authenticator } from "@app/lib/auth";
+import { RemoteMCPServerModel } from "@app/lib/models/assistant/actions/remote_mcp_server";
+import { RemoteMCPServerResource } from "@app/lib/resources/remote_mcp_servers_resource";
+import { getWorkspaceByModelId } from "@app/lib/workspace";
+import logger from "@app/logger/logger";
+
+export async function syncRemoteMCPServers(): Promise<void> {
+  logger.info({ msg: "Starting sync of remote_mcp_servers" });
+
+  try {
+    const servers = await RemoteMCPServerModel.findAll();
+
+    for (const s of servers) {
+      // Retrieve the resource
+      const server = await RemoteMCPServerResource.fetchByModelId(s.id);
+      if (!server) {
+        logger.error({
+          msg: "Remote MCP server not found",
+          serverId: s.id,
+        });
+        continue;
+      }
+
+      // Retrieve the authentication
+      const workspace = await getWorkspaceByModelId(server.workspaceId);
+      if (!workspace) {
+        logger.error({
+          msg: "Workspace not found for remote MCP server",
+          workspaceId: server.workspaceId,
+          serverId: server.sId,
+        });
+        continue;
+      }
+      const workspaceId = workspace.sId;
+      const auth = await Authenticator.internalAdminForWorkspace(workspaceId);
+
+      // Fetch the remote server metadata
+      const r = await fetchRemoteServerMetaDataByURL(auth, server.url);
+
+      if (r.isErr()) {
+        logger.error({
+          msg: "Error fetching remote server metadata",
+          workspaceId,
+          serverId: server.sId,
+          url: server.url,
+          error: r.error.message,
+        });
path: front/temporal/remote_tools/activities.ts
user: login: dust-agent[bot]
body: **[GEN6]** _Comments must be sentences wrapped at 100 characters_

This comment "nit" from the reviewer should be addressed. The logger calls should follow our standard format with the message as the second parameter.
created_at: 2025-05-15T15:46:35Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,53 @@
+import { WorkflowNotFoundError } from "@temporalio/client";
+
+import { getTemporalClient } from "@app/lib/temporal";
+import logger from "@app/logger/logger";
+import type { Result } from "@app/types";
+import { Err, Ok } from "@app/types";
+
+import { QUEUE_NAME } from "./config";
+import { syncRemoteMCPServersWorkflow } from "./workflows";
+
+export async function launchRemoteMCPServersSyncWorkflow(): Promise<
+  Result<string, Error>
+> {
+  const client = await getTemporalClient();
+  const workflowId = "remote-mcp-servers-sync";
+
+  try {
+    try {
+      const handle = client.workflow.getHandle(workflowId);
+      await handle.terminate();
+    } catch (e) {
+      if (!(e instanceof WorkflowNotFoundError)) {
+        throw e;
+      }
+    }
+
+    await client.workflow.start(syncRemoteMCPServersWorkflow, {
+      args: [],
+      taskQueue: QUEUE_NAME,
+      workflowId: workflowId,
+      cronSchedule: "0 12 * * 0", // Run every Sunday at 12:00
path: front/temporal/remote_tools/client.ts
user: login: dust-agent[bot]
body: **[GEN6]** _Comments must be sentences wrapped at 100 characters_

The comment `// Run every Sunday at 12:00` is not a full sentence. It should be something like: "This schedule runs every Sunday at 12:00."
created_at: 2025-05-15T15:46:35Z
author_association: CONTRIBUTOR
diff_hunk: @@ -83,6 +88,7 @@ async function handler(
     cachedDescription: metadata.description,
     cachedTools: metadata.tools,
     lastSyncAt: new Date(),
+    clearError: true,
path: front/pages/api/w/[wId]/mcp/[serverId]/sync.ts
user: login: dust-agent[bot]
body: **[GEN6]** _Comments must be sentences wrapped at 100 characters_

Missing comment explaining what this `clearError: true` parameter does.
created_at: 2025-05-15T15:46:35Z
author_association: CONTRIBUTOR
diff_hunk: @@ -267,6 +269,23 @@ export class RemoteMCPServerResource extends BaseResource<RemoteMCPServerModel>
       cachedDescription,
       cachedTools,
       lastSyncAt,
+      lastError: clearError ? null : this.lastError,
+    });
+  }
+
+  async markAsErrored(
path: front/lib/resources/remote_mcp_servers_resource.ts
user: login: dust-agent[bot]
body: **[GEN6]** _Comments must be sentences wrapped at 100 characters_

Missing comment explaining this new `markAsErrored` method.
created_at: 2025-05-15T15:46:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,81 @@
+import { fetchRemoteServerMetaDataByURL } from "@app/lib/actions/mcp_metadata";
+import { Authenticator } from "@app/lib/auth";
+import { RemoteMCPServerModel } from "@app/lib/models/assistant/actions/remote_mcp_server";
+import { RemoteMCPServerResource } from "@app/lib/resources/remote_mcp_servers_resource";
+import { getWorkspaceByModelId } from "@app/lib/workspace";
+import logger from "@app/logger/logger";
+
+export async function syncRemoteMCPServers(): Promise<void> {
+  logger.info({ msg: "Starting sync of remote_mcp_servers" });
+
+  try {
+    const servers = await RemoteMCPServerModel.findAll();
path: front/temporal/remote_tools/activities.ts
user: login: adrsimon
body: yeah you just copied flav no need to brag here
created_at: 2025-05-15T16:06:56Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,53 @@
+import { WorkflowNotFoundError } from "@temporalio/client";
+
+import { getTemporalClient } from "@app/lib/temporal";
+import logger from "@app/logger/logger";
+import type { Result } from "@app/types";
+import { Err, Ok } from "@app/types";
+
+import { QUEUE_NAME } from "./config";
+import { syncRemoteMCPServersWorkflow } from "./workflows";
+
+export async function launchRemoteMCPServersSyncWorkflow(): Promise<
+  Result<string, Error>
+> {
+  const client = await getTemporalClient();
+  const workflowId = "remote-mcp-servers-sync";
+
+  try {
+    try {
+      const handle = client.workflow.getHandle(workflowId);
+      await handle.terminate();
+    } catch (e) {
+      if (!(e instanceof WorkflowNotFoundError)) {
+        throw e;
+      }
+    }
+
+    await client.workflow.start(syncRemoteMCPServersWorkflow, {
+      args: [],
+      taskQueue: QUEUE_NAME,
+      workflowId: workflowId,
+      cronSchedule: "0 12 * * 0", // Run every Sunday at 12:00
path: front/temporal/remote_tools/client.ts
user: login: dust-agent[bot]
body: **[GEN6]** _Comments must be sentences wrapped at 100 characters_
This comment should be a full sentence, starting with a capital letter and ending with a period (e.g., "This cron schedule runs every Sunday at 12:00.").
created_at: 2025-05-16T07:07:54Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,105 @@
+import { fetchRemoteServerMetaDataByURL } from "@app/lib/actions/mcp_metadata";
+import { Authenticator } from "@app/lib/auth";
+import { RemoteMCPServerResource } from "@app/lib/resources/remote_mcp_servers_resource";
+import { getWorkspaceByModelId } from "@app/lib/workspace";
+import logger from "@app/logger/logger";
+
+export async function syncRemoteMCPServers(ids: number[]): Promise<void> {
+  logger.info({ msg: "Starting sync of remote_mcp_servers" });
+
+  try {
+    for (const id of ids) {
+      // Retrieve the remote MCP server
+      const server = await RemoteMCPServerResource.fetchByModelId(id);
+      if (!server) {
+        logger.error({
+          msg: "Remote MCP server not found",
+          serverId: id,
+        });
+        continue;
+      }
+
+      // Retrieve the workspace
+      const workspace = await getWorkspaceByModelId(server.workspaceId);
+      if (!workspace) {
+        logger.error({
+          msg: "Workspace not found for remote MCP server",
+          workspaceId: server.workspaceId,
+          serverId: server.sId,
+        });
+        continue;
+      }
+      const workspaceId = workspace.sId;
+      const auth = await Authenticator.internalAdminForWorkspace(workspaceId);
+
+      // Fetch the remote server metadata
+      const r = await fetchRemoteServerMetaDataByURL(auth, server.url);
+
+      if (r.isErr()) {
+        logger.error(
+          {
+            workspaceId,
+            serverId: server.sId,
+            url: server.url,
+            error: r.error.message,
+          },
+          "Error fetching remote server metadata"
+        );
+        await server.markAsErrored(auth, {
+          lastError: r.error.message,
+          lastSyncAt: new Date(),
+        });
+        continue;
+      }
+
+      const metadata = r.value;
+
+      // Update the server metadata
+      await server.updateMetadata(auth, {
+        cachedName: metadata.name,
+        cachedDescription: metadata.description,
+        cachedTools: metadata.tools,
+        lastSyncAt: new Date(),
+        clearError: true,
+      });
+
+      logger.info({
+        msg: "Successfully synced remote MCP server",
+        workspaceId,
+        serverId: server.sId,
+        url: server.url,
+      });
+    }
+  } catch (error) {
+    logger.error({
+      msg: "Failed to resync remote_mcp_servers",
+      error,
+    });
+    throw error;
+  }
+}
+
+const EMPTY_BATCH = {
+  servers: [],
+  next: async () => EMPTY_BATCH,
+};
+
+/**
+ * Returns a batch of up to 100 RemoteMCPServerResource servers and a function to get the next batch.
+ */
+export async function getBatchRemoteMCPServers(offset = 0, limit = 100) {
+  const servers = await RemoteMCPServerResource.dangerouslyListAllServersIds(
+    offset,
+    limit
+  );
+  return {
+    servers,
+    next: async () => {
+      if (servers.length < limit) {
+        // No more servers left
path: front/temporal/remote_tools/activities.ts
user: login: dust-agent[bot]
body: **[GEN6]** _Comments must be sentences and properly wrapped_ :cite[cc]
This comment is not a full sentence. Please rephrase to be a full sentence, for example: "Indicates that there are no more servers left to process in the current batch." or remove if the subsequent code is self-explanatory.
created_at: 2025-05-21T09:20:30Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,71 @@
+import { WorkflowNotFoundError } from "@temporalio/client";
+
+import { getTemporalClient } from "@app/lib/temporal";
+import logger from "@app/logger/logger";
+import type { Result } from "@app/types";
+import { Err, Ok } from "@app/types";
+
+import { QUEUE_NAME } from "./config";
+import { syncRemoteMCPServersWorkflow } from "./workflows";
+
+export async function launchRemoteMCPServersSyncWorkflow(): Promise<
+  Result<string, Error>
+> {
+  const client = await getTemporalClient();
+  const workflowId = "remote-mcp-servers-sync";
+
+  try {
+    try {
+      const handle = client.workflow.getHandle(workflowId);
+      await handle.terminate();
+    } catch (e) {
+      if (!(e instanceof WorkflowNotFoundError)) {
+        throw e;
+      }
+    }
+
+    await client.workflow.start(syncRemoteMCPServersWorkflow, {
+      args: [],
+      taskQueue: QUEUE_NAME,
+      workflowId: workflowId,
+      memo: {
+        workflowId,
+      },
+    });
+
+    await client.schedule.create({
+      action: {
+        type: "startWorkflow",
+        workflowType: syncRemoteMCPServersWorkflow,
+        args: [],
+        taskQueue: QUEUE_NAME,
+      },
+      scheduleId: workflowId,
+      spec: {
+        cronExpressions: ["0 12 * * 0"], // Every Sunday at 12:00 PM
+      },
path: front/temporal/remote_tools/client.ts
user: login: dust-agent[bot]
body: **[GEN6]** _Comments must be sentences and properly wrapped_ :cite[cc]
This comment is not a full sentence. Please rephrase to be a full sentence, for example: "This cron expression schedules the workflow to run every Sunday at 12:00 PM."
created_at: 2025-05-21T09:20:30Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,71 @@
+import { WorkflowNotFoundError } from "@temporalio/client";
+
+import { getTemporalClient } from "@app/lib/temporal";
+import logger from "@app/logger/logger";
+import type { Result } from "@app/types";
+import { Err, Ok } from "@app/types";
+
+import { QUEUE_NAME } from "./config";
+import { syncRemoteMCPServersWorkflow } from "./workflows";
+
+export async function launchRemoteMCPServersSyncWorkflow(): Promise<
+  Result<string, Error>
+> {
+  const client = await getTemporalClient();
+  const workflowId = "remote-mcp-servers-sync";
+
+  try {
+    try {
+      const handle = client.workflow.getHandle(workflowId);
+      await handle.terminate();
+    } catch (e) {
+      if (!(e instanceof WorkflowNotFoundError)) {
+        throw e;
+      }
+    }
+
+    await client.workflow.start(syncRemoteMCPServersWorkflow, {
+      args: [],
+      taskQueue: QUEUE_NAME,
+      workflowId: workflowId,
+      memo: {
+        workflowId,
+      },
+    });
+
+    await client.schedule.create({
+      action: {
+        type: "startWorkflow",
+        workflowType: syncRemoteMCPServersWorkflow,
+        args: [],
+        taskQueue: QUEUE_NAME,
+      },
+      scheduleId: workflowId,
+      spec: {
+        cronExpressions: ["0 12 * * 0"], // Every Sunday at 12:00 PM
+      },
+      memo: {
+        workflowId,
+      },
+    });
+
+    logger.info(
+      {
+        workflowId,
+      },
+      "Started weekly remote MCP servers sync workflow."
+    );
+
+    return new Ok(workflowId);
+  } catch (e) {
+    logger.error(
+      {
+        workflowId,
+        error: e,
+      },
+      "Failed to start remote MCP servers sync workflow."
+    );
+
+    return new Err(e as Error);
path: front/temporal/remote_tools/client.ts
user: login: dust-agent[bot]
body: **[ERR2]** _Do not rely on `err as Error`_ :cite[cc]
Casting an error with `e as Error` can hide the actual error type and lead to runtime issues if `e` is not an `Error` instance. Please use `normalizeError(e)` to ensure a proper `Error` object is handled.
created_at: 2025-05-21T09:20:30Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,71 @@
+import { WorkflowNotFoundError } from "@temporalio/client";
+
+import { getTemporalClient } from "@app/lib/temporal";
+import logger from "@app/logger/logger";
+import type { Result } from "@app/types";
+import { Err, Ok } from "@app/types";
+
+import { QUEUE_NAME } from "./config";
+import { syncRemoteMCPServersWorkflow } from "./workflows";
+
+export async function launchRemoteMCPServersSyncWorkflow(): Promise<
+  Result<string, Error>
+> {
+  const client = await getTemporalClient();
+  const workflowId = "remote-mcp-servers-sync";
+
+  try {
+    try {
+      const handle = client.workflow.getHandle(workflowId);
+      await handle.terminate();
+    } catch (e) {
+      if (!(e instanceof WorkflowNotFoundError)) {
+        throw e;
+      }
+    }
+
+    await client.workflow.start(syncRemoteMCPServersWorkflow, {
+      args: [],
+      taskQueue: QUEUE_NAME,
+      workflowId: workflowId,
+      memo: {
+        workflowId,
+      },
+    });
+
+    await client.schedule.create({
+      action: {
+        type: "startWorkflow",
+        workflowType: syncRemoteMCPServersWorkflow,
+        args: [],
+        taskQueue: QUEUE_NAME,
+      },
+      scheduleId: workflowId,
+      spec: {
+        cronExpressions: ["0 12 * * 0"], // Every Sunday at 12:00 PM
+      },
+      memo: {
+        workflowId,
+      },
+    });
+
+    logger.info(
+      {
+        workflowId,
+      },
+      "Started weekly remote MCP servers sync workflow."
+    );
+
+    return new Ok(workflowId);
+  } catch (e) {
+    logger.error(
+      {
+        workflowId,
+        error: e,
+      },
+      "Failed to start remote MCP servers sync workflow."
+    );
+
+    return new Err(e as Error);
path: front/temporal/remote_tools/client.ts
user: login: adrsimon
body: :eyes:

created_at: 2025-05-21T09:24:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -156,6 +156,16 @@ export class RemoteMCPServerResource extends BaseResource<RemoteMCPServerModel>
     return servers.length > 0 ? servers[0] : null;
   }
 
+  // Admin operations - don't use in non-temporal code.
+  static async dangerouslyListAllServersIds(offset = 0, limit = 100) {
path: front/lib/resources/remote_mcp_servers_resource.ts
user: login: flvndvd
body: `nit`
```suggestion
  static async dangerouslyListAllServersIds({limit, offset}: {limit: number, offset: number}) {
```
created_at: 2025-05-22T07:29:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,105 @@
+import { fetchRemoteServerMetaDataByURL } from "@app/lib/actions/mcp_metadata";
path: front/temporal/remote_tools/activities.ts
user: login: flvndvd
body: ```suggestion
import { fetchRemoteServerMetadataByURL } from "@app/lib/actions/mcp_metadata";
```
created_at: 2025-05-22T07:31:00Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,71 @@
+import { WorkflowNotFoundError } from "@temporalio/client";
+
+import { getTemporalClient } from "@app/lib/temporal";
+import logger from "@app/logger/logger";
+import type { Result } from "@app/types";
+import { Err, normalizeError, Ok } from "@app/types";
+
+import { QUEUE_NAME } from "./config";
+import { syncRemoteMCPServersWorkflow } from "./workflows";
+
+export async function launchRemoteMCPServersSyncWorkflow(): Promise<
+  Result<string, Error>
+> {
+  const client = await getTemporalClient();
+  const workflowId = "remote-mcp-servers-sync";
+
+  try {
+    try {
+      const handle = client.workflow.getHandle(workflowId);
+      await handle.terminate();
+    } catch (e) {
+      if (!(e instanceof WorkflowNotFoundError)) {
+        throw e;
+      }
+    }
+
+    await client.workflow.start(syncRemoteMCPServersWorkflow, {
path: front/temporal/remote_tools/client.ts
user: login: flvndvd
body: You don't need this, schedule should be enough.
created_at: 2025-05-22T07:34:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -156,6 +156,16 @@ export class RemoteMCPServerResource extends BaseResource<RemoteMCPServerModel>
     return servers.length > 0 ? servers[0] : null;
   }
 
+  // Admin operations - don't use in non-temporal code.
+  static async dangerouslyListAllServersIds(offset = 0, limit = 100) {
+    const servers = await RemoteMCPServerModel.findAll({
path: front/lib/resources/remote_mcp_servers_resource.ts
user: login: flvndvd
body: `nit`
We usually use a single where clause on `id` field if we order.
created_at: 2025-05-22T07:39:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,29 @@
+import type { Context } from "@temporalio/activity";
+import { Worker } from "@temporalio/worker";
+
+import { getTemporalWorkerConnection } from "@app/lib/temporal";
+import { ActivityInboundLogInterceptor } from "@app/lib/temporal_monitoring";
+import logger from "@app/logger/logger";
+import * as activities from "@app/temporal/remote_tools/activities";
+
+import { QUEUE_NAME } from "./config";
+
+export async function runRemoteToolsSyncWorker() {
+  const { connection, namespace } = await getTemporalWorkerConnection();
+  const worker = await Worker.create({
+    workflowsPath: require.resolve("./workflows"),
+    activities,
+    taskQueue: QUEUE_NAME,
+    connection,
+    namespace,
+    interceptors: {
+      activityInbound: [
+        (ctx: Context) => {
+          return new ActivityInboundLogInterceptor(ctx, logger);
path: front/temporal/remote_tools/worker.ts
user: login: flvndvd
body: Let's leverage the tsconfig here so we can't stop using the relative path import. See relocation worker.
created_at: 2025-05-22T07:40:44Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,71 @@
+import { WorkflowNotFoundError } from "@temporalio/client";
+
+import { getTemporalClient } from "@app/lib/temporal";
+import logger from "@app/logger/logger";
+import type { Result } from "@app/types";
+import { Err, normalizeError, Ok } from "@app/types";
+
+import { QUEUE_NAME } from "./config";
+import { syncRemoteMCPServersWorkflow } from "./workflows";
+
+export async function launchRemoteMCPServersSyncWorkflow(): Promise<
path: front/temporal/remote_tools/client.ts
user: login: flvndvd
body: Bundle this in a CLI command so we can start it.
created_at: 2025-05-22T07:41:26Z
author_association: CONTRIBUTOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-05-15T14:24:30Z
body: 
<!--
  1 failure:  Please include a ...
  1 warning:  Files in `**/lib/...
  
  
  DangerID: danger-id-Danger;
-->

<table>
  <thead>
    <tr>
      <th width="50"></th>
      <th width="100%" data-danger-table="true">Fails</th>
    </tr>
  </thead>
  <tbody><tr>
      <td>:no_entry_sign:</td>
      <td>Please include a detailed Deploy Plan section in your PR description.</td>
    </tr>
  </tbody>
</table>


<table>
  <thead>
    <tr>
      <th width="50"></th>
      <th width="100%" data-danger-table="true">Warnings</th>
    </tr>
  </thead>
  <tbody><tr>
      <td>:warning:</td>
      <td>

  Files in `**/lib/models/` have been modified and the PR has the `migration-ack` label. Don't forget to run the migration from prodbox.
  </td>
    </tr>
  </tbody>
</table>



<p align="right">
  Generated by :no_entry_sign: <a href="https://danger.systems/js">dangerJS</a> against ae71141f60e0da668c1fd5821ede2d44002db06b
</p>

author_association: CONTRIBUTOR
user: login: claude[bot]
created_at: 2025-05-21T09:19:55Z
body: **Claude finished @adrsimon's task** —— [View job](https://github.com/dust-tt/dust/actions/runs/15158414480)

---
### Code Review for PR #12691

I've reviewed the code changes in this PR that add automatic syncing for remote MCP servers and persistence of error states. Here's my analysis of the changes:

#### Issues to Address

1. **Resource Pattern Violation (CRITICAL)**: 
   - In `front/temporal/remote_tools/activities.ts:91-94`, direct usage of the model `RemoteMCPServerResource.dangerouslyListAllServersIds` violates [BACK3] - "Resource invariant: no sequelize models outside of resources." The `fetchByModelId` should be used or a proper resource method should be created.

2. **Comment Format Issues (MEDIUM)**:
   - In `front/temporal/remote_tools/client.ts:45`, the comment `// Every Sunday at 12:00 PM` is not a complete sentence with proper capitalization and period, violating [GEN6].
   - Missing proper documentation for the new methods, especially `markAsErrored` in `front/lib/resources/remote_mcp_servers_resource.ts:285`.
   - Missing comment explaining the purpose of `clearError: true` parameter in `front/pages/api/w/[wId]/mcp/[serverId]/sync.ts:91`.

3. **Error Handling (MEDIUM)**:
   - In `front/components/actions/mcp/RemoteMCPForm.tsx:129`, `error` is used directly without using `normalizeError()`. This should be fixed to adhere to [ERR2].

4. **Performance Concern (MEDIUM)**:
   - The current implementation syncs all MCP servers in a single activity. As noted in the review comments, if the number of servers grows significantly, this might lead to timeouts. Consider implementing batch processing with smaller chunks.

#### Strengths

1. The code properly tracks errors in MCP servers and has good UI integration to display these errors.
2. The implementation of a scheduled weekly sync is well structured using Temporal.
3. The error handling mechanisms are generally well integrated across the system.
4. The database migration is appropriately minimal and focused.
5. The code generally follows the codebase's patterns and conventions.

#### Security Considerations

The implementation appears secure, with proper authorization checks in the API route and appropriate resource access controls.

#### Recommendations

1. **Resource Pattern**:
   - Create a proper Resource method in `RemoteMCPServerResource` for listing server IDs that doesn't violate the resource pattern, rather than using `dangerouslyListAllServersIds`.

2. **Comments**:
   - Fix comment formatting to comply with [GEN6] by ensuring all comments are complete sentences with proper capitalization and periods.
   - Add detailed documentation for the new methods and parameters.

3. **Error Handling**:
   - Replace direct error usage with `normalizeError()` in `RemoteMCPForm.tsx`.

4. **Performance**:
   - Consider implementing batch processing to handle large numbers of MCP servers more efficiently.

5. **Logger Formatting**:
   - Adjust the logger call in `activities.ts` as suggested in the review comments to properly separate the message from the metadata.

Overall, this is a solid PR with a few fixable issues. The most critical one is the violation of the resource pattern, which should be addressed before merging.
author_association: NONE