Spaces:

huggingchat

/

chat-ui

like 965

Running on CPU Upgrade

What's your deployment process ? Where are you hosting chat-ui? 👀

I used the docker for mongo: docker run -d -p 27017:27017 --name mongo-chatui mongo:latest
I built the chat UI from source. Nodejs is 20.5.1. Just ran npm install then npm dev to run. Its just reaching out to some text-generation-inference endpoints.
Only config modifications are the TGI endpoints and the mongo db thing.

I have started having the same issue today. First message in a chat is fine, second message results in the same stack trace. I'm running via docker compose. The compose file creates a mongodb container and an ollama container.

The issue is present in both of these images:

ghcr.io/huggingface/chat-ui         latest          ef44d918bdb8   5 hours ago     688MB
ghcr.io/huggingface/chat-ui         <none>          d0429d46f7f8   32 hours ago    688MB

From the browser dev tools, it looks like it's sending a POST to https://my.domain.com/conversation/<conversation_id> and getting a 500 back.

As a workaround, I can select a different chat, then select the chat again, and then ask a follow-up. However, asking a second question after navigating to the chat doesn't work, whether it's a new chat or an old chat.

I bisected the published container images, and it looks like the last good image was sha256:def9c80dc77d0f37dd8bb2f1ba5728cb058366b14123043c1fb3178a4d766d14 (find it here). As a workaround, one can pin this image with docker tag for now.

The next published image (sha256:486a0f652ab43a1dcd332dcca97b393fcfe767c67cb4581098c0600820b9c6d7) is an arm64 image so I can't test that on my server, and the very next one (sha256:235588f6b9793a20607e8fc479eab4abc81aad1a591d82fe53e8af93ec16d287) has the "Ancestor not found" error.

Hope that helps narrow it down.

Got this, too. First failing commit is the current head e6addfc54e52 ("Conversation trees (#223) (#807)")
Unfortunately, no history of the merged branch or useful commit messages.

Will take a look!

I had the same problem when building my docker image locally, for me it was caused by not having ORIGIN set when running PM2.
running locally like this made the problem go away:
docker run -p 3000:3000 -e ORIGIN='http://localhost:3000' containername

This problem was independent of docker for me. npm run preview worked, but just running node or pm2 did not.

But there is something strange still going on. I get everything to run fine when using localhost, but as soon as I run the container on another server and use an actual IP address the POST to /conversation gives a 405.

Hi I merged some changes to main, can you pull the latest version and tell me if things are working better on your hand ? Feel free to let me know if not!

I am still getting the ancestor not found issue, not sure about UUID as that was something I resolved by switching to https.

@oxaronick could you share your docker compose file (without any secrets you might have) so I could try to reproduce the issue locally?

Can confirm, error still appears with 00443e1 ("Pass websearch to preprocessMessages (#876)")

However, new information: The error only appears when I connect via a proxy nginx (for https), not when I connect directly to vite. Sorry, that I missed that!

Relevant nginx config:

   # chat.ai.molgen.mpg.de
    server {
        listen 443 ssl;
        server_name chat.ai.molgen.mpg.de

        ssl_certificate       fullchain.pem;
        ssl_certificate_key   privkey.pem;
        ssl_session_cache     shared:SSL:1m;
        ssl_session_timeout   24h;

        ssl_prefer_server_ciphers on;
        ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';
        ssl_dhparam /etc/nginx/dhparams.pem;
        add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;

        location / {
            client_max_body_size 100M;
            proxy_pass http://ai.molgen.mpg.de:5174;
            proxy_set_header Host $host;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "Upgrade";
        }
    }

So http://ai.molgen.mpg.de:5174/ seems to work an https://chat.ai.molgen.mpg.de/ fails.

buczek@ai:~/chat-ui$ npm run dev -- --host ai.molgen.mpg.de --port 5174   

> chat-ui@0.7.0 dev
> vite dev --host ai.molgen.mpg.de --port 5174

  VITE v4.5.2  ready in 2248 ms

  ➜  Network: http://ai.molgen.mpg.de:5174/
  ➜  press h to show help
Error: Ancestor not found
    at /home/buczek/chat-ui/src/lib/utils/tree/buildSubtree.ts:22:26
    at Array.map (<anonymous>)
    at Module.buildSubtree (/home/buczek/chat-ui/src/lib/utils/tree/buildSubtree.ts:20:27)
    at POST (/home/buczek/chat-ui/src/routes/conversation/[id]/+server.ts:236:23)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Module.render_endpoint (/home/buczek/chat-ui/node_modules/@sveltejs/kit/src/runtime/server/endpoint.js:43:18)
    at async resolve (/home/buczek/chat-ui/node_modules/@sveltejs/kit/src/runtime/server/respond.js:404:17)
    at async Object.handle (/home/buczek/chat-ui/src/hooks.server.ts:150:19)
    at async Module.respond (/home/buczek/chat-ui/node_modules/@sveltejs/kit/src/runtime/server/respond.js:274:20)
    at async file:///home/buczek/chat-ui/node_modules/@sveltejs/kit/src/exports/vite/dev/index.js:510:22

Maybe obvious but could you make sure that PUBLIC_ORIGIN in your .env. is set to the public-facing URL you use to access your app ?

For example for HuggingChat it would be PUBLIC_ORIGIN=https://huggingface.co (see here)

Let me know if that fixes it.

Pushed some new changes that should hopefully fix this now!

Thanks!
I did not sent PUBLIC_ORIGIN. However, setting PUBLIC_ORIGIN=https://chat.ai.molgen.mpg.de does not fix the problem.
I don't see new commits on https://huggingface.co/spaces/huggingchat/chat-ui yet..

Sounds like you might have it already figured out, but here's my compose file.

version: '3'
services:
  mongodb:
    image: mongo:latest
    restart: unless-stopped
    volumes:
      - ./mongodb:/data/db
    networks:
      - internal
  chat:
    image: ghcr.io/huggingface/chat-ui:latest
    restart: unless-stopped
    networks:
      - webserver
      - internal
    volumes:
      - "./.env.local:/app/.env.local"
      - "./assets:/app/static/chatui"

networks:
  webserver:
    name: https_proxy_net
    external: true
  internal:
    name: chat
    external: true

The NGINX compose file is elsewhere; lmk if you want to see that.

I've only ever connected to chat through an NGINX proxy, so it's probably the same for me as for @buczek - issue only appears when connecting through a reverse proxy.

FWIW, I have had PUBLIC_ORIGIN set for a while.

@buczek @oxaronick the fix should be available on the space now and on github too! let me know if it works for you

Yes, that fixed it for me. Thanks!

latest is working for me as well - thanks!