Spaces:

huggingchat

/

chat-ui

like 958

Running on CPU Upgrade

I think I've asked this elsewhere but I'm not sure what the answer was. Do you use a paid API to query Google search? I'm asking because I can imagine that if it's through something hacky like selenium, Google won't like it (and they'll miss ad revenue). So, in short what does the technical pipeline look like for this from user query to generated output?

I think I've asked this elsewhere but I'm not sure what the answer was. Do you use a paid API to query Google search? I'm asking because I can imagine that if it's through something hacky like selenium, Google won't like it (and they'll miss ad revenue). So, in short what does the technical pipeline look like for this from user query to generated output?

You can check out the feature here!

@BramVanroy , they use the SerpAPI which, as far as I understand, is paid and legal. see the source code here: https://github.com/huggingface/chat-ui/blob/main/src/lib/server/websearch/searchWeb.ts

Question regarding sources/citations: Do I understand correctly that you currently display all URLs as sources, which the retriever retrieved and gave the LLM as context (regardless of whether the LLM actually used/refers to the source)? In Bing chat, they somehow managed to attribute sources to specific parts/sentences of the generated output and not only the generated output as a whole. I've always been wondering how they made that work (direct citations / attributing sources to specific sub-parts of a generated output). does anyone know?

Question regarding sources/citations: Do I understand correctly that you currently display all URLs as sources, which the retriever retrieved and gave the LLM as context

Yes

In Bing chat, they somehow managed to attribute sources to specific parts/sentences.

Interesting. I guess one simple way would be: for every generated sentence calculate its similarity against the sources and decide the highest scoring source as the source of that sentence. Not sure if that's what they are doing

Any chances for a search API that isn't Google's? IE: DuckDuckGo or Bing :)
Awesome work though! Really cool :D

In Bing chat, they somehow managed to attribute sources to specific parts/sentences.

Interesting. I guess one simple way would be: for every generated sentence calculate its similarity against the sources and decide the highest scoring source as the source of that sentence. Not sure if that's what they are doing

@mishig , yeah true. My first intuition was that I would be hesitant to trust embeddings from a bi-encoder to be reliable enough for this. But if you took a cross encoder, that should actually work quite well. especially if you set a high enough threshold to avoid false positives (then it could even work with a bi-encoder sentence transformer). could be a nice feature to have direct citations :)

@mishig first of all thanks for working on the project and making this feature better for everyone , the interface looks like perplexity AI and it was better then i thought , i have been having this issues where it just searches something but then never shows me the answer it shows max tokens
i just keep seeing this
first i see the links of the resources

but then it does nothing at all

should i create the issue on Github

but then never shows me the answer it shows max tokens

Nice catch, never had this.

Any chances for a search API that isn't Google's? IE: DuckDuckGo or Bing :)

Yes that would be cool if someone wants to PR https://github.com/huggingface/chat-ui

I tried Llama 2 70B with web search. I asked it about myself and what is my expertise and that it shall include linkedin in its research. It was a complete fail. I tried it three times and it always failed. It didn't search linkedin and it made things up. It said that I'm a medicine professor although I'm a software architect. For many years I had been a freelance developer, you get countless results if you google my name and I have a detailed public profile on linkedin. One answer had search hits on Xing where I also have profile, but it was a different person. Something is going terribly wrong with the web search.

that it shall include linkedin in its research.

Yes, the content of LinkedIn pages is not parsed by the web search. For the Xing page the rendering of users informations seems to happens client-side so it wont work.

It hallucinates very hard with anything scholarly, sources are almost non existent (The sources provided do not match with the text bot gives) It should be better with integrating pubmed etc free engines to web search.

It hallucinates very hard with anything scholarly, sources are almost non existent (The sources provided do not match with the text bot gives) It should be better with integrating pubmed etc free engines to web search.

@zelosleone I see. Yes right now we are just performing simple google search and picking the top 10 organic results. But as you have said, it is prone to hallucinating on scholarly topics. As a next step, what we can potentially do is: a routing mechanism (i.e. classifier) that classified user search query to an appropriate search engine (pubmed for biomed, arxiv for ML, etc.)

Any chances for a search API that isn't Google's? IE: DuckDuckGo or Bing :)

As Victor said, @DarwinAnim8or not at the moment. But it shouldn't be too hard to abstract away search engine interface and let the user plugin the choice of engine.
Specifically, you can start looking at this part of the codebase: https://github.com/huggingface/chat-ui/blob/main/src/lib/server/websearch/searchWeb.ts

@bilalazhar50 I see that you are getting the max new tokens error. Did it happen on the first try or was it a continuation of a long existing chat (in which case, you hit the limit of max new tokens) ?

I tried Llama 2 70B with web search. I asked it about myself and what is my expertise and that it shall include linkedin in its research. It was a complete fail

@krassmann atm, linkedin has a mechanism against scraping, which is why your linkedin was not included in the context. However, we will keep iteratively improving the feature 🤗

Any chances for a search API that isn't Google's? IE: DuckDuckGo or Bing :)
Awesome work though! Really cool :D

I worked on a simple and similar chatbot and I found DuckDuckGo doesn't provide links as Google Search API does. I managed to extract the "title" of the 5 organic results, extract the "snippet text" also concatenate it and send to LLM (using Openai functions). I would like to improve it.

Will you be adding a feature to upload files like pdfs or txt files?

Will you be adding a feature to upload files like pdfs or txt files?

yes

Will you be adding a feature to upload files like pdfs or txt files?

I have an open issue and PR for PDF support, but more specially PDF support with an OCR integration to handle math equation and table. This leverage the Mathpix private API so this feature is not for everyone and will likely not be include in the public version of chat-ui. But at least you can try to copy my code and deploy an instance of chat-ui with this. Maybe this will be include as an optional feature in the future.

Mathpix is currently the only viable OCR solution I know, I don't know any open source solution that could produce such result at the time.

However 90% of the user don't need advanced OCR features and a simple text extraction is enouth, Claude and Perplexity PDF feature work this way and that already quite impressive.

@mishig Did you have more info about the current work on this feature ? I don't see anything on GH expect my issue and PR

Any chances for a search API that isn't Google's? IE: DuckDuckGo or Bing :)

Yes that would be cool if someone wants to PR https://github.com/huggingface/chat-ui

Look like a good idea, if I find the time soon I might try to do this

@blanchon have you checked https://facebookresearch.github.io/nougat/ yet ?

Any chances for a search API that isn't Google's? IE: DuckDuckGo or Bing :)

Yes that would be cool if someone wants to PR https://github.com/huggingface/chat-ui

Look like a good idea, if I find the time soon I might try to do this

https://github.com/huggingface/chat-ui/issues/338#issuecomment-1631135948

Maybe this can help

The LLM seems to forget any previous conversation information. I asked it first about some code for Godot (game engine) without web search. Then afterwards, I asked it to convert this code into the new Godot 4 GDScript and I gave it internet access for that. It asked me, what code I was talking about. Then I prompted it to summarize the conversation, and it only rembers how it was supposed to summarize some non existent code.

The LLM seems to forget any previous conversation information. I asked it first about some code for Godot (game engine) without web search. Then afterwards, I asked it to convert this code into the new Godot 4 GDScript and I gave it internet access for that. It asked me, what code I was talking about. Then I prompted it to summarize the conversation, and it only rembers how it was supposed to summarize some non existent code.

This is kinda why I asked about the ability to read pdf/txt files. It seems not so good with memory retention.

Good!

Would be nice if the websearch could be used ONLY when needed. If I enable the web-search, it performs a web search on EVERY SINGLE QUERY. If I say "Hi, how are you", why does it need to perform a websearch?

@yahma We've been working on this, it'll come with the agents feature!

Websearch issues
searches on every single msg;
indifferent search: doesn't comply with the context of the chat, gets too robotic (ik, ik);
hallucinates;
connection error lots of times

IMG 01

IMG 02

bye openassistant

searches on every single msg;

Would be nice if the websearch could be used ONLY when needed. If I enable the web-search, it performs a web search on EVERY SINGLE QUERY. If I say "Hi, how are you", why does it need to perform a websearch?

At the moment. as long as the Websearch radio is checked on, it will perform websearch on every single message. However, I agree that on the next updates, we should make the LLM decide whether to use websearch or not depending on the user prompt.

indifferent search: doesn't comply with the context of the chat, gets too robotic (ik, ik);
hallucinates;
connection error lots of times

We will work on the prompting as well so that the LLM generations make more sense and follow the context of the chat better

alright! thank you very much!! :)

Hi, what is the hardware used to run the LLama model that is used in current version, to get this level of response time that can be seen on the current version ?

The immediate effect is awesome but I found an issue and that's when the model with websearch enabled keeps the context of previous questions and try to respond to them each time asked a new question and answers them first over and over , again and again! All answers are awesome but the repeating acculative response is an issue!

Hi! Is the app down today? Thanks

i had problems with it. it keeps on giveing a junk error saying network error but my network is fine.

I keep getting network error message so I can not use it.

So, frustrating

I keep getting network error message so I can not use it.

Yes sorry for the inconvenience we are looking at it.

@hopefulhappy issue should be fixed!

It is happening again.

I do not understand why this keeps happening. What am I doing wrong? Do I use it to much or something? What can I do in the future to prevent this from happening?

It says, "Network Error" and won't respond to anything I input into the chat. It happens often but usually resolves itself after a few tries. Lately, it keeps doing it though and lasting indefinently.

@hopefulhappy yeah I think more features & all should be paused temporarily & focus on current issues now.
*Network error
*Recent message disappearing suddenly & reappearing suddenly after new message, sometimes doesn't reappear.
*When web search is on, chat goes to the moon 🌚
*Another model can't be selected or given sys promt as the appy button is beyond screen resolution on some phones, can't scroll that window either.
Need to toggle desktop mode for that.
:(

Now it says it is generating without actually generating anything.

Now back to "Network error". I don't understand what is happening.

Any chances for a search API that isn't Google's? IE: DuckDuckGo or Bing :)

@DarwinAnim8or there has been updates since the last comment here. You can see this PR on how you.com is being added as a search engine. And do same for DuckDuckGo or Bing :)

Does any one has some information about infrastructure required to run such service on-prem (including LLAMA model) with a similar performance that we see here on HF page ?

Is there direct support to help with this "network error" issue? It keeps happening. Also, can anyone explain why it keeps happening?

Is this just happening to me? Repeated frequent "network errors". I don't understand why it keeps happening and I wonder if it can be fixed.

I'm afraid not. Maybe spam on GitHub too :3 XD
Edit: oh wait, that's the chat-ui there.
Also, I think it's due to overloaded server maybe.

This sucks because I am using this to help me write a complaint to the state and every time it does this I get pushed back. It is a wonderful resource but the network errors are becoming more frequent. This is so frustrating.

@SvCy I could not find a lawyer so this was the next best thing. It has really helped me in this process when I could not find another person to help. Now, it is doing this prolonging my ability to finish this. I have developmental disabilities so it has been helpful to fill in where my disbalities complicate my ability to finish this complaint.

and now the junk errors keep comeing. the error says network error when its fine!

@hopefulhappy if the problem persists & can't be resolved as of now, to get it done, I only can suggest to use ChatGPT/Bing AI. & Goodluck there!

Your HuggingFace provides the best answers and revisions. I want to donate to the project I am so grateful.

ChatGPT is good but the revisions and answers HuggingFace provides is more natural. ChatGPT can offer very robotic type replies and revisions. HuggingFace issues more human like resposnes. It is an amazing invention. Kudos to the creators and the people who maintain it. It has been a lifesaver when it does actually work. It is a great invention/AI.

Welp, that's your call which llm you'd use. See ya then! :)

to me hugging chat may be better in fluid reposes
but it is riddled with glitches

It is back at it again with "network error" messages.

@foronia I totally agree. As I previously said when it works it is better than the other AI chat bots however it rarely works. I encounter frequent "network error" glitches and it had become stressful to use it. I wish it worked consistently.

@foronia I wonder what the developers of Huggingface did differently than than the other creators of the other AI chat bots that Hugging AI produces more fluid answers? It sucks that the other AI chat bots, I never experience glitches but their AI chat bots aren't as fluid as this one. This one is great but is not always functional.

Tbh i have had 2 full weeks of junk errors i think there might be a lot of stress on the system.

It is not working at all for me. SIGH

@hopefulhappy none of the model works? Which model facing the problem more? Make a post/thread about it directly. With details & if possible screenshots ig

i believe the one sending junk errors is Llama-2-70b-chat-hf but it seems to show no reason why.

@hopefulhappy it seems to stop working at night prominently but the middle of the day is somewhat fine but you do get boughts of error days at a time sometimes

@foronia I get it during the day too. At night is worse. It wasn't like this when I first started using it. I am currently looking for an alterative. It is unstable and unreliably.

Hi can you try again? We've had some API issues recently that should now be fixed. @foronia @hopefulhappy

ok it seems to be more stable but its day for me right now so i have no assurances.

so the model says its overloaded what is the matter with that?

@nsarrazin it says its over loaded is that a problem on your side or ours?

Model shows a bit of overload on our side for Llama 2 70B, I'll bring it up internally. Feel free to retry if that occurs again. It's a button in the top right above the chat input:

So @nsarrazin it may not be extremely important but i think you misunderstoo i have had the error for about 3 days now.

Which model are you using ? It could occasionally happen (as there can be some delay between usage increase and us scaling our infra) but if you get 100% overloaded something else is wrong.

meta-llama/Llama-2-70b-chat-hf

hmm odd... hey @nsarrazin ? the redo button is immidiately just not working. you click it and it is as if you never did.

I am only getting fetch failed no matter what model i use

hey i got a 503 error what does that exactly intail? @mishig it has not worked for the past 1 hr (time 9:55 EST)

it also will not resolve

Hugging Chat is undergoing some Friday maintenance today and will be back up as soon as this is done 🛠️

when is it scheduled to be done? @julien-c

hey @nsarrazin why do i keep on getting fetch failed?

Hey @nsarrazin i am getting fetch failed everytime i use websearch

@nsarrazin

yeah seems to be down now, was working a while back too! all models.

Hi! Y'all are super fast haha, we're having some issues with the API at the moment. I'll let you know as soon as it's back up!

XD Thanks!

Should be back up! @shirohawkeye @foronia @SvCy

It's back. You rock! ✌

Hey! @nsarrazin why is the chat saying i have to pay for a subscription?

@foronia It's hallucinating I think, did you have the websearch enabled? it probably parsed a newspaper with a paywall

@nsarrazin so what i have not had the wed search enabled but its odd it comes in the form of a error message

@nsarrazin

Yep, just noticed this as well. Working on a fix, the error shouldn't be there :)

Thank you! good luck! :-) @nsarrazin

welp looks like hugging chat broke @nsarrazin

Lol nice, which model was this ? @foronia

meta-llama/Llama-2-70b-chat-hf @nsarrazin

why did this happen? @nsarrazin

this was suupsed to be a scene matching a song not random gobblygook @nsarrazin

ok no this is happening way to much

pls add support for plugins like chatgpt plugins -- thank you

1. Hey, can you add a 3 line bar on left side so it hides conversation history and other settings.

2. Also It is showing number of times as Network errorr, I know mainting an Open Source Chat App is a very tough, appreciate your work, but If this is working better, it can defenitely beat ChatGPT, as HuggingChat has better UI than ChatGPT's boring UI.

Hey, can you add a 3 line bar on left side so it hides conversation history and other settings.

Yes I think we are probably going to add it soon.

Also It is showing number of times as Network errorr, I know mainting an Open Source Chat App is a very tough, appreciate your work, but If this is working better, it can defenitely beat ChatGPT, as HuggingChat has better UI than ChatGPT's boring UI.

Thank you for the kind words, we're trying to keep improving the infrastructure of HuggingChat, it should keep getting better and better.

@victor @nsarrazin
guys falcon is missing, is it gone? please brink it back guys! falcon was awesome 😭
here is the thread #343 request. Please consider it!

Hi everyone, just a ping to share with you that a bug was fixed in the Web search and it's now working 100% of the time!

Thank you for your hard work guys!!

Hi everyone, just a ping to share with you that a bug was fixed in the Web search and it's now working 100% of the time!

Hi Victor! I'm having issues that it always performs searching in English, doesn't matter what I ask or what I put in the prompt. The thing is, when I ask some stuff in Portuguese, and it perform the search in English, it doesn't find the answer and gives a error.

Any plans to make it perform the search in multiple languages (e.g following the language of original question)?

🚨 Important update: 🌐Internet Access for Assistants, This makes some new use cases possible!