Spaces:

huggingchat

/

chat-ui

like 786

Running oncpu upgrade

I think I've asked this elsewhere but I'm not sure what the answer was. Do you use a paid API to query Google search? I'm asking because I can imagine that if it's through something hacky like selenium, Google won't like it (and they'll miss ad revenue). So, in short what does the technical pipeline look like for this from user query to generated output?

I think I've asked this elsewhere but I'm not sure what the answer was. Do you use a paid API to query Google search? I'm asking because I can imagine that if it's through something hacky like selenium, Google won't like it (and they'll miss ad revenue). So, in short what does the technical pipeline look like for this from user query to generated output?

You can check out the feature here!

@BramVanroy, they use the SerpAPI which, as far as I understand, is paid and legal. see the source code here: https://github.com/huggingface/chat-ui/blob/main/src/lib/server/websearch/searchWeb.ts

Question regarding sources/citations: Do I understand correctly that you currently display all URLs as sources, which the retriever retrieved and gave the LLM as context (regardless of whether the LLM actually used/refers to the source)? In Bing chat, they somehow managed to attribute sources to specific parts/sentences of the generated output and not only the generated output as a whole. I've always been wondering how they made that work (direct citations / attributing sources to specific sub-parts of a generated output). does anyone know?

Question regarding sources/citations: Do I understand correctly that you currently display all URLs as sources, which the retriever retrieved and gave the LLM as context

Yes

In Bing chat, they somehow managed to attribute sources to specific parts/sentences.

Interesting. I guess one simple way would be: for every generated sentence calculate its similarity against the sources and decide the highest scoring source as the source of that sentence. Not sure if that's what they are doing

Any chances for a search API that isn't Google's? IE: DuckDuckGo or Bing :)
Awesome work though! Really cool :D

In Bing chat, they somehow managed to attribute sources to specific parts/sentences.

Interesting. I guess one simple way would be: for every generated sentence calculate its similarity against the sources and decide the highest scoring source as the source of that sentence. Not sure if that's what they are doing

@mishig, yeah true. My first intuition was that I would be hesitant to trust embeddings from a bi-encoder to be reliable enough for this. But if you took a cross encoder, that should actually work quite well. especially if you set a high enough threshold to avoid false positives (then it could even work with a bi-encoder sentence transformer). could be a nice feature to have direct citations :)

@mishig first of all thanks for working on the project and making this feature better for everyone , the interface looks like perplexity AI and it was better then i thought , i have been having this issues where it just searches something but then never shows me the answer it shows max tokens
i just keep seeing this
first i see the links of the resources

but then it does nothing at all

should i create the issue on Github

but then never shows me the answer it shows max tokens

Nice catch, never had this.

Any chances for a search API that isn't Google's? IE: DuckDuckGo or Bing :)

Yes that would be cool if someone wants to PR https://github.com/huggingface/chat-ui

I tried Llama 2 70B with web search. I asked it about myself and what is my expertise and that it shall include linkedin in its research. It was a complete fail. I tried it three times and it always failed. It didn't search linkedin and it made things up. It said that I'm a medicine professor although I'm a software architect. For many years I had been a freelance developer, you get countless results if you google my name and I have a detailed public profile on linkedin. One answer had search hits on Xing where I also have profile, but it was a different person. Something is going terribly wrong with the web search.

that it shall include linkedin in its research.

Yes, the content of LinkedIn pages is not parsed by the web search. For the Xing page the rendering of users informations seems to happens client-side so it wont work.

It hallucinates very hard with anything scholarly, sources are almost non existent (The sources provided do not match with the text bot gives) It should be better with integrating pubmed etc free engines to web search.

It hallucinates very hard with anything scholarly, sources are almost non existent (The sources provided do not match with the text bot gives) It should be better with integrating pubmed etc free engines to web search.

@zelosleone I see. Yes right now we are just performing simple google search and picking the top 10 organic results. But as you have said, it is prone to hallucinating on scholarly topics. As a next step, what we can potentially do is: a routing mechanism (i.e. classifier) that classified user search query to an appropriate search engine (pubmed for biomed, arxiv for ML, etc.)

Any chances for a search API that isn't Google's? IE: DuckDuckGo or Bing :)

As Victor said, @DarwinAnim8or not at the moment. But it shouldn't be too hard to abstract away search engine interface and let the user plugin the choice of engine.
Specifically, you can start looking at this part of the codebase: https://github.com/huggingface/chat-ui/blob/main/src/lib/server/websearch/searchWeb.ts

@bilalazhar50 I see that you are getting the max new tokens error. Did it happen on the first try or was it a continuation of a long existing chat (in which case, you hit the limit of max new tokens) ?

I tried Llama 2 70B with web search. I asked it about myself and what is my expertise and that it shall include linkedin in its research. It was a complete fail

@krassmann atm, linkedin has a mechanism against scraping, which is why your linkedin was not included in the context. However, we will keep iteratively improving the feature 🤗

Any chances for a search API that isn't Google's? IE: DuckDuckGo or Bing :)
Awesome work though! Really cool :D

I worked on a simple and similar chatbot and I found DuckDuckGo doesn't provide links as Google Search API does. I managed to extract the "title" of the 5 organic results, extract the "snippet text" also concatenate it and send to LLM (using Openai functions). I would like to improve it.

Will you be adding a feature to upload files like pdfs or txt files?

Will you be adding a feature to upload files like pdfs or txt files?

yes

Will you be adding a feature to upload files like pdfs or txt files?

I have an open issue and PR for PDF support, but more specially PDF support with an OCR integration to handle math equation and table. This leverage the Mathpix private API so this feature is not for everyone and will likely not be include in the public version of chat-ui. But at least you can try to copy my code and deploy an instance of chat-ui with this. Maybe this will be include as an optional feature in the future.

Mathpix is currently the only viable OCR solution I know, I don't know any open source solution that could produce such result at the time.

However 90% of the user don't need advanced OCR features and a simple text extraction is enouth, Claude and Perplexity PDF feature work this way and that already quite impressive.

@mishig Did you have more info about the current work on this feature ? I don't see anything on GH expect my issue and PR

Any chances for a search API that isn't Google's? IE: DuckDuckGo or Bing :)

Yes that would be cool if someone wants to PR https://github.com/huggingface/chat-ui

Look like a good idea, if I find the time soon I might try to do this

@blanchon have you checked https://facebookresearch.github.io/nougat/ yet ?

Any chances for a search API that isn't Google's? IE: DuckDuckGo or Bing :)

Yes that would be cool if someone wants to PR https://github.com/huggingface/chat-ui

Look like a good idea, if I find the time soon I might try to do this

https://github.com/huggingface/chat-ui/issues/338#issuecomment-1631135948

Maybe this can help

The LLM seems to forget any previous conversation information. I asked it first about some code for Godot (game engine) without web search. Then afterwards, I asked it to convert this code into the new Godot 4 GDScript and I gave it internet access for that. It asked me, what code I was talking about. Then I prompted it to summarize the conversation, and it only rembers how it was supposed to summarize some non existent code.

The LLM seems to forget any previous conversation information. I asked it first about some code for Godot (game engine) without web search. Then afterwards, I asked it to convert this code into the new Godot 4 GDScript and I gave it internet access for that. It asked me, what code I was talking about. Then I prompted it to summarize the conversation, and it only rembers how it was supposed to summarize some non existent code.

This is kinda why I asked about the ability to read pdf/txt files. It seems not so good with memory retention.

Good!

Would be nice if the websearch could be used ONLY when needed. If I enable the web-search, it performs a web search on EVERY SINGLE QUERY. If I say "Hi, how are you", why does it need to perform a websearch?

@yahma We've been working on this, it'll come with the agents feature!

Websearch issues
searches on every single msg;
indifferent search: doesn't comply with the context of the chat, gets too robotic (ik, ik);
hallucinates;
connection error lots of times

IMG 01

IMG 02

bye openassistant

searches on every single msg;

Would be nice if the websearch could be used ONLY when needed. If I enable the web-search, it performs a web search on EVERY SINGLE QUERY. If I say "Hi, how are you", why does it need to perform a websearch?

At the moment. as long as the Websearch radio is checked on, it will perform websearch on every single message. However, I agree that on the next updates, we should make the LLM decide whether to use websearch or not depending on the user prompt.

indifferent search: doesn't comply with the context of the chat, gets too robotic (ik, ik);
hallucinates;
connection error lots of times

We will work on the prompting as well so that the LLM generations make more sense and follow the context of the chat better

alright! thank you very much!! :)

Hi, what is the hardware used to run the LLama model that is used in current version, to get this level of response time that can be seen on the current version ?

The immediate effect is awesome but I found an issue and that's when the model with websearch enabled keeps the context of previous questions and try to respond to them each time asked a new question and answers them first over and over , again and again! All answers are awesome but the repeating acculative response is an issue!

Hi! Is the app down today? Thanks

i had problems with it. it keeps on giveing a junk error saying network error but my network is fine.

I keep getting network error message so I can not use it.

So, frustrating

I keep getting network error message so I can not use it.

Yes sorry for the inconvenience we are looking at it.

@hopefulhappy issue should be fixed!

It is happening again.

I do not understand why this keeps happening. What am I doing wrong? Do I use it to much or something? What can I do in the future to prevent this from happening?

It says, "Network Error" and won't respond to anything I input into the chat. It happens often but usually resolves itself after a few tries. Lately, it keeps doing it though and lasting indefinently.

@hopefulhappy yeah I think more features & all should be paused temporarily & focus on current issues now.
*Network error
*Recent message disappearing suddenly & reappearing suddenly after new message, sometimes doesn't reappear.
*When web search is on, chat goes to the moon 🌚
*Another model can't be selected or given sys promt as the appy button is beyond screen resolution on some phones, can't scroll that window either.
Need to toggle desktop mode for that.
:(

Now it says it is generating without actually generating anything.

Now back to "Network error". I don't understand what is happening.

Any chances for a search API that isn't Google's? IE: DuckDuckGo or Bing :)

@DarwinAnim8or there has been updates since the last comment here. You can see this PR on how you.com is being added as a search engine. And do same for DuckDuckGo or Bing :)

Does any one has some information about infrastructure required to run such service on-prem (including LLAMA model) with a similar performance that we see here on HF page ?

Is there direct support to help with this "network error" issue? It keeps happening. Also, can anyone explain why it keeps happening?

Is this just happening to me? Repeated frequent "network errors". I don't understand why it keeps happening and I wonder if it can be fixed.

I'm afraid not. Maybe spam on GitHub too :3 XD
Edit: oh wait, that's the chat-ui there.
Also, I think it's due to overloaded server maybe.

This sucks because I am using this to help me write a complaint to the state and every time it does this I get pushed back. It is a wonderful resource but the network errors are becoming more frequent. This is so frustrating.

@SvCy I could not find a lawyer so this was the next best thing. It has really helped me in this process when I could not find another person to help. Now, it is doing this prolonging my ability to finish this. I have developmental disabilities so it has been helpful to fill in where my disbalities complicate my ability to finish this complaint.

and now the junk errors keep comeing. the error says network error when its fine!

@hopefulhappy if the problem persists & can't be resolved as of now, to get it done, I only can suggest to use ChatGPT/Bing AI. & Goodluck there!

Your HuggingFace provides the best answers and revisions. I want to donate to the project I am so grateful.

ChatGPT is good but the revisions and answers HuggingFace provides is more natural. ChatGPT can offer very robotic type replies and revisions. HuggingFace issues more human like resposnes. It is an amazing invention. Kudos to the creators and the people who maintain it. It has been a lifesaver when it does actually work. It is a great invention/AI.

Welp, that's your call which llm you'd use. See ya then! :)

to me hugging chat may be better in fluid reposes
but it is riddled with glitches

It is back at it again with "network error" messages.

@foronia I totally agree. As I previously said when it works it is better than the other AI chat bots however it rarely works. I encounter frequent "network error" glitches and it had become stressful to use it. I wish it worked consistently.

@foronia I wonder what the developers of Huggingface did differently than than the other creators of the other AI chat bots that Hugging AI produces more fluid answers? It sucks that the other AI chat bots, I never experience glitches but their AI chat bots aren't as fluid as this one. This one is great but is not always functional.