CharGPT as a research tool - Page 1
Forum section: electronics / general

author: PlainName
author_rank: Super Contributor
timestamp: April 06, 2023, 11:38:13 am
content: Article in The Grauniad this morning warning about the perils of using ChatGPT for research.Quote from:https://www.theguardian.com/commentisfree/2023/apr/06/ai-chatgpt-guardian-technology-risks-fake-articleMy emphasis.

author: AndyC_772
author_rank: Super Contributor
timestamp: April 06, 2023, 11:56:28 am
content: It still never ceases to amuse me that people cite this as a weakness specific to AI.How many of us have ever thought along the lines of "I'm sure I read about <topic> somewhere", but simply misremembered the details?ChatGPT sometimes makes stuff up. It makes mistakes. It's not even "sure" (assuming such a concept makes sense in the context of a thing that's not conscious) what is says that's real vs what's not - but the same is true of people. Especially people who are not experts on the topic being discussed, which is most of us, most of the time.

author: PlainName
author_rank: Super Contributor
timestamp: April 06, 2023, 12:07:42 pm
content: People that can't recall properly don't quote the non-existent article in a way that fools even the alleged author. And 'computer says ...' is often enough proof on its own.

author: golden_labels
author_rank: Super Contributor
timestamp: April 06, 2023, 05:12:08 pm
content: Imagine asking ageneratortool togeneratenew content and be surprised it… generated new content. So unexpected. Must be a fault in the tool.PlainName: it seems we do not need ChatGPT to fool readers, if they get confused even by human-written article. Even if the statement is slapped right in their face in plain words. The author never quoted a non-existent article and you yourself provided a paragraph, which explains they could not even recall writing one:QuoteIt merely sounded to them like a subject they would touch, which is exactly what AndyC_772 mentioned.Even without that, you would still be wrong. Human brain generates false content and treats it as true all the time.False memoriesare a thing. That includes not only simple statements, but entire complex events, vividly recalled with details, the person believing they were part of.

author: PlainName
author_rank: Super Contributor
timestamp: April 06, 2023, 05:53:32 pm
content: Quote from: golden_labels on April 06, 2023, 05:12:08 pmIt merely sounded to them like a subject they would touch, which is exactly what AndyC_772 mentioned.You must have missed: "Worried that there may have been some mistake at our end, they asked colleagues to go back through our systems to track it down." That isn't "merely sounded" in my books - they were unsure enough to try and find it. (It's not uncommon to come across something one has written previously without recalling actually writing it. Done it myself on this forum a couple of weeks back.)QuoteI pointed out that when someone created false content in their mind, they don't actually create it all ('merely', in your throwaway style, give a precis). And they certainly don't provide a URL to go with it.Quote from: Comments to the articleAnd, as I also pointed out, if someone says something it's very easy indeed to discount them. Happens all the time here, and in this very thread. If acomputersays it, you believe it. [OK, notyoubecause you would never be taken in, but in general, which means most of the population. The problems with this aren't for anyone in this topic but for thosenotof your mindset.]

author: pdenisowski
author_rank: Super Contributor
timestamp: April 06, 2023, 09:06:35 pm
content: I had the same experience the other day: asked for references on roll mode and got 5 very authoritative sounding references (including one from R&S!) that I had .... somehow ... never seen during my research. I asked for links and ChatGPT provided those too. One small problem: none of the very plausible-sounding references exist, and all of the equally plausible-looking links didn't point to anything. Kind of disturbing when ChatGPTmakes upreferences.I decided to test it by asking for something very specific which I know for sure doesn't exist. The answer (attached) was priceless

author: SiliconWizard
author_rank: Super Contributor
timestamp: April 06, 2023, 09:13:33 pm
content: Quote from: golden_labels on April 06, 2023, 05:12:08 pmIndeed.The other day I took a hammer from my garage and got confused when it couldn't even tell me what time it was.Aren't those stupid things supposed to do that already?Next thing you know, someone's gonna ask Biden a question and expect to get an articulate answer. Come on.

author: golden_labels
author_rank: Super Contributor
timestamp: April 06, 2023, 11:23:31 pm
content: Quote from: pdenisowski on April 06, 2023, 09:06:35 pmIt does not make upreferences. It makes up every single letter of its output. It does that, because generating text is the sole purpose of GPTs.(1)It models language, not knowledge or structured data.Not disturbing, if one understands the nature of these algorithms. As horrible as finding “help me” in/dev/urandom. Fear of the unknown is, what is disturbing.Quote from: SiliconWizard on April 06, 2023, 09:13:33 pmHearing 5 people tell you, that hammers are tools that apply huge force in small area and are used to drive nails, taking a hammer with “applies huge force in small area to drive nails” written on it, hitting your finger… and then writing an article about how horrible hammers are, because it did not remove your nail polish, but your finger hurts.(1)With “text” understood loosely. That may also include machine languages, and structures the model picked up as a language features from the learning data set.

author: PlainName
author_rank: Super Contributor
timestamp: April 06, 2023, 11:46:23 pm
content: QuoteObviously, most people don't understand that. Being a pedant and pointing out a screwdriver isn't a hammer is meaningless if most people are driving in nails with one.

author: golden_labels
author_rank: Super Contributor
timestamp: April 07, 2023, 01:42:04 am
content: Not, if the person may receive the notice about a screwdriver not being a hammer. Otherwise I would not waste my time on writing this — in general I try to avoid xkcd-386-ing myself.

author: Siwastaja
author_rank: Super Contributor
timestamp: April 07, 2023, 06:59:04 am
content: Quote from: AndyC_772 on April 06, 2023, 11:56:28 amYou clearly make the usual mistake. You seem to assume that ChatGPT's design goal is to provide correct information and logical thinking, and failure to do that is just "making mistakes". This is not the case. ChatGPT was never designed to do that. All it does isliterally,by definition"make stuff up", and not sometimes, but all the time. Also by definition, if the resulting text is factually or logically incorrect, it is not a "mistake", as ChatGPT does notunderstandthe concept of facts or truth, it just... makes stuff up.By statistical coincidence, the information is often correct. It is dangerous to assume this is because of intelligence.BTW, some people operate the same way, you can see them on this forum as well: they take random words from a suitable context and glue them together, producing text that looks convincing to people who have no idea about the subject. For example, look at this poster:https://www.eevblog.com/forum/microcontrollers/rant-371852/msg4798079/#msg4798079. If you rewrite that text to be grammatically correct, fix punctuation and add some weasel words, it could be ChatGPT output. And it makes zero sense.But with actual people (politicians aside),mostof us have the decency of not generate bullshit like that; we still sometimes do, but that is a mistake, not the primary modus operandi except for very few people. ChatGPT is physicallyunableto operate in such honest way most people operate most of the time.TLDR, ChatGPT is a hardwired (hence unfixable) pathological bullshitter, just with very good input data.

author: pdenisowski
author_rank: Super Contributor
timestamp: April 07, 2023, 10:47:11 am
content: Quote from: golden_labels on April 06, 2023, 11:23:31 pmExcellent observation.The danger is of course that we tend to assume that higher language competence is strongly correlated with informational accuracy. People are inclined to trust the accuracy of information that is provided in a clear, authoritative, plausible manner.A lot of humor involves saying absolutely ridiculous things with a perfectly straight face, e.g. the famous "retro-encabulator" video (or most Monty Python sketches), but it stops being funny and starts being ... concerning ... if people start believing (and acting on) the information they get from ChatGPT.

author: PlainName
author_rank: Super Contributor
timestamp: April 07, 2023, 11:05:13 am
content: I can't verify the veracity of this since it appears in the DailyMail, but I bet it's not something ChatGPT made up...ChatGPT falsely accuses a law professor of a SEX ATTACK against students during a trip to Alaska that never happened...QuoteAll very well to sit in your ivory tower to pontificate on the minutiae of ChatGPT, but people have been seriously cancelled for far less than this. Pertaining to the title of this topic, the article goes on:QuoteI accept and agree that ChatGPT is only doing what it's mean to do and that it's the users who are to blame for "using the wrong tool". However, in other instances where proles can't be trusted to use powerful tools there are regulations and even laws designed to prevent harm to, and by, the unwary.Edit: the article asks: will ChatGPT replace Google? and it occurs to me that if we were talking numbers then ChatGPT might be averaging to Google's taking the mean.