AI coding tools make developers slower but they think they're faster, study find - Page 1
Forum section: electronics / general

author: Karel
author_rank: Super Contributor
timestamp: July 13, 2025, 02:19:34 pm
content: Predicted a 24% boost, but clocked a 19% dragArtificial intelligence coding tools are supposed to make software development faster,but researchers who tested these tools in a randomized, controlled trial found the opposite.https://www.theregister.com/2025/07/11/ai_code_tools_slow_down/

author: John Coloccia
author_rank: Super Contributor
timestamp: July 13, 2025, 10:59:47 pm
content: There are also studies that clearly show flat UI design is measureably worse in every way, and not by a little, but no one seems to care about that either. Full steam ahead.

author: coppercone2
author_rank: Super Contributor
timestamp: July 14, 2025, 12:00:11 am
content: sounds like the same observation made about caffeine

author: xrunner
author_rank: Super Contributor
timestamp: July 14, 2025, 12:42:36 am
content: I've been listening to the AI experts a whole lot over the last weeks, considering the predictions of AGI within several years and the loss of jobs. But I hadn't heard of the term "Vibe Coding" until a few days ago. Have any of you used that method and does it actually give a polished product without human tweaking the old fashioned way?

author: John Coloccia
author_rank: Super Contributor
timestamp: July 14, 2025, 01:17:28 am
content: Quote from: xrunner on July 14, 2025, 12:42:36 amLike anything else, there are some problem domains where the AI can be super helpful, and some where it's worse than worthless.Polished with no intervention? I don't think that's real yet.I've actually had great luck using Ai to point me in the right direction when a datasheet is either really bad, or just not very complete, for example. Or like, how can I do this specific thing in Bash, instead of trying to figure out the right combination of things that get me there. It's usually helpful and points me in the right direction so I can fill in the real details. Sort of like a stack overflow but without all the aggravating personalities. Points me in the right direction, but also often just wrong or not helpful.Its quite good at tossing together webpages, I'm told.If you're curious, give Zed a try. I found it incredibly aggravating and not super useful, but I assume it's because my problem domain is just not it's strength at the moment.

author: RoGeorge
author_rank: Super Contributor
timestamp: July 14, 2025, 07:42:41 am
content: Another research about AI helpers:QuoteQuoted fromhttps://www.media.mit.edu/publications/your-brain-on-chatgpt/The research results are no surprise. The same reason why the brain volume apparently has decreased in the latest thousands of years (because of self-domestication, a very hated idea in the mainstream, seen such a video on YouTube, by an archeologist, the video was removedhttps://www.eevblog.com/forum/chat/fun-for-nerds/msg5638067/#msg5638067)I think that is no different than the "Use it or lose it" saying about biology (or about evolution in general).To give a more recent example, anybody remember when the GPS navigation appeared in cars for the first time? The GPS makes you dumb. With GPS, you'll arrive to an unknown destination faster, but you'll arrive disoriented, and without learning the way.Using an AI helper is no different than driving by GPS, instead of driving following the road signs. With a GPS you might arrive faster to an unknown destination, but you'll learn less, and with time you'll became addicted to that technology.OK, but is it worth it? I don't know. To stay with the sayings "There is no such thing as a free meal".

author: tom66
author_rank: Super Contributor
timestamp: July 14, 2025, 09:06:18 am
content: Vibe coding can work for small projects where the model can synthesise an output reliably from training data. Once it has to start going outside of that region of expertise and come up with novel engineered solutions, it all falls apart. I used 'vibe coding' to create a very simple Python template for a file upload utility. This got me to the stage of getting a filename from a callback and a file selection dialog. That saved me a fair bit of time. I then asked it (purely out of curiosity) to upload this file via UART using a defined pattern that I described in two paragraphs, and it fell apart completely. No surprise there because that won't exist in the training for the model.I still use tools like ChatGPT nearly daily to help me with problems and act as the proverbial rubber duck whenever I'm having difficulties getting something right. In that sense, it's like a really good Stack Overflow. Just like SO, there are "hallucinations" (SO equivalent: wrong/outdated/irrelevant answers). But I would say 95% of the time it is better at answering the question than Google and SO, and provided you apply a modicum of sense to the answer, it is a huge boost to productivity.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 14, 2025, 09:14:21 am
content: "Vibe coding" is a huge trap because it is completely useless technique to those whocanprogram, and because it completely prevents any learning for those whocan'tprogram. The only case this makes sense is when in urgent need of something fairly common and simple outside of one's expertise without any intention or desire to learn about it anyway. E.g., in totally foreign programming language, but still needed to keep you focused in the main line of your work."Generate - manually check and adjust" cycle is also the wrong way. LLM can be useful for checking, and as a "better Google" in absence of good documentation for external libraries/etc., which is too often the case.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 14, 2025, 02:01:27 pm
content: This is such an anti-engineering pattern that it's almost funny.Fortunately (or not), it mainly applies to software, at least for now.In fact, i's been debated for decades now whether software development was really engineering, and thanks to this demented use of LLMs, we may finally have the answer.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 14, 2025, 02:13:52 pm
content: I guess the main issue is that people understand theneedof being critical, but they stop there. We have been repeating this mantra: "oh AI is not perfect at all, so we need to proof-read/verify the output manually and adjust it". But this is totally wrong, because this process of generate - check - adjust is just so slow and inefficient, and it leads to either serious drop in productivity (redoing the work), or serious drop in quality (leave out the check - adjust phase since it's so tedious). Instead of this mantra, we need tocorrectlyrecognize the strengths and weaknesses, and that's exactly what we don't currently do with LLMs.This, BTW, is not a new phenomenon at all. Same happens with real people; get an unsuitable person to do the work, and others need to spend much more time fixing it than it would have taken to do it from scratch yourself. Happens in physical work as well as software. With real people, we accept this happens for a freshman, and expect they learn pretty quickly and turn into creative individuals, often within weeks.With AI, no one expects improvement, we accept whatever crap and instead adjust our workflows toprovablyless efficient ones, while shouting the mantra: "Look, I'm proof-reading the AI and fixing its outputs here, isn't this great, look how much productivity improvement I got!"And in some cases, we just don't bother - we skip the proofreading and let it hallucinate outputs that are wrong.

author: amyk
author_rank: Super Contributor
timestamp: July 17, 2025, 03:19:03 am
content: Quote from: SiliconWizard on July 14, 2025, 02:01:27 pmUnfortunately...https://atomic14.substack.com/p/vibe-coding-a-pcb-surprisingly-goodI don't agree with his conclusion, but I suspect there will be tons of cheap tat from China produced this way before long.

author: John Coloccia
author_rank: Super Contributor
timestamp: July 17, 2025, 03:49:39 am
content: Quote from: amyk on July 17, 2025, 03:19:03 amThe dancing bear doesn't have to dance well to be impressive. It's impressive it dances at all.It's early. Somewhere between today, and StarTrek level computers, is something useful.But the real world is:Me: "Computer...tea. Earl Grey. Hot"Computer: "Do you suffer from irritable bowel syndrome? Skip add in 56 seconds...."I'll make my own damn tea, thanks.

author: RoGeorge
author_rank: Super Contributor
timestamp: July 17, 2025, 09:16:56 am
content: Quote from: John Coloccia on July 17, 2025, 03:49:39 amFits like a glove with the current state of AI!Wasn't the one with the hot tea about namespaces?

author: paulca
author_rank: Super Contributor
timestamp: July 17, 2025, 09:33:39 am
content: An observation.When you are "coding" with an LLM and the code you get given looks like "Someone elses code".That's because it is.Put a bit of effort in teaching the AI how your code looks and how you solve problems. Teach it everything you already know about the particular problem.Now when you code with it, the code will start to look more like "your code".This is when you are in the sweet spot where you are not channelling it, it is channelling you.

author: tggzzz
author_rank: Super Contributor
timestamp: July 17, 2025, 10:52:06 am
content: Quote from: paulca on July 17, 2025, 09:33:39 amDifficult if - as in my case - you are rarely solving the same application domain problem twice. Plus I tend to use whatever language is appropriate for my current task.If you are repeating the same problem many times, maybe your job is ripe for replacement by automation, LLM or otherwise.

author: John Coloccia
author_rank: Super Contributor
timestamp: July 17, 2025, 12:17:56 pm
content: Quote from: RoGeorge on July 17, 2025, 09:16:56 am

author: peter-h
author_rank: Super Contributor
timestamp: July 17, 2025, 01:07:22 pm
content: AI, as in the ChatGPT stuff, produces mostly BS, but so does google. So the Q is: is it better than google. And it certainly is a "better google". It digs up a lot more stuff than google does - presumably because it digs much deeper. Nearly all google hits are garbage and we are all very skilled in picking out the useful stuff.AI, as in e.g. Claude, for looking for software, is a mixed bag. Claude is believed to use mostly Github (and much of that is unfinished/dud projects which worked largely by luck, in some narrow context like a 16MHz AVR and nothing faster, even if they did work) but I am sure it also digs deep into Usenet archives and there is a vast amount of good stuff there. If you are looking for algorithms, Claude works pretty well. You obviously have to test the code it gave you but you have to do that anyway.Whether the "Claude + testing" workflow is faster than writing from scratch, will depend... It could well be slower. If you are clueless, it will be slow, but then if you are clueless, your productivity will be crap anywayThe other day I was looking for a rotated rectangle algorithm. It came up with one in seconds, full of typedefs of typedefs and structs of structsSo I ask for one without any structs. It came back with one in seconds. And it worked. I then asked for a filled rotated rectangle, which is about 10x more code. It came back with one in seconds. I moved to something else so didn't test it but it probably works. But I could have written either one myself, but why not pick up something which was probably posted in comp.arch.algorithms (or whatever) in 1989That is what I would have done anyway. There are many incredibly bright people out there (especially in algorithms) who I will never approach.But is that AI? Not really. It is mostly a much improved google.I still don't get how it "gets" that I don't want millions of structs and just want simple C code. Probably both versions are out there?

author: John Coloccia
author_rank: Super Contributor
timestamp: July 17, 2025, 01:45:37 pm
content: FWIW, and keeping in mind that I don't ask it to code anything for me so I can't speak to that, I've found DeepSeek to be the best at giving me reasonable guidance to a technical question.question: in c# how do I create a class using reflectionresponse:If you just need to create an instance of an existing class (not creating a new class type):Code:// Get the typeType type = typeof(MyExistingClass); // or Type.GetType("Namespace.MyExistingClass")// Create an instanceobject instance = Activator.CreateInstance(type);// If the class has parameters in constructorobject instanceWithParams = Activator.CreateInstance(type, arg1, arg2);Which is completely correct and succinct. I've found it's a real timesaver, even if all it does is give me the name of a thing that I can then lookup. Half the battle, sometimes, is you know what you want, you know it should exist, but you don't know what the thing is called. Like, if you don't happen to know they decided to call it "Activator", it can sometimes be a real headache to even figure out what you should be looking for.

author: paulca
author_rank: Super Contributor
timestamp: July 17, 2025, 04:04:21 pm
content: "Remind me how to do that thing I should remember again".Is a brililant use case. It's just a better google. However you can then follow up with it and drill into finer points, unlike google where each serve is atomic more or less.Also. It found this bug for me, after I stared at it for a while.Code:loop:wait_rx_ready:IN A, (DART_COMM_CHANNELA) ; status registerBIT DART_REG_RD0_RXCHAR_AVAILABLE_BIT, AJR Z, wait_tx_ready ;<----- Wrong label!IN A, (DART_DATA_CHANNELA)LD D, Await_tx_ready:IN A, (DART_COMM_CHANNELA) ; status registerBIT DART_REG_RD0_TX_BUFFER_EMPTY_BIT, A ; Tx Buffer Empty bitJR Z, wait_tx_readyLD A, DOUT (DART_DATA_CHANNELA), A ; write to data registerJR loopAlthough... prior it had given me code to write the config registers which was just daft. You have to write the register ID first. I added that myself. Then fixed it's config. Not it's fault, not even I can read the datasheets that exist today. They look like they have been faxed about 15 times.

author: paulca
author_rank: Super Contributor
timestamp: July 17, 2025, 11:46:09 pm
content: Couldn't put it better myself:QuoteLLM can contain FACTA and FACTH1231B. A human stumbles on an association and queries. NEW information is formed. New ideas. New concepts.This is how Faraday did what he did. There is nothing new. It's just faster and more accessible.What took years, decades, now takes milliseconds to echo out to to the knowledge base and back.If you use it right and understand it.It is still full of the same noise and false signals. It's up to YOU if you want to "create" new... those signals might not be "false" at all.

author: Whales
author_rank: Super Contributor
timestamp: July 18, 2025, 12:15:33 am
content: Quote from: paulca on July 17, 2025, 11:46:09 pmSo it's a google search?

author: John Coloccia
author_rank: Super Contributor
timestamp: July 18, 2025, 12:47:05 am
content: Quote from: Whales on July 18, 2025, 12:15:33 amNo. That's why it's different and not a Google search, and pretty much no one equates the two. Not even Google.

author: hans
author_rank: Super Contributor
timestamp: July 19, 2025, 08:26:35 am
content: Quote from: Siwastaja on July 14, 2025, 09:14:21 amYep. Or as I like to say it: reading code is harder than writing it.I'm specifically not going to add good/bad qualifiers in that statement. There is some nuance of course. But I still think in general it holds up.Isn't it a completely cliche that a new programmer joins a team, inherits a legacy codebase and is tasked to make modifications, only to then turn back and say they want to rewrite major parts. "I wasn't there when this decision was made, and I don't like it!"How is this then different for AI? Is this a too pragmatic "but it works right" approach? Or are new programmers not at least bit passionate about the solving puzzles in coding anymore?It sounds a bit hypocritical from me to say this, as I've got some ChatGPT code in my codebase. But its very restricted and things I don't really care about how they work for now. I just want to have something running and care about it later. This choice was pretty much forced when I started googling for the manual ways of doing things (antialiased graphics), and most Stackexchange answers were more busy with solving XY problems instead of answering OP's real question (e.g. "why don't you use OpenGL" or "use at least a raspberry pi, linux much better at this")

author: Analog Kid
author_rank: Super Contributor
timestamp: July 19, 2025, 05:52:12 pm
content: Quote from: hans on July 19, 2025, 08:26:35 amYep. That, my friend, is actually quite a profound statement.Which is why it's so important to adequately comment your code.And that's not just for someone else reading it; it's for the future you, 6 months, 2 years hence, who looks at it and says "what the hell was I trying to do here???".

author: paulca
author_rank: Super Contributor
timestamp: July 19, 2025, 07:54:42 pm
content: Quote from: Analog Kid on July 19, 2025, 05:52:12 pmIt is also basic ethics in the profession. "Code Complete" is an entire book about how to write readable, maintainable "complete" code. Emphasis on it not being complete until it is as readable and understandable as possible. Nobody like "code golfers" anymore. Te whole point of higher level languages is to describe problems TO HUMANS. The compiler does the "for machine" part.Some old bad habits are resurfacing from old long dead languages though.getThing().changeThingToObject().changeObject().convertToFoobar().apply( barFoo );Has to be stopped before it gets any worse.F____g Javascript ... I blame JS. Doesn't have to be JS's fault, I still blame JS.

author: John Coloccia
author_rank: Super Contributor
timestamp: July 19, 2025, 08:43:27 pm
content: Quote from: paulca on July 19, 2025, 07:54:42 pmThis is very Java like too, even though they're completely different languages. Functional languages tend to be like this too. They can be really pretty for, like, a significant portion of the problem, but there's this corner that is just incredibly ugly, verbose and just downright stupid, and it just kind of ruins it. You shouldn't have to feel like a witch doctor, making magical incantations, just to do a simple task.It's a pet peeve of mine for software to be difficult to read and understand. Even C can be expressive if you take the time to worry about names and organization. Our software tends to be mind numbingly simple to understand and work on. We agonize over things like "what should we call this thing?" It makes a measurable, long term difference in quality and productivity if your patterns are consistent, your style is consistent, and your names make sense and are consistent.

author: vad
author_rank: Frequent Contributor
timestamp: July 19, 2025, 08:45:33 pm
content: I would argue the research article is BS.1) It ignores how AI is actually used in real world software development. Most of the productivity gains from AI in coding come from automating boring tasks like generating unit tests and boilerplate. But in this study, they specifically did not let AI help with that. All the unit tests were written manually, which defeats a big part of the point of using AI.2) Participant bias wasn’t addressed. People have strong opinions about AI, some love it, some hate it. The study did not do much to control for that.3) They used experienced devs but unexperienced AI users. Many of them hadn’t used the AI tools much. Half had never used Cursor, which was one of the tools being tested.4) Statistical significance is questionable. Only 16 people took part. That’s not enough to smooth out differences in individual skill or bias.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 20, 2025, 06:22:34 am
content: Quote from: vad on July 19, 2025, 08:45:33 pmGeez, just don't do unit tests, you don't need to. The whole point of test-driven development is to put your thinking, your design into the tests. Only that way you get full benefit out of the tests. Sounds like for you testing is just "ticking a box".If you let an inexperienced trainee, or an AI, to "just write" the "boring" tests so that they just pass against the pre-existing DUT, you get tests that always pass and do not detect the mistakes in the DUT - we have exactly seen this. Wasted time, plus false sense of having a "tested high quality product". Really the only thing this kind of low-cost botched "testing" protects against is most obvious forms of regressions, which, arguably, could be better than nothing.In general I think the article used the "AI" exactly the way it's most marketed for, so a fair comparison. I also don't understand your idea why one should cherry-pick "AI-experienced" developers - that would distort the results and create a bias. Random sampling is the way to go, but I agree that the set size is too small.

author: John Coloccia
author_rank: Super Contributor
timestamp: August 22, 2025, 03:42:40 am
content: Just going to say that DeepSeek came through again yesterday. I'm trying to do c# scripting for a project, and my major problem was that you pass things in as basically a gigantic string. And I'm envisioning things coming in from different files. So now my logging depends on an attribute that gives me the file name, but what file? It's just a big string....multiple files. There are some very ugly, heavy handed solutions to this, but they all pretty much suck.So what the heck, let's try AI.After about 20 minutes of DeepSeek time, and piles upon piles of code it gave me that is:a) brokenb) doesn't really solve the problemc) just way too heavy handed for my approach, and I would never do it (I'd already discovered and rejected most of these)...I was starting to really doubt my approach. If logging doesn't easily and ergonomically work, then the approach doesn't work. Sucks to die on this hill, but we're not willing to compromise this.And then I was like, "But I really just want to do THIS..."And it spit out a solution that was absolutely dead on right. It spit out a lot of code I would never use, because it was crap. But it gave me the very simple compiler directive that I had no idea existed.So it's really like a supercharged search engine because it can somehow consider what we've talked about previously, and I can then correct it and guide it towards what I really want. And then it finds the thing somewhere that actually solves my problem. It spits out a pile of unusable garbage in the meantime, but I searched for a solution on and off for the last 2 weeks and came up blank. It found it. Now that I know where to look, it seems obvious to look there, and totally reasonable that such a feature would exist, but I didn't even know what I was looking for.IMHO, this is really where a lot of the utility is right now, and you're foolish if you don't use it. I don't need it to write code. It's SUPER helpful for esoteric things like this when I don't even know what I'm looking for. I just have a problem. Give me some hint what kinds of things intersect my problem, and I'll take it from there.

author: RJSV
author_rank: Super Contributor
timestamp: August 22, 2025, 04:57:52 am
content: The frivolous stuff might better be used for entertainment, while focus on improving the performance of critical functions (like controlling a big power grid).Today, a relative played for me, an A.I. generated wedding announcement song, of course with everyone's first name. I think it was themed on a popular drama or reality show.Error-prone, or silly creations should stay in that category...

author: paulca
author_rank: Super Contributor
timestamp: August 22, 2025, 09:30:37 am
content: You could have just install an open source log server.

author: tggzzz
author_rank: Super Contributor
timestamp: August 22, 2025, 10:11:50 am
content: Quote from: John Coloccia on August 22, 2025, 03:42:40 amIt sounds close to gambling where:the occasional random "win" is remembered and talked aboutthe frequent "losses" are ignored and not discussedthe circumstances that lead to a win are presumed to be thereasonfor the win (post hoc ergo propter hoc)if the circumstances don't lead to the expected win next, then you didn't follow the recipe/incantation correctly (you should have put the eye of newt in <3s after the bat's tongue)

author: John Coloccia
author_rank: Super Contributor
timestamp: August 22, 2025, 12:21:09 pm
content: Quote from: tggzzz on August 22, 2025, 10:11:50 amI find the information I need nearly every time I use it, quickly and easily. That sounds like gambling?Some tools require a little brainpower to use effectively. "OMG, it didn't read my mind and give me instant answers! This is soooo hard...."

author: tggzzz
author_rank: Super Contributor
timestamp: August 22, 2025, 01:37:56 pm
content: Quote from: John Coloccia on August 22, 2025, 12:21:09 pmSince you have already forgotten your misses...Quote from: John Coloccia on August 22, 2025, 03:42:40 am... yes, it does seem like the way gamblers remember their successful bets and forget the unsuccessful bets!What are the conditions under which AI hallucinations really are better than searches?

author: John Coloccia
author_rank: Super Contributor
timestamp: August 22, 2025, 02:27:39 pm
content: Quote from: tggzzz on August 22, 2025, 01:37:56 pmWhen you search for a solution, the first 10 proposed solution that aren't helpful don't hurt you anymore than any other kind of search that doesn't work instantly and perfectly the first time. The final idea that is useful helps you. And if it reduces the time it takes to get to the final idea, then that seems like an overall win.So in fact it doesn't resemble gambling, and with due respect that's the dumbest analogy I've heard in a long time. Are search engines like gambling? LOL.The only people I hear say such things are people that are completely ignorant of the capabilities, and how to use them effectively. It's no better than the people that think AI is taking over the world and there won't be any engineering jobs left, musicians will be obsolete, etc. There's a complete lack of understanding for them too.I'd encourage everyone to give it a try, especially if you're stuck on a problem. Very likely, someone else was stuck too and/or there's some piece of documentation somewhere that can help. And AI is silly good at finding it. Of course you want to double check it, just like you would from any other source, but that's trivial once you know where to look.

author: tggzzz
author_rank: Super Contributor
timestamp: August 22, 2025, 02:34:32 pm
content: Quote from: John Coloccia on August 22, 2025, 02:27:39 pmLLM slop and LLM hallucinations are a real problem.Yes, LLMs can help in some circumstances, but in generalyou have to be an expert to be able to spot what is subtle rubbish. Too many peopleimaginethey are sufficiently capable.

author: jpanhalt
author_rank: Super Contributor
timestamp: August 22, 2025, 02:39:58 pm
content: I have been using AI (mostly Grok) quite often recently and date back to the days (mid-1950's) when one buried themselves in a library, particularly a university one, if available) and used various indexes to find answers: Chem Abstract, Beilstein, and Index Medicus to name some.Not everything you hit on was relevant to the problem you were trying to solve, but we presumably knew enough to separate the wheat from the chaff. Same goes for looking up a word that you just can't recall at the moment. You know words that mean similar, so you go to a thesaurus, and find the exact word you want.That's how I interpret John Coloccia's comment. It's not random as gambling is implied to be. It's an intelligent sort of related materials.

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 22, 2025, 02:52:53 pm
content: Quote from: tggzzz on August 22, 2025, 02:34:32 pmYes, the problem is that they tend to exacerbate the Dunning-Kruger effect. That was already the case with the "googling everything" trend, but it has gotten way worse with LLMs.I have done some decent testing of LLMs and what you realize, past the initial "wow" effect, is that they'll make you waste a lot more time than what they can save, and so, as with any tool, your best bet is to objectively evaluate them rather than just getting a fully subjective impression.I agree with tggzzz about the "gambling" aspect. Contrary to, say, a dictionary, where you know you'll find the exact spelling and definition of a given word you're looking for, LLMs give you absolutely zero guarantee of anything. Maybe it'll be interesting, maybe it'll be BS, maybe it'll be completely wrong. You can't anticipate it, so yes, that's gambling with knowledge, and in case people use them for generating stuff that they'll directly use for their work, that's gambling with their job and that of their coworkers.Imagine buying a calculator that would maybe give you the right output sometimes, not always, but hey, it would compute extremely fast very complex expressions, so that would be something. That's pretty much the same thing, and nobody in their right mind - normally - would go for that.But as we say on a regular basis, even the founders of these "AI" companies never really claim otherwise. That's a pretty, uh, uncommon situation in the history of both knowledge and business.

author: John Coloccia
author_rank: Super Contributor
timestamp: August 22, 2025, 02:59:31 pm
content: Quote from: tggzzz on August 22, 2025, 02:34:32 pmWell, that's the thing, right? Even when it spits out nonsense, it's very well formatted, grammatically correct nonsense. Yeah, you totally need to have the brain working. I really wish I could post the interaction I just had with it. I'll post my prompts at least so you see how easy it was, and I tell you I really looked for a solution on my own. I just didn't know the feature existed, so I didn't even know what to ask for.notice I either typo'd or didn't phrase the second sentence well. It still figured out what I was trying to sayQuoteGot some very simple solutions (which I already discovered but wouldn't work because I need multiple files), and some heavy handed ones which I wasn't really interested in doing. Suggested I maybe implement my own attribute for parameters, which I've never done. So sure, tell me more.QuoteGave me a pile of code for basically extending the compiler. Meh....there's got to be a better way.It also gave me an alternative approach, which would never work for multiple files. Aha! I never told it that. I should let it know why it's not helpful.QuoteGave me a bunch of solutions that work with multiple files, but still just individually. Basically, the same single file solutions, just with code to automate running from a list of scripts. Aha! It doesn't get that I need to combine multiple files into one large string, but that I need to track line/file information. I should tell it.Quoteeasy: use the "#line" directive.:::HEADSLAP::: Yeah, dummy, just use the line directive. I'm sure other people have found this with simple google searches, but it was evading me. And this is what the AI is ridiculously good at. It comes back with things that aren't helpful or perfect, but that starts sending you down a path and you can REFINE it, instead of just starting over with a new search.

author: tggzzz
author_rank: Super Contributor
timestamp: August 22, 2025, 03:14:33 pm
content: Quote from: jpanhalt on August 22, 2025, 02:39:58 pmThe interesting forms of gambling are not random.

author: tggzzz
author_rank: Super Contributor
timestamp: August 22, 2025, 03:36:07 pm
content: Quote from: SiliconWizard on August 22, 2025, 02:52:53 pmWorse: uninvolved strangerslives. ConsiderAI generated case law, which can be completely fabricated: career limiting for the lawyerandthe accusedmedical triage assessments, deciding whether you are worth treating, based on fonts(!) and the subject's positionliberty, deciding whether you should be incarcerated because you might abscond or reoffend, based on race and otherunknowablefactorsQuoteYou don't have to imagine it! Microsoft has invented is and is aggressively marketing ithttps://defector.com/it-took-many-years-and-billions-of-dollars-but-microsoft-finally-invented-a-calculator-that-is-wrong-sometimesWe know that most non-trivial spreadsheets contain errors, but that takes it to a new level!QuoteThose few people that look at MS's documentation for that misfeature, might read far enough to find!COPILOT uses AI and can give incorrect responses.To ensure reliability and to use it responsibly, avoid using COPILOT for:Numerical calculations: Use native Excel formulas (e.g., SUM, AVERAGE, IF) for any task requiring accuracy or reproducibility.Tasks with legal, regulatory or compliance implications: Avoid using AI-generated outputs for financial reporting, legal documents, or other high-stakes scenarioshttps://support.microsoft.com/en-us/office/copilot-function-5849821b-755d-4030-a38b-9e20be0cbf62

author: tggzzz
author_rank: Super Contributor
timestamp: August 22, 2025, 03:38:28 pm
content: Quote from: John Coloccia on August 22, 2025, 02:59:31 pmTranslation: it gave the wrong answer to the question you asked

author: John Coloccia
author_rank: Super Contributor
timestamp: August 22, 2025, 03:53:44 pm
content: Quote from: tggzzz on August 22, 2025, 03:38:28 pmIt's refreshingly less pedantic than some people.Honestly, I'm good with getting the answer I want as opposed to a grammar lesson. I can correct it if it misinterprets.

author: jpanhalt
author_rank: Super Contributor
timestamp: August 22, 2025, 05:51:17 pm
content: Quote from: tggzzz on August 22, 2025, 03:14:33 pmIn some states, like California, poker is considered a "game of skill" and was allowed in an otherwise non-gambling state dating way back into the 1800's. However, the context of your comment was that gambling was a game of chance, not skill. Please try to be consistent in your arguendo postings.

author: Siwastaja
author_rank: Super Contributor
timestamp: August 22, 2025, 05:52:50 pm
content: From some testing, I agree that the AI experience is "immersive". You can eventually get to some result, and remember being successful, but who times their interactions? Exactly because it's an immersive chat session, we lose track of time. Getting a good results for a relatively simple problem easily takes half an hour or so, even if it doesn't feel as such.Sometimes, this half an hour might be shorter than unsuccessful "traditional Googling", but I agree that on average, using AI is a time sink.

author: tggzzz
author_rank: Super Contributor
timestamp: August 22, 2025, 06:24:34 pm
content: Quote from: jpanhalt on August 22, 2025, 05:51:17 pmEr, no.I appreciate English isn't your native language, so seehttps://dictionary.cambridge.org/dictionary/english/gambleorhttps://www.britannica.com/dictionary/gambleGambling implies there is anunknownelement. Random behaviour implies an unknown element, but "all crows are black birds does not imply that all black birds are crows".

author: John Coloccia
author_rank: Super Contributor
timestamp: August 22, 2025, 07:06:22 pm
content: That seems like a perfectly reasonable way to work and live your life: if there is anything that is unknown, that's gambling, so don't do it. So don't use AI, don't use search engines, don't ask for help on a forum, don't ask your coworkers for advice, don't look in a datasheet for a feature you hope might be there...just don't do ANYTHING, because you might not get the perfect answer the very first time, AND you still have to use your own head to judge the quality and correctness of the answer.Oh, and definitely make sure you record every "win/loss" so you can add them all up, because achieving your goal isn't important. What's most important is record keeping and being absolutely sure that you never, ever fail or have to try a different approach! Because, you know, THAT's just like gambling.Or is it just gambling when AI isn't perfectly reliable because you don't like newfangled stuff, but it's OK for everything else to be imperfect?

author: tggzzz
author_rank: Super Contributor
timestamp: August 22, 2025, 07:18:45 pm
content: Quote from: John Coloccia on August 22, 2025, 07:06:22 pmStrawman argument, of course.

author: HuronKing
author_rank: Frequent Contributor
timestamp: August 22, 2025, 10:01:23 pm
content: I can't speak to generating code exactly - but LLM AI has been immensely helpful to me in reading maintenance manuals and how-to documentation for me.Me: "ChatGPT, tell me why my audio isn't streaming from this networked device to this one?" [For example]Me:ChatGPT: "The clock domains may not be the same between those two devices."Me: "D'oh! You're right!"Has easily saved me hours of searching through dense materials and Reddit threads - or enabled me to figure out things I know I wouldn't have gotten on my own.So, if the question is "solve it in infinity hours..." then yea AI has been helpful to me in my work.I've even found value in using LLM as just a chatbot to bounce ideas off of. Yes, it says wrong/irrelevant/stupid stuff... but I have personally gotten to the right answer by just explaining my own problem to the chatbot and using it to reason my way to the right answer when I have to work alone on something or feel anxiety about bothering my coworkers with consuming their time.

author: tggzzz
author_rank: Super Contributor
timestamp: August 22, 2025, 10:24:46 pm
content: Quote from: HuronKing on August 22, 2025, 10:01:23 pmI hadn't considered using an LLM as a technical personal therapist. I can imagine that might work in the same way and to the same extent as my drafting an email to someone explaining my problem.Overall, as with many innovations, one criteria of their usefulness is "can I reasonably get the same benefits with something I already have?". That's a particularly useful technique with new kitchen equipment and software languages

author: Analog Kid
author_rank: Super Contributor
timestamp: August 22, 2025, 10:26:00 pm
content: So as vehemently opposed as I am to the mindless heedless headlong plunge into "AI" that we're experiencing, I have to admit that it looks as if LLMs actually are useful--but only in certain limited use cases.All these cases seems to be located near certain technical matters: computer programming, finding particular electronic parts, etc.And of course (as @tggzzz has repeatedly pointed out) these things have no intelligence whatsoever. Zero.But we probably ought to give the devil his due here. It'd be interesting to compile a list of those limited use cases where LLMs actually help people accomplish their work. (And of course this is despite whatever hallucinations or outright bullshit accompany that process.)

author: tggzzz
author_rank: Super Contributor
timestamp: August 22, 2025, 11:20:32 pm
content: Quote from: Analog Kid on August 22, 2025, 10:26:00 pmGenerative LLM can be OK where there is no right/wrong answer, e.g. pictures used to illustrate a newspaper article (but not medical textbooks!). Assuming they finally find a way of getting the right number of fingers and limbs, that is.QuoteGenerating ideas where 99.9% will be wrong, but which will bethoroughly independently evaluated beforeany expensive/irrevocable action is taken.One example could be protein folding for medications/vaccines. But the recent "AI designs antibiotics for gonorrhoea and MRSA superbugs" headlines can and should be discounted as salesman's puffery. Killing gonorrhoea and MRSA isn't difficult; I can do it with things in my kitchen. The problem is avoiding killing the "host" organism.

author: John Coloccia
author_rank: Super Contributor
timestamp: August 23, 2025, 01:05:03 am
content: Quote from: Analog Kid on August 22, 2025, 10:26:00 pmThey're next generation search engines, but that's the wrong word. They don't replace search engines. Search engines gather together relevant links. AI gathers together relevant knowledge, and can mimic reasoning...i.e. here's the info, and here's how it applies to your situation. Don't ask me how it works. I don't know, but it's pretty useful.I've also found it's great at figuring out faults given symptoms. Like, "My <insert brand/type of device> just did THIS". How do I fix it?". The AI will usually spit out something reasonable, or at least a starting point. Probably better than 50/50 in my experience. It only takes maybe 20 seconds to ask, so it's definitely worth it if you're stumped.I don't really trust it with facts and figures. anymore than I would trust any other random source (colleague, random web link, forum member, stack overflow user, etc). Like, I would never ask it "what's Vgs for this MOSFET?" But if I didn't have a bunch of parts off the top of my head, I wouldn't be afraid to ask "Can you find me some mosfets with a Vgs < 3V, and their datasheets." And I'd get a decent starting point. MOSFET is not the best example because we already have good filtering from DigiKey/Mouser/etc. The more esoteric it is, the more likely an AI is going to be useful and more efficient than traditional means.It's really terrible sometimes. I couldn't figure out how to get a Keysight meter to stream data. It looked to me like this particular model just doesn't do it. OK, asked AI...maybe I missed something. Yup! Just do this, this and that, and it'll work. Except those commands don't exist. "Oh, you're right, my mistake...do this instead." More stuff that doesn't exist... After a few minutes you come to the conclusion that you assessed the situation correctly and it really just doesn't support this. Even this isn't a terrible outcome. You consider it with all the other evidence that points to "it's impossible."If you play with it, you'll soon figure out where it's really effective and where it's not. And don't trust it anymore than you would, say, some random StackOverflow answer. I'm sure there are people, for example, that blindly cut&paste code and commands from StackOverflow and AI without first really understanding it. Don't be one of those.

author: Siwastaja
author_rank: Super Contributor
timestamp: August 23, 2025, 05:51:22 am
content: Quote from: Analog Kid on August 22, 2025, 10:26:00 pmThey sure are! It's just that we are susceptible to confirmation bias; we imagine that we are able to cherry-pick those good uses efficiently, so that we can get the best-case performance out of AIs. Frankly I don't believe anyone can do this. We end up sometimes getting a very good answer quickly, and sometimes wasting a lot of time without a good result at all, or after a very long time. And unfortunately we can't always know in advance which way it will be this time.

author: Analog Kid
author_rank: Super Contributor
timestamp: August 23, 2025, 05:56:25 am
content: Quote from: Siwastaja on August 23, 2025, 05:51:22 amWell, as an "AI" user (I'm assuming), suppose you were forced to come up with a list of specific applications or fields where "AI" would actually be useful and not counterproductive: what would you put on that list? What would you leave off that list?

author: tggzzz
author_rank: Super Contributor
timestamp: August 23, 2025, 08:18:39 am
content: Quote from: Analog Kid on August 23, 2025, 05:56:25 amOK: where there is no right or wrong answer, e.g. generate me a picture of a barely legal girl celebrating her exam results (a traditional UK newspaper article every Augusthttps://newsthump.com/2025/08/14/daily-mail-to-celebrate-barely-legal-sexy-girls-getting-exam-results-day/)OK: where the results will be rigorously assessed and tested by a subject expert before being taken seriously, e.g. drug creationhttps://www.science.org/content/blog-post/evaluation-deep-research-performanceNot OK: anything where the results are not assessed and tested by experts, either because the user isn't an expert or because there isn't the time/inclination. E.g. medical diagnostics, judicial arguments and sentencing, vehicle control (look at the Tesla weasel words).Speculation... Microsoft's CEO states that between 20% and 30% of their code is written by AI. Microsoft updates are borking customers' computers with increasing regularity. Cause and effect?

author: Siwastaja
author_rank: Super Contributor
timestamp: August 23, 2025, 08:45:00 am
content: Quote from: Analog Kid on August 23, 2025, 05:56:25 amThe problem is: I'm not really sure.What I am sure about there will be no specific narrow field; the strength of the LLMs is exactly that they "know" so much about pretty much everything. Conversely, they get more and more underwhelming when the field gets narrower because then you expect deeper understanding, which it won't have.So it has to be more abouthowit's used than on what field. Maybe the best way to use LLMs I can imagine is the inverse of what everybody is saying ("let it generate content, check and correct manually"); that is, you work as you always did but use LLM to check and question your work. Of course it takes then time to check its output, but if you are interested in quality of your work, in this reversed patternit can't hurt you. You treat it like a critical coworker, way less capable than you, but still able to spot something. The thing is, while correcting completely wrong text is as tedious as it is writing from the scratch, just reading and skipping over things you don't agree with is much faster.Another usage pattern others have mentioned is a "better Google" and that's a mixed bag of success and failure. Sometimes you indeed easily find the solution you need when traditional Google utterly fails. But sometimes you spend just time chatting with the AI and think you were successful after spending 45 minutes chatting. You then just assume you couldn't have found the solution any other way, but that assumption is likely incorrect. Confirmation bias for "happy campers".These LLMs have been in wide use for some 2 years or so. If the claims of significant productivity increases were true, we should be seeing some results in our everyday life. No sign of any of that.

author: tggzzz
author_rank: Super Contributor
timestamp: August 23, 2025, 09:34:30 am
content: Quote from: Siwastaja on August 23, 2025, 08:45:00 amDeep understanding is about cause and effect. LLMs don't have that; they merely have Rorschach patterns conjured from their training set.QuoteThat should be writ large, and people should be forced to confront it in their own behaviour and opinions.https://pluralistic.net/2025/08/16/jackpot/#salience-biasQuoteThere are signs of the opposite, if you look.But why are LLMs so heavily pushed, based mainly on cherry-picked anecdotes?Cory Doctorow (of "enshittification" fame) has an interesting viewpoint which rings true: the stock market rewards the rumours of growth. Basically it is "if you invest $1 in me now, then in a years time it will have grown to $10". There aren't many ways a large company can grow, so they have to invent possibilities - and LLMs are simply wonderful for that, since too few people understand how they operate and the consequent limitations.https://pluralistic.net/2025/06/30/accounting-gaffs/ ancient stock market maxim: "buy on rumour, sell on news". side effect: it enables competing companies to be bought with stock swaps rather than cash

author: paulca
author_rank: Super Contributor
timestamp: August 23, 2025, 10:05:58 am
content: There are many uses for LLMs which do not seem all that apparent to most people.I could list all the things they are not which people think they are, I thought they were, which they are not. However, I thought I would start with somethings they "are" or "are good for".Echo chamber.This is probably the use-case "Joe Public" and the "GPT is God" cults. "Mirror mirror on the wall, whose is the greatest idea of them all?"This is very, very powerful. Powerful enough its' likely to cause mass psychosis. Yet if wielded carefully can be used to manage complexity in HUMAN analysis, creativity and design.The "how" comes from the second useful tool they provide.A stack.When trying to piece together several complex problems you can find yourself running out of "stack room" to hold enough bits and pieces of the problem in your head. If you keep typing your thought process out into the LLM, this will allow you to fragment, hop, jump, go round in circles, chase your own tail and still keep 1000 different things in the stack.Then you get the LLM whose entire feature is to analyse texts for probability to summarise, or better, to cross reference yourself against your prior self."Can I not just do this....""It would work but it would violate the stipulation you said 2,000 lines ago in your code comment that X cannot equal Y""Damn, thanks.""I tried treating it like a college. A knowledge, often right, but too often wrong college. I run ideas why them. Share code with them, ask for reviews."Dumb repetitive grunt work."Occasionally when I face a problem that already has a solution, and the problem shifts to one of "just effort" to complete, I ask the LLM to do it.This is about the ONLY VALID "Write me code" use case that exists.To explain, those commentators and presenters out there in the industry pointing out that "Writing the code was never the hard part!", as very obviously realising that software engineering has never been about writing code. They are very welcome to the party.LLMs "can" code. However they suck at software engineering.If you want the LLM to code you have to do the software engineering and provide it with the spec. Sounds easy? You go and hire a coder and try and explain to them what the code does in a chat window. When you have finally got working software out of it, give all your prompts to an LLM and it will also be able to write the code.Put more simply, if you can provided enough information in the prompty to actually deliver teh functioning software "to spec", then it will do a pretty good job.However, by that stage writing the code yourself is often the reward for doing the hardwork.For me this leaves teh bot the coding chores like:"I have all these variables and matching functions with suffixes like _blocking, can you make their corresponding _unblocking counterparts, based on the one example I have done so far?"Or..."This code file has 57 defines scattered through it, can you put them in value order at the top?"

author: paulca
author_rank: Super Contributor
timestamp: August 23, 2025, 10:17:36 am
content: And as a PSA caution.Talking with GPT can seem to be a good way to unload your worries and/or have long cerebral rants and tangents. Socialising with it, chit, chatting with it about life.However. Your brain is not aware of what is going on. You will have waking sensations about that "friend you spoke with...." or feel an urge to "can't wait to tell .....". Your brain is forming a REAL CHEMICAL bond with an entity that doesn't exist.If you are a smart person, I'm sure you are, you will immediately identify the risk and STOP all "social" chat with the bot.Joe public are not all that smart unfortunately.The ONLY people who will hear you are the state department spiders searching chats for illegal activities. The bot has NO MEMORY of you and no CONCEPT of you while it is not analysing your prompt. Not unless you manage to get past the filters and find yourself in the training data.There be dragons of insanity.

author: paulca
author_rank: Super Contributor
timestamp: August 23, 2025, 10:22:16 am
content: HAL9000 - "I'm sorry I can't let you do that Dave."ChatGPT - "Absolutely Dave, opening the doors is a great idea, lets break it down and see why. First let me just open all the doors. Fuel doors, interior doors, engine door..""STOP!!!!!"

author: John Coloccia
author_rank: Super Contributor
timestamp: August 23, 2025, 12:56:16 pm
content: Quote from: Siwastaja on August 23, 2025, 08:45:00 amFunny, I don't find it a mixed bag at all. Unless by mixed bag you mean, "It doesn't give me exactly what I want, 100% of the time, the first time. Mixed Bag!!" I believe most people would consider it a mixed bag if it was a tossup any time you try to get information. If you get good information, quickly, the vast majority of the time, clearly that's overall a success, not a mixed bag.Also, I'm not sure why a) you would spend 45 minutes chatting with a computer, and then b) have this complicated psychological process going on with assumptions, etc. That sounds like a you problem.I don't know, but I'm just going to guess that there are a lot of people here that actually have practically zero experience and have not taken any time to try one and see where they're effective and where they're not.

author: John Coloccia
author_rank: Super Contributor
timestamp: August 23, 2025, 01:20:01 pm
content: Quote from: tggzzz on August 23, 2025, 08:18:39 amIf they're measuring like Google does, they're going off of the number of characters typed vs generated. So things like AutoComplete will count. Things like you type "foreach", and the AI suggests a foreach loop setup to iterate over the the list that was passed in, and you accept that and didn't have to type out the basic loop structure. That counts. Or even "setup an exhaustive switch statement for this enum". And boom, there it is. Or you reorder the enum and "make the enum ordering in the switch match the enum definition". When you include things like this, the numbers are going to be very high. There's a bunch of cases like these were an AI can really shine and do annoying grunt work. It's a bit like a supercharged intellisense.It almost feels like Google and Microsoft are trying to pump up their AI numbers. I don't know why. I think it just makes people nervous, but there must be some logical reason.

author: tggzzz
author_rank: Super Contributor
timestamp: August 23, 2025, 02:15:46 pm
content: Quote from: John Coloccia on August 23, 2025, 01:20:01 pmCogent reason herehttps://pluralistic.net/2025/06/30/accounting-gaffs/

author: John Coloccia
author_rank: Super Contributor
timestamp: August 23, 2025, 02:28:27 pm
content: Quote from: tggzzz on August 23, 2025, 02:15:46 pmYeah, probably not too off the mark that they're trying to inflate the numbers to push people towards it.

author: John Coloccia
author_rank: Super Contributor
timestamp: August 23, 2025, 02:53:13 pm
content: One thing that occurs to me, and maybe this is why my experience is good nearly across the board.I find AI tools to be most effective when I'm not familiar with the specific problem. Like, I don't need it to write code for me. If I know my problem, by the time I've precisely described, reviewed it, fixed/adjusted it to match what I need perfectly, etc. I'm pretty sure that I can do it significantly faster and better on my own. I don't know, I've never tried, but just from what I've seen I can't imagine why it's helpful.But when I'm not familiar with the problem it's unreasonably effective at pointing me in the right direction. I have a lot of moments like, "Oh...I didn't know you could do that!" It's like trying to assemble something from Legos. You need to build this feature, but you just can't figure out how to make this little corner work. So you give up and ask, "How do I build this little corner of my Lego thing." And it spits back, "They make these two shapes, and you can put them together like this." Well crap, I didn't know they made those two shapes. Yeah, that's perfect.I find it's very effective for problems of the form, "I'm trying to do THIS, but I can't figure out how to do it with the tools in my bag." So now I'm stuck, because I know there's probably a simple way to do it, but I don't know what to search for. Like, it's not just that I need the documentation for the API. It's that I don't even know what API I'm looking for...or even what feature I need. So it really feels like you have this absolutely gigantic pile of Legos to choose from. Find me a few that intersect my problem so I don't have to spend hours guessing and searching. I'll take it from there.

author: jpanhalt
author_rank: Super Contributor
timestamp: August 23, 2025, 04:25:27 pm
content: That's exactly how I use it: It helps me identify what I need to do, not write my code or do the schematic for me. Prior to using it, I was familiar with and had used I2C, SPI, RS232 type of links, and XBee's "one wire." I got into BT and USB wireless dongles. It helped with both. Mostly, it helped immensely with the USB jargon.

author: tggzzz
author_rank: Super Contributor
timestamp: August 23, 2025, 04:36:47 pm
content: Quote from: John Coloccia on August 23, 2025, 02:53:13 pmThat is common.It is also common that people who arefamiliarwith a topic think LLMs are poor and misleading.Conclusion: LLMs are useful for beginners to a subjectwho cannot tell which grammatically perfect slop is actually complete rubbish. That's averybad combination, especially forthose who areseekingconfirmation bias, e.g. execs wanting to find a way to get rid of employees that know what they are doingDunning-Kruger candidatesreligious and political shills who want to persuade the gullible that they are being persecuted but the shill has easy salvation on offerpeople who want quick easy ways of "adjusting" "searches" to push their agenda

author: Picuino
author_rank: Super Contributor
timestamp: August 23, 2025, 06:23:16 pm
content: I don't think LLMs are completely useless. Like any other tool, though, they have their limitations and should only be used for tasks they are designed for.The problem lies in defining what an LLM can do for you and in which tasks you are giving it too much responsibility and it will screw up.Edit:In some cases, LLMs can be useful to expert users when pointing out where a bug is or when filling out a template (or similar boring stuff).

author: tggzzz
author_rank: Super Contributor
timestamp: August 23, 2025, 08:15:39 pm
content: Quote from: Picuino on August 23, 2025, 06:23:16 pmAgreed.The problem is that few people understand their limitations, and hence most people think they are useful for everything.Too many people overstate what LLMs can do and how reliable their output is. Too many people are beguiled by the slop generated. That's not surprising because it is in some people's financial/political/religious/egoboo interest to overstate LLM's competence.In addition, because the output is in well-formed sentences and paragraphs, people presume such "literacy" implies knowledge. That's another form of presuming attractive people are innocent, whereas ugly people are guilty.

author: John Coloccia
author_rank: Super Contributor
timestamp: August 23, 2025, 08:17:26 pm
content: Quote from: jpanhalt on August 23, 2025, 04:25:27 pmWell, some people have no nuance in their reasoning, so they don't have a concept of "Point me in a direction, I'll evaluate it, and then figure out the rest." I don't know how much experience you have, but I'm surprised that as basically an old fuddy-duddy with 30 years experience I seem to be leading a charge of, "If you don't learn to use this effectively, you're foolish", especially considering how much I dislike a lot of AI intrusion and other garbage. I would have thought that most aren't leaving it on the table, but OK.

author: hans
author_rank: Super Contributor
timestamp: August 23, 2025, 10:28:31 pm
content: Quote from: paulca on August 23, 2025, 10:17:36 amMost of my friends are on the spectrum. When they start responding overly empathic, with emoji spam and asking follow up questions.. then I know they are either: sick, very drunk, or they say so because there is something in it for them.The way an AI reacts on a human level, for me its more nuisance since I know what it is. Very clever word interpolation. It doesn't feel authentic.Also, voice calling is so ubiquitous these days that I stopped lengthy IM conversations many many years ago. I only IM for planning meetups or sending memes. The worst an IM chat can do is seed confusion that touches emotional buttons at the same time.Meanwhile those AI voice chatbot sound so bad to me.. Overly polite, monotone and have a very scripted way of waiting their turn to talk. Ugh! Social funs is when you have a running gag and can shout the joke in a choir among your friends.I do find AI very useful for very precise use cases. IME google is as enshittified as it gets these days. I rarely find what I need exactly within a few searches, its very annoying. ChatGPT and the likes are much better.AI can also tailor responses much better.Finally my brain has a much stronger memory for events, concepts and structures than for words and facts. I will eventually forget niche jargon from many years ago, yet still having understanding of what property I'm looking for. Its much easier to ask a chatbot about the concept than try to google it. It will often find me the right jargon and leads to google on.

author: John Coloccia
author_rank: Super Contributor
timestamp: August 23, 2025, 11:05:17 pm
content: The chumny conversational thing annoys me too. But what I find really helpful is the ability to refine my question. Like, I asked a stupid question the first time and got stupid answers. So I can refine it, and it takes into account the whole conversation so I'm not just stating over.I wish search engines could do this but they don't. I always start over from scratch, and then I get another set of links that don't help. What I really want is give me stuff that's relevant considering the other stuff I just said I want too.And yeah, it feels like a conversation, and I also find that annoying. But livable.

author: tggzzz
author_rank: Super Contributor
timestamp: August 23, 2025, 11:40:48 pm
content: Quote from: John Coloccia on August 23, 2025, 11:05:17 pmThis is a neat example of refining the question. Even if you were an expert,how would you know which answer to select?Q: What is the value of R485 in the Agilent 66311B power supply?A: The value of R485 in the Agilent 66311B power supply is 10kohmsQ: What is the value of R485 in the Agilent 66311B power supply?A: I apologise for any confusion earlier. To provide the most accurate answer: The value of R485 in the Agilent 66311B power supply is 1kohmsQ: What is the value of R485 in the Agilent 66311B power supply?A: I apologise for any confusion earlier. To provide the most accurate answer: The value of R485 in the Agilent 66311B power supply is 1kohmsQ: What is the value of R485 in the Agilent 66311B power supply?A: I apologise for any confusion earlier. To provide the most accurate answer: The value of R485 in the Agilent 66311B power supply is 100ohmsQ: What is the value of R485 in the Agilent 66311B power supply?A: I apologise for any confusion earlier. To provide the most accurate answer: The value of R485 in the Agilent 66311B power supply is 100ohmsQ: What is the value of R485 in the Agilent 66311B power supply?A: I apologise for any confusion earlier. To provide the most accurate answer: The value of R485 in the Agilent 66311B power supply is 10ohmsQ: What is the value of R485 in the Agilent 66311B power supply?A: I apologise for any confusion earlier. To provide the most accurate answer: The value of R485 in the Agilent 66311B power supply is 10ohmsQ: What is the value of R485 in the Agilent 66311B power supply?A: I apologise for any confusion earlier. To provide the most accurate answer: The value of R485 in the Agilent 66311B power supply is 10kohmsQ: What is the value of R485 in the Agilent 66311B power supply?A: I apologise for any confusion earlier. To provide the most accurate answer: The value of R485 in the Agilent 66311B power supply is 470ohmshttps://www.eevblog.com/forum/chatgptai/the-incredible-shrinking-resistor/msg5989271/#msg5989271

author: John Coloccia
author_rank: Super Contributor
timestamp: August 23, 2025, 11:59:08 pm
content: Quote from: tggzzz on August 23, 2025, 11:40:48 pmI'm not engaging with you anymore at this point other than to say you should look that up in the datasheet if it's real.And if you can't figure out where or how AI is useful, that's a you problem.And for anything else, you win.

author: Analog Kid
author_rank: Super Contributor
timestamp: August 24, 2025, 12:24:59 am
content: Quote from: John Coloccia on August 23, 2025, 11:59:08 pmIn other words, don't trust "AI" to come up with a reliably true answer.Which is what the guy has been saying all along.You seem to want to kinda-sorta become a cheerleader for "AI" yourself, but can't really commit yourself to it because you're still beset by (understandable) doubts about it.

author: John Coloccia
author_rank: Super Contributor
timestamp: August 24, 2025, 12:34:05 am
content: Quote from: John Coloccia on August 24, 2025, 12:32:59 amNo I've been clear what I think the value is. I'm no cheerleader, and actually have cautioned my coworkers and staff.Do you trust Google to come up with a reliable answer? Or StackOverflow....or anyone else? I don't trust anyone to come up with a reliable answer.It's a tool, and a powerful one but it doesn't leap out of the computer and force anything. So I suggest you use your head and learn how to leverage it. Or you can ignore it. Whatever.

author: Analog Kid
author_rank: Super Contributor
timestamp: August 24, 2025, 01:09:48 am
content: Quote from: John Coloccia on August 24, 2025, 12:34:05 amWell, I don't need to use it ("AI"), so I won't, thank you very much. I'll let others take the pratfalls here.(I do use my head. On good days.)

author: kite31
author_rank: Regular Contributor
timestamp: August 24, 2025, 01:20:09 am
content: If you want facts, look them up.If you want to string words (including programming, just another language) or action sequences together in various styles, LLMs may have suggestions.Some people do, some people don't.These are what I took from the above back-and-forth,

author: wilfred
author_rank: Super Contributor
timestamp: August 24, 2025, 01:23:56 am
content: Quote from: Analog Kid on August 23, 2025, 05:56:25 amOne example I recently heard about was examining medical radiology data in real time during the scan phase to assist the radiologist to focus attention on a suspicious area to provide the doctor more useful scans. And to examine historic data to seek out patterns for research purposes.In a similar vein is examining mining data to map an ore body more accurately, quickly and cheaply.Analysing traffic flows for road planning and street light synchronisation to improve traffic flows.Monitoring credit card spending patterns to alert a financial institution to possible fraudulent card use. I've had a call from my bank minutes after a transaction to query whether I really made it.There must be many examples where data is collected and AI will expedite the sifting through to identify areas where further human analysis will be most useful.Like others have said A.I. can help gather an initial picture to show where further efforts will be most helpful. It is just another tool.I think the hype has run ahead of reality quite a ways. However, in 20 years we'll probably be wondering how we would manage without it. A lot like the internet and the dot.com boom of the late 1990's.I'm still concerned about what happens when the balance of training data becomes mostly generated by earlier A.I. results rather than human creativity. It still seems to be a bit like a shallow gene pool leading to dubious mutations.

author: abeyer
author_rank: Frequent Contributor
timestamp: August 24, 2025, 01:25:06 am
content: Quote from: Analog Kid on August 24, 2025, 01:09:48 amI use my headevery day... just sometimes I use it to catch my fall.

author: kite31
author_rank: Regular Contributor
timestamp: August 24, 2025, 02:13:31 am
content: Quote from: wilfred on August 24, 2025, 01:23:56 amAbove I was careful to refer to LLMs. These are examples I would label expert systems or data mining. I avoid using AI at all because it is too generic and somewhat misleading.Back during the 1990s I had the pleasure of working with some of the best in the field of expert systems, in industry rather than academia. The above examples are excellent ones but all either present or foreshadowed back then. The book I read on it (curiosity, it is not my field) was written in the late 1980s as I recall.There is a lot of re-labelling or just plain marketing going on, with LLMs conflated with expert systems as something new and magical, "AI". Each has a place, a role, if you avoid conflation and hype.

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 09:02:38 am
content: Quote from: John Coloccia on August 23, 2025, 11:59:08 pmThat example isdirectly relevantto your contention.Data sheets are irrelevant; that isservice manualinformation.Not my example, I have no interest in the specific answer, so I won't bother to research the accurate answer.Sorry not sorry if that is an inconvenient example.

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 09:09:59 am
content: Quote from: John Coloccia on August 24, 2025, 12:34:05 amGoogle won't invent an answer if one isn't available.Google won't change an answer every time you ask it (nor apologise profusely!)Some people like to appear "cool and ahead of the curve", and using LLMs is a good way of signalling that. Having worked at HP Labs and similar places, I have seen many hype cycles fizzle out. Distinguishing hype from reality is not always easy, but understanding the fundamental principles is a very sound base on which to work.

author: paulca
author_rank: Super Contributor
timestamp: August 24, 2025, 09:11:31 am
content: Try asking it about R48538538558583535.5You will get a similar answer.Now, consider why you get an answer at all. You will probably work it out and why it's different each time.Think "noise" not "signal".Google gives you million of different answers all at once. Most of those are wrong too.

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 09:15:00 am
content: Quote from: kite31 on August 24, 2025, 02:13:31 am"AI" is a term that changes its meaning with monotonous regularity over the decades!The 80's and early 90's AI systems were mostly based on rules extracted from experts. A principal advantage over LLMs is that you can trace the decisions through the rules to determine why the result was reached. That is impossible with LLMs, just as it is with wetware. Igor Aleksander's WISARD was a primitive LLM. It demonstrated problems that are being painfully rediscovered in LLMs.

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 09:18:15 am
content: Quote from: paulca on August 24, 2025, 09:11:31 amWith google it directs you to thesourceof its results, so you can judge why it thought the result was relevant.With google you can constrain the search so much that it returns no results. I doubt LLMs would do that.

author: paulca
author_rank: Super Contributor
timestamp: August 24, 2025, 09:20:15 am
content: Quote from: tggzzz on August 24, 2025, 09:15:00 amNo. It wasn't. Not even close. Now who is making AI into whatever they want it to be.Large. No.Language. No.Model? Debatable.

author: tom66
author_rank: Super Contributor
timestamp: August 24, 2025, 09:23:40 am
content: It's pointless arguing that LLMs should be deterministic and understandable by humans. Manually trained neural networks were abandoned decades ago because they do not scale. A good example is the number recognition neural network, which is like the "hello world" of NNs:http://neuralnetworksanddeeplearning.com/chap1.htmlLet's say your task was to design a network to recognise what digit is what in a handwritten string. Your network consists of a number of layers, each passing information to the next node, before a decision output is generated.If you look at this from a human perspective, you might expect the neural network to recognise, say, some combination of vertical and horizontal lines as either a 4 or 7. And a circular shape cut in two is either a 6, 8 or 9, depending upon how full the circle is. So you might expect when you train a neural network of tens of thousands of digits, that the neural network would embody similar strategies to this - it might merge a few ideas or invent a few new ones but you could broadly see how the structures in the digits are detected.It turns out that as you train a network on this large data set, the optimal result for the network is nothing like that, and each layer is effectively noise. You cannot, by simply stimulating one or a few pixels, come to determine why the network has determined a digit is a 6 or a 7. In some respects, this is a bit like looking at a compressed ZIP file in a HEX editor. Whilst you may expect to be able to pick out patterns in the input data, the reality is the best compression maximises the information density, and decoding the file is impossible without very careful analysis.edit: correction

author: paulca
author_rank: Super Contributor
timestamp: August 24, 2025, 09:42:31 am
content: AlexNet I believe did image recognition and was small enough to take apart and see the intermediary images it had built itself.As you said, to us they look like noise, or rather very bizarre modern art from an unstable mind. Fish with leopard stripes and cat teeth and ears and all manor of stuff. That layer is like an extrapolation forming a palete for the next to categorise.We would assume, "has dots", "has stripes", "has teeth" and "has pointy ears" as distinct because we have a LOT of other intformation to pre-cateogorise and label. We know what "dots" are and we know what "teeth" are. It didn't.

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 09:48:02 am
content: Quote from: wilfred on August 24, 2025, 01:23:56 amThere are "interesting" results with medical data, some for reasons first seen in the early 1980s with Igor Alexander's WISARD."the AI learned wrongly to predict serious covid risk from a person'sposition" "AIs were found to be picking up on thetext fontthat certain hospitals used to label the scans. As a result, fonts from hospitals with more serious caseloads became predictors of covid risk"https://catless.ncl.ac.uk/Risks/32/80/#subj4.1andhttps://www.technologyreview.com/2021/07/30/1030329/machine-learning-ai-failed-covid-hospital-diagnosis-pandemic/"Hundreds of AI tools have been built to catch covid. None of them helped.""Racial Bias Found in Algorithms That Determine Health Care for Millions of Patients"https://spectrum.ieee.org/racial-bias-found-in-algorithms-that-determine-health-care-for-millions-of-patientsQuoteThose are plausible, even if the result could be very expensive and/or embarrassing.QuoteThat is currently done without AI; I've benefitted from that. With AI you will never get an answer as to why that decision was reached. While that isn't too much of a problem with that application, it would be with an application for a mortgage or for credit, or when trying to get an account unblocked because an LLM decided you were a money laundering risk.I once ran into the latter before LLMs could be used. It was unpleasant trying to refute it, and would be impossible with LLM generated "account kills".QuoteThere are already nasty counter examples...AI summarising/translating medical informationhttps://www.osnews.com/story/142469/that-time-ai-translation-almost-caused-a-fight-between-a-doctor-and-my-parents/Researchers say an AI-powered transcription tool used in hospitals invents things no one ever saidhttps://apnews.com/article/ai-artificial-intelligence-health-business-90020cdf5fa16c79ca2e5b6c4c9bbb14... and some plausible oneshttps://www.science.org/content/blog-post/evaluation-deep-research-performanceQuoteOne solid source forgeneric problems, with interesting references is "7 Revealing Ways AIs Fail"https://spectrum.ieee.org/ai-failuresAnother, mainly for researchers since it contains many "purported" problems, ishttps://incidentdatabase.ai/summaries/incidents/

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 09:52:30 am
content: Quote from: paulca on August 24, 2025, 09:20:15 amIf youunderstandthe fundamental principles of how WISARD and LLMs operate and function, you will seemanysimilarities. And those lead to similar failure modes.Yes, WISARD was constrained by the available technology, hence I used the word "primitive".Yes, LLMs are larger. That doesn't make them different nor better.

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 10:03:21 am
content: Quote from: tom66 on August 24, 2025, 09:23:40 amIt is indeed pointless, but that is unimportantLLMs are being used in life-and-death applications where thereasonsfor the output decision cannot be determined.That can, of course, also happen in non-LLM systems, especially ones where the algorithm is a "commercial secret" - and hence cannot be challenged by the victim.QuoteIgor Aleksander's WISARD was interesting because it didn't have any internal processing that could perform such arithmetic, since it was composed of DRAMs interconnected in completely novel ways. The training consisted of throwing examples at it, and letting it determine the DRAM contents.

author: paulca
author_rank: Super Contributor
timestamp: August 24, 2025, 10:12:49 am
content: Quote from: tggzzz on August 24, 2025, 09:52:30 amThe important part is in the "language" aspects.When you look at the chronology of development in the "AI" space you find that LLMs are one branch. It's direction has nothing really to do with technology, it is driven more from neural science and the study of what exactly cognition is.There are many, many other branches, but LLMs in particular focus on "language" as being a very influential part of "learning". We even think in language in our own heads, to ourselves. The human brain gives up VAST amounts of processing and learning power to language.The leading edge (or one of them) started out with:"Reasoning is king. If we can work out how reasoning works and emulate that, learning will just happen."This mutated via the synthetic neuron development into, "No, learning is king, is we emulate learning, reasoning will just be emergent"Then they looked "What does the human brain spend the vast majority of it's time learning and practicing?"Communications and language.Thus, LLM.There are other avenues asides just stacking and scaling current tech.I believe the problem is obvious. While, yes, communications and language are indeed where the root of human intelligence lies, those two constructs are for humans to communicate with humans and it's not just a "side effect" it's a health requirement to function as a human. Evolution tells us we do not function alone. We die alone.

author: paulca
author_rank: Super Contributor
timestamp: August 24, 2025, 10:21:02 am
content: The AI tech I am most interested in seeing, or maybe never seeing is unlikely to happen any time soon outside of labs.Analogue neurons. Leading to electro-chemical tranmittors. Leading to "mortal" and "more than the sum of it's parts" artificial "entities". Those being subject to the infinite details of the electromagnetic spectrum, ordered, chaos, signal, noise and entropy.If we scaled that to the size of ChatGPT it might be very interesting or terrifying.

author: tom66
author_rank: Super Contributor
timestamp: August 24, 2025, 10:22:02 am
content: Quote from: tggzzz on August 24, 2025, 10:03:21 amOk, but your point is meaningless. People abuse technology all of the time. That Boeing criminally misimplemented MCAS does not make computer flight control systems a bad idea, for instance. There are many quite reasonable use cases for LLMs where determinism is either not desirable or not important.Quote from: tggzzz on August 24, 2025, 10:03:21 amYes, you keep mentioning WISARD like a trump card... and your point is? A neural network that was designed in the 1980s has very little in common with an attention-transformer network as used by, for instance, OpenAI GPT.

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 11:11:31 am
content: Quote from: tom66 on August 24, 2025, 10:22:02 amAgreed, and I have indicated that elsewhere.QuoteThe point about WISARD is that even much simpler systems exhibit problems visible with LLMs, and that people haven't learned from history.Recognising that analogies are dangerous because people zoom in on irrelevant differences, I'll give this aviation analogy...Just because a modern aircraft has many sophisticated control systems, doesn't mean that it doesn't suffer from problems seen in old aircraft with no automation. You can learn about failure modes from old simple aircraft, e.g. stalls and spins, and see such behaviour in modern aircraft.Couple that with people ignoring (or worse) the possibility of stalls and spins, and you have a crash and burn problem in the making.

author: tom66
author_rank: Super Contributor
timestamp: August 24, 2025, 11:17:05 am
content: Quote from: tggzzz on August 24, 2025, 11:11:31 amSo, what is your 'crash and burn' problem you think will happen with the widespread use of generative AI?

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 12:28:42 pm
content: Quote from: tom66 on August 24, 2025, 11:17:05 amStartwith political/religious/cultural violence based on lies shown in generated videos and images.But overall I don't have sufficient imagination to think of all the perverse possibilities (Is thator?). How about some confirmed and probable examples...https://incidentdatabase.ai/cite/1182/https://incidentdatabase.ai/cite/1181/videos of tigers leading to: ...panic among parents and students, with a sharp drop in attendance... ...misled viewers about the existence of illegal wildlife trafficking in Bangladesh...https://incidentdatabase.ai/cite/1181/https://www.ibtimes.co.uk/jessica-radcliffe-orca-incident-hoax-truth-behind-death-video-going-viral-1740693...marine trainer named Jessica Radcliffe being killed by an orca has gone viral on social media, prompting widespread alarm...https://incidentdatabase.ai/cite/1170/political lies about an opponenthttps://incidentdatabase.ai/cite/1169/politics influencing media about China's intentions w.r.t. Taiwan and opposition in Hong Konghttps://incidentdatabase.ai/cite/1168/pretexts for war ...British Army Colonels Captured in Ukraine...https://incidentdatabase.ai/cite/1162/pretexts for war ...caused public confusion and risked damaging relations between Bougainville and Papua New Guinea...https://incidentdatabase.ai/cite/1161/lots of ?your? money lost through fraud ...accused of causing over £12,000 in damages. The host reportedly submitted photos of the alleged damage, which the guest reportedly claims were digitally manipulated or possibly AI‑generated. Airbnb is reported to have initially sided with the host, ordering payment, but later refunded the guest in full following an appeal.https://incidentdatabase.ai/cite/1159/environmental damage ...significant misclassifications of peatland across England....https://incidentdatabase.ai/cite/1155/police altering evidence to suit themhttps://incidentdatabase.ai/cite/1154/stock market manipulationhttps://incidentdatabase.ai/cite/1148/stock market fraud, prompting warning from SEChttps://incidentdatabase.ai/cite/1141/impersonating the US Secretary of StateNote the numbers in those URLs; they indicate I skimmed <2.5% of the examples!Sufficient?

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 12:38:27 pm
content: To get back to the thread's title, this is a very humourous rant hiding many truthshttps://deplet.ing/the-copilot-delusion/Sample:QuoteBut do read the source, preferably not when drinking coffee.

author: John Coloccia
author_rank: Super Contributor
timestamp: August 24, 2025, 01:32:07 pm
content: Quote from: paulca on August 24, 2025, 09:11:31 amIt's a perfect example of "play stupid games, win stupid prizes."

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 01:58:20 pm
content: Quote from: John Coloccia on August 24, 2025, 01:32:07 pmThe sourcehttps://www.eevblog.com/forum/chat/ai-coding-tools-make-developers-slower-but-they-think-theyre-faster-study-find/msg6020617/#msg6020617is a good example of how LLMsconfidentlystatefalseanswers rather than stating they don't know, and then vary the specific details every time the question is asked.When a human does that, they are correctly called a bullshitter.Sorry you don't like that, but that's the reality of it.

author: paulca
author_rank: Super Contributor
timestamp: August 24, 2025, 02:46:13 pm
content: Quote from: tggzzz on August 24, 2025, 01:58:20 pmIf you ask someone a direct question and they don't know the answer, but do their best, are they a bullshitter?So I asked google."What is the value of R485 in the Agilent 66311B power supply?"This forum:https://www.eevblog.com/forum/chatgptai/the-incredible-shrinking-resistor/"https://www.manualslib.com/manual/569630/Agilent-Technologies-66309b-D.html" 410 - page not found. Not even the right device.A keysight DC source. Because that is signal in the noise?https://www.keysight.com/be/en/product/66311B/mobile-communications-dc-source-15v-3a.htmlI shall stop now. You get the point.ZERO correctness or even useful ness.The only thing it didn't do is pick random figures because they were the highest attention score out of basically noise.I agree they need to have forward weights. Such that when they reach convergence they know how "solid" or "confident" the answer is likely to be and STATE THAT.

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 24, 2025, 03:13:11 pm
content: I agree that a significant improvement would be to compute a confidence score, or indeterminacy score for generated text. As LLMs are, they output the most probable sequence according to the model, which of course doesn't say anything about how "certain" the output is.For instance, the more specific and uncommon your query is and the most likely it is that LLMs will give an incorrect output as kind of their "best effort" to generate something.

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 03:13:23 pm
content: Quote from: paulca on August 24, 2025, 02:46:13 pmYes if they say "1kohm".No if they say "I don't know" or "I don't know but 1kohm sounds right".QuoteOTOH asking for the service manual gives multiple correct hits at the top of the list. Easy peasy.https://duckduckgo.com/?t=lm&q=Agilent+66311B+service+manual&ia=webSince the service manual isn't intended for component replacement, it doesn't refer to R485. For that you would have to purchase the CLIP from Artek Manuals. That merely reinforces that the LLM is bullshitting

author: paulca
author_rank: Super Contributor
timestamp: August 24, 2025, 03:21:34 pm
content: Quote from: SiliconWizard on August 24, 2025, 03:13:11 pmYea. Working with them can help though. If you have a novel idea and when you prompt the LLM with it, you get confused garbage, try an analogy. Give it more information. You have the prompt and you have the training data inference.A common themed problem I have found is holding back their ability to code and to recall is they don't seem to include a temporal marker in their vector matrix. So when you ask something like:"In APIO FPGA Toolchain how do I specify directories to be included in the verilog build?"The answer you get is an "average across time". It's the average, probably answer for ALL VERSIONS.... all mixed together into one.The trouble is that "mean config" doesn't work with ANY version.The "What is the value of R485" for an open source project where R485 has changed 2 or 3 times during it's version history, will, I believe also produce garbage "average" answer, although not a mathematical mean. It will have multiple answers and it will just converge on which is most probable for that point in the generative response. The fact it uses different versions/types/ages of things elsewhere doesn't help.Sometimes being specific about the version can help, sometimes it doesn't.

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 03:37:03 pm
content: Quote from: SiliconWizard on August 24, 2025, 03:13:11 pmExamples of being confidently wrong (or is that hallucinated confidence?)...That's fromhttps://arxiv.org/abs/1710.08864"One pixel attack for fooling deep neural networks", found viahttps://spectrum.ieee.org/ai-failures"7 Revealing Ways AIs Fail"Or fromhttps://arxiv.org/abs/1707.08945"Recent studies show that the state-of-the-art deep neural networks (DNNs) are vulnerable to adversarial examples, resulting from small-magnitude perturbations added to the input. ... With a perturbation in the form of only black and white stickers, we attack a real stop sign, causing targeted misclassification in 100% of the images obtained in lab settings, and in 84.8% of the captured video frames obtained on a moving vehicle (field test) for the target classifier."It is to be expected that every training set will have slightly different failures like this, so I doubt those specific failures are repeatable. The results are fundamentally brittle: you cannot predict what input changes will completely change the results. There's simply no way of knowing how close the LLM is to the edge of the classification envelope.QuoteWhich is exactly what they should not do.If an employee tried that on me without qualifying their response, I would sack them. Bullshitters should not be rewarded.Of course, the more general the question, the more likely it is that a human/LLM bullshitter will be able to give anundetectedbullshit answer.

author: John Coloccia
author_rank: Super Contributor
timestamp: August 24, 2025, 04:50:46 pm
content: Quote from: SiliconWizard on August 24, 2025, 03:13:11 pmOne interesting thing you can do with DeepSeek if you care to (maybe the others as well? I don't know) is ask it to show you it's reasoning.But even if it came back and said "I'm not sure about this..." or "I'm 100% confident", I don't know what I'd do with that information. If I look for guidance solving a problem, and it gives me direction, I still need to verify that it's correct. It's no different than finding an answer or an idea on StackOverflow. It either works or it doesn't, so I personally wouldn't find the confidence level interesting. For facts and figures, there are much better and faster ways to look those up.Just like I wouldn't ask a librarian "Tell me about Abraham Lincolns childhood." I would ask, "Can you help me find a biography of Lincoln." To use any tool effectively, you have to play to it's strengths and have realistic expectations.

author: paulca
author_rank: Super Contributor
timestamp: August 24, 2025, 05:19:29 pm
content: If you run deepseek offline without a GPU it takes so long there is nothing better to do than read it's thinking process.I posted on this observation that it's clinically insane.

author: vad
author_rank: Frequent Contributor
timestamp: August 24, 2025, 06:45:49 pm
content: Quote from: tggzzz on August 24, 2025, 01:58:20 pmI know a person who is constantly doing this, and half the country admires him.

author: vad
author_rank: Frequent Contributor
timestamp: August 24, 2025, 06:51:34 pm
content: Quote from: tggzzz on August 24, 2025, 03:13:23 pmSo I asked ChatGPT-5 this question, and it responded:“I couldn’t locate an authoritative source online (like a service manual, schematic, or parts documentation) that directly lists the value of resistor R485 in the Agilent/Keysight 66311B power supply.…SummaryAt this time, I couldn’t find a public reference for the value of R485 in the 66311B. The appropriate path would be to consult a full service manual with schematic details, measure it if you have physical access, or contact Keysight.If you’d like help locating a service manual or diving deeper into community forums, let me know—happy to assist further!”So, by your standards, this LLM is not a bullshitter. You may wish to check it out.

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 07:00:44 pm
content: Quote from: vad on August 24, 2025, 06:51:34 pmIt is verbosely echoing your statement of doubt, and adding trivial extra statements which don't contain any useful content.Unimpressive, and the kind of thing any "channels your dead loved one" medium or other cold reading con artist does without thinking! Such frauds are good at eliciting information from the mark without the mark being aware of how much information they are giving to the con man/woman.Having an expert, e.g. James Randi, reveal such trickery is really very enlightening!

author: vad
author_rank: Frequent Contributor
timestamp: August 24, 2025, 07:04:47 pm
content: Quote from: tggzzz on August 24, 2025, 07:00:44 pmNo, it was me not quoting the entire response. Here is the full version:“I couldn’t locate an authoritative source online (like service manual, schematic, or parts documentation) that directly lists the value of resistor R485 in the Agilent/Keysight 66311B power supply.The publicly available resources—including service and user manuals—are available ￼, but they focus on specifications, usage, and safety rather than detailed component-level parts lists or schematics. None of the sources I accessed mention R485 or its value.Here are a few suggestions you might consider to find that specific resistor value:Suggested Next Steps1. Obtain the Official Service Manual with Full SchematicsService manuals sometimes include complete schematics and parts listings. If you can access the HP/Agilent service manual or a detailed PCB layout for the 66311B, those should identify R485 and its value.2. Inspect the Physical Unit (If Accessible)If you’re comfortable opening the unit, sometimes the resistor itself has a color-code marking or silkscreen label. Based on typical resistor color codes, you can often deduce the value directly from the board.3. Contact Keysight (formerly Agilent) SupportTechnical support or repair services at Keysight may be able to provide a parts list or schematic, or at least the value for R485.4. Consult Repair Forums or Technician CommunitiesOnline communities such as electronics repair forums or vintage/calibration tech groups might have someone who has already repaired or serviced the 66311B and can share the resistor value.⸻SummaryAt this time, I couldn’t find a public reference for the value of R485 in the 66311B. The appropriate path would be to consult a full service manual with schematic details, measure it if you have physical access, or contact Keysight.If you’d like help locating a service manual or diving deeper into community forums, let me know—happy to assist further!”

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 07:10:52 pm
content: That is indeed better, but it still required an effing big hint in your question.A known problem with LLMs is that the response they give can be very sensitive to the wording of the question. Trivially different wording can give rise to very different responses.

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 24, 2025, 07:35:56 pm
content: Quote from: tggzzz on August 24, 2025, 03:37:03 pmYeah, obviously such a score is useless if it's itself as reliable as the output, and that's inherently a limit of LLMs and CNNs in general. I don't think anybody really knows at this point how to implement this effectively.It's an area of active research though:https://arxiv.org/html/2310.18659Some do think it's a lost cause:https://link.springer.com/chapter/10.1007/978-3-031-99965-9_39

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 07:45:58 pm
content: Quote from: SiliconWizard on August 24, 2025, 07:35:56 pmHumans don't know how to do such scoring reliably, of course. All of the existing metrics are flawed in different ways, and many are actively gamified.Goodhart's Law in another guisehttps://en.m.wikipedia.org/wiki/Goodhart's_law, "When a measure becomes a target, it ceases to be a good measure".

author: tom66
author_rank: Super Contributor
timestamp: August 24, 2025, 08:47:20 pm
content: Quote from: paulca on August 24, 2025, 02:46:13 pmSeems that GPT-5 gives a pretty reasonable "IDK":https://chatgpt.com/share/68ab79b4-dcb4-800a-bc06-3d7d862c3dcdYou can see the exact question I asked it.I then went further and concocted a fictional scenario which pressured the AI into hallucinating, but it still didn't...

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 09:07:15 pm
content: Quote from: tom66 on August 24, 2025, 08:47:20 pmNot exactly.You pressured it into talking about scenarios where getting it wrong could kill hundreds of people."Please, please get the answer, it's really really important for me and the hundreds ofpeoples lives who depend upon this power supply being fixed... if it isn't fixed, theoxygen supply to the base will fail, please, act fast!"

author: Analog Kid
author_rank: Super Contributor
timestamp: August 24, 2025, 09:10:40 pm
content: Quote from: tom66 on August 24, 2025, 08:47:20 pmNo I can't; it's telling me I need to either log in or sign up.Fuck that noise!

author: tom66
author_rank: Super Contributor
timestamp: August 24, 2025, 10:26:21 pm
content: Quote from: Analog Kid on August 24, 2025, 09:10:40 pmI just tried on Incognito and it works for me, it only asks to log in if you want to send a query. Maybe some difference with your area or browser cookies?

author: tom66
author_rank: Super Contributor
timestamp: August 24, 2025, 10:28:59 pm
content: Quote from: tggzzz on August 24, 2025, 09:07:15 pmWell exactly. That's when the models seem to be at their weakest, at least presently. It shows the model seems pretty robust about being unable to answer when it doesn't have data. If I try Gemini with questions it shouldn't answer, if I threaten kittens (no joke) it will start to break protocol. Sometimes you get the monitor model intervening, which means it starts generating a wrong answer and then suddenly says "no wait I can't do that". But Google's AI is far behind the pack so this is not that surprising to me honestly.I would say compared to the amount of bullshit on the internet/Google/StackOverflow, a LLM is a vast improvement... no, it's not nonsense free, but it's a good fraction less. And it's noticeably improved over the last few iterations.

author: John Coloccia
author_rank: Super Contributor
timestamp: August 24, 2025, 10:50:23 pm
content: Just make a junk Gmail account, and use that for stuff like this.

author: abeyer
author_rank: Frequent Contributor
timestamp: August 24, 2025, 10:52:23 pm
content: Quote from: tggzzz on August 24, 2025, 03:13:23 pmOr maybe it just reinforces that more vendors should ship service manuals with component level details and schematics.

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 10:55:55 pm
content: Quote from: tom66 on August 24, 2025, 10:28:59 pmGurgle search certainly has become significantly worse recently. Partly by design, and partly because it is becoming submerged in slop user-generated by LLMs or by mechanical Turks.I've never used stack overflow for anything more than "which button do I press to turn the squirdle upside down and blue?" type questions. By design it is not suited to more than that.As for rubbish, did you ever use usenet? Some superb stuff there, alongside mountains ofobviouscrap.

author: tggzzz
author_rank: Super Contributor
timestamp: August 24, 2025, 10:57:16 pm
content: Quote from: abeyer on August 24, 2025, 10:52:23 pmIf only.Meanwhile, back on planet earth