by ninetyninenine | 2025-08-14 19:43:20+00:00
>I think you're spooked for the same reason I think that all the "AI alarmists" whose alarmism is based on our lack of understanding of LLMs are spooked. That because we "lack understanding" it follows that AI is "out of our control" or is on the verge of becoming "conscious" or "intelligent", whatever that means.

Yeah, well you made this shit up out of thin fucking air. I'm not spooked. We lack understanding of it, but it doesn't mean we can't use it. It doesn't mean there's going to be a skynet level apocalypse. You notice I didn't say anything like that? There's literally no evidence for it AND i never said anything like that. Here's what I think, I don't know how an airplane works. And I'm fine with it, I still can ride an airplane. I also don't know how an LLM works. I'm also fine with it. It just so happens, nobody knows how an LLM works. I'm also fine with that.

This spooked bullshit came out of your own hallucination. You made that shit up. My initial post is NOT meant to be alarmist. It's meant to be fucking annoyed at people who want to utterly deny everything to be totally simple and we totally get it. The fact of the matter is, we may not understand it, but I don't think anything catastrophic is going to emerge from the fact we don't understand it. Even if the LLM is sentient I don't think there's much to fear.

However this doesn't mean that what the alarmists say isn't real. We just don't know.

>Except this isn't true to me. Yes, we can't predict how inputs will map to outputs, but that's nothing unexpected? This has been true of nearly every ML model in practical use (not just those based on neural nets) for a very long time.

Doesn't change the fact we don't fucking know what's going on. Like I said. This is something you're spooked about IF what I said was true. I'm not spooked about it period. Your adding shit to the topic that's OFF topic.

>I don't perceive this as a "lack of understanding", in the same way I don't consider it a "lack of understanding" the inability to predict the output of a Support Vector Machine classifying email as spam, or not being able to predict how the coefficients of a radial basis function end up accurately approximating the behavior of a complex physical system. To me they're all a "lack of interpretability", which is a different thing.

This isn't a perception problem. It's not as if you perceive something in a different way suddenly your perception is valid. NO. We Categorically DO NOT understand it. Stop playing with words. Lack of understanding IS lack of interpretability. It's the same fucking thing. If you can't interpret what happened you don't understand what happened.

Maybe what you're trying to say here is that we understand LLMs enough in such a way that you aren't spooked. Since you made up all that bullshit about me being spooked, I'm guessing that's what you mean. But the fact of the matter remains the same: We UNDERSTAND LESS about LLMs then what we currently know.

>This is, to me, qualitatively different from our lack of understanding of the human brain. We know the algorithm an LLM is executing, because we set it up. We know how it learns, because we invented the algorithm that does it. We understand pretty well what's happening between the neurons because it's just a scaled up version of smaller models, whose behavior we have visualized and understand pretty well. We know how it "reasons" (in the sense of "thinking" models) because we set it up to "reason" in that matter from how we trained it.

Sure there are differences. That's obvious. But the point is we STILL don't understand LLMs in essence. That is still true despite your comparison here.

>Our understanding of the human brain is not even close to this. We can't even understand the most basic of brains.

If we understand 1 percent of LLMs but only 0.1% of the human brain. That's a 10x dramatic increase in our understanding of LLMs OVER the brain. But it still doesn't change my main point: Overall we. don't. understand. how. LLMs. work. This is exactly the way I would characterize our overall understanding holistically.

>Even postulating that LLMs are conscious, whatever that actually is in reality, is nonsensical. They're not even alive! What would "consciousness" even entail for a pure function? There's no reason to even bring that up other than to hype these things as more than what they are (be it positively or negatively).

Your statement is in itself nonsensical because you don't even know what consciousness or being alive even means. Like there are several claims here made about things you don't know about: LLMs and human brains made using words you can't even define: "alive" and "conciousness". Like the rational point by point thing you need to realize is that you're not rationally constructing your claim from logic. You're not saying we know A therefore B must be true. <--- that is how you construct an argument.

While I'm saying you're making claim A, using B, C and E but we don't know anything about B, C and E so your claim is baseless. You get it? We don't know.

>They're just as intelligent as a chess engine is intelligent. They're algorithms.

But you don't understand how the emergent of effects of the algorithm works so you can't make the claim that they are as intelligent as a chess engine. See? You make claim A and I said your claim A is based on fact B but B is something you don't know anything about. Can you counter this? No.

>We understand enough about how they work that we know just forcing them to output more tokens leads to better results and we have a good intuition as to why (see: Karpathy's video on the subject). It's why when asked a math question they spit out a whole paragraph rather than the answer directly, and why "reasoning" is surprisingly effective (we can see from open models that reasoning often just spits out a giant pile of nonsense). More tokens = more compute = more accuracy. A bit similar to the number of noise removal steps in a diffusion model.

This is some trivial ball park understanding that is clearly equivalent to overall NOT understanding. You're just describing something like curve fitting again.