by imiric | 2025-07-11 10:00:16+00:00
> I don't think that COBOL, BASIC, SQL have failed. They allowed many non-technical people to get started building things with computers.

Those didn't fail, but they're certainly not used by non-technical people. That was my point: that all technologies that previously promised to make software development accessible for non-technical people didn't deliver on that promise, and that they're used by software engineers today. I would chalk up the Low-Code and No-Code tools as general failures, since neither business people nor engineers want to use them.

> In the context of people not learning "real programming", you can equate LLMs to say, wordpress plugins or making a squarespace site.

I don't think that's an accurate comparison, as website builders only cover a small fraction of what's possible with "real programming". Web authoring and publishing tools have existed since the dawn of the web, and the modern ones simply turned it into a service model.

LLMs OTOH allow creating any type of software (in theory). They're much broader in scope, and lower the skill requirements to create general-purpose software much more than any previous technology. The software in TFA was an iOS app. This is why they're a big deal, and why we're seeing scam artists and grifters pump out these low-effort applications in record time and volume. They were already enabled by WordPress and Squarespace, and there are certainly a lot of scam and spam sites on the web thanks to website builders, but their scope, reach and productivity got multiplied by LLMs.

> If we care about the security of it all, then let's build the platforms to have LLMs build secure applications.

That's easier said than done, if it's possible at all. Security, privacy, and bug-free software is not something that can be automated, at least with current technology. It requires great care and attention to detail from expert humans, which grifters have zero interest in, and non-expert non-grifters don't have the experience or patience to do. Vibe coding, after all, is the idea that you keep pasting errors to the LLM and prompting it until the software on the surface works as you expect it to. Code is just the translation layer for the LLM to write and interpret; vibe coders don't want to know about it.

Could we encode some general security and privacy hints in the LLM system prompt so that it can check for specific issues? Sure. It will never be exhaustive, though, so it would just give a false sense of security.

> As far as the OP goes, these kind of security issues due to hardcoded credentials are basically the hallmark of someone shipping a (mobile|web) app for the first time, LLMs or not.

Agreed. What I think you're not taking into account is the fact that there is a large swath of the population who just doesn't care about this. The only thing they care about is having an easy way to pump out a service that attracts victims who they can quickly exploit in some way. Once that service is no longer profitable, they'll replace it with another. What LLMs have given these people is an endless revenue stream with minimal effort.

This is not the same group of people who cares about software, the product they're building, and their users. Those are a small minority of the new generation of software developers who will seek out best practices and figure out how to use these tools for good. Unfortunately, I don't think they will ever become experts at anything other than interacting with an LLM, but that's a separate matter.

So the key point is: building high quality software starts with caring. Good practices that ensure high quality are discovered by intentionally seeking out established knowledge, or by trial and error. But the types of issues we're seeing here are not because the developer is inexperienced and made a mistake—it's because they don't care. Which should be criticized and mocked in public, and I would argue regulated and fined, depending on the severity. I even think that a software development license is even more important today than ever before.