MurphyA/hacker-news huggingface dataset:
by kubb | 2025-06-08 08:07:43+00:00
It would be awesome to develop some theory around what kind of problems LLMs can and cannot solve. That should deter some leads pushing for solving the unsolvable with the technology.

That being said, this isn’t a knockout blow by any stretch. The strength of LLMs lies in the people who are excited about them. And there’s a perfect reinforcing mechanism for the excitement - the chatbots that use the models.

Admit for a second that you’re a human with biases. If you see something more frequently, you’ll think it’s more important. If you feel good when doing something, you’ll feel good about that thing. If all your friends say something, you’re likely to adopt it as your own belief.

If you have a chatbot that can talk to you more coherently than anyone you’ve ever met, and implement these two nested loops that you’ve always struggled with, you’re poised to become a fan, an enthusiast. You start to believe.

And belief is power. As in the case of neuroscience development not being able to retire the concept of the dualism of body and soul, so will the testing of LLMs not be able to retire the concept of AI poised to dominate everything soon.
by 1dom | 2025-06-08 08:08:33+00:00
"They're super expensive pattern matchers that break as soon as we step outside their training distribution" - I find it really weird that things like these are seen as some groundbreaking endgame discovery about LLMs

LLMs have a real issues with polarisation. It's probably smart people saying all this stuff about knockout blows, and LLM uselessness, but I find them really useful. Is there some emperor's new clothes type thing going on here - am I just a dumbass who can't see he's excited at a random noise generator?

It's like if I saw a headline about a knockout blow for cars because SomeBigBame discovered it's possible to crash them.

It wouldn't change my normal behaviour, it would just make me think "huh, I should avoid anything SomeBigName is doing with cars then if they only just realised that."
by kassner | 2025-06-08 08:09:17+00:00
> tad over 6W

That is the key. The RPi works for idling, but anything else gets throttled pretty bad. I used to self host on the RPi, but it was just not enough[1]. Laptops/mini-PCs will have a much better burstable-to-idle power ratio (6/3W vs 35/8W).

1: https://www.kassner.com.br/en/2022/03/16/update-to-my-zfs-ba...
by burnt-resistor | 2025-06-08 08:09:30+00:00
Sounds similar to backpacking through {India | Camino de Santiago}.

I think it's important for every young adult who becomes a well-rounded adult to have experienced a short term of deprivation so they have a frame-of-reference what others in less fortunate situations experience. <my-two-centidollars>The problem today is that there are too many mean, spoiled individuals with way too much power lacking theory of mind, a sense of community, and basic human compassion.</my-two-centidollars>
by pbhjpbhj | 2025-06-08 08:11:31+00:00
Downvotes sadly are endorsed by pg (the owner of HN) for use to indicate disagreement.

Flags are not downvotes and are not to show disagreement. They do seem to get used that way.

I like the others above have show-flagged enabled. "90%" of things I vouch are things I disagree with that represent what I consider a point of view that deserves to be known, has been at least reasonably well presented, and isn't flame-bait.
by golol | 2025-06-08 08:12:43+00:00
The first figure in the paper with Accuracy vs Complexity makes the whole point moot. The authors find that the performance of Claude 3.7 collapses around complexity 3 while Claude 3.7 thinking collapsed around complexity 7. A massive improvement in the complexity horizon that can be dealt with. It's real, it's quantitative, so what's the point of philosophical atguments about whether it is truly "reasoning" or not. All LLMs have various horizons, a context horizon/length, a complexity horizon etc. Reasoning pushes this out further, but not to some infinite algorithmically perfect recurrent reasoning effect. But I bet humans pretty much just have a complexity horizon of 12 or 20 or whatever and bigger models trained on bigger data with bigger reasoning posttraining and better distillation will push the horizons further and further.
by wqaatwt | 2025-06-08 08:12:49+00:00
> wealth indescribable

That’s kind of the problem.. even if we knew the amount of gold he actually had and multiplied it by its current market price the resulting figure would be entirely pointless. The question is how much stuff he could buy for it back in the 1300s. If there wasn’t enough stuff/people to buy having a massive hoard of illiquid gold doesn’t necessarily make you extremely wealthy.

Including absolute monarchs and dictators in the list is also semi meaningless.

Also the the way you chose to measure this wealth results in massive variance. E.g. Jacob Fugger was allegedly worth $400 billion if adjusted by inflation but he’s not even here.
by genewitch | 2025-06-08 08:13:10+00:00
> neural networks of various kinds can generalize within a training distribution of data they are exposed to, but their generalizations tend to break down outside that distribution.

so the AI companies really took that to heart and tried to put everything into the training distribution. My stuff, your stuff, their stuff. I remember the good old days of feeding the wikimedia dump into a markov chain.
by ookblah | 2025-06-08 08:13:17+00:00
how about lets stop making AI (i guess LLM here) some monolithic block that applies to every problem we could ever have on this earth? claude might suck at tower of hanoi, but in the end we will just use something better suited to the job. nobody complains that ML or vision models "fail spectacularly" at soemthing it's not suited for.

i get the criticism, but "on the ground" there's real stuff getting done that couldn't be done before. all of this boils down to an intellectual study which, while good to know, is meaningless in the long run. the only thing that matters is if the dollars put in can be recouped to the level of hype created and that answer is probably "maybe" in some areas but not others.

this AI doomerism is getting just as annoying as people claiming AI will replace everyone and everything.
by burnt-resistor | 2025-06-08 08:13:53+00:00
For retrofit purposes, it's probably attainable to use solid state (no moving parts) floppy disk emulators that use USB thumb drives or CF/SD cards instead of error-prone, real floppy disks. Every time a floppy drive moves over a sector to read or write, it wears that area mechanically. Magnetically, bits just seem to rot from floppy disks randomly with time more likely failure mode for previously good floppies.

Let me complain you about how error-prone and unreliable are real floppy disks. ):
by IshKebab | 2025-06-08 08:14:22+00:00
Argh please stop. Everyone knows LLMs aren't AGI currently and they have annoying limitations like hallucinations. Even the "giving up" thing was known before Apple's paper.

You aren't winning anything by saying "aha! I told you they are useless!" because they demonstrably aren't.

Yes everybody is hoping that someone will come up with a better algorithm that solves these problems but until they do it's a little like complaining about the invention of the railway because it can only go on tracks while humans can go pretty much anywhere.
by muzani | 2025-06-08 08:14:35+00:00
Agreed with the article. Some will say just do iOS/Cordova/React Native/Flutter. But these have all the same problems eventually.

Do web? You're just pushing the problem of accessing lower levels (esp. files and camera) to the browser. Browsers are not consistent. Some devices don't even update their browsers and it would break POST on a very specific version of Android.

There's a high end Samsung gallery that has a very inefficient way of displaying thumbnails. This is the default gallery. If the user has over 10000 photos or so, it freezes. People who buy this phone will often have a lot of photos. And they're rich; usually the investor. This bug doesn't show up in typical tests because QA can't afford this phone. So the fix is to avoid built in galleries, and write a custom gallery for the app that utilizes binary search or something. Major apps like FB and WhatsApp will have implemented their own in built gallery software so these device owners end up blaming the apps that don't.
by closewith | 2025-06-08 08:15:04+00:00
> Not really because buses get stuck in traffic all the time because there’s a point where they need to share roads with cars.

Like many tram lines, CVLR is being laid in-road and not segregated. In fact, while not mentioned here, the it's 15 m turning radius is so important is because it's planned to traverse roundabouts in-lane.
by Den_VR | 2025-06-08 08:15:31+00:00
Marcus’s writing is from a scientific perspective, it’s not general artificial intelligence and probably not a meaningful path it GAI.

But the ivory tower misses the point of how LLM improved the ability of regular people to interact with information and technology.

While it might not be the grail they were seeking, it’s still a useful thing what will improve life and in turn be improved.
by drewcoo | 2025-06-08 08:16:36+00:00
>> Back then, using audio tapes to hold data was revolutionary, in terms of storage space and price.

> Those aren't ordinary cassettes

Years earlier, my Color Computer (from that other leather company, Tandy) was reading and writing to ordinary cassettes (also sold by Radio Shack).

The folks I knew who could afford Apple IIs had the cash to spend on floppy drives but I think their computers supported cassettes.

I had friends in pre-Adam days who had Commodores and one with a TI-99, all of which had plain old cassette storage, too. In the 70s-80s that was slow but affordable. This was anything but revolutionary for the Adam.
by sweetjuly | 2025-06-08 08:17:17+00:00
The point they draw is that the results are non sequiturs. Reasoning models produce chains of thought (and are sometimes even correct in the chain of thought) and still produce a wrong, logically inconsistent answer. The extreme examples of this is giving the model a step by step guide to complete the program (the Towers program) and it being unable to produce answers which are consistent with the provided plan. So, not only is it unable to produce robust chain of thought, even were it correct and explicit, it cannot mash this information into a reasoned response.
by taklimakan | 2025-06-08 08:18:08+00:00
This unfortunately resonates with my experience. I used to attend GoLab, the event held in Florence, Italy.

Over time, as the event gained in popularity, speakers shifted from developers to developer advocates.

Last time I attended (2023 edition, IIRC) one of the talks was from some Intellij dev-relations person: the talk was about the importance of unit testing or something like that, completely devoid of any value for anyone who hasn’t started coding yesterday. The speaker was obviously not a technical person and gave a strong impression they were presenting content they barely knew anything about. I seriously doubt the slides deck was put together the night before. It was an utter disappointment. That was it for me. That’s a glaring example of what the OP is talking about but other talks I had listened to were only marginally better or on par with that one. That’s when I decided the thing was so much not worth the (hefty) ticket price + cost of transportation + accommodation.

It’s a sad state of affairs but not unexpected.
by genewitch | 2025-06-08 08:18:34+00:00
PCem does it relatively pain free; also emulates voodoo 2 so you can play GPU accelerated games on it. The network is pig-slow and i haven't figured out why, though. All in all PCem is the exact amount of jank and awesome to use for retro-emulation.

it emulates ~8086 through Pentium II or so. maybe a bit further on both sides; my machine struggled to maintain 100% emulation speed with the highest end CPU selected.
by threeseed | 2025-06-08 08:19:16+00:00
> aha! I told you they are useless

You said this. Neither Apple nor the author did.

The focus was specifically on LLM's reasoning capabilities not whether they are entirely useless or not.

This is relevant because countless startups and investment is predicated on LLM's current capabilities being able to be improved and built on top of. If it is a technological dead-end then we could be in for another long lull in progress. And companies like OpenAI should have their valuations massively cut.

It also constrains the level of investment Apple would need to be comparable to top tier LLM companies.
by elmerfud | 2025-06-08 08:19:29+00:00
They are getting some coverage but in no way is it extensive. News is still reporting that it's illegal immigrants being picked up, and they act like it's the exception that legal immigrants have been deported. Because they only mention one or two high-profile ones. Not the thousands upon thousands of legal immigrants that are detained and being deported that do not have the money or the luck to get picked up by the ACLU so their case can be heard.

Also I'm hard pressed to find any news media that's talking about how immigrants are shifted from the detention facility to detention facility, to actively deny them their day in court to hear why they're being detained. So go ahead show me the extensive coverage because it doesn't exist. The only people who seem to know about what's going on are those who have people who have been arrested by ICE for no reason and everyone else pretends like it must be the exception and an accident because that's how the news media is reporting it.

The occasional government, oh my bad we didn't mean to do that, but most of the people being picked up are legal and documented immigrants because how else would they know where to go get them at. Basic logic says if you're undocumented they don't know where you're at.
by IshKebab | 2025-06-08 08:20:30+00:00
Sure but VCs always exaggerate. Remember the dot com bubble? Or "there's an app for that"?

The internet and smartphones were still extremely useful. There's no need to refute VC exaggeration. It's like writing articles to prove that perfume won't catch you Bradd Pitt. Nobody literally believes the adverts but that doesn't mean perfume is a lie.

I'm not saying that VCs only push good ideas - e.g. flying cars & web3 aren't going to work. Just that their claims are obviously exaggerated and can be ignored, even for useful ideas.
by macrolime | 2025-06-08 08:21:40+00:00
From a quick glance it seems to be about spatial reasoning problems. I think there is good reasons for why it's tricky to become extremely good at these from being trained on text and static images. Future models being further multimodally trained with video and then physics simulators should deal with this much better I think.

There's a recent talk about this by Jim Fan from Nvidia https://youtu.be/_2NijXqBESI
by therein | 2025-06-08 08:22:14+00:00
It is an interesting read. I can imagine a future where the "tools" we make available become numerous enough and poorly thought out enough that an AI could actually figure out how to escalate privileges and execute stuff outside the defined security boundaries by combining them.

It isn't hard to think of a simple example in which Claude.md can be written to by the LLM to allow accessing endpoints not whitelisted by the user by smuggling a base64 encoded payload that then gets decoded by a subroutine it wrote to a file without you noticing. Or realizing it can't use the WebFetchTool but it can write a script to do manual DNS resolution and then use bash TCP sockets instead of curl in case it is hardened to not be able to use curl.
by closewith | 2025-06-08 08:22:41+00:00
> A 15 meter turning radius is tight, but not revolutionary. San Francisco's MUNI system's tightest turns are 45 feet, or 13.7 meters.

Revolutionary as the turning circle will be used at speed with passengers to traverse roundabouts in-lane. CVLR doesn't need turning loops, as the vehicles can be driven from either end.
by zmgsabst | 2025-06-08 08:25:56+00:00
Citing a few points to justify my own conclusion:

> Many (not all) humans screw up on versions of the Tower of Hanoi with 8 discs.

> LLMs are no substitute for good well-specified conventional algorithms.

> will continue have their uses, especially for coding and brainstorming and writing

> But anybody who thinks LLMs are a direct route to the sort AGI that could fundamentally transform society for the good is kidding themselves.

I agree with the assessment but disagree with the conclusion:

Being good at coding, writing, etc is precisely the sort of labor that is both “general intelligence” and will radically change society when clerical jobs are mechanized — and their ability to write (and interface with) classical algorithms to buttress their performance will only improve.

This is like when machines came for artisans.
by whstl | 2025-06-08 08:26:24+00:00
Not Meta but I once got yelled at not by a real manager, but by a PM because I said I wouldn't let the team do something shady without legal signing off. I'm in Europe so it was GDPR related.

The PM tried shopping the task to other teams, but nobody took the bait after I raised it publicly, and both legal and the external law firm sided with me after about three months of delay.

In the meantime I raised the topic of yelling with HR but every step of the way the company made me feel like I was the one in the wrong for not complying.

I believe if I were meeker I would probably have complied right there.
by seanalltogether | 2025-06-08 08:27:25+00:00
This is one area I will give Apple credit for. While their app store is a pain to navigate, their framework support and dev patterns have stayed pretty consistent since the early days of iOS. My companies iOS app today looks very similar to how it looked 9 years ago when we started it, aside from changing from objc to swift, it's still driven by core data, view controllers and storyboard views. Our android app on the other hand is kind of a Frankenstein app that started with Activities, Java, Sqlbrite, Butterknife, manual state management and Dagger, and has mostly transitioned to Fragments, Kotlin, Room, Data Binding, ViewModels and no Dagger. It's still quite a mess that we'll never have the budget to fix up properly.
by sneak | 2025-06-08 08:29:11+00:00
Same thing at Google or Apple. Google has everyone’s email and browsing history, Apple has the complete copy of everyone’s iMessage and SMS history (in the non-e2ee iCloud backups, readable by Apple).

Anything these companies know, the FBI and CIA can know, without a warrant thanks to FAA702 (did we all forget about PRISM?).

The state now has leverage over almost every normal citizen, thanks to what these companies have built.

Turnkey tyranny. Built by silicon valley.
by brabel | 2025-06-08 08:29:26+00:00
Bugs like these are pervasive in languages like Java that give no protection against even the most basic race condition causes. It’s nearly impossible to write reliable concurrent code. Freya only helps if you actually use it to test everything which is not realistic. I am convinced, after my last year long struggle to get a highly concurrent Java (actually Kotlin but Kotlin does not add much to help) module at work, that we should only use languages that provide safe concurrency models, like Erlang/Elixir and Rust, or actor-like like Dart and JavaScript, where concurrency is required.
by closewith | 2025-06-08 08:29:52+00:00
CVLR is projected to cost £10 MM per km, but we all know what projections are worth.

Still, there's no point comparing build costs between France and the UK as they're completely different cultures and jurisdictions. Instead, a more reasonable approach is to compare to recent similar project in the UK.

Edinburgh's tram covers 18.5 km and cost at least (they're still uncovering overruns years later) £1 BN. That's ~£50 MM or ~€60 MM per km. That's what CVLR should and will be benchmarked against.
by tsurba | 2025-06-08 08:30:26+00:00
I would argue the other way around. I have ADHD, but the thing that really helped me with work procrastination, which I think would help even without ADHD, was to find a job that is actually interesting.

In approx 7 years I went through working at all the top software companies in my country, but what really fixed my problems was moving on to being a researcher at the university. I’m now paid less than half from before, but it’s still enough, and I couldn’t be happier.

Getting to work on what I think is actually important and interesting every day is what helped. I also seem happier than the younger researchers who didn’t work at companies first, who don’t know how good they have it.
by safety1st | 2025-06-08 08:31:41+00:00
I'm trying to understand what you're asking for that isn't available. Reliable portable storage? Isn't that what a portable SSD is? They start at like $60 for 1TB and get way cheaper per TB as you go up. Are you talking about a gaming-specific storage product/complaint specifically?

If the complaint is that large game distributors work to make it hard for you to store the bits on that SSD, yeah I totally hear you.
by Lerc | 2025-06-08 08:31:47+00:00
There are quite a lot of options out at the extremes and both ends seem to presuppose things about consciousness that even specialist's in the field have debated for years.

I'm ok with thinking it's possible that some subset of consciousness might exist in LLMs while also being well aware of their limitations. Cognitive science has plenty of examples of mental impairments that show that there are individuals that lack some of the things that LLMs also lack that. We would hardly deny those individuals are conscious. The distinction for what is thought is lower but no less complex.

Before we had machines pushing at these boundaries, there were very learned people debating these issues, but it seems like now some gut instincts from people who have chatted to a bot for a bit are carrying the day.
by youngtaff | 2025-06-08 08:33:28+00:00
Efficiency — mainly through lower friction

“Thanks to the very low friction between the steel wheel and steel rail, railways can move a load using as little as 15% of the energy that road haulage needs.”

Gareth Dennis, How Railways will Fix the Future (https://www.penguinrandomhouse.com/books/761930/how-the-rail...)
by hoseyor | 2025-06-08 08:34:25+00:00
The irony of the illusion is that even though “the iPhone is locked down”, practically speaking it is more your property than an android, unless things have changed since I last really cared about Android, even every single update that was pushed on you incrementally made the device and user experience slower and worse, to the point where basically your phone would accumulate performance lag incrementally and one year in you would be thinking about a new Android device. Meanwhile, a several years old iPhone basically has far closer performance to what it was on day one, even if not perfect.

It is yet another one of those oversights that people have just ignored, that when you buy a product like a phone, there is absolutely zero reason why it should not perform at the very least just as well as it did on day one if all the other conditions are the same, i.e., you have a choice about including any additional features. If you do not have a choice, e.g., because an update imposes some feature, then what you have on your hands is really just a company damaging your property.

Imagine if you took your car into mandatory services or if Tesla pushed mandatory updates that made your acceleration, breaking, turning response times slower and increased your gasoline or electric consumption; and then tried telling your how fast and smooth and efficient the new model is that behaves the same way that the old one did when you bought it.

It’s just fraud! If your update diminishes the performance and function of core capabilities, then you are liable for those damages, because you caused damages.