MurphyA/hacker-news huggingface dataset:
by tomnicholas1 | 2026-01-17 16:22:14+00:00
The generalized form of this range-request-based streaming approach looks something like my project VirtualiZarr [0].

Many of these scientific file formats (HDF5, netCDF, TIFF/COG, FITS, GRIB, JPEG and more) are essentially just contiguous multidimensional array(/"tensor") chunks embedded alongside metadata about what's in the chunks. Efficiently fetching these from object storage is just about efficiently fetching the metadata up front so you know where the chunks you want are [1].

The data model of Zarr [2] generalizes this pattern pretty well, so that when backed by Icechunk [3], you can store a "datacube" of "virtual chunk references" that point at chunks anywhere inside the original files on S3.

This allows you to stream data out as fast as the S3 network connection allows [4], and then you're free to pull that directly, or build tile servers on top of it [5].

In the Pangeo project and at Earthmover we do all this for Weather and Climate science data. But the underlying OSS stack is domain-agnostic, so works for all sorts of multidimensional array data, and VirtualiZarr has a plugin system for parsing different scientific file formats.

I would love to see if someone could create a virtual Zarr store pointing at this WSI data!

[0]: https://virtualizarr.readthedocs.io/en/stable/

[1]: https://earthmover.io/blog/fundamentals-what-is-cloud-optimi...

[2]: https://earthmover.io/blog/what-is-zarr

[3]: https://earthmover.io/blog/icechunk-1-0-production-grade-clo...

[4]: https://earthmover.io/blog/i-o-maxing-tensors-in-the-cloud

[5]: https://earthmover.io/blog/announcing-flux
by ben_w | 2026-01-17 16:22:42+00:00
> What they all seem to be just glossing over is how the project unfolded: without human intervention, using computers, in an exceptionally accelerated time frame, working 24hr/day.

The reason I have yet to publish a book is not because I can't write words. I got to 120k words or so, but they never felt like the right words.

Nobody's giving me (nor should they give me) a participation trophy for writing 120k words that don't form a satisfying novel.

Same's true here. We all know that LLMs can write a huge quantity of code. Thing is, so does:

yes 'printf("Hello World!");'

The hard part, the entire reason to either be afraid for our careers or thrilled we can switch to something more productive than being code monkeys for yet-another-CRUD-app (depending on how we feel), that's the specific test that this experiment failed at.
by torginus | 2026-01-17 16:23:27+00:00
I think the main issue is lack of competition. If company A makes a 10% worse car that costs 10% more than company B, they very quickly go out of business.

There is no 'Reddit 2' substitute product (or indeed for lots of software), and network effects tend to dominate, so your benchmark is 'is it bad enough so people would rather use nothing than your product', which is a very low bar to clear.
by glitchc | 2026-01-17 16:23:30+00:00
> Even then this seems much more promising to me compared to other areas. Writing theorem statements is much much easier than coming up with proofs so it's not a big deal if a human has to do that.

Not at all. Theorem crafting is hard. What's easy is standing up and proving a mathematical strawman which may or may not have any relation to the original problem.
by Der_Einzige | 2026-01-17 16:23:38+00:00
Who were the primary class of people drawn to the SS and the SA? At least in the SA's case it was working class to lower middle class people.

Also, so many reds (as in communists) became fascists it was a meme in Nazi Germany.

https://en.wikipedia.org/wiki/Beefsteak_Nazi

I don't think white collar tech workers are uniquely predisposed to fascism. Blue collar tradesmen are more likely to be disposed to it and capable of getting their hands dirty.
by Lions2026 | 2026-01-17 16:23:46+00:00
I put together a small reference implementation of a deterministic settlement
control layer for systems that rely on external outcome resolution
(oracles, APIs, AI agents, or humans).

The goal is to prevent premature or conflicting payouts by enforcing
finality, reconciliation, and exactly-once settlement semantics.

This is intentionally minimal and includes runnable scenarios
(clean resolution, conflicts, late signals, and majority-oracle cases).

I’m looking for technical feedback from people who’ve dealt with settlement
disputes, replay issues, or payout integrity in real systems.

I put together a small reference implementation of a deterministic settlement
control layer for systems that rely on external outcome resolution
(oracles, APIs, AI agents, or humans).

The goal is to prevent premature or conflicting payouts by enforcing
finality, reconciliation, and exactly-once settlement semantics.

This is intentionally minimal and includes runnable scenarios
(clean resolution, conflicts, late signals, and majority-oracle cases).

I’m looking for technical feedback from people who’ve dealt with settlement
disputes, replay issues, or payout integrity in real systems.

I put together a small reference implementation of a deterministic settlement
control layer for systems that rely on external outcome resolution
(oracles, APIs, AI agents, or humans).

The goal is to prevent premature or conflicting payouts by enforcing
finality, reconciliation, and exactly-once settlement semantics.

This is intentionally minimal and includes runnable scenarios
(clean resolution, conflicts, late signals, and majority-oracle cases).

I’m looking for technical feedback from people who’ve dealt with settlement
disputes, replay issues, or payout integrity in real systems.
https://github.com/azender1/deterministic-settlement-gate
by JohnFen | 2026-01-17 16:23:59+00:00
I agree.

Also, I have become very suspicious of articles like these. There are quite a lot them these days. It's just odd that so many people are taking such an extreme interest in pressuring other people to make the "right" decisions, cloaked in a selfless and self-righteous tone.

It smells like a PR campaign to me.
by utopiah | 2026-01-17 16:24:04+00:00
Thanks for the wiki link, very interesting, in particular

- the long tail aspect of the problem space ; 'a "long tail" of under-explored problems at the other, many of which are "low hanging fruit" that are very suitable for being attacked by current AI tools'

- the expertise requirement, literature review but also 'Do I understand what the key ideas of the solution are, and how the hypotheses are utilized to reach the conclusion?' so basically one must already be an expert (or able to become one) to actually use this kind of tooling

and finally the outcomes which taking into consider the previous 2 points makes it very different from what most people would assume as "AI contributions".
by deaux | 2026-01-17 16:24:15+00:00
Ive used Langfuse. It's completely unrelated to tools like Langchain and Autogen. It's just logging/tracing for LLMs. Sure they added stuff like "prompt management" and "epxeriments" etc. probably to keep investors happy but those are entirely optional sidedishes.

The tools you mentioned are indeed to be avoided. I trialed them early on and quickly realized in 99.9% they do nothing but bog you down. Pretty sure they'll be dead sooner rather than later.
by deaux | 2026-01-17 16:24:49+00:00
Correct! Will be moving away immediately for this reason.

Or well, technically incorrect, as someone will surely point out. US companies can be legally compliant with GDPR, it's just that the likes of the CLOUD Act and FISA make it completely meaningless.

Before anyone comes in talking about how it's farfetched that those matter, it's 100x as far-fetched that self-hosted Chinese LLM models would exfiltrate your data (you can even airgap them) yet 90% of corporate America is avoiding them based solely on the country they were trained in. Compared to that insanity, above US acts are a very real threat.

And that's of course on top of that now an adversarial state's company has the power to immediately dissolve Langfuse.
by mwillis | 2026-01-17 16:25:31+00:00
Fantastic technique and deep dive. I will say, I was hoping to see an improved implementation of the Cognition cube array as the payoff at the end. The whole thing reminded me of the blogger/designer who, years ago, showed YouTube how to render a better favicon by using subpixel color contrast, and then IIRC they implemented the improvement. Some detail here: https://web.archive.org/web/20110930003551/http://typophile....
by pron | 2026-01-17 16:25:31+00:00
A fundamental problem is that program verification is intractable in the computational complexity sense. The question of whether a program satisfies a spec or not is called in the literature the model checking problem (not to be confused with model checkers, which are various algorithms for automatically solving some model checking problems). In the worst case, there is no faster way to determine whether a program satisfies a spec or not than explicitly testing each of its reachable states.

The question is, of course, what portion of programs are much easier than the worst case (to the point of making verification tractable). That is not known, but the results are not necessarily encouraging. Programs that have been deductively verified proof assistants were not only very small, and not only written with proofs in mind, but were also restricted to use simple and less efficient algorithms to make the proofs doable. They tend to require between 10x and 1000x lines of proof per line of code.

(an old blog post of mine links to some important results: https://pron.github.io/posts/correctness-and-complexity)

There is a belief that programs that people write and more-or-less work should be tractably provable, as that's what allowed writing them in the first place (i.e. the authors must have had some vague proof of correctness in mind when writing them). I don't think this argument is true (https://pron.github.io/posts/people-dont-write-programs), and we use formal methods precisely when we want to close the gap between working more-or-less and definitely always working.

But even if verifying some program is tractable, it could still take a long time between iterations. Because it's reasonable that it would take an LLM no less than a month to prove a correct program, there's no point in stopping it before. So a "practical" approach could be: write a program, try proving it for a month, and if you haven't finished in a month, try again. That, of course, could mean waiting, say, six months before deciding whether or not it's likely to ultimately succeed. Nevertheless, I expect there will be cases where writing the program and the proof would have taken a team of humans 800 years, and an LLM could do it in 80.
by Aurornis | 2026-01-17 16:25:41+00:00
In theory, the name is supposed to imply that they're pursuing rational thinking and philosophies, not that their decisions are the rational choice.

That said, I was surrounded by rationalists in my younger years by pure coincidence and spent some time following the blog links they sent and later reading the occasional LessWrong thread or SSC comment section that they were discussing each day in chat.

It's pretty easy to see that the movement attracts a lot of people who have made up their minds but use rationalisim as a way to build a scaffold underneath their pre-determined beliefs in a way that sounds correct. The blogs and forums celebrate writing of a certain style that feels correct and truthy. Anyone who learns how to write in that style can get their ideas accepted as fact in rationalist communities by writing that way. You can find examples throughout history where even the heroes of the rationalist movement have written illogical things, but they've done it in the correct way that makes it appear to be "first principals" thinking with a "steelmanning" of the other side along with appropriate prose to sound correct to rationalists.
by lkjdsklf | 2026-01-17 16:25:43+00:00
The real question is, do you even need to fix it? Does it matter?

The reason those things matter in a traditional project is because a person needs to be able to read and understand the code.

If you're vibe coding, that's no longer true. So maybe it doesn't matter. Maybe the things we used to consider maintenance headaches are irrelevant.
by smj-edison | 2026-01-17 16:26:02+00:00
Yeah. It's really important that you have a platform with your users, because that's how you introduce them to features (not always a bad thing). It becomes a touch point with your customers. Google and Apple have their phones, Meta is desperately trying to get their smart glasses working, Amazon is also desperately trying to get something together with Alexa, and Microsoft is... Throwing theirs away?
by dahart | 2026-01-17 16:26:09+00:00
Yes!! The software market is still growing quickly.

What moat are you referring to? AFAICT, moats are bigger now than ever before, but the moats I see are centered around things like consumer habits and B2B contracts and hub/platform experiences. People tend to like the software they know & have, or the contractors they know already, even when they complain about it. The need to overcome static friction is a moat, one that makes it harder for startups to get in, but easier to stay once you’re in.

Big companies are slow. It seems like it would be easy for them to copy products, but they rarely do, and even when they try they take forever. Worry about yourself first, whether you can deliver, whether your product is superior, whether you can get customers, whether your business is sustainable, etc.. Worry about other startups second. They can copy faster than big companies. Do you have an edge over other startups? Don’t worry about the big companies. If they aren’t already solving the problem you want to solve, they won’t start until after you’re successful.

Note acquisitions are a thing! Big companies that want to copy your product are aware that the fastest way to do that is to acquire a working solution and a pile of new customers. You should be thinking about big companies more as opportunity than competition.
by dpark | 2026-01-17 16:26:18+00:00
> Accepting white inferiority?

Because if you reject white supremacy, obviously the implication is white inferiority…

White supremacists generally deny that they have societal advantages and frame any attempt to give minorities equal opportunities as a plot to subjugate whites.

> Full blown resistance marketing market?

MAGA and the rise of neonazis.
by mancerayder | 2026-01-17 16:26:36+00:00
Unfortunately this is the most dangerous when it is most needed - on the New York City subway, especially when the train has a cell signal, when jackasses listen to reels of IG and TikTok and scroll at high volume.

It's an incredibly rude world we live in - this behavior two decades ago would have led to a fight. Now everyone is scared to tell anyone anything, since punching or stabbing are risks. Also, I believe the new generation is hyper tolerant of such things compared to us olds.
by thomascountz | 2026-01-17 16:27:04+00:00
I share your awe of the NTSB and their reports. Here's my favorite quote from the Alaskan Airlines door plug incident[1]:

We determined that the probable cause of this accident was the in-flight separation of the left MED plug due to Boeing’s failure to provide adequate training, guidance, and oversight necessary to ensure that manufacturing personnel could consistently and correctly comply with its parts removal process, which was intended to document and ensure that the securing bolts and hardware that were removed to facilitate rework during the manufacturing process were properly reinstalled.

[1]: https://www.ntsb.gov/investigations/Pages/DCA24MA063.aspx
by microtonal | 2026-01-17 16:27:47+00:00
I am not sure where you live, but I cannot remember the last time our power went out (Western Europe).

I have gas-cooked since I was a kid (living in an area with a lot of natural gas, so houses were connected to gas since the 50ies), but induction is so much nicer that I'm happy to not be able to cook during a once in a ~10-20 year outage. Also a lot safer (it still happens quite frequently that a house blows up because of a gas leak, just this week there was a huge explosion in Utrecht what was presumably a gas leak).

Of course, the equation may change for countries with less stable power.
by dijit | 2026-01-17 16:27:55+00:00
Spot on. And it's worse than that: you're not choosing between a 10% slower product and a faster alternative, you're choosing between a slow product and losing access to everyone still using it. That's not a market choice, it's a hostage situation.

We can see this works in reverse: developer tools, CLIs, and local apps where network effects don't apply (ripgrep over grep, esbuild over webpack) performance actually matters and gets rewarded. Developers switch because they can switch without losing anything. But Instagram users can't switch to a lighter alternative without abandoning their social graph.

This is why the "developer time is expensive, user hardware is cheap" argument only works in the absence of competition. In genuinely competitive markets, efficient code becomes a competitive advantage worth investing in. The fact that it's "not worth optimising" is itself evidence of market power, not sound economics.

Your automotive analogy actually understates it: imagine if switching to a better car meant your old car's passengers couldn't ride with you anymore, and that's closer to what we're dealing with.
by CodingJeebus | 2026-01-17 16:28:17+00:00
> Don't take it personal. All business want to reduce costs. As long as people cost money, they'll want to reduce people.

"Don't take it personal" does not feed the starving and does not house the unhoused. An economic system that over-indexes on profit at the expense of the vast majority of its people will eventually fail. If capitalism can't evolve to better provide opportunities for people to live while the capital-owning class continues to capture a disproportionate share of created economic value, the system will eventually break.
by nospice | 2026-01-17 16:28:24+00:00
>> AI designed for use by children and teens should be less sycophantic and more "antagonistic"

> Genius. I love this idea.

I don't think it would really work with current tech. The sycophancy allows LLMs to not be right about a lot of small things without the user noticing. It also allows them to be useful in the hands of an expert by not questioning the premise and just trying their best to build on that.

If you instruct them to question ideas, they just become annoying and obstinate. So while it would be a great way to reduce the students' reliance on LLMs...
by blahnjok | 2026-01-17 16:28:26+00:00
> We’re still in that same fundamental situation. We have better tools—vastly better tools—but the thinking remains essential.

But less thinking is essential, or at least that’s what it’s like using the tools.

I’ve been vibing code almost 100% of the time since Claude 4.5 Opus came out. I use it to review itself multiple times, and my team does the same, then we use AI to review each others’ code.

Previously, we whiteboarded and had discussions more than we do now. We definitely coded and reviewed more ourselves than we do now.

I don’t believe that AI is incapable of making mistakes, nor do I think that multiple AI reviews are enough to
understand and solve problems, yet. Some incredibly huge problems are probably on the horizon. But for now, the general “AI will not replace developers” is false; our roles have changed- we are managers now, and for how long?
by harrisi | 2026-01-17 16:28:59+00:00
Just to be clear, I think the situation is disgusting, entirely unprofessional, and intentionally violent. Maybe the shooter didn't make those mistakes to have a reason for murder, but the idea that someone of his experience would make such a completely foolish mistake is absurd.

Nobody, as far as I know, has any intention of hitting me with their car. Yet, for about a dozen simple reasons, if I'm crossing a walkway, driveway, or whatever with a car also trying to enter the road by driving through the area I'm walking, I often will walk behind the vehicle. People make mistakes. There are blind spots. Maybe they're having an emergency. Maybe they're intoxicated. Or maybe they do want to hit someone with their car.

It's just absurd to think this was appropriate behavior by a seasoned professional. It's not even the appropriate behavior for a reasonably developed child.
by torginus | 2026-01-17 16:30:09+00:00
I think people overestimate the necessity of 'high level' conveniences or the difficulty of writing C/C++ to the metal.

For example, take Dear.IMGUI which is a c++ UI framework with a kind of data binding, which generates vertex buffers which can be directly uploaded to the GPU and rendered.

It supports most of the fancy layout stuff of CSS afaik (flexbox etc), yet its almost as low level as it gets.

The code is also not that much harder to write than React.
by maryPoppins23 | 2026-01-17 16:30:31+00:00
This is beautiful. By inferring exp() gets exp()', the author's
`metabolizing' eventually life's shocking and upstart singularities,
we are on are way to AGI if and only if exp() is, drum roll please..

Concious!

This is a rhetorical question:

"Well why not?"

Twelve million tech educated Chinese who do actually work for a living,
could not care less about this or whatever possible hn comments say,
but if they do, may they also metabolize Hegseth's DOW still has the B-52.
by spockz | 2026-01-17 16:30:32+00:00
I think NG outperforms in high efficiency heaters when the outside temperature is around 1-4 degrees Celsius with humidity as it causes ice buildup on the external unit which then has to be cleared using resistive heating. Also if only little hot water is required sporadically, heating it just in time with gas is more efficient than keeping a buffer heated for long times.

Also, heat pumps do best when the temperature differential is lower. So in older housing without floor heating or duct heating, it is typically not as efficient to use a heat pump when the water to heat has to be above 55 degrees Celsius.

For any new residential construction I think there is very little value in natural gas.
by mindslight | 2026-01-17 16:31:14+00:00
The problem is the capital cost of any of that type of equipment sitting around idle or under-capacity, ready to go when the electricity price goes down. It's likely more profitable to run them most of the time, even with positive electric rates, and then only stop using them when rates are exceptionally high ("load shedding").

This is why you see most opportunistic electricity usage systems doing resistive heating - this equipment is inexpensive.
by simonw | 2026-01-17 16:31:16+00:00
> I believe that explicitly teaching students how to use AI in their learning process

I'm a bit nervous about that one.

I very firmly believe that learning well from AI is a skill that can and should be learned, and can be taught.

What's an open question for me is whether kids can learn that skill early in their education.

It seems likely to me that you need a strong baseline of understanding in a whole array of areas - what "truth" means, what primary sources are, extremely strong communication and text interpretation skills - before you can usefully dig into the subtleties of effectively using LLMs to help yourself learn.

Can kids be leveled up to that point? I honestly don't know.
by MontyCarloHall | 2026-01-17 16:31:45+00:00
Were many of the aforementioned advancements marketed as "replacing developers"? Absolutely. Did that end up happening? Quite the opposite; each higher-level abstraction only caused the market for software and demand for developers to grow.

That's not to say developers haven't been displaced by abstraction; I suspect many of the people responsible for re-wiring the ENIAC were completely out of a job when punchcards hit the scene. But their absence was filled by a greater number of higher-level punchcard-wielding developers.