MurphyA/hacker-news huggingface dataset:
by mingtianzhang | 2025-04-07 08:56:21+00:00
Hi Bsenftner, thanks for your interest.

The motivation behind building PageIndex is to build a reasoning-based RAG. When we previously designed a RAG system for financial documents, we encountered two main challenges:

1. Traditional embedding-based RAG often returns redundant information because all the financial terms are semantically similar.

2. We want to incorporate expert experience into the RAG process—specifically, experts often have a preferred order of where to look first.

To address these, we developed PageIndex, which transforms long documents into a structured “table of contents.” This allows the LLM to selectively retrieve relevant nodes based on reasoning. With this approach, we can do few-shot learning by providing examples of expert preferences directly in the prompt, enabling the LLM to choose nodes more like a domain expert would.

-----

In your case, it sounds like you're looking for a system that can automatically build and continuously update a knowledge base as new data arrives. You might benefit from something like:

1. Using expert knowledge to define a template knowledge graph—e.g., specifying entity types, link types, or a rough graph structure.

2. Building an agent that updates the knowledge graph when new documents are received. The agent’s tasks could include:

a. Identifying new information relevant to existing nodes or links.

b. Determining whether this new information changes the current knowledge graph.

c. Updating the graph accordingly.

Since your use case involves logical reasoning (not just semantic similarity), PageIndex and reasoning-based RAG could play a helpful role here. In other words, while a traditional graph-based RAG might still be used at inference (question answering) time, PageIndex and reasoning-based RAG can assist during the knowledge graph update phase by identifying related information in the new documents that are related to the graph. Additionally, the tree structure produced by PageIndex can be used as an initialization for building your knowledge graph.

Hope this is helpful! Mingtian
by DLion | 2025-04-07 08:56:22+00:00
So did they just discover pair programming? :D

Also, I'm reading lots of comments that are pointing out that they don't like doing pair programming because they do it wrongly.

In the past I wrote an article about it: https://domenicoluciani.com/2022/07/22/misleading-pair-progr..., I hope it helps to clarify some concepts behind this way of working
by hnlmorg | 2025-04-07 08:56:29+00:00
> Not to argue the point, but in the article, he can't get the snare sound right so he needed to revert to a sample.

That’s a different problem. He is trying to exactly recreate a sample but without using the sample.

You wouldnt do that in released music because if you didn’t have clearance to use that sample then you’d be still breaking copyright law if you were to recreate it exactly. So what you’d do is recreate the essence of that sample but without making it exactly the same.

Also you ignored my point that professional studios have access to more resources and professionals too.

> I think you're underestimating the difficulty of recreating sounds.

I never said it was easy ;)

> It would be impossible for instance, to recreate the amen break with a sequencer because nothing would be on a quantized beat and every single drum hit is different.

Sequencers don’t quantise beats. You’re thinking of trackers and they are a completely different beast.

You can have your samples aligned any which way you like in sequencers. And even if you were to user a drum machine, you can add padding to your samples. Some drum machines also support dequantisation where they deliberately trigger the samples off beat to give the drums a more natural sound.

To give an example of how sequencers don’t quantise: I was hopelessly bad at producing and some of my earliest efforts had little to no quantization at all (if you can forgive the made up word) and thus I required an engineer to come in an redo most of what I did but have those samples triggered on beat. So I’m very familiar with how easy or hard it is to get drums aligned and how quantisation works in a DAW.

I did get better at producing as time went on but never good enough to even consider making a career out of it.
by InsideOutSanta | 2025-04-07 08:57:45+00:00
>intelligent culling makes it easier to find things

That's true, but things get easier to find over time. I have thousands of digital photographs from the 1990s that I burned on CDs and then copied to a NAS later. Today, for the first time, they're actually searchable because they're now indexed in Immich. So they're sorted by the person in them and searchable by their contents.

If I had culled the photos back then, I would have lost many photos that have become searchable (and thus valuable) only recently.
by m000 | 2025-04-07 08:57:49+00:00
I believe this might have been the case in the past, where (a) sensor resolutions were lower - so the raw images less bulky, (b) camera CPUs were slower - so you would like to take them out of the equation.

These days, the bottleneck for achieving continuous shooting rate is probably writting to the sd card (which is the standard for the consumer/pro-sumer models).
by ViewTrick1002 | 2025-04-07 08:57:55+00:00
Safe rust prevents you from writing data races. All concurrent access is forced to be guarded by synchronization primitives. Eliminating an entire class of bugs.

You can still create a mess from logical race conditions, deadlocks and similar bugs, but you won’t get segfaults because you after the tenth iteration forgot to diligently manage the mutex.

Personally I feel that in rust I can mostly reason locally, compared to say Go when I need to understand a global context whenever I touch multithreaded code.
by arkh | 2025-04-07 08:58:34+00:00
> Of course this means >200ms to even hit the server when requests don't come from the same data center.

What world are you living in?

When using a European server from Europe I expect < 10ms ping. Less than 100ms for East coast server, less than 150ms for west coast ones. Only cases I'd expect more than 200ms (and not a lot more) is when hitting Australian servers.
by StackRanker3000 | 2025-04-07 08:59:11+00:00
I’m guessing this is the article you submitted: https://arstechnica.com/tech-policy/2025/04/eu-may-make-an-e...

This is the first sentence in it, which links to the New York Times article the submission was changed to:

> European Union regulators are preparing major penalties against X, including a fine that could exceed $1 billion, according to a New York Times report yesterday.

These are the Hacker News guidelines (you’ve been on here for 18 years, perhaps time to give them a read): https://news.ycombinator.com/newsguidelines.html

> Please submit the original source. If a post reports on something found on another site, submit the latter.

This isn’t creepy, this is in line with the rules.
by vintermann | 2025-04-07 09:00:34+00:00
There are many ways to define opposites. This author demands that the "dark mirror" person wants the exact opposite, but that is way too specific in my opinion.

Then it's more interesting with cases like William Cowper. Who certainly believed in anything a Christian in his time and place was supposed to believe, including double predestination. It's just that unlike most who believed that, he was convinced he was predestined for hell. He didn't go around doing evil things, of course, what would be the point of that? But he was very, very distressed.

Likewise, I think the author underestimates how much Marxist historical materialism was viewed as a hard science in much of the world, until the cold war. There were certainly capitalists who were terrified it was true, but could not see any way out of it for themselves, because just like Cowper's double predestination, orthodox Marxism denies that there is such a thing.
by petre | 2025-04-07 09:01:22+00:00
MacOS userspace was forked from FreeBSD, that's why it bundles non-GNU extensions. Also the FreeBSD userspace has since then incorporated GNUisms.

Why they went with Bash 2 as the defualt shell is beyond me. I always switched to and used Zsh which had a more recent version. Now I'm also using it on Linux and FreeBSD, because I want a consistent shell.
by FirmwareBurner | 2025-04-07 09:01:28+00:00
>I’ve also found some small companies have even worse politics and infighting than large companies.

Also matches my experience and I think I can explain why.

Small companies usually have tight social cliques (for better and for worse) which also translates to group think, more gossip, and less diversity of though as most people working there likely know each other form before or went to the same schools, worked at the same company before, etc.

Therefore if you don't click from the start with the core "gossip" group on all wavelengths, then your career prospects in the company are fucked.
by rob74 | 2025-04-07 09:02:27+00:00
That might be true, but creating the Amiga 500 and 2000 was also a no-brainer: the original Amiga (later retconned to A1000) had a detachable keyboard (= expensive), but limited expansion capabilities, so launching two new models, a cheaper all-in-one design and a more expensive version with detachable keyboard and a larger case with more expansion capabilities was pretty obviously the way to go.

What all the Commodore CEOs unfortunately failed to do was invest adequately into R&D to keep the Amiga competitive. Instead, they treated the existing lineup as a cash cow for far too long. The new models which hit the market from 1990 onwards were too little too late and couldn't fend off the eventual insolvency...
by stereolambda | 2025-04-07 09:02:39+00:00
I have another somewhat related observation. If you don't think or don't know how to think about your ideas your broadly (because the communication is saturated, or you lack any context), your opponents often get to dictate your beliefs. This is when you know either them or what their propaganda describes as the worst caricatural evil.

For a model example, you are a peasant in old Christian Europe, you don't know any learned humanism or anything like that, so you turn to witchcraft--as it is described by the preachers to you. Also it is common among young people who can be raised in a very controlled environment where the authority figures shut out any opposing views: so either you submit, or your only alternative is the (often objectively) horrible things they describe as the enemy. Of course they won't tell you there are other options besides that, and you are very unlikely to come up with them by yourself.

I would also say this plays a role in the society's spiral of internet radicalization. People rarely know basic boring political theory and ideologies, non-ragebait history post WW2 etc. If you are fully jacked into twitter-likes like that, your worldview is gonna largely consist of someone's Satans adopted out of ignorance and spite.
by whatnow37373 | 2025-04-07 09:02:57+00:00
I get what you are saying, but surely their definition of "peace" itself differs quite dramatically.

I venture to say they in fact are not after the same thing, while they may be using the same words. Their use of the word "evil/not good" only serves as a distraction as that is a word used to determine how aligned or misaligned a subject thinks it or some other subject is on our or its way to whatever goal we have in mind, but this doesn't mean the goal is the same if two actors use that word.

What Siths want is autocratic control for its own sake. If they could somehow have robots or whatever handle the galaxy's "peace", they wouldn't suddenly grab a martini and start relaxing on the beach singing beautiful songs of cosmic harmony. They would still exert control in their characteristic sadistic, vindictive ways. They are in it for themselves, they don't care about anything else and I'd argue they are incapable of knowing. They are wired that way. In a way it's sad. They want personal control and personal aggrandizement. When they use the word "peace" they mean "lack of resistence (to me)" and not "freedom from violence and liberty for all".

Jedis fundamentally want a different thing. They don't want to become personal dictators over a faceless populace subjected to their every arbitrary whim. They do want "freedom of violence and liberty for all". They want an order based on rules and those rules must be born out of concern for life itself, not the indiscriminate use of power to get whatever the hell you want when you want it. That's why Siths are obsessed with personal power, not because their aims are equivalent. It's because their aims are not equivalent.
by Sharlin | 2025-04-07 09:02:59+00:00
Sony is famous for having the worst interfaces of all the big camera manufacturers.

Lightroom most likely has “obsolete paradigms” for the same reason Photoshop does: because professionals want to use what they know rather than what is fashionable. Reprogramming their muscle memory is not something people want to be doing. Anyway, I find Lightroom’s UI very nice to work with.
by HenryBemis | 2025-04-07 09:03:48+00:00
A very friend of mine was going through a nasty divorce. Although we weren't talking about the divorce/case he set our messages on auto-delete. Apparently his (now, ex-wife) had SMS and WhatsApp messages in court to be used for her cased.

Any 'loving message' to her from the early days ("you are so perfect") and any 'nasty message' to others ("oh that bitch!!") sent to anyone was presented in court. So for caution he auto-deleted even the messages that were innocuous, just in case it could be used against him "oh he wanted to spend money for a new phone/laptop, thus he has money, thus I will take it"
by fastball | 2025-04-07 09:05:00+00:00
The comment I replied to was holding out the federal government as an especially stable employer, as compared to the private sector. Sure, you don't want a reputation for firing people a week after you hired them, but A. that is not what is happening here B. not doing that doesn't mean it is actually desireable for the federal government to be seen as a more stable alternative to the private sector.

Said another way: any company can have the reputation GC claims the federal government does, but most don't. Why is that?
by autoexec | 2025-04-07 09:05:01+00:00
The most universally hated companies are also among the richest. Voting with your wallet is a myth. The entire point of a private company is to confiscate your income. They must charge you as much as they possibly can while providing you with as little as they can possibly get away with. Maybe you've even noticed prices going up while enshittification and shrinkflation increases.
Scaling AI agent tooling: ideas for dynamic discovery and execution workflows
by artski | 2025-04-07 09:05:10+00:00
I've been working on designs for scaling tool use in AI agents and wanted to share some ideas for feedback.

The problem is familiar: as agents get access to more tools, either you overload the system with static tool lists, or you hard-code the tools at design time. Both approaches hit scaling limits quickly.

What I’ve been exploring is letting agents discover and use tools dynamically. Instead of loading every tool into the system upfront, the agent could explore an external registry at runtime, inspect tool descriptions and parameters, and decide what to use based on its current goal.

There are a few workflows I’m testing:

Manual exploration: agent lists tools, inspects them, and selects what to use.

Fuzzy auto-selection: agent describes its goal, and the system suggests a matching tool.

External LLM-assisted selection: agent outsources tool selection to another agent or service, which queries the registry and recommends a tool.

Each of these has trade-offs. Manual exploration is transparent but slow. Fuzzy matching is faster but depends on query quality. External assistance can scale to complex environments but adds system complexity.

The goal is to move away from static APIs and let agents explore tools like a developer browsing an API catalog. Search, inspect, and use tools as needed, rather than loading them all upfront.

Open questions I’m still thinking about:

Should these workflows be combined, escalating from manual to auto-selection if needed?

Should the system suggest parameter defaults or usage examples?

Should I move from basic matching to semantic search?

Would chaining tools together unlock more complex workflows?

I've written up a more detailed research note here, if anyone's interested: https://github.com/m-ahmed-elbeskeri/MCPRegistry/tree/main

Would appreciate any feedback or experiences if you've explored similar patterns.
by fakedang | 2025-04-07 09:05:48+00:00
Yes, but in our very recent history, the US has enabled policies that have driven countries to purchase using currencies other than the USD.

Only if you're buying direct from the gulf countries or you're buying on the spot markets are you forced to pay with the USD. For example, the EU had a long standing agreement to pay for Russian oil and gas with Euros, India has recently begun paying for Russian gas with Indian rupee, as does China with yuan. While Saudi Arabia and the rest of the oil dependent GCC are forced to curtail production to force prices up, Russia floods the oil market to account for the shortfall.
by varispeed | 2025-04-07 09:05:52+00:00
The issue is that you're treating class as a static category based purely on occupation or income - effectively saying that once someone earns a decent wage or becomes "a professional", they're no longer working class. That's a very narrow, top-down view, and ironically, it erases one of the few success stories of social mobility.

Many working-class people entered fields like IT, engineering, or contracting not because they were "born middle class", but because those were accessible paths that didn't require elite credentials, family connections, or private education. They built businesses, gained skills, and carved out independence - often still without the security, assets, or cultural capital traditionally associated with the middle class.

By your definition, the moment they succeed, they're no longer working class - which conveniently absolves the system of any responsibility for making life harder for them. It's circular: "If you’re struggling, you're working class. If you succeed, you were never working class." That's what's classist—defining people's identity by a fixed socioeconomic role and then erasing their background the moment they transcend it.

Social class isn't just about job title - it's about access to capital, power, mobility, and resilience in the face of economic shocks. IR35 disproportionately affected people who were just starting to get a foothold in those areas - often without the safety net others take for granted.

And yes, we may be using different definitions - but mine accounts for lived experience and systemic barriers, not just an abstract Wikipedia definition from a table written decades ago.
by pjc50 | 2025-04-07 09:07:03+00:00
> How do you put numbers in there usually?

Make them up. If you're uncomfortable with this, ask ChatGPT to make them up for you. Most places don't have any real way to measure productivity or rate of feature delivery, so as long as you promise a reasonable number (5-10%) they have no way of contradicting you.

Incremental refactoring is also the other classic way to do this. Slip bits into other tickets until you've slowly dragged the ship around. But it's not always possible.
by namaria | 2025-04-07 09:07:36+00:00
> killing the "stochastic parrot" meme

This was clearly the goal of the "Biology of LLMs" (and ancillary) paper but I am not convinced.

They used a 'replacement model' that by their own admission could match the output of the LLM ~50% of the time, and the attribution of cognition related labels to the model hinges entirely on the interpretation of the 'activations' seen in the replacement model.

So they created a much simpler model, that sorta kinda can do what the LLM can do in some instances, contrived some examples, observed the replacement model and labeled what it was doing very liberally.

Machine learning and the mathematics involved is quite interesting but I don't see the need to attribute neuroscience/psychology related terms to them. They are fascinating in their own terms and modelling language can clearly be quite powerful.

But thinking that they can follow instructions and reason is the source of much misdirection. The limits of this approach should make clear that feeding text to a text continuation program should not lead to parsing the generated text for commands and running these commands, because the tokens the model outputs are just statistically linked to the tokens inputted to them. And as the model takes more tokens from the wild, it can easily lead to situations that are very clearly an enormous risk. Pushing the idea that they are reasoning about the input is driving all sorts of applications that seeing them as statistical text continuation programs would make clear are a glaring risk.

Machine learning and LLMs are interesting technology that should be investigated and developed. Reasoning by induction that they are capable of more than modelling language is bad science and drives bad engineering.
by kwon-young | 2025-04-07 09:08:20+00:00
Curl is one of the very few projects I managed to contribute to with a very simple PR.

At the time, I was a bit lost with their custom testing framework, but was very imprest by the ease of contributing to one of the most successful open-source project out there.

I now understand why. It is because of their rules around testing and readability (and the friendly attitude of Daniel Stenberg) that a novice like me managed to do it.
by thworp | 2025-04-07 09:09:03+00:00
I think you went into stable genius territory yourself in the second half.

> I'd guess Ukraine falls within months at most.

Even if the US stops all deliveries -- which is far from certain, judging by the noise Trump made in the past week -- Ukraine won't run out of supplies. They'll keep trading territory and even if that accelerates, it will take a long time to reach Kiev (which isn't guaranteed to lead to capitulation anyway, Ukrainians are pretty determined). Meanwhile it looks like Russia is running out of armaments itself and the domestic situation is heating up such that another mobilization is quite risky.

> Pooh Bear is laughing too and will probably try to invade Ukraine.

This is a laughable suggestion. Sure, if Russia annexes parts of Ukraine, China will gladly send construction and mining crews for a friendly price and a cut of the profits, but they have absolutely no reason to send soldiers to Ukraine. In the first place it is not their conflict. They are only "allied" to Russia until they can fully bring Central Asia under their control and build up the infrastructure to replace Russian hydrocarbons and raw materials. Sending the PLA also runs the risk of exposing their weakness, which would damage China's position in the region. Much better to keep it as the huge threat it is on paper.

> Did I mention that US antipiracy efforts are likely being strongly cut which means world shipping is going to get a lot more dangerous, expensive, and unreliable?

Yes, but the PLN is more than ready to take up this role. They have the bases, the ships and have been training like mad. Since freedom of navigation is very important to China (for now), we might see some strange bedfellows there.