MurphyA/hacker-news huggingface dataset:
Stop Blaming Embeddings, Most RAG Failures Come from Bad Chunking
by wehadit | 2025-12-03 14:25:15+00:00
Everyone keeps arguing about embeddings, vector DBs, and model choice, but in real systems, those aren’t the things breaking retrieval. Chunking drift is. And almost nobody monitors it. A tiny formatting change in a PDF or HTML file silently shifts boundaries. Overlaps become inconsistent. Semantic units get split mid-thought. Headings flatten. Cross-format differences explode. By the time retrieval quality drops, people start tweaking the model… while the actual problem happened upstream. If you diff chunk boundaries across versions or track chunk-size variance, the drift is obvious. But most teams don’t even version their chunking logic, let alone validate segmentation or check adjacency similarity. The Industry treats chunking like a trivial preprocessing step. It’s not. It’s the single biggest source of retrieval collapse, and it’s usually invisible. Before playing with new embeddings, fix your segmentation pipeline. Chunking is repetitive, undifferentiated engineering, but if you don’t stabilize it, the rest of your RAG stack is built on sand.
by daveguy | 2025-12-03 14:25:29+00:00
The election systems are managed at the state level exactly as they should be. Corruption has a much lower chance when there are barriers.

That said, there should be national minimum standards for the elections.

Chain of custody is something all elections adhere to in the US, but if it's not a federal requirement, it should be.

What's really needed are hand marked paper ballots, electronically counted using open, independently audited machines with no network connectivity (obviously). "Scantron" style machines That should be a minimum requirement.

Nobody stole the election from the doddering old fool in 2020. Internet bullshit doesn't stand up in court where they do have evidence standards.

Edit: Back on topic ... individual stock trading by representatives clearly should be illegal. And apparently we need more independent watchdogs with better protections against a puppet who mindlessly goes along with the last person to manipulate them.
by impossiblefork | 2025-12-03 14:25:39+00:00
I don't think that's true. I see ICE vehicles as complex, with lots of parts that wear etc. and which are problematic, while I see BEVs as straightforward.

Maybe the batteries will wear out, but what will a replacement battery cost in 10 years? Presumably even less than now.

The comment by Lio beside this one also makes it hard for me to take a view like yours.
by PurpleRamen | 2025-12-03 14:25:58+00:00
It kinda reads like a case of survivorship bias. He is the one in a million to reach the good ending, despite starting with the wrong choice; though in this case, the wrong choice brought him on the road to the good ending.

Now the real question is, does the game loads significant better now, or does the performance still suck? In which case it might be more an excessive case of yak-shaving. And if yes, when can we except the release?
by fragmede | 2025-12-03 14:26:19+00:00
In this modern era of easily accessible knowledge, how gate keepy is it though? It's inscrutable at first glance, but ChatGPT is more than happy to explain what the hell ℵ₀, ℵ₁, ♯, ♭, or Σ mean, and you can ask it to read the arxiv pdf and have it explain it to you.
by kragen | 2025-12-03 14:26:37+00:00
I thought Martian soil was full of perchlorate, which produces oxygen if you just get it wet and expose it to somewhere the gas can escape?

I guess we'll never know, because this article is just blogspam linking another blogspam article that doesn't link the actual preprint, just says, "A recent paper from Daniella Billi of the University of Rome Tor Vergata , [sic] published in pre-print form in Acta Astronautica, reviews how one particular extremophile fills the role of both useful test subject and useful tool all at once."
by olalonde | 2025-12-03 14:27:01+00:00
> We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome.

They must be really glad to have so much competition then.

> If a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project.

I wonder if OpenAI will start assisting Google now?
by jerf | 2025-12-03 14:27:11+00:00
It seems to me that even if AI technology were to freeze right now, one of the next moderately-sized advances in AI would come from better filtering of the input data. Remove the input data in which humanity teaches the AI to play games like this and the AI would be much less likely to play them.

I very carefully say "much less likely" and not "impossible" because with how these work, they'll still pick up subtle signals for these things anyhow. But, frankly, what do we expect from simply shoving Reddit probably more-or-less wholesale into the models? Yes, it has a lot of good data, but it also has rather a lot of behavior I'd like to cut out of my AI.

I hope someone out there is playing with using LLMs to vector-classify their input data, identifying things like the "passive-aggressive" portion of the resulting vector spaces, and trying to remove it from the input data entirely.
by Aurornis | 2025-12-03 14:27:17+00:00
The fact that Nancy Pelosi became the meme for congress members trading stocks is more political than anything. The actual top performing congressional portfolios never make as much news. Pelosi doesn’t make as many trades as other members like Marjorie Taylor Green, who gets a free pass for some reason.

And everyone ignores the fact that her husband is a professional in the finance and venture capital world. Investing is literally his job.

Anyway, if you want to track her trades there are ETFs to do it. It’s not an exclusive investment thing. If you’re in the US you could buy them from your broker right now.

There are also countless sites that will push the Nancy Pelosi portfolio to your inbox, but they’re universally spammy and almost always pushing some other agenda too.

As a rule: If they’re focused specifically on Nancy Pelosi and not looking at top congressional trades as a whole, they’re more interested in pushing a political content than financial content.

Oh, and it’s important to know that you can’t actually track congressional portfolios in real time. The disclosures are delayed so you can only track them after the fact. If you believe they’re front-running insider info then the changes may have happened before you could possibly get the signal to buy or sell.
by bluGill | 2025-12-03 14:27:19+00:00
I don't want people in congress as a career. Do it for at most 2 terms and get back to the real world. When that is the reality it means we cannot take away their ability to have stocks because when they return "to the real world" they will need that savings. Though while in office they should be limited to buying and selling from blind trusts and such things where they cannot know what happens except the economy in general. However if they own stock before it isn't fair to make them sell it: this is a really hard problem and there probably isn't an answer I will be happy with.
by ah979 | 2025-12-03 14:27:24+00:00
> If it was simply pairing with juniors from time to time to hack on things and show them cool stuff, it would be wonderful.

I think that's a lot of it! (Author here, btw) I think that doing more of that actually makes performance discussions easier and takes the place of 1:1s a lot of times. The whole point of performance management is to nurture the relationship with the junior so that they grow into the type of senior that they need to be to be a contributing member of the team. And if you can achieve that better through hacking and pairing, then by all means.

As far as all the other BS meetings, I think that a lot of people in leadership positions aren't intentional/strategic enough about which meetings they allow themselves to get "hauled into", so they end up totally jaded and burnt out. They think they can't say no to these things, and so they lose the reins on their actual goals. If you step back and ask yourself what stakeholders actually need from you to feel heard, seen, validated, informed, etc, then it may look different from a calendar full of meetings.
by Closi | 2025-12-03 14:27:25+00:00
It's still better than anything else and the de-facto standard so you need it.

Clients will send you their PowerPoint template and want you to use it. They will send you their complex spreadsheets riddled with VBA macros and you will need to fix them. They will invite you to a Teams site because that's where their project updates go. I just don't see how you can avoid it as a consulting company!

For things like Excel - We can say it's 'bad' but I've not seen anything do the job it does better. And besides, even if it was bad, it doesn't matter - as a consultant you need to use it because your clients probably want your workings as part of the deliverables, and if it's on Google Sheets they often won't want that.
by thesuitonym | 2025-12-03 14:27:44+00:00
Chris Sawyer lived and breathed assembly, earlier in the article he states that he just felt more efficient writing it than higher level languages. Then you've got the modern team of devs who probably haven't worked with asm since university, and it becomes difficult for them to review the original source code. Also Chris probably wasn't doing a lot of the actual programming, so instead of one guy working on a passion project, you have a team of devs doing a job.
by wolrah | 2025-12-03 14:27:47+00:00
> (Even for a fully self-hosted system you'd still have to figure out how to interface the certificate renewal mechanism with your DNS provider, so not as easy to set up as individual certificates for each subdomain.)

That's exactly what the new DNS-PERSIST-01 challenge is for, being able to authorize a specific system or set of systems to request certs for a given FQDN and optionally subdomains without having to give that system direct control over your DNS as the existing DNS-01 challenge requires.
by zerofor_conduct | 2025-12-03 14:28:15+00:00
"The unknown thing to be known appeared to me as some stretch of earth or hard marl, resisting penetration... the sea advances insensibly in silence, nothing seems to happen, nothing moves, the water is so far off you hardly hear it... yet finally it surrounds the resistant substance."

A. Grothendieck

Understanding mathematical ideas often requires simply getting used to them
by stevenjgarner | 2025-12-03 14:28:25+00:00
Great intention, but too simplistic. The lawmaker has a third party set up a trust for the benefit of his children and the trust owns a company in yet another jurisdiction that just happens to not only take a position in a certain stock, but probably provides "consulting services" for a healthy fee. And that is the simple version. By the time the lawmaker's interest is fully obfuscated, the legal structure is fully compliant with every letter of the law, just not the intention. Better to have fewer laws that are actually enforceable.
by ACCount37 | 2025-12-03 14:28:37+00:00
There are a few weirder training methods that involve wiring explicit bits of knowledge into the model.

I imagine that if you use them hard enough with the same exact text, you can attain full word for word memorization. This may be intentional, or a side effect of trying to wire other knowledge into the model while this document is also loaded into the context.
by ethbr1 | 2025-12-03 14:28:47+00:00
There's definitely a few phases of Github, feature and popularity wise.

1. Free hosting with decent UX 2. Social features 3. Lifecycle automation features

In this vein, it doing new stuff with AI isn't out of keeping with its development path, but I do think they need to pick a lane and decide if they want to boost professional developer productivity or be a platform for vibe coding.

And probably, if the latter, fork that off into a different platform with a new name. (Microsoft loves naming things! Call it 'Codespaces 365 Live!')
by ApolloFortyNine | 2025-12-03 14:28:54+00:00
>This is saying that people in congressional leadership positions do 47% better than other members of Congress.

It's even more damning than that I think.

>we find that lawmakers who later ascend to leadership positions perform similarly to matched peers beforehand but outperform them by 47 percentage points annually after ascension.

The same person makes more after being put in a leadership position than before. That essentially removes any possibility of 'well maybe they're just more knowledgable and that's why they're in leadership.'
by trentnix | 2025-12-03 14:29:01+00:00
Every single one of these scoundrels will stand up at a college graduation and exhort the virtues of public service and sacrifice for country. But somehow they all manage to get rich in the midst of their own service and sacrifice.

They shouldn't be allowed to trade stocks. When you do so, corruption is inevitable. It should be an expected price of public service that you remove yourself from most other ambitions.

I'd hope that if they were required to live off of their (rapidly depreciating) savings and income, like most Americans, they might consider some of their legislative choices differently.
by VikingCoder | 2025-12-03 14:29:16+00:00
Can I find one of these people in congressional leadership positions, who is willing to accept a sizeable campaign contribution in exchange for joining their mailing list which contains their list of stock purchases in real time?

Or is that one of those, "Of course that exists, but if you have to ask how much it costs, you can't afford it" kind of things?
by HarHarVeryFunny | 2025-12-03 14:29:46+00:00
> The problem I had that the larger your project gets, the more mistakes Claude makes

I think the reason for this is because these systems get all their coding and design expertise from training, and while there is lots of training data available for small scale software (individual functions, small projects), there is much less for large projects (mostly commercial and private, aside from a few large open source projects).

Designing large software systems, both to meet initial requirements, and to be maintainable and extensible over time, is a different skill than writing small software projects, which is why design of these systems is done by senior developers and systems architects. It's perhaps a bit like the difference between designing a city and designing a single building - there are different considerations and decisions being made. A city is not just a big building, or a collection of buildings, and large software system is not just a large function or collection of functions.
by lordnacho | 2025-12-03 14:30:53+00:00
For instance, I have a friend with a phd in designing medical experiments. Someone from the biosciences field will come to him and ask for how exactly his experiment needs to be set up to make valid conclusions.

One of the guys I met on HN got into marine telecoms. Something about the necessity of compressing data back before it became ubiquitous.

I work in quant trading stuff, where we try to use stats to decide what to buy and sell.
by aitchnyu | 2025-12-03 14:31:00+00:00
1 liter can contain upto 490ml oxygen but supplemental oxygen is upto 60 liters per minute. Do they plan to put a bubbler to replenish oxygen? And did they compute maximum oxygen transfer for the intestines?

They administered a "dose of non-oxygenated perfluorodecalin" but there is "oxygenation effect" observed. How?
by myrmidon | 2025-12-03 14:31:13+00:00
Peoples understanding of what "corruption" is differs by a surprising amount.

The boundaries between corruption, lobbyism, incompetence and nepotism can get very blurry (sometimes even actually representing your constituents interests can look this way!).

But policy makers enriching themselves by insider trading are seen as corrupt by pretty much everyone.

What is the less benign corruption seen in Washington that you feel is most pressing?
by ikamm | 2025-12-03 14:31:21+00:00
Also Car Park Capital [1], an upcoming indie tycoon game in the style of RCT1/2 where you contribute to urban sprawl. It's being published by Microprose and runs on a custom engine [2] that makes it look incredibly similar to the original games.

[1] https://hilkojj.nl/

[2] https://hilkojj.nl/projects
by voidhorse | 2025-12-03 14:31:26+00:00
As someone who has always struggled with mathematics at the calculational level, but who really enjoys theorems and proofs (abstract mathematics), here are some things that help me.

1. Study predicate logic, then study it again, and again, and again. The better and more ingrained predicate logic becomes in your brain the easier mathematics becomes.

2. Once you become comfortable with predicate logic, look into set theory and model theory and understand both of these well. Understand the precise definition of "theory" wrt to model theory. If you do this, you'll have learned the rules that unify nearly all of mathematics and you'll also understand how to "plug" models into theories to try and better understand them.

3. Close reading. If you've ever played magic the gathering, mathematics is the same thing--words are defined and used in the same way in which they are in games. You need to suspend all the temptation to read in meanings that aren't there. You need to read slowly. I've often only come upon a key insight about a particular object and an accurate understanding only after rereading a passage like 50 times. If the author didn't make a certain statement, they didn't make that statement, even if it seems "obvious" you need to follow the logical chain of reasoning to make sure.

4. Translate into natural english. A lot of math books will have whole sections of proofs and /or exercises with little to no corresponding natural language "explainer" of the symbolic statements. One thing that helps me tremendously is to try and frame any proof or theorem or collection of these in terms of the linguistic names for various definitions etc. and to try and summarize a body of proofs into helpful statements. For example "groups are all about inverses and how they allow us to "reverse" compositions of (associative) operations--this is the essence of "solvability"". This summary statement about groups helps set up a framing for me whenever I go and read a proof involving groups. The framing helps tremendously because it can serve as a foil too—i.e. if some surprising theorem contravene's the summary "oh, maybe groups aren't just about inversions" that allows for an intellectual development and expansion that I find more intuitive. I sometimes think of myself as a scientist examining a world of abstract creatures (the various models (individuals) of a particular theory (species))

5. Contextualize. Nearly all of mathematics grew out of certain lines of investigation, and often out of concrete technical needs. Understanding this history is a surprisingly effective way to make many initially mysterious aspects of a theory more obvious, more concrete, and more related to other bits of knowledge about the world, which really helps bolster understanding.
by stetrain | 2025-12-03 14:31:30+00:00
The idea is that if their salary does not actually cover the expenses of living in and traveling to DC to do their job, you are limiting the pool of potential congressional representatives to those who are independently wealthy or who are finding other ways to make money via their position.

Paying enough to cover the expenses of the job isn't a solution to all problems, just kind of a minimum bar if we want a normal person to be able do the job without relying on other income sources.
by micromacrofoot | 2025-12-03 14:31:47+00:00
it's not really possible considering that most retirement accounts are stock portfolios — it also doesn't matter too much on an individual level because an individual ban just gets pushed to sharing stock tips with extended family (already happens), family bans get pushed to friends, etc...

I really think the best and maybe only way to avoid this wealth building nonsense are much harsher term limits — public service should be severely limited to prevent people from being in decades long positions to abuse it. Term limits also prevent the incessant campaign fundraising to some extent, because say you're limited to 2 terms... there's not much point in personal fundraising on your last one
by patates | 2025-12-03 14:32:08+00:00
As someone who abuses gemini regularly with a 90% full context, the model performance does degrade for sure but I wouldn't call it massively.

I can't show any evidence as I don't have such tests, but it's like coding normally vs coding after a beer or two.

For the massive effect, fill it 95% and we're talking vodka shots. 99%? A zombie who can code. But perhaps that's not fair when you have 1M token context size.
by ethbr1 | 2025-12-03 14:32:15+00:00
Mercurial was/is nice and imho smooths off a lot of the unnecessarily rough git edges.

But VCS has always been a standard-preferring space, because its primary point is collaboration, so using something different creates a lot of pain.

And the good ship SS Linux Kernel was a lot of mass for any non-git solution to compete with.
by amlib | 2025-12-03 14:32:24+00:00
Even though I have two SSDs in my main machine I still use a hard drive as an overflow for games that I judge are not SSD worthy.

Because it's a recent 20TB HDD the read speeds approach 250MB/s and I've also specifically partitioned it at the beginning of the disk just for games so that it can sustain full transfer speeds without files falling into the slower tracks, the rest of the disk is then partitioned for media files that won't care much for the speed loss. It's honestly fine for the vast majority of games.
by TheDong | 2025-12-03 14:32:48+00:00
> Most devs don’t work at Google or contribute to Google projects, so there is no reason for them to know anything about Gerrit.

Most devs have never worked on Solaris, but if I ask you about solaris and you don't even know what it is, that's a bad sign for how competent a developer you are.

Most devs have never used prolog or haskell or smalltalk seriously, but if they don't know what they are, that means they don't have curiosity about programming language paradigms, and that's a bad sign.

Most competent professional developers do code review and will run into issues with their code review tooling, and so they'll have some curiosity and look into what's out there.

There's no reason for most developers to know random trivia outside of their area of expertise "what compression format does png use by default", but text editors and code review software are fundamental developer tools, so fundamental that every competent developer I know has enough curiosity to know what's out there. Same for programming languages, shells, and operating systems.
by MallocVoidstar | 2025-12-03 14:33:06+00:00
According to the RFC:

>The messages in classic UDP-based DNS [RFC1035] are inherently unordered and have low overhead. A competitive HTTP transport needs to support reordering, parallelism, priority, and header compression to achieve similar performance. Those features were introduced to HTTP in HTTP/2 [RFC7540]. Earlier versions of HTTP are capable of conveying the semantic requirements of DoH but may result in very poor performance.

I'd bet basically all their clients are using HTTP/2 and they don't see the point in maintaining a worse version just for compatibility with clients that barely exist.
by akramachamarei | 2025-12-03 14:33:16+00:00
> Or society could decide that people shouldn't have to pay for [healthcare]

Society has already basically decided that: it's called health insurance. Having a "single" "government" payer where claimants actually pay nothing at PoS can be thought of like a very low deductible health insurance plan, can it not? Everyone still pays their premium in taxes, unless we mean to enslave doctors and nurses.