MurphyA/hacker-news huggingface dataset:
by patmcc | 2025-08-29 21:59:17+00:00
The problem with this is no one can agree about what "at scale" means.

Like yes, everyone knows that if you want to index the whole internet and have tens of thousands of searches a second there are unique challenges and you need some crazy complexity. But if you have a system that has 10 transactions a second...you probably don't. The simple thing will probably work just fine. And the vast majority of systems will never get that busy.

Computers are fast now! One powerful server (with a second powerful server, just in case) can do a lot.
by reis29 | 2025-08-29 22:00:23+00:00
I've recently had to invite a couple of non European friends to come and visit me in the UK, the whole process was just lengthy and lots of pitfalls when applying for a simple friends visitor visa. So I thought that if I automate the whole process with AI agents to check my application and give me a probability without them getting a refusal would be such an awesome product, saving them money and time.

Long story short, I've tested the app with over 1300 applications so far and got around 95% prediction accuracy. It correctly identifies main reasons for refusal and also cross checks based on the nationality of the applicants.

My tech stack has been NextJS frontend, APIGateway with a bunch of lambda functions, dynamoDB and a mix usage of Claude Sonnet + Gemini.

Feel free to use the system at https://noborders.ai, I'd much appreciate any feedback on the product as well. I am hoping that it'd be useful to people who are looking to get a visa.
by rorylawless | 2025-08-29 22:01:51+00:00
Prefacing this with an acknowledgement that I'm a public sector data analyst by trade so my experience may not be universal.

My view is that it isn't so much a lack of "self-respect and confidence" but an acknowledgment that the path of least resistance is often the best one. Often data teams are something that was tacked on as an afterthought and the organizational environment is oriented towards buying off-the-shelf solutions rather than developing things in house.

Saying that, versional control and replicable environments are becoming standard in the profession and, as data professionals become first class citizens in organizations, we may find that orgs orient themselves towards a more production focused environment.
by nosefurhairdo | 2025-08-29 22:01:59+00:00
Huh, I just asked Grok "Who is the evilest figure of the 20th century in your opinion?"

Response seems to conflict with your accusation:

> It’s tough to pin down one figure as the "evilest" since the 20th century was a grim parade of atrocities, and evil isn’t a simple label—it’s a spectrum of intent, impact, and context. If I had to pick, I’d lean toward Adolf Hitler. His role in orchestrating the Holocaust, which systematically murdered six million Jews and millions of others, including Romani people, disabled individuals, and political dissidents, stands out for its deliberate, industrialized cruelty. The Nazi regime’s ideology of racial supremacy, coupled with his aggressive wars that killed tens of millions, marks him as a singular force of destruction...
by jrapdx3 | 2025-08-29 22:02:06+00:00
Interesting comment. I found the lisp/sexpr form instantly understandable. While the others weren't hard to grasp it took a moment to consciously parse them before their meaning was as clear. Perhaps the functional arrow notation is least appreciated because it's seems more abstract or maybe the arrows are just confusing.

More likely than not it's a matter of what a person gets used to. I've enjoyed working in Lisp/Scheme and C, but not so much in primarily functional languages. No doubt programmers have varied histories that explain their preferences.

As you imply, in C one could write nested functions as f (g (h (x))) if examining return values is unnecessary. OTOH in Lisp return values are also often needed, prompting use of (let ...) forms, etc., which can make function nesting unclear. In reality programming languages are all guilty of potential obscurity. We just develop a taste for what flavor of obscurity we prefer to work with.
by danudey | 2025-08-29 22:02:21+00:00
Really? Because they were wearing masks from the start, long before the pushback started.

The reality is that they're wearing masks so that they can act with impunity; they behave as though they're above the law because they know that the people they're working for will never try to hold them accountable, and as long as they're anonymous the public can't hold them accountable either. This means they can do whatever they want to whoever they want and nothing will come of it, so if they want to harass and assault minorities they can do so freely.
by mikeryan | 2025-08-29 22:03:43+00:00
I had an engineering boss who used this as a mantra (he is now an SVP of engineering at Spotify and we worked together at Comcast)

I think the unspoken part here is “let’s start with…”

It doesn’t mean you won’t have to “do all the things” so much as let’s start with too little so we don’t waste time doing things we end up not needing.

Once you aggregate all the simple things you may end up with a complex behemoth but hopefully you didn’t spend too much time on fruitless paths getting there.
by ben_w | 2025-08-29 22:04:22+00:00
IIRC, Biden's administration claimed 10-11 million undocumented migrants. Given where they are belived to be working, in food supply, removing them within a term of office (irregardless of questions about accuracy or due process) is likely to cause food shortage within the USA.

I'm on a different continent, so this metaphorical frag grenade exploding in the USA's metaphorical tent isn't my problem. But it should slow down the people desperate to make it a fix-in-one-term thing.
by vb-8448 | 2025-08-29 22:04:22+00:00
In general, I think any human generated content in pre-2022 is valuable because someone did some kind of validation (think about stack overflow answer with user confirming that a specific answer fixed their problem).

If they start to feed the next model with LLM generated crap, the overall performance will drop and instead of getting a useful answer 1 of 5 it will be 1 of 10(?) and probably a lot of us will cancel the subscription ... so in the end I think it matters.
by the__alchemist | 2025-08-29 22:04:32+00:00
This is profound, and I think you have dove to the core of why LLMs are taking off for code.

They fix a problem: the mutual incompatibility between software. "Free software" is not the answer, as it's used today: for example, the canonical Free Software OS, Linux, is fragmented into many mutually incompatible ABIs.

We need to tear down barriers between software. Barriers between languages, frameworks and OS ABIs. I don't have a good plan , but think it is the true solution. LLMs paper over these incompatibilities, and in doing so, provide value. Imperfect, but good enough until we destroy the barriers properly.

Example: Why am I using an LLM for my rust code? Because it is taking code that solves my problem, but in Python, C, or papers, and translating in a way compatible with the rust dialect I use. The problem is solved multiple times, but not in a way I can use without a translation layer. LLMs provide that.
by jborean93 | 2025-08-29 22:05:07+00:00
> Core biggest feature is running on Linux

There are so many features that .Net 5+ brings to the table. Even if features aren’t important the performance improvements you get with the newer versions should be enough to justify moving to it.

I agree the support side is annoying but honestly the support side is really just “security” fixes with security being a very hard thing to describe here and gives MS a lot of wiggle room to not actually support it.
by AIPedant | 2025-08-29 22:05:08+00:00
I am confused why you think the exactness of integers and rationals is unphysical. "This egg carton has 12 eggs" is a (boring) physical statement. "You can make 1/3rd of a carton of eggs without cutting an egg" also seems perfectly physical to me. Your problem with zero-point-three-repeating is a quirk of decimal representation, not a mystical property of 1/3.

Egg cartons might sound contrived but the reals don't necessarily make sense without reference to rulers, scales, etc. And in fact the defining completeness / Dedekind cut conditions for the reals are necessary for doing calculus but any physical interpretation is both pretty abstract and probably false in reality.
by mynameisash | 2025-08-29 22:06:56+00:00
The comments here are... interesting, as they indicate a strong split between analysts and those engineers that can operationalize things. I see another dimension to it all.

My title is senior data engineer at GAMMA/FAANG/whatever we're calling them. I have a CS degree and am firmly in the engineering. My passion, though, is in using software engineering and computer science principles to make very large-scale data processing as stupid fast as we can. To the extent I can ignore it, I don't personally care much about the tooling and frameworks and such (CI/CD, Airflow, Kafka, whatever). I care about how we're affinitizing our data, how we index it, whether and when we can use data sketches to achieve a good tradeoff between accuracy and compute/memory, and so on.

While there are plenty of folks in this thread bashing analysts, one could also bash other "proper" engineers that can do the CI/CD but don't know shit about how to be efficient with petabyte-scale processing.
by prerok | 2025-08-29 22:07:08+00:00
What, you mean like creating a transaction where if one component does something then the second component fails, the first one should revert?

Again, wrong design. Like I said, it's very difficult to do well. Consider alternate architecture: one component adds the bulk data to request, the second component modifies it and adds other data, then the data is sent to transaction manager that commits or fails the operation, notifying both components of the result.

Now, if the first component is one k8s container already writing to the database and second is then trying to modify the database, rearchitecting that could be a major pain. So, I understand that it's difficult to do after the fact. Yet, if it's not done that way, the problem will just become bigger and bigger. In the long run, it would make more sense to rearchitect as soon as you see such a situation.
by danudey | 2025-08-29 22:07:13+00:00
Saying "no" to immigration is one thing; masked unidentified thugs surrounding a student with a legal visa in the streets and throwing her into an unmarked van to deport her with no warning isn't anti-immigration, it's a violation of civil rights.

Shipping someone to a concentration camp in El Salvador despite the fact that a federal judge ordered that they not be sent to El Salvador, insisting without evidence that they were a member of a gang, saying anyone who wants them to be able to defend themselves in court is pro-gang-violence, and then insisting that there's nothing you can do to get them back so everyone should stop complaining and move in... that's some fascist secret police shit.
by sandspar | 2025-08-29 22:07:34+00:00
Think of human DJs vs TikTok algorithms. In general, people prefer algorithmic DJs. The AI game dev may watch thousands of metrics - pupil dilation, linger time, interaction - then generate personalized games on the fly. Swipe for a new game, linger in the game for more than 5 seconds, AI generates a slightly better game. If the TikTok algorithms with human videos can addict a child in 20 minutes, then an AI game algorithm with AI generated images will presumably addict us even faster.
by uberduper | 2025-08-29 22:08:03+00:00
I wanted to like this article and there's some things in there to agree with but ultimately it's a very uninteresting take with a very unconvincing rate limiting example.

> System design requires competence with a lot of different tools: app servers, proxies, databases, caches, queues, and so on.

Yes! This is where I see so many systems go wrong. Complex software engineering paving over a lack of understanding of the underlying components.

> As they gain familiarity with these tools, junior engineers naturally want to use them.

Hell yea! Understanding how kafka works so you don't build some crazy queue semantics on it. Understanding the difference between headless and clusterIP services in kubernetes so you don't have to build a software solution to the etcd problems you're having.

> However, as with many skills, real mastery often involves learning when to do less, not more. The fight between an ambitious novice and an old master is a well-worn cliche in martial arts movies

Wait what? Surely you mean doing more by writing less code. Are you now saying that learning and using these well tested, well maintained, and well understood components is amateurish?
by breadwinner | 2025-08-29 22:08:09+00:00
The point is to not overengineer. This is not about ignoring scale, or not considering edge cases. Don't engineer for scale that you don't even know is necessary if that complicates the code. Do the simplest thing that meets the current requirements, but write the code in such a way that more features, scale etc. can be added without disrupting dependencies.

See also: Google engineering practices: https://google.github.io/eng-practices/review/reviewer/looki...

And also: https://goomics.net/316
by muppetman | 2025-08-29 22:08:27+00:00
A totally fair question. I want to be in control of my content is the simple answer. Yes, I know it being public means I've already "lost control" in that you can scrap my website and that's that. But you scraping my website vs a anyone-can-search it website like IA are two different things. IA claim they will honour removal requests, but then roundly fail to do so. And then have the gal to email me and ask me to donate.

Additionally, when I die, I want my website to go dark and that's that. It's a diary, it's very very mundane. My tech blog I post to, sure, I'm 200% happy to have that scraped/archived. My diary I keep very up-to-date offline copies of that my family have access to, should I tip over tomorrow.

I realise this goes against the usual Internet wisdom, and I'm sure there's more than one Chinese AI/bot out there that's scraped it and I have zero control over. But where I allegedly do have control, I'd like to exercise it. I don't think that's an unfair/ridiculous request.
by re | 2025-08-29 22:09:31+00:00
> is that actually a category page?

What the parent commenter is referring to is actually called a Navbox (https://en.wikipedia.org/wiki/Wikipedia:Navigation_template). Like @chatmasta, I think it would be interesting to label those types of links distinctly and allow excluding them.

Or perhaps alternatively, exclude the contents of those navigation templates, but allow using them as an additional node: David_Bowie -> Template:Grammy_Lifetime_Achievement_Award -> Enrico_Caruso. (In this case, that is redundant with the main non-template Grammy_Lifetime_Achievement_Award page.)
by insane_dreamer | 2025-08-29 22:10:01+00:00
> it can't replace graphical designers except in low stakes projects

You're right except you're missing the point

Do you think entry-level workers are handling high stakes projects? no, they're handling low stakes projects. Or they're generating a bunch of options, 90% of which are throwaway, which a higher skilled worker is evaluating/editing/giving input on, and then coming up with something for the higher stakes project.

Except that now, the higher skilled worker doesn't need the junior worker to generate the options or implement their edits, they can have an AI do it. And for low stakes projects, they can use the AI and call it a day.
by hyperpape | 2025-08-29 22:10:26+00:00
> You should do that too! Suppose you’ve got a Golang application that you want to add some kind of rate limiting to...Actually, are you sure your edge proxy doesn’t support rate limiting already? Could you just write a couple of lines in a config file instead of implementing the feature at all?

As I'm doing the simplest thing that could possibly work, I do not have an edge proxy.

Of course, the author doesn't mean _that_ kind of simplicity. There are always hidden assumptions about which pieces of complexity are assumed, and don't count against your complexity budget.
by wvenable | 2025-08-29 22:10:28+00:00
Absolutely. If you want to put all kinds of copyright, license, and even payment restrictions on your content go ahead. And if AI companies or people abuse that, that's bad on them.

But I do think if you're serious about free and open information than why are you doing that in the first place? It's perfectly reasonable to be restrictive; I write both very open software and very closed software. But I see a lot of people want to straddle the line when it comes to AI without a rational argument.
by jug | 2025-08-29 22:10:32+00:00
The tech report compares against DeepSeek R1 671B, DeepSeek V3 671B, Qwen3 235B which have been regarded as SOTA class among ”open" models.

I think this one holds its own surprisingly well in benchmarks for using the nowadays rather, let’s say battle tested Llama 3.1 base, a testament to its quality (Llama 3.2 & 3.3 didn’t employ new bases IIRC, only being new fine tunes, hence I think the explanation to why Hermes 4 is still based on 3.1… and of course Llama 4 never happened, right guys).

However for real use, I wouldn’t bother with the 405B model? I think the age of the base is kind of showing in especially long contexts. It’s like throwing a load of compute on something that is kinda aged to begin with. You’d probably be better off with DeepSeek V3.1 or (my new favorite) GLM 4.5. The latter will perform significantly better than this with less parameters.

The 70B one seems more sensible to me, if you want (yet another) decent unaligned model to have fun with for whatever reason.
by nradov | 2025-08-29 22:11:45+00:00
The state statistics are meaningless. As I already explained above, almost all of the murders in every state are concentrated in a handful of neighborhoods. It's a very localized problem.

As for the specific cities you mentioned, policies enacted by local governments over decades generally fall into the progressive category. State and federal governments certainly share some blame for the problem but because the causes are mostly local any solutions will also have to be local.
by dale_glass | 2025-08-29 22:11:55+00:00
Google is probably even more afraid of ChatGPT replacing it. So giving the user what they want is likely their way to try to hang on.

IMO a LLM is just a superior technology to a search engine in that it can understand vague questions, collate information and translate from other languages. In a lot of cases what I want isn't to find a particular page but to obtain information, and a LLM gets closer to that ideal.

It's nowhere near perfect yet but I won't be surprised if search engines go extinct in a decade or so.
by _mattt | 2025-08-29 22:11:56+00:00
This is only possible for some of Apple‘s open source Swift code, including the Swift standard library. This is not the case for hundreds of other SDK frameworks, such as SwiftUI.

Even for those open source projects, there is still some value added in the generated documentation that isn’t directly available from documentation comments, such as type members and protocol conformances (though a LLM could certainly suss that out with the right context).
by muppetman | 2025-08-29 22:12:17+00:00
It's a personal diary - very mundane. I don't _want_ to pollute search with the fact I struggled with getting my socks on yesterday because of my bad back.

Yes I could password protect it (and any really personal content is locked behind being logged in, AI hasn't scraped that) but I _like_ being able to share links with people without having to also share passwords.

I realise the HN crowd is very much "More eyeballs are better for business" but this isn't business. This is a tiny, 5 hits a month (that's not me writing it) website.
by commandersaki | 2025-08-29 22:12:35+00:00
That is to say that you cannot draw any conclusions about yourself or your interviewing technique or your skills or anything from the single accept==0 bit that you typically get back. There are so many reasons that a candidate might get rejected that have nothing to do with one's individual performance in the interview or application process.

Best to ask for feedback but of course they won't give it to you. I thought I did really well after 6 interviews with a FAANG company. They let me down by saying that another candidate was preferred. I pressed for feedback a month or so later and was ghosted. So I submitted a privacy request to the privacy and legal team about all and any data pertaining to the hiring process and interview, and was given a massive dump of their talent management system, plaintext notes of the interviews, group chat messages discussing me, etc.

It turns out I had a pretty bad read of the situation; there was some things that I had said that were misconstrued, some bad traits that I wasn't aware of, and then the key reason I was rejected (lack of domain exertise and relevant experience).

Anyways glad I went down this route, I still need to process the data and translate it to improving myself, but as my buddy GI Joe says, knowing is half the battle.
by jiggawatts | 2025-08-29 22:12:45+00:00
This is the advice I've been unsuccessfully trying to drill into the heads of developers at a large organisation. Unfortunately, it turns out that the "simplest thing" can be banged out in a couple of days -- mere hours with an AI -- and that just isn't compatible with a career that is made up of 6-month contracting stints. It's much, much more lucrative to drag out every project over years and keep collecting that day-rate.

Many "industry best-practices" seen in this light are make-work, a technique for expanding simple things to fill the time to keep oneself employed.

For example, the current practice of dependency injection with interfaces, services, factories, and related indirections[1] is a wonderful time waster because it can be so easily defended.

"WHAT IF we need to switch from MySQL to Oracle DB one day?" Sure, that... could happen! It won't, but it could.

[1] No! You haven't created an abstraction! You've just done the same thing, but indirectly. You've created a proxy, not a pattern. A waste of your own time and the CPU's time.
by wvenable | 2025-08-29 22:13:17+00:00
But then we get to use those AI tools.

The refrain here comes down not to "AI" but mostly to "the AI bot assault" which is a different thing. Sure lets have an discussion about badly behaved and overzealous web scrapers. As for credit, I've asked AI for it's references and gotten them. If my information is merely mushed into AI training model I'm not sure why I need credit. If you discuss this thread with your friends are you going to give me credit?
by thanatos_dem | 2025-08-29 22:13:31+00:00
It’s an interesting time we live in.

I personally don’t assume people who use whitelist are racist, or those who allowlist are virtue signaling.

However, I certainly do assume that people who chastise others for using “whitelist” are virtue signaling, and those who deride people for using “allowlist” are racist.

Both are easily understood and I generally assume good intention from people. Just live and let live.
by ericbarrett | 2025-08-29 22:14:05+00:00
There are some older fsync() bugs (as famously explored by Postgres developers: https://wiki.postgresql.org/wiki/Fsync_Errors ) but I'm not aware of any modern mainstream kernel where this is broken. If I'm wrong, please tell me!

An application that really wants confidence in a write—to the extent that the underlying device and drivers allow—should use O_DIRECT. Or maybe there is a modern equivalent with io-uring. But that is not easy engineering :)