MurphyA/hacker-news huggingface dataset:
by CharlieDigital | 2025-12-08 16:00:15+00:00
I've implemented a distributed worker system on top of this paradigm.

I used ZMQ to connect nodes and the worker nodes would connect to an indexer/coordinator node that effectively did a `SELECT FROM ORDER BY ASC`.

It's easier than you may think and the bits here ended up with probably < 1000 SLOC all told.

- Coordinator node ingests from a SQL table - There is a discriminator key for each row in the table for ordering by stacking into an in-memory list-of-lists - Worker nodes are started with _n_ threads - Each thread sends a "ready" message to the coordinator and coordinator replies with a "work" message - On each cycle, the coordinator advances the pointer on the list, locks the list, and marks the first item in the child list as "pending" - When worker thread finishes, it sends a "completed" message to the coordinator and coordinator replies with another "work" message - Coordinator unlocks the list the work item originated from and dequeues the finished item. - When it reaches the end of the list, it cycles to the beginning of the list and starts over, skipping over any child lists marked as locked (has a pending work item)

Effectively a distributed event loop with the events queued up via a simple SQL query.

Dead simple design, extremely robust, very high throughput, very easy to scale workers both horizontally (more nodes) and vertically (more threads). ZMQ made it easy to connect the remote threads to the centralized coordinator. It was effectively "self balancing" because the workers would only re-queue their thread once it finished work. Very easy to manage, but did not have hot failovers since we kept the materialized, "2D" work queue in memory. Though very rarely did we have issues with this.
by neom | 2025-12-08 16:00:29+00:00
Just to give you another little titbit if you're interested. I work in go to market, and part of that is awareness, and part of that is advertising. Where people use the platform has a huge impact on the prices you pay to advertise on the platform, for example reddit is very expensive because they have a very high mobile traffic population, and the ads can't be blocked, advertising on X is hard because the people I want to reach all pay for premium, so the traffic you get from it now is basically useless, linkedin skews towards desktop, but their targeting is amazing, but because they skew towards desktop people run ad blocks, some platforms let you pick the devices you serve to, some don't, all of it impacts the price you pay to serve the ads.
by dirkc | 2025-12-08 16:00:45+00:00
Story time - A few years back I had to update a company linkedIn page, couldn't find who was admin and the normal process was broken somehow - can't remember the exact details.

Found an obscure reference to a support page where you could contact them about exactly the issue I had, but the form was broken...

I could see a coding error in the dev console, so I hot-patched the code and submitted a ticket!

Lo and behold, a day later my problem was solved and I gained admin access to the company page! I still wonder what the support team thought when they received the support ticked?
by maratc | 2025-12-08 16:00:46+00:00
I don't live in the US and I'm rather indifferent to either understanding, so it would be correct to say that I'm neither in favour nor in opposition of any one of these. Being personally unaffected, I simply find it an interesting subject to observe.

With that, I still want to point out that even if we both agree that "changing the text is the correct way to handle," it's not a possibility that is even remotely practical or achievable; any meaningful progress that the US have had in the last 100 years or so was achieved solely by the means of "reinventing" the meaning of already written (and unchangeable) words. That seems to be the exact thing you seem to be in opposition of.
by FabHK | 2025-12-08 16:02:06+00:00
Sure, but fiat is indispensable to running a modern economy. And the abuses that occur, of course, are curbed by vigorous regulation and enforcement. Sure, HSBC laundered money for the Mexican cartels. When it came out, it was a big scandal, they were fined, and the money laundering was stopped.

Crypto is entirely dispensable. It is pseudonymous, by design resists regulation, and has no enforcement. Some chains are deliberately opaque (Monero, Zcash), and there are tumblers to obscure flows even further. Its approach to money laundering is "sure, bring it on".

Fiat is indispensable, and better for legitimate purposes than crime. Crypto is dispensable, and better for crime than legitimate purposes. We should dispense with it.
by Jtsummers | 2025-12-08 16:02:16+00:00
> Once Netflix buys all of these companies, you won't ever be able to watch a WB movie without a $25 netflix sub per month. (and yeah, when they are done buying all the competition that's what the monthly will be.

That's kind of a silly argument. "People are better off paying $100+/month for 4+ streaming services than $25/month for one that has everything."

If your argument were that you'd have to pay more than the current combined cost, it'd be a better argument against mergers. Arguing against something because it's a better deal is just strange.
by SoftTalker | 2025-12-08 16:02:19+00:00
Not really. Cable TV started as a better way for people to get OTA channels when they were in marginal reception areas. My family had cable TV in the 1970s and it was a maybe eight or ten OTA channels and except for the PBS station they all had commercials, between shows and during shows.

HBO was the first offering that didn't have ads during the show.
by afavour | 2025-12-08 16:02:21+00:00
I'm curious how often tactics like this work. It is essentially asking the Warner stockholders to act against the wishes of their elected board.

It seems the main thrust of the pitch is "we're friends with Trump therefore more likely to win approval" which is so deeply gross but also probably persuasive to many. Jared Kushner is involved in the Paramount bid so you know they're greasing the right wheels.
by ProllyInfamous | 2025-12-08 16:02:44+00:00
I haved lived the majority of my life in working class communities (as an electrician). Despite sporadic eligibility, I've never applied for the benefits that many of my near poor neighbors receive (I am a simple homebody, with zero dependents).

>negative incentives for people receiving support, it doesn't uplift and enable success, it keeps people trapped in poverty and a mindset of helplessness.

For the majority of my recipient neighbors, I would disagree: [single] parent households simply are too expensive to operate without temporary community support.

Conversely, I have a few childless neighbors that simply ride out "disability" payments while working cash jobs part-time, typically along the lines of handyman and/or dealer. This bothers me.

Few of my neighbors are incentivized to work harder (at least on paper), out of fear of losing healthcare/housing/dining benefits.

Several of my wealthier clients had PPP loans "pre-emptively forgiven," and pay my neighbors cash for housework, so I know all sides are gaming this system...

----

But it just seems so obvious that single-payer healthcare and subsidized childcare would solve most of society's problems (much more simply than our current failures of welfare). These are the legitimate grievances of my working class neighborhood.
by rzwitserloot | 2025-12-08 16:02:46+00:00
I don't think these LLMs were explicitly designed based on the CEO's detailed input that boils down to 'reproduce these servile yes-men in LLM form please'.

Which makes it more interesting. Apparently reddit was a particularly hefty source for most LLMs; your average reddit conversation is absolutely nothing like this.

Separate observation: That kind of semi-slimey obsequious behaviour annoys me. Significantly so. It raises my hackles; I get the feeling I'm being sold something on the sly. Even if I know the content in between all the sycophancy is objectively decent, my instant emotional response is negative and I have to use my rational self to dismiss that part of the ego.

But I notice plenty of people around me that respond positively to it. Some will even flat out ignore any advice if it is not couched in multiple layers of obsequious deference.

Thus, that raises a question for me: Is it innate? Are all people placed on a presumably bell-curve shaped chart of 'emotional response to such things', with the bell curve quite smeared out?

Because if so, that would explain why some folks have turned into absolute zealots for the AI thing, on both sides of it. If you respond negatively to it, any serious attempt to play with it should leave you feeling like it sucks to high heavens. And if you respond positively to it - the reverse.

Idle musings.
by monerozcash | 2025-12-08 16:02:48+00:00
> But even in 2010 lots of important traffic was over https, and anyone worth their salt used https for important things.

In 2010 almost all messaging traffic on the internet was plaintext (or using badly broken encryption). Telephony? Hah.

These days nobody even uses regular phone calls or SMS, except US-based android users.

> That was just icing on the cake, the main purpose as I understood was metadata mining, and not just the internet but phone calls and sms as well

Metadata mining was just the fallback when they absolutely couldn't legally capture the content, or were not able to do so for logistical reasons. If you hack China Mobile and get access to all the call content, you'll still have a hard time sending that to the US. Metadata? Much easier.

These days even metadata collection has been gimped, most of the interesting metadata is encrypted. When I text someone, the NSA can see an encrypted connection from my phone to Apple. They can not feasibly see who that message goes to. They might not even be able to tell that I sent a message at all.

> I don't think any information as to what organizations they compromised has ever been revealed, but they certainly had the capability and it is only reasonable to presume they improved upon that capability

TSB leaks include tons of such information. Snowden leaks include some specific cases too, like Gemalto. Although just for the sake of accuracy I'm not sure which of these are actually TAO and which are other similar teams inside the NSA, but as I recall at least the TSB stuff seems to primarily originate from TAO.

There have also been a bunch of public and non-public incidents attributed to the Equation group (almost certainly NSA TAO) by the private sector.

I think these capabilities were already so good a decade ago that it would be hard to significantly improve upon them, you just slap in new exploits and keep doing what you're doing.

>I strongly disagree. I wish i had the time to compile evidence to back that up but plenty exists if you look it up. Matter of fact, I recall some of NSA's leadership oppose things like backdooring encryption or apps because they don't need it, and it only hurts the nation's security.

I was trying to suggest that the NSA is mostly interested in spying on foreign governments and maybe sometimes catching terrorists, not exactly "they want to know who's downloading tails, who's using signal, who the source of some journalist is".
by konz | 2025-12-08 16:03:28+00:00
ML6 | Machine Learning Engineer, Data Engineer | Python, TensorFlow, PyTorch, GCP, AWS, Azure | Full-time | Amsterdam, Berlin, Munich, Eindhoven, Ghent (EU) On-site/hybrid

We are a Machine Learning consulting company that builds end-to-end Machine Learning solutions. By applying the latest AI research, we keep our clients at the forefront of innovation. If you are interested check out: https://www.ml6.eu/knowledge-hub/blog and https://www.ml6.eu/customers/cases

Work on innovative projects for the biggest clients across Europe such as Randstad, ASML, FUNKE, and many more! Whether it’s about leveraging LLMs to improve customer support, building data lakes on cloud platforms to improve storage or implementing models using sensor data for quality control. You can find it all at ML6. You will mostly work with Python and a range of ML frameworks such as TensorFlow, PyTorch, or HuggingFace Transformers to solve hard Machine Learning tasks and help bring these application into production by building data pipelines and cloud infrastructure on all of the major cloud providers (GCP, AWS, Azure).

Open roles include:

• Senior/medior AI Engineer

• Senior AI Project Manager

• Chief Product Owner

and many more!

Apply now at the https://jobs.ml6.eu/
by gcr | 2025-12-08 16:03:33+00:00
Yeah I was confused about that bit too. AWS and GCP's metadata servers know which instances were deployed, so they presumably have some way of verifying the instance's identity out-of-band, such as being tagged by an internal job or machine identifier.

As for deploying from a trusted service without managing credentials, PyPI calls this "trusted publishing": https://docs.pypi.org/trusted-publishers/

From the docs:

1. Certain CI services (like GitHub Actions) are OIDC identity providers, meaning that they can issue short-lived credentials ("OIDC tokens") that a third party can strongly verify came from the CI service (as well as which user, repository, etc. actually executed);

2. Projects on PyPI can be configured to trust a particular configuration on a particular CI service, making that configuration an OIDC publisher for that project;

3. Release automation (like GitHub Actions) can submit an OIDC token to PyPI. The token will be matched against configurations trusted by different projects; if any projects trust the token's configuration, then PyPI will mint a short-lived API token for those projects and return it;

4. The short-lived API token behaves exactly like a normal project-scoped API token, except that it's only valid for 15 minutes from time of creation (enough time for the CI to use it to upload packages).

You have to add your github repository as a "trusted pulbisher" to your PyPI packages.

Honetsly the whole workflow bothers me -- how can PyPI be sure it's talking to github? what if an attacker could mess with PyPI's DNS? -- but it's how it's done.
by nunez | 2025-12-08 16:04:08+00:00
So I, too, was motivated to watch Steve Jobs unveil Aqua [^0] and compare it against Alan Dye's Liquid Glass reveal [^1]. The difference is unbelievable.

Jobs might have been an asshole, but his enthusiasm for the details is incontestable. He couldn't wait to show people the Save window, in its many forms. That he cared about the small details in everything is easy to see.

Contrast that with Alan Dye's inspiration reel for Liquid Glass. He's clearly reading from a script, which is quite a downgrade but understandable given the production value of keynotes these days. However, the real problem is that this intro is all about how it looks, not how it functions.

Microsoft tried this move with Vista (and their Aero design, which mostly failed), then again with Metro (which also by and large failed). Meanwhile, the key concepts of Aqua remain timelessly in macOS 25 years later. Function over form always!

[^0]: https://www.youtube.com/watch?v=dHrVGk0WwYM&t=381s

[^1]: https://www.youtube.com/watch?v=jGztGfRujSE
by throwaway34564 | 2025-12-08 16:04:22+00:00
Slightly off-topic - I was momentarily excited this was a replication of https://journals.aps.org/prapplied/abstract/10.1103/PhysRevA... (Electric Power Generation from Earth’s Rotation through its Own Magnetic Field)

I've been stalking the citations for this paper for a while now. Surely people would be scrambling to replicate these results. It could truly be transformative for the world if it works and is scale-able.

The science looks good to me
by mmaunder | 2025-12-08 16:04:26+00:00
This is why TDD is how you want to do AI dev. The more tests and test gates, the better. Include profiling in your standard run. Add telemetry like it’s going out of fashion. Teach it how to use the tools in AGENTS.md. And watch the output. Tests. Observability. Gates. Have a non negotiable connection with reality.
by HarHarVeryFunny | 2025-12-08 16:04:41+00:00
We likely learn/generate language in an auto-regressive way at least conceptually similar to an LLM, but this isn't just self-contained auto-regressive generation...

Humans use language to express something (facts, thoughts, etc), so you can consider these thoughts being expressed as a bias to the language generation process, similar perhaps to an image being used as a bias to the captioning part of an image captioning model, or language as a bias to an image generation model.
by heresie-dabord | 2025-12-08 16:04:53+00:00
> The thing that bothers me the most about LLMs is

What bothers me the most is the seemingly unshakable tendency of many people to anthropomorphise this class of software tool as though it is in any way capable of being human.

What is it going to take? Actual, significant loss of life in a medical (or worse, military) context?
by neom | 2025-12-08 16:04:56+00:00
As someone who has spent their whole career in growth and awareness for business building, I see why people hate ads so much now - personally I love running ads, trying to place a good ad in a good spot for someone who will genuinely appreciate it, it can be very rewarding... but late stage capitalism, aka fervent consumerism, has driven business into a real bad place, it's a shame because business and commerce is pretty fun, better than conquering via killing anyways.
by SoKamil | 2025-12-08 16:04:56+00:00
FoundationDB is awesome testing wise as they have deterministic simulation testing [1] that can simulate distributed and operating system failures.

> We wanted FoundationDB to survive failures of machines, networks, disks, clocks, racks, data centers, file systems, etc., so we created a simulation framework closely tied to Flow. By replacing physical interfaces with shims, replacing the main epoll-based run loop with a time-based simulation, and running multiple logical processes as concurrent Flow Actors, Simulation is able to conduct a deterministic simulation of an entire FoundationDB cluster within a single-thread! Even better, we are able to execute this simulation in a deterministic way, enabling us to reproduce problems and add instrumentation ex post facto. This incredible capability enabled us to build FoundationDB exclusively in simulation for the first 18 months and ensure exceptional fault tolerance long before it sent its first real network packet. For a database with as strong a contract as the FoundationDB, testing is crucial, and over the years we have run the equivalent of a trillion CPU-hours of simulated stress testing.

[1]https://pierrezemb.fr/posts/notes-about-foundationdb/#simula...
Working in 2 startup at the same time
by rafruf | 2025-12-08 16:04:56+00:00
I’m in a tough spot. I’m working for two startups, each with a group of close friends. Both startups are equally successful. I don’t want to leave either because I’m the guiding force for both. What should I do? How can I manage my work and both groups are planning to apply to YC?
by Yizahi | 2025-12-08 16:05:05+00:00
And other countries have freedom to scoff at that or even sanction trade with it (gasp, the horror!). Freedom works both ways, you know.

A fitting quote for the moment:

"Pacifism is objectively pro-fascist. This is elementary common sense. If you hamper the war effort of one side, you automatically help out that of the other." (c) George Orwell

Also regarding India and tokens, sure they are officially banned. But they are still a key link a in trade chain, that' why I mentioned Dubai and its neighbors. They act as a laundromat and a tumbler, obscuring financial flows. And one part of the Russia-India trade or Russia-Iran trade or even Russia-China trade involves tethers (USDT), without which exchanging rubles to rupee or to yuan directly would be highly problematic and risky. Tokens mitigate some of the risk by hiding financial flows through multiple jurisdictions.
by WorldMaker | 2025-12-08 16:05:37+00:00
I tend to refer to this as a Mixtape versus Remix spectrum distinction. A Mixtape plays the "hits" in about their original form, just in a new order. A Remix might sample the hits, but adds in new original content and takes it in a new direction.

Personally, I bounced out of Stranger Things S1 pretty hard because it seemed far too much like a Mixtape and not enough like a Remix than I wanted. The part that hit me was a feeling that entire monologues were lifted from the originals nearly verbatim with maybe a couple Proper Nouns swapped Mad Libs style. That can be incredibly fun (one that worked for me: William Shakespeare's Terminator the Second, was a creative mixtape of Shakespeare dialog reordered to retell Terminator 2), but at least for Stranger Things I kept having too many moments pulling me out of the story with "I've heard this before" because of the worse follow up feelings of "I'd just rather watch the originals, because they did it better" or "This isn't really adding much new or original to this moment".

But I realize there are a lot of opinions in the Mixtape versus Remix spectrum and also a lot of opinions on where something like Stranger Things falls. If it feels more like a Remix to you, that doesn't necessarily change that I found it too much like a Mixtape. (And vice versa, just because I find it too much of a Mixtape doesn't mean that I missed things that others felt made it a useful Remix, just that maybe my opinions on Mixtapes are stronger.)
by level09 | 2025-12-08 16:05:41+00:00
This is my actual stack, been using Enferno for years, ReadyKit is just the SaaS layer on top. No straying, more like cleaning up what I already run in production.

Stack choices: Flask for its elegant simplicity without hidden conventions, Vue with Vuetify over CDN to skip build-tool pain (massive productivity and time win btw), PostgreSQL because boring is reliable, Redis (optional) for sessions and caching, and Celery when background jobs are needed (optional too)
by FabHK | 2025-12-08 16:06:10+00:00
lol. No, PoW existed before. Putting it all together in this specific form was an innovation, if only a minor incremental one on the large body of work of state machine replication, enabling for the first time sybil resistance and thus permissionless distributed consensus. But all that gave us were crime tokens; haven't seen a use case but crypto yet.

Invest in stocks and bonds to maintain purchasing power.
by burnt-resistor | 2025-12-08 16:06:11+00:00
Austerity for military pork and welfare for the rich first, because there is critical underinvestment in economic and social infrastructure. Cutting healthcare and welfare the poor isn't going to solve anything except increase net costs and drive regional rural hospitals into insolvency. To the former, curtailment in an un-DOGE, organized, and evidenced-based manner would be appropriate.
by johnfn | 2025-12-08 16:06:22+00:00
“Setting up playwright” is about two sentences of a prompt to Claude. I know this because I’ve done it many times. The authors prompt in the post is only a few hundred words. (Most of the post is just LLM output.) I’m certain I could type that out in 5 minutes.

Do we really need another post where I time how long it takes to prompt Claude to create the Space Jam website?
by ryandrake | 2025-12-08 16:06:57+00:00
> You wouldn't go to a doctor, hear that you need an appendix removed, and feel "belittled and undermined"!

It happens more than you'd think, even in the HN comment section! Go to any thread where the topic is medical or diseases. Plenty of people distrust their doctor and advocate going to the doctor with your own crackpot theory you "researched" on WebMD. There's a huge anti-credential streak, even here. A lot of people see professional service providers of all kinds as "mere gatekeeping implementors of my own ideas" rather than experts in the field.