MurphyA/hacker-news huggingface dataset:
Show HN: Kafka-compatible streaming on S3 with stateless brokers
by 2pk03 | 2026-01-06 16:53:58+00:00
Hi HN,

KafScale is Kafka-compatible streaming, k8s native, where S3 is the source of truth and brokers hold no persistent state. Written in Go, runs on Kubernetes.

Built this after years of operating Kafka and hitting the same walls: broker failures that take hours to recover, partition rebalancing that blocks deploys, disk capacity planning that never ends.

How it works:

- Producers and consumers use standard Kafka clients
- Brokers buffer in memory, flush to S3
- etcd stores metadata and consumer group state
- Recovery means restarting a pod and reading from S3
- Optional Iceberg processor reads segments directly from S3, bypasses brokers entirely for batch/analytical workloads

What you give up: latency is 400-500ms (S3 round-trip), no transactions, no compacted topics. It's not a 100% replacement.

What you get: brokers are disposable, scaling is just replica count, no disk management, direct access to streamed data over S3 ACL

License: Apache 2.0
GitHub: https://github.com/novatechflow/kafscale
by Noaidi | 2026-01-06 16:54:00+00:00
> China is essentially a pinko branded fascist

Sorry, China is a unitary communist state. Fascism and communism are the opposite. Both have state control but the fascist control is from the corporations.

The reason China has better chips is because they have a form of socialism

Don't be a propagandist regurgitating propagandists ideas, it does nothing for intelligent conversation.
by 1970-01-01 | 2026-01-06 16:54:21+00:00
A 45/65 balance feels like it's at the optimal balance for interesting. Users are expected to continually upvote more and more boring posts if the user pool grows with noise. If the system stabilizes to 50/50, the content would trend toward mediocre but harmless.. Ergo, HN really is a cut above social media.
by 7777777phil | 2026-01-06 16:54:24+00:00
OP here :)
On skepticism being lumped with negativity: partially true. The SST-2 training task treats critical evaluation as negative sentiment. I should clarify that "negative" here means evaluative or critical, not hostile. HN's culture of substantive critique registers as negative by these metrics, but that's arguably a feature of technical discourse rather than toxicity.

On the neutral category: the model outputs continuous scores from 0 to 1, so neutrality does exist around 0.5. The bimodal distribution with peaks at roughly 0.0 and 0.95 reflects how HN users tend toward strong evaluative positions. Three-class models could provide additional perspective, and that's worth exploring in future work.

Also love your meta-observation. Imo your comment is critical, substantive, and engaging. By sentiment metrics it's "negative," but functionally it's high-quality discourse. But that's exactly how I read the data: HN's negativity is constructive critique that drives engagement, not hostility.
by Ajedi32 | 2026-01-06 16:54:41+00:00
I think the problem is that telling someone they're wrong without hurting their ego is a very difficult skill to learn. Even if you're rally good at it, you'll still often fail because sometimes people just don't want to be disagreed with regardless of how you phrase it. It's far easier for the AI to learn to be a sycophant instead (or on the opposite side of the spectrum, to learn to just not care about hurting people's feelings).
by rvnx | 2026-01-06 16:55:07+00:00
I think in the mid-term, many of these AI companies are going to run out of cash.

Users are more and more going to be able to run models locally so it's a race to nowhere (all you need to have a very good model, is to have a Mac with 128 GB of memory, but at 16 GB you already have something usable but not so nice)

The big winners are going to be the memory makers
by veilrap | 2026-01-06 16:55:42+00:00
The fact that Jellyfin lacks a AppleTV/tvOS app seems like it continues to make it a dealbreaker... at least for my setup.

I hear people recommending clients like Infuse, but it feels odd to swap out Plex at this point if I can't go all in on the open source side of things.

Am I missing something here wrt Jellyfin clients? I guess I could try running it side-by-side with Plex and see how it goes.
by GoatInGrey | 2026-01-06 16:55:48+00:00
At its worst, Uber had a net margin of -~60%. The AI labs are all running at least negative triple digit margins, some running negative quadruple digit margins. This is why AGI has been "forecasted" to death by the labs, because investors need the promise of infinite automation to stomach the losses.

Anyway, in this instance, what you received for $20 in 2025 will run you somewhere in the range of $60-$90 in 2027/2028. In the interim, you will likely see that $30-$40 of service gets you what cost $20 in 2025. The most likely avenue for this will be reduction in subscription user limits, and for API customers premiumization through substitution. The latter being a situation where what would be the next Claude Sonnet model is now sold as Claude Opus, for example.

The only way the math works for the consumer is if the user base has become dependent on the service instead of remaining in a conventional cost/benefit relationship.
by rvz | 2026-01-06 16:55:55+00:00
This was the xmas rug-pull. Remember [0], even worse if they stopped it and reduced it further.

L̶e̶a̶r̶n̶ ̶t̶o̶ ̶c̶o̶d̶e̶

L̶e̶a̶r̶n̶ ̶t̶o̶ ̶c̶l̶a̶u̶d̶e̶

Learn to think.

[0] https://support.claude.com/en/articles/13163666-holiday-2025...
by gosub100 | 2026-01-06 16:56:23+00:00
Rolling coal is rude an obnoxious but doesn't "destroy" anything. I think you are projecting something personal against the stereotype guy that does that. Just like someone who was bullied might irrationally hate tough looking bikers.

Some men (and women!) like large and overpowered trucks. You don't have to like them, but you should praise the freedom that this country gives us to choose our own pursuits.
by apparent | 2026-01-06 16:56:28+00:00
I think a lot of people bought AWs because they seemed like the right thing to get, integrated easily, and were more or less easy to use.

But most people I know who have AWs don't use most of the functionalities they provide. If you went up to 20 random AW wearers and ask them if they would give up a bunch of features they don't use (like the awful Siri assistant) in exchange for 15-30x the battery life, I think a lot of them would say yes.

Add onto that the fact that Pebbles are cheaper than AWs, and I think we're going to see a non-trivial number of people "upgrading" from AWs to Pebbles when the batteries start to degrade.
by 7777777phil | 2026-01-06 16:56:58+00:00
You're making a distinction the paper should address more directly. The classifier can't tell the difference between "this API design is fundamentally flawed because X" and "this company is terrible" (as noted in an earlier reply). Both register as negative by models trained on reviews and social media.

You're also right that HN's moderation probably removes hostile content quickly (which is why I prefer this platform to other roptions tbh). So the negativity we observe is mostly substantive critique rather than personal attacks.

That said, I'd push back a bit on whether this makes the finding less interesting. If anything, the opposite seems true. The fact that HN's "negativity" is constructive criticism, and that this criticism correlates with 27% higher engagement, tells us something about how technical communities value critical analysis over promotional framing. The classifier limitation is real (also see my other replies), but the engagement correlation holds whether we call it "negative sentiment" or "evaluative critique."

I'll add a limitations section to make the terminology clearer: "negative sentiment" as used here means evaluative criticism detected by SST-2-trained models, not personal attacks or toxic comments. Thanks for your feedback!
by Workaccount2 | 2026-01-06 16:57:26+00:00
The assholes on reddit aren't the problem, often they are the people who are closest to breaking free from the swamp (yes, some are just assholes).

The problem reddit has is the celebration of it's doomerism, even in the small hobby subs the vibe is still present. The highest upvoted comments are so nauseatingly repetitive and formulaic, ridden with whatever the contemporary dogma of reddit is, substantiated by snowballs of echo-chamber fallacy with pebbles of truth in the middle.
by forkbikash | 2026-01-06 16:57:27+00:00
Stop building software. Start building the company that builds software.

Most "AI tools" are just fancy wrappers for ChatGPT. You ask a question, it gives you a suggestion, and then YOU do the work.

I got tired of the manual labor. So I built Agent Rabbit.

We’re an execution engine.

Imagine an AI that doesn’t just tell you how to write code, but:
Spins up its own dev environment.
Navigates your GitHub, AWS, and Slack, jira and many more tools.
Actually clicks the buttons, fills the forms, and ships the features while you’re getting coffee.

This is the world’s first AI Software Company. We’ve built agents that don’t just plan—they execute. Whether it's a multi-repo complex task or a simple browser action, the "Rabbit" does the heavy lifting.

If you’re a founder, an indie hacker, or a dev who is tired of being the middleman between the LLM and the terminal... this is for you.
by shuka | 2026-01-06 16:57:42+00:00
I've used Apple Notes for years because it syncs fast and stays out of the way. But when I'm writing in the terminal, there's always been friction getting Markdown into Apple Notes.

Existing tools were either bloated or read-only. So I built Stash: push a Markdown file to Apple Notes, pull changes back. It uses YAML frontmatter to track which note belongs to which file.

Built with Bash, AppleScript, and Pandoc. No databases, no daemons, no config files. Install via Homebrew.
Happy to take suggestions and answer questions about the quirks I ran into along the way.
by MadnessASAP | 2026-01-06 16:57:44+00:00
I would also like to thank Jellyfin and the other software packages in its orbit for motivating me to keep my homelab in good running order. That and Home Assistant.

When compared to the current breed of streaming services it really shows the difference between something designed to drive up engagement and revenue while driving down cost vs something designed to actually be useful and pleasant.

Also I hadn't heard of OCIS, but it looks like something I want. So thanks for that.
by fooey | 2026-01-06 16:57:48+00:00
I keep asking our product team what the plan is when our API/Infra costs eventually skyrocket and our AI features become unsustainable, but there's absolutely zero preparedness.

We throw in all these cute little AI features to fill out marketing bullet points because they're basically free, but if they had real cost we're going to have no choice but to take them away.
by cootsnuck | 2026-01-06 16:57:51+00:00
Problem is that the frontier models are nowhere near 99% reliable. Orchestration and good system design is how you get reliability. Yes, the frontier models still are going to be better by default than open source models. But the LLM is still only a component in a broader system. What's seeming to be actually necessary for any high-usage worthwhile use case is making your model task specific (via fine-tuning / post-training / RL). I build these systems for enterprises. The frontier models are not enough.
by DrBazza | 2026-01-06 16:57:54+00:00
In what scenarios are they terrible? I hope not every scenario. I've found Codex adequate for refactoring and unit tests. I've not used it in anger to write any significant new code.

I suppose part of the problem is that training a model on publicly available C++ isn't going to be great because syntactically broken code gets posted to the web all the time, along with suboptimal solutions. I recall a talk saying that functional languages are better for agents because the code published publicly is formally correct.
by estebank | 2026-01-06 16:58:26+00:00
> the C-style approach is to extract a minimal subset of the dependency and tightly review it and integrate it into your code. The Rust/Python approach is to use cargo/pip and treat the dependency as a black box outside your project.

The Rust approach is to split-off a minimal subset of functionality from your project onto an independent sub-crate, which can then be depended on and audited independently from the larger project. You don't need to get all of ripgrep[1] in order to get access to its engine[2] (which is further disentangled for more granular use).

Beyond the specifics of how you acquire and keep that code you depend on up to date (including checking for CVEs), the work to check the code from your dependencies is roughly the same and scales with the size of the code. More, smaller dependencies vs one large dependency makes no difference if the aggregate of the former is roughly the size of the monolith. And if you're splitting off code from a monolith, you're running the risk of using it in a way that it was never designed to work (for example, maybe it relies on invariants maintained by other parts of the library).

In my opinion, more, smaller dependencies managed by a system capable of keeping track of the specific version of code you depend on, which structured data that allows you to perform checks on all your dependencies at once in an automated way is a much better engineering practice than "copy some code from some project". Vendoring is anathema to proper security practices (unless you have other mechanisms to deal with the vendoring, at which point you have a package manager by another name).

[1]: https://crates.io/crates/ripgrep

[2]: https://crates.io/crates/grep/
by amluto | 2026-01-06 16:58:59+00:00
C lacks safe primitives or non-error-prone ways to build abstractions to refer to business objects. There are no safe string references, let along ways to safely manipulate strings. Want to iterate over or index into a result set? You can try to remember to put bounds checks into every API function.

But even with explicit bounds checks, C has an ace up its sleeve.

int cost_of_nth_item(int n) {
if (n < 0 || n >= num_items)
return -1; // error handling
…
}

Safe, right? Not so fast, because if the caller has a code path that forgets to initialize the argument, it’s UB.
by sylware | 2026-01-06 16:59:19+00:00
I am talking about C syntax, not absurdely grotesque ultra-complex syntax like the ones from c++, java and similar (is that true that rust syntax is not that much less worse than c++ one now?).

We need a new C, fixed, leaner and less complex: primitives are all sized (u64, u8, f64, etc), only one loop primitive (loop{}), no switch, no enum, no typedef, no typeof and other c11 _generic/etc, no integer promotion, no implicit casts (except for void* and literal numbers?), real hard compiler constants, current "const" must go away (I don't even talk about "restrict" and similar), no anonymous code block, explicit differentiation between reinterpret,runtime,etc casts?, anonymous struct/union for complex memory layouts, and everything else I am currently forgetting about.

(the expression evaluation of the C pre-processor would require serious modifications too)

We need clean and clear cut definitions, in regards of ABIs, of argument passing by value of aggregates like structs/arrays, you know, mostly for those pesky complex numbers and div_t.

We need keywords and official very-likely-to-be-inline-or-hardware-accelerated-functions, something in between the standard library and the compiler keywords, for modern hardware architecture programming like memory barriers, atomics, byte swapping, some bitwise operations (like popcnt), memcpy, memcmp, memmove, etc.

In the end, I would prefer to have a standard machine ISA (RISC-V) and code everything directly in assembly, or with very high level languages (like [ecma|java]script, lua, etc) with a interpreter coded in that standard machine ISA assembly (all that with a reasonable SDK, ofc)
by pluralmonad | 2026-01-06 16:59:48+00:00
Limiting human liability through coercion has always been the problem. I signed no contract with any LLC founder to limit their personally liability. Yet somehow some bureaucrat hundreds of miles away signed for me and now none of us can hold people accountable for their actions without armed men preventing it. Just look at what companies like Dupont, Monsanto, Bayer have done to entire towns. Limited liability is the problem. Uncle Sam is the worst uncle.
by lproven | 2026-01-06 16:59:48+00:00
There were at least 2 separate and independent TV series, but I highly recommend going to the original: the 2 later novels from Douglas Adams, after he wrote the Hitchhiker's Guide to the Galaxy.

https://en.wikipedia.org/wiki/Dirk_Gently%27s_Holistic_Detec...

https://en.wikipedia.org/wiki/The_Long_Dark_Tea-Time_of_the_...

In the novels, a stuck sofa is revealed to have got there because a retired Time Lord, Professor Urban Chronotis, briefly materialised his TARDIS in such a place it provided a doorway. So the books are extremely loosely tied into the Dr Who universe.

Don't let this put you off.
by rozap | 2026-01-06 16:59:59+00:00
Both things can be true. It's true there's a whole circle jerk around complaining about how the web used to be better, and the much of indie web movement is just talking in circles about this phenomenon.

It is also true that Facebook and its ilk did destroy huge swaths of the online community. Example: I'm a automotive tinkerer, and online forums used to be a rich source of information, community, crazy builds...people actually creating stuff for the sake of creating. All that is gone now and the purity of an open space to put creative pursuits has been infiltrated with perverse engagement incentives, ads, algorithmic curation and the like.

I get what you're saying, but it doesn't mean that OP is wrong even if you find it exhausting.
by cloverich | 2026-01-06 17:00:16+00:00
This is well known, and why forums that wanted to maintain their quality would consistently lock such threads going back at least 20+ years when I started using forums. Reddit, Facebook, et al, do the opposite. Its why they feel so bad to use over time - they are engineered to tap into this and to promote it. HN thrives because they very consciously do the opposite.

I'm sure many of us would take it much further, but I hope we can appreciate its not an easy task.
by _jab | 2026-01-06 17:00:25+00:00
I've often wondered whether the world would be better without ads. The incentive to create services (especially in social media) that strive to addict their users feels toxic to society. Often, it feels uncertain whether these services are providing actual value, and I suspect that whether a user would pay for a service in lieu of watching ads is incidentally a good barometer for whether real value is present.

Don't get me wrong, I'm well aware this is impractical. But it's fun to think about sometimes.
by catapart | 2026-01-06 17:00:30+00:00
Not a great regulatory move, in my opinion. But I really wish ad companies would implement this rule across the board. If you can't sell me on your ad in 5 seconds, it's unlikely you can sell me on your product in 15 or 30 seconds. And if your product is of any interest to me whatsoever, I'm happy to continue watching the ad. I sit through movie trailers and tech ads all the time, even with an option to skip. But I have no use for seeing the entire Dawn dish soap's aw-shucks, faux-folksy ad play out. In five seconds, you can remind me that dawn exists, fulfilling the main purpose of the ad, and I can get on with the content I'm actually interested in.
by azornathogron | 2026-01-06 17:00:45+00:00
There isn't sufficient space in the stations or underground sections of the line for these hypothetical train-carriage-sized buses to manoeuvre around each other. Plus with this many people per "bus" and this many people using the system, how do you intend to guide everyone onto the right bus that's going to stop at the station they want, without every bus just stopping at every station?
by osamagirl69 | 2026-01-06 17:01:27+00:00
The second to last bullet point on the post is about the Samsung client, which (as you are aware) has been written but is currently held up in review Samsung. The first round of review took about 3 months(?!), and they are working through reproducing the vaguely stated issues found in the review.

It is thankless work, but they are actually quite close!
by mindcrash | 2026-01-06 17:01:30+00:00
> Losing 31 ally nations (possibly more) overnight and all those facilities would be a long-term military disaster, and a longer-term political disaster.

And they should also consider all equipment stored at those facilities, including the B61 nukes, null and void. Because their former NATO allies will never allow the US to take equipment which could be used against them later on back home.
by iso1631 | 2026-01-06 17:01:38+00:00
Yet at a mere £4.4m a year to become a billionaire a FTSE100 boss would have to work 227 years before looking into tax.

As it stands someone on £4.4m a year only picks up £2.4m, with £2m going to the taxman, as well as an extra £660k the employer pays.

Someone picking up £1b through share price increase will pick up £720m after tax -- it will take the FTSE boss 300 years to earn that much.
by epolanski | 2026-01-06 17:02:28+00:00
> The C language is old and boring. It is a well-known and well-understood language.

This is such an under appreciated feature.

I'm tired of languages that just keep adding and adding to a language.

Work on the standard library or ecosystem, but stop changing the language otherwise you'll never end up anywhere because there is always going to be something that would benefit from some love.

Hell, I'd say the same is true for many libraries.
by CharlesW | 2026-01-06 17:02:42+00:00
There Swiftfin, Jellyfin Mobile, and Streamyfin at least. My forthcoming iOS-only music player has first-class Jellyfin support (beta sign-up: https://forms.gle/AGLePh9RtaYEfDH6A) if you're looking for a dedicated, offline-capable music app.
by bombcar | 2026-01-06 17:03:06+00:00
Yeah he brings up the points I would (which is that "online image" is not the precursor of webcomics, but print and other "indie" comics are).

The first scanner was made in 1957, using that to scan a Peanuts comic would be some form of "digital comic image" but not a webcomic as we consider it.

I do also thing there's something the kids would call a "meme" that these might be closer to; a comic does NOT have to have a narrative or overarching story (though they often seem to devolve to that) - if the only Far Side comic to exist was Cow Tools it'd still clearly be a comic.
by mort96 | 2026-01-06 17:03:21+00:00
Democrats lost, in part, by giving in to the republican narrative around immigration. They gave up on arguing that immigration can be good, and started accepting the republican framing of it as a problem to be solved. This strategy is doomed from the start, since republicans will always be willing to go harder on anti-immigration rhetoric and action than the democrats will.

According to Gallup [1], only 47% of US Americans thought that immigration should decrease in 2020. That number had held more or less steady since at least 2000. But it grew steadily over Biden's term, reaching 88% in 2024. This, I believe, is a reflection of how the democrats shifted their rhetoric to be "tough on immigration". And it handed Trump a populace that was primed to be more susceptible than ever to his much more aggressive immigration rhetoric.

Of course, this is just a small part of a much bigger picture, but I don't exactly think it helped.

[1] https://news.gallup.com/poll/692522/surge-concern-immigratio...