MurphyA/hacker-news huggingface dataset:
by vouwfietsman | 2025-09-27 09:59:14+00:00
Ok, since you're looking for sincere feedback.

Great vision, challenging the "scale" of current AI solutions is super valid, if only for the reason that humans don't learn like this.

Architecture: despite other comments, I am not so bothered with MMAP (if read only) but rather with the performance claims. If your total db is 13kb you should be answering queries at amazing speeds, because you're just running code on in-cache data at that point. The performance claim at this point means nothing, because what you're doing is not performance intensive.

Claims: A frontal attack on the current paradigm would at least have to include real semantic queries, which I think is not currently what you're doing, you're just doing language analytics like NLP. This is maybe how you intend to solve semantic queries later, but since this is not what you're doing, I think that should be clear from the get-go. Especially because the "scale" of the current AI paradigm has nothing to do with how the tokenization happens, but rather with how the statistical model is trained to answer semantic queries.

Finally, the example of "Find all Greek-origin technical terms" is a poor one because it is exactly the kind of "knowledge graph" question that was answerable before the current AI hype.

Nevertheless, love the effort, good luck!

(oh and btw: I'm not an expert, so if any of this is wrong, please correct me)
by danabramov | 2025-09-27 09:59:30+00:00
> doesn’t this imply that displaying comments under a post is a very expensive operation?

It would be expensive if you don’t do your own aggregation. The recommendation for social atproto apps is to aggregate the records you care about into a local database and essentially build your app-specific index/cache of the network.

Ctrl+F to this part: “Coincidentally, that’s the exact mechanism you would use for aggregation. You listen to events from all of your app users’ repositories, write them to a local database, and query that database as much as you like with zero extra latency.”.

(One interesting consequence is that indexing the network can also be done by someone on your behalf, see https://slices.network/ for an upcoming experiment in that area.)

> Also not totally clear to me (apologies if I missed it in the post) is where repositories live, I have a bsky account with my own domain, but I am not running anything to host a repo.

Yeah, Ctrl+F here: ”Note that https://alice.com and at://alice.com do not need to resolve to the same server. This is intentional so that having a nice handle like @alice.com doesn’t force Alice to host her own data, to mess with her website, or even to have a site at all. If she owns alice.com, she can point at://alice.com at any server.”

You can inspect where at:// points for your handle in an online browser like http://pdsls.dev. If you put your handle there you’ll see the physical server it resolves to at the top.

Presumably you signed up from Bluesky so yes, Bluesky is hosting your repo by default. You can move it somewhere else with no disruption. See https://whtwnd.com/bnewbold.net/3l5ii332pf32u for how to do this from CLI and https://pdsmoover.com/info.html for something less technical.
by NoiseBert69 | 2025-09-27 09:59:41+00:00
Looking at their GitHub there are some things upcomming

- They want to make device telemetry opt-in. This will cause the network to be more quiet

- A few patches ended up in master for handling corner cases of their flooding and next-hop routing algorithm

- There's also a merge request ("Add packet replay feature") which maybe can solve the reliability problem

Currently bigger Meshtastic networks are very very unreliable. I almost cannot traceroute a device that is one or two hops away. And our channels are not very saturated (10-20% ChUtil).
by mr-karan | 2025-09-27 09:59:47+00:00
It's great to see Typst getting more visibility. We migrated a real-world workload to it a couple of years back: generating and e-mailing 1.5M+ PDFs daily at my org, Zerodha.

Our previous pipeline was LaTeX-based (first pdflatex, then lualatex), but we were constantly fighting cryptic memory errors on large documents and huge Docker image sizes that slowed down boot times of our ephemeral workers.

Switching to Typst was a massive win for us. The single static binary resulted in tiny images and faster boot times. More importantly, the performance gains were huge. Overall compile times were ~3–4× faster than LaTeX. On really large documents (2000+ pages, mostly tables), Typst compiles in ~1 minute vs. ~18 minutes with lualatex.

Beyond performance, the better developer experience and good error messages was a nice bonus too.

We wrote a detailed post about the entire architecture - from the job orchestration with Nomad to the S3 optimizations and the Typst migration in particular. If you're curious, you can read it here: https://zerodha.tech/blog/1-5-million-pdfs-in-25-minutes
by ZeroGravitas | 2025-09-27 10:00:35+00:00
The 10% figure bothers me for other reasons.

Only 75% of people have clean water.

Why would anyone use that figure to argue against expanding clean water access to 100%? Or for rolling that progress backwards.

And if we can't even sort out clean water for a quarter of the globe then it makes sense they're not going to be recycling.

Perhaps more relevantly, nearly half of all global waste is "uncontrolled" so isn't recycled, composted, incinerated or even landfilled (and that's not even accounting for the quality of any of those systems).

So should we give up on landfills, indoor plumbing etc?

It's such a fundamentally stupid talking point, it could only sustained by propaganda.
by al_borland | 2025-09-27 10:01:46+00:00
When I was growing up Reduce, Reuse, Recycle was the manta that was all over mainstream media. Today, I never hear it anymore, it’s only about recycling.

I always thought Reduce, Reuse, Recycle was the correct order.

Reducing consumption is the best thing you can do, because things don’t need to be made in the first place.

Reuse means something else you may have otherwise bought doesn’t need to get made, which is another big win. I remember my grandpa’s shop growing up was full of old glass baby food jars to organize all his screws and things. Today someone would probably buy a plastic organizer and throw out the plastic baby food containers.

Recycle… something is still getting made. Some of the raw materials might be reused in the process, but it’s questionable how much, and how efficient the whole process is.

From what I’ve read, recycling program and push was paid for by the plastics industry. The hidden agenda was to get the public comfortable with single-use plastics, with the idea that they could be recycled, so it was no big deal. It seems they were amazingly successful in this effort to deceive the public.
by kelvinquee | 2025-09-27 10:01:53+00:00
I'm not sure. The GFW is rather tricky and is known to do DPI.

For now, yes we are communicating moves in exactly the way you described EventSource! Eg:

{ "fen": "rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq e3 0 1", "status": "active", "turn": "b", "white_ready": true, "black_ready": true, "spectators": 1, "in_check": false, "last_move": "e2e4", "game_over": false, "available_colors": [], "reservations": { "white": {"reserved": true, "expires_in": 0}, "black": {"reserved": true, "expires_in": 0} } }
by spacechild1 | 2025-09-27 10:02:31+00:00
> But you know what is really fun? Creating something from scratch. Sculpting it. Refactoring it. Trying things. Cutting things. Learning things. Exploring new ideas. All of the stuff that typically doesn't fit well when money is involved.

All my programming happens in the context of art (computer music and media art). I do all the things you describe while (indirectly) earning money with it. If you build something that you deeply enjoy, money won't take that joy away. I guess it's just hard to find this kind of satisfaction in typical corporate jobs where you're just a replacable wheel in the machine and your code will be replaced in a few years anyway.
by closewith | 2025-09-27 10:04:24+00:00
> And so it goes, until someone advises them to go to Ireland, where they can walk over the border to the UK, as in Northern Ireland.

This is absurd. The problem is the exact opposite, nearly all IPAs in Ireland come from the UK via the common travel area. Ireland is not in Schengen and is not reachable from the continent by small boat, so there is negligible migrant flow in the opposite direction.
by j4coh | 2025-09-27 10:04:50+00:00
It doesn’t even have to be art. If someone told me they were a chef and cooked some food but in reality had ordered it I’d think they were a bit of a moron for equating these things or thinking that by giving someone money or a request for something they were a creator, not a consumer.

It may be nice for society that ordering food is possible, but it doesn’t make one a chef to have done so.
by xg15 | 2025-09-27 10:10:02+00:00
If OP wants to approach this using machine learning instead of a deterministic algorithm, wouldn't this be exactly what they need?

Use the completed traces and part locations (complete with human post adjustments and all) as labels and the bare connectivity graph + "constraints" in some form as inputs.

Of course, as with all machine learning projects, the interface is deceptively simple but gives you no information how well the system can work or whether it can work at all...
by danigleba | 2025-09-27 10:10:05+00:00
I got my first customer from a reddit post.

Since then I've generated millions of views off reddit, and most of my revenue has come from it.

But writing high-quality posts that go viral took all my time.

So I built Karma kit.

A chain of AI agents that write viral reddit posts for me.

Making dozens of posts an hour, getting thousands of views and hundreds of upvotes.
by Yokolos | 2025-09-27 10:10:23+00:00
I think it's an incredibly spoiled perspective that anything that becomes a chore is no longer fun or worth doing. Practicing piano is a chore. Repeating the same piece, the same finger movements for an hour or two isn't necessarily fun. But it is satisfying at the end of the day and it is fulfilling. Fun isn't necessarily the goal. I can spend hours banging my head against the wall while trying to build a piece of software. It can be tedious, frustrating, and tiring. It's rarely "fun". But in the end, it's satisfying and it's fulfilling and tomorrow I'll do it again because I want to.
by wtetzner | 2025-09-27 10:10:35+00:00
> The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by the exertion of the imagination.

This is both a reason that programming is enjoyable, as well as one of the reasons it's so easy to end up with a giant mess that kinda sorta works well enough to get by.
by ZeroGravitas | 2025-09-27 10:11:00+00:00
A couple of UK focused nerd YouTubers pushing back on this narrative:

https://www.youtube.com/watch?v=iOtrvBdRx8I

https://www.youtube.com/watch?v=fuwaOiTHl7U

Tl;Dr charge the producers of packaging for what happens to it after use.

They respond almost immediately to that financial incentive by both reducing unnecessary usage but also improving recycling of what is still necessary because it's the second best option after funding cheap propaganda to pass the buck onto others.

Contrast the latest form of that cheap propaganda, which is to point fingers at oil companies for promoting recycling that "can't economically work" with the reality that when their own money is on the line they adopt it rapidly.

Meanwhile America is doing that "Only developed country that is failing to do this claims it's not possible" thing again.
by KoolKat23 | 2025-09-27 10:11:15+00:00
I guess it depends on its design (does it set up the dystopian infrastructure or not) and then more so on the legislation around it. It's already not uncommon to need your passport for work, making it a law to produce for any work would be the actual thing that changes this.

I get your point about IrishID, but be realistic, none of this is surprising considering the current state of things. I could make some controversial statements in this regard but I'll avoid the flames (I sympathise with you before you place me in the wrong box).
by spiritplumber | 2025-09-27 10:11:53+00:00
Cool! We made https://www.robots-everywhere.com/cellsol/ in 2020 but I think Meshtastic is further along. However, ours can run on an atmel328.

We also respect and forward Meshtastic and DisasterRadio packets, but there is currently no reciprocity. What can be done about it?
by mikaraento | 2025-09-27 10:12:48+00:00
Sorry, I meant to emphasize _different_ mechanical connections. That a sensor that detects pressure has a different mechanical linkage than the one detecting vibration. So you need multiple different manufacturing techniques to replicate that at correspondingly higher cost.

The “more than 10000” also has a large impact in size (sensors need to be very small) and cost (you are not paying for one sensor but 10000).

Of course some applications can do with much less. IIUC the article is all about a _universal_ humanoid robot, able to do _all_ tasks.
by rdmuser | 2025-09-27 10:13:17+00:00
Personally I like the newsminimalist rss feed for simple ai evaluated and summarized news. I don't have much experience with the site itself but it's likely also worth a check. The dev is an active part of hn.

https://www.newsminimalist.com/

The dev and the launch post from 2023:

https://news.ycombinator.com/item?id=35795388

https://news.ycombinator.com/user?id=t0bia_s
by thaumasiotes | 2025-09-27 10:14:33+00:00
> Even Kids can formalize the multivariate equation in Litex in 2 minutes, while it [takes] an experienced expert hours of work in Lean 4.

Well, I propose an alternative proof in lean4:

import Mathlib.Tactic example (x y : ℝ) (h₁ : 2 * x + 3 * y = 10) (h₂ : 4 * x + 5 * y = 14) : x = -4 ∧ y = 6 := by have hy : y = 6 := by linear_combination 2 * h₁ - h₂ have hx : x = -4 := by -- you'd think h₁ - 3 * hy would work, but it won't linear_combination 1/2 * h₁ - 3/2 * hy exact ⟨hx, hy⟩

---

One thing I like about the lean proof, as opposed to the litex proof, is that it specifies why the steps are correct. If litex's strategy is "you describe the steps you want to take, and litex will automatically figure out why they're correct", how are you supposed to do any nontrivial proofs?
by noodlesUK | 2025-09-27 10:15:11+00:00
The UK has no notion of a person number or national ID number that is tied to citizenship. Therefore it is not possible to prove British citizenship except with a British Citizen passport, naturalisation certificate or pre-1983 birth certificate.

It’s therefore a lot harder to prove citizenship for an initial passport application in certain circumstances than you might expect. You need to prove that you have an unbroken link of people born in the UK to someone born before 1983, and as time goes on that will mean even more generations. Right now you typically need to provide your birth certificate, up to 2x parents birth certificate, and up to 4x grandparents birth certificates.

In many other countries the birth certificate will have the person numbers of the parents, which will mean there’s essentially guaranteed to be a record of the citizenship of the parents that the state can check. Alternatively there’s a national ID scheme that helps bootstrap this information early in life.
by LightBug1 | 2025-09-27 10:15:25+00:00
There's a pendulum swinging, for better or worse. But people do care.

When I was in growing up in the 90's 00's ... I distinctly remember saying to myself: I know, logically, that racism exists but I cannot remember the last time I experienced direct racism, if ever. I know it existed, indirectly, but I, nor my close family or friends, experienced it. The first time I did experience, and increasingly so, has been in the last 10 years.

I think it was terrible until the 70's and 80's (history and family anecdotes prove this, you're right) ... it went relatively underground in the 90's and early 00's which made life great ... and when society was strained by the global financial crash, etc, things have swung back again.

I really put this whole reality at the feet of the bankers and it's amplified by social media. They were the lead dominos in all of this. They released the wankers and populism.

And getting back on topic, society has broken apart again and exposed racism again. You're right. Nothing has changed. But life is better when the assholes are in their caves stewing their hatred rather than emerging and sharing it with everyone.

As above, it's different in how it manifests at the moment, but it definitely rhymes, and at its heart, it's the same hatred manifest in a different way.

The only thing we can hope for is the pendulum swinging back, so the wankers are put back in their caves and we can get on with a relatively normal life again.
by Sophira | 2025-09-27 10:15:32+00:00
A lot of these arguments also apply to newsgroups (Usenet), however. It has the same features of being able to use whatever client you want, and can be archived.

Archives can be searched fairly easily, although admittedly most will be incomplete (hence Deja and Google Groups back in the day).

Some newsgroups are still active, but not many. Why? What went wrong?
by bluenose69 | 2025-09-27 10:17:30+00:00
I've started using typst for small local documents that I would previously have written in markdown (or R-markdown). Typst offers programming features that are very helpful for small writing tasks that need more customization than markdown provides but that don't need the wildly expansive set of templates available in the latex world.

Like other academics, I plan to stick with latex for journal articles and books, unless publishers provide support for typst.

Markdown still has a place for files on github, because that means that web browsers will display formatted material, not just the raw code. A similar thing applies to code documentation, with many systems (R, Julia, etc) supporting various flavours of markdown.

I recommend typst to students for small tasks like assignments. It offers more typesetting power than markdown, it's a great replacement for msword, and it's easier to learn than latex.

Long story short, typst fills a niche. But it's not the be-all and end-all, at least in my (natural science) field.
by rcxdude | 2025-09-27 10:17:35+00:00
I mean, it was. It was also a genuinely big problem, but given the scope for hype, even massive problems can be overblown as well.

(And the media is pretty good at it. I'm pretty sure if a comet was about to hit the planet tomorrow and wipe out humanity there would still be an article that somehow manages to make it sound worse)
by AlgebraFox | 2025-09-27 10:19:22+00:00
Reports say IPv6 is gaining market share. But most ISPs are delegating /64 prefix. This is as useless as carrier grade NAT. Atleast with IPv4, I can create NATed subnets for guests, IoT devices etc. That's not possible with IPv6.

Overall it's a negative progress. Worse than IPv4+NAT.
by unsungNovelty | 2025-09-27 10:19:23+00:00
CoC should facilitate free speech and thought without the worry of repurcussions. But in reality it is now a tool to say "I don't like you/your thoughts, so here comes the hammer". I have seen some of the flimsiest excuses used to ban someone. But the same reason doesn't warrant ban on the people the admins like.

If you are actually right, then talk about in the open. Show restraint from name calling. Not everybody is out there with a hidden agenda. Lobste.rs is a great example of this. No shadow banning. Everything has a reason and accountability.

Shoutout to people like JT from the BSD Now podcast as well. I am a fan of his moderation on the Telegram channel. Atleast as far as I have witnessed, I have seen some of the controversial topics going on and on but not banned for difference of opinions. A lot of de-escalation and patience. It would've been so easy for him to just do some Hammering. But that's not how it goes most of the time.

We need more spaces like this.
by burnt-resistor | 2025-09-27 10:19:25+00:00
I'd like to know what kind of magical Flavor Aid I should add to water served to VCs for them to hand me piles of money to work on something Philip E. Mason will eventually get around to debunking. Or do I just need a Harvard CS degree or 3 EE PhDs and then it's automatic mana from heaven via pedigree?
by eigenspace | 2025-09-27 10:20:39+00:00
The nice thing about Typst is that I think they've built up a community and business model that should disincentivize enshittification. The compiler is open source, and there's a large number of people using it now. If they start enshittifying, it can just be forked and maintained by the community at the very least at the level of functionality it's currently at, which is good enough for a large segment of people.
by jstimpfle | 2025-09-27 10:21:17+00:00
From a skim, my impression was that there was a specific stream processing problem discussed but I probably have gotten that wrong, or a comment was edited. If I understand better now, in the OP, Rust streaming features are being used here to achieve essentially a typecast, and the streaming transformations (looking at each individual element) are supposed to be optimized away. And I restate that this is essentially solving a language-inflicted problem, not some interesting performance problem.

For sure, linking "graphs" generally have random connections so obviously I'm not saying that linking in general is a trivial stream processing problem.

Thinking about it though, linking graphs are _almost_ free of cyclic dependencies in practice (even when looking at entire chunks of code or whole object files as units), so there are some basic tricks we can use so we don't need to load all compilation units into RAM as I think you claimed we need to. I don't think it's necessary to load all the data in RAM.

The way I understand it, linking is essentially concatenating object files (or rather the sections contained in them) into a final executable. But while doing so, relocations must be applied, so object code must not be written before all symbols referenced by that code are known. This can be done in a granular fashion, based on fixed-size blocks. You collect all relocations in per-block lists (unsorted, think std::vector but it can be optimized). When a symbol or code location becomes known (i.e. looking at the library that has the required symbol), that information gets appended to the list. When a list is full (i.e. all relocations known) that block can be committed to the file while applying all fixups. Applying fixups is still random writes but only to fixed size blocks.

That way the problem is almost reduced to streaming fixed size blocks in practice. Typically, all symbols that some object file depends on are typically resolved by the object files that were already added before. So most new chunks that we're looking at can immediately be fixed up and be streamed to the output file. (In fact some linkers are kind of strict about the order of objects that you pass on the command line). What does grow more-or-less linearly though is the set of accumulated symbols (e.g., name + offset) as we add more and more objects files to the output executable.

I don't know a lot about LTO but it seems just like an extension of the same problem. Balancing memory consumption and link times with output performance is partly the user's job. Most projects should probably just disable LTO even for release builds. But making LTO fast in a linker is probably all about using the right approach, doing things in the right order, grouping work to avoid reads and writes (both RAM and disk).
by ryangibb | 2025-09-27 10:22:27+00:00
Given the website, is worth noting that keyboards like this can avoid the infamous Emacs pinky. I've got the keys two rows below the homerow bound to the modifiers right alt, super, alt, control, and shift. And reversed on the other hand. This makes any modifier shortcuts very ergonomic -- one hand 'chords' the modifiers and the other hits the non-modifier key.