MurphyA/hacker-news huggingface dataset:
by anishathalye | 2025-09-11 15:29:38+00:00
Hi HN!

I've been thinking a lot about semantic data processing recently. A lot of the attention in AI has been on agents and chatbots (e.g., Claude Code or Claude Desktop), and I think semantic data processing is not well-served by such tools (or frameworks designed for implementing such tools, like LangChain).

As I was working on some concrete semantic data processing problems and writing a lot of Python code (to call LLMs in a for loop, for example, and then adding more and more code to do things like I/O concurrency and caching), I wanted to figure out how to disentangle data processing pipeline logic from LLM orchestration. Functional programming primitives (map, reduce, etc.), common in data processing systems like MapReduce/Flume/Spark, seemed like a natural fit, so I implemented semantic versions of these operators. It's been pretty effective for the data processing tasks I've been trying to do.

This blog post shares some more details on the story here and elaborates what I like about this approach to semantic data processing. It also covers some of the related work in this area (like DocETL from Berkeley's EPIC Data Lab, LOTUS from Stanford and Berkeley, and Palimpzest from MIT's Data Systems Group).

Like a lot of my past work, the software itself isn't all that fancy; but it might change the way you think!

The software is open-source at https://github.com/anishathalye/semlib. I'm very curious to hear the Hacker News community's thoughts!
by colechristensen | 2025-09-11 15:29:55+00:00
What was needed is your own personal "algorithm" to provide you a feed of the things you'd subscribed to so you didn't have to read and check off every published thing. Imagine if you actually had control of this feed and could have the social aspect but tweaked to your preferences instead of somebody else's

In fact it's not such a bad idea to write this software now
by businessmate | 2025-09-11 15:30:12+00:00
Enterprise AI Visibility is the new frontier of brand management. As generative AI systems like ChatGPT, Gemini, and Perplexity increasingly serve as the first point of discovery for consumers, enterprise leaders face a new strategic challenge: is our brand visible and accurately represented when customers ask AI assistants about our category? This white paper introduces a rigorous methodology for conducting an Enterprise AI Visibility Audit using the Prompt Share of Search (PSOS™) metric, a cornerstone of the AIVO Standard™. It provides a step-by-step framework for:\n\n- Defining the scope of relevant categories, products, and markets. \n- Designing natural-language prompt sets aligned to customer intent. \n- Systematically querying major LLMs (e.g., ChatGPT, Gemini, Claude, Perplexity). \n- Measuring brand presence with PSOS™ and benchmarking against competitors. \n- Detecting risks of invisibility or misinformation as models retrain over time. \n- Translating results into board-ready governance KPIs. \n\nFor CMOs, Chief Digital Officers, and corporate boards, this audit process creates transparency, mitigates reputational risk, and establishes a new class of performance metrics suited to an AI-mediated discovery environment. The paper also situates the audit within the broader context of enterprise marketing strategy, drawing on insights from the AIVO 100™ Global Report (Open Future, 2025), which revealed that fewer than 40% of leading CPG brands appear in the top three generative AI responses to purchase-driven queries.
by thebruce87m | 2025-09-11 15:30:12+00:00
> Don't make it sound like Apple is some sort of victim of regulation who's being picked on the whole time.

I didn’t say this. I would merely accept other companies being held to the same standard.

> They will not get fines and lawsuits if they make a foldable phone with a display that degrades over time.

This will 100% happen. When it does I will come back and post here.
by mingus88 | 2025-09-11 15:31:02+00:00
I’ve been using RSS feed for nearly 20 years now. Why we ever strayed to some endless algorithm is beyond me.

I have a list of chronologically sorted articles from sources I trust. That’s it and that’s all.

I read them until they are read. Then I close the app and do other stuff until tomorrow.

Is it any wonder Google killed off Reader around the same time it tried to launch Google+? Managing our own feeds was never going to peak capitalism.
by javchz | 2025-09-11 15:31:05+00:00
Yes, but even lifestyle changes (like a diet low in glycemic load and building muscle) can help reduce many of the harmful effects of type 2 diabetes, even sending it into remission for some people in early stages.

Type 1 is a different story. It’s the lack of natural insulin production (due to a damaged pancreas, autoimmune or other causes), basically the opposite problem to type 2, and no amount of lifestyle changes will replace of need of insulin doses.
by bogwog | 2025-09-11 15:31:14+00:00
Try it before you criticize it.

> It means that users will have to build a custom system image or fiddle with FS overlays just to do system management tasks that are straightforward on all other systems.

What system management tasks? /etc and /var are usually writeable, which is all you need to configure the software on your system. Overlays are for installing new software on the base system, which is only really necessary for something like nvidia drivers because all other software is installable through other means (it's also usually a trivial process). Even if you don't want to use containers, you can use a separate package manager like Homebrew/Nix/Guix/Pacman/etc.

It requires a bit of a mental shift to adapt to if you're only used to traditional systems. It's kind of like the move from init scripts to systemd: it's objectively an improvement in all the ways that matter, but cultural/emotional push back is inevitable :)
by buster | 2025-09-11 15:31:16+00:00
So..

a) I get that a lot of users use KDE. And they love the Desktop Environment. But is there demand for an OS? Would those users switch? I hope so, but for such a big decision to build, support, maintain a whole OS, i'd expect some kind of poll maybe? Some input saying "30% of KDE users would switch to KDE OS"? Is there some kind of proof? I've been using Gnome OS for years but never felt i would want to switch to some Gnome OS. The Desktop Environment is one of many tools in my distro (for me, at least).

b) Supporting lots of hardware (expecially Laptops!) seems to be a huge time sink for people not primarly involved in kernel/driver stuff, or not?

c) ok..

d) Same as a): Will all KDE devs use KDE OS? And is it good to have the KDE Devs use KDE OS, when the majority of users use Arch/Debian/Ubuntu/Fedora? I'd rather have a good chunk of those devs use my distro...
by anarazel | 2025-09-11 15:31:20+00:00
> Nowhere in my comment have I used Linux kernel as an example. It's not a great example neither since it's mostly trivial to compile in comparison to the projects I had experience with.

It's true across a wide range of projects. I build a lot of stuff from source and I routinely look at performance counters and other similar metrics to see what the bottlenecks are (I'm almost clinically impatient).

Building e.g. LLVM, a project with much longer per-translation unit build times, shows that memory bandwidth is even less of a bottleneck. Whereas fetch latency increased as a bottleneck.

> Core can be 100% busy but as I see you're a database kernel developer you must surely know that this can be an artifact of a stall in a memory backend of the CPU. I rest my case.

Hence my reference to doing a topdown analysis with perf. That provides you with a high-level analysis of what the actual bottlenecks are.

Typical compiler work (with typical compiler design) has lots of random memory accesses. Due to access latencies being what they are, that prevents you from actually doing enough memory accesses to reach a particularly high memory bandwidth.
by thiago_fm | 2025-09-11 15:31:26+00:00
I can understand that, but both the elite and the bottom will be pulled down with the policy.

As I've mentioned, industrialization will happen benefiting jobs for blue collars, but you'll have lots of inflation.

And inflation hurts a lot the poor, as they have little to zero discretionary income. It's like a sales tax, it affects the poor more than the rich, proportional to how much they have available to spend.

A country with lots of factories can't afford suburbs, F150s, houses (instead of small condos).

I know this because I live in Germany, and we do have factories, but the life you can afford working in one of them is much more humble than a construction worker in the US can already right now.

Americans will need to learn to live on cramped spaces and stuff that doesn't make sense, as they'll really hate it.

There's no way the US government will have money to maintain all that suburban infrastructure, huge defense expenses without being high in the value chain.
by latexr | 2025-09-11 15:32:44+00:00
> whats the difference between this and bookmarking the actual site and just manually checking?

For one, not having to manually check is huge. If someone posts once every two months, I’m not going to manually open their website every day (times twenty for every one) or even every week and then have to think “have I read this?”. That would be a colossal waste of time, attention, and mental energy.

For another, there’s a plethora of cases where RSS filters. I subscribe to YouTube channels exclusively through RSS, I never open the website (or even have an account) otherwise. Same for Reddit, they have specific feeds for each subreddit, and for each they can even list just the top daily, weekly, monthly, or yearly posts. I can easily get just the most interesting posts of something like r/toolgifs without ever having to deal with the faff or getting distracted by other junk.

Content comes to me, and on my terms.
by swiftcoder | 2025-09-11 15:33:31+00:00
It's not language agnostic in the sense that some languages don't have the necessary functionality to allocate everything up front and avoid allocations in the hot loop.

For example, if you were to benchmark this in Java, the HashMap<> class allocates (twice!) on every insertion. Allocations are a bit cheaper with the GC than they would be via malloc/friends, but still we'd expect to see significant allocator and hence GC overhead in this benchmark
by truelson | 2025-09-11 15:34:12+00:00
Network effects have a certain amount of unpredictability. A highly political group split off from X to BSky, and, imo, gave the network an extra amount of escape velocity. We're seeing a receding, but it did show X's network effect wasn't almighty. BSky will be better prepared for a next wave, wherever that may come from.

I don't think any of us can predict how this will play out, but it certainly is interesting to watch the user growth/receding and watch the waves.
by zlonmask | 2025-09-11 15:34:17+00:00
"AI-Powered Birthday Image Generator: Create stunning, personalized birthday graphics in seconds . Add names, photos & custom details to unique AI-generated designs 1. Simply describe your vision ("e.g., colorful cake with balloons in anime style" 4) and get multiple image options 3. No design skills needed - first card free! Perfect for social posts, e-cards or premium printed cards with worldwide shipping options 1. Try now!
by tedggh | 2025-09-11 15:34:34+00:00
That was fast, which feels oddly positive. My brother died from cancer and spent a full year in agony. I do cancer screenings every 3 years given my brother died relatively young from stomach cancer. Lately I have been thinking on adding whole body MRI scans to my screening process but I keep reading conflicting evidence on their effectiveness. My brother’s cancer was detected late and would have been easy to treat and perhaps even survivable if detected early. I also have two close friends diagnosed with metastatic breast cancer in their 30s, again both with high survival rate on early detection.
by samiv | 2025-09-11 15:34:36+00:00
A week ago, KDE-Plasma 6 whatever is the latest on Arch.

Using NVIDIA proprietary.. glitching like MOFO. Looks slick but just way too buggy to be used.

Some things to try:

* Try turning your display on/off * Try using several virtual displays and spread graphics apps on each one (I use 4 normally) * Try opening 20 firefox windows with ~50 tabs each * Try opening a 8k png in firefox tab (or in some other image viewer)

So yeah... pre-alpha.

P.s. I also tried XFCE and Enlightenment.. and those are not any better (not that claim to be anything but pre-alpha).

Honestly.. on Windows11 the experience is just so damn smooth and slick. Nothing glitches or hangs. The Linux graphics stack just lags behind decade after decade... never catches up...
by dsissitka | 2025-09-11 15:34:59+00:00
I really want to like Bun and Deno. I've tried using both several times and so far I've never made it more than a few thousand lines of code before hitting a deal breaker.

Last big issue I had with Bun was streams closing early:

https://github.com/oven-sh/bun/issues/16037

Last big issue I had with Deno was a memory leak:

https://github.com/denoland/deno/issues/24674

At this point I feel like the Node ecosystem will probably adopt the good parts of Bun/Deno before Bun/Deno really take off.
by rickydroll | 2025-09-11 15:35:41+00:00
I suspect this counts as phone addiction, but I'm reading many books on Kindle with the All You Can Eat Kindle subscription. One thing I've become addicted to is the never-ending series of science fiction stories, for example, such as Backyard Starship, No Stress, Space Express, The Worst Ship in the Fleet, Homeworld Lost, and Frontlines (Martin Kloos, worth paying for). Then there are the numerous series by Alma T. C. Boykin. She's able to spin a story out of everyday life, drawing on history and mythology, while adding enough fantasy to hold my interest.

Some of them are good, while others are cheesy. There are also series I will not admit to having started and given up on when I realized how bad they were. You can thank me later when you realize how much of your life I've just wasted.
by cjbgkagh | 2025-09-11 15:35:51+00:00
It’s a tight feedback loop, use of phones is a symptom caused by problems caused by use of phones. To break it you have to stop using the phone. This is far simpler and more broadly applicable than figuring out who has dopamine disregulation and putting them on meds.

My ADHD is clearly genetic and I’m heavily medicated for it and even still I have difficulty with phone addiction and self control. I would appreciate an environment that aided in this by making tempting things harder to access.

For a long time we were told it was self control causing the issues of weight gain and not changes to the food, diets, and eating patterns. We were told that such problems couldn’t be solved with a magic pill, well for me that magic pill was ozempic and it really did solve 95% of my problems. I had uncontrolled weight gain after taking the Covid vaccine and now 4 years later, two on a rather low dose of ozempic and I’m finally back to normal. I was as disciplined before taking ozempic as I was after so it’s clear that the ozempic had a drastically positive effect.

I think an aversion to empathy leads us to blame people as the cause of their own predicaments, but this blinds us to other causes and fixes. Sometimes it really is the environment.
by rangerelf | 2025-09-11 15:36:06+00:00
You're being willfully dense, I do not believe it's up for debate.

Governments that public force to kidnap, torture, murder, "disappear" their own citizens, are bad. Plenty of examples to go around, both historically and currently: China, Russia, México, North Korea, Belarus, the balcans, plenty of African governments, etc.

It shouldn't matter that "34% of my neighbors" want me sent to a concentration camp, personally I wouldn't want to end up there.

The example you're giving, the whole "it really depends on people's views, ..." is a bad government.

And the truth is that it's easy to be a good government: don't be bad.

Edit: fixed a word.
by elictronic | 2025-09-11 15:36:09+00:00
Bluesky looks to go up then fall back to about half of it's peak. It has the look of a consistently used platform that slowly rolls off users but maintains a solid base. Clubhouse went to the moon then died over a 6 month period. Bluesky would need to drop another 2/3 of it's active users 6 months ago if you wanted your statement to match your data. Bluesky isn't growing exponentially, and is not falling catastrophically. Not a good comparison.
by lpandong | 2025-09-11 15:36:35+00:00
TigerData (FKA Timescale) [ Global | 100% Remote | Multiple Roles: Eng, Product + more ] We began in 2017 as Timescale, focused on making PostgreSQL the best database for time-series data. As our customers evolved, so did we.

TigerData is the fastest PostgreSQL platform for real-time, analytical, and agentic applications. Built for a new era—where data streams nonstop, insights happen instantly, and intelligent agents reason and act—TigerData delivers speed, scale, and simplicity on the foundation developers already trust: PostgreSQL.

Trusted in production with 3 million databases across 2,000+ companies worldwide. We’re the creators of TimescaleDB and pgvectorscale, and backed by $180 million from top-tier investors, TigerData is building the new standard for operational data infrastructure:

TigerData is growing! Check us out: https://www.tigerdata.com/careers

Engineering Manager - Database: https://www.tigerdata.com/careers/08ddff61-b67a-42c5-a799-2e...

Database Support Engineer - US West: https://www.tigerdata.com/careers/0aca17c7-7222-4f75-9c8e-59...

Database Support Engineer - US Central/East: https://www.tigerdata.com/careers/2d64b51e-681b-4197-8290-4e...

Senior/Staff Product Manager, AI: https://www.tigerdata.com/careers/542068ed-6f51-4994-9f58-0c...
by pessimizer | 2025-09-11 15:37:15+00:00
No, it's down posts year over year, which I wouldn't have believed was possible if it hadn't happened. It's got half as much post traffic as its peak(s), which were on exactly the days that Trump was elected and inaugurated.

It's basically what Truth Social would be if it didn't even have Trump.

-----

edit: it's not the fault of the technology, it's the fault of the awful company. They represent people strongly aligned with the Democratic Party, but with no power in the Democratic Party, and their philosophy has been stated over and over again: if you don't agree with whatever we believe today, we don't even want your support, or want you here, or want you to be employed. A lot of conservatives would add "they also don't want us to be alive."

The worst part is that they're all upper-middle class, and when as the Gini coefficient goes up with the right-wingers they're locking into power by being so repulsive, they'll just get wealthier and wealthier and more self-righteous.

I've been suspecting for years that there's a lot of botted support for the dumbest most out of touch liberals that is paid for by conservatives. I don't meet people like this in real life, and I am very left-wing. The liberals I meet are generally humble and thoughtful (if in love with their television sets.)
by HankStallone | 2025-09-11 15:37:22+00:00
When I used Google Reader, I didn't read everything together under one heading for that reason. I clicked on a feed, read the latest, and moved on to the next. So on a slower feed I might always read everything, while on a very busy one I might just occasionally check in or read a little while and then mark the rest read.

Now I use TT-RSS, which looks a lot like Google Reader did, but you run it on a local system. And I still read some feeds completely every day and let others pile up for weeks.
by bediger4000 | 2025-09-11 15:37:24+00:00
I had a very different experience - I saw very few gloating comments, and those were vastly outweighed by comments saying we don't know, nobody deserves this, violence isn't the answer, political violence spirals.

About the worst I saw were, in essence, saying this is bad, but those who live by the sword, die by the sword, which is a biblical thing.

Why is Bluesky an echo chamber but X, Rumble, Gab, Gettr are not? I bet there's huge vile "damn leftists" commentary there, even though we don't know who shot Kirk or why. Why isn't that an echo chamber?
by crtasm | 2025-09-11 15:37:37+00:00
Hundreds sounds overwhelming! Did Google Reader let you look at a single feed at a time? I've never found readers that present a single "inbox" combining all my feeds to be usable.

Some of the feeds I subscribe to are effectively curated news from around the web, and I use filters to hide certain topics - e.g. from a newspaper I might filter the entire sport section.
by softwaredoug | 2025-09-11 15:37:40+00:00
I don’t get this criticism.

The models he - and others like him - make are probabilistic. 70% probability Hillary would win was actually accurate. 30% probability events happen all them time.

The ones with “skin in the game” in 2016 said there was a 99% chance that Hillary would win. And that they’d eat their shoe, etc, if Trump won. They were in so much disbelief that Trump could win they built models just for confirmation bias.
by willahmad | 2025-09-11 15:37:42+00:00
It could have been, but you still need to compress and transform OpenAPI, here are some examples:

* Stripe OpenAPI has ~300 endpoints, imagine adding it to 128k context window, you will run out of context

* Most APIs return unix timestamp, which LLMs don't understand and can't convert to human readable format without additional tools

* With OpenAPI you are trying to be extensive (see Stripe API, contains everything), with LLMs you want to provide basic functionality (only transaction related functionality for humans, even that is limited to last month or week)

With MCP, I think authors envisioned long running processes with duplex communication (e.g. notification about progress).

You can obviously achieve all these with OpenAPI, but it requires some engineering effort to adjust for LLMs. And when people feed OpenAPI as it is, they would start complaining about sub-par quality, because it didn't understand the timestamp
by dahcryn | 2025-09-11 15:38:13+00:00
yeah indeed, regression testing for chatbots that use RAGs would involve making sure the correct response comes from the RAG.

Today we have a extremely hacky workaround by ensuring that at least the desired chunk from the RAG is selected, but it's far from ideal and our code is not well written (a temporary POC written by AI that has been there for quite some months now ...)
by fullstop | 2025-09-11 15:39:18+00:00
I have been using Aurora DX for the last month, and it has been a good experience but has also required a shift in my thinking.

If anything is not included in the base image, you have a few options:

1. use distrobox to install it in a container, and export the app to the desktop. 2. use rpm-ostree to install it as a layer. This is on the slow side, and will slow down weekly updates. 3. Make your own base image with what you want included. This is probably cumbersome and requires some infrastructure.

I have a few things in distrobox containers, things which aren't available as flatpaks. The biggest hurdle, for me, was getting wireshark running since the flatpak version can't capture traffic. I had to make a root distrobox container and export the app to my desktop. It works, but there were definitely some hoops to jump through.

I like that updates come through once a week and they aren't applied until I reboot. If I have problems, it is easy to roll back to what I was running before.

I would be comfortable giving my parents an Aurora setup, knowing that they can't easily break it.