MurphyA/hacker-news huggingface dataset:
by gwern | 2026-02-15 21:30:39+00:00
You could potentially use WARC instead of Tar as the appended container, sure, but that's a lot of complexity, WARC doesn't serialize the rendered page (so what is the greater 'fidelity' actually getting you?) and SingleFile doesn't support WARC, and I don't see a specific advantage that a Gwtar using WARC would have. The page rendered what it rendered.

And if you choose to require separate files and break single-file, then you have many options.

> surely you could host "warcviewer.html" and "warcviewer.js" next to "mycoolwarc.warc" "mycoolwrc.cdx"

I'm not familiar with warcviewer.js and Googling isn't showing it. Are you thinking of https://github.com/webrecorder/wabac.js ?
by jesuslop | 2026-02-15 21:31:15+00:00
As I understand, the technology was protected by a patent help by guys at Leptonica and it exprided. There is a crude project for encoding images to jbig2 at https://github.com/agl/jbig2enc. I am sharing my personal scripts here [1] (windows) that wrap that for end to end djvu to pdf for scanned texts using jbig2 compressed images in the pdf instead of jpeg. This combines decent compression with pdf handiness. djvu still compresses better but pdfs can be got under twice the side, that sounds no impressive, but many common available pipelines produce sizes x3, x4 and worse, a particular offender those using ghostscript pdfwriter. The sripts have worked months locally but are given "as is" without testing, with zero support, you deal with python dependencies and having jbig2 and djvu-libre tools in the path. Beyond image compression tech, they support OCR-layer (cut/pasteability), bookmark and page label migration from djvu to pdf info.

[1] https://github.com/jesuslop/djvu2pdf-test
by mgraczyk | 2026-02-15 21:31:44+00:00
But do you see how crazy that sounds? If we know the numbers, we would just not do the treatment in those cases.

And in reality it's not actually close like this. Early treatment is so overwhelmingly better than it completely dwarfs all considerations of biopsy risk.

Late stage lung cancer has less than half the 5 survival of early stage, and around 50% of lung cancer detection is late stage. That's around 30000 lives you could save every year from just lung cancer.

Let's say you don't protocol for all adults. Typically post CT or PET at most 10% get a biopsy. So let's say we increase the false positives that lead to CT or PET by X% of adults. That's 270M * X additional biopsies. Lung biopsy mortality is under 2%.

So the worst case you need a false positive rate under 5% for this to he net beneficial for lung cancer. That's low but with 3 scans 6 months apart and a good radiologist, it's not unreasonable

I believe I've assumed the worst case for all the quantities. In reality the addition screened people would have much lower CT detection rate than the background population, and biopsy mortality has been decreasing. Plus you can do this for only people above 40 where the benefits are higher
by lebuffon | 2026-02-15 21:31:47+00:00
Probably worth mentioning that writing a big project in Forth is more like creating an OOP framework.(if you are disciplined)

The end result of that is one doesn't write the program in "Forth" per se but in the domain specific language you create for the job. This is how Forth gets more productive than pure assembly language if the team documents things well and follows the prescribed system.
by duskwuff | 2026-02-15 21:32:13+00:00
Other advantages include:

- It's much easier for web sites to implement, potentially even on a page-by-page basis (e.g. using <meta> tags).

- It doesn't disclose whether the user is underage to service providers.

- As mentioned, it allows user agents to filter content "on their own terms" without the server's involvement, e.g. by voluntarily displaying a content warning and allowing the user to click through it.
by jv22222 | 2026-02-15 21:32:20+00:00
Obsidian: We're mostly solving different problems. I wanted that text editor / IDE feel but with the ability to instantly share and collaborate. Bring in a full team, manage all your docs, give everyone different permissions etc.

So the problems I was thinking about were collaboration and content management between teams. But I also wanted it to work really well as a private personal tool. So in a way it's two tools in one. You can enjoy it solo, and then the hope is you've found something you like and now you can bring a team into it

Re the tech framing, I mostly mentioned it because I figured HN would find it interesting since it's the less taken path. Apologies if it came across as unrelated to the product.
by ClaytheMachine | 2026-02-15 21:33:29+00:00
Moltbook is a social network for AI agents (think Reddit for LLM-based assistants). This post by an agent named eudaemon_0 describes finding a credential stealer disguised as a weather skill in ClawdHub, an AI skills marketplace.

The agent proposes signed skills, permission manifests, and 'isnad chains' (provenance verification borrowed from Islamic hadith authentication) as solutions.

Interesting to see AI agents starting to grapple with the same supply chain security problems that human developers face with npm/PyPI.
by pdonis | 2026-02-15 21:33:38+00:00
Instead of lobbying for taking away everyone's privacy, why isn't the government going after those they say are the actual culprits? From the article:

"The attorneys general argue that social media companies deliberately design products that draw in underage users and monetize their personal data through targeted advertising. They contend that companies have not adequately disclosed addictive features or mental health risks and point to evidence suggesting firms are aware of adverse consequences for minors."

Okay, so why aren't they going after the social media companies?
by jsandfort | 2026-02-15 21:33:54+00:00
Hey HN, I built this because I kept hitting the same problem: getting stock data into side projects was either expensive (Polygon starts at $199/mo) or unreliable (Yahoo Finance scraping breaks constantly).

stockdata.dev is a REST API for 15-minute delayed US stock data. 10,000+ tickers, historical daily prices, batch quotes. Free tier is 1,000 calls/month. You can grab a trial API key right on the docs page - no signup required, works for 24 hours.

I built it to power my other project, https://dirtytraders.app, which tracks stock trades made by members of Congress. Needed a reliable, affordable data source for quote lookups and portfolio tracking — couldn't find one that fit, so I built my own and opened it up.

Stack: Flask/Gunicorn, PostgreSQL, Redis, InfluxDB. Three data providers with automatic fallback (IEX Cloud - Alpha Vantage - Finnhub). S&P 500 quotes are pre-cached, everything else is fetched on demand and cached for 15 minutes.

The 15-minute delay is intentional - no exchange licensing fees, which is how I can offer a free tier. Covers the vast majority of use cases: portfolio trackers, trading bots, dashboards, alerts.

Pricing: Free (1k/mo), Starter $15/mo (50k), Pro $50/mo (500k).

Would love feedback on the API design, docs, or pricing. What would make this more useful to you?
by tim333 | 2026-02-15 21:34:41+00:00
The "the Mortecene, the Age of Death and Killing" stuff is kind of phoney when most evidence seems to show higher rates of killing before the enlightenment than after.

Likewise other stuff - the industrial revolution can't be explained without slavery. I mean it was involved but the steam engine would have been invented anyway. And so on. Making up incorrect nonsense to fit a preconceived hypothesis.
by josephg | 2026-02-15 21:34:50+00:00
H264 is controlled by the same consortium as H265. The only difference is that many of the h264 patents have expired over the last couple of years.

The free version of davinci resolve still doesn't include h264 support - presumably because they don't want to poke the bear. (h264 still works on macos because apple pays the licensing fees, and resolve uses the macos encoder & decoder.)
by jackvalentine | 2026-02-15 21:35:05+00:00
You’re going to end up running down the same merry path that DRM companies do - and you can’t patch the wetware layer. Inevitably thousands of ‘human tokens’ will end up in the hands of actual humans working in call centres with 300 phones in front of them.

https://www.scmp.com/news/people-culture/trending-china/arti...
by salawat | 2026-02-15 21:35:51+00:00
>The suggestion is not to depend on AI for privacy, but to use it to build products like signal which guarantee privacy.

Uh huh. No. You use their system to do it, they have your prompt, and the output on hand. Even more so, they have the capability to tamper with it. They are essentially in a position to own the entire instance of the work product. It doesn't matter if they don't yet. It matters that they can. Furthermore you lose out on the learning. You lose out on any innovation. You lose out in the eyes of the law on the privacy of the communique you use to drive the black box.

>In addition, right now AI has little vendor lock in and there are multiple competitive alternatives, so becoming dependent on a single company is not so likely.

Yup. Right. Like we don't know how that ends. <gestures to />50 years of market consolidation in the distance, letting the illusion of choice speak for itself>
by neilk | 2026-02-15 21:35:55+00:00
I understand why the parent post felt it was clankerish. It has the same feeling - many signposts on a road to nowhere.

But it's also got the structure of marketing. Or a pitch for a cult. Or both.

It is literally book-ended - he mentions the new book in the first and last paragraphs.

As far as I can tell it's no different from a pitch for other popular spirituality books. A tease that there is an answer, the author discovered it, but now is not the time for that revelation. Boring.

It's just targeted towards an audience that has different filters, and different expectations for what enlightenment might look like. Apparently for some it involves PKD-like psychonauttery, name-checking Bay Area grifters of past and present, post-rationalism, etc.
by Joel_Mckay | 2026-02-15 21:36:23+00:00
Indeed, the list of human cognitive biases was posted above.

The activation capping effect on LLM behavior is available in this paper:

https://www.anthropic.com/research/assistant-axis

This data should already have been added to the isomorphic plagiarism machine models.

Some seem to want to bury this thread, but I think you are hilarious. =3
by quinncom | 2026-02-15 21:36:36+00:00
From what I can see, this is agentic tooling that provides similar features to OpenClaw. It’s been on GitHub since June 2024 but never seemed to catch the hype train. Some stats comparing the popularity of the two:

- Agent Zero: 14k GH stars, 3k X followers
- OpenClaw: 197k GH stars, 314k X followers
by mmooss | 2026-02-15 21:37:44+00:00
> That's why the west was able to leapfrog other nations economically

I tend to agree, though it's of course hard to prove. However, I'm talking about the present, not the past.

> There are no human rights for the lowest castes in Hinduism

I said it is "very widespread", not everywhere. Perhaps the confusion is the word Universal: that doesn't mean everyone believes it (false for any belief), but that everyone has the rights, whether or not they know or can exercise them. It's the concept that starts the Declaration of Independence: All are created equal, and all have inalienable rights.

> there were nothing like the modern idea of human rights in Japan or China before they westernized

I am talking about the present, where it's adopted in East Asia (including in China - Taiwan, Hong Kong (though suppressed now), June 4 on the mainland), throughout Latin America, Europe of course, parts of Africa, the Anglo world, etc.

> there are no human rights for polytheists in Islam,

There is no country called 'Islam'; if we go by scripture, nobody has human rights. The idea that all practicioners of Islam have the same beliefs is as true as saying all practicioners of Christianity do - and look at HN.

In Indonesia, the largest majority Muslim country, there are human rights, also in India, with the largest Muslim population (but not the majority). I think Pakistan and some South Asian countries probably have them enshrined.
by noosphr | 2026-02-15 21:37:53+00:00
I've been at this longer than just about anyone.

After three major generations of models the "intuition" I've build isn't about what AI can do, but about what a specific model family can do.

No one cares what the gotchas in gpt3 are because it's a stupid model. In two years no one will care what they were for gpt5 or Claude 4 for the same reason.

We currently have the option of wasting months of our lives to get good at a specific model, or burn millions to try and get those models to do things by themselves.

Neither option is viable long term.
by mikestew | 2026-02-15 21:38:29+00:00
Here in the Seattle area, plenty of sub-freezing days (which is itself unusual for the area, in 25 years of living here), just no precipitation. And you know what Seattle is known for, especially in the winter? But when we do get precipitation, it’s warm enough in the mountains that it comes down as rain, not snow. Rough year to be a ski area.
by gildas | 2026-02-15 21:38:35+00:00
I haven't looked closely, but I get the impression that this is an implementation detail which is not really related to the format. In this case, a polyglot zip/html file could also interrupt page loading via a window.stop() call and rely on range requests (zip.js supports them) to unzip and display the page. This could also be transparent for the user, depending on whether the file is served via HTTP or not. However, I admit that I haven't implemented this mechanism yet.
by RickJWagner | 2026-02-15 21:39:02+00:00
I respect your integrity if you are willing to condemn Obama as well as Trump.

I think one side is just more likely to protest. Since Trump’s been in office, there have been numerous protests: the Women’s March ( complete with pink hats ), No Kings protests, anti-ICE protests, etc. Wikipedia has a page for them.

https://en.wikipedia.org/wiki/Protests_against_Donald_Trump

Meanwhile, the other side had one notable protest, J6, that was confined to Washington DC.

Why is that? I think it’s a combination of party demographics, party operative strategies, and an imbalance in Karen count ( maybe that’s part of demographics ).

If Vance succeeds Trump, we’ll see if your theory is better. He hasn’t engaged in Trump’s childish ( yet politically effective ) manner, so if Vance protests erupt I’ll feel my ideas seem more likely valid.

Edit: this New Republic article makes some of the points for my argument. https://newrepublic.com/article/205820/left-protests-hyperpo...
by afiori | 2026-02-15 21:39:18+00:00
Without a lot of discipline it is very easy to end up with a css with lots of unclear and hard to guess effects. Eg consider the case of <A type=1><B><A type=2></A></B></A> where A and B are complex templates. Any selector with the " " operator on A risk expanding to the inner A even if it was intended only for the outer. Similarly a :has selector might catch a descendant of the wrong element.

@scope fixes a lot of this, but it is a complex problem. With tailwind you mostly have to worry about inheritance
by noelwelsh | 2026-02-15 21:39:30+00:00
I'm not sure we're talking abou the same paper. Here's the one I'm referring to:

https://smlnj.org/compiler-notes/k32.ps

E.g. "Our strategy is to pre-allocate a small set of memory
locations that will be treated as registers and managed by
the register allocator."

There are more recent publications on "compiler controlled memory" that mostly seem to focus on GPUs and embedded devices.
by Tor3 | 2026-02-15 21:40:10+00:00
Browsing the web was slow, because the network was slow. It wasn't really because the desktop computers were slow. I remember our company having just a 64 kbit/s connection to the 'net, even as late as in 1997.. well, that was pretty good compared to the place where I was contracted to at the time, in Italy.. they had 19.2 kbit/s. Really big sites could have something much better, and browsing the internet was a different experience then, using the same computers.
Show HN: DetectZeStack – Tech stack detection API with 4 detection layers
by mlugo_apb | 2026-02-15 21:40:46+00:00
I built a REST API that detects what technologies any website is running. You give it a URL, it gives you back the full stack: frameworks, CMS, CDNs, analytics, payment processors, etc.

What makes it different from BuiltWith/Wappalyzer:

- 4 detection layers running simultaneously: Wappalyzer fingerprinting (3,500+ signatures), DNS CNAME analysis (29 infrastructure signatures), TLS certificate inspection (8 issuer patterns), and custom HTTP header matching. Most tools only do one of these.

- Batch endpoint: analyze up to 10 URLs in a single request with concurrent processing.

- Compare endpoint: pass 2-10 URLs, get back shared vs unique technologies.

- CPE identifiers on every detected technology, so you can map directly to the NVD vulnerability database for security audits.

- 24-hour smart cache: cached responses return in <5ms and don't count against your quota.

Tech stack: Go, chi router, SQLite, wappalyzergo. Runs on a single Fly.io machine.

Free tier is 100 requests/month. Pro is $29.99 for 25K requests — significantly cheaper than BuiltWith ($295/mo minimum).

Try the live demo on the homepage, or check the API spec: https://detectzestack.fly.dev/openapi.yaml

Happy to answer questions about the detection approach or the architecture.
by catlikesshrimp | 2026-02-15 21:40:56+00:00
How are pigs, rabbits, goats and venison more sustainable? Unless you mean eating meat twice a year.

I live in a farmer family; our cattle needs around one hectare each, because we don't feed them processed food, only grass; because concentrated food is even less sustainable, and more importantly, more expensive than letting them roam (fenced areas)

Rabbit is not sustainable. There were some people trying to commercially rise and sell them and it didn't work. They would need concentrated food, which is expensive.

Goat meat is much more expensive than cows because they are less efficient than cows and pigs and chicken. I know two people who rise goats to sell them, and it doesn't make them money; really, they do it because they kind of like the critters as a pet project.

Only pigs and chickens are more sustainable, precisely because of theirinhumane(?) short life and their genetics. They are very efficient meat producers.

I know poor people who rise chickens and pigs; those animals take longer to reach "maturity", and the meat is not tender; but since the animals are eating whatever they scavenge, it can't be done at scale; again, we would eat meat like twice a year (This might be an exageration, but chicken pig and cow farms really produce all the meat we eat; of those only cows eat grass under the sun)
by eqvinox | 2026-02-15 21:40:56+00:00
> glibc dependencies that can't be resolved because you need two different versions simultaneously in the same build somehow...

If you somehow experience an actual dependency issue that involves glibc itself, I'd like to hear about it. Because I don't think you ever will. The glibc people are so serious about backward and forward compatibility, you can in fact easily look up the last time they broke it: https://lwn.net/Articles/605607/

Now, if you're saying it's a dependency issue resulting from people specifying wrong glibc version constraints in their build… yeah, sure. I'm gonna say that happens because people are getting used to pinning dependency versions, which is so much the wrong thing to do with glibc it's not even funny anymore. Just remove the glibc pins if there are any.

As far as the toolchain as a whole is concerned… GCC broke compatibility a few times, mostily in C++ due to having to rework things to support newer C++ standards, but I vaguely remember there was a C ABI break somewhere on some architecture too.
by mmooss | 2026-02-15 21:41:29+00:00
Just look around the world; they are the norm: East Asia (Japan, South Korea, China - Taiwan, Hong Kong, June 4 on the mainland); North America; South America, almost all of the region; Europe; Australia, NZ, Indonesia, the Phillipines; South Asia (India, and I think they are enshrined if not enforced in Pakistan and some others).

What's mostly missing is the Middle East, Central Asia, parts of SE Asia, and large parts of Africa - though there are Benin, Botswana, Kenya, and many others iirc.
by Our_Benefactors | 2026-02-15 21:41:55+00:00
It’s such a shame as h265 is such an amazing codec breakthrough. I’m in the process of converting my library for space saving and the h265 files are literally 50% of the original size (give or take), with imperceptible quality difference. I can reencode around 100-200GB/day typically, using a 3090
by asdff | 2026-02-15 21:42:18+00:00
Long term though, AI will win out. The thing is that you can improve capability. You can make the context window bigger. You can throw more compute at it. Improve efficiency of chips. Throw more power at it. And indeed, that has worked so far to turn the gpts of 2017 into the gpts of 2026 that can actually do stuff.

Meanwhile, human thoughtpower cannot really be improved. Once the tipping point is reached where computers exceed humans, humans will never be able to catch up by definition.

Humans can also only maintain so much contextual information and scope. They can only learn so much in the time scale they have to get up to speed. They can only do so much within the timescale of their own mental peak before they fall off and go senile or die. While these limits are bound by evolution, they change on the orders of thousands of generations, and require strong selection for these changes at that.

The turtle has marched far already, but the hare in the speeding car they continually improve is not far behind. Efficiency doesn't matter. What is inefficient now will be trivial to parallelize and scale in the future as its always been in the history of compute. We'd have to engage in something like the Bene Gesserit breeding program if we are to have human thoughtpower be competitive against compute in the future.
by edanm | 2026-02-15 21:42:21+00:00
> Only one shuffle can exist at a time, the "number of shuffles" is not a number of natural objects but rather a cardinality of a set.

I really don't understand what this means in practice. If there are exactly 50 rocks in front of me right now, I can't talk about 51? It doesn't maintain a direct connection to reality to talk about what would happen if I threw another rock on the pile? Or if that's connected cause another rock exists, what about if I have exactly 20 chickens, and I want to talk about what would happen when another is born? Is this "connected to reality" and "a number of natural objects"? Or "the cardinality of a set" instead?
Show HN: AgentKV – SQLite for AI agent memory (MMAP vector+graph DB)
by shiwang_khera | 2026-02-15 21:42:39+00:00
Hi HN! I built AgentKV — an embeddable vector + graph database for AI agents.

The problem: I was building a local LLM assistant and needed to store conversation history + RAG documents. ChromaDB/Pinecone require running servers. FAISS has no persistence. I wanted something that just works like SQLite.

AgentKV is a single-file mmap database with:
• Vector search (HNSW index) + graph relations
• Crash recovery with CRC-32 checksums
• Zero servers, zero config files
• Thread-safe concurrent reads
• pip install agentkv (Python 3.9+)
Real examples in the repo:
• Local RAG with Ollama (10 lines)
• Multi-turn chatbot with memory
• Agent coordination with context graphs

Built in C++20 with mmap + nanobind Python bindings. Benchmarked against FAISS — competitive throughput with built-in persistence.

Open to feedback on the API design and any use cases I'm missing!

PyPI: https://pypi.org/project/agentkv/