MurphyA/hacker-news huggingface dataset:
Show HN: Ran an AI agent 100x – pass rate 70%, not 100%
by alepot55 | 2026-02-09 09:35:43+00:00
I tested Claude 3 Haiku on "What is 247 * 18?" across 100 trials.
Pass rate: 70%. 95% CI: 48%-85%. A task any calculator solves 100%
of the time.

This is the core problem with agent evals today: one run tells you
nothing. The same prompt, same model, same tools — different result
every time.

I built agentrial to fix this. It's a pytest-style CLI that runs
your agent N times and gives you:

- Wilson confidence intervals on pass rate
- Step-level failure attribution (Fisher exact test pinpoints which
tool call or reasoning step diverges between pass/fail runs)
- Real API cost from response metadata
- A GitHub Action that blocks PRs when reliability drops

Usage is minimal — write a YAML config, run "agentrial run":

pip install agentrial

Tested extensively with LangGraph agents. 100 trials cost $0.06.
MIT licensed, no telemetry, runs locally.

Looking for feedback on what metrics matter most when you're
shipping agents to production.
by hilti | 2026-02-09 09:35:50+00:00
I always wanted a minimalistic CSS framework for my projects, so I started to create my own: THINK

THINK is a modern CSS-first UI framework built on semantic HTML, custom elements, and data attributes. Uses :has(), container queries, and density scaling. No classes, no build step.

It‘s work in progress but I‘m pretty happy with the outcome so far, especially the data table component and automated Insights. I know it‘s not AI driven - but it works pretty okay for quick insights on the loaded data.

https://think.iotdata.systems/
by Aachen | 2026-02-09 09:36:08+00:00
I've thought about making such a system before, but never considered making it a single flat file¹. How are you going to identify who keeps inviting these bad actors?

Assuming the list is under source control, the commit history can answer this question but it's manual work whereas a tree/graph system shows you directly who is making the bad judgement calls (may be intentional or not, so this person can keep contributing so long as those contribs are good, but not invite further people). I don't understand the added value of a bunch of software around what is essentially an allowlist where the commit history already shows why someone was added or removed

¹ https://github.com/mitchellh/vouch?tab=readme-ov-file#vouche...
by cyrus_kelly | 2026-02-09 09:37:38+00:00
Thank you for the feedback, really appreciate this and we've shipped some of your recommendations:

- We've just released carousel / grid posts. Multi-page reports aren't directly supported as PDFs yet, but vertical carousels get you most of the way there. All panels sit on the same canvas in this mode, which allows continuity between different elements (for example: https://www.instagram.com/p/DUhHDZQE86J/?img_index=1 )

- We've made some improvements to the design editor since, but there are still some significant issues. We're currently working on further fixes to the design editor to make it more reliable and hope to have a much more polished experience within the next week.

- The agency angle is really interesting and not something we'd deeply explored yet. We're looking at launching a referral program soon so let me know if you'd like to be looped into that given your connections to agencies.

Thanks again for giving the product a shot!
by boltzmann-brain | 2026-02-09 09:39:16+00:00
pay-to-commit has been discussed in the article linked here

https://news.ycombinator.com/item?id=46938811

escrow is a more complex system, and there are multiple possible implementations, but the nice thing is you can skip it and get the same results.

let's assume for a second that the repo owner spends time on PR review, and that time needs to be reimbursed. let's also assume that the person pushing a PR expects some sort of bounty. then as long as the review price is less than bounty price, there's no need for escrow. the pushing party goes out on a limb paying the reviewer to merge their PR, but also expects (rightly or not) to be remunerated for solving the bounty. whether they really did solve it is in the remit of the bounty originator, who might or might not be part of the group controlling the repository. if there's escrow, then the bounty giver probably has to be part of that group. not having escrow allows for crowd funding by interests outside of the repo controlling party.

escrow is only usefully different in a situation when there is no bounty, you want to push code, and then you want to say "ok, here's some money, and here's a PR, either accept the PR and give me money or don't accept it and take my money" as a means of skipping the line or getting a shot at pushing in the first place. however, at that point two things are apparent: 1. you expect the reviewer to do work required to implement your desired changes for free and 2. this might start getting abused, with PRs getting rejected (to gain money) but then modified / refactored versions of this code being pushed via commits or from another user who is the repo owner's puppet (refactoring code is becoming super cheap due to AI). so that degenerates escrow-to-push into a scam.

there are more considerations like that in the article I linked to. I agree that an economy around FOSS pushing would be desirable. it also doesn't preclude free-as-in-money contributions - there are at least two mechanisms that would allow it: 1. you get sponsored by someone who sees your talent (either gives you money to push, or they have push access to that repo and can hand it out free) 2. you create a fork that becomes so good and valuable that upstream pulls from you for free

ultimately becoming a respected developer with free push access to contended repositories should be something that you can monetize to some extent that's purely within your remit, and it would greatly reduce unserious bullshit coming from third parties (especially all those weird hardware developers) and make it easier to be a FOSS dev.
by armchairhacker | 2026-02-09 09:39:20+00:00
> and couldn't upgrade because Apple didn't support my hardware anymore.

If you really want, you may be able to upgrade the OS anyways with https://github.com/dortania/OpenCore-Legacy-Patcher.

Or use Homebrew on the old OS with TigerBrew (https://github.com/mistydemeo/tigerbrew), but people online suggest MacPorts, not only because it has first-party support but also because it’s apparently better designed.
by cess11 | 2026-02-09 09:40:34+00:00
Yeah, it's not a problem that a particular person does not know it all, but if no one knows any of it except as a black box kind of thing, that is a rather large risk unless the system is a toy.

Edit: In a sense "AI" software development is postmodern, it is a move away from reasoned software development in which known axioms and rules are applied, to software being arbitrary and 'given'.

The future 'code ninja' might be a deconstructionist, a spectre of Derrida.
by Kon5ole | 2026-02-09 09:41:14+00:00
>I'm feeling people are using AI in the wrong way.

I think people struggle to comprehend the mechanisms that lets them talk to computers as if they were human. So far in computing, we have always been able to trace the red string back to the origin, deterministically.

LLM's break that, and we, especially us programmers, have a hard time with it. We want to say "it's just statistics", but there is no intuitive way to jump from "it's statistics" to what we are doing with LLM's in coding now.

>That's why "the easy part" is easy because the easy problem you try to solve is likely already been solved by someone else on GitHub, so the template is already there.

I think the idea that LLM's "just copy" is a misunderstanding. The training data is atomized, and the combination of the atoms can be as unique from a LLM as from a human.

In 2026 there is no doubt LLM's can generate new unique code by any definition that matters. Saying LLM's "just copy" is as true as saying any human writer just copies words already written by others. Strictly speaking true, but also irrelevant.
by iLoveOncall | 2026-02-09 09:41:43+00:00
Funnily enough we have had those comments with every single model release saying "Oh yeah I agree Claude 3 was not good but now with Claude 3.5 I can vibe-code anything".

Rinse and repeat with every model since.

There also ARE intrinsic limits to LLMs, I'm not sure why you deny them?
by hdjrudni | 2026-02-09 09:42:05+00:00
The problem is that whole thing should have been its own script, which it coulda been with npm-run-all:

https://github.com/mysticatea/npm-run-all/blob/HEAD/docs/npm...

"scripts":{
"buldrun":"run-p check test docs && run-p zip zip:firefox"
}

I guess you can put `bun run --parallel` into the script too but it's a bit more verbose.
Show HN: MaiMap – Generate live architecture docs and diagrams for microservices
by ev_dev3 | 2026-02-09 09:42:26+00:00
I am working on a tool called MaiMap which combines static code parsing with LLM-based analyses to generate architectural docs and diagrams for microservices.

The link I have posted here is to a demo of what the generated docs look like. My plan is to provide this knowledge to user's LLMs so you can ask it questions about your distributed system. I would also like to have this read documentation and incorporate that into its knowledge base. Any feedback here would be much appreciated - would you like to use a tool like this at your workplace or on a project? What use-cases do have for a tool like this?

This will be a paid-product with a self-hosted option. I have provided more details and some docs at https://maimap.dev/. The code is not currently open-source. I am aiming to release a beta version around April 2026, however it will only support Go-based microservices. Java, Typescript and C# support will be added next.

Thank you!
by enum | 2026-02-09 09:42:44+00:00
Nice article. I believe the Claude C Compiler is an extraordinary research result.

The article is clear about its limitations. The code README opens by saying “don’t use this” which no research paper I know is honest enough to say.

As for hype, it’s less hyped than most university press releases. Of course since it’s Anthropic, it gets more attention than university press.

I think the people most excited are getting ahead of themselves. People who aren’t impressed should remember that there is no C compiler written in Rust for it to have memorized. But, this is going to open up a bunch of new and weird research directions like this blog post is beginning to do.
by Semaphor | 2026-02-09 09:42:54+00:00
> the series only really began

It ended with the new GTA, with the new one obviously being III ;)
I spent so much time playing I and especially II, then was very disappointed with III.

GTA II also had one of my favorite bugs (that kinda required cheating): As long as you were throwing grenades, you wouldn’t move on the z-axis. So if you had unlimited grenades (or I guess just a lot of them without cheating), you could fly from one skyscraper to where ever else as long as you kept throwing those grenades. Just needed to remember to be on something as high when you stop throwing, or you’d still splat :D
by ProllyInfamous | 2026-02-09 09:43:10+00:00
=D

Django has low re-watchability (unlike most of Tarantino's work) but incredible acting/twists/cinematography.

Once Upon a Time is too much for me (bottom-tier Tarantino IMHO), but it does have many great actors/scenes (the overall storyline/premise is what I didn't care for).

Haven't seen Deathproof, but Basterds is wonderful storytelling.
by hobofan | 2026-02-09 09:43:28+00:00
The corner radius discussed in the article is utopian and also only works in street layouts that prioritize car driveability over everything else.

A "properly rounded corner" at an intersection as alluded in the article is one that's great for cars (as they don't have to slow down much or have a tight turn. However, they are awful as pedestrian crossings if you want to do a double crossing, as you have to traverse additional distance. They are also much more space-inefficient, which isn't viable in a densely built urban environment.
by troupo | 2026-02-09 09:44:02+00:00
"It's just another Markdown file, bro".

LLMs do not learn. So every new session for them will be rebuilding the world from scratch. Bloated Markdown files quickly exhaust context windows, and agents routinely ignore large parts of them.

And then you unlessh them on one codebase that's more than a couple of days old, and they happily duplicate code, ignore existing code paths, ignore exhisting conventions etc.
by vladvasiliu | 2026-02-09 09:44:37+00:00
It's a question of priorities, I guess.

I admit I love the mbp hardware, but I can't stand macos anymore. So when my work computer was up for replacement, I didn't think twice and went with a PC, the latest thinkpad p14s. Everything works out of the box on Linux.

Is it as nice as a mac? No, especially the plastic case doesn't feel as nice under the hands as a mac's aluminum, the touchpad is quite good but worse than a mac's, and there are some gaps around the display hinge. But the display itself is quite nice (similar resolution, oled, although not as bright as a mac's), it's silent and it's plenty fast for what I do. I didn't pay for it, so I don't directly care about this point in this situation, but it also cost around half of what an equivalent mbp would have cost.

I also haven't tried the battery life yet, but it should hold at least as well as my 5-yo hp elitebook, which still held for around 5 hours last year. I basically never use it for more than an hour unplugged, so battery life is low on my priorities.
by rrobukef | 2026-02-09 09:46:01+00:00
The fact that the height of adult males with ADHD is greater than that of males without ADHD (first figure/graphs, G & H) should indicate that there is no causative link that the medication has an impact. For females, the average does not show this, but if you look carefully, the variation is bigger. The same selection bias is possible in this case.

Instead, to me it looks like there is selection bias going on. Shorter males are more likely to get MPH prescribed is an equally valid explanation.

Luckily, the conclusion makes no assertions about height either, nor does the paper make conclusions about any causative relation.
by Jn2G3Np8 | 2026-02-09 09:47:10+00:00
Built a bit of software[1] with Claude this weekend to stop Apple Wireless Direct Link from interrupting GeForce NOW on macOS. Bit of an upgrade from a shell script[2].

Started with Rust, then swapped to Swift to cut the dependencies right down.

[1]: https://github.com/sjparkinson/geforcenow-awdl0

[2]: https://uncomplicated.systems/2026/02/08/geforcenow-macos
by spacedcowboy | 2026-02-09 09:48:05+00:00
I think American society is very much road-focussed, having lived there for a couple of decades. I think UK (and European in general) society is very much street-focussed.

A lot of that comes down to geography - the UK is a high-density population compared to the USA but the impact on our lives is significant. In the US, I would drive everywhere. Literally everywhere - to the shops, to the library, to the beach, everywhere. Yesterday I took my son to his archery practice, we walked along the coast road for about 20 minutes, and picked up a "Mr Whippy" 99er ice-cream (yes, even in the cold weather) along the walk back. It was pleasant, and healthier.
by renegat0x0 | 2026-02-09 09:48:15+00:00
This might sound stupid, but I have my own index, of trusted domains:

https://github.com/rumca-js/Internet-Places-Database

I start with it, to find stuff I know. If there is stuff I don't know and is important to me, I add it to my database.

Also it enforces me to verify each link I visit. So links I visit are mostly ok.

Though I sometimes use chatgpt for instructions, and if someone poinsed the well "well enough" it might spread malware.
by usrusr | 2026-02-09 09:48:42+00:00
Not sure what exactly you're referring to, but legal is a very interesting field to observe, right? I've been wondering about that since quite early in my LLM awareness:

A slightly sarcastic (or perhaps not so slightly..) mental model of legal conflict resolution is that much of it boils down to throwing lots of content at the opposing side, claiming that it shows that the represented side is right and creating a task for the opposite side to find a flaw in that material. I believe that this game of quantity fits through the whole range from "I'll have my lawyer repeat my argument in a letter featuring their letter head" all the way to paper-tsunamis like the Google-Oracle trial.

Now give both sides access to LLM... I wonder if the legal profession will eventually settle on some format of in-person offline resolution with strict limits to recess and/or limits to word count for both documents and notes, because otherwise conflicts fail to get settled in anyone's lifetime (or won by whoever does not run out of tokens first - come thinking of it, the technogarchs would love this, so I guess this is exactly what will happen barring a revolution)
by joe_mamba | 2026-02-09 09:48:49+00:00
>48-days old account

Since when is the understanding of economics and politics tied to the age of your account? Is this how arguments are won here?

>posting doubious takes like this

Universally recognized and factually proven facts = dubious to you?

What (counter-)arguments do you bring to the discussion, other than throwing baseless accusations at people as a strategy to discredit their opinions you dislike?

That's exactly the opposite of the HN rules.
by MrBuddyCasino | 2026-02-09 09:48:57+00:00
Very interesting!

> As such, during my investigation I found a bit of code mapped to an unknown memory area 0x0081_0000. It contained what seemed to be a vector table, code for bootstrapping the system, and then some USB code with a small protocol to do some debug operations. The code was written as it was supposed to live at address 0x00 (which is where the Flash is mapped to in normal conditions).

Is this something you can pattern-match directly from the assembly, or was this dis-assembled to C or so?
by bob1029 | 2026-02-09 09:49:41+00:00
> For functions that can be fast or slow, the decision as to whether to inline or not depends on the input.

This is one area where modern JIT runtimes can dominate static compilations. Dynamic profile guided optimization adjusts things like inlining based upon how the live workload actually performs. You don't need to have any data ahead of time.

There are very few cases where I would trade this sort of long tail optimization capability for something like faster startup. Cold start happens once. If the machine rarely stops, it's not very relevant to the typical operational concerns.
by noduerme | 2026-02-09 09:49:55+00:00
Regarding modern roadways and viaducts that pass around medieval city cores, it's fascinating how they can be seen and still not seen. I tried to do this in Cities Skylines II for awhile, building very narrow streets in "organic" city centers with low-to-mid density, and then transitioning to a new city grid around the old center. Finally, superimposing highways and tunnels without destroying the original historical core. There were some beautiful results, but the traffic jams were astounding, even with limits on the city center roads and highway exits. This is the way cities actually grew, so to me it's much more interesting to simulate than a "perfect" city built all at once. One thing that was fun in SimCity 3000 was that certain technologies didn't become available until you had a certain population. I would almost wish that a city-builder now would allow you to take that to the extreme: Spend 200 years with horse-drawn carts before you can pave a road, and then figure out what to do with the mess. But more than anything: Oddly shaped buildings that fit into oddly shaped lots, which are not limited to hotdog stands.
by usefulposter | 2026-02-09 09:50:15+00:00
This is one of the myriad manifestations of the "keep4o" people spilling onto HN.

They are especially aggrieved because OpenAI plan to deprecate the 4o model one day before Valentine's (https://openai.com/index/retiring-gpt-4o-and-older-models).

https://x.com/search?q=%23keep4o
by vladvasiliu | 2026-02-09 09:51:27+00:00
> Wayland, fractional scaling, 120+ Hz, HDR

What's wrong with those? I don't have a single screen which does 120 Hz + HDR, but I'm typing this on a 120 Hz laptop, with variable refresh rate, at 125% scaling, and everything works great with Plasma (haven't tried anything else). I also have an external HDR screen, but it only does 60 Hz. It works great, too, doing HDR on it but not on the laptop screen (running at the same time, of course). They also run at different scaling (125% and 100%).

Now I don't know how to confirm that VRR is actually doing anything, but I can tell there's a difference between setting the monitor to 60 and to 120 Hz. HDR on the other screen also produces a clear difference.

This is all running from integrated intel graphics, maybe with other GPUs it's more of a crapshoot, no idea.
by soufron | 2026-02-09 09:52:05+00:00
I like the half-life of knowledge a lot, but it should better take into account that using your craft actually improves what you know. This half-life is more a thing for stuff you never use in your life. And is there anything good to remember things that have no use for you? It's just wasting space in your head.

So having your skills expire after a while might actually be a good thing.
by AJRF | 2026-02-09 09:52:20+00:00
A project builder.

I often have ideas, then spin cycles on starting the project, getting auth in place, making a marketing page, doing SEO, building and configuring pipelines for mobile app release, etc, etc.

My project builder just takes a name, a few configuration options (do you need payments? Analytics?) and spits out a templated build with Terraform that I can 1 (okay maybe 3) click deploy to GCP + App Stores.

The nice thing (I got help with Claude Code) is that now all my projects are in one place, I have a dashboard where I click in to and edit the code (with hot reloading - it deploys code-server and the applications in a small Kubernetes cluster, each project has its own pod) and when I am done editing I just click Deploy and it updates the "production" service in Cloud Run.

Not really interested in selling it as a service or anything (it's a bit too opinionated for that), but it's a very fun project to work on. I need to make Git + Versioning of the code work right now you only have a single copy of the code which isn't great!
by joe_mamba | 2026-02-09 09:52:40+00:00
>Ok, well I guess if Europe is fine with a continued slide in global economic relevance, they can keep their status quo.

EU citizens understood and recognize that economic supremacy of some private sector industries is pointless if the gains all go to the hands of a few tax dodging trillionaires with sex trafficking private islands, while the externalities get outsourced to the environment and the public sector to deal with.

That's why they want to see policies that address the environment and quality of life first, before shareholder returns, even if that makes them less economically dominant.