MurphyA/hacker-news huggingface dataset:
by squigz | 2026-01-13 22:35:47+00:00
> The /metadata/lookup API endpoints (GET and POST versions) now require the caller to send an Authorization token in order for this endpoint to work.

> The ListenBrainz Labs API endpoints for mbid-mapping, mbid-mapping-release and mbid-mapping-explain have been removed. Those were always intended for debugging purposes and will also soon be replaced with a new endpoints for our upcoming improved mapper.

> LB Radio will now require users to be logged in to use it (and API endpoint users will need to send the Authorization header). The error message for logged in users is a bit clunky at the moment; we’ll fix this once we’ve finished the work for this year’s Year in Music.

Seems reasonable and no big deal at all. I'm not entirely sure what "nice things" we can't have because of this. Unauthenticated APIs?
by efreak | 2026-01-13 22:36:13+00:00
GPT2 at 774m is considered a LLM. I wouldn't say there's much difference between that and 700m, or even 123M.

Having said that, looking up small language model these days returns tons of results calling 7B models small language models.

------

My understanding of small language models is that they're generally intended for specific purposes, like analysis and classification (whatever you'd call the text equivalent of image interrogation with clip models), translation, etc; that there small because they don't need to be big to do their intended functions, not because they're just smaller versions of bigger models.
by TeMPOraL | 2026-01-13 22:36:43+00:00
> Employees are under contract and are screened for basic competence. LLMs aren't

So perhaps they should be.

> and can't be.

Ah but they must, because there's not much else you can do.

You can't secure LLMs like they were just regular, narrow-purpose software, because they aren't. They're by nature more like little people on a chip (this is an explicit design goal) - and need to be treated accordingly.
by testacc74 | 2026-01-13 22:36:48+00:00
Well I don't know much about French aside from the joke but Cannes obviously has way better taste than Hollywood.

The neat part is, with social media and global internet, culture are generally degenerating regardless of countries. If I were American citizen(gladly Im not) I might as well just let them in to get some tax money, and potentially see if the industry can be used as a soft propaganda machine. Just like Hollywood.
by racl101 | 2026-01-13 22:36:52+00:00
One thing I still love about blogs like this that AI can never replace is that it takes a topic that might seem complicated to some people such as myself (Git Rebase does actually terrify me) and it talks about it in great detail in a way where I don't have to know what to ask. I don't have to prompt about something I don't even know where to begin. The topic is introduced and elaborated on while speaking to a general audience. So that's why I still like to see tech blogs out there.
by kouteiheika | 2026-01-13 22:37:20+00:00
> With the one AI, we can do word-to-image to generate an image. Clearly, that is a derived work of the training set of images

> The question of whether AI is stealing material depends exactly on what the training pathway is; what it is that it is learning from the data.

No it isn't. The question of whether AI is stealing material has little to do with the training pathway, but everything to do with scale.

To give a very simple example: is your model a trillion parameter model, but you're training it on 1000 images? It's going to memorize.

Is your model a 3 billion parameter model, but you're training it on trillions of images? It's going to generalize because it simply doesn't physically have the capacity to memorize its training data, and assuming you've deduplicated your training dataset it's not going to memorize any single image.

It literally makes no difference whether you'll use the "trained on the same scene but one in daylight and one at night" or "generate the image based on a description" training objective here. Depending on how you pick your hyperparameters you can trivially make either one memorize the training data (i.e. in your words "make it clearly a derived work of the training set of images").
by lep_qq | 2026-01-13 22:37:31+00:00
This is frustrating to watch. MetaBrainz is exactly the kind of project AI companies should be supporting—open data, community-maintained, freely available for download in bulk.
Instead they’re:
∙ Ignoring robots.txt (the bare minimum web courtesy)
∙ Bypassing the provided bulk download (literally designed for this use case)
∙ Scraping page-by-page (inefficient for everyone)
∙ Overloading volunteer-run infrastructure
∙ Forcing the project to add auth barriers that hurt legitimate users
The irony: if they’d just contacted MetaBrainz and said “hey, we’d like to use your dataset for training,” they’d probably get a bulk export and maybe even attribution. Instead, they’re burning goodwill and forcing open projects to lock down.
This pattern is repeating everywhere. Small/medium open data projects can’t afford the infrastructure to handle aggressive scraping, so they either:
1. Add authentication (reduces openness)
2. Rate limit aggressively (hurts legitimate API users)
3. Go offline entirely (community loses the resource)
AI companies are externalizing their data acquisition costs onto volunteer projects. It’s a tragedy of the commons, except the “commons” is deliberately maintained infrastructure that these companies could easily afford to support.
Have you considered publishing a list of the offending user agents / IP ranges? Might help other projects protect themselves, and public shaming sometimes works when technical measures don’t
by raw_anon_1111 | 2026-01-13 22:37:41+00:00
Once you move beyond a mid level developer [1], line level managers are more or less useless in my experience. They don’t control raises, promotions, budgets or strategies.

I worked at two companies before I got into consulting where I was recruited to lead major hairy initiatives, the conditions of employment for both was that I wanted to report to the director/cto - someone who controlled budgets.

[1] no matter what your title is, if you are just pulling well defined tasks off the board, you’re a mid level developer.
by OhMeadhbh | 2026-01-13 22:37:48+00:00
we also sort of effed up a while ago with changes to section 174... suddenly software devs in the states were 10%-25% more expensive. once that happened it made sense to see if moving devs to europe for situations where you have a european based product and sales team made sense.

in the states we've sort of repaired the damage of the section 174 changes, but i think they were rolled into a tax bill that sunsets in a few years. so we may see this again in 2029.
by sgtwompwomp | 2026-01-13 22:37:52+00:00
It's not a business's job to make their documentation accessible to their potential and current customers?

I would ask if you've started a real business but it's clear you haven't. It is 100% on a developer tool startup to provide documentation that is easily accessible. If they don't, customers will struggle to get value. If you think this isn't true, then you are ignoring the gigantic market of companies purchasing documentation products (look at Mintlify's customer base for reference)

There is no way I'm asking my customers to scrape my docs and build their own MCP server and AI assistant just to access it easily.
by tensegrist | 2026-01-13 22:37:53+00:00
the more time passes the more i'm convinced that the solution is to—somehow—force everyone to have to go through something like common crawl

i don't want people's servers to be pegged at 100% because a stupid dfs scraper is exhaustively traversing their search facets, but i also want the web to remain scrapable by ordinary people, or rather go back to how readily scrapable it used to be before the invention of cloudflare

as a middle ground, perhaps we could agree on a new /.well-known/ path meant to contain links to timestamped data dumps?
by pizzafeelsright | 2026-01-13 22:38:20+00:00
Following up - I am the most excited about using computers because the barrier from intent to product are being dropped. At this point my children can 'code' software without knowing anything other than intent. Reality is being made manifest. Building physics into a game would take a decade of experience but today we can say "allow for collision between vehicles".

If you have ever gone running the ability to coordinate four limbs, maintain balance, assert trajectory, negotiate uneven terrain, and modify velocity and speed at will is completely unknown to 99.9% of mortals who ever lived and yet is possible because 'biological black box hand wave'.
by solid_fuel | 2026-01-13 22:38:25+00:00
> He was directly told that he won't be promoted because he was a white man.

Even if that was true (I don't believe his allegation), that's just _one company_. He obviously considered himself a very intelligent and capable person, so it seems the obvious next step would be to go work basically anywhere else? The Dilbert comics never seemed to push the ideal of company loyalty, so I don't think he felt trapped by obligation there.

One only needs to look at the upper management and board of any fortune 500 to disprove the idea that only non-white women are getting promoted.
by dgl | 2026-01-13 22:38:32+00:00
This.

Instead you can create multiple Wireguard interfaces and use policy routing / ECMP / BGP / all the layer 3 tricks, that way you can achieve similar things to what vxlan could give you but at layer 3.

There's a performance benefit to doing it this way too, in some testing I found the wireguard interface can be a bottleneck (there's various offload and multiple core support in Linux, but it still has some overhead).
by rly0nheart | 2026-01-13 22:38:59+00:00
Someone is impersonating me. If you try to contact me, please use the links on my blog.

These aren't mine:

https://xcancel.com/rly0nheart
https://about.me/rly0nheart

Unfortunately, I don't think I can report impersonation on X without having an account, and X is a no go for me.
by Pet_Ant | 2026-01-13 22:39:12+00:00
I wish more resources were available legitimately. There is a dataset I need for legitimate research that I cannot even find a way to contact the repo owners.

Mind you I take effort to not be burdensome by downloading only what I need and taking time between each request of a couple seconds, and the total data usage is low.

Ironically, I supposed you could call it "AI" what I'm using it for, but really it's just data analytics.
by tyre | 2026-01-13 22:39:14+00:00
I've been an early employee (sub 10 and 20) in two unicorns and another (a presidential campaign) that didn't have a valuation but did the equivalent. People did not work 40 hours per week, and I feel comfortable saying that the companies could not have been as successful if people had.

The common threads were:

- incredible ICs

- founders who spiked in the most important areas for that market

- a mission that everyone truly believed in

- a culture of people who deeply cared about one another but were comfortable pushing back (as you said!)

It's incredibly rare to find all of these together. I agree that management is responsible for helping others thrive, but not necessarily that they should shape the environment to fit any engineer. Some people want things (projects, challenges, roles) that don't make sense in that company's context. It's okay, especially when it's hard, to agree that this isn't the place for someone.
by Animats | 2026-01-13 22:39:21+00:00
If you need motivation, maybe the organization is designed badly.

It was once said of the Roman legions "The Legion is not composed of heroes. Heroes are what the Legion kills."
Field Marshall the Viscount Slim, who commanded in the China-Burma-India theater in WWII, once wrote "Wars are won by the average performance of the line units."
He wrote negatively on various special forces type units, preferring to use regular infantry and training them up to a good, but not superhuman, standard.
Arthur Imperatore, who had a unionized trucking company in New Jersey, is profiled in "Perfecting a Piece of the World" (1993) for how he made his trucking company successful despite a very ordinary workforce.

There's an argument for winning by steady competently managed plodding. The competently managed part is hard. Steve Bechtel, head of the big construction company that bears his name, once said that the limit on how many projects they could take on was finding bosses able to go out to a job site and make it happen. Failure is a management problem, not a worker problem.
by tonymet | 2026-01-13 22:39:35+00:00
The battery + thermals give you either a large cube (like a Beelink++) or cigarette carton. It's wise to go with a cigarette carton and include a keyboard.

Especially since remote work took off, offices are full of vacant workstations. Why drag a small monitor around when you can hook into a 32'' monitor at the office?
by Rebelgecko | 2026-01-13 22:39:47+00:00
What gives you that impression? Unless you're looking at the pre-9/11 grid powered by Enron™, rolling blackouts aren't a big deal in California. I think since 2001 there was a single 2 day event limited to certain regions, I don't know anyone personally who lost electricity due to rolling blackouts.

The CA grid has also scaled up battery storage surprisingly quickly. A few years ago it was in the single digit mWh, not really a meaningful fraction of the grid. Now it's measured in gigawatt-hours.
by smallerfish | 2026-01-13 22:40:00+00:00
Bear in mind that some portion of this could be human directed research. I'm doing a research project right now with 1000 things that I'm building profiles on; to build a full profile requires an agent to do somewhere around 100 different site lookups. Where APIs exist, I've registered API keys and had the agent write a script to query the data in that manner, but that required me to be deliberate about it. Non technie plebs aren't likely to be directed to use an API by the agent.
by TeMPOraL | 2026-01-13 22:40:39+00:00
Reality doesn't have a distinction between "code" and "data"; those are categories of convenience, and don't even have a proper definition (what is code and what is data depends on who's asking and why). Any such distinction requires mechanically enforcing it; AI won't have it, because it's not natural, and adding it destroys generality of the model.
by dannyobrien | 2026-01-13 22:40:44+00:00
Metabrainz is a great resource -- I wrote about them a few years ago here: https://www.eff.org/deeplinks/2021/06/organizing-public-inte...

There's something important here in that a public good like Metabrainz would be fine with the AI bots picking up their content -- they're just doing it in a frustratingly inefficient way.

It's a co-ordination problem: Metabrainz assumes good intent from bots, and has to lock down when they violate that trust. The bots have a different model -- they assume that the website is adversarially "hiding" its content. They won't believe a random site when it says "Look, stop hitting our API, you can pick all of this data in one go, over in this gzipped tar file."

Or better still, this torrent file, where the bots would briefly end up improving the shareability of the data.
by nostrademons | 2026-01-13 22:40:49+00:00
That's sorta what MetaBrainz did - they offer their whole DB as a single tarball dump, much like what Wikipedia does. I downloaded it in the order of an hour; if I need a MusicBrainz lookup, I just do a local query.

For this strategy to work, people need to actually use the DB dumps instead of just defaulting to scraping. Unfortunately scraping is trivially easy, particularly now that AI code assistants can write a working scraper in ~5-10 minutes.
by bb88 | 2026-01-13 22:40:53+00:00
But did all the nodes and repeaters update to the version of meshtastic with routing? Meshcore started with routing out of the box.

I don't think it's the noise that's too high, my original understanding of the meshtastic 2.0 protocol (from this video: https://www.youtube.com/watch?v=7v6UbC5blJU) was that an edge device would receive priority. So the message might go the wrong direction, say. Or, worse, a meshtastic radio in a basement might get priority.

Because they didn't understand (or care about) network topologies from the start and reliable message transmission, not to mention the versions of devices that are still on their network, I'm guessing it's gonna be a couple more years before they finally get their act together.
by Groxx | 2026-01-13 22:41:21+00:00
I also get tons of mileage out of This One Weird Trick: make a new branch first. Every single time it's non-trivial, just abort and branch before continuing.

Then if you screw up, even several steps later when it's hard to un-rebase a portion, you just go back to the original. No need to dig through the ever-confusing reflog format, just use branches.

Once you're fully happy with it, you can push that new one, or just go back to your original and `git reset --hard after-rebase-branch` to adopt the new history.
Show HN: Capture anything you see, here or think in your own knowledge base
by chriswright1664 | 2026-01-13 22:41:35+00:00
Something I have been working on for a while. Pet project for my own use that a bunch of people asked me if they could play with.

Chrome extension and desktop widget (Electron) to make it easy to capture notes, screenshots, links, audio, files almost anything. Pushed into your own knowledge base. We apply a little AI to fix typos, pull out topics, people, but of summarisation (needs a lot of work), and relationships to other entries.

You can search, quick and dirty or AI natural language and use grids, kanban and other UIs so sort and interact with whatever you have captured - projects, tasks, knowledge.

Lots of customisation to make it work your way.

There are lots of bugs, issues, and maybe poor design choices. And HN is tough crowd. But I also think there is the bones of something cool here so would love feedback.
by Imustaskforhelp | 2026-01-13 22:41:38+00:00
If someone wants to scrape. I mean not levels of complete internet similar to how google does but at a niche level (like you got a forum you wish to scrape)

I like to create tampermonkey scripts regarding these. They are like more lightweight/easier way to build extensions mostly imo

Now I don't like AI but I don't know anything about scraping so I used AI to generate the scraping code and paste it in tampermonkey and let it run

I recently used this for where I effectively scraped a website which had list of vps servers and their prices and I built myself a list of that to analyze as an example

Also I have to say this that I usually try to look out for databases so much so that on a similar website like this related to something, I contacted them about db but no response, their db of server prices were private and only showed lowest

So I picked the other website and did this. I also scraped all headlines of lowendtalk ever with their links for semi purposes of archival and semi purposes of scraping the headlines and parsing it to LLM to find a list of vps providers as well
by tomhow | 2026-01-13 22:41:57+00:00
When you do the job we do, and spend hours each day looking at the discussions and the data, it becomes fairly easy to spot inauthentic actors on HN. It stands out when newly registered accounts (which appear in green) or older accounts without much posting history, suddenly start posting comments in support of a particular position.

This is another reason why HN's primary purpose – gratifying intellectual curiosity – is so important. Curious conversation is hard to fake. You can't really be inauthentically curious, at least not for long. So the more we can succeed at nudging the community's conduct towards curious conversation, the more it stands out when accounts are acting to promote a particular agenda, and the faster we can weed them out.
by tonymet | 2026-01-13 22:42:16+00:00
W & watts has become a colloquialism for watt-hours in the EV space at least.

e.g. "how much did consume at the super charger" -- about 10 watts (WH).

So it's not literally correct, but it's becoming commonplace to say that.

slang , pigeons and dialects all develop based off of context like this.
by bigstrat2003 | 2026-01-13 22:42:26+00:00
If GW couldn't drive their customers away with:

* deprecating people's models so that they have to buy new ones

* making any number of rules changes that were widely hated

* making lore changes that were widely hated

They aren't going to lose customers because some other company is using AI. They effectively don't have any competition, because people love the Warhammer settings and want to play games set in them.
by spankalee | 2026-01-13 22:42:29+00:00
Wait, so you would personally say things like "Black people are a hate group", except that you need to stay employed?

That is exactly what I mean about making bigotry shameful again. You should worry about losing your job, and your friends and family, for that.

If you stay quiet about your hateful views, then others are more likely too, and maybe some day we can eventually, slowly, move past all of you.
by kgarten | 2026-01-13 22:42:55+00:00
“His peaks—the biting humor about corporate absurdity, the writing on systems thinking and compounding habits, the clarity about the gap between what organizations say and what they do—unquestionably made me healthier, happier, and wealthier.”

Maybe I’m getting cynical, yet every time I see an mdash and rules of 3, it triggers the feeling of “This sounds like AI” …

Here’s another example:

“ I can avoid the ugliness—the racism, the grievance, the need to be right at any cost.”
by SahAssar | 2026-01-13 22:43:10+00:00
> So perhaps they should be.

Unless both the legalities and technology radically change they will not be. And the companies building them will not take on the burden since the technology has proved to be so unpredictable (partially by design) and unsafe.

> designed to be more like little people on a chip - and need to be treated accordingly

Deeply unpredictable and unsafe people on a chip, so not the sort that I generally want to trust secrets with.

I don't think it's that complex, you can have secure systems or you can have current gen LLMs. You can't have both in the same place.
by didgetmaster | 2026-01-13 22:43:43+00:00
Once upon a time, you could enjoy the works of a creative person at face value; mainly because you didn't know much of anything about their personal life.

That seems to have all changed in this age of the Internet; where every aspect of your life is exposed for all the world to judge (at least if you are famous). All your words (written or spoken) are presented as proof positive that you and your works are not to be tolerated; even if they are from your teenage years.

It seems like you cannot say anything these days without offending a large number of people; some of whom will try to lead a boycott against you.

I generally like to enjoy a good book, movie, blog, or comic strip without letting politics get in the way.
by 19-84 | 2026-01-13 22:44:25+00:00
ive created tooling for an instance registry and team based leaderboard. the API has function to support this as well, so that we can collectively host archives in a decentralized and distributed manner

registry readme:
https://github.com/19-84/redd-archiver/blob/main/docs/REGIST...

register instances:
https://github.com/19-84/redd-archiver/blob/main/.github/ISS...