MurphyA/hacker-news huggingface dataset:
by Twirrim | 2025-04-26 13:37:51+00:00
Backblaze uses erasure encoding, which is currently the best and most efficient way to do storage. It's how every major object storage platform works.

The very quick high level explanation is that in storage you talk about "stretch factor". For every byte of file, how many bytes do you have to store to get the desired durability. If your approach to durability is you make 3 copies, that's a 3x stretch factor. Assuming you're smart, you'll have these spread across different servers, or at least different hard disks, so you'd be able to tolerate the loss of 2 servers.

With erasure encoding you apply a mathematical transformation to the incoming object and shard it up. Out of those shards you need to retrieve a certain number to be able to reproduce the original object. The number of shards you produce and how many you need to recreate the original are configurable. Let's say it shards to 12, and you need 9 to recreate. The amount of storage that takes up is the ratio 9:12, so that's a 1.3x. For every byte that comes in, you need to store just 1.3x bytes.

As before you'd scatter them across 12 shards and only needing any 9 means you can tolerate losing 3 hard disks (servers?) and still be able to retrieve the original object. That's better durability despite taking up 2.7x less storage.

The drawback is that to retrieve the object, you have to fetch shards from 9 different locations and apply the transformation to recreate the original object, which adds a small bit of latency, but it's largely negligible these days. The cost of extra servers for your retrieval layer is significantly less than a storage server, and you wouldn't need anywhere near the same number as you'd otherwise need for storage.

The underlying file system doesn't really have any appreciable impact under those circumstances. I'd argue ZFS is probably even worse, because you're spending more resources on overhead. You want something as fast and lightweight as possible. Your fixity checks will catch any degradation in shards, and recreating shards in the case of failure is pretty cheap.
by rdtsc | 2025-04-26 13:37:59+00:00
> Australian Border Force superintendent, James Ryan, said he hoped the case would make more people aware of the regulatory frameworks around what can and cannot be imported into Australia

Ah yes, the truth comes out. It was about making an example out of him. They knew immediately it wasn’t a big deal but they figured to have some “fun”. I guess people who weren’t aware are now aware that of the kind of people who work in Border Force.
by chamomeal | 2025-04-26 13:38:25+00:00
If you’re not knowledgeable about cars, getting a 20 year-old truck is probably just not an option. Sometimes it’s totally fine, sometimes it craps out and needs a whole engine replaced!

My sister got a 2003 subaru last year for about $3,000. The oil leaked out while she was driving, and the whole engine sorta melted together and just totally died.

So I’d say for non car people, this Slate truck isn’t competing with old cars, if only on the basis of potential hidden catastrophes.
by roenxi | 2025-04-26 13:39:48+00:00
> Margret Thatcher wasn’t a “faction of the British Government”. She was Prime Minister for 11 years.

Fair enough, faction of British politics. She didn't have the power to shut down the BBC, so she obviously didn't represent the consensus position. Again, the argument seems like it would be that the BBC isn't propaganda because the British PM is relatively weak. That doesn't hold together. Besides, Putin isn't the PM of Russia, Wikipedia tells me that is Mikhail Mishustin.

> But more broadly, you’re arguing at a level of abstraction that rises above actually looking at the content produced by the BBC in comparison to the content produced by RT. You only have to watch each for 15 minutes to see the very clear difference.

So if RT was better written then it wouldn't be propaganda? Because the fact that the BBC has better journalists and targets the middle and upper class in style doesn't particularly mean much except they're better at their jobs than the RT people. You're mistaking propaganda for low quality writing with that one. Good propaganda relies on truth and being mostly credible (see also - the model pioneered by the BBC with enormous success).

> You asked “If [the BBC] isn't pushing a British viewpoint, wouldn't it be incumbent on the British government to shut it down?” That clearly suggests that the government of the day could defund the BBC if it displeased them.

"displeased" is a bit vague but yes if there was a consensus in the Houses of Lords and Commons that the BBC wasn't advancing the interests of Britain I imagine it'd not last long. The parliament is quite powerful when it unites on a question of policy. That doesn't mean a PM can just snap their fingers and the BBC disappears, it'd be a long process.
by mfer | 2025-04-26 13:39:49+00:00
Imagine the time we would have if....

Instead of doom scrolling on social media we called a friend.

Instead of binge watching another meh show, we had friends over to play cards or a board game.

Instead of over scheduling kids with constant activities, parents had a regular night out with friends while kids spent quality time at home with the other parent.

The time is there. It was in the past. We just have a finite amount of time and use it differently.
by barry-cotter | 2025-04-26 13:41:12+00:00
This reminds me of a tweet I saw in 2023, during the height of the Floyd Riots, that has since unfortunately been deleted, that said approximately that suburbanisation was a long term bet on the inability of the US State to keep order in urban areas. Basically, once a generation (last one before the Floyd riots were the Rodney King riots) the US has massive riots, everyone learns of remembers that if you don’t live in walking distance of an urban centre you’re safe from having your house burned down by a mob and after that there’s a durable uptick in desire for suburban rather than urban property.
by deng | 2025-04-26 13:41:35+00:00
Well, the police also said he bought mercury, which "can be used in switches for a dirty bomb", which is such a stupid thing to say, because a mercury switch is just an old form of a tilt switch. The idea that someone would buy mercury for making his own tilt switch is just so wild, but of course, they just put this BS out there to scare people and justify their completely overblown reaction.
by kasey_junk | 2025-04-26 13:42:36+00:00
That’s _just_ the peg. The other currency controls include the prevention of currency outflows by Chinese capital and the restriction on foreign holders of Chinese debt. All of that drives the costs down.

My personal opinion is that the Chinese EV would dominate in a completely free market, but we will never know.

My broader point is that it’s weird to say that the cash subsidies make up for the lack of freer markets capital, that’s double dipping.
by camel-cdr | 2025-04-26 13:42:52+00:00
I quite like highway.

As mentioned, last time I tried vqsort for RVV it was surprisingly slow.

I tried to replicate it yesterday, but noticed that vqsort is now disabled for RVV: https://github.com/google/highway/blob/400fbf20f2e40b984be12...

Does highway support sorting networks for non-128-bit vector registers?

When I tried to compile it for AVX512, the BaseCase seems to only use xmm registers: https://godbolt.org/z/qr9xoTGKn
by codewench | 2025-04-26 13:43:11+00:00
Seeing as how this article is talking about the deportation of US citizens, I'm going to question what exactly you mean by "here illegally".

Expanding the argument: I've just decided that you are illegally, and will thus be deported. As there is no due process, my word is law, have fun wherever you end up I literally do not care.

Does that seem fair? And before arguing "well this wouldn't happen, I'm not here illegally", again, this is an article about the deportation of US citizens. Children no less.
by zettabomb | 2025-04-26 13:43:39+00:00
>It’s not like his activities had zero impact in his community.

They didn't. The ridiculous and uninformed government reaction caused this. Nothing he did was even remotely dangerous.

>You don’t mess around with radioactive materials; even small amounts can be extremely hazardous to life and the environment.

These materials were not dangerous, it was literally a capsule from a smoke detector. As in, an average person would've had it in their house.

>There’s a reason they’re not easy to obtain.

Right, so difficult to obtain that he was able to simply order them online and have them delivered through the mail.
by fidotron | 2025-04-26 13:43:49+00:00
Nest have done a really good job of poisoning the well of consumer IoT. If a well funded startup acquired by a tech giant isn't going to do this properly then what chance does anyone else have? Absolutely disgraceful.

Fundamentally the idea end point devices should be connected to the Internet is wrong, and this means the idea the manufacturer should run any associated cloud service or be responsible for "security updates" is also wrong.

The only long term solution to this is that "smart devices" not be given direct access to the Internet, where the security problems are, and live instead in their own strictly segregated networks controlled by something conceptually like Home Assistant but where it also acts as a broker between all local devices, with a whitelist only policy for networking access. Until that becomes the default mode for deploying these things the consumer IoT space will continue to fail to meet expectations.
by loginatnine | 2025-04-26 13:43:51+00:00
This week, Hydro-Québec, the nationalized company that provide electricity to residents of the province, has announced a major investment program to reduce electricity usage by using smart thermostats. I'm extremely worried about the life expectancy of those smart thermostats in the long run and whether it's a good use of public funds. I'm also not super thrilled of the amount of functional regular thermostats that will end up in landfills because of that initiative.

[1]https://news.hydroquebec.com/en/press-releases/2172/hydro-qu...
by gosub100 | 2025-04-26 13:43:57+00:00
this is where the FTC should step in and say "FOSS it" so at least consumers can have a choice and companies compete. Google could argue that it costs some nonzero amount (which is infinitesimal, I know), therefore they can't offer it perpetuity, but they could be required to unlock it so a 3rd party can compete and offer you the same service or let FOSS users control it.
by lolinder | 2025-04-26 13:44:00+00:00
> asked it to look up their URLs and give me a list

Something missing from this conversation is whether we're talking about the raw model or model+tool calls (search). This sounds like tool calls were enabled.

And I do think this is a sign that the current UX of the chatbots is deeply flawed: even on HN we don't seem to interact with the UI components to toggle these features frequently enough that they're the intuitive answer, instead we still talk about model classes as though that makes the biggest difference in accuracy.
by true_religion | 2025-04-26 13:45:19+00:00
I have two sets of grand parents. One was relatively well off, and the other not.

I can say, the cutlery inherited from the poorer pair is not great. Some is bent. Some was broken and then repaired with different materials. Some is just rusted. And the designs are very basic.

It’s one of the few surviving things from them, so I haven’t thrown it away but I doubt my kids will want to inherit it since they don’t even know them.

I think survivorship bias plays into effect here strongly.
by tpmoney | 2025-04-26 13:45:25+00:00
> We have too much fake-niceness, and we are over-enthused quite often on things that turn out to be nothing, at least in the U.S. We don't need more of it, at least IMO.

I don't think the original article is advocating for "fake niceness". It's advocating for enthusiastic uplift. That spirit that tends to pervade small hobbyist communities, where everyone is iterating and building on everyone else. If you've been in the 3d printing space over the last decade or so, there's plenty of honest criticism and knowledgable discussion. But it's also infectiously enthusiastic and there's a sense of exploration and desire to see everyone* succeed that's very genuine. Things don't always succeed or work out, but it's pretty rare to encounter hard "that's stupid, you're stupid and you should feel bad" sort of commentary.

Compare this to the broader tech community as it appears in places like HN. Open any thread on someone's new project or experiment here and count the number of comments that are genuinely positive and encouraging vs the number that are nit-picking for the sake of nit-picking, dismissive and just generally unhappy or are outright actively tearing down the item in question. Even the ones that are "Oh this is nice, but hey you could do XYZ to improve it" very rarely have any follow up that submits the suggested improvements, even when the original item is an open source project that they clearly could submit improvements to. People love to talk about the shortcomings, but unlike "fans" rarely tend to put out the time and effort to make it better.

It's bad enough that my general approach to HN these days is to only read the articles and make an effort to avoid the comment threads or at a minimum make sure I've read the original link in its entirety and thought about my own opinions of it before heading into the comments. And when I find something in a project that I think could be improved, I make an effort to ensure that I'm putting proper effort into trying to improve it, either with a patch/fix or if I'm incapable of patching/fixing, as much detail and testing as I can document in order to have contributed more than just a drive by criticism.

Also, I'm not sure a "too much niceness", fake or otherwise, is the problem in the US right now.
by sokoloff | 2025-04-26 13:46:43+00:00
> Were this a public project it would have persisted indefinitely and have a lobbying constituency to keep setting money on fire.

And, more importantly, keep going on with a strategy that isn't actually helping the kids in the program... I can easily look past the government wasting an extra $50M/yr here and there, because that barely matters in the grand scheme of government waste, fraud, and inefficiency. But getting poor educational and life-preparation outcomes for generations of kids because of inertia is an entirely greater concern, at least to me.
by lifeisstillgood | 2025-04-26 13:47:02+00:00
I have long assumed that we won’t be getting robot butlers partly because it’s really really hard, but also because most of not all things we want robots for it’s easier to reconfigure the environment than make a flexible humaniod

So factories are obvious but the real mass uptake is the home - and honestly I think something that cleans and tidies an hour a day might actually be achievable
by Twirrim | 2025-04-26 13:47:17+00:00
If they care about compression at the ZFS level, that means your file is going to be visible to anyone able to log in to the server, because they're relying on (at best) encryption at rest. That's not a great security model for a storage service. You don't want anyone to be able to log in to a server and see your actual files unencrypted.

If they're going to compress/decompress, ideally you'd want them to have that at the point of ingestion, then encrypted, and then store that on the target drive.

That way you can put very strong controls (audit, architecture, software etc) around your encryption and decryption, and be at reduced risk from someone getting access to storage servers.
by Ichthypresbyter | 2025-04-26 13:47:44+00:00
>Not only does the system know exactly where you are at every moment, it knows who your friends are, what they are interested in, and who you are spending time with

This actually makes sense of an anecdote a colleague uses to say that he thinks his phone is listening to him.

I am a keen skier. He used to ski a lot, but hasn't been for several years. Around the start of ski season this year, we talked about my plans to go skiing that weekend, and later that day he started seeing skiing-related ads.

He thinks it's because his phone listened into the conversation, but it could just as easily have been that it was spending more time near my phone (I had only recently started at that job) on which I regularly search for skiing-related things like conditions reports and directions to ski areas.
by fidotron | 2025-04-26 13:48:11+00:00
That doesn't mention the key detail of the Hydro Quebec smart thermostats: they keep remote control of the thermostat.

The whole idea is they can turn down your thermostat and sell the electricity to the US instead, or it was. Somehow they also encourage the adoption of electric cars, while also claiming we should save electricity. It's a very confused message.
Show HN: Diffulab, a library to train diffusion models from scratch
by AdilZtn | 2025-04-26 13:48:17+00:00
Lately a friend of mine has been working on a personal project and I wanted to share it. The goal of Diffulab is to provide a flexible and modular framework for training diffusion models from scratch. The project is still in its early stages, and he is actively working on adding new features and improvements.
by swores | 2025-04-26 13:49:29+00:00
Ah, yes you're right - I didn't clarify this in my original comment, but my anecdote was indeed the ChatGPT interface and using its ability to browse the web[#], not expecting it to pull URLs out of its original training data. Thanks for pointing that out.

But the reason I suggested model as a potential difference between me and the person I replied to, rather than ChatGPT interface vs. plain use of model without bells and whistles, is that they had said their trouble was while using ChatGPT, not while using a GPT model over the API or through a different service.

[#] (Technically I didn't, and never do, have the "search" button enabled in the chat interface, but it's able to search/browse the web without that focus being selected.)
by p_ing | 2025-04-26 13:49:44+00:00
The issue is you're picking out bits and pieces that /seem/ correct to you, but you don't seem to have experience defending a public website from intrusions.

These rules do in fact work. Like I've said previously, these rules require tuning for your particular website. If I'm "talking about math" then I would modify or disable that rule as needed.

I think this is the forest you're missing. WAF isn't "install it and walk away". WAF needs to be tested in conjunction with your release, like any other code would.

The WAF can and does protect against attacks your code would never think of. It also /logs requests/ in a way your web server will not, making it invaluable for auditing.

And when running 3rd party software that has a function you cannot control, but need to prevent, WAFs can do that, too. I have a particular query string that must work from an internal but not external network while external/internal users leverage the same URL -- WAF can do that with a custom rule examining the query string and denying access to the outside world.

Or if I need to prevent [AI] bot scraping. WAF can do that with a couple of clicks.

WAF also unloads the web server from malicious traffic. Instead of having to size up or out a web server, I can have a WAF appliance prevent that traffic from ever reaching the server.

> Every WAF ruleset I've read so far

You don't appear to have any experience with implementation or operation of a WAF, but are attempting to be authoritative and dismiss a WAFs utility.
by kashunstva | 2025-04-26 13:50:05+00:00
> When Hillary called half the nation "deplorables", that was closer to Hitler rhetoric than anything I've heard out of this administration.

Technically Secretary Clinton called half of her opponent’s supporters a “basket of deplorables.” So 0.25 of the voting population at most.

But if that sounds worse than anything uttered by this administration, you’re not listening closely. I’m Canadian and we’ve been called “one of the nastiest countries.”
by somenameforme | 2025-04-26 13:50:17+00:00
Every day across the world thousands of people are removed from countries around the world for violating immigration laws. Except in cases of where it coincided with criminality, it's always going to be very ugly, because it means somebody had built up a life for themselves somewhere and that is now ended due to them having been born in a different place and then overstayed their permission, or never received such, to stay somewhere else.

Like in this case, what do you propose as an alternative with a precedent that you think could be agreeable to most people? The parents were in the country illegally, and the children's citizenship was solely one of birthright. Any sort of "pleasant" outcome would effectively require turning birthright citizenship into defacto citizenship for the parents as well, at least if they can stay illegally for long enough. That's not only completely unrealistic, but also a complete slap in the face to the millions of people who try to migrate legally and are refused entry.
by skydhash | 2025-04-26 13:50:50+00:00
> Art and things created by other people doesn’t have to pander to you, it expresses something they made and you are allowed to observe or interact with it.

And that's why you need to curate as it's an expression of your choice. Both by choosing which feeds to follow and which articles to read. Social media is like being in a public square alone while everyone is shouting at you while feeds are more like being in a gathering. In the latter, everyone is already vetted so you just skip from one interesting conversation to another while recognizing keeping track of everything is pointless.
by heresie-dabord | 2025-04-26 13:51:21+00:00
Unless Saint Friedman got his "doctrine" from some higher power, it's just the oligarchy's first commandment.

In the first line of GP's reference in Wikipedia:

"The Friedman doctrine, also called shareholder theory, is a normative theory of business ethics advanced by economist Milton Friedman that holds that the social responsibility of business is to increase its profits."
by Phanteaume | 2025-04-26 13:51:48+00:00
Hmm. I sometimes think that people in the past were "the same as us", just in a different environment. Could you imagine such a young woman writing such a book today ?

The damage done by instant messaging and endless entertainment is unfathomable.

We, and more importantly our kids, are becoming dumber and this is but another proof on the pile. What is to be done about it ? Raise luddites ?
by pcthrowaway | 2025-04-26 13:52:07+00:00
We haven't had any issues with the free notification service. They did release the code to host your own but I think you'd also have to rebuild and release their mobile app to use it, and that sounds like a massive hassle.

I'm using it for a nonprofit, so $10/month per user is out of the question. Self-hosting and open source are critical to us also.
by LightBug1 | 2025-04-26 13:52:21+00:00
"Trump appointee Ed Martin accuses the online encyclopedia of “allowing foreign actors to manipulate information and spread propaganda to the American public.” April 25, 2025 at 6:54 p.m. EDT"

How about these wankers turn their attention to their own administration ...

Oh, pardon me ... what a ludicrous thing to suggest.