MurphyA/hacker-news huggingface dataset:
by fc417fc802 | 2025-04-22 21:14:42+00:00
Yes thank you I am aware of what fragmentation is.

There are two sorts of fragmentation that can occur with btrfs. Free space and file data. File data is significantly more difficult to deal with but it "only" degrades read performance. It's honestly a pretty big weakness of btrfs. You can't realistically defragment file data if you have a lot of deduplication going on because (at least last I checked) the tooling breaks the deduplication.

> If your free space is spread across the volume in small chunks, and at 95% full it almost certainly will be

Only if you failed to perform basic maintenance. Free space fragmentation is a non-issue as long as you run the relevant tooling when necessary. Chunks get compacted when you rebalance.

Where it gets dicey is that the btrfs tooling is pretty bad at handling the situation where you have a small absolute number of chunks available. Even if you theoretically have enough chunks to play musical chairs and perform a rebalance the tooling will happily back itself into a corner through a series of utterly idiotic decisions. I've been bitten by this before but in my experience it doesn't happen until you're somewhere under 100 GB of remaining space regardless of the total filesystem size.
by anonym29 | 2025-04-22 21:14:58+00:00
But there is also no publicly known way to implant unwanted telemetry, backdoors, or malware into modern model formats either (which hasn't always been true of older LLM model formats), which mitigates at least one functional concern about trust in this case, no?

It's not quite like executing a binary in userland - you're not really granting code execution to anyone with the model, right? Perhaps there is some undisclosed vulnerability in one or more of the runtimes, like llama.cpp, but that's a separate discussion.
by sokoloff | 2025-04-22 21:15:17+00:00
Seemingly not, at least not for the moment, based on the experiences of people migrating arrays [successfully] from older units and the massive backlash that would result if they trashed currently working arrays.

If you’re going to buy a DSM unit, I’d definitely buy a 2024 or earlier model. But even as an overall happy user under their old go-to-market approach, I can’t recommend them now.
by neaden | 2025-04-22 21:15:56+00:00
"Ditto for The Iliad and The Odysee, yet Troy existed." - As I said in a different comment this comparison makes no sense. Troy was continuously inhabited up until around 1300, we have artifacts like coins from there and multiple attestations from contemporary sources. The only thing that was debated was if the ancient city was underneath the more contemporary one or a few miles away. That is nothing like Atlantis.
by Adityav369 | 2025-04-22 21:16:06+00:00
Depends on your document types.

If you're using txts, then plain RAG built on top of any vector database can suffice depending on your queries (if they directly reference the text, or can be made to, then similarity search is good enough). If they are cross document, setting a high number of chunks with plain RAG to retrieve might also do a good job.

If you have tables, images, etc. then using a better extraction mechanism (maybe unstructured, or other document processors) and then creating the embeddings can also work well.

I'd say if docs are simple, then just building your own pipeline on top of a vector db is good!
by woodruffw | 2025-04-22 21:16:41+00:00
There are two things that work in `uv`'s favor (which, to be clear, is an incredible tool that I'm a big fan of):

1. Python packaging, unlike Homebrew, does have a compute-heavy phase in the form of dependency resolution. `uv` can make significant gains over `pip` in that phase.

2. `uv` performs parallel downloads and other operations, in part because of Rust's fearless parallelism.

Homebrew doesn't really have (1), since resolution is just a linearization of the dependency tree. And parallelism of downloads in (2) poses a problem for Homebrew that's been mentioned in other threads (whereas parallelism is not a significant problem for PyPI's CDN-fronted distribution).
by stevenAthompson | 2025-04-22 21:17:07+00:00
The thoughts in your head, and the information you gather belongs to you and you get to make decisions about how it's used. If it helps think of it as intellectual property, or perhaps an extension of bodily autonomy in less corporate terms.

It's so fundamental a right that most places only enshrine more complicated versions of it as law. Most societies now even extend that right to offer control over the knowledge you store electronically or write down and the way other people get to use "your" knowledge.

When we seek to impose privacy limitations on the data that others can gather we're curtailing those rights to some extent. We're telling them there are specific things they can't know and that does limit their freedom in a very fundamental way.
by bzmrgonz | 2025-04-22 21:18:09+00:00
I think our creativity has not yet been duplicated in AI, so for maximum results, we need to pair AI with a human expert or a panel of human experts and innovate by committee. AI brings to the table vast memory, instant recall and most importantly, tired-less pursuit and the human element can provide creative guidance and prompt. The trick is in curating the BOK(body of knowledge) used to train GENERATIVE AI. I wonder what a curricula designed specifically for AI would look like?
by mcswell | 2025-04-22 21:18:48+00:00
Decades ago, I read a book (written, I think, around 1890) about Atlantis. 99% of the evidence it gave was, of course, bogus. But the one piece that seemed reasonable was an account of depth soundings by the SS Great Eastern when it laid the second cable across the Atlantic in 1866. I haven't seen a recent account of those soundings, but the chart in the book did show the Atlantic to be shallower in the middle--which the author took to be the sunken continent of Atlantis. Of course, now we know that the shallower depths there are the Mid-Atlantic ridge, which was never above water (except up at Iceland).
by lxgr | 2025-04-22 21:19:10+00:00
Hm, I wouldn't say that that's the key point of open software. There are many open source projects that don't have reproducible builds (some don't even offer any binary builds), and conversely there is "source available" software with deterministic builds that's not freely licensed.

On top of that, I don't think it works quite that way for ML models. Even their creators, with access to all training data and training steps, are having a very hard time reasoning about what these things will do exactly for a given input without trying it out.

"Reproducible training runs" could at least show that there's not been any active adversarial RHLF, but seem prohibitively expensive in terms of resources.
by serjester | 2025-04-22 21:20:09+00:00
I think this such an awesome innovation but does anyone understand what incentivizes a ship’s captain to take the warnings seriously? Is this just reliant on people being good people? I haven’t worked with sailors but I did a lot of blue collar work when I was younger and there’s a pessimistic side to me that struggles to see many of those guys caring. I do hope I’m wrong.
by AnthonyMouse | 2025-04-22 21:20:09+00:00
> Freedom of speech doesn't require privacy in most constructions as it typically concerns, first and foremost, publicly-made political speech.

Suppose you're Thomas Paine and you want to advocate for the independence of America from Britain, and that Britain doesn't allow the government to punish anyone for what they say, but does prohibit anyone from publishing their writings anonymously.

Then everyone knows exactly who Thomas Paine is, and even if there is no law against writing those things, there is now a constable poking into his business looking for some other pretext to arrest him, several legislators drafting bills that would negatively impact his livelihood and which will quietly be withdrawn if he would simply stop publishing such things, and a variety of private enterprises who now refuse to have anything to do with him in any capacity because they have business before the crown and fear any public association with their critic.

How is it free speech if you're still getting punished for it?
by codingmoh | 2025-04-22 21:20:10+00:00
Hey, that is a very good question, I have answered that before. I hope you don't mind, if I simply copy paste my previous answer:

Technically you can use the original Codex CLI with a local LLM - if your inference provider implements the OpenAI Chat Completions API, with function calling, etc. included.

But based on what I had in mind - the idea that small models can be really useful if optimized for very specific use cases - I figured the current architecture of Codex CLI wasn't the best fit for that. So instead of forking it, I started from scratch.

Here's the rough thinking behind it:

1. You still have to manually set up and run your own inference server (e.g., with ollama, lmstudio, vllm, etc.). 2. You need to ensure that the model you choose works well with Codex's pre-defined prompt setup and configuration. 3. Prompting patterns for small open-source models (like phi-4-mini) often need to be very different - they don't generalize as well. 4. The function calling format (or structured output) might not even be supported by your local inference provider.

Codex CLI's implementation and prompts seem tailored for a specific class of hosted, large-scale models (e.g. GPT, Gemini, Grok). But if you want to get good results with small, local models, everything - prompting, reasoning chains, output structure - often needs to be different. So I built this with a few assumptions in mind:

- Write the tool specifically to run _locally_ out of the box, no inference API server required. - Use model directly (currently for phi-4-mini via llama-cpp-python). - Optimize the prompt and execution logic _per model_ to get the best performance.

Instead of forcing small models into a system meant for large, general-purpose APIs, I wanted to explore a local-first, model-specific alternative that's easy to install and extend — and free to run.
by aaronbaugher | 2025-04-22 21:21:38+00:00
Jan 1, 2024, I made a resolution to change my diet (basically small, very simple meals with one cheat meal a week). I'd probably made a dozen such resolutions before without much success, but for some reason this one stuck right away. I never even had trouble sticking with it, and lost a pound a week over the next 10 months. Then I met my girlfriend and we went out to eat a lot, and that broke the habit. It seems like big life changes can reset habits.

I wish I knew how I did it, because I've been unable to regain that same "this is just what I do now and it's fine" level of acceptance that seemed to be the key.
by fc417fc802 | 2025-04-22 21:22:03+00:00
Which is exactly why you periodically do maintenance to compact the free space. Thus it isn't an issue in practice unless you have a very specific workload in which case you should probably be using a specialized solution. (Although I've read that apparently you can even get a workload like postgres working reasonably well on zfs which surprises me.)

If things get to the point where there's over 1 TB of fragmented free space on a filesystem that is entirely the fault of the operator.
by thesz | 2025-04-22 21:22:32+00:00
https://arxiv.org/pdf/2503.02113

This paper shows that polynomials show most features of deep neural nets, including double descent and ability to memorize entire dataset.

It connects dots there - polynomials there are regularized to be as simple as possible and author argues that hundredths of billions of parameters in modern neural networks work as a regularizers too, they attenuate decisions that "too risky."

I really enjoyed that paper, a gem that puts light everywhere.
by taeric | 2025-04-22 21:22:34+00:00
Its funny what you can get used to. We had so many storms and general bad weather events growing up, that I don't really give them any thought. Certainly didn't back then.

Out in Seattle, though, if there is a single crack of thunder, everyone is at the windows trying to see what happened. It is almost comical on how this place never really gets a storm.

Does make me somewhat at odds with the crowds that hate firework noise, "because it scares pets." I'm in agreement that it is just obnoxious and I don't miss it. I'm pretty sure thunder was far more frightening for any pets I had, growing up.

All that is to say, probably wise advice on unplugging things. I know that quality of power has gotten a lot more relevant in recent years, such that you should only be worried about very local events. Still, seems safe enough not to take a risk, if you can avoid it.
by frollogaston | 2025-04-22 21:22:45+00:00
The symptom was something like https://stackoverflow.com/questions/16432071/how-to-fix-home... , which btw has a terrible accepted answer. I forget the root cause (no pun intended), possibly some bad interaction with stuff I installed outside of Homebrew. It's also possible I did something wrong, but even then there's something to be said about so many users having the same problem uniquely with Homebrew.

Lemme try to repro again when I'm home.
by NoTeslaThrow | 2025-04-22 21:22:55+00:00
Well we have plenty of records of people who did pay attention. (American?) Society as a whole isn't very good at focusing on what will care about in the future.

Anyway, it's not easy to turn oral history into speculation about the literal past. There's a reason such work focuses so much on geology! It's likely oral traditions encode a lot more literal history than we realize, but the ability to verify or interpret this as "history" in the western sense of "historiography" may be fundamentally impossible for large swathes of it.
by reneherse | 2025-04-22 21:23:31+00:00
Living in storm prone regions for most of my life has given me the same habit. All my sensitive electronics get unplugged when storms approach.

Two of my family members have had devices fried by lightning strikes over the years, and not even in regions known for the worst electrical storms.

I keep some portable battery packs handy in case I need to charge a phone, and if I'm working will switch to my laptop and tablet screens.

Of course, one can't conveniently unplug everything (HVAC, big kitchen appliances, etc.) but it's easy enough to safeguard work and lifestyle electronics.

Turning the TV off and listening to the storm is usually a nice change of pace, too.
by kopecs | 2025-04-22 21:23:37+00:00
There already was a verdict (for the Times) but it was thrown out on appeal. This is a re-do.

ETA: I looked up the docket [0] and in fact, this was the second appeal (See ECF 64). There was also an appeal on a prior MTD, hence the extreme delay.

[0]: https://www.courtlistener.com/docket/6081165/palin-v-the-new...
by namaria | 2025-04-22 21:24:19+00:00
If the effort to produce websites goes tends to zero, the value of websites will surely tend to zero. Either issues with security and maintainability will be a break on this tendency or we will get to a point where generating a custom website will be something trivial that will be done on demand.

The thing is, the cost of producing websites is already pretty low, but the value of websites mostly derives from network effects. So a rising flood of micro crud saas products will not be likely to generate much added value. And since interoperability will drive complexity, and transformer based LLMs are inherently limited at compositional tasks, any unforeseen value tapped by these extra websites will likely be offset by the maintainability and security breaks I mentioned. And because there is a delay in this signal, there is likely to be a bullwhip effect: an explosion of sites now and a burnout in a couple of years in which a lot of people will get severely turned off by the whole experience.
by Aloisius | 2025-04-22 21:24:41+00:00
> Filtered milk. Tastes the same as ordinary homogenized milk but double (or better) the shelf life.

I had to look this one up. I don't think I've ever seen filtered milk in the US, but it looks like the ones sold here are specialty high protein/low lactose milks which I can't imagine tastes the same.

They also appear to expire 9-14 days after opening which doesn't seem all that different from other milk with practice?
by parpfish | 2025-04-22 21:24:42+00:00
Maybe you could do it with math research?

First, give it the abstract for a fresh paper that it couldn’t have been trained on, then see if it can come up with the same proofs to see if it can replicate the logic knowing the conclusion.

Second, you could give it all the papers cited in the intro and ask a series of leading questions like “based on this work, what new results can you derive”?
by NoTeslaThrow | 2025-04-22 21:24:46+00:00
Back? This country was built on taking and retaining land with violence. We just call it "property" now and act like everyone thinks this is fine. we now refer to human sacrifices as "essential workers" and "terrorists". Same fundamental dynamic tho—we just replaced the gods with the economy; but either way, people gotta die to keep it running.
by MattGrommes | 2025-04-22 21:25:09+00:00
Ok, both of these comments made me doubt my memory so I just checked and my current employer, a very large consumer company, and the limits of the program are that you get a C or above, and the class is "related" to your job or any job you can get at the company. But I've gotten classes paid for that only tangentially related to my job with no problem. So I concede that you might not get a biology degree as an engineer but my particular company does a lot of different things so my guess is in practice you'd have no problems. I also worked at a now-defunct mid-size startup and a hospital system with similarly loose requirements but I don't have access to their docs anymore.
by tasuki | 2025-04-22 21:25:18+00:00
> The way I had it makes it very easy for me to read it in reverse; map, filter, map, collect

Yes, sure, who cares. But the way you wrote it, it's impossible to match those "map, filter, map, collect" to their parameters: `, |w| w.toWingding()), |w| w.alive), |w| w.id))`. Impossible!

You just include all the parameters to the various function calls on one very long line! To top it off, it's the most indented line that you decide to make the longest! If I could I'd put you in jail for this!
by johnwalkr | 2025-04-22 21:25:23+00:00
An American thermostat would be using around 920MHz which is the equivalent unlicensed band in the US. Funnily enough, the author warns Americans as an example but he is running more foul of the local rules an American would be (assuming an American would use 920MHz and not 868MHz).

In the US you are allowed to tinker around there with home-built RF devices for personal use and prototyping in unlicensed bands, to some extent. Although, using an SDR for this requires a certain interpretation of the rules. In Europe it's basically not allowed, one shall only use pre-certified modules. Or only use the device you've built in an RF anechoic chamber until you've undertaken the certification process (totally impractical for a hobbyist).
by nine_k | 2025-04-22 21:25:28+00:00
Tamagochi had a very limited way to interact with the environment, because it was an egg (tamago,卵) with a creature (chi) inside. It had needs like food and play, but could only receive care, and adjust its state a little bit.

This squid can interact with the environment in many ways, hence it can learn new stuff, and maybe do new stuff.
by thomastjeffery | 2025-04-22 21:25:32+00:00
I think the foundational problem is the privatization of information.

Information is not an object. It's not a countable good. Yet copyright demands we frame it as such.

Why? To structure our society around competition. I would much rather give collaboration a try. Unfortunately, the demand of intellectual property reframes collaboration as "derivative work", and forces us to make explicit contracts with specific people before creating it.

---- edit:

To elaborate this in the context of the article,

The very idea that we need to protect a person's right to monopolize information is ignorant of the underlying right to oblivion. Intellectual property frames the very act of existing as a person as the creation of information. That alone infringes on the right to oblivion.
by 7bit | 2025-04-22 21:25:41+00:00
> We realize this may come as a shock and disappointment to our contributors but we simply do not have the expertise or resources within the organization to continue to maintain this project.

Resources ... Okay. What truly comes as a shock is the fact that Microsoft says they don't have expertise to continue this project. I mean, who is building Windows then? It's the same building blocks, no?!
by jagged-chisel | 2025-04-22 21:25:45+00:00
They are not selling the hardware. They are taking your money in exchange for the privilege of utilizing the hardware they created to use a service they built so their customers can obtain data about your usage of the device and whatever it connects to.

I fear that small companies without VC are the only ones interested in making a thing they sell that doesn’t require some ongoing commitment from buyers. And those companies run the risk of folding overnight. It’s fine for the customers since their hardware continues to function, but it’s not an attractive business model.
by nartho | 2025-04-22 21:25:55+00:00
You're right, I was ignorant about the Nibelungen and assumed it was about Achilles. If you had just provided the information to me instead of resorting to name calling (twice now) without adding anything to the conversation (comments on HN are supposed to add something to the conversation, I imagine you know that), I would have learned something new, we both would have wasted less time by arguing back and forth, and I would have thought "Hey what a nice and educated guy this Beijinger is" instead of "I guess he is one of those people who have to put other people down in order to feel better about themselves". But you do you.
by TheOtherHobbes | 2025-04-22 21:26:05+00:00
There's no coordinated strategy anywhere.

The people with real power voted him in to lower taxes, and in return he is nuking the economy, neutering the Pentagon, trashing public health, isolating the US from tourists and skilled immigrants, walking public education back at least a century, killing the next generation of R&D, murdering rational climate policy with a coal-powered chainsaw, and handing government software and data to enemy powers.

The entire machinery of governance - official and public, as well as off-the-record "do as I say and I'll fund your campaign" private - has folded like tissue paper.

The only pushback is from the judiciary, and their orders are as likely to be ignored as implemented. And parts of academia.

It's astonishing.
by ray__ | 2025-04-22 21:26:12+00:00
Fun questions! My takes:

1) Sadly there isn't really. There are a few good blogs like Derek Lowe's "In the Pipeline" that centralize news, but no anonymous online forum like this.

2) Google scholar alerts, Twitter, Bluesky, and word of mouth.

3) I think our understanding of biological processes at the mesoscale is about to hit an inflection point, largely through advances in electron microscopy (cryo-ET) and the ability to perform simulations at this scale.

4) Not harder but definitely more messy and progress is less linear.
by sargstuff | 2025-04-22 21:27:23+00:00
Ah, no concept of time in Lewis Carrol universe, so find and eliminate expressions to make things 'N' before the maxwell equations[1][2][3] differentiate to much / become "NP" and crash the multi-universe.

------------

[1] : https://en.wikipedia.org/wiki/Maxwell%27s_demon

[2] : https://en.wikipedia.org/wiki/Maxwell%27s_equations

[3] : https://en.wikipedia.org/wiki/P_versus_NP_problem