MurphyA/hacker-news huggingface dataset:
by mort96 | 2026-01-22 19:07:15+00:00
It seems like zstd is somehow compressing really badly when many zstd processes are run in parallel, but works as expected when run sequentially: https://news.ycombinator.com/item?id=46723158

Regardless, this does not make a significant difference. I ran hyperfine again against a 37M folder of .pdf.zst files, and the results are virtually identical for zstd and gzip:

+-------+-------+--------+-------+
| gzip | zstd | brotli | xz |
+-------+-------+--------+-------+
| 142ms | 165ms | 269ms | 994ms |
+-------+-------+--------+-------+

Raw hyperfine output:

~/tmp/pdfbench $ du -h zst2 gz xz br
37M zst2
38M gz
38M xz
37M br

~/tmp/pdfbench $ hyperfine ...
Benchmark 1: for x in zst2/*; do zstd -d >/dev/null <"$x"; done
Time (mean ± σ): 164.5 ms ± 2.3 ms [User: 83.5 ms, System: 72.3 ms]
Range (min … max): 162.3 ms … 172.3 ms 17 runs

Benchmark 2: for x in gz/*; do gzip -d >/dev/null <"$x"; done
Time (mean ± σ): 142.2 ms ± 0.9 ms [User: 87.4 ms, System: 43.1 ms]
Range (min … max): 140.8 ms … 143.9 ms 20 runs

Benchmark 3: for x in xz/*; do xz -d >/dev/null <"$x"; done
Time (mean ± σ): 993.9 ms ± 9.2 ms [User: 896.7 ms, System: 99.1 ms]
Range (min … max): 981.4 ms … 1007.2 ms 10 runs

Benchmark 4: for x in br/*; do brotli -d >/dev/null <"$x"; done
Time (mean ± σ): 269.1 ms ± 8.8 ms [User: 176.6 ms, System: 75.8 ms]
Range (min … max): 261.8 ms … 287.6 ms 10 runs
by inopinatus | 2026-01-22 19:07:16+00:00
You were not the root cause of that outage.

> IT were not stupid

Everything else you described points to them being blundering morons. From an email forwarder that didn’t build loop detection into its header prepending, fucking up a restore, and then malware’ing the student that exposed them into kafkaesque technology remand, all I’m taking away here is third-degree weaponised incompetence
by godelski | 2026-01-22 19:07:26+00:00
Funding is definitely a problem, but frankly reproduction is common. If you build off someone else's work (as is the norm) you need to reproduce first.

But without repetition being impactful to your career and the pressure to quickly and constantly push new work, a failure to reproduce is generally considered a reason to move on and tackle a different domain. It takes longer to trace the failure and the bar is higher to counter an existing work. It's much more likely you've made a subtle mistake. It's much more likely the other work had a subtle success. It's much more likely the other work simply wasn't written such that a work could be sufficiently reproduced.

I speak from experience too. I still remember in grad school I was failing to reproduce a work that was the main competitor to the work I had done (I needed to create comparisons). I emailed the author and got no response. Luckily my advisor knew the author's advisor and we got a meeting set up and I got the code. It didn't do what was claimed in the paper and the code structure wasn't what was described either. The result? My work didn't get published and we moved on. The other work was from a top 10 school and the choice was to burn a bridge and put a black mark on my reputation (from someone with far more merit and prestige) or move on.

That type of thing won't change in a reproduction system but needs an open system and open reproduction system as well. Mistakes are common and we shouldn't punish them. The only way to solve these issues is openness
by pbasista | 2026-01-22 19:07:36+00:00
Those numbers are meaningless unless you specify what you get in return.

It is like saying that you pay $30,000 for a car. But the most important question is: For which car?

Also, if the installation services are so expensive, you can always install everything yourself.

Study how to do it, get the tools and materials, and then do it. It would be time-consuming, challenging and perhaps it would carry extra risks. Absolutely.

But it is not rocket science. It can be done. As long as there is a motivation to do it, i.e. a good value you will get out of it in return, it should be a valid approach to consider, in my opinion.
by daveguy | 2026-01-22 19:07:55+00:00
Republicans are always trying to increase and protect oil subsidizes, cheer drill baby drill, and have social media tools peddling their bs who are funded by foreign influence campaigns. The Democats may not be perfect, but they are much more likely to cut oil subsidies or at least subsidize solar and other renewables to balance. This is especially true as the next generation takes over by primarying the tepid fossils currently in office. Meanwhile, dear leader Dumpty likes to suggest windmills cause cancer because he doesn't like what they look like on the horizon of his golf dumps. Saying this is a "both sides" problem is laughable.
by victorbjorklund | 2026-01-22 19:07:58+00:00
> The only reason the IMF hasn't taken the reins of France yet is because France has the nuclear weapon and is the only country in the eurozone to have it, so they have some leverage with the other countries in the eurozone.

First of all IMF has nothing to do with the Eurozone. And second of all, we are Europeans. We don’t threaten to bomb our neighbors if they don’t give us what we want. That’s just a Russian/American thing.
by 10729287 | 2026-01-22 19:08:12+00:00
I come from punkrock and buying cheap compilations released by labels like it was the norm back in the days (Get Punk O Rama 2 by Epitaph records now if you’re curious) really made the music lover I am today, spinning them like crazy because that was almost the only music I had access to.

I dreamt so much about unlimited streaming access back then. 30 years later I realize it was a chimera that broke a lot of my will to dig into music and spend time on more rewarding albums. I’m just sick of the non stop novelty.

MP3 era was the best for this. Because a lot of music was then easy to get, and communities on the Internet were great. Streaming is just mass consumption, or I just don’t have the discipline to restrain.

Well, I’m now back to my very own library, supporting bands by buying their stuff on bandcamp or spending time on random old records I slept too much time on.
by mindslight | 2026-01-22 19:08:28+00:00
From the patterns I've seen there are roughly two buckets of people. Some get ahead of this when they're still quite young, embrace "retirement" and "downsizing", etc. This makes the switching costs worth it - they're paying to start a new deliberate life.

And others just want to stay in their same home they know and love even as they slow down and do less and less. I myself feel I'm in this latter camp, and I don't know what I'd do differently to change that.

I wrote another comment elsewhere in this topic about the lack of multi-generation households, and I feel that is directly part of the problem here too. Of course it's a very tall ask to expect your kid(s) to stick around in the American individualist culture. Although the economics of this might change with where we're headed...
by mike_hearn | 2026-01-22 19:08:42+00:00
Thanks for reading it, or scan reading it maybe. Of the 18 papers discussed in the essay here's what they're about in order:

- Alzheimers

- Cancer

- Alzheimers

- Skin lesions (first paper discussed in the linked blog post)

- Epidemiology (COVID)

- Epidemiology (COVID, foot and mouth disease, Zika)

- Misinformation/bot studies

- More misinformation/bot studies

- Archaeology/history

- PCR testing (in general, discussion opens with testing of whooping cough)

- Psychology, twice (assuming you count "men would like to be more muscular" as a psych claim)

- Misinformation studies

- COVID (the highlighted errors in the paper are objective, not subjective)

- COVID (the highlighted errors are software bugs, i.e. objective)

- COVID (a fake replication report that didn't successfully replicate anything)

- Public health (from 2010)

- Social science

Your summary of this as being about a "valid and common but subjective political dispute" I don't agree is accurate. There's no politics involved in any of these discussions or problems, just bad science.

Immunology has the same issues as most other medical fields. Sure, there's also fraud that requires genuinely deep expertise to find, but there's plenty that doesn't. Here's a random immunology paper from a few days ago identified as having image duplications, Photoshopping of western blots, numerous irrelevant citations and weird sentence breaks all suggestive that the paper might have been entirely faked or at least partly generated by AI: https://pubpeer.com/publications/FE6C57F66429DE2A9B88FD245DD...

The authors reply, claiming the problems are just rank incompetence, and each time someone finds yet another problem with the paper leading to yet another apology and proclamation of incompetence. It's just another day on PubPeer, nothing special about this paper. I plucked it off the front page. Zero wet lab experience is needed to understand why the exact same image being presented as two different things in two different papers is a problem.

And as for other fields, they're often extremely shallow. I actually am an expert in bot detection but that doesn't help at all in detecting validity errors in social science papers, because they do things like define a bot as anyone who tweets five times after midnight from a smartphone. A 10 year old could notice that this isn't true.
by uxcolumbo | 2026-01-22 19:08:53+00:00
Not sure what your definition of 'Design Thinking' is.

Design Thinking isn't about people thinking "that they can just come in and design user interfaces, etc. without really having an expertise in the particular field."

It's a problem solving approach using UCD methods amongst others and working with experts in the field to come up with solutions and ideas to a given problem space.

Key thing is you work with the people who are experts in the field, for example working with medical experts to design a new health related application etc.
by magicalhippo | 2026-01-22 19:08:54+00:00
The HF demo space was overloaded, but I got the demo working locally easily enough. The voice cloning of the 1.7B model captures the tone of the speaker very well, but I found it failed at reproducing the variation in intonation, so it sounds like a monotonous reading of a boring text.

I presume this is due to using the base model, and not the one tuned for more expressiveness.

edit: Or more likely, the demo not exposing the expressiveness controls.

The 1.7B model was much better at ignoring slight background noise in the reference audio compared to the 0.6B model though. The 0.6B would inject some of that into the generated audio, whereas the 1.7B model would not.

Also, without FlashAttention it was dog slow on my 5090, running at 0.3X realtime with just 30% GPU usage. Though I guess that's to be expected. No significant difference in generation speed between the two models.

Overall though, I'm quite impressed. I haven't checked out all the recent TTS models, but a fair number, and this one is certainly one of the better ones in terms of voice cloning quality I've heard.
by 10000truths | 2026-01-22 19:10:46+00:00
I think it's natural to be uncomfortable with black-box optimizations with unclear boundary conditions, especially when there's no escape hatch or override. Postgres's query planner is notorious for this - some small change in a table's computed statistic shifts the cost estimate ever so slightly to favor some other "optimization" that actually ends up performing significantly worse. It happens rarely, but no one wants to "rarely" be paged at 3 AM on a Saturday.

Optimize for the p99/p99.9/worst case scenarios. Minimize unpredictability in performance where possible, even if it comes at a small cost of median/average performance. Your SREs will thank you.
by f1shy | 2026-01-22 19:10:51+00:00
Sorry, the standards imposed to meat from south america is way higher than the European. Wine is nearly impossible. And spicies like paprika also require higher standards than in Europe. I don’t know what you mean with that. Free trade does not mean there are no standards to met. Same standards will be imposed (as are already imposed) to anything imported in the EU. Also in south America, because of size you don’t need nearly as much chemicals as in EU anyway.

Last but not least, that of quality standards and chemicals doesn’t hold anyway, as there are already loads of products coming from those countries already… I look always where things come from, and fruits come up to 80% from South America (including Mercosur). Dang even apples from Argentina in Germany, which is frankly non sense to me! It’s just not about quality, is good all protectionism and imposing tariffs, just as Trump is doing, but if we do, is ok.

Yes, I agree with the standards, but has absolutely nothing to do whatsoever with the agreement Mercosur/EU. The standard will be imposed for ANY product sold in the EU, doesn't matter where it comes from, as it should be.
by anigbrowl | 2026-01-22 19:10:53+00:00
Your previous statement was flagged so I couldn't reply, but I wanted to pick up on this one thing you said:

MAGA is a necessary response to the AOC/Mamdani left

But MAGA officially began in 2015, when Trump announced the launch of his Presidential campaign with a speech including a string of vicious remarks against immigrants. AOC was elected in 2019 and Mamdani (to the NY state assembly) in 2021. MAGA has been a populist movement from the outset, seemingly motivated by conservative dislike of the Obama administration (anything but populist) and the prospect of a Hillary Clinton administration (likewise), as well as an atavistic dislike of immigrants.

The devastation from AOC/Mamdani politics is far worse than right-wing terrorism.

The populist politics of AOC and Mamdani seem motivated by exasperation with the prejudice, corruption, and general lawlessness of the Trump administration and the larger MAGA movement, and I doubt they would have enjoyed their electoral success if the Republican party had selected a staid institutionalist over Trump in 2016.

It's interesting to hear context about the economic history of Bangladesh, but I don't think comparing the grinding poverty of cold-war era Bangladesh with the economic and strategic hegemony of early 21st century USA is even slightly illuminating or useful.
by felarof | 2026-01-22 19:10:58+00:00
Good question. We think the browser is becoming the new OS. It doesn’t really matter anymore if you’re on Windows, macOS, or Linux—the browser is where most work already happens.

We see a future where it’s the main gateway to everything, and where agents live and work alongside you inside the browser. That’s why we call it BrowserOS. :)
by rtkwe | 2026-01-22 19:11:57+00:00
That is a factor most people miss when thinking about the replication crisis. For the harder physical sciences a wrong paper will fairly quickly be found because as people go to expand on the ideas/use that data and get results that don't match the model informed by paper X they're going to eventually figure out that X is wrong. There might be issues with getting incentives to write and publish that negative result but each paper where the results of a previous paper are actually used in the new paper is a form of replication.
by doctorpangloss | 2026-01-22 19:12:15+00:00
i confess, i've read both of your comments on this - your analogy and a deeper explanation of the analogy - and i still have no idea what you are saying. i'm not stupid. so first, my feedback here is, it sounds like you are an educator or in an education-adjacent role, and you should focus on making more sense haha. like lay out your beefs clearly, it sounds like you have a beef with interdisciplinary work, specifically between some STEM departments and especially with humanities and STEM departments, which is subjective. you don't have to be objective about everything. you can just say, "i don't like this design thinking thing because i don't like the people involved" or whatever. but i don't know! i cannot figure out what you are saying.

it sounds like your point is: "some ways of solving problems are superior to others." i've heard this take a million times. one perspective i'll offer to you is, math is not the only way to solve problems. it's not even the best way in many cases. not everything can be solved by defining a narrow goal, and then having a dispute about the methods, and then picking some objective method and then applying it very optimally, or whatever. this is also on you, as an educator, to understand! i could give a bajillion specific examples.

but first, you have to concede: an analogy nobody understands is bad, and you have to own that, and two, it's not really clear, what exactly is your dispute with Design Thinking? it doesn't have anything to do with user interfaces... so why the hell are you talking about it? why "Design Thinking people"? What is your beef here?
by falloutx | 2026-01-22 19:12:15+00:00
> People will forget what it was like to have authentic content.

This is already true. Most people on the internet are stuck in tiktok loop anyway. Even chatgpt would struggle to get them. By the time LLMs get good enough, a lot of people who were willing to pay for them would have lost their jobs and they would not be paying for something that costed them their job, many will quit the internet fully. We are looking at value being driven out of internet at a grand scale.
by giancarlostoro | 2026-01-22 19:12:36+00:00
I don't fully know, however, I will note that to my understanding, since Moses wrote the fist five books (Genesis, Exodus, Leviticus, Numbers, and Deuteronomy) if you will note... Moses came long after these events. I don't know what his source material was, but if it was word of mouth or other scrolls, its possible people in his time had access to those other scrolls which are now lost to time. If he felt that anyone could read the other scrolls for more information I could understand why there's not more information about these people.

I have no idea why some people do what they do. I will say I am very jealous of the Amish because they don't have the stresses I have, or half of the issues I have. No money for gas? I don't think they need to worry or care about it.

The other thing is, what does it really mean that he made a city? It could mean that he started an encampment elsewhere. we don't know how many other people God would have made during Adam / Cains time, I would imagine God would have made Cain a wife at some point.
by boznz | 2026-01-22 19:13:39+00:00
>it'll be friendlier for the average person than Linux itself.

I think the myth that Windows is easier needs to die. The builds targeted at Windows users are very easy to use; You would likely go into the Command Prompt as much as you would with Windows, and the "average person" spends more time on their non-windows phone than they do in Windows.

I am a 30+ years Windows developer, who thought he would never move, but who migrated literally a week ago, the migration was surprisingly painless and the new system feels much more friendly, and surprisingly, more stable. I wrote it up on my blog, and was going to follow it up with another post about all the annoyances in my first full week, but they were so petty I didn't bother.
by ianferrel | 2026-01-22 19:13:55+00:00
>Because agentic AI can parse unstructured data and make purchasing decisions regardless of whether your site allows it, which avoids the chicken-and-egg problem.

People can do that too, and also benefit from actual structured data. But the avoidance of the chicken and egg problem didn't seem to result in widespread structured data stores beating out the SEO-spam-style listings.
by isr | 2026-01-22 19:14:32+00:00
No, it isn't a wild take. Israel's DIRECT involvement has been boasted about by former CIA directors on X, and WIDELY REPORTED in Israeli media (where it is presented as an obvious thing).

Plus, it has been exposed for what it is by many independent journalists in the West.

These attempts at creating a "fake reality" don't work well any longer in their targeted environments. Witness the MULTI-MILLION marches last week oppos8ng all this. Not to mention, as soon as Starlink was bl9cked, almost by magic, the burning, looting & shootings stopped.

Funnily enough, it seems like the only place where these "fake reality" narratives are holding ... is in the West itself. Where commentators try to pretend that the aforementioned MULTI-MILLION marches didn't happen (despite the fact that multiple non-Iranian news sources, from all Jazeera all the way to CGTN, had massive coverage of them - some even had a rolling 24hr style on-the-ground roving coverage).

However, it seems like the VAST majority of the Iranian people (real Iranians, who live in Iran, not "Persians from LA", who haven't a clue) have burned through the fakery. Including recognising outlets like "Iran International" and even BBC Persian for being the clearing houses for CIA/Mossad propaganda.
by jimbokun | 2026-01-22 19:15:14+00:00
Slough House denizens screw up in blatant, over the top ways. While the Park screw up in ways that leave geopolitical consequences festering for years or decades while being good at covering their own asses.

The plot is generally some evil, corrupt actions the Park took in the past are coming home to roost and only the bumbling losers in Slough House can fix it (kind of, eventually, in a "at least London wasn't blown off the face of the earth" kind of way).
by biophysboy | 2026-01-22 19:15:16+00:00
> look at any HN thread on the topic of fraud in science.

HN is very tedious/lazy when it comes to science criticism -- very much agree with you on this.

My only point is replication is necessary to establish validity, even if it is not sufficient. Whether it gives a scientist a false sense of security doesn't change the math of sampling.

I also agree with you on quality issues. I think alternative investment strategies (other than project grants) would be a useful step for reducing perverse incentives, for example. But there's a lot of things science could do.
by soiltype | 2026-01-22 19:15:38+00:00
But that seems almost trivially solved. In software it's common to value independent verification - e.g. code review. Someone who is only focused on writing new code instead of careful testing, refactoring, or peer review is widely viewed as a shitty developer by their peers. Of course there's management to consider and that's where incentives are skewed, but we're talking about a different structure. Why wouldn't the following work?

A single university or even department could make this change - reproduction is the important work, reproduction is what earns a PhD. Or require some split, 20-50% novel work maybe is also expected. Now the incentives are changed. Potentially, this university develops a reputation for reliable research. Others may follow suit.

Presumably, there's a step in this process where money incentivizes the opposite of my suggestion, and I'm not familiar with the process to know which.

Is it the university itself which will be starved of resources if it's not pumping out novel (yet unreproducible) research?
by torginus | 2026-01-22 19:15:46+00:00
Yeah politics or not, the US stock market has a very high exposure to just a couple tech companies, and many of these companies have a very high P/E, and likewise hugely invested in AI (which itself is a risk). Add to that the recent (entirely self-inflicted) geopolitical questions of US reliability, I think it's a smart idea to reduce US exposure in one's portfolio.

Circling back to AI, my (not politically motivated) opinion, is that most of the tremendous supposed value was priced in into AI stock back in 2024, with 2025 gains being either relatively modest or stagnant. With the risks involved, I think it's fair to expect that AI companies can go down a lot, but it's hard to imagine them going up by that much.

Like, for example if NVIDIA gained another $1T in market cap, that'd increase the stock price by 22%, but if they lost that much, it would make it go down by 36%. If we consider both outcomes equally likely (not suggesting this is a reasonable assumption), we're more likely to lose money.