MurphyA/hacker-news huggingface dataset:
by zahlman | 2025-05-30 23:41:15+00:00
> When LLMs came out, and they now produce text that sounded like it was written by a native speaker (in major languages).

But they still often get things completely wrong, especially in high-context languages such as Japanese. There often isn't a way to convey the necessary context in text. For example, a Japanese live-streamer who says "配信来てくれてありがとう" means "Thank you for coming to (watch) the stream", but DeepL will give "Thanks for coming to the delivery." - because 配信 actually does mean "delivery" in ordinary circumstances, and that's just the word they idiomatically use to refer to a stream. No matter how much you add from the transcript before or after that, it doesn't communicate the essential fact that the text is a transcript of a livestream.

(And going the other way around, DeepL will insert a に which is grammatically correct but rarely actually uttered by livestreamers who are speaking informally and colloquially; and if you put "stream" in the English, it will be rendered as a loanword ストリーム which is just not what they actually say. Although I guess it should get credit for realizing that you don't mean a small river, which would be 小川.)

(Also, DeepL comes up with complete incomprehensible nonsense for 歌枠, where even basic dictionaries like Jisho will get it right.)

More obviously, they get pronouns and roles wrong all the time - they can't reliably tell whether someone is saying that "I did X" or "you did X" because that may depend on facts of the natural world outside of the actual text (which wouldn't include more information than the equivalent of "did X"). A human translator for, say, a video game cutscene may be able to fix these mistakes by observing what happened in the cutscene. The LLM cannot; no matter how good its model of how Japanese is spoken, it lacks this input channel.
by fjasdfwa | 2025-05-30 23:41:46+00:00
> Virtual Threads Ate My Memory

From what I see, It's a developer trade off.

If you want to save on memory then you need to drop down to callbacks, promise chains, or async-await sugar that is a compiler transform to a state machine.

But if you do that, then you will write a blog complaining about the boilerplate!

I think Zig had an ability to kind of give you the best of both worlds. A zig expert would need to correct me on this.
by tialaramex | 2025-05-30 23:42:01+00:00
"Data race which doesn't seem to modify the data" is actually often not a false positive, they're called "Benign data races" and well, go read the literature, it's often difficult to be sure they're truly "benign" and if there's a subtle reason why they aren't that's now a crazy hard to spot bug in your program.

So yeah, just fix 'em. Most of them you dodged a bullet, and even when you didn't it will now be easier for the next person to reason about the software.
by baby_souffle | 2025-05-30 23:42:03+00:00
The question isn't IF but WHEN.

I suspect spinning rust is still going to be critical to long-term and small/medium scale archive applications for at least another 5 years. It may not be getting significantly denser but it's a well understood, reliable-ish and affordable technology.

I don't expect to have an "all flash" NAS at home (~100TB) for another decade :(.
by MelancholyMiser | 2025-05-30 23:42:07+00:00
signCount would need to be synchronised as well as the passkey by whatever method you elect to use. If your synchronisation method has been persistently compromised you're hosed, but if the passkey is cloned as a one off, the server would continue to increment signCount everytime either copy is used, while the passkey in your possession would only increment when used by you ie. half as often. You'd run into trouble if the sync service can't tally multiple device uses in quick succession, which is the likely reason for the article - if I use my synced passkey on three separate devices in a few minutes, all three copies would have the same signCount, but it would be lower than the server's signCount. Either you'd have to prompt the user everytime this happens or record and sync a lot more information about every passkey use and let the sync service count them.
by xnyan | 2025-05-30 23:42:54+00:00
The most established option is Ceph, which has a (optional) web ui too. It's what I use and I'm happy with it.

Before investigating it for eventual production use, I heard quite a bit about how complicated it was to use, but for my use case (storage for enterprise and academic k8s clusters) it's actually been quite simple to deploy and use. cephadm (one of many ceph management tools) can handle nearly all our bootstrapping and management needs. Little to no tweaking or configuration needed. Fairly low overhead. Very reliable and resilient to adverse conditions. Easy to handle different storage types and data retention needs.

One thing I will warn you, if you go the ceph docs site right now and just start browsing, it is in fact quite overwhelming because ceph has the capacity to handle a ton of edge cases and unusual environments. I'd recommend taking 15 minutes and build this [1], which gives you a fully functional toy-sized ceph cluster on a single node. Then, hit the docs to fill in the gaps of what you need to know for your deployment.

1: https://hackmd.io/@yujungcheng/B1lCV3UH9
by koito17 | 2025-05-30 23:43:47+00:00
> Without any back-pressure, the program kept stuffing memory with pending results.

The key phrase in the whole article.

This is the reason I am not fully confident in the "pervasive async" mindset overtaking web development. In JavaScript, pervasive async makes sense, because all functions eventually return control to a central event loop anyway. In environments with real threading, the move from OS threads to "green threads and async everywhere" implicitly gives up the tried-and-tested scheduler of your OS and its mechanisms to handle back-pressure. Developers must now take full responsibility for avoiding buffer bloat.

> they fundamentally change how we think about concurrency limits and resource management.

Virtual threads make concurrency cheaper, but nothing about them (or any other concurrency abstraction) eliminates the need for flow control. By switching to virtual threads, you traded off the safeguards once provided by your (bounded!) thread pool and OS scheduler. It's now your responsibility to put safeguards back in.

As an aside: in Clojure's core.async, channel limits (like 1024 pending puts) are intentionally there as a warning sign that your async pipeline isn't handling back-pressure correctly. This is exactly the "hyperactive downloader with no brakes" problem described in the article. With core.async, you would have eventually hit an exception related to excessive puts on a channel, rather than silently expanding buffers until resource exhaustion.
by jmyeet | 2025-05-30 23:43:50+00:00
This is basically what happened after 2008. The entry level jobs college grads did basically disappeared and didn't really come back for many years. So we kind of lost half a generation. Those who missed out are the ones who weren't able to buy a house or start a family and are now in their 40s, destined to be permanent renters who can never retire.

The same thing will happen to Gen Z because of AI.

In both cases, the net effect of this (and the desired outcome) is to suppress wages. Not only of entry-level job but every job. The tech sector is going to spend the next decade clawing back the high costs of tech people from the last 15-20 years.

The hubris here is that we've had a unprecedented boom such that many in the workforce have never experienced a recession, what I'd call "children of summer" (to borrow a George RR Martin'ism). People have fallen into the trap of the myth of meritocracy. Too many people thing that those who are living paycheck to paycheck (or are outright unhoused) are somehow at fault when spiralling housing costs, limited opportunities and stagnant real wages are pretty much responsible for everything.

All of this is a giant wealth transfer to the richest 0.01% who are already insanely wealthy. I'm convinced we're beyond the point where we can solve the problems of runaway capitalism with electoral politics. This only ends in tyranny of a permanent underclass or revolution.
by kelseyfrog | 2025-05-30 23:46:00+00:00
The point is that I sometimes wonder, if I were born in ancient Rome, wouldn't I have also felt, just as deeply, that defending family honor with violence was righteous? Not in a barbaric way, just in a 'this is obviously the right thing to do' kind of way?

It's not that there's no such thing as right and wrong, it's that what counts as 'moral clarity' seems to shift, and everyone thinks they're the ones seeing it clearly. That includes people we now think were totally wrong. What do we do with that?

Imagine someone 500 years from now looking back at us with horror and clarity, seeing our blind spots as clearly as we see slavery or witch-burning. Don't you ever worry, like me, that you might be the one taking a stand for what will one day be unthinkable? Not because we didn't believe that right and wrong didn't exist, but because we were so certain that we had it correct?
by pron | 2025-05-30 23:47:26+00:00
> By switching to virtual threads, you traded off the safeguards once provided by your (bounded!) thread pool and OS scheduler.

The OS scheduler provides no safeguards here (you can use the thread-per-task executor with OS threads - you'll just get an OOME much sooner), and as the virtual thread adoption guide says, you should replace bounded thread pools with a semaphore when needed because it's just a better construct. It allows you to limit concurrency for different resources in an easy and efficient way even when different threads make use of different resources.

In this case, by switching to virtual threads you get to use more appropriate concurrency constructs without needing to manage threads directly, and without sharing them among tasks.
by jwrallie | 2025-05-30 23:48:01+00:00
If I understood correctly, the ECP5 FPGA can be designed for with open source tooling [0][1], which makes this even more awesome.

OP, if you are planning to commercialize these, try to confirm compatibility, that will definitely make it more attractive!

[0] https://hackernoon.com/getting-started-using-open-source-fpg...

[1] https://github.com/YosysHQ/prjtrellis
by trhway | 2025-05-30 23:48:46+00:00
>but that the rate of expansion itself is increasing over time.

Take for example galaxies running from us at "c". They happen to be exactly at 13.7B light years - coincidence? I don't think it is a coincidence, and some simple logic leads to conclusion that the same galaxies will be running from us at the same "c" say 300M years later - ie. then it will be "c" at 14B light years distance from us - which means that the Hubble constant is actually decreasing.
by Spooky23 | 2025-05-30 23:49:30+00:00
In context of tech, H1B is great for the money people in the US and India. It suppresses wages in both countries and is a powerful plum for employee “loyalty”. There’s a whole industry of companies stoking the pipeline of cheap labor and corrupting the hiring process.

In big dollar markets, the program is used more for special skills. But when a big bank or government contractor needs marginally skilled people onshore, they open an office in Nowhere, Arizona, and have a hard time finding J2EE developers. So some company from New Jersey will appear and provide a steady stream of workers making $25/hr.

The calculus is that more H1=less offshore.

The smart move would be to just let skilled workers from India, China, etc with a visa that doesn’t tie them to an employer. That would end the abusive labor practices and probably reduce the number of lower end workers or the incentive to deny entry level employment to US nationals.
by jonahx | 2025-05-30 23:50:58+00:00
> The intuition that such a thing is "too powerful" is actually a pretty bad intuition here.

I still disagree. Just focus on theorem proving and not the examples that are too simple. If the halting problem could be solved, we'd be able to magically solve all these "impossible" problems. But our intuition is that just doesn't make sense, it's "too good to be true", "the universe is just more complex than that", etc. This intuition isn't a proof, but since we do have a proof, it makes sense as an intuition. That's all that's being said.

Scott Aaronson has a similar sentiment about P =/= NP:

"If P=NP, then the world would be a profoundly different place than we usually assume it to be. There would be no special value in “creative leaps,” no fundamental gap between solving a problem and recognizing the solution once it's found. Everyone who could appreciate a symphony would be Mozart; everyone who could follow a step-by-step argument would be Gauss; everyone who could recognize a good investment strategy would be Warren Buffett."

Certainly not a proof -- and ofc we have none in this case -- but still a powerful intuition.
by techpineapple | 2025-05-30 23:51:08+00:00
This is a fragile point I’m going to make, and I’m sure somebody has made it before me, but the weird thing about knowledge or philosophy or whatever, and I real plenty of history or philosophy, but you can basically get it to say anything you want it to say.

Maybe it’s sort of like, you can’t become wise without reading history, but you can read history without becoming wise.

And it’s kind of weird to me that someone can read hundreds of thousands of pages of history and write thousdans of pages of blog posts, that basically boils down to Anakin sky walker asking, “but, what if the dictator was good”
by evanelias | 2025-05-30 23:51:14+00:00
> I do a lot of personal knowledge management and I use a shit ton of sections and lists in that.

That's because these are notes, not a book. A list-heavy outline format makes sense for notes, as these are summaries that supplement your own memory and knowledge you've already taken in. They're not a sole/primary source of conveying knowledge to others on their own.

> Perhaps we're just way too used for books etc. to an approach that is suboptimal.

If you truly believe books are "suboptimal", I can only suggest that you consider looking inward and do some reflection:

Is the "problem" really with books and long-form writing, which is the dominant form of knowledge transfer across several thousand years of human civilization?

Or is the problem with people's attention spans in the past decade, due to dopamine-fueling social media doom scrolling and AI usage?
by antithesizer | 2025-05-30 23:52:05+00:00
It's certainly possible that the author believes their blog post is going to spark a movement or contribute in some way, as a sort of political act, to society's rejection of AI and therefore the ultimate salvation of their career as an AI-illiterate "creative". But giving them the benefit of the doubt for a moment, it might be more likely that they simply are aware that many such people exist already and are apt to click on a post such as this in order to have their opinions repeated back to them cathartically. The idea that a growing mass of social media posts can go critical and blast a hole in reality is, on the other hand, extremely popular these days.
by LexiMax | 2025-05-30 23:52:05+00:00
> I do think we do need to solve the male part of the equation after spending the last 50 years raising women up.

When left-wing progressives talk about dismantling...let's just call it the "p-word", dismantling the expectations about what a man should and shouldn't be is very much part of that.

The system victimizes men as much as women, and we should be tearing that system down, not leaning back into it in a misguided attempt of trying to rebalance the scales.
by trade2play | 2025-05-30 23:52:57+00:00
I joined in 2018.

Honestly it was 10 years too late. The big innovations of the 2010 era were maturing. I’ve spent my career maintaining and tweaking those, which does next to zero for your career development. It’s boring and bloated. On the bright side I’ve made a lot of money and have no issues getting jobs so far.
by tialaramex | 2025-05-30 23:53:20+00:00
C++ auto isn't type inference, as so often in C++ it's something weirder which they call type deduction.

In languages with type inference when it could be A or B and it's left to infer the type the compiler just always says it could be A or B, so the programmer needs to disambiguate.

But with deduction in some cases C++ will deduce that you meant A even though you could instead have explicitly chosen B here instead, and too bad if you did mean B because that's not what it deduced.
by dminvs | 2025-05-30 23:54:44+00:00
I always liked going to well-stocked stores and browsing for stuff. That was Fry's until they shot themselves in the foot.

Micro Center might not be optimal on price, but sometimes you just want to wander a store full of cool stuff and maybe walk out with something you didn't expect, instead of another anonymous box of schmutz from Amazon or wherever
by gruez | 2025-05-30 23:54:45+00:00
What are your thoughts on trickle down economics, then? Everything you said above seems suspiciously similar to the reasoning use by advocates of lower taxes to "grow the pie". After all, it's not like billionaires use that much roads, schools, and defense compared to the typical American, definitely not as much in relation to how much more income/wealth they have. Maybe we should give them a "bulk discount" on taxes, in the hopes that it increases the tax take overall? You said something earlier about how taxes totally can't be compared to interchange rates, but your exact reasoning is unclear.

Is this just word lawyering over what "subsidize" means? I can understand how strictly speaking, rich people getting more cashback (or lower taxes) isn't exactly a "subsidy", because they're still paying more than "their share" into the system, but most people would still think the arrangement is unfair. Call it "regressive" or whatever, but I still think it's a valid complaint.
by CSMastermind | 2025-05-30 23:55:17+00:00
Huge amounts of white collar jobs have been automated since the advent of computers. If you look at the work performed by office workers in the 1960s and compared it to what people today do it'd be almost unrecognizable.

They spent huge amounts of time on things that software either does automatically or makes 1,000x faster. But by and large that actually created more white collar jobs because those capabilities meant more was getting done which meant new tasks needed to be performed.
by carpo | 2025-05-30 23:55:25+00:00
Yeah, it's probably too expensive and complicated to send all the frames to an LLM. I only send about 30 images at 576x324 which is around 15000 tokens a video, and comes to about $0.045 per video. First I save one frame every second and then loop through them comparing each to find the differences, and send just screenshots which have changed significantly, up to a max of 30. Claude only allows 100 images per API call, so it would be a bit fiddly and costly to handle 7000 frames.

Though, now that I'm thinking about it, you could probably do this locally and just look at the part of the image that has the current score, do some local OCR on it to check if the score has changed each frame, if it has, store the timestamp and then use ffmpeg to extract the correct parts. Probably wouldn't need an LLM at all.

As for editing, one thing I do in my videos is audio keywords so my app can do specific things. For example, I can say "AI, mark what I just said as important." Then when it transcribes the audio and the LLM processes it, it will mark that part as a Distinct Moment with a start and end timestamp, a title and description that will show in my app as a clickable link to that part of the video. I'm thinking of adding more commands for more complex editing too.
by MegaDeKay | 2025-05-30 23:55:29+00:00
I checked out his post on Reddit [0]. OP (cyao12) wrote a CPU in Verilog at the age of 13 and is now only 16. Mind. Blown.

cyao12: I'm going to try and put the old cpu I made in verilog when I was 13 on it! The sdram is okay, the traces are short enough that the distance difference doesn't matter :D Collez_boi: You made a freaking CPU in Verilog when you were 13?! That's crazy. cyao12: Yeaaah, but tbh the design wasnt really good lol. Im 16 now so Im quite happy about my progress

[0] https://www.reddit.com/r/FPGA/comments/1kwxvk8/ive_made_my_f...
by Aurornis | 2025-05-30 23:56:32+00:00
No one said the AI created new algorithms nor that there weren’t pre-existing solutions.

The implication was that the FP32 versions of these kernels have lagged behind the more popular versions. There was opportunity to translate the advancements from other kernels into these. Someone would need to look closely to see exactly what was done, but it’s premature to suggest anything like “new algos” or “no pre-existing solutions”

This is a great use case for LLMs, though. I often do something similar where I make improvements to something I use most frequently and ask an LLM to translate that pattern to other similar parts of the code.
by lexandstuff | 2025-05-30 23:57:02+00:00
I've never had a job like that. My job has always involved helping my company, not just figure out how to build something, but what to build. We typically collaborate on a few ideas and then go away, let them percolate in our brains, before coming back with some new ideas to try. The whole point of the Agile Manifesto is that we don't know what to build in the first place.

Sometimes my boss has asked me to do something that in the long run will cost the company dearly. Luckily for him, I am happy to push back, because I can understand what we're trying to achieve and help figure the best option for the company based on my experience, intuition and the data I have available.

There's so much more to working with a team than: "Here is a very specific task, please execute it exactly as the spec says". We want ideas, we want opinions, we want bursts of creative inspiration, we want pushback, we want people to share their experiences, their intuition, the vibe they get, etc.

We don't want AI agents that do exactly what we say; we want teams of people with different skill sets who understand the problem and can interpret task through the lens of their skill set and experience, because a single person doesn't have all the answers.

I think your ex-boss Mike will very soon find himself trapped in local minima of innovation, with only his own understanding of the world, and a sycophantic yes-man AI employee that will always do exactly as he says. The fact that AI mostly doesn't work is only part of the problem.
by esperent | 2025-05-30 23:57:35+00:00
> just enough skilled to be worried their life's worth is going to be replaced

Everyone starts out that way. When you start work as a coder, or an artist, or writer, you are just enough skilled to make money. You can bypass some of this step by university or other forms of education, or you can be lucky enough to have spent your teenage years developing a skill that makes you money as an adult. But nobody starts out highly skilled.

It's a genuine source of worry that AI will subvert the process of people going from "just skilled enough to make money" to "highly skilled". I'm not fully convinced that it will - I think it's likely that in the long term society will adapt, just as it did to other tools.

But nonetheless it's a valid concern that shouldn't be dismissed. As with other new tools, it's going to disrupt a lot of people's livelihoods and we should approach that with empathy rather than telling people they simply weren't skilled enough to deserve to continue their career in this new reality.