skysight-inc/HackerNews huggingface dataset:
This looks really interesting, congrats! It checks a few key boxes I was always hoping to see solved in Astro, namely first-party support for web components and a clean option for HTML-partials and tools like HTMX. I'm guessing a bit on the second one, bases on the site I'm assuming routes that ship raw HTML partials works out of the box.

My main hesitation is RPC event handlers. That's been the trend for a couple years now, for me its always a bit too much magic. I always prefer controlling the API URL being called, both for clarity and to ensure that URLs being called are always stable and more easily testable and reusable.
war crimes, if you ask me
Yup, it's a fantastic read. I based most of my post off it (I clearly mention so) and it's worth it to read at least the first part of it before reading my post.
To paraphase [1]:

> Some people, when confronted with a problem, think “I know, I'll use micro-services.” Now they have two problems.

As soon as I read this example where there's users and orders microservices, you've already made an error (IMHO). What happens when the traffic becomes such an issue that you need to shard your microservices? Now you've got session and load-balancing issues. If you ignore them, you may break the read-your-write guarantee and that's going to create a huge cost to development.

It goes like this: can you read uncommitted changes within your transaction or request? Generally the answer should be "yes". But imagine you need to speak to a sharded service, what happens when you hit a service that didn't do the mutation but it isn't committed yet?

A sharded data backend will take you as far as you need to go. If it's good enough for Facebook, it's good enough for you.

When I worked at FB, I had a project where someone had come in from Netflix and they fell into the trap many people do of trying to reinvent Netflix architecture at Facebook. Even if the Netflix microservices architecture is an objectively good idea (which I honestly have no opinion on, other than having personally never seen a good solution with microservices), that train has sailed. FB has embraced a different architecture so even if it's objectively good, you're going against established practice and changing what any FB SWE is going to expect when they come across your system.

FB has a write through in-memory graph database (called TAO) that writes to sharded MySQL backends. You almost never speak to MySQL directly. You don't even really talk to TAO directly most of the time. There's a data modelling framework on top of it (that enforces privacy and a lot of other things; talk to TAO directly and you'll have a lot of explaining to do). Anyway, TAO makes the read-your-write promise and the proposed microservices broke that. This was pointed out from the very beginning, yet they barreled on through.

I can understand putting video encoding into a "service" but I tend to view those as "workers" more than a "service".

[1]: https://regex.info/blog/2006-09-15/247
I think attempting to automatically "undo" partially failed distributed operations is fraught with danger.

1. It's not really safe, since another system could observe the updated data B (and perhaps act on that information) before you manage to roll it back to A.

2. It's not really reliable, since the sort of failures that prevent you from completing the operation, could also prevent you from rolling back parts of it.

The best way to think about it is that if a distributed operation fails, your system is now in an indeterminate state and all bets are off. So if you really must coordinate updates within two or more separate systems, it's best if either

a) The operation is designed so that nothing really happens until the whole operation is done. One example of this pattern is git (and, by extension, the GitHub API). To commit to a branch, you have to create a new blob, associate that blob to a tree, create a new commit based on that tree, then move the branch tip to point to the new commit. As you can see, this series of operations is perfectly fine to do in an eventually-consistent manner, since a partial failure just leaves some orphan blobs lying around, and doesn't actually affect the branch (since updating the branch itself is the last step, and is atomic). You can imagine applying this same sort of pattern to problems like ordering or billing, where the last step is to update the order or update the invoice.

b) The alternative is, as you say, flag for manual intervention. Most systems in the world operate at a scale where this is perfectly feasible, and so sometimes it just makes the most sense (compared to trying to achieve perfect automated correctness).
Chinese hackers breached US court wiretap systems, WSJ reports
> The fraudulent behaviour is on the part of the streamers.

He was also coordinating the botnet streamers. From the first sentence of the first paragraph of TFA:

> a scheme to create hundreds of thousands of songs with artificial intelligence and use automated programs called “bots” to stream the AI-generated songs billions of times.
Thanks to Docker, whenever I start a new project or team, I always ensure that everything can be run locally (the DB, Redis, services, website, and mobile app). But it's hard to be disciplined, especially when reproducing bugs, so developers usually end up using a "shared" test server.

Also, these days, I use Cloudflare more and more. They're very affordable, and deployment is a breeze for the simplest cases. But local development seems to be an afterthought. I built a service that uses some of their dev offerings. Some work locally (using Miniflare), and some can only work remotely (dev environment in your Cloudflare account). Imagine when you need both kinds of offerings!
What‘s your problem? Typing http in the search field at the bottom results in https://hn.algolia.com/?q=http

All results have http in their titles.
For just hosting something as long as possible I would highlight two options stolen blatantly from previous threads:

If you can throw enough money at the problem and are in the US, set up a trust to host your content for as long as the funds last.

If not you could bet on The Internet Archive to outlive you (and maybe donate to them).

But I think the bigger issue than hosting is discoverability. How will your great grandchildren find your content even if it was still available?

To cover both issues the best bet is to focus on somehow convincing your offspring on keeping your content and memory alive in what ever way they see fit, enough so that they pass it on. I have no ideas for exact details here, maybe someone else does?
This also happened to me.

I do still get their email invitations to participate in various OCI webinars and developer conferences. Which is, you know, what I really wanted all along.
Ask HN: Is there an API that enables something similar to notebooklm?
I've been blown away by notebooklm and would love to use it via an API. There currently isn't an official API but is there some alternative way of enabling something similar via an API or combination of APIs?
>"The guy lied for financial gain. That makes it fraud."

Isn't lying for financial gain is what almost any company does?
Dupe: https://news.ycombinator.com/item?id=41753826
If you work in Exeter you've probably already got an hour's commute each way. It's not unreasonable to consider making that two hours a hardship for many people, whether or not you consider that a worthwhile trade-off.
Grail Demo from Computer History Museum Archive [video]
Whenever I see a flake.nix in a project I'm more likely to contribute because I know I'm not going to get stuck in dependency hell before I can even test a single change.

Reproducible builds are what I've found to be the most important part of any local dev experience; standing up local databases, message queues etc required for whatever is being developed has always been relatively simple in comparison.
Also, (2010).
I'm really enthused by it not delving in too deeply...
On a completely unrelated note, I call for the repeal of the HN rule that forbids me from calling out commenters for having not read the first sentence of the first paragraph of the fine article.

rightbyte, your laziness contributes nothing to this forum and it would be better if you deleted your account. Your comment was bad and you should feel bad.
I would put it on IPFS.
Fluidsynth: Open-Source Software Synthesizer Built on SoundFont 2 Specs
Similar discussions from before:

https://news.ycombinator.com/item?id=36065088

https://news.ycombinator.com/item?id=41584922
Andrej Karpathy: llm.c (GPU Mode IRL 2024 Keynote) [video]
FWIW, `bun` produced larger output files

~/P/s/ffmpeg-over-ip bun build ./src/client.ts --compile --minify --sourcemap --bytecode --outfile ffmpeg-over-ip-client ; ls -lah | grep ffmpeg-over-ip-client
[6ms] minify -90.92 KB (estimate)
[3ms] bundle 6 modules
[44ms] compile ffmpeg-over-ip-client
-rwxrwxrwx 1 steelbrain staff 56M Oct 6 16:53 ffmpeg-over-ip-client

and `deno` produced even bigger outputs

~/P/s/ffmpeg-over-ip deno compile --allow-read --allow-net ./lib/client.js ; ls -lah | grep client
Compile file:///Users/steelbrain/Projects/steelbrain/ffmpeg-over-ip/lib/client.js to client
-rwxr-xr-x 1 steelbrain staff 65M Oct 6 16:56 client
Fraud is defined as deception in a business transaction for financial gain.
I build a replica of the first Chinese domestically produced single (germanium) transistor radio but using ‘modern’ components. This model 636 radio was released in 1953 or 1954 (sources are not clear). Check out the files here[1] and build your own!

Sadly I still need to write up the very interesting and complex history of this radio but do not have the time/funding at the moment.

[1] https://dennisdebel.nl/2017/2023-636_Radio_DIY/
We've entered the AI grift era
Proof that the statement is an axiom. And the fact that it's an axiom also falls under the axiomatic principle of guaranteed human unhappiness.
> pricing is astronomical. you also have to "contact sales" to get access to anything outside of a free trial

You don't have to contact sales if you are a Chinese-speaking customer. And pricing is fine. ByteDance has a different brand for their cloud services in China: https://www.volcengine.com/ [1]. But of course the underlying infrastructure are all the same.

This is very likely done by a Chinese customer using ByteDance's cloud service.

[1] Well, Alibaba Cloud did this too, and ByteDance is copying Alibaba 1:1 (who in turn is copying AWS) so I'm not surprised. But at least Alibaba named their international brand "Alibaba Cloud" and their CN one "AliCloud", similar enough.
Funny how exploiting uninitialized variables is evil in (nonstrict) JavaScript but good in Awk. I agree that’s true, mind you, I just can’t pinpoint what about the languages’ designs makes it work in one case but not the other.
Abused in what way? Content? How would they know, if it was encrypted. Or volume?
Famously, the most litigious and generally obnoxious to deal with enterprise software company on the planet. Or ale doesn’t have customers, they have hostages.
Trusting any single actor with your data is a bad idea, no matter if it's Oracle, Cloudflare or $CurrentInternetDarling.
I view JS as necessary because of the Web but honestly I don't like doing Python for anything serious because it's a dynamic language and I don't want to write unit tests for spelling mistakes. I realize it's the tool of choice for AI/ML/DS (for good reasons).

But I'm with you on fast compilation and at Google this was the case, even with C++. There was a ton of infrastructure made around this that wouldn't be easy to replicate but I could certainly compile and run a C++ unit test in <20 seconds most of the time.
If I decided to order something (through Amazon) that I can't afford to have left out on the porch, I'd have it sent to one of their lockers about 200 meters away at the local Gas Station, and pick it up there.

If this happens often, consider getting a UPS drop box, or a box at your Post Office.

Otherwise, a neighbor who is home at all hours.
Then there are actually two "AI centers", one in Zurich and one in Lausanne? And what about the Manno CSCS center? Is it somehow involved?
Devbox aims to solve this very problem: https://www.jetify.com/devbox
AI generated music isn't the issue. I'm pretty sure pop song production is tool-assisted.
Once you've learned the syntax of the language, long expressions like that are about as readable as however-many-dozen lines of JS/Python with 1-to-3-character variable names; i.e. some parts may be obvious if they're a common pattern or simple enough, but the big picture may take a while to dig out.

Probably the biggest readability concern of overly-golfed expressions really is just being dynamically typed, a problem shared with all dynamically-typed languages. But array languages have the problem worse, as nearly all operations are polymorphic over array vs number inputs, whereas in e.g. JS you can use 'a+b' as a hint that 'a' and 'b' are numbers, & similar.

If you want readable/maintainable code, adding comments and splitting things into many smaller lines is just as acceptable as in other languages.
"may" is that English word that non-natives learn too late that it actually means "will".

I remember when I first moved to England and my boss told me I may work on x and I thought that it is optional so I did something else.
He said this in 2022 when it was definitely not possible to run FB on a laptop either.
Jokes are fun and all, but we should at least try to answer the question.
WiFi4EU initiative provides free Wi-Fi in public spaces across Europe
Maybe run every build version in their own container?
Nah I'm good, thanks anyway. I prefer to preserve the commenting culture by gently ridiculing people who don't get it. Downvotes and flags don't discourage nearly as well.
We went remote-only dev years ago; definitely never going back. It's such a pleasure to always have the same stuff everywhere, no matter what and on any device.
The best book I've partially read is the New Testament. The best book I've actually read is Taleb's Antifragile.
You are just wrong.

Before the camera, portrait painting was how most painters would make their living and the camera upended that completely.

On the last line, look up hyperrealist painter on a search engine. That is the reverse , an artistic movement in painting inspired by the photograph.
These days, you have to remove the public access block AND explicitly write a bucket policy (or set up deprecated ACLs) to allow public access.
archive.org, assuming their legal troubles don't take them down. Upload it to there, and ta-da! Body of work. If text, collect it into a book and upload it to LibGen or something also - plenty of shadow sites mirror it. (Yes, it'll be harder to charge for it, but it will probably live on.)
Very nice! I like the readability-- not sure if thats just indicative of your style or the language, and the map construct is also nice. I don't remember any off-the-shelf map construct, at least not in Dyalog.
I agree with the free speech part and so for Xitter to be a tool for free speech we need to make it public and federated. Otherwise it is just Elon Musk's manipulation tool that you love.
as a potential step up from overly sensitive pattern matching: somewhere I ran across the idea that our close primate relatives enjoy sleight-of-hand magic tricks, but more distant ones do not.
How is mojo doing ? Has it made its way as a niche language in some places ?
Unfortunately this one isn't easy to navigate on mobile.
It's techno-opportunism at its worst:

"I can have this postmodern values but if you don't like it I can have these other ones. Repeat until something sticks."
This would be a very dumb way of phrasing the situation, because the “AI musician” doesn’t have any understanding what the scheme is, it was entirely a plan of the human conspirators. It’s exactly as childish and stupid as saying “gun aided by human shooter in murder.”
Just a few weeks ago Elon went DEFCON 3 kanye West style against this judge and apparently, lost badly. Now he is begging to get back to Brazil.
I've found that the challenge is to "think in vector operations" rather than of iterating over the same data. The tricky part is figuring out how to get an operator to do the right thing over an array of stuff on the left hand side and this list/bag/etc of arguments on the right
Observing
https://www.youtube.com/watch?v=A-brgkkjnHc
Zero-Shot Text Classification on a low-end CPU-only machine?
I want to do zero-shot text classification either with the model [1] (711 MB) or with something similar. Want to achieve high throughput in classification requests per second. Classification will run on low-end hardware: some Hetzner [2] machine without GPU (Hetzner is great, reliable and cheap, they just do not have GPU machines), something like this:

* CCX13: Dedicated vCPU, 2 VCPU, 8 GB RAM

* CX32: Shared vCPU, 4 VCPU, 8 GB RAM

Now there are multiple options for deploying and serving LLMs:

* lmdeploy

* text-generation-inference

* TensorRT-LLM

* vllm

There are more and more new frameworks for this. I am a bit lost. Would you suggest the best option for deploying the above-listed model (No-GPU hardware)?

[1] https://huggingface.co/MoritzLaurer/roberta-large-zeroshot-v2.0-c

[2] https://www.hetzner.com/cloud/
It would have taken time to do this and consequently Python would have missed the race and some other language would now be #1.
Hacker News and GitHub. Your contributions to these platforms are syndicated in datasets like The Pile https://en.wikipedia.org/wiki/The_Pile_%28dataset%29 Stay away from platforms like Reddit that gatekeep your data to extract rents.
I take it you've never worked with Oracle.
You have a single shared server that everyone has to book time on to run and debug their code?

Or does everyone have their own server that they ssh into? Cause I consider that as close to local as you can get without running stuff on your own laptop. Would be very annoyed though those few times I work on a plane or a train with spotty 4G/5G.
My bank sells CDs to me, at a rate that's clearly disclosed to me. What type of "lying for financial gain" is happening here?
So my take is that music streaming platforms do zero quality control. Not that fraudulent streams are right either.
I can’t even register an account at Oracle.

I’ve tried many things and their 3rd party check refuses to consider me a good actor.
Publish it as a physical book, deposit it with the Library of Congress (or comparable institution in your jurisdiction).
Outside of literal cults and criminal organizations, I can't think of an industry that has duped, robbed, swindled, abused, and destroyed people at the same pace as the music industry. And generally, it is just hand-waved over in court.
Smartest people do on weekend what everyone else will do in ten years (2013)
For this reason, I strongly recommend Google Firebase.

The local emulator suite is one of the best I've seen[0] (would love to see others). Powerful and easy to set up[1].

It includes a top notch emulator for auth which gives you a full SSO flow as if going through Google's OAuth flow and makes it easy to get otherwise complicated auth flows nailed down. The database and Functions runtime emulators are excellent and make it easy to prototype and ship. Comes with a Pub/Sub emulator to boot for more complex async processes. You can export the emulator state to disk so you can share it or bring it back up into the same state.

If you need to interface with relational DBs, you just use a Pg or MySQL container.

Really phenomenal and would love to find other stacks with a similarly solid emulator suite. It's a strong recommend from me for any teams that value speed because it really allows much, much faster iteration.

Edit: Dear GCP team, please, please - never kill this thing.

[0] https://firebase.google.com/docs/emulator-suite

[1] https://github.com/CharlieDigital/coderev/blob/main/web/util...
Sounds like he also tried to launder the profits of the fraud. If you set up a bunch of different bank accounts registered to different names or LLCs to collect all the profits of an illegal scheme so it doesn't look like a huge amount of money in each account, that's an additional money laundering charge versus if you put all the profits from a scheme in the same bank account.
I was about to make a snarky comment about starlink. It's getting harder to take a shot of the sky without one of those pesky floaties.
I love awk as a language / framework. If it got an uplift to make it useful for more complex problems it would be an absolute winner for lots of basic data processing tasks.

I’ve often written incantations many lines long and even broken out to actually writing awk scripts from time to time.

Once the penny drops with it it’s great fun but it’s absolutely useless once your problems get to any degree of sophistication.

I typically move into Python at this stage. Perl and Ruby are probably a more elegant fit here but those aren’t rows I want to have.

In this day and age, awk really needs CSV (RFC-4180) support and better semantic scoping and library support.

I’d also think it would be neat as an embedded language for various data processing platforms but if we haven’t seen it yet I doubt we will ever see it.

EDIT support for file formats beyond plain text would also be a winner.
Sure. Finney+Sassaman is a not-that-fringe theory. Le Roux might not have been alone but seem far-fetched to have coupled with any of the others on this list, specifically.
I regularly use may in a way that doesn’t mean “will”. Sorry that you had a bad experience or whatever.
Jack’s seeing lots of phantoms—why not also the viewer, whom the camera brings “in” to the scene?

(When you start pulling on that thread—the viewer as a vital part of the fiction, as really present, as encouraging and complicit in what takes place—and make that the focus of a horror film, you get Funny Games)
The State of Prod ML: 36% take 1-3mo to prod, 10% use spreadsheets for tracking
Sharing some interesting (preliminary) results for the 2024 State of Production ML Survey:

Deploying ML: 36% take 1-3 months, 21% 3-6 months

Experiment Tracking: 42% use MLFlow, 10% Spreadsheets

Feature Stores: 53% use none, 28% Custom-built

Vector DB: 55& use none, rest unconsolidated

Training: 27& use custom-built, 21% Databricks

Serving: 56% use custom-built (+ FastAPI/Flask)

Monitoring: 50% use none, 24% Custom-built

Diversity: Only 4% identify as female

This is a community initiative and the data will provide all of us in the community with actionable insights to improve the ecosystem. We aim for this input will help create a comprehensive overview of common practices, tooling preferences, and challenges faced when deploying models to production, ultimately benefiting the entire ML community

We are opening this survey until the end of October, and we'll publish the results for the community to derive useful insights! If you can please take two minutes to share your experience: https://bit.ly/state-of-ml-2024

You can also check out the preliminary results here: https://ethical.institute/state-of-ml-2024 - we are building an interface for basic slice and dice to enable extracting further insights (but still early WIP so feedback appreciated). Final results / report will be published end of October!
AI can predict how some code behaves when run?

So AI can predict whether some program halts?

Seriously?
The smokes don’t seem like that big of a deal but the fire seems like it is a complete disaster for fairness. That’s actually pretty horrific - the other contestants out there working and suffering their asses off to survive and he gave a huge leg up to arbitrary competitors? I should hope he got in serious trouble - can you imagine an official in a sports competition casually messing with the game w/ $1M on the line???