skysight-inc/HackerNews huggingface dataset:
When we replaced a venerable Tru64 cluster with Linux about 15 years ago, the head of the department (who was skeptical of linux) came to me with a problem:

he had scheduled 9 "use 100% of a core" processes on an 8 core machine and expected each process to get 8/9ths of a core on the machine, with each process proceeding at the same rate. (If I got that math wrong, basically I mean that all processes should receive the same amount of CPU time over an interval, with the expectation that the CPUs will be at 100%).

Instead, he found that one of the 9 processes always went slower than all the others. Tru64 did what he expected. I believe at this point, we were running a kernel before the completely fair scheduler, but I've always wondered if there was a right answer here- I don't think UNIX or Linux truly 'guarantee' fully predictable and evenly divded CPU performance when you have n+1 (or n+whatever) processes running on n cores.

(also, he used "yes > /dev/null" as the CPU-generating task and I had to spend a lot of time explaining to him that wasn't really 100% cpu)
Today's answer is that the federal reserve has announced that they feel recently high interest rates have achieved their goals and even see a possibility of rate decreases.
Show HN: I scraped 25M Shopify products to build a search engine
Hi HN! I built Agora as a side-project leading up to the holiday season. I wanted to find an easier way to find Christmas gifts, without needing to go store-by-store.

My wife asked me for a a pair of red shoes for Christmas. I quickly typed it into Google and found a combination of ads from large retailers and links to a 1948 movie called 'Red Shoes'. I decided to build Agora to solve my own problem (and stay happily married). The product is a search engine that automatically crawls thousands of Shopify stores and makes them easily accessible with a search interface. There's a few additional features to enhance the buying experience including saving products, filters, reviews, and popular products.

I've started with exclusively Shopify stores and plan to expand the crawler to other e-commerce platforms like BigCommerce, WooCommerce, Wix, etc. The technical challenge I've found is keeping the search speed and performance strong as the data set becomes larger. There's about 25 million products on Agora right now. I'll ramp this up carefully to make sure we don't compromise the search speed and user experience.

I'd love any feedback!
> LLM tools - Ollama, llama.cpp

I was including these since they're LLM-related cli tools.
true but it still causes frictions with friends, family.
I think the big issue is that everyone like pydantic and are willing to put it in places it may not really see a big benefit, over say, DRF.

OTOH, if you're a big fan of NoSQL databases, then pydantic is a godsend.
I don't know where 1-2% comes from, but for many scale production workloads I studied it was so close to 0% that it was tough to measure beyond noise on the cloud. That's not to say that 1-2% is wrong, but that it's likely someone's workload and other people see less.

Helping people find ~30-3000% perf wins, helping debugging and automated bug reports, is huge. For some sites it may be like 300 steps forward, one step back. But it's also not the end of the road here. Might we go back to frame pointer ommision one day by default if some other emerging stack walkers work well in the future for all use cases? It's a lot of ifs and many years away, and assumes a lot of engineering work continues to be invested for recoving a gain that's usually less than 1%, but anythings possible.

There's a couple of problems with an apt reinstall. One is that people often don't work on performance until the system is melting down -- many times I've been handed an issue where apt is dreadfully slow due to the system's performance issue and just installing a single package can take several minutes -- imagine reinstalling everything, it could turn the outage into over an hour! The other is I'd worry that reinstalling everything introduces so many changes (updating library versions) that the problem could change and you'd have no idea which package update changed it. If there was such an apt reinstall command, I know of large sites (with experience with frame pointer overheads) that would run it and then build their BaseAMI so that it was the default. Which is what Ubuntu is doing anyway.
Out of curiosity, what is your development setup for that?
What you said did not make much sense, if it got poorly remade how can the local one be bad? that would mean the local one is better because the remake is worse.

Most horror remakes are seen as bad and a lot are forgotten, The mist, The Shining, Psycho. Shows imported from britain are bad, like The it crowd, a good deal of panel shows from britain crashed and burned.

Hollywood also has the capability of throwing away movies if they think are bad enough. "Release the snyder cut". There's plenty of bad remakes, I'm not young but a lot of things are remakes from the 50s 60s 70s 80s and we just don't know, apart from the adaptations of foreign movies and films.
Yeah, sorry if my jest wasn't clearly jesting. I'm actually not overly concerned with successful VC-backed companies, I'm just pointing out that the term "unicorn" is fairly contrived.
The military transformation of medieval Europe: Stirrups vs. social cohesion
Voyager 1 in Trouble as Engineers Scramble to Debug Issue with Flight System
Here's mine! https://skeoh.com/othello/

It's not nearly as polished but I had a lot of fun making it. It uses React and the now-no-longer-maintained https://boardgame.io/.
Summary: "don't be bad at cybersecurity".

Cybersecurity is special in a real sense: it's one of the few groups typically charged with enforcing required compliance controls. I was a senior engineer for many years before moving full-time into security, and it's amazing how many of those annoying inconveniences are actually mandated from external customers or frameworks[0]. As an engineer, I don't really care if every PR has an associated Linear/Jira/whatever ticket; sometimes it's enough to say "I'm fixing this thing because it needs to be fixed". As a compliance person, SOC2 says that all change requests must have an associated ticket, and my personal feelings about that aren't part of the equation.[1] Think Security is a pain in the neck? Work in a regulated industry like healthcare or finance where you have lawyers instead of colleagues giving you a list of commandments.

Also, I've never slept so well as I did the month after I was allowed to hire my CISO replacement. It's like looking up and seeing the Milky Way instead of the Sword of Damocles.

[0] Having worked at shops where Security didn't take time to explain those things, I've vowed never to be that person. If I ask you to do a thing, I'll tell you why it's something we have to do. We should all collectively be better about that.

[1] Although there's an analogy between "compliance" and "unit testing". A side benefit of thorough test coverage is that you tend to write better structured, more loosely coupled, easily testable code. That's all good to have even if you never ran a single test. Same with compliance. The ability to audit all the things means you're more likely to design your systems so that they can be inspected, instrumented, and monitored. In the "where's the ticket?" audit case, know what's more annoying than having to match every PR to a specific ticket? Having to individually assess a couple hundred exceptions to very that each of them reflects a reasonable judgment to violate the guidelines.
https://en.wikipedia.org/wiki/Claudine_(film)

> ... Throughout the film, Miss Kabak, a social worker, visits Claudine at her home and asks her if she is employed and if she is dating anyone. Claudine always claims to be unemployed and single, to make sure to get the maximum amount of benefits, which she desperately needs. If Claudine has a job or dates anyone and receives gifts from her boyfriend, the social worker has to deduct any money or gifts from her benefits, forcing Claudine to lie.

This movie was released in 1974. Though fiction, it's one of the examples used when people say that when government gets involved ($$$), it moves slow and breaks things for a long time.

I think people will always try to game/pervert a system, and unfortunately we all will have to suffer the consequences. For now.
There is no "pretentious tech positivity" but by all means save your money and stay away. They won't miss your sparkling personality.
Is This How Amazon Ends?
Cone and CBVision look amazing. I know several people who could really benefit.
I think plenty of entrepreneurs emphasize that they are scrappy underdogs, that it’s a long shot, and that it’s going to take hard work. That’s you know, the optimistic side of the truth that they’re going to fail. I’m sure Musk has said things like this. He’s also just lied about stuff. He’s remarkably careless about what he says/tweets for a CEO. Might have something to do with his drug use, who knows.
The article does acknowledge the existence of real fraud. It distinguishes it from honest errors. It goes on to point out that sometimes institutions (financial and state) sometimes fail to adequately distinguish the two.

Maybe it could have included more on bona fide fraud. But it doesn't contain an assumption that true check fraud is non existent. And it certainly doesn't devolve into any sort of rant.
The printing press's impact was in ending the Catholic Church's monopoly over information, and thereby "the truth". It took 400 years for that process to take place.

The Gutenberg Era lasted all the way from its invention to (I'd say) the proliferation of radio stations.
https://archive.is/blJuU
Very apt, because the mosquito will be fine in most situations where a shotgun is used.
Not trying to wade into a continental dispute, but weren't gas pumps in the US also fill-then-pay until maybe 10-15 yr ago? I find it strange that you now have to basically guess about how much it will cost and then authorize a purchase up to that amount, instead of, you know, paying for however much actually got pumped (at today's price).
Year in Search 2023
"It really doesn’t make sense for cybersecurity to have its own special snowflake process for things. It does not make sense operationally, philosophically, or socially."

In general, I believe it does - cybersecurity is unique in the sense that you're dealing with intelligent adversaries that are actively trying to break your code. It's not like SRE work or normal software engineering. But there's also a lot of snake oil in the industry and most "cybersecurity" professionals don't actually know how to find vulnerabilities.

"Attackers accessing our systems without our consent is one type of failure, but not the only kind. Reliability failures are arguably both more frequent and more damaging when they occur".

This will completely depend on your product, but I've seen this sentiment across industries. "Risk = probability * impact" doesn't work when you throw security into the mix. The author sounds like a very common type of software developer who takes security issues in their code like a personal attack.
I'd really like to see 1 and 2!
Misleading title

> ICNS to build brain-scale supercomputer

> The world's first supercomputer capable of simulating networks at the scale of the human brain has been announced by researchers from the International Centre for Neuromorphic Systems (ICNS) at Western Sydney University.
I have never considered tipping by check - and have occasionally stiffed people who deserved a tip because I didn’t have enough cash. Is using checks for this really a thing people do?

When doing this do you name of the person you’re tipping, or just give them a check with the first line blank?
OK,

1) This is awesome :)

and

2) How much of this is you, and how much is ChatGPT? (Or a thesaurus?)

Still, I tip my hat to you! :)
> Slavery is like an old disease such as Polio: it still exists in some part of the world, but we're progressively eradicating it.

Slavery has actually been on the increase lately, the following is just one of the statistics confirming this. There are more recent claims that covid and new wars have increased it further, but it is a hard thing to measure

> An estimated 50 million people were living in modern slavery on any given day in 2021, an increase of 10 million people since 2016. [1]

That's roughly 1:162 people who are in slavery today

Also, it's rather dehumanizing to compare slavery to a disease. One is biological, the other a choice to enslave another human being

[1] https://www.walkfree.org/global-slavery-index/
Fantastic novel. Fans of science fiction should not miss it. Watts' other books, such as The Freeze-Frame Revolution, are also excellent. I also quite liked the "sidequel", Echopraxia.
Very impressed by what you've come up with - it will be brilliant not to be worrying about failed paste!
I see the liquid metal pad is now in the store. How easy will replacement be if that's needed?
https://frame.work/gb/en/products/16-liquid-thermal-pad-amd-...
I think that's correct in principle, but the important difference is that you don't have to know the priority in advance.

E.g., to implement an Israeli queue using a standard priority queue, you'd have to track the priority of each "friend group" in a separate data structure.

Concrete priority values are also difficult to reuse without accidentally inserting a new group in the middle of the queue - so the values are at the risk of overflowing.
Eschew obfuscation.
Yeah, it's pretty usual Jewish humor, which is mostly self deprecating.
> Making IRS payments without a fee

I think you are exaggerating, conflating "checks" with "electronic transfers from my (checking) account".

IRS has an excellent electronic debit system. Sure, the money comes from my checking account. Are you saying that is the same as a check?

> Paying the gas bill. My gas company charges $5+ to pay by credit or debit card.
> Paying the electric bill. The electric company charges $5+ to pay by credit or debit card.

e-pay by the bank... is the same as a check? Maybe...

The whole thing about the rent is surprising. I'm surprised you don't get billed for checks. I had a direct-debit-only landlord recently. Surcharge for everything else.

You forgot: your cable company also wants direct access to your bank account and will give you $5/month for the privilege. So consider that a fee for paying any other way.
Definitely true at startups where it's that or what else you could/should be doing, and maybe in a few months a pivot means it's mostly discarded for a new adventure.
Perhaps provide some links, and explain what "The Goal (1,2,3)" is? And who are "they"?
> So what a civilised country that values human life (especially of its own citizens) should do?

Not bombing people (killing some of the hostages doing so). Make a serious offer for peace. Nothing serious has been done since Rabin's assassination. Both Barak and Olmert's tries were ham-handed, at a time when both Barak and Olmert were about to be ousted from power.

Can't you understand that violence won't solve anything at all? Retaliation won't resuscitate any of the dead. "Eye for an eye" will only make everyone blind in the end.

If Israel want to get back its lost moral up-hand, it has to offer peace, for real. Nothing else will do. If Israelis are serious about being better than the opposing side, than they have to do better than the opposing side, not worse.
Hey, it's not stupid if it works! In the context of this particular game, I can't see why it would be a bad idea. Even if you need to conserve some kind of state... gzip the whole thing and copy it elsewhere, it's going to take seconds!
Judging someone by how many citations they have is like saying The Pope is right about everything because millions of cardinals and priests will refer to his proclamations in their masses.
Where can you get the Pico for only 200?
Dunno, I couldn’t help but exfiltrate the model from Azure first thing in the morning (ugh) and I’m staring at “data/model/modeling_mixformer_sequential.py”

it looks like standard OOP Multi Head Attention but has both self and cross attention instead of just self attention (also standard, just personal preference to avoid cross attention, they are equivalent if you know what you’re doing)

except for the Mixformer from 2203.11082, the ParallelBlock looks like just MHA, MLP, Residual, and the rotary embedding from 2104.09864 is a big chunk of it. Otherwise just einsum you saw already.

The main reason this model is more capable is A) high quality dataset of text many brilliant humans worked very hard to produce and won’t get paid for and B) Microsoft broke the [almost surely felonious](https://www.justice.gov/atr/page/file/1091651/download ) terms of use of Microsoft OpenAI by using “synthetic” data aka “Microsoft OpenAI Output no one else is allowed to use but Microsoft” (obligatory Cartel evidence link: https://i.postimg.cc/MGqPvPz5/cartel-microsoft-microsoft-ope...) to develop their model (they used GPT-3.5-Turbo to generate many responses and used GPT-4 to outsource the hard part of AI (quality evaluation) 100,000 times) … also they have a lot more GPUs than we do.

If not for the fact Microsoft OpenAI was bought and paid for by Microsoft already, breaking the [almost surely felonious](https://www.justice.gov/atr/page/file/1091651/download) Microsoft OpenAI Terms of use by using the output to develop a Microsoft model would deserve a[n almost surely unwinnable](https://en.wikipedia.org/wiki/Illegal_agreement?wprov=sfti1) lawsuit from OpenAI, but I doubt Microsoft OpenAI will sue Microsoft, that would bite the hand that feeds them.

It’s a risky maneuver which leaks GPT-4’s most precious and unique capability (being heavily manually fine tuned on human evaluation of text) and just goes to show how much you can get away with if you have enough money and power…

(I am not a lawyer, by the way!)
Very impressive! Any information on how this compares to 3D Gaussian splatting in terms of performance, quality or data size?
GM's Cruise robotaxi unit dismisses nine execs after safety probe
Ah but the internet was about connecting every human being to all information ever, anywhere, instantly.

LLMs are hardly much more than a very lossy and error prone compression algorithm for the same.
I know of all that, hell I was listening to Plastic Beach when it came out. My point is that EDM and modern Pop coexist with older forms of music. I don't think new ways of creating art are zero-sum, they create new demand.
Building an app that scales to enterprise size is hard. Building an app that does so while also scaling down to single-team size is hard. Every feature you build two backends for is one more thing to go wrong. Writing a sqlite backend for your redis usage, your search, etc... it's not insane, and it's a sign of good Faith, but it's not something that should be expected, either.
is it true to assume that in some cases, some messages in the queue might never get processed, if any other messages in the queue have an infinite amount of friends?

Reminds of a triage in a hospital.
please say why it is a must-read.
Fair. I'm more than a little worried on the perceptions, though. Having arguments with people who think that the English alphabet is not phonetic really hurt my views there.
The UK planned, in 2009, to phase out cheques by 2018 - but backtracked in 2011: https://www.gov.uk/government/news/frequently-asked-question...
PDF: copy-paste entire link into new tab or `base64 -d` encoded section

data:text/plain;base64,aHR0cHM6Ly96bGlicmFyeS50by9maWxlZG93bmxvYWQvZGVzaWduLXJ1bGVzLXZvbC0xLXRoZS1wb3dlci1vZi1tb2R1bGFyaXR5LTA

Not a converted EPUB, either. ^^
I felt like I wrote this comment because I did exactly the same once long time ago. Made directories like 0/1/2/3/4/5/6/7/8/9 starting from 0 all the way to 9 billion. I was doing it on a Windows XP machine. I don't exactly remember if I completed the process or stopped it midway but in the end I had this structure which windows could never search through in any reasonable time. Only I would know where my files are. After that I tried zipping all that structure and remember taking it like 65MB or something and took many hours. Deleting this whole thing was also very painful. It also grew the size of MFT file significantly.

Never tried this again. And I don't remember availability of USB drives back then.
>If you have a competent and fully staffed SRE Platform engineering team - you will NOT need a separate cybersecurity team.

So the SRE team is supposed to find vulnerabilities in the codebase and pentest the network?
they would stay there as they please as long as they aren't subjugating palestine as occupiers.
When I first learned about Bitcoin, I thought "this is interesting, but the person telling me this is using a 'numbers go up' graph to try to convince me to buy. They specifically, irregardless of the technology, smell like scammers".

(That was at $250/BTC, so of course I have a certain level of regret from that which may cloud my thinking…)

AI? I've been interested in it since the late 90s when I first learned that ANNs were an actual thing one could program and which could learn for themselves, but that just means I'm disappointed by seeing obvious scammers and grifters trying to make a quick buck from what I regard as a legit tech — and because I can see the bad actors, I can sympathise with people who want to sit it out, even though I won't.
Sidenote, I really loved and miss my Asus Eee PC, and that netbook era of laptops. That and my og Motorola droid with the slide keyboard was all i needed (along with my big chucky remote server).
The Magic of Chrome's $0
The New York Times' First Director of AI
PyInstaller runs well in a CI I have, what is this first run experience issue?
Also, Tesla's FSD.
Of course some women buy trucks but they are obviously dick magnifiers for incels. Walk around the parking lot this holiday season. How many of those truck beds are pristine? Or should I say virginal?
I guess there is a distinction but considering how people usually talk, the thing said registers the same, especially as the guy did wrote in the form that he was "biding his time" expecting Twitter to actually die and for everything being done there being uncontroversially bad for the platform. I would differentiate those things the same way you did in a different context.

Reddit was by far the most common platform for me to visit. Overall community there has degraded over the years (more than in places like HN!), not helped by moderation being seemingly all over the place nowadays, possibly affected by the culling. I reported someone for a relatively minor but reportable infraction and their 10 year account got instantly permabanned. All while I had to send appeals 15 days continuously to get my alternate account and subreddit unbanned, with a very reasonable plea that my subreddit is doing the same as another one that has existed for ~7 years, after which it got unbanned for one day of no action.
In most US banks, sending a wire is an "event" since wires often go overseas and so KYC, suspicious transactions and sanctions are implicated. The fact that someone is sending a wire--as opposed to a multinational--is itself evidence of a suspicious transaction.
Does the an open source toolchain exist for capturing, processing, and hosting navigable 3D walkthroughs like this (e.g. something like an open-source Matterport)?