skysight-inc/HackerNews huggingface dataset:
>> Transformers are RASP programs, which includes sorting programs. See the Weiss paper (https://arxiv.org/pdf/2106.06981.pdf).

That's one preprint on arxiv, that makes a wild claim about a new concept that they acronymise as "RASP". It's not any kind of established terminology, nor is it anything but a claim.

What is certainly established is that a function, and an algorithm, are different objects. To clarify, a function is a mapping between the elements of two sets, whereas an algorithm is a sequence of operations that calculates the result of a function and is guaranteed to terminate. Algorithms are also typically understood to be provably correct and to have some provable asymptotic complexity (as opposed, for example, to heuristics) but that's not a requirement.

So for example, if you have a function ƒ between sets X and Y, and an algorithm P that calculates the result of ƒ, then you can give any element of X to P and it will return (in fact, construct) an element of Y. Crucially, ƒ is not P, and P is not ƒ.

Now, when you train a machine learning model, you are typically training a function ƒ̂ (with a little hat) to approximate ƒ. That means that your trained ƒ̂ is a function that maps some of the elements of X to the same elements of Y as ƒ, but not all. It's an approximation. So you get some amount of error, as in your experiment.

So what you've done in your experiment is that you trained a model to approximate a mapping between the set of lists, to itself (where the input list is any of the lists in your training set and the output is the same list, sorted). Your model is not an algorithm, and you cannot train an algorithm with a language model.

I appreciate that, learning an algorithm, is what you wanted to achieve, but in science we don't choose the answer that pleases us, we choose the answer that makes the most sense- and a good heuristic for that is that the answer that makes more sense is the simplest one. Here, in order to convince yourself that you have trained a language model to learn an algorithm, rather than an approximator, you have chosen to rely on a preprint with a completely novel and untested concept that someone put on the internet, rather than the well-understood abstractions of elementary computer science, so not at all the simplest explanation. That is not a good idea. You will not understand what is going on, if you rely on that kind of explanation. I assume you are trying to understand?

Edit: incidentally, you don't need a transformer to train an approximator to a sorting function. You can do that with a multi-layer perceptron, or a logistic regression, certainly with an LSTM. Ceteris paribus, you'll get the same results.

>> The probability of a test list existing in the training set is less than 10^-70.

But the same probability if you held the test set out would be 0, so why not do that? It's not hard to do.

Is there a good reason not to do that?

Btw, lists are composite objects. How much overlap is there between your training and test lists? Do you know?

Edit: meh. HN messes up my nice f-with-hook-and-combining-circumflex-accent. DAAAAANG!!!!
> I see no reason why AI would have any difficulty defining its own reward functions

The first problem is epistemological... If you think that creative decisions are made by complying with a "reward function," you are entirely missing something. Most values are fundamentally based on irrationality. I've literally spent an entire life doing things that everyone else told me was wrong and being interested in things that almost no one else saw the value in, but which ended up being "correct" (for me, at least, and also leading to tangible success). I have no reason to believe that any of my decisions were rational, functional, or acted according to a "reward function"... and I'm a programmer! So I COMPLETELY understand the appeal of the explanatory power of "reward functions." And yet, I can assure you that this is a piss-poor explanation for many creative decisions that literally no one else understands but the person making it, but which then bears fruit despite all reason to the contrary. Some might call this "intuition"
Sure, but lack of safety nets can definitely turn you homeless OR force you to forego crucial medical interventions (in the US at least). For example, my daughter has some developmental delays she might be able to grow out of with the proper therapies. These therapies cost thousands of dollars a month so when my wife and I talk about what the best thing to do for our daughter is, we usually land on "making more money, getting that next promotion", because one of us staying home with the kid makes a number of things become unaffordable.
Looks nifty— basically it's the "let me try that in a container first" except on your live system with no setup to get it going.

That said, as a NixOS user for the last year or so, I think I've gotten a bit spoiled by just not worrying about this kind of thing any more— eliminating the filesystem (other than home and var) as a big ol' chunk of globally mutable state is so liberating.
The most likely answer for what is going on here is the one right on the surface. Prigozhin realized that he could try to take Moscow, but he'd ultimately be defeated, Putin was already gone. He acted without any political support and without having a bunch of aces up your sleeve an attempted coup is destined to fail. It would also drag units away from Ukraine to deal with him and undermine Russian efforts in the face of the Ukrainian offensive. Prigozhin is still fighting against Ukraine and probably recognized that he would go down in Russian history as a traitor. It was very likely just a rash miscalculation by Prigozhin. He's hoping to use this as political leverage now, but my guess would be that he winds up dead pretty soon. I doubt there's any 4 dimensional chess going on.

If this is the case, then this isn't going to be the Russian Civil War/Coup that people had hoped to end the war. Russia is certainly a mess though, and this might cause other political forces in Russia to act.
Doesn't work on popos:

$ try pip3 install libdash
Warning: Failed mounting /boot as an overlay, see /tmp/tmp.BrLiRj0Brb
Warning: Failed mounting /home as an overlay, see /tmp/tmp.BrLiRj0Brb
Warning: Failed mounting /snap as an overlay, see /tmp/tmp.BrLiRj0Brb
/tmp/tmp.c7hp4nI6lE: line 4: cd: /home/user: No such file or directory
American Farm subsidies to over produce > Lots of cheap food pretty much everywhere it even created the school lunch programs we have today > fed/overfed people. We even have billions of blocks of cheese in a cave system because of the dairy subsidies. I'm all for subsidies, except that the people that have farms couldn't survive without it and think they are doing it all on their own and voting regressive even though they likely wouldn't exist without the generations of subsidies their families took.
It was run by such a society - the British. Capitalism was even cited as a justification at the time. If the "market" decided to export food while the Irish starved... well, thats ok. Market knows best.

It essentially mirrored the Holodomor - an occupying power exports all the food from an agrarian colony during a bad harvest. Famine happens. Occupying power shrugs. People wonder if it was a covert attempt at ethnic cleansing. Probably it was.

The difference is that the Irish famine was oh, just a big whoopsie while the Holodomor was an evil plot.
The chatbot they have seems has access to knowledge graph, which is a very good way to make sure access to more updated data.

This means an access to text classifier and NER are needed to create a good graph queries.
> Having roommates in late 30s out of necessity rather than choice

1. It's not as bad as you describe it. Not for tech workers, at least. Please go to the lady working at Walmart and ask her about her income and living arrangements before you rant about $300k a year.

2. Of course it's by choice. Nobody is forced to live in SF or the bay area. Especially not people in tech.
Regarding North Korea, it's likely because the populance is under tight control of a ruthless military dictatorship. During the famines in the early 2000's Germany sent a larger part of it's BSE beef reserves (meat from ~400k head of cattle) [0] to North Korea, under the condition to ensure the issuance of the meat to the civil populance. Otherwise it was feared that this meat would just be confiscated by the military.

As a result certain German media outlets got unprecedented access to North Koreans going about their daily lives, because German humanitarian workers were allowed to freely travel across the country. I cannot find it right now, but ARTE/Discovery Channel produced a documentary where these workers were followed. There were some remarkable scenes in that documentary, it was like someone travelled back in time into the soviet union at it's lowest point. They also interviewed farmers and they put it in no certain terms that they knew they were living in a planned economy, but couldn't to much about it, since there was a constant threat of violence if they retaliated.

[0] https://www.theguardian.com/world/2001/feb/20/northkorea.fam...
You realize is you ban this kind of service, fraud rates are going to multiply 10x in the EU, and new, very inconvenient barriers to fraud will be put in place.

I'd like to see one (1) person harmed by any of the data sharing that TeleSign has done here in the history of the company.
I don't know about drugs. Ego, messiah complex, and delusions of invincibility? Sure.

Putin got where he was through reality-TV-level ostentatious displays of strength. Prigozhin likely figured he was the only one who could boast measurable gains in the war, and the same tactics would work for him.

My guess is his miscalculation was assuming he was more popular among the rank-and-file Russian military and police than he is, and it's too early to tell if that's actually a miscalculation.
i really tried to guess what this would be, and couldn’t. laughed out loud, this is a great idea
Propping up an entire failed county is hard. China does a lot better in NK than the US did in Iraq or Afghanistan.
As a native Russian speaker, can assue you it is exacly the same thing.
At this point this guy will be arrested I think, how many lives were lost and billions of rubles of equipement destroyed due to this?

No blood shed is a lie too if reports of downed choppers and planes are to be believed.

He didn't have the guts or the support to go through, and even announcing of trying to reach Moscow was stupid.

Amongst all this, where the heck is the Russian Air Force? They were planning to reach Moscow via usual roads, would have been very easy for a couple of MIGs to smoke them like the US Air Force did in 1991 in Iraq's Highway to Hell.

This whole war in Ukraine is very bizarre, no Navy, no Air Force, just ground war and drones, it's like the really expensive pieces of equipment are too costly to be risked for Ukraine.
Never heard of a hardware vendor accepting payment in the form of company culture.
The “leave it broken but easily fixed” technique sounds somewhat similar / complementary to this. Apologies for the LinkedIn url, it’s the only reference I could find. https://www.linkedin.com/pulse/leave-broken-easily-fixed-shö...
I've got over ten years experience, working in "highly regarded" tech companies. Most of the time, management is both incapable of recognizing who is truly writing the best code and politically their hands are tied in terms of firing half the team. This is why startups like OpenAI can take on companies like Google. I've seen dozens of products fail because engineers making 400k+ could not handle the complexity of some code base.
You know s1e8! The quote fits so well with the website's intentions.
I only got a few years out of it. I'm running an Arista 7280SR-48C8 now.

The problem is that the million tcam entries are split between IPv4 and IPv6, so I really ran out of space.
On the contrary, "precise rules" make things harder for many people in the community, because the more precise you get, the more attention people pay to them, and the more work they put into getting as far with those precise rules as they can (after all, if the rules are precise, surely it's OK to come right up to their edge, like the railings at a scenic overlook).

One weird subtext of this discussion is the idea that imprecision in the guidelines is costing "the subjects" something. But getting moderated doesn't cost you anything; on the contrary, it costs Dan. You just adjust and move on.
> difference between mean and marginal

I was writing mean rates, not marignal.

For 300K in CA:

- marginal is 35% federal, 9.3% state, 2.35% FICA

- mean is 24.74% federal and 8.06% state, 4.79% FICA.

> I can see that paying for 3 generations on one income can be hard.

Yes, most working class people have to care for 3 generations. Parents being wealthy enough to take care of their own retirement expenses is a small, small number. Working class people in their middle ages choosing to not have kids is also a small number.
>Constitutional rights generally refer to what the government can and cannot do to a person.

So the constitution only protects your human rights(life, privacy, no discrimination) from the government? in USA you need other laws to protect your rights from other people and companies ?
No public statements by officials of any government or organization can be believed. They may or may not be telling the truth. The story will be spun in accordance to whims/wishes of whoever is spinning it. It could be a coup attempt by Wagner or maybe the longtime Putin loyalist is doing this for Putin to see who in the military end up demonstrating a lack of loyalty to Putin. There are lots of possibilities but whatever the truth is it is a fascinating story.

Churchill’s quote seems apt. Russia is a riddle, wrapped in a mystery inside an enigma.
This doesn't hold up because there are a number of ways to invest your wealth to protect it from inflation.
For the downvoters, if you can’t tell this is an AI you’re gonna be in real trouble if these models actually get good.
You are correct in terms of raw talent. And I agree, there are a significant number of NNPPs in many teams who could be culled for a net increase in productivity.

However what this viewpoint doesn't account for are team dynamics. A strong TL can turn NNPPs into incremental positive contributors. A great programmer without leadership capabilities will not be able to outpace the technical debt. There are also more subtle dynamics depending on the structure and personality traits of the individuals. Ultimately programmer productivity is not an absolute value, it depends on the whole ecosystem (including other functions, leadership stance, etc). After doing this for 25 years (IC, TL, EM, CTO), I strongly believe a healthy team is about harnessing and orchestrating different individuals unique strengths rather than trying to set too high a bar—the latter will lead to counter-productive competition and ultimately burn out your best folks.
This headline and press release is misleading.
In internet comments people think "Constitutional rights" means "Stuff that I want."
What performance? That’s not a single thing. Do you pay in throughput or latency?

It certainly has a price but it is waaay too overblown in many discussions. What it mostly does entail is a slightly larger p99 latency. Where it actually matters is entirely another question.
Reminder: https://githubcopilotlitigation.com/
I'm going to skip over telemetry and config resetting, I think everyone hates these.

But as for dazzling new changes, the enterprise market probably doesn't love it but it's candy for consumers, judging from the hype around Apple's announcements. They can definitely drive sales. There are downsides to having the same product in both positions, it appears.

But Microsoft often flubs the execution whoever the changes are for.
Of course not. Communication is still important, and you discuss and plan features that need to be implemented. What you don't do is hover over their shoulder and monitor how they solve problems.
Calling in orders was unthinkable most times, unless I knew that the business was accustomed to delivery orders anyway. But usually, there was just no need or reason for me to call in ahead for orders. (I don't own a car, so I certainly wouldn't be picking up, always dining in.)

However, ordering pickup food online is an amazing service now. I picked up this habit quite recently, when I realized that Google Maps makes it super easy. They present your choices between GrubHub, or the restaurant's own website, or other services. So what I do is get on the bus, then when I'm about 20 minutes away, I'll just open up my phone and order ahead. Credit card payments are no problem; I often use PayPal with GrubHub, or Google Pay.

Calling in an order on the bus would be unthinkable, since I would not be reciting my credit card details by voice in a public space. But online ordering makes it a piece of cake.

This Toast fee may be affecting me soon; I recently started patronizing a coffeehouse that uses Toast, and ordering ahead seemed like a really good idea. We'll see.
I'm familiar with the widespread practices of how AI models are trained, yes. There's an implicit "and we must be able to do this" in your argument, which is not at all evident. "But what about AI" is not an argument that suddenly it's OK to violate Open Source licenses or the licenses of small copyright holders (e.g. of online posts).

I'm well aware that this is widely done by large and small entities alike; I'm not just concerned about the practices of large companies, I'm also concerned about small ones, and individuals.

This is not "more draconian" copyright; this is not a change to copyright at all. This is the same copyright we already have, equally enforced for all copyright holders, large and small. You want to "democratize" this space? Get rid of copyright, and a lot of things become better and easier, not just AI.

I don't want the dystopia where copyright still exists for large publishers and studios and proprietary software companies to prevent sharing and remixing things, but at the same time all the small and Open Source entities don't get to set their own terms because AI will just remix them away.

Do you think you'd get away with training an AI on a bunch of animated Disney movies, and asking it to generate new images in that style, and using the result in commercial endeavors? Or is it just the myriad of smaller copyright holders, like independent artists, that you're comfortable stepping on?

> It's not going to stop OpenAI. It's not going to stop Microsoft. They'll figure it out.

Will they? If there isn't, in fact, a legal solution, they're not in any better shape than anyone else. If anything, they're in a worse position, because they have deep pockets for potential lawsuits, while non-commercial efforts tend to not be interesting targets to sue (at most, they get shut down, and others pop up in their place).

Question your assumptions about the world that results from requiring AI to respect Open Source licensing and other small copyright holders such as independent artists or online comment/story authors. It's not a corporate dystopia. It's a level playing field.
How do you find yourself not saving enough for early retirement when you are saving $8k per month? Only a a tiny fraction of your condo is being consumed within a month. Far less than 8k.
> contains nothing that is scientifically falsifiable

It's observable. You simply ask people what they have been or are thinking or sensing. If they are able to relate that, then, by the definition, they were conscious about it. Of course they might be lying or misremember; introspection is not intersubjective, and therefore hard to create reproducible experiments around. But that doesn't mean introspective information doesn't exist.

> says nothing about the phenominalogical experience of consciousness in a meaningful way

What kind of experience is observable, except by introspection?

The only observations of experiences that anyone can have are through either one's own introspection, or through what others have related of their introspection.

If you are to say that you have had a "phenomenological experience of consciousness" that you want me to explain, then you are giving an introspective report of a brain process that left an imprint that you can now describe. If the experience was not available to introspection, then you wouldn't be asking, because you wouldn't realise there was an experience at all.
If you can get within 120 kilometers of the Kremlin while tripping balls, then the problem is not the drugs.
A fact I havent seen posted about this incident, initially the reports stated he had been stabbed through the back and the front of his torso. Later these reports were retracted and never mentioned again.

Also there were 2 Freano police officers present during Lang's autopsy, from start to finish. These are 100 percent facts, irrefutable. These facts may allow for others to better decide what they think happened to John Lang.
Why would be hard, they can put them to work in factory to do cheap shit and give them food.
Price controls work perfectly well as long as the government implements the necessary policies to backstop the market, whether that means nationalizing the suppliers and eating the difference in cost or subsidizing private companies. Venezuela and North Korea just don't have the foreign currency reserves required to do that.

Tons of countries like China and the United Kingdom have energy price controls. Rental price controls are everywhere around the world, both good and bad implementations abound.

Lots of countries have put price control on agricultural staples like rice and wheat. It's a relatively cheap way of guaranteeing social stability since the difference between the market price and the set price is relatively small and predictable per person, except in emergency situations like the world experienced with Russian gas and Ukrainian wheat. The best governments squirrel money away for rainy days like that, so that the price controls don't fall apart.
There will be multiple persons that didn't get the loan or mortgage they applied for, as their reputation score was deemed to low due to data quality issues.
Russian MoD was trying to bring Wagner troops under their direct control.
And what about unchecked agression in Iraq?
I've been following this on Twitter, and so far as I can tell it's a bar fight between a couple of crazy old guys who both happen to have an army.

Prigozhin's rationale for this episode seems to be intense annoyance that he wasn't getting respect.

And now - as he believes - he's got that after a show of strength, everyone can go home and pretend it never happened.

It's insanity. Just bonkers. All of it.

Some Ukrainians I know tell me there's local suspicion that was a feint or a ruse to see where Ukraine would attack. It's an interesting idea, but that seems a lot of effort and drama for a very small reward.
Sure, Red Hat may be obligated to distribute everything GPL/within kernel space.

But how about patches for packages licensed in a more permissive license, like MIT?
I take it back, I didn't know it was that bad in the US. I guess I only speak from my bubble as a single, low paid SW dev in the EU. Having kids that need expensive medical expenses is something I didn't think about, because one, I don't have kids, and two, it's normal here to have most critical medical expenses covered, although, even here I still have plenty out of pocket costs due to how thinly stretched the public health system is, so you get the care you need for free, IF you happen to be dying right NOW, otherwise it could mean suffering for months long waits and/or out of pocket fees. Still better than the US probably.