skysight-inc/HackerNews huggingface dataset:
The key differences between multi-head attention in Transformers and the proposed multi-scale retention (MSR) in RetNets are:

- Retention replaces the softmax in attention with an exponential decay along the sequence dimension. This allows formulating retention in a recurrent form for efficient O(1) inference.

- Retention heads use different decay rates (gamma values) for multi-scale modeling. Attention heads use the same softmax.

- Retention outputs are normalized per-head with GroupNorm before concatenation. Attention uses LayerNorm on the concatenated output.

- Retention can be computed in parallel, recurrent, or chunkwise recurrent modes. Attention is only parallel.

- The recurrent form enables RetNets to summarize long previous context into a fixed-size state during inference. Attention recomputes on the full context each step.

- So in summary, retention adapts attention to enable recurrent modeling and multi-scale decays. This provides efficiency benefits and competitive performance to Transformers.
The type of pedantry rife in ChatGPT imploring one to accommodate to the opinion of sanctioned views, aka "..and they are generally not accepted by mainstream historians, archaeologists, and scientists."

Take one turn towards curiosity or speculation and BAM, no outlier thinking!
this is rly awful "research". ederer prob got bullied on there and got mad, cause he has a reputation as a bitch.

more importantly, ejmr has been important in uncovering multiple cases of research fraud (including one of the literal damn authors, this is some vindictive ass bs) and is the best source for actual unfiltered info on opinions of econ departments. includes important info like info abt people's political biases, potential toxic departmental cultures (or rather, which are more or less toxic, this being econ), and even info on sexual harassment allegations that depts would rather cover up. chilling effect of stripping anonymity is awful.
Also you don't have to either, you really only have to make airplanes, ships, weapons and missiles, everything else is kind of unnecessary if you have those. As long as you have the bigger stick nothing else really matters, of course you'll be hated by everyone else but that's not really a problem. You just call that stick "liberalism" and call it a day.
Going back to your original post:

“ We can house feed and care for every person on the planet but we don’t because money. A made up concept that has no actual value outside of what we place on it.”

Try do anything you are talking about without the involvement of money or barter. It ain’t going to happen.

No one is going to work in a crappy job if they can avoid it.

Who is going build new properties or repair dilapidated buildings without money or some other incentive?

Why would anyone build more than they personally need?

Who is going to build/repair/modify homes for disabled people to live in without financial incentive?

You need money for all of this.
Personally I’d never buy a car made in China. Related, I find it pretty strange Volvo is trying to establish a luxury brand (Polestar) but instead of building Polestars in Sweden they are building them in China.
I can't believe the level of whataboutism in quick replies to this post. Have you considered maybe if policing authorities had been held to account in the first place, then these protests wouldn't have existed at all? You don't even have to think that far back to see it in action. The Ferguson protests happened in 2014. The executive had a measured response that addressed the issue, not even fixed, just addressed.

When public peaceful protests happened between then and now those groups were told to fuck off by the president and vice president. "Get that son of a bitch off the field". There's your catalyst, not groups of people who were so motivated by bad policing they went out on the streets to protest. The executive in power that was made aware of an issue, and said we not only reject it, but we will antagonize you for bringing it up.

These people were subject to violent policing that they were protesting. The irony is palpable, but we see it get excused as just political hacks egging them on, or it was justified because of adjacent activities.
by that logic, the SAGE computer from 1957, being the biggest computer ever built, should be the one to run it on.
I hope the day comes when we look upon the 100th birthday the way we now look at a second birthday.
I tried that (primarily because I didn't know about it and was excited), but it didn't work. It needs me to install "lighttpd" to work. And the steps to install that are not straightforward.

I like that a lot of functionality is not bundled up in the git that I have already installed in my computer, but at the same time, I agree with the fact that adding these separate binaries is not easy as a user.
If they are in a consolidated market they will just tell you to go ahead and chargeback…they’ll deny list you and make you pay a big fee when yo I get tired of their (lack of) competitors and come crawling back.
WikiHow:

How to make cheese: https://www.wikihow.com/Make-Cheese

How to make Feta cheese: https://www.wikihow.com/Make-Feta-Cheese

How to make goat cheese: https://www.wikihow.com/Make-Goat-Cheese

These are the ones with quality/expert badges; there are more where that came from.
I thought they both started at the same time? Unless you are implying early versions of git were worse, which wouldn't surprise me.

I know git has a learning curve, but the only times I really see junior people have issues is when they are trying to do things that wouldn't have been easy with earlier VCS systems either. And it seems like git isn't going away, so it's worth putting the time into learning it.
He got me started down the rabbit hole. Now I have a cheese cave, homemade cheese press and a fridge full of cultures. Still haven’t mastered mozarella though…
IMHO, for software developers, a version management system is a tool, just like an IDE, or the underlying OS. The less they need to fiddle with these tools, the more productive they are.

Having worked with multiple version management systems, for me, Fossil hits the perfect sweet spot. You literally just set it up and forget about it.
tim dettmers is such a resource, cheers for this
So what's next? Get these people fired from their jobs for having unapproved ideas? Also, who decides what is or is not toxic?
Matrix as an alternative backend for things you might use Discord, phpBB or ActivityPub for. Really neat idea, I hope it works out and makes "Matrix but you don't have to know or care that it's Matrix to use it" a viable strategy.

It's a very neat and tidy strategy to avoid having to reimplement the hard parts of Discord (global scale messaging and integrations) by using Matrix.

I'm surprised they're currently organizing on Discord instead of Matrix, really shows how much being on Discord matters for attracting community.
good point, for some reason i always leave out rwkv when thinking of the transformer models.. perhaps because it is more of a redux
Humans are certainly the absolute kings of tool usage. Examples in the animal world that we find impressive are things that even a human infant will trivialize in short order. And if you think about it in this context, communication itself is really just another tool. It's taking something that's not particularly useful, the ability to make the same sounding grunt repeatedly, and turning it into something useful.

Any animal that can move its body in a consistent way, make the same noise repeatedly, or do any other sort of repeated 'thing' has the ability to create an arbitrarily complex language. But they lack something enabling them to do that, and it seems whatever this something is - it probably wouldn't be terribly far from the same thing that enables meaningful, complex, and intricate tool creation.
I don't think it takes doing anything wrong for diffuse spatial patterns to become well-defined when you substantially compress the space. What's dispersed across quite a few pixels in the full-size is now condensed into an easily discerned sharper edge of pixels.

If I defocus my vision, kind of like when looking at those old "MagicEye" 3D-like images, I can see the words pretty well in the full-size...

In the full-size there's a bunch of high-frequency noise (like building windows) interfering, but that's necessarily lost in the down-scaling, and now the low-frequency information forming the name is clearly visible.
There was also a time when my computer playing mp3 in winamp while trying to browse the web would glitch the music due to buffer underun. Then I guess faster computers came along...
I’m more concerned about the Chinese software. I rather like the Volvo EVs.
What's the modern version of WASTE? It seems to me that the progression from Twitter & Facebook to Discord probably ends in something similar to WASTE, especially if it includes voice chat and video streaming.
Counterpoint: Lots of people actually use and like the D aspect. It’s one of the best things about Git!

Local git repos can be shallow or partial clones if you insist on it. You say elsewhere that a VCS should support an offline mode instead, but if that’s not a copy of the repository, how is it in any way equivalent and what exactly is the difference?

I’ve never understood why people like Perforce; I’ve assumed it’s some kind of weird nostalgia for mainframe user interfaces. I understand that it handles some kinds of large assets better than Git, but Perforce is so bad at everything else.
It's the potency and accessibility of drugs, now you can get fentanyl or meth that are dozens of times more pure and cheaper and available all over with virtually zero consequences. If you want to use drugs, there are no cops or doctors who are going to intervene and your family can't just force you into rehab. This new reality is simply sum of the cumulative bad decisions we've made--get rid of mental institutions, make health care unaffordable, tie insurance to employment so mentally ill and drug addicts have a harder time getting access to treatment, the breakup of the nuclear family, more females working and less time to raise their kids, decriminalize drugs, decriminalize sleeping outside in public places, decriminalize theft... this is the end result.
It is also much harder to make a good aged cheese then a good fresh cheese. There's a lot more chance for bad bacteria to contribute a bad flavor or spoil the cheese if you give it more time.

Speaking as the son of a small commercial cheesemaker, who started out (~23 years ago) making cheddar that only stayed good for about a week in the fridge. Now we make aged cheddar.
These is exactly what my first thought was! Installing open-ssh and git is kind of my first activity on a new computer.
But I believe they are talking about setting up a git server. The thing that we rely on GitHub/GitLab/Bitbucket for. Richard counts them as an added dependency.

(I mean, he is not wrong in being worried about that. You are basically giving your all your code to a company. Which does not matter for open-source projects like SQLite, but it does for many private projects with code as their primary IP)
That's neat! There really is a XEP for everything, huh.
These are not mutually exclusive ideas and replacing an existing device is much harder to detect.
When I was a kid growing up in the '90s, there was only one visibly homeless person and he was just a schizophrenic guy who pushed a tv around in a shopping cart. Now, there are tent cities and visibly mentally ill using drugs openly in most cities on the west coast. Something has definitely changed.
But maximum efficiency does not scale linearly with organisational size/product complexity. So yes, lots of wastage, but still possibly the least wastage of other tried-and-tested organisational models.

Also, effectiveness of software relates to the attributes that align to the dimensions of "goodness" as defined by each user not the developer. The software does not have to be the best (among its peers) to be the most effective for me. The same software may be completely ineffective for you, if what you want out of it is different to what I want.
The choice is actually between dealing with complexity and shifting responsibility for that to someone else. The tools themselves (e.g. virtual environments) can be used for both. Either people responsible for packaging (authors, distribution maintainers, etc.) have some vague or precise understanding of how their code is used, on which systems, what are its dependencies (not mere names and versions, but functional blocks and their relative importance), when they might not be available, and which releases break which compatibility options, or they say “it builds for me with default settings, everything else is not my problem”.
Last time I needed to traverse millions of files the fastest was 9yo https://github.com/MichaelTJones/walk

I wonder how this compares in speed.
He’s going to put flames on the Twitter logo, phoenix away all the debt.
the words per second i believe from the first graph in the paper
Every time this comes up, mostly people talk about driving less, eating less meat, etc. How about we fix stuff instead of throwing it out? In early 2000s my parents bought a washer machine, it still runs, it does run pretty well, then had to repair heating element at least once, but it runs just fine even now. It’s has every cycle you can possibly have. We had to replace our bosch after about 8 years of use. I can’t buy spare parts. Had to replace drain pump twice, not there is a hose that cracked and I just can’t buy a replacement with matching connector. Similar stories for a microwave, fridge, dishwasher, laptops and even vacuum cleaner.

We need to force manufacturers to make things fixable. And mandate certain level of support for spare parts and batteries.

Same with all types of connectors. I wish the US passed the passed the law requiring common charging port, common audio port(audio jack was perfect), etc. Audio jack removal by Apple and subsequently Google is an atrocity.
This started in the 60s, when manufacturing started moving to Japan. "Made in Japan" was the mark of garbage (I inherited some of those old Japanese-made tools from my Dad. I only keep them for nostalgia reasons.)

Then in the 80s Japan got very good at manufacturing. And they got expensive as a result. So cheap manufacturing moved to Taiwan, and "Made in Taiwan" became the mark of garbage.

Now Taiwan has become a very good manufacturer, but it's expensive, so cheap manufacturing moved to China.

And now high-end Japanese brands outsource their manufacturing to...China.
Yes, but I don't think you can see them without being logged in. The argument is that it doesn't disable XSS by default. Here is the example video https://twitter.com/nsjames_/status/1682846144118902788 or hypothetical scenario in text form https://twitter.com/nsjames_/status/1682834706671235072

Responses are mostly negative even though the docs seem to agree. https://twitter.com/gurleen_s_/status/1682815674828111872 Commenters say that many frameworks do similar things and that defending against XSS is up to the developer.
Not a fan of this name. Dbag. Why not datasack? /s
We had a version (not in USA) of an Oppenheimer movie.

They really, really played up the angle: The entire atmosphere might ignite.

The last scene was at the test site, and the last few seconds were just all white in the frame.

I suppose in part this saved on production cost.

For years I was left wondering: "Did the entire atmosphere burned up?"
How do you ensure what the LLM generates is legally accurate?
We used no-code tools(Retool) to build our Admin interface to administrate our App, e.g manage users, products etc.

But for anything novel and complex no-code tools simply don't suffice, you can start with them for an MVP or example app but you'll very quickly have to move to coding it up fully custom(in my experience atleast).

When you go from no-code -> MVP you will likely have to throw away most of the no-code version and rebuild from scratch, so it is a question of how far can no-code get you and does it make sense for your use case?

So, it depends, what are you building?
Paw Patrol’s dark secret explained [video]
It turns out people really don’t like Paw patrol and I’m starting to understand why!
Here's a video from Beta, a wrench manufacturer in Italy.[1] Think about what it would take to build a plant like that from a cold start. What you'd need to know. All the ways you could screw up. How much it would cost to recover from a bad decision. Notice what they automated, and what they didn't.

Who wants to be a millwright? Or a production engineer? To build a factory, you need both. Once you've built the factory, you don't need most of them them any more. It's a job with high layoff potential. Top pay is maybe $125K.

You need people who've built your kind of factory before. Here, they need people who've built mechanized hot forging shops. Since few of those are built in the US each year, there's no pool of such people.

Stanley, the company, had forgotten how to build a plant like that.
They relied on the equipment manufacturer, and perhaps some consultants. That didn't work out.

It takes a long time to get a good factory going from a cold start. It took Airbus 21 years before they made a good airplane. It took Tesla 15 years from start to significant production volume.

[1] https://www.youtube.com/watch?v=azDUFQ-e0EM
I was once responsible for doing this with a Perforce server. Only time in my career a VCS has lost data. This was decades ago now and the data loss might have been due to hardware; I cannot definitively blame Perforce, but man was it shitty to deal with at the time. We migrated to Subversion and never had another issue.
> I think coming up with a comprehensive mental health policy is a little beyond the scope of a HN comment

True, but that's clearly not what I asked for.

> If there are, we already have an example in homosexuality of how we can stop considering them mental illnesses. It's clearly a thing that's possible to do, so if necessary, let's do that.

Also true, but (1) our understanding of what should be treated as a mental illness can regress, and (2) that's not much solace for all the people harmed between now and when we as a society agree that they shouldn't be considered mentally ill.
The point was that systems are multi-faceted, complicated, nuanced, and work by mechanisms and principles that are not obvious. Experts have a really hard time nailing down a working definition of intelligence to begin with, even when given thousands of pages to describe it - the idea that such an elusive thing is simply, directly correlated to the size of an object, is frankly, fairly unbelievable.

It used to be used all the time. For instance, to defend the idea that women and asians weren't as intelligent as anglo saxons because they were smaller in stature. There's plenty of other ludicrous nonsense claims that were made based on this supposed principle.

We all agree that the conclusions are nonsense but somehow we hold on to the premise.
Hi, I am the author of ANS and of these diagrams you don't like - made more than a decade ago. Sure, animations are nicer, but you cannot put them into articles ... and indeed somebody else could make them through this decade.

Regarding the Standard Model, its development is mostly new experimental surprises and fitting new corrections to the Lagrangian - currently more than a hundred. It would be great to see them as kind of Taylor expansion of some simpler more fundamental model, not guessing but deriving new terms.

Where to search for such more fundamental model? Maybe in topological defects like recently growing in popularity skyrmion models. For charge quantization: by interpreting curvature of some deeper field as EM field, Gauss law counts topological charge - which has to be quantized.

I am developing this kind of approach since 2009, and seems quite promising, e.g. naturally unifying EM+QM+GEM gravity vacuum dynamics, has 3 leptons, baryons e.g. with proton lighter than neutron ...

Slides with links: https://www.dropbox.com/s/9dl2g9lypzqu5hp/liquid%20crystal%2...

Invited Wolfram introductory article "Framework for Liquid Crystal Based Particle Models": https://community.wolfram.com/groups/-/m/t/2856493
>>And you are ok with Governments stepping in to prop up competition to Monopolies. Guess that is a different way of doing it. Instead of breaking up monopolies to foster competition, seems you are arguing that it is better to allow monopolies to exist (to be free to do what they want), but to help keep a free market the government can step in and support their competitors until they can compete? Not sure how well that would work in practice. I think when the US had had the stomach to break up monopolies it has worked out better for the free market, because it keeps competition. I don't think AT&T and Standard Oil would have been taken down by more efficient rivals. That is the entire problem with monopolies, they don't have to be efficient.

Yes, I want the government to prop up competition to monopolistic entities, and I want that competition to become the new monopolist. The difference would be that that government-propped-up competition would be an open standard or a decentralized protocol —think TCP/IP, or a public blockchain— that doesn't require active management by the state. Once it has become the market standard, it is self-sustaining, and more efficient than any private company would be in that role, since it doesn't have a proprietor that would abuse its monopolistic position to extract economic rent.

>>The Free Market created Unions. If workers could actually compete for fair wages, their wouldn't be the need for unions.

Absolute nonsense. The "free market" doesn't create laws that make it illegal for an employer to refuse to employ unionized or striking workers. Greedy humans, who have no respect for the free market, do. And yes, people can convince themselves that they're being treated unfairly, and that they are entitled to use the state's apparatus of violence to restrict their fellow man's freedom to contract, for reasons of greed. Just because they're workers, doesn't mean their grievances are automatically legitimate.

>>Almost all of the 'freedom's people take for granted today were from unions fighting for them. Weekends? 40 hour weeks? Vacation? Do you think the free market would have provided those? The free market treats humans as cattle.

Unions destroyed or severely handicapped nearly every industry they took control of. The mandated benefits they encouraged to be imposed are immoral, in restricting the freedom of contract. As for improving work conditions and wages, that was already happening —and at an even faster pace— before industry was captured by unions.

Detroit had the highest capita GDP in the US in 1950, and that prosperity emerged over the course over several decades where unions were a non-factor in the auto industry. 50 years after the UAW took over the US auto industry, and Detroit was a ghost town.

Anecdotal, yes, but entirely typical.
All that is because of lack of understanding of underlying physics. Standard model is used as substitute to physics, which results in explanations like that: "there is no Everest (physics), it's just excitement in the height field (model). Gravitation (physics) bends space-time (model), so students just follow the shortest path.", and so on.
As someone in his position 20+ years ago who did a lot of stretching and yoga for recovery, I can attest that this will only take you so far. Real gains come when you lift weights, which I started at the age of 49. The deadlift is a particularly good remedy for lumbar pain. Start on low weights with good form. Programmers develop weak lumbar support and stretching weak muscles can bring short term relief, but will not resolve the underlying issue. In time, stretching no longer becomes effective without over-stretching, which brings its own problems. Building that muscular support is key.