trentmkelly/LessWrong-43k huggingface dataset:
Forget Everything (Statistical Mechanics Part 1)
by J Bostock — 2024-04-22T13:33:35.446Z

EDIT: I somehow missed that John Wentworth and David Lorell are also in the middle of a sequence have written one post on this same topic here. I will see where this goes from here! This sequence will continue!

Introduction to a sequence on the statistical thermodynamics of some things and maybe eventually everything. This will make more sense if you have a basic grasp on quantum mechanics, but if you're willing to accept "energy comes in discrete units" as a premise then you should be mostly fine.

The title of this post has a double meaning:

Forget the thermodynamics you've learnt before, because statistical mechanics starts from information theory.
The main principle of doing things with statistical mechanics is can be summed up as follows:

Forget as much as possible, then find a way to forget some more.

Particle(s) in a Box

All of practical thermodynamics (chemistry, engines, etc.) relies on the same procedure, although you will rarely see it written like this:

Take systems which we know something about
Allow them to interact in a controlled way
Forget as much as possible
If we have set our systems correctly, the information that is lost will allow us to learn some information somewhere else.

For example, consider a particle in a box.

What does it mean to "forget everything"? One way is forgetting where the particle is, so our knowledge of the particle's position could be represented by a uniform distribution over the interior of the box.

Now imagine we connect this box to another box:

If we forget everything about the particle now, we should also forget which box it is in!

If we instead have a lot of particles in our first box, we might describe it as a box full of gas. If we connect this to another box and forget where the particles are, we would expect to find half in the first box and half in the second box. This means we can explain why gases expand to fill space without reference to anything except information theory.

A new question might be, how much have we forgotten? Our knowledge gas particle has gone from the following distribution over boxes 1 and 2

P(Box)={1 Box 1 0 Box 2

To the distribution

P(Box)={0.5 Box 1 0.5 Box 2

Which is the loss of 1 bit of information per particle. Now lets put that information to work.

The Piston

Imagine a box with a movable partition. The partition restricts particles to one side of the box. If the partition...
On Llama-3 and Dwarkesh Patel’s Podcast with Zuckerberg
by Zvi — 2024-04-22T13:10:02.645Z

It was all quiet. Then it wasn’t.

Note the timestamps on both of these.

Dwarkesh Patel did a podcast with Mark Zuckerberg on the 18th. It was timed to coincide with the release of much of Llama-3, very much the approach of telling your story directly. Dwarkesh is now the true tech media. A meteoric rise, and well earned.

This is two related posts in one. First I cover the podcast, then I cover Llama-3 itself.

My notes are edited to incorporate context from later explorations of Llama-3, as I judged that the readability benefits exceeded the purity costs.

Podcast Notes: Llama-3 Capabilities

(1:00) They start with Llama 3 and the new L3-powered version of Meta AI. Zuckerberg says “With Llama 3, we think now that Meta AI is the most intelligent, freely-available assistant that people can use.” If this means ‘free as in speech’ then the statement is clearly false. So I presume he means ‘free as in beer.’

Is that claim true? Is Meta AI now smarter than GPT-3.5, Claude 2 and Gemini Pro 1.0? As I write this it is too soon to tell. Gemini Pro 1.0 and Claude 3 Sonnet are slightly ahead of Llama-3 70B on the Arena leaderboard. But it is close. The statement seems like a claim one can make within ‘reasonable hype.’ Also, Meta integrates Google and Bing for real-time knowledge, so the question there is if that process is any good, since most browser use by LLMs is not good.

(1:30) Meta are going in big on their UIs, top of Facebook, Instagram and Messenger. That makes sense if they have a good product that is robust, and safe in the mundane sense. If it is not, this is going to be at the top of chat lists for teenagers automatically, so whoo boy. Even if it is safe, there are enough people who really do not like AI that this is probably a whoo boy anyway. Popcorn time.

(1:45) They will have the ability to animate images and it generates high quality images as you are typing and updates them in real time as you are typing details. I can confirm this feature is cool. He promises multimodality, more ‘multi-linguality’ and bigger context windows.

(3:00) Now the technical stuff. Llama-3 follows tradition in training models in three sizes, here 8b, 70b that released on 4/18, and a 405b that is still training. He says 405b is already around 85 MMLU and they expect leading benchmarks. The 8b Llama-3 is almost as good as the 70b Llama-2.

...
Motivation gaps: Why so much EA criticism is hostile and lazy 
by titotal — 2024-04-22T11:49:59.389Z
Link: https://titotal.substack.com/p/motivation-gaps-why-so-much-ea-criticism

Disclaimer: While I criticize several EA critics in this article, I am myself on the EA-skeptical side of things (especially on AI risk).

Introduction

I am a proud critic of effective altruism, and in particular a critic of AI existential risk, but I have to admit that a lot of the critcism of EA is hostile, or lazy, and is extremely unlikely to convince a believer.

Take this recent Leif Weinar time article as an example. I liked a few of the object level critiques, but many of the points were twisted, and the overall point was hopelessly muddled (are they trying to say that voluntourism is the solution here?). As people have noted, the piece was needlessly hostile to EA (and incredibly hostile to Will Macaskill in particular). And he’s far from the only prominent hater. Emille Torres views EA as a threat to humanity. Timnit Gebru sees the whole AI safety field as racist nutjobs. In response, @JWS asked the question: why do EA critics hate EA so much? Are all EA haters just irrational culture warriors?

There are a few answers to this. Good writing is hard regardless of the subject matter. More inflammatory rhetoric gets more clicks, shares and discussion. EA figures have been involved in bad things (like SBF’s fraud), so nasty words in response are only to be expected.

I think there’s a more interesting explanation though, and it has to do with motivations. I think the average EA-critical person doesn’t hate EA, although they might dislike it. But it takes a lot of time and effort to write an article and have it published in TIME magazine. If Leif Weinar didn’t hate EA, he wouldn’t have bothered to write the article.

In this article, I’m going to explore the concept of motivation gaps, mainly using the example of AI x-risk, because the gaps are particularly stark there. I’m going to argue that for certain causes, the critiques being hostile or lazy is the natural state of affairs, whether or not the issue is actually correct, and that you can’t use the unadjusted quality of each sides critiques to judge an issues correctness.

No door to door atheists

Disclaimer: These next sections contains an analogy between logical reasoning about religious beliefs and logical reasoning about existential risk. It is not an attempt to smear EA as a religion, nor is it an attack on religion.

Imagine a man, we’ll call him Dave, who, for whatever reason, has never once thought about th...
Should we break up Google DeepMind?
by Hauke Hillebrandt — 2024-04-22T09:16:31.201Z

Regulators should review the 2014 DeepMind acquisition. When Google bought DeepMind in 2014, no regulator, not the FTC, not the EC's DG COMP, nor the CMA, scrutinized the impact. Why? AI startups have high value but low revenues. And so they avoid regulation (and tax, see below). Buying start-ups with low revenues flies under the thresholds of EU merger regulation[1] or the CMA's 'turnover test' (despite it being a 'relevant enterprise' under the National Security and Investment Act). In 2020, the FTC ordered Big Tech to provide info on M&A from 2010-2019 that it didn't report (UK regulators should urgently do so as well given that their retrospective powers might only be 10 years).[2]

Regulators should also review the 2023 Google-DeepMind internal merger. DeepMind and Google Brain are key players in AI. In 2023, they merged into Google DeepMind. This compromises independence, reduces competition for AI talent and resources, and limits alternatives for collaboration partners.

Though they are both part of Google, regulators can scrutinize this, regardless of corporate structure. For instance, UK regulators have intervened in M&A of enterprises already under common ownership—especially in Tech (cf UK regulators ordered FB to sell GIPHY​).

And so, regulators should consider breaking up Google Deepmind as per recent proposals:

A new paper 'Unscrambling the eggs: breaking up consummated mergers and dominant firms' by economists at Imperial cites Google DeepMind as a firm that could be unmerged. [3]
A new Brookings paper also argues that if other means to ensure fair markets fail, then as a last resort, foundation model firms may need to be broken up on the basis of functions, akin to how we broke up AT&T.[4]
Relatedly, some top economists agree that we should designate Google Search as 'platform utilities' and break it apart from any participant on that platform, most agree that we should explore this further to weigh costs and benefits.[5]
Indeed, the EU accuses Google of abusing dominance in ad tech and may force it to sell parts of its firm.[6]
Kustomer, a firm of a similar size to DeepMind bought by Facebook, recently spun out again and shows this is possible.
Finally, DeepMind itself has in the past tried to break away from Google.[7]

Since DeepMind's AI improves all Google products, regulators should work cr...
What should our containers do?
by Richard Henage — 2024-04-22T06:17:04.017Z

If we don't have free will, we're just minds stuck inside (biological) robot bodies and brains. Some argue that this would mean that there's no point in trying to determine which courses of action are right, or what facts are true, since we wouldn't have control over how we behave or what we believe. In other words, we can ignore the possibility of being stuck in robot bodies because if it's true, nothing matters anyway. Let's call this the "unimportant possibility" conjecture. It's like assuming that there's not something weird like a false vacuum bubble wall hurtling towards us since there'd be nothing we could do anyway to stop it from destroying Earth almost instantly[1].

But that's a confusion of levels. Sure, it doesn't matter what I, Richard's experiencing mind, thinks about free will. But it sure matters what Richard's brain thinks and how it makes Richard's body behave. Richard could end up hurting people if he started thinking the wrong things, and that would cause their minds to suffer. And since all the convincing and arguing that goes on in the world happens between these brains and bodies, the "unimportant possibility" conjecture doesn't apply. Richard's brain should use its body to type words and try to explain this idea to other brains (and their minds will listen in). Imagine you're watching a movie, and someone is trying to convince the main character that they have no free will. You may be chanting "don't listen, you have free will!" under your breath. It doesn't matter what you think as the viewer (it will have no impact on the plot of the movie), but it does matter what the character thinks.

Moreover, it could be important for all these brains and bodies to understand that there are minds housed inside of them that have feelings, and are ultimately what matter. For example, maybe science could one day unlock the ability for minds to send messages back to brains. And this discovery would only be found and utilized effectively once the brains understand that these minds exist as separate entities. It's actually rather defeatist to assume that there will never, in the billions of years of the future, be a way for minds to speak back.[2]

There are lots of other ways this knowledge of the nonexistence of free will could matter, especially if the link between brains and minds isn't perfect. It could turn out that in some when the brain experience...
Goal oriented cognition in "a single forward pass"
by dxu — 2024-04-22T05:03:18.649Z

The below is me (habryka) and dxu talking about a shortform that dxu had published a few months ago, going into the relationship between goal-oriented cognition and the myopic nature of current large language model training setups. Some key quotes if you don't want to read the whole thing:

I think it's interesting to think about how brains get around the issue [of having limited context]. Obviously the patterns in which biological neurons fire don't carve cleanly into "forward passes" the way ANNs do, but ultimately there's only so much computation that can occur in the brain within some (small) unit of time. In that sense, I think I want to claim that the brain's ability to engage in something that looks a whole lot like long-term, goal-oriented reasoning clearly can't depend on being able to hold that entire context in memory to be attended to, even selectively.

When I think about the internal experience of problem-solving, there's a mental move of going up and down the ladder of abstraction, where you zoom in on some particularly difficult and/or confusing part of the problem, solve it, and then use what you learned from that to zoom back out and fill in a gap in the larger problem you were trying to solve. For an LLM, that seems like it's harder, and indeed it's one of the reasons I inside-view suspect LLMs as-currently-trained might not actually scale to AGI. (Though there are obviously some terminological issues surrounding "AGI" at the moment, what I mean by that is something like the classical concept of "something better than humans at pretty much every cognitive task humans can do".)

An LLM doesn't learn how it can best reason within a 2048-token context. The human cognition that it is imitating has been shaped by lots of feedback that propagates back through multiple tokens. The human has learned to avoid cognitive traps and routes around the things that would cause the human to go off track. But if there is something that would cause the LLM to go off-track on a task, it will just do it again and again every time.

dxu
Okay, but doesn't that also suggest that an LLM trained on human-generated data would reach human-level intelligence internally before its output began to successfully mirror the human output?

habryka
I would strongly predict it would develop many vastly superhuman capabilities, yes. (Like having memorized vastly more facts than any human

...
Time complexity for deterministic string machines
by alcatal — 2024-04-21T22:35:36.062Z

This was a project conducted during MATS 5.0 under the mentorship of Vanessa Kosoy and supported by a grant from BERI. It builds off the String Machines framework (and depends on the linked post for certain definitions), which models category-theoretic generalizations of finite-state transducers. [EDIT: The results here can also be found presented in self-contained form, in this paper.]

The framework as it previously existed did not have representation-independent ways of bounding (analogues of) time complexity, or natural guarantees that output size would not grow exponentially in input size. We introduce "filtered" transducers, which operate on categories enriched over filtered sets (sets equipped with a function to a partially ordered monoid, where morphisms are functions respecting order), and then, restricting our attention to transducers with a finite state space, prove constraints on the time complexity growth and expressivity of string machines.

Parameterizing complexity in string machines

Filtered transducers

Definition 1. The category FiltSet of filtered sets is the category such that

an object is a tuple (S,degS), where S is a set and degS:S→N is a function,
a morphism f:(S,degS)→(T,degT) is a function S→T such that degT(f(s))≤degS(s) for all s∈S.

We will generally refer to objects in FiltSet solely by the symbol corresponding to the underlying set going forward. One can observe that the identity function on a set S by definition satisfies degS(idS(s))=degS(s) for all s∈S and is thus a morphism in FiltSet. One can also observe that given f:S→T and g:T→V, degV(g(f(s)))≤degT(f(s))≤degS(s) for all s∈S, and therefore g∘f is also a morphism in FiltSet. Therefore, FiltSet is indeed a category.

Definition 2. Given two objects S,T∈Ob(FiltSet), we define their filtered product S⊗T to be the set S×T equipped with the function degS⊗T:S×T→N satisfying degS⊗T(s,t)=degS(s)+degT(t) for all (s,t)∈S×T. Given a morphism f:S→U and a morphism g:T→V, we define the morphism f⊗g:S⊗T→U⊗V to be the function f×g. Indeed, we have that degU⊗V(f(s),g(t))=degU(f(s))+degV(g(t))≤degS(s)+degT(t)=degS⊗T(s,t), so f⊗g is a morphis

...
Transfer Learning in Humans
by niplav — 2024-04-21T20:49:42.595Z

I examine the literature on transfer learning in humans. Far
transfer is difficult to achieve, best
candidate interventions are to practice at the edge of one's
ability and make many mistakes,
evaluate mistakes after one has made
them, learn from training programs modeled after expert tacit
knowledge, and talk about on one's strategies
when practicing the domain.

When learning, one would like to progress faster, and learn things
faster. So it makes sense to search for interventions that speed up
learning (effective learning techniques), enable using knowledge
and knowledge patterns from one learned domain in a new domain if
appropriate (transfer learning), and make it easier to find further
learning-accelerating techniques (meta-learning).

Summary

I've spent ~20 hours reading and skimming papers
and parts of books from different fields, and
extracting the results from them, resulting spreadsheet
here,
google doc with notes
here.

I've looked at 50 papers, skimmed 20 and read 10 papers and 20% of a book.
In this text I've included all sufficiently-different interventions I've
found that have been tested empirically.

For interventions tried by scientists I'd classify them into (ordered
by how relevant and effective I think they are):

Error-based learning in which trainees deliberately seek out situations in which they make mistakes. This has medium to large effect sizes at far transfer.

Long Training Programs: These usually take the form of one- or two-semester long classes on decision-making, basic statistics and spatial thinking, and produce far transfer at small to medium effect sizes. Such programs take a semester or two and are usually tested on high-school students or university students.

Effective Learning Techniques: Things like doing tests and exercises while learning, or letting learners generate causal mechanisms, which produce zero to or best small amounts of far transfer but speed up learning.

OODA-loop-likes: Methods that structure the problem-solving process, such as the Pólya method or DMAIC. In most cases, these haven't been tested well or at all, but they are popular in the business context. Also they look all the same to me, but probably have the advantage of functioning as checklists when performing a task.

Transfer Within Domains: Methods that are supposed to help with getting knowledge about a particular domain from an expert to a trainee, o

...
I created an Asi Alignment Tier List
by TimeGoat — 2024-04-21T18:44:44.329Z

Website:

https://www.AsiATL.net/

Check out the

mission page

to learn about the mission of AsiATL.
For now, I’m looking for

General feedback / constructive criticism
People to join the discord and hang out
Feedback on the tier list content

Making a case to demote a singular item is the easiest for me to process. Example: X is worse than Y because of Z

Connections within the Ai Safety space
The losing identity of Twitter
by Itay Dreyfus — 2024-04-21T13:43:15.145Z
Link: https://productidentity.co/p/6-twitter-whats-my-name-again

What’s Twitter for you?

That's a long-lasting trend I often see on my feed when people praise the blue bird for getting them a job, introducing them to new people, investors, and all this and that.

What about me? I just wanted to get into

dribbble

— the then invite-only designer's social network which was at its peak at the time. When I realized invites were

given away

on Twitter, I set up an account and went on a hunt. Soon, the

mission was accomplished

.

For the next few years, I went on a radio silence. Like many others, I was lurking most of the time. Even today I don't tweet excessively. But Twitter has always been a town square of mine. Suited best for my interests, it’s been a place for belonging to Design and Tech Twitter for many years.

It feels a bit intimidating to write about Twitter. I don’t have a badge of honor representing my high social graph—my reach is barely going places. But as another person who consistently addictively checks Twitter many times during the day, it’s an exciting Pandora’s box to open. And despite all the recent noise since Elon’s takeover, I still view Twitter as a novel product. Like many others, I get horrified by some tweets from Elon’s gang, but luckily this isn't another (or at least a full) ranting post about the upcoming "death" of Twitter.

My hot take is an old one: Twitter is one of the most intellectual places on the internet. Sure, it’s also a fertile ground for toxicity to spread, but as a theme for this post, I choose to embrace

Nabeel S. Qureshi

serendipity approach

:

But a well-curated Twitter feed is worth a lot of IQ points -- so I think it's worth doing.

Old Duck Twitter has been endowed with a special community and unusual design. It has been wearing many hats throughout the years. Although some cracks in its identity have arguably started to emerge, it’s still the king of short-form messages, capturing words of wisdom into bite-sized tweets.

The subculture of singing birds

Twitter is the most mainstream and niche place both at the same time, as counterintuitive as it may sound.

To put it adequately, I like how the team of

Metalabel

framed the crypto scene:

Crypto is a subculture. A self-contained universe, and a relatively small one.

Climbing Out Of The Rabbit Hole

Twitterverse has been forever understood only by its loyal citizens. And like in any subculture, it has developed its own unique language—an odd style of ...