trentmkelly/LessWrong-43k huggingface dataset:
The Progress Paradox
by Ben Turtel — 2023-12-27T18:26:52.506Z
Link: https://bturtel.substack.com/p/the-progress-paradox

LessWrong Intro: I'm new to the LessWrong community, but was inspired to write this based on recent reading of The Sequences. It seems to me there's a paradox to social progress that tends to undermine itself - societies making rapid ethical progress will (necessarily) become ashamed of their recent past and question their own legitimacy. Without awareness of this effect, it can undermine subsequent progress or influence such societies are willing to have in the world.

Humans are uniquely capable of self-reflection and learning from the past.

This capacity to change and grow over time also alters our perspective on our own previous actions, so that deeds and decisions that used to make us proud can later seem petty or unenlightened in light of newfound wisdom. Since we judge everything in relation to our present perspective, the more we progress, the less alignment we feel with our past selves in comparison.

We experience this as both individuals and as societies.

Individual Progress

For example, as a child, I once played a prank on a classmate by taping a pencil to his chair so that it would poke him in the butt when he sat down.

My friends were all laughing, and I thought I was hilarious.

After 30 years of learning, this isn’t exactly a moment I’m proud of. I mostly feel bad for the poor kid. I lacked empathy and prioritized getting a laugh over the well-being of another.

As individuals, we remember what it felt like to be our former, unenlightened selves. We can remember the confidence we once placed in bad ideas.

Comprehending our own transformation gives us a meta-perspective. We not only improve our values, our beliefs, our wisdom, but we develop an appreciation for the learning process itself. We develop practices - like reading, journaling, and meditation - to accelerate our learning. We acquire humility as we realize how often we’ve been wrong and how much more we have to learn.

We might think about this journey in terms of Position, Velocity, and Acceleration. Position describes where we are today, velocity describes how we’re changing, and acceleration is created by forces that increase our velocity, to speed up our learning.

Of course, not all forces create a positive acceleration of our learning. Diseases like Alzheimer’s can reverse our velocity, pushing our progress backwards. Ideological dogmatism might pu...
The virtuous circle: twelve conjectures about female reproductive agency and cultural self-determination
by Miles Saltiel — 2023-12-27T18:25:50.149Z

Extract of flowchart of female reproductive agency (for full schematic and explanatory notes, see below)

Summary

This post aims to reconcile conflicting aspects of economics and evolution. The intention is strong, testable predictions about human conduct, both individual and collective.

Why bother? Because after millennia of effort, humans still struggle for comprehensive models of our behaviour, either individual or collective[1]. Only two fields claim to offer accounts which are broad enough to be interesting and which also lend themselves to Popperian falsifiability, that is testing with data. Unfortunately, neither is satisfactory.

Economics and its subsidiary disciplines in finance, games theory etc, make robust predictions, in principle meeting the test of falsification by measurement. The trouble is that their concepts can be inconsistent, and their subject-matter is too unstable to be reliable and too narrow to be generally applicable. Nonetheless they offer useful tools relating to agency, information and risk.
Darwinian biology is comprehensive, supported by evidence and testable, but is indifferent to - if not, at odds with - the thrust of economics, by denying individual agency and bearing only obliquely upon information and risk. In addition, evolution’s central assertion regarding individuals, that they are given to reproduce, offers only the disappointing irony of scant further fruit.

Any such reconciliation must overcome the near two centuries of justifiable zealotry, which has seen off past attempts to modify the central element of Darwin’s model, whereby random mutations in genotype are selected only by the brute fact of survival. The zealots have been right to stand their ground: all such attempts have erred, either as whimsical in introducing advertence, generally divine, into the evolutionary process, eg, the various flavours of “intelligent design”; or as conflating genotype with phenotype, eg, Lamarck, Lysenko. This post aims to avoid such errors[2].

Introduction to the issue

For countless generations, our species needed to understand other animals. First, homo sapiens had to survive predators and hunt game. Then, our forebears turned to domesticating animals into sources of clothing, energy, milk, meat, other raw materials and the variety of duties assumed by dogs. Eventually breeding, drafting, herding, mi...
In Defense of Epistemic Empathy

by Kevin Dorst — 2023-12-27T16:27:06.320Z
Link: https://kevindorst.substack.com/p/in-defense-of-epistemic-empathy

TLDR: Why think your ideological opponents are unreasonable? Common reasons: their views are (1) absurd, or (2) refutable, or (3) baseless, or (4) conformist, or (5) irrational. None are convincing.

Elizabeth is skeptical about the results of the 2020 election. Theo thinks Republicans are planning to institute a theocracy. Alan is convinced that AI will soon take over the world.

You probably think some (or all) of them are unhinged.

As I’ve argued before, we seem to be losing our epistemic empathy: our ability to both (1) be convinced that someone’s opinions are wrong, and yet (2) acknowledge that they might hold those opinions for reasonable reasons. For example, since the 90s our descriptions of others as ‘crazy’, ‘stupid’ or ‘fools’ has skyrocketed:

I think this is a mistake. Lots of my work aims to help us recover our epistemic empathy—to argue that reasonable processes can drive such disagreements, and that we have little evidence that irrationality (the philosophers’ term for being “crazy”, “stupid”, or a “fool”) explains it.

The most common reaction: “Clever argument. But surely you don’t believe it!”

I do.

Obviously people sometimes act and think irrationally. Obviously that sometimes helps explain how they end up with mistaken opinions. The question is whether we have good reason to think that this is generically the explanation for why people have such different opinions than we do.

Today, I want to take a critical look at some of the arguments people give for suspending their epistemic empathy: (1) that their views are absurd; (2) that the questions have easy answers; (3) that they don’t have good reasons for their beliefs; (4) that they’re just conforming to their group; and (5) that they’re irrational.

None are convincing.

Absurdity.

“Sure, reasonable people can disagree on some topics. But the opinions of Elizabeth, Theo, and Alan are so absurd that only irrationality could explain it.”

This argument over-states the power of rationality.

Spend a few years in academia, and you’ll see why. Especially in philosophy, it’ll become extremely salient that reasonable people often wind up with absurd views.

David Lewis thought that there were talking donkeys. (Since the best metaphysical system is one in which every possible world we can imagine is the way some spatio-temporally isolated world actually is.)

Timothy Williamson thinks that it’s impossible f...
Critical review of Christiano's disagreements with Yudkowsky
by Vanessa Kosoy — 2023-12-27T16:02:50.499Z

This is a review of Paul Christiano's article "where I agree and disagree with Eliezer". Written for the LessWrong 2022 Review.

In the existential AI safety community, there is an ongoing debate between positions situated differently on some axis which doesn't have a common agreed-upon name, but where Christiano and Yudkowsky can be regarded as representatives of the two directions[1]. For the sake of this review, I will dub the camps gravitating to the different ends of this axis "Prosers" (after prosaic alignment) and "Poets"[2]. Christiano is a Proser, and so are most people in AI safety groups in the industry. Yudkowsky is a typical Poet [sort-of Poet, but there's an important departure from my characterization below], people in MIRI and the agent foundations community tend to also be such.

Prosers tend to be more optimistic, lend more credence to slow takeoff, and place more value on empirical research and solving problems by reproducing them in the lab and iterating on the design. Poets tend to be more pessimistic, lend more credence to fast takeoff, and place more value on theoretical research and solving problems on paper before they become observable in existing AI systems. Few people are absolute purists in those respects: almost nobody in the community believes that e.g. empirical research or solving problems on paper in advance is completely worthless.

In this article, Christiano lists his agreements and disagreements with Yudkowsky. The resulting list can serve as a reasonable starting point for understanding the differences of Proser and Poet positions. In this regard it is not perfect: the tone and many of the details are influenced by Christiano's reactions to Yudkowsky's personal idiosyncrasies and also by the specific content of Yudkwosky's article "AGI Ruin" to which Christiano is responding. Moreover, it is in places hard to follow because Christiano responds to Yudkowsky without restating Yudkowsky's position first. Nevertheless, it does touch on most of the key points of contention.

In this review, I will try to identify the main generators of Christiano's disagreements with Yudkowsky and add my personal commentary. Since I can be classified as a Poet myself, my commentary is mostly critical. This doesn't mean I agree with Yudkowsky everywhere. On many points I have significant uncertainty. On some, I disagree with both Christiano and...
AGI will be made of heterogeneous components, Transformer and Selective SSM blocks will be among them
by Roman Leventov — 2023-12-27T14:51:37.713Z

This post is prompted by two recent pieces:

First, in the podcast "Emergency Pod: Mamba, Memory, and the SSM Moment", Nathan Labenz described how he sees that we are entering the era of heterogeneity in AI architectures because currently we have not just one fundamental block that works very well (the Transformer block), but two kinds of blocks: the Selective SSM (Mamba) block has joined the party. These are natural opposites on the tradeoff scale between episodic cognitive capacity (Transformer's strong side) and long-term memorisation (selective SSM's strong side).[1]

Moreover, it's demonstrated in many recent works (see the StripedHyena blog post, and references in appendix E.2.2. of the Mamba paper) that hybridisation of Transformer and SSM blocks works better than a "pure" architecture composed of either of these types of blocks. So, we will probably quickly see the emergence of complicated hybrids between these two.[2]

This reminds me of John Doyle's architecture theory that predicts that AI architectures will evolve towards modularisation and component heterogeneity, where the properties of different components (i.e., their positions at different tradeoff spectrums) will converge to reflect the statistical properties of heterogeneous objects (a.k.a. natural abstractions, patterns, "pockets of computational reducibility") in the environment.

Second, in this article, Anatoly Levenchuk rehearses the "no free lunch" theorem and enumerates some of the development directions in algorithms and computing that continue in the shadows of the currently dominant LLM paradigm, but still are going to be several orders of magnitude more computationally efficient than DNNs in some important classes of tasks: multi-physics simulations, discrete ("system 2") reasoning (planning, optimisation), theorem verification and SAT-solving, etc. All these diverse components are going to be plugged into some "AI operating system", Toolformer-style. Then Anatoly posits an important conjecture (slightly tweaked by me): as it doesn't make sense to discuss some person's "values" without considering (a) them in the context of their environment (family, community, humanity) and (b) their education, it's pointless to discuss the alignment properties and "values" of some "core" AGI agent architecture without considering the whole context of a quickly evolving "open agency" of various tools a...
5. Moral Value for Sentient Animals? Alas, Not Yet
by RogerDearnaley — 2023-12-27T06:42:09.130Z

Part 5 of AI, Alignment, and Ethics. This will probably make more sense if you start with Part 1.

TL;DR In Parts 1 through 3 I discussed principles for ethical system design, and the consequences for AIs and uploads, and in Part 4, I discussed a principled way for us to grant moral value/weight to a larger set than just biological humans: all evolved sapient beings (other than ones of types we cannot form a cooperative alliance with). The history of liberal thought so far has been a progressive expansion of the set of beings accorded moral value (starting from just the set of privileged male landowners having their own military forces). So, how about animals? Could we expand our moral set to all evolved sentient beings, now that we have textured vegetable protein? I explore some of the many consequences if we tried this, and show that it seems to be incredibly hard to construct and implement such a moral system that doesn't lead to human extinction, mass extinctions of animal species, ecological collapses, or else to very ludicrous outcomes. Getting anything close to a good outcome clearly requires at least astonishing levels of complex layers of carefully-tuned fudge factors baked into your ethical system, and also extremely advanced technology. A superintelligence with access to highly advanced nanotechnology and genetic engineering might be able to construct and even implement such a system, but short of that technological level, it's sadly impractical. So I regretfully fall back to the long-standing solution of donating second-hand moral worth from humans to animals, especially large, photogenic, cute, fluffy animals (or at least ones visible with the naked eye) because humans care about their well-being.

[About a decade ago, I spent a good fraction of a year trying to construct an ethical system along these lines, before coming to the sad conclusion that it was basically impossible. I skipped over explaining this when writing Part 4, assuming that the fact this approach is unworkable was obvious, or at least uninteresting. A recent conversation has made it clear to me that this is not obvious, and furthermore that not understanding this is both an x-risk, and also common among current academic moral philosophers — thus I am adding this post. Consider it a write-up of a negative result in ethical-system design.

This post follows on logically from Part 4. so...
Differential Optimization Reframes and Generalizes Utility-Maximization
by J Bostock — 2023-12-27T01:54:22.731Z

Consider the following world: a set P of past nodes, a set of environment nodes E a set of actor/agent nodes A, and a set of future nodes F. Consider our previously defined function Op(A; p, f) for p∈P,f∈F, which represents how much A is optimizing the value of f with respect to p. For some choices of A and f, we might find that the values Op(A; p, f) allow us to split the members of P into two categories: those for which Op≫0, those with Op≈0, and those with Op≪0.

Picking P Apart

We can define Vi={p∈P∣Op(A; p,fi)≫0}. In other words, Vi is the subset of P which has no local influence on the future node fi whatsoever. If you want to be cute you can think of Vi as the victim of A with respect to fi, because it is causally erased from existence.

Remember that Op is high when "freezing" the information going out of A means that the value of fi depends much more on the value p. What does it mean when "freezing" the information of A does the opposite?

Now lets define Ui={p∈P∣Op(A; p, fi)≪0}. This means that when we freeze A, these nodes have much less influence on the future. Their influence flows through A. We can call Ui the utility-function-like region of P with respect to fi. We might also think of the actions of A as amplifying the influence of Ui on fi.

For completeness we'll define Xi={p∈P∣Op(A; p, fi)≈0}≡P−U−V.

Let's now define U=U0∪...∪Un i.e. the set of points which are utility-like for any future point. V=V0∪...∪Vm will be the set of points which are victim-like for any future points. We might want to define V′=V0∩...∩Vm as the set of "total" victims and U′=U0∩...∩Um similarly. X=P−U−V meaning X only contains nodes which really don't interact with A at all in an optimizing/amplifying capacity.

We can also think of U and V as the sets of past nodes which have an outsizedly large and small influence on the future as a result of the actions of A, respectively.

Describing D

D={d∈E∪F∣∃v∈V∣Op(A; v, d)≫0}, in other words D ...
More Thoughts on the Human-AGI War
by Seth Ahrenbach — 2023-12-27T01:03:18.812Z

In https://www.lesswrong.com/posts/xSJMj3Hw3D7DPy5fJ/humanity-vs-agi-will-never-look-like-humanity-vs-agi-to the author argued that in an active conflict, a human-AGI war would look very different from how many expect. Namely, it would likely involve many slow burn, clandestine moves by AGI that do not trigger an obvious collective alarm response from humans, humans would be unlikely to be able to coalesce into a unified force, and once positioned, AGI would not have difficulty sweeping us up. I think this is correct, but I want to expand the scope a bit and claim that we are already in the conflict game and have the time to make N unanswered shaping moves, which we squander at our peril.

The conflict is such that we humans can make a number of moves before AGI gets a chance to respond, including the move to delay its being constructed. Thane’s point about disjointed humanity is playing out before our eyes here, as we are frozen with indecision and uncertainty about whether there even is a real threat. It’s as if the AGI is coming back in time and playing shaping moves against us!

But due to Thane’s third point, the capability asymmetry we are likely to see, we should consider our available N moves now to likely be the only moves we can play. Once unaligned AGI is created, which is likely to occur in the next decade, we are left relying entirely on the N moves we play now, and we either survive and solve alignment later or get defeated in some way that we can’t specifically predict.

What types of moves can we play, given we do not know the value of N, and humanity is disjointed? Likely whatever moves we play will be known entirely by the future AGI, so secrecy cannot be an element. Also, we should count on present and future humans attempting to undermine the moves, due to ideology or self interest.

I think our available moves exist in three main categories: physical, regulatory, and game theoretic. A physical move would be some sort of mechanism built into the chips on which the AGI is to run, that does something like pauses work after a certain number of FLOPS until a secret key is entered to unlock. So I suppose some secrets can maybe exist, but the existence of and strategic details of the moves cannot rely on secrecy. A regulatory move would be like a global pause on development enforced by treaties, or a global treaty binding all chip manufacturers to incl...
METR is hiring!
by Beth Barnes — 2023-12-26T21:00:50.625Z

This is a quick update that

METR (formerly ARC Evals)

is recruiting for four positions. I encourage you to err on the side of applying to positions that interest you even if you’re unsure about your fit! We’re able to sponsor US visas for all the roles below except Research Assistant, and all applications are rolling with no set closing date.

Engineering Lead

and

Senior Software Engineer

. You’ll work on our internal platform for evaluating model capabilities (think: 100 docker containers running agents in parallel against different tasks). The work is technically fascinating and you get to be on the cutting edge of what models can do, as well as collaborate with our partners (e.g. major world governments).

Human Data Lead

. High-quality feedback on agent behavior is a key bottleneck to improving agent performance, and you’ll manage this data generation process by recruiting and managing skilled contractors.

Research Assistant

. You'll help our Model Evaluation Researchers test model capabilities by designing and implementing tasks, testing agent designs, and reviewing agent performance. Many of our research assistants from earlier this year are now full-time researchers, and we both found that experience useful to gauge fit for a longer-term work relationship. This is a full-time, fully-remote role that requires substantial overlap with North American Pacific Time working hours.

If you know anyone who’d be a good fit, please let them know about these roles or recommend that we reach out to them! If we reach out to and hire a candidate because you filled out

this referral form

, we will pay you a referral bonus of 5,000 USD. (See referral form for conditions.)
Environmental allergies are curable? (Sublingual immunotherapy)
by Chris Lakin — 2023-12-26T19:05:08.880Z

I used to have severe environmental allergies to cats, dogs, and the outdoors. As I kid, I woke up crying most mornings because of my allergies, and I would occasionally wake up with croup and difficulty breathing. I even had to be taken the hospital once.

Anyways, I cried enough that my mother found and enrolled me in a study for an experimental treatment called sublingual immunotherapy (SLIT). For the next two years I took under-the-tongue drops, and I presume the drops were formulated with small amounts of the allergens I was reactive to.

My allergies have been nearly non-existent since then.

I’m sharing this post because I keep telling people about sublingual immunotherapy and they’re very surprised. No one seems to know about this treatment! I’m mad about this.

Maybe my improvement was unusual? I don’t know. A few random studies. Please share additional information in the comments.

To be clear, I still have a few mild symptoms:

If I pet a dog and then rub my eyes, my eyes get slightly itchy.
If a dog licks me, I get mild hives in that area.

But that’s all! (And I haven’t observed any side effects, either.)

FWIW, I might also go back on sublingual immunotherapy at some point so I can pet dogs without worry. (Because maybe my treatment was stopped too soon?)

Other details:

My mother says the particular drops I took costed $25 a week. They weren’t FDA approved, but they were still available for purchase.
From a quick brief search, I found a few that sell sublingual immunotherapy in the US: Wyndly, Curex, and Quello. I looked a few months ago and I couldn’t find any significant reason to prefer one brand over the others. Please comment you have a recommendation.

Note: SLIT has been available for longer in Europe than in the US, so the European brands might be better if you have access to them.