trentmkelly/LessWrong-43k huggingface dataset:
Progress, humanism, agency: An intellectual core for the progress movement
by jasoncrawford — 2022-01-12T00:06:51.387Z
Link: https://rootsofprogress.org/progress-humanism-agency

I’ve said that we need a new philosophy of progress for the 21st century. This implies that the world needs, not just progress studies, but a progress movement: the advocacy of a set of ideas.

What are those ideas?

I see three premises at the core of this movement: progress, humanism, and agency.

Progress as a historical fact

The starting point and motivation for progress studies is the historical fact of the enormous improvements in material living standards in the last ~200 years. This observation is so generally acknowledged and incontrovertible that Deirdre McCloskey calls it “the Great Fact.” Everyone in the progress community looks back on the last few centuries and concludes that, no matter how we interpret or caveat it, something obviously went very right.

A sharply contrasting position is declinism: the idea that that world is getting worse. A declinist might think that the benefits of energy are not worth the costs of pollution, that the value of cars does not redeem their role in accidents or congestion, and that the pleasures of social media are outweighed by its psychological and social harms. Perhaps even hunter-gatherers were better off than us moderns, and agriculture was a mistake. (Some won’t go this far, but express agnosticism on the question, or are simply indifferent to material progress, greeting it with a shrug.)

But if progress is real and important—how do we judge this? How do we justify that improvements to material living standards are good? That technological and industrial progress represents true progress for humanity?

Humanism as the standard of value

Humanism says that the good is that which helps us lead better lives: longer, healthier, happier lives; lives of more choice and opportunity; lives in which we can thrive and flourish. This is the standard proposed, for instance, by Steven Pinker in Enlightenment Now.

To be clear, this need not mean simply satisfying our base material needs, such as full bellies and warm beds. It can encompass everything that makes life worth living, including psychological needs such as excitement, adventure, romance, beauty, knowledge, exploration, and human connection.

Opposition to humanism often comes from some form of romanticism. One form is the romanticization of nature: nature as a loving, protective “mother”; or a “natural” lifestyle as clean, safe, and healthy. Another is the romanticization of ...
Understanding the two-head strategy for teaching ML to answer questions honestly
by Adam Scherlis — 2022-01-11T23:24:21.751Z

This post is the result of my attempts to understand what’s going on in these two posts from summer 2021:

Paul Christiano:

https://www.alignmentforum.org/posts/QqwZ7cwEA2cxFEAun/teaching-ml-to-answer-questions-honestly-instead-of

Evan Hubinger: https://www.alignmentforum.org/posts/gEw8ig38mCGjia7dj/answering-questions-honestly-instead-of-predicting-human

The underlying problem is similar in some ways to Eliciting Latent Knowledge, although not quite the same. According to Paul, understanding the two-headed algorithm (described in these posts) is likely to be fairly useful to people working on ELK. A similar algorithm is described in the ELK report, under Strategy: penalize reporters that work with many different predictors and in related appendices.
I initially found the algorithm pretty confusing, so I wrote this post for anyone else who's confused by it.

I start with the basic strategy underlying Paul’s post, but in a simpler / more general setting without any ML. Then I add back in the ML context, followed by the specific context of the problem Paul and Evan discuss in their posts. (This is not quite the same as the ELK context.)

Along the way, I outline a simple application of this strategy: finding the second-simplest “meaningfully distinct” solution to a problem. I note that Paul’s application is sort of a special case of this.

Simple abstract version: conditioning on G(a, b)

Suppose we have a probability distribution over some discrete space A × B:

p:A×B→R

This is shown in the figure below -- dots represent probability mass:

Suppose there’s some point (a+, b+) in A × B, and we want to obtain it by sampling from this distribution. Unfortunately, it’s not very high probability, so we probably won’t succeed:

(a+, b+) indicated by (+).

Conditioning on D(·, ·)

An obvious strategy, if sampling is very cheap, is to sample repeatedly until we get (a+, b+).

Unfortunately, we don’t even know how to recognize it if we had it -- there are other points that look very similar. Worse still, some of them are higher in probability.

Aside: rejection sampling might sound too expensive to be worth thinking about. We really care about it here as a proxy for processes like gradient descent, which are doing something that looks like “sample from a distribution conditional on a constraint” -- the constraint being “loss is low” and the distribution being an architecture- and optimizer-depend

...
Has anyone had weird experiences with Alcor?
by Jaevko — 2022-01-11T21:47:56.938Z

After years of procrastinating, I've finally taken steps towards signing up for cryonics, thanks in no small part to things I've read here on LessWrong. I view it as a very long shot, but worth a try nonetheless.

However, I'm getting some very weird signals from the Alcor representative. The two default options for criteria for preservation were:

I wish Alcor to place into cryopreservation any biological remains that they may be able to recover, regardless of the severity of the damage from such causes as fire, decomposition, autopsy, embalming, etc. Similarly, members who have chosen Neurocryopreservation will have any remains of their brain placed into cryopreservation regardless of damage.

I wish Alcor to place into cryopreservation any remains of my brain that they may be able to recover, regardless of the severity of the damage. If none of my brain tissue is recoverable, do not proceed with my cryopreservation.

I found neither of these satisfactory: if I died from a blunt impact to the head, or if my brain has been burnt, crushed, has a bullet through it, etc., I do not want to be cryopreserved, and I want the money to go to my spouse. The instructions said to write out custom criteria if desired, so I wrote my criteria (in a lot more detail), and part of that was that my spouse should make the final decision if there was any possible doubt about my criteria in the event of my death.

The representative emailed back and stated that the second option above should meet my needs. It obviously does not, so I insisted on my criteria. She came back with "with the wording [I] asked for. [She] changed it slightly." Digging into the document, here is the "slight" change:

The Member might die under circumstances which would cause considerable damage to his/her human remains. Under such conditions, the Member specifies the following quantifiable and objective conditions under which his/her human remains should not be cryopreserved. The Member acknowledges and accepts that the Alcor Board of Directors shall have the sole and absolute authority
to determine if these conditions apply at the time of legal death. Further, the Member releases Alcor from any and all liability for its good faith decisions in this regard. Member requests Alcor to place into cryopreservation any remains of their brain that they may be able to recover,

...
Future ML Systems Will Be Qualitatively Different
by jsteinhardt — 2022-01-11T19:50:11.377Z

In 1972, the Nobel prize-winning physicist Philip Anderson wrote the essay "More Is Different". In it, he argues that quantitative changes can lead to qualitatively different and unexpected phenomena. While he focused on physics, one can find many examples of More is Different in other domains as well, including biology, economics, and computer science. Some examples of More is Different include:

Uranium. With a bit of uranium, nothing special happens; with a large amount of uranium packed densely enough, you get a nuclear reaction.

DNA. Given only small molecules such as calcium, you can’t meaningfully encode useful information; given larger molecules such as DNA, you can encode a genome.

Water. Individual water molecules aren’t wet. Wetness only occurs due to the interaction forces between many water molecules interspersed throughout a fabric (or other material).

Traffic. A few cars on the road are fine, but with too many you get a traffic jam. It could be that 10,000 cars could traverse a highway easily in 15 minutes, but 20,000 on the road at once could take over an hour.

Specialization. Historically, in small populations, virtually everyone needed to farm or hunt to survive; in contrast, in larger and denser communities, enough food is produced for large fractions of the population to specialize in non-agricultural work.

While some of the examples, like uranium, correspond to a sharp transition, others like specialization are more continuous. I’ll use emergence to refer to qualitative changes that arise from quantitative increases in scale, and phase transitions for cases where the change is sharp.

In this post, I'll argue that emergence often occurs in the field of AI, and that this should significantly affect our intuitions about the long-term development and deployment of AI systems. We should expect weird and surprising phenomena to emerge as we scale up systems. This presents opportunities, but also poses important risks.

Emergent Shifts in the History of AI

There have already been several examples of quantitative differences leading to important qualitative changes in machine learning.

Storage and Learning. The emergence of machine learning as a viable approach to AI is itself an example of More Is Different. While learning had been discussed since the 1950s, it wasn’t until the 80s-90s that it became a dominant paradigm: for instance, IBM’s first...
An Open Philanthropy grant proposal: Causal representation learning of human preferences
by PabloAMC — 2022-01-11T11:28:01.842Z

This is a proposal I wrote for the recent Open Philanthropy call on AI Alignment projects. I am finishing my Ph.D. in quantum computing and I would like to start working on AI Alignment. Thus, this is my proposal for a postdoc-style grant to learn more about one area I think the community could pay more attention to. However, since I am quite new to this area, there might be things that I have overlooked. For that reason, if you see something that doesn't sound promising, or you disagree with it, I would be happy to know about it.

The proposal received some light comments from Victor Veitch, Ryan Carey, and José Hernández-Orallo. I also received some comments from Jaime Sevilla during the ideation phase. I am really grateful to all of them for their help. Any remaining errors and inaccuracies are solely my fault.

Summary: Causal Representation Learning aims to extract a causal and disentangled representation of the mechanisms in the environment. I propose a postdoctoral project studying its applicability to learning human preferences, which should make the learned model robust and interpretable. In other words, human preferences have a causal structure, can we learn its concepts and their causal relations?

Introduction

One of the most promising and straightforward research lines in AI Safety is what is known as value learning (Russell, 2019): developing ways for the system to infer our preferences to subsequently optimize for them. While a good initial step, there are two main limitations with this approach: (1) inferring these preferences is often very complex, and (2) it only addresses the problem of outer alignment, that of aligning the optimizer with the user. In contrast, inner alignment can be seen as a robustness problem: the optimized model might have learned a representation of the objective that is correlated with the true objective but breaks on out-of-distribution situations. Consequently, to solve both problems, we need a way to robustly infer human preferences that extends to such cases.

Causal Representation Learning (CLR) offers a possible solution: the key objective of CLR is to learn the causal structure of the environment. Using this tool might enable AI systems to learn causal representations of what people want, and could as a result generalize better and be more interpretable. Furthermore, it may enable the application of causal incentive ...
Template For Effective Action
by njv — 2022-01-11T09:30:09.186Z

Hey LessWrong -

This community has been extremely beneficial to my life in more ways than I can describe. I'm working on a new company and am in the process of creation documentation before launching, including some project plan templates. I figured I would share one of the templates (see here) hopes that it could be helpful to some folks tackling operational complex projects. If useful, I have a bunch more.
Emails from your Gratitude Journal
by karlkeefer — 2022-01-11T06:05:28.952Z

Lots has been written here about gratitude journaling.

In spite of knowing about the benefits, I've never managed to make the habit stick. I do, however, already have a very strong habit of responding to email and keeping my inbox under control.

That's what pushed me to build Email Notebook, which I'm hopeful will facilitate other folks' writing habits, too.

It's beta quality at the moment, but my email is on the contact page and I plan to put some effort into feature requests if there are any common themes. Already on the roadmap is to improve the variety of prompts, touching on some of the specific practices outlined by @david_gross.

If you do give it a try, please report any bugs you encounter :)
Calibration proverbs
by Malmesbury — 2022-01-11T05:11:53.490Z

Cross-post from Telescopic Turnip.

Every problem is a calibration problem. That’s why most advice is basically useless: it tells you what to do, but doesn’t tell you when to stop. Therefore, the best pieces of advice are calibration advice. What we need is a metric to know if we are doing well, or if we should change our habits (and in which direction). For example, here are two poor pieces of advice:

"You should spend more time reading blogs, because compared to traditional media, bloggers have more freedom to communicate in original ways, and are more accountable when they say something false."

"You should spend less time reading blogs, because keeping up with a big pile of subscriptions takes a lot of time and makes you anxious."

Neither of these is really useful. Instead, we can produce one piece of calibration advice:

"After catching up with your backlog of blog posts, ask yourself what you remember out of it. If you can’t say what half of the posts were about, you should probably clean up your subscriptions."

Of course, that doesn’t sound like the timeless advice my grandmother received from her grandmother. To make it compelling, we need to make it rhyme:

"Before you subscribe to another blog,
check what you took home from your backlog."

Now that sounds like ancient wisdom.

Thereafter is my humble attempt at writing calibration proverbs. As a non-native, English’s pronunciation is still a mystery to me, so the rhymes might be a bit wonky. Anyways, may these proverbs contribute to making 2022 a better year than 2021.

"He who thinks a claim by evidence is backed,
can he predict the effect size just from the abstract?"

"Only when you leave a long conversation,
you can tell if it warranted your apprehension."

"If you don’t enjoy the taste of tea,
seek how to infuse it properly."

"Always check for bikes when you get out of your car.
Always."

"When checking the source teaches you something new,
the source of the source you should check too."

"One hour with your kids is worth two hours online,
according to Han et al twenty twenty one."

"Man whose roommates wear t-shirts in winter,
would better calm down with the goddamn heater."

What other calibration proverbs should we transmit to our grandchildren?

Thanks to Justis for the proofreading and rhyme consulting.
D&D.Sci Holiday Special: How the Grinch Pessimized Christmas Evaluation & Ruleset
by aphyer — 2022-01-11T01:29:59.816Z

This is a follow-up to last week's D&D.Sci scenario: if you intend to play that, and haven't done so yet, you should do so now before spoiling yourself.

Full generation code is available here if you are interested, or you can read on.

RULESET

Each of the six toys makes a different amount of noise depending on what child receives it:

A Blum-Blooper makes 6 noise.
A Fum-Foozler is more popular with female Who children - it makes 8 noise with a female Who child but only 4 noise with a male one.
A Gah-Ginka is very noisy, but too simple to hold the attention of older children for long. It makes 11 noise - 1/2 (rounded down) the child's age (so anywhere from 11 for an Age 1 child to 5 for an Age 12 child).
A Sloo-Slonker is complicated, and small Who children won't use it very much. It makes 5 noise + 1/2 (rounded down) the child's age (so anywhere from 5 for an Age 1 child to 11 for an Age 12 child).
A Trum-Troopa can be used to hold very noisy Trum-Troopa Battles with other Trum-Troopas. If another Who child in the same family (having the same middle name e.g. Cindy Lou Who is in the Lou family) also has a Trum-Troopa, it makes 10 noise. If not, it makes only 5.
A Who-Whonker is more popular with male Who children (who prefer Whonking things for some reason). It makes 9 noise with a male Who child but only 5 noise with a female one.

Each child's noise is the sum of their two toys' noises. Additionally, however, each distinct child has a favorite toy. Any noise a child makes with their favorite toy is doubled.

For example, Johnny Drew Who, in our dataset, is a 4-year old Male Who Child whose favorite toy is the Blum-Blooper. If he is given:

A Blum-Blooper and a Fum-Foozler, he will make 16 noise in total: 6x2 from the Blu-Blooper and 4 from the Fum-Foozler.
A Gah-Ginka and a Trum-Trooper, he will make either 14 or 19 noise in total: 9 from the Gah-Ginka, plus 10 if one of his siblings (Betty, Phoebe and Cindy Drew Who) also gets a Trum-Trooper and 5 if none do.
A Sloo-Slonker and a Who-Whonker, he will make 16 noise in total: 7 from the Sloo-Slonker and 9 from the Who-Whonker.

DATASET GENERATION

There are three families: the Drew Whos, Lou Whos and Sue Whos.

Each year, Who Children age and may be born - each family rolls a d4 and has a new child if the roll is greater than the number of children they currently have. As su...
The "gestalt" operator
by Alex Power — 2022-01-10T22:10:13.358Z

I have developed a convention which I must define by example. I plan to call it the “gestalt” operator.

To take a {card|page} from REF1’s book - if you take a card from REF1’s book, it is metaphorically physically taking the card from REF1’s possession. Presumably you intend to give the card back to REF1. If you take a page from REF1’s book, it is metaphorically copying the page, with that page never leaving REF1’s possession.

To {fork|mirror} a repository - to create a replica of a data repository with the expectation of continued updates. A fork is one extreme: both sides will make independent changes after the cutover. A mirror is a different extreme: one side will make changes, and the other will faithfully replicate them. There are also various combinations. Before the trigger time, they are visibly identical.

To {partition|appraise} a collection - the executor divides the collection into several lots based on price, and offers (on a commission basis) to either divide the collection equally on price, to purchase some or all of the items at the stated price, or to duplicate the items at the stated price. When there is not a clear market price for the items, having multiple appraisers is recommended.
No Abstraction Without a Goal
by dkirmani — 2022-01-10T20:24:02.879Z

Abstractions are important.

In order to think in terms of "trees" and "birds" instead of raw retinal activation rates, human minds need to form abstractions.

In order to keep Nature red in tooth and claw, organisms must first evolve abstractions like "tooth" and "claw".

Neural networks can learn human abstractions.

Figuring out abstraction is possibly "the main foundational piece" of successful AI alignment.

Abstractions are functions that map a high-dimensional space to a low-dimensional space. They have more possible inputs than possible outputs, so abstractions have to shed some information.

Abstractions filter out useless information, while keeping useful information.

In order for there to be such a thing as "useful information", there must be some goal(s) being pursued.

You might argue that "abstraction" only means preserving information used to predict faraway observations from a given system. However, coarse modeling of distant objects is often a convergent subgoal of the kind of organisms that Nature selects for.

The scout does not tell the general about the bluejays he saw. He reports the number of bombers in the enemy's hangar. Condensation of information always selects for goal-relevant information. Any condensation of information implies that the omitted information is less goal-relevant than the reported information; there is no abstraction without a goal.
Is state a (semi-) rational agent?
by Birke — 2022-01-10T19:53:24.367Z

Look at the following statements*:

- “India and Pakistan can agree on one thing: neither wants the other one around.”

- “The Americans don’t really want to fight for the South Koreans, but nor can they afford to be seen to be giving up on a friend.”

- “What is now the EU was set up so that France and Germany could hug each other so tightly in a loving embrace that neither would be able to get an arm free with which to punch the other.”

I don't want to argue whether these statements are true or not. What's bugging me is that they all contain an implicit assumption that a state acts as an intelligent agent at least and as a rational agent at most**.
Is this assumption true?
A state is a complex system which comprise civilians, military, businesses and different government branches. These parts may have different and even opposite goals and values and they can constrain each other. Will interactions between those parts lead to a (more or less) rational behavior of a whole?
Do you know a good book or article which reflects on this issue?

---

* Quotes from "Prisoners of Geography" by Tim Marshall

** I guess many people assume that states are human-like rational