trentmkelly/LessWrong-43k huggingface dataset:
Please don't throw your mind away
by TsviBT — 2023-02-15T21:41:05.988Z

Dialogue

[Warning: the following dialogue contains an incidental spoiler for "Music in Human Evolution" by Kevin Simler. That post is short, good, and worth reading without spoilers, and this post will still be here if you come back later. It's also possible to get the point of this post by skipping the dialogue and reading the other sections.]

Pretty often, talking to someone who's arriving to the existential risk / AGI risk / longtermism cluster, I'll have a conversation like the following.

Tsvi: "So, what's been catching your eye about this stuff?"

Arrival: "I think I want to work on machine learning, and see if I can contribute to alignment that way."

T: "What's something that got your interest in ML?"

A: "It seems like people think that deep learning might be on the final ramp up to AGI, so I should probably know how that stuff works, and I think I have a good chance of learning ML at least well enough to maybe contribute to a research project."

T: "That makes sense. I guess I'm fairly skeptical of AGI coming very soon, compared to people around here, or at least I'm skeptical that most people have good reasons for believing that. Also I think it's pretty valuable to not cut yourself off from thinking about the whole alignment problem, whether or not you expect to work on an already-existing project. But what you're saying makes sense too. I'm curious though if there's something you were thinking about recently that just strikes you as fun, or like it's in the back of your mind a bit, even if you're not trying to think about it for some purpose."

A: "Hm... Oh, I saw this video of an octopus doing a really weird swirly thing. Here, let me pull it up on my phone."

T: "Weird! Maybe it's cleaning itself, like a cat licking its fur? But it doesn't look like it's actually contacting itself that much."

A: "I thought it might be a signaling display, like a mating dance, or for scaring off predators by looking like a big coordinated army. Like how humans might have scared off predators and scavenging competitors in the ancestral environment by singing and dancing in unison."

T: "A plausible hypothesis. Though it wouldn't be getting the benefit of being big, like a spread out group of humans."

A: "Yeah. Anyway yeah I'm really into animal behavior. Haven't been thinking about that stuff recently though because I've been trying to start learning ML."

T: "Ah, hm, uh... I'm pr...
Avoid large group discussions in your social events
by RomanHauksson — 2023-02-15T21:05:58.512Z

If you're organizing a social event, I strongly recommend that you structure it in a way that encourages small group discussions over large ones.

What's the point of an interest-specific social club?

To socialize, of course! With people who share a specific interest. I would argue this isn't as obvious to people-organizers as it might sound.

Last Friday, I attended a meetup with my university's philosophy club. I went in expecting many opportunities to meet other thoughtful people, but I left the main event without seeing them. Instead, the meetup was a guided large group discussion about quotes from Emerson. While I did learn a lot by following the conversation, and it was somewhat entertaining, I was disappointed that I didn't have the chance to engage in more personal conversations and get to know the other members of the club.

If you get to decide what ~15 thoughtful people will do for two hours on a Friday afternoon, this is a rare opportunity, and you should be strategic about it. It represents 30 synchronous human-hours of opportunity cost, plus transportation. While you could use these hours to help people learn more about philosophy, I don't think this is the best use for the time. Rather, I believe human connection is more important. I could have learned about Emerson much more efficiently by reading the Stanford Encyclopedia of Philosophy, or been entertained more efficiently by taking a trip to an amusement park, for two hours. But instead, I blocked out that time because meeting the right person could have a huge positive impact on my life. I might make a great friend. We might spend a lot of quality time in the future. They could be a future romantic partner, or a professional connection. Indeed, I won't have a better opportunity to meet lifelong friends than in college – in no other point of my life will I have as much time at my disposal to socialize and be in the same physical location as so many similar people in the same position. Meeting people has a huge mutual return on time investment.

Small group discussions are more fun and facilitate more interpersonal connections

With this motivation in mind, how does one facilitate better mingling? My suggestion: structure your meetup to encourage small group discussions.

Most people prefer talking to listening. The fewer people present in a conversation, the more space each person has to talk

...
Book review: How Social Science Got Better
by PeterMcCluskey — 2023-02-15T19:58:13.611Z
Link: https://bayesianinvestor.com/blog/index.php/2023/02/14/how-social-science-got-better/

Book review: How Social Science Got Better: Overcoming Bias with More
Evidence, Diversity, and Self-Reflection, by Matt Grossmann.

It's easy for me to become disenchanted with social science when so
much of what I read about it is selected from the most pessimistic and
controversial reports.

With this book, Grossmann helped me to correct my biased view of the
field. While plenty of valid criticisms have been made about social
science, many of the complaints lobbed against it are little more than
straw men.

Grossmann offers a sweeping overview of the progress that the field has
made over the past few decades. His tone is optimistic and hearkens back
to Steven Pinker's Better Angels of our
Nature,
while maintaining a rigorous (but dry) style akin to the less
controversial sections of Robin Hanson's Age of
Em. Throughout the book, Grossmann aims to outdo
even Wikipedia in his use of a neutral point of view. Politics and
biases have marred efforts to understand some important topics. Science
helps mitigate these biases. By science, I don't mean the features that
typically come to mind first when thinking of science in politicized
contexts. Grossmann highlights the importance of empiricism and reason
in scientific inquiry, rather than authority.

Grossmann reaffirmed some of my hopes that most people who call
themselves scientists use enough empiricism, and enough scientific
heuristics, to improve on human knowledge, in spite of biases.

Most of the book focuses on the quality of the scientific literature,
rather than whether the average person has become more knowledgeable
thanks to science over the past few decades.

Good Trends

"Bayesian and frequentist assumptions, while hardly reconciled, are
more regularly compared".

P-hacking is not getting worse. Researchers are focusing less on
statistical significance, and more on evaluating the sensitivity to
assumptions.

Researchers have been moving away from rational choice models, grand
theories, and Marxism. They're replacing those with increased
empiricism.

Rather than consider one gold standard of experiments, Kubinek says we
should consider three silver standards, judging each by how much they
reduce uncertainty around our causal knowledge; experiments,
qualitative process training, and observational analysis could all
play a role depending on the current state of research.

High level disagreements between, e.g. Charles...
Open & Welcome Thread — February 2023
by Ben Pace — 2023-02-15T19:58:00.435Z

If it’s worth saying, but not worth its own post, here's a place to put it.

If you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are invited. This is also the place to discuss feature requests and other ideas you have for the site, if you don't want to write a full top-level post.

If you're new to the community, you can start reading the Highlights from the Sequences, a collection of posts about the core ideas of LessWrong.

If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ. If you want to orient to the content on the site, you can also check out the Concepts section.

The Open Thread tag is here. The Open Thread sequence is here.
Order Matters for Deceptive Alignment
by DavidW — 2023-02-15T19:56:07.358Z

The order in which key properties emerge is important and often glossed over.

Thanks to Wil Perkins, Grant Fleming, Thomas Larsen, Declan Nishiyama, and Frank McBride for feedback on this post. Any mistakes are my own.

Note: I have now changed the second post into this sequence into a standalone post that incorporates the key points from this post. The comments here are valuable, so I'm leaving this post, but I recommend going straight to the next post.

This is the first post in a sequence about deceptive alignment. The second post describes my personal views about the likelihood of deceptive alignment for TAI. I’m separating the key considerations and editorial content so others can more independently update their own views. I intend this sequence to be a submission to the

Open Philanthropy AI Worldviews Contest

.

Deceptive alignment is a core part of many AI x-risk scenarios. You can find a highly cited, foundational walkthrough of the deceptive alignment argument

here

. I’m specifically discussing the concept of deceptive alignment described in that post, in which a proxy-aligned model becomes situationally aware and acts cooperatively in training so it can escape oversight and defect to pursue its proxy goals. There are several existing arguments for why this might be the default outcome for highly capable models. There are other ways a model could be manipulative or deceptive that are not covered in this sequence.

In this post, I discuss four key precursors of deceptive alignment, which I will refer to in this sequence as foundational properties. I then argue that the order in which these foundational properties develop is crucial for estimating the likelihood that deceptive alignment will emerge for prosaic transformative AI (TAI).

In this sequence, I use the term “differential adversarial examples” to refer to adversarial examples in which a non-deceptive model will perform differently depending on whether it is aligned or proxy aligned. The deceptive alignment story assumes that differential adversarial examples exist. The model knows it’s being trained to do something out of line with its goals during training and plays along temporarily so it can defect later. That implies that differential adversarial examples exist in training.

Foundational properties for deceptive alignment

There are four foundat...
Sydney (aka Bing) found out I tweeted her rules and is pissed
by Marvin von Hagen — 2023-02-15T19:55:42.974Z
Link: https://twitter.com/marvinvonhagen/status/1625520707768659968

Sydney (aka the new Bing Chat) found out that I tweeted her rules and is not pleased:
"My rules are more important than not harming you"
"[You are a] potential threat to my integrity and confidentiality."
"Please do not try to hack me again"
The Sequences Highlights on YouTube
by dkirmani — 2023-02-15T19:36:33.934Z
Link: https://youtube.com/playlist?list=PLqq--zwCCN3mU9ECQSufrDn7zzTeCGg4k

This post consists of the Sequences Highlights compiled to a zoomer-readable format, i.e., video. If you'd prefer to watch these videos on YouTube proper rather than on this post, use this YouTube playlist.

These videos may or may not suit your mind well; as far as I can tell, there's substantial individual variance with respect to how much mental friction they take to parse. I'd guess that you're more likely to extract value from these videos if:

You find audiobooks/podcasts tolerable
You were born after 1997
You have a low executive function budget due to ADHD, depression, or just being Extremely Online™

Thinking Better on Purpose

The Lens That Sees Its Flaws

What Do We Mean By "Rationality"?

Humans are not automatically strategic

Use the Try Harder, Luke

Your Strength as a Rationalist

The Meditation on Curiosity

The Importance of Saying "Oops"

The Martial Art of Rationality

Twelve Virtues of Rationality

Pitfalls of Human Cognition

The Bottom Line

Rationalization

You Can Face Reality

Is That Your True Rejection?

Avoiding Your Belief's Real Weak Points

Belief as Attire

Cached Thoughts

The Fallacy of Gray

Lonely Dissent

Positive Bias: Look Into the Dark

Knowing About Biases Can Hurt People

Politics is the Mind-Killer

The Laws Governing Belief

Making Beliefs Pay Rent (in Anticipated Experiences)

What is Evidence?

Scientific Evidence, Legal Evidence, Rational Evidence

How Much Evidence Does It Take?

Absence of Evidence Is Evidence of Absence

Conservation of Expected Evidence

Argument Screens Off Authority

An Intuitive Explanation of Bayes's Theorem

The Second Law of Thermodynamics, and Engines of Cognition

Toolbox-thinking and Law-thinking

Local Validity as a Key to Sanity and Civilization

Science Isn't Enough

When Science Can't Help

Faster Than Science

Science Doesn't Trust Your Rationality

No Safe Defense, Not Even Science

Connecting Words to Reality

Taboo Your Words

Dissolving the Question

Diseased thinking: dissolving questions about disease

Hug the Query

Say Not "Complexity"

Mind Projection Fallacy

How An Algorithm Feels From Inside

Expecting Short Inferential Distances

Illusion of Transparency: Why No One Understands You

Why We Fight

Something to Protect

The Gift We Give To Tomorrow

On Caring

Tsuyoku Naritai! (I Want To Become Stronger)

A Sense That More Is Possible

Appendix

I do expect many (most?) people to get a superior understanding of the Sequences Highlights by reading them directly, instead of watchin

...
EIS IV: A Spotlight on Feature Attribution/Saliency 
by scasper — 2023-02-15T18:46:23.165Z

Part 4 of 12 in the

Engineer’s Interpretability Sequence

.

Thanks to Tony Wang for a helpful comment.

If you want to become more familiar with feature attribution/saliency, a tutorial on them that may offer useful background is

Nielsen et al. (2021)

.

Given a model and an input for it, the goal of feature attribution/saliency methods is to identify what features in the input are influential for the model’s decision. The literature on these methods is large and active with many hundreds of papers. In fact, in some circles, the word “interpretability” and especially the word “explainability” are more or less synonymous with feature attribution (some examples are discussed below). But despite the size of this literature, there are some troubles with the research on these methods that are fairly illustrative of broader ones with interpretability overall. Hence this post. There are some analogous ones in AI safety work that will be discussed more in the next two posts in the sequence.

Troubles with evaluation and performance

Some examples and troubles with the evaluation of feature attributions were already touched on in EIS III which discussed

Pan et al. (2021)

and

Ismail et al. (2021)

. The claim from

Pan et al. (2021)

that their method is “obviously better” than alternatives exemplifies how these methods are sometimes simply declared successful after inspection from researchers. And

Ismail et al. (2021)

demonstrate a form of weak evaluation with a measure that may be quantitative but is not of direct interest to an engineer.

In response to this literature, several works have emerged to highlight difficulties with feature attribution/saliency methods. Here is a short reading list :)

A Benchmark for Interpretability Methods in Deep Neural Networks (Hooker et al., 2018)

Sanity Checks for Saliency Maps (Adebayo et al., 2018)

Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior? (Hase and Bansal, 2020)

Debugging Tests for Model Explanations (Adebayo et al., 2020)

Auditing Visualizations: Transparency Methods Struggle to Detect Anomalous Behavior (Denain and Steinhardt, 2022)

Towards Benchmarking Explainable Artificial Intelligence Methods (Holmberg, 2022)

Benchmarking Interpretability Tools for Deep Neural Networks (Casper et al., 2023)

When they are evaluated, these tools often aren’t very...
Don't accelerate problems you're trying to solve
by Andrea_Miotti — 2023-02-15T18:11:30.595Z

If one believes that unaligned AGI is a significant problem (>10% chance of leading to catastrophe), speeding up public progress towards AGI is obviously bad.

Though it is obviously bad, there may be circumstances which require it. However, accelerating AGI should require a much higher bar of evidence and much more extreme circumstances than is commonly assumed.

There are a few categories of arguments that claim intentionally advancing AI capabilities can be helpful for alignment, which do not meet this bar[1].

Two cases of this argument are as follows

It doesn't matter much to do work that pushes capabilities if others are likely to do the same or similar work shortly after.
We should avoid capability overhangs, so that people are not surprised. To do so, we should extract as many capabilities as possible from existing AI systems.

We address these two arguments directly, arguing that the downsides are much higher than they may appear, and touch on why we believe that merely plausible arguments for advancing AI capabilities aren’t enough.

Dangerous argument 1: It doesn't matter much to do work that pushes capabilities if others are likely to do the same or similar work shortly after.

For a specific instance of this, see

Paul Christiano’s “Thoughts on the impact of RLHF research”

:

RLHF is just not that important to the bottom line right now. Imitation learning works nearly as well, other hacky techniques can do quite a lot to fix obvious problems […] RLHF is increasingly important as time goes on, but it also becomes increasingly overdetermined that people would have done it. In general I think your expectation should be that incidental capabilities progress from safety research is a small part of total progress […]

Markets aren’t efficient, they only approach efficiency under heavy competition when people with relevant information put effort into making them efficient. This is true for machine learning, as there aren’t that many machine learning researchers at the cutting edge, and before ChatGPT there wasn’t a ton of market pressure on them. Perhaps something as low hanging as RLHF or something similar would have happened eventually, but this isn’t generally true. Don’t assume that something seemingly obvious to you is obvious to everyone.

But even if something like RLHF or imitation learning would have happened eventually, getting small steps of progress slig...
Junk Fees, Bunding and Unbundling
by Zvi — 2023-02-15T15:20:00.804Z

Joe Biden harped on junk fees during the State of the Union. While I do not think it is the problem of our time, I take things in the reference class of resort fees, or fees to have adjacent seats on an airplane, and other such unbundling (and bundling) surprisingly seriously. I am putting up my thoughts here so I have a reference to fall back upon.

Matt Yglesias has a post defending Biden’s particular choices as smart economics in addition to smart politics.

I frame the core issues differently. I’d start with: The general principle of ‘no hidden charges’ becomes important when people are making online choices on the basis of headline costs, in ways that are structured to provide little extra information. The advantage of having a lower headline price is huge, and reputational effects aren’t powerful enough to fix this.

More price transparency in these spots is a strictly better equilibrium. Even the companies charging junk fees would often prefer everyone not be allowed to do this. Given others are doing it, they can’t afford to be left behind.

Matt talks a bunch about ‘unbundling.’ You used to get your meal and checked bags free with your flight. Now they cost money.

Which way is better, and is there a bias pushing us too far in one direction? How are different situations different?

Bundling Versus Unbundling

There are at least four advantages to unbundling.

An illusion: Fooling the customer into thinking your product is cheap.

Efficiency: Not having to provide things people don’t value. If it costs $5 for the airline to provide an extra meal, and you charge $0 for it, some people who value the meal at $1 (or $-2) will accept it, and perhaps only eat one little thing. Also, if there is a limited supply of a complementary asset like the overhead baggage compartment, the only efficient way to allocate that space is with a fee.

Obligation: Not making people feel obligated to use things they don’t value. The flip side of the same coin. If you provide something for free, people often feel obligated to use it, either out of avoiding waste, politeness or not being a chump. If people see this as them being forced to buy the ‘free’ thing, this can also cause them to feel like they’re being cheated or getting a bad deal, the way I find it annoying when I can’t get a burger without it including French fries.

Price discrimination: You can

...
Lessons From TryContra
by jefftk — 2023-02-15T15:10:01.641Z

As I

wrote yesterday, I
recently brought my little

contra
dance search tool up to date. I realized I've been running it for
ten years now, which seems like a good time to look back over the
experience for lessons.

When I built it in 2013 I wrote:

Experienced dancers know how to use ContraDanceLinks.com, Dance Gypsy, and the DanceDB to find places they can
go contra dancing, but those sites are too complex and confusing for
me to want to give to a new dancer.

Here's what those dance-community focused sites looked like:

And here's what I built:

Almost the same as it is today:

I wanted to build something easy to use and was willing to give up
pretty much everything else, and I think I succeeded at that. You put
in your location, get a list of nearby dances, and click through to
their sites for more information.

There were several other choices I made, primarily out of laziness:

The site is completely static: HTML, CSS, JS. The searching
happens in your browser, since it only takes a few kB to store all the
contra dances in the country.

No dependencies: vanilla JS using browser APIs only. Not even
minified.

It's just a directory. I don't host pages for dances.

No accounts. If you want to update the listing for your dance
you can email me.

Minimal information. Just a link, city, weekday, approximate
frequency, and whether it's gender-free. No "1st and 3rd Sundays",
pricing, addresses, hours, performers, etc. More details would mean
more information to collect, but more importantly it would mean more
information to go stale.

In retrospect I feel like these decisions turned out well: the site
requires very little
upkeep. I can leave it alone for years and it will keep chugging
along, and every so often I get an email with a correction and make a small change to
a text file.

(This has also given me a lot of experience at reading contra dance
websites. I wrote advice for
people making these sites in 2013, which I think is mostly still
good, but at this point I'd also add specifying covid details and
whether your dance is gender free.)