trentmkelly/LessWrong-43k huggingface dataset:
Relationship between subjective experience and intelligence?
by Q Home — 2022-07-24T09:10:10.494Z

I want to investigate with you if my belief about intelligence is rational or irrational. Beware, it may turn out to be based on wishful thinking.

To put it crudely: I believe there should be a "cheat code" for high intelligence (problem-solving skills) for people.

Down below is my evidence for this. For me this evidence is abstract (not based on very specific facts), but strong. Basically, all my evidence boils down to what I perceive as conflicts between everyday subjective experience and the idea of intelligence. Those conflicts are "cruxes of my feelings" that push me to the conclusion.

And if my opinion turns out to be rational enough (for common enough sets of priors), then I think it's important: it means there's a significant chance that people have an yet unknown way to boost their "learning to learn" ability. Finding this way would be as important as learning rationality.

Conflict 1

99% of your experience is subjective experience and 1% is problem solving. Because even when you're solving problems you have subjective experience about it.

For me problem solving is like a drop in the ocean. Like rational numbers on the real number line.

Intelligence appears to be related to subjective experience (you may need to think hard about stuff or to do quite a bit of world modeling and self-reflection and whatnot in order to get certain feelings). And at the same time there appears to be a large disconnect between subjective experience and intelligence. I find it strange.

Crazy analogy: if subjective experience were like mass and problem solving were like energy, there would be a neat equivalence.

Conflict 2

I think there's another conflict between subjective experience and intelligence. Because of the self-defining properties of the subjective experience.

Imagine you're solving a problem

If you feel that a problem is hard, it's objectively hard inside of your subjective experience.
If you don't use all the available information to solve the problem, that's because this information just doesn't exist inside of your subjective experience.

In a way, having consciousness is like feeling that you're maximally smart. From the outside it looks like people solve problems with different efficiency, but from the inside everyone's using the most effective method. And subjective experience of people is similar.

So I think there's a big tension between the nature of consciousness and t...
Double Crux
by CFAR!Duncan — 2022-07-24T06:34:15.305Z

Author's note: There is a preexisting standalone essay on double crux (also written by Duncan Sabien) available here. The version in the handbook is similar, but has enough disoverlap that it seemed worth including it rather than merely adding the standalone post to the sequence.

Missing from this sequence is a writeup of "Finding Cruxes," a CFAR class developed primarily by instructor Eli Tyre as a prerequisite to Double Crux, giving models and concrete suggestions on how to more effectively introspect into one's own belief structure. At the time of this post, no writeup existed; if one appears in the future, it will be added.

Epistemic status: Preliminary/tentative

The concepts underlying the Double Crux technique (such as Aumann’s Agreement Theorem and psychological defensiveness) are well-understood, but generally limited in scope. Our attempt to expand them to cover disagreements of all kinds is based on informal theories of social interactions and has met with some preliminary success, but is still being iterated and has yet to receive formal study.

There are many ways to categorize disagreements. We can organize them by content—religion, politics, relationships, work. We can divide them into questions of opinion and matters of fact. We can talk about disagreements that “matter” (in which we try very hard to convince others of our view) versus those where we can calmly agree to disagree.

In most cases, disagreements revolve around the outputs of models. What we mean by that is that each person in a disagreement has a 1) certain set of base assumptions about the world, 2) a certain toolkit of analyses, algorithms, and perspectives that they bring to bear on those assumptions, and 3) a number of conclusions that emerge from the combination of the two. Every person is essentially a black box which takes in sense data from the universe, does some kind of processing on that sense data, and then forms beliefs and takes actions as a result.

This is important, because often disagreements—especially the ones that feel frustrating or unproductive—focus solely on outputs. Alice says “We need to put more resources into space exploration,” and Bob says “That’d be a decadent waste,” and the conversation stops moving forward because the issue has now become binary, atomic, black-and-white—it’s now about everyone’s stance on space, rather than being about ...
Example Meetup Description
by Julius — 2022-07-24T05:38:06.773Z

This post contains sample meetup descriptions that people have used to promote their communities to the outside world. Feel free to copy and adapt anything to fit your needs.

Here is one created by Eliot from the Sydney meetup:

If you are not sure if you should come, you are welcome! Do show up anyway!

What is the science and art of human rationality? We are a group of deep thinkers. We talk about thinking, we talk about human coordination, science, and futurism. You will most likely find us trying to solve the problems of the world. A task that usually forces us to read widely and be curious about everything we can be curious about.

This is our regular monthly pub meetup where we talk all things math, statistics, medicine, science, economics, technology, engineering, thinking, communicating, personal growth, reasoning, and beliefs.

There is no set program and we usually share what we are working on at the moment (which is why our topics are so varied).

If you are an aspiring rationalist, a nerd, a geek, a scientist, or just a quiet thinker - we can't wait to meet you to hear what you are thinking about. We love to share ideas, discuss, debate, learn and grow together.

If you are interested in our monthly rationality dojos, ask us about them in person.

Bring a book, bring a paper, bring a friend along too!

We usually get 5-10 attendees through Facebook, LessWrong, SSC/ACX and Meetup.

Here is one from the Portland meetup:

Welcome to the hub for all things in Portland related to effective altruism and rationality aka PEAR! Effective altruism is about using reason and evidence to do the most good we can with our limited resources. For more information about how we do this, visit effectivealtruism.org. In the rationality community, we aspire to improve our reasoning and decision-making and address the biases that prevent us from holding true beliefs and accomplishing our goals. For more information on rationality, here’s a brief explanation on LessWrong, or dive into some of the posts on Scott Alexander’s blog.

Please feel free to come out, even if, like many of us, you feel awkward, feel that you're not the typical rationalist or effective altruist, are worried that people won't like you, are just curious, or have read this far, etc. Any event is welcoming but PEAR Social dinners are especially casual. We would love to have you join Portland's effective a

...
Eavesdropping on Aliens: A Data Decoding Challenge
by anonymousaisafety — 2022-07-24T04:35:40.880Z

This is my reply to faul_sname's claim in the "I No Longer Believe Intelligence To Be Magical" thread:

In terms of concrete predictions, I'd expect that if we

Had someone² generate a description of a physical scene in a universe that runs on different laws than ours, recorded by sensors that use a different modality than we use
Had someone code up a simulation of what sensor data with a small amount of noise would look like in that scenario and dump it out to a file
Created a substantial prize structured mostly³ like the Hutter Prize for compressed file + decompressor

we would see that the winning programs would look more like "generate a model and use that model and a similar rendering process to what was used to original file, plus an error correction table" and less like a general-purpose compressor⁴.

...

If you were to give me a binary file with no extension and no metadata that is

Above 1,000,000 bytes in size
Able to be compressed to under 50% of its uncompressed size with some simple tool like gzip (to ensure that there is actually some discoverable structure)
Not able to be compressed under 10% of its uncompressed size by any well-known existing tools (to ensure that there is actually a meaningful amount of information in the file)
Not generated by some tricky gotcha process (e.g. a file that is 250,000 bytes from /dev/random followed by 750,000 bytes from /dev/zero)

then I'd expect that

It would be possible for me, given some time to examine the data, create a decompressor and a payload such that running the decompressor on the payload yields the original file, and the decompressor program + the payload have a total size of less than the original gzipped file
The decompressor would legibly contain a substantial amount of information about the structure of the data.

Here is such a file: https://mega.nz/file/VYxE3T5A#xN3524gW4Q68NXK2rmYgTqq6e-2RSaEF2HW8rLGfK7k.

It is 2 MB in size.
When I compress it with zip using default settings, I see it's ~1.16 MB in size. or a little over 50%.
I have tried to compress it to smaller sizes with various configurations of 7zip and I've been unable to get it significantly smaller than 50% of the file size.
This file represents the binary output of a fake sensor that I developed for the purpose of this challenge.

There is no "gotcha" on what type of sensor is used, or what the sensor is being used to detect. It will be obvious

...
Information theoretic model analysis may not lend much insight, but we may have been doing them wrong!
by Garrett Baker — 2022-07-24T00:42:14.076Z

Introduction

For a SERI MATS research sprint under John Wentworth, teams were given the following research prompt:

Everyone's assignment will be to figure out how to operationalize the "broad peaks induce data compression" argument. Here's the intuitive argument: I train some neural net to reproduce the training data. Conceptually, the trained net has to "store" all the relevant information from the dataset in its weights. If it "compresses the data", then it should be able to store the data using fewer weights, so more weights are free to vary without increasing loss much. "Lots of weights left free to vary without increasing loss much" = broad peaks. Or, to put it differently: there are more parameter-values for which the net reproduces the training data using a more compressed representation, so training is more likely to find a more compressed representation.So, the assignment is to figure out how to operationalize things like:

"the representation of the data" (or the embedding of that representation in the weights)
what it means for the representation to "use" some parameter-degrees-of-freedom but not others (bear in mind that "weights" might not be the right ontology, e.g. maybe it uses some directions-in-parameter-space but not others, or maybe something else entirely)
what it means for two different parameter-settings of the net to "use the same representation" (or, more generally, to "do the same thing" internally)

When operationalizing this, we're looking for three main things:

Testability: the operationalization should be in terms of things which you can directly look for in a trained net (possibly relying on some approximations). This is mainly to make sure the operationalization stays grounded.
Generalizability: the operationalization should be clearly generalizable to new kinds of systems which don't look like the kinds of systems we're currently working with (maybe imagine biological systems). We're not looking for a narrow ad-hoc model, here.
Provability: using the operationalization, we should be able to turn the intuitive argument into a mathematical proof (up to approximation/events of high probability). We should be able to argue mathematically that training probably finds more compressed representations.

Our team decided to try to tackle what it means for two different parameter-settings of the net to use the same representation, as this seemed th...
What's next for instrumental rationality?
by Andrew_Critch — 2022-07-23T22:55:06.185Z

Preceded by: Curating "The Epistemic Sequences" (list v.0.1)

Epistemic status: speculations and ideations from me about the potential for further progress on broadly accessible instrumental rationality content.

In Curating "The Epistemic Sequences", I explained how the epistemic content of the LessWrong sequences has a different epistemic status than the instrumental content, i.e.., content on how to behave. So what's next for instrumental rationality? It would be great if there were a how-to-behave version of the sequences that was built on foundations as strong as logic, probability, statistics, and causal inference.

Unfortunately, those foundations don't yet exist. There aren't formal foundations for decision theory, game theory, ethics, meta-ethics, and political theory that are "tried and true" the way logic, probability, statistics, and causal inference ("L+P+S+C") are.

Many people argue that universally-useful how-to-behave instructions can't exist, on the the grounds that "philosophers have been trying to solve these issues for millennia", but I'm not sure that's a strong case. After all, philosophers had been trying to develop truth-seeking techniques for millennia prior to the 20th century, and then along came a bunch of progress in L+P+S+C with widespread applications, enabling what might be called an "epistemic enlightenment (for individuals)", which arguably culminated in the epistemic content of the LessWrong sequences. And, perhaps in the next decade, there could be progress in the theory of embedded agency and multi-agent rationality ("E+M") leading to real-world applications as robustly useful and well-vetted as L+P+S+C are today. If breakthroughs in embedded & multi-agent rationality then remained in practice for something like 30 years of applications in broad-sweeping domains (or, an AI-augmented equivalent of 30 human-civilization-years) the way L+P+S+C have, perhaps then will be a good time for someone to write the "The Instrumental Sequences", and a new generation of instrumentally enlightened people will look back and wonder why it was considered impossible to derive a principled account of how individuals should behave.
Easy guide for running a local Rationality meetup
by nsokolsky — 2022-07-23T22:52:02.233Z

This weekend I've participated in a retreat in Berkeley for organizers of prior ACX meetups, with around 50 meetup organizers attending. This post serves to write down my learnings from the sessions on the basic logistics of meetups and hopefully provide a reference document for future. This is an addition to the canonical meetup organizer guide from 2012, trying to integrate some new lessons over the past decade.

I'm not an organizer but want to run a new meetup in city [X]. Can I do this?

Yes, you can. Anyone can become an organizer right now or cease being an organizer at any moment of time. There's no authority or gate keeper bestowing the honor on would-be organizers. Just try to follow some basic common sense rules and hopefully your event will turn out fine. If there are already organizers in your city, you can check in with them on potentially co-hosting something, but you can also just go ahead and create a new meetup.

Where to announce the event?

The key to event announcements is to be mindful of the fact that no single platform is visited by all Rationality fans and that no platform is necessarily the place where people go looking for meetups. Just because you or I read LessWrong doesn't mean that every person reading LessWrong will necessarily visit the Events section to check out upcoming events in their city. They might not know about the Events tab, never check it because it almost never has events in their city, not interested in checking it because it doesn't integrate well with their favorite calendar, etc. So you want to advertise your event on every relevant platform that allows for such announcements.

Online platforms

LessWrong events page
LessWrong community page (scroll to "Local Groups"). If you don't have a local group yet, you can create one. Some people only check their local group page for events.
Open threads and Schelling meetup posts by Scott Alexander. Great way to find people who are LW-adjacent but don't actually read LW.
Rationality groups on Meetup.com, i.e. here's one for Seattle. If your city doesn't have a Meetup group, create one!
Facebook rationality groups. You can find one for your city in this wiki.
Relevant subreddits, especially /r/slatestarcodex
Rationality-adjacent Discord/Slack/Whatsapp channels for your city
Email lists with a lot of Rationality community members, if one exists for your city

If there's a platfo...
Curating "The Epistemic Sequences" (list v.0.1)
by Andrew_Critch — 2022-07-23T22:17:08.544Z

Summary for regular readers: The epistemic content of The Sequences — i.e., the advice on finding true beliefs — has a different epistemic status than the instrumental content — i.e., the advice on how to behave. Specifically, the epistemic content is based upon techniques from logic, probability, statistics, and causal inference that have already been heavily explored and vetted, both formally and empirically, in a wide range of disciplines and application areas. This is not true of the how-to-behave content of the sequences, such as the stances presented on individual heroism and what does or not not constitute humble behavior. Indeed, the analogous underpinning fields — namely, decision theory, game theory, ethics, meta-ethics, and political theory — are not nearly as well explored-and-vetted as the epistemic underpinnings. As such, the epistemic content of the sequences should be made available in a separate compendium, curated for its special epistemic status. This post is one attempt at that, which I'd be happy to have replaced or superseded by a more official attempt from, say, the LessWrong team, or Eliezer Yudkowsky.

Followed by: What's next for instrumental rationality?

Introduction for newcomers

The good

The "Epistemic Sequences" curated below are taken from the writings of Eliezer Yudkowsky. Yudkowsky's writings cover some of the best introductions I've seen for helping individual people to start forming true beliefs about the world, or at least, beliefs that become progressively less wrong over time. The Epistemic Sequences describe processes to follow to form true beliefs, rather than merely conclusions to reach, and the prescribed processes are well backed by logic, statistics, and causal inference. They draw inspiration from well-written, tried-and-true textbooks like these:

(2000) H. B. Enderton: A Mathematical Introduction to Logic
(2003) E.T. Jaynes: Probability Theory: The Logic of Science
(2000/2009) J. Pearl: Causality: Models, Reasoning, and Inference*

(* J. Pearl: Causal Inference in Statistics: A Primer is a more recent and friendlier intro.)

The questionable

Epistemic status: broadly verifiable claims about fields of theory and practice and their relation to the LessWrong Sequences.

For better or worse, the broader LessWrong Sequences and Sequence Highlights also contain a lot of strong stances on how...
Room Opening
by jefftk — 2022-07-23T21:00:08.296Z

One of the housemates we share our 8br 2-unit Somerville MA

house with is moving across the
street, and we're looking for another one! Their room is opening up
October 1st, though they could potentially be out by September 1st if
need be. We also have an adjacent bedroom that's open now but isn't
as nice, and also an office-type room in the backyard. We could rent
out any combination of these rooms. We'd prefer to leave the rooms
empty than go with a housemate that's a poor fit, and we could
potentially hold the space for someone who'd be a great fit but won't
be in the area for a while.

Current housemate constellation:

Upstairs apartment: Our family, which is Jeff, Julia, Lily (8),
Anna (6), Nora (1). This is where the spare rooms are.

Downstairs apartment: David, Al, their two cats. If you like to
hang out with cats, you could probably visit Willow and Betsy.

Across the street: Ruthie, eventually probably other friends of
Ruthie's.

There may be more kids around at some point, either Ruthie's,
David and Al's, or both.

House culture:

We eat dinner together about 5 nights a week, with each adult
cooking once a week. We plan to continue something like that (with one
night happening at Ruthie's house), though the details might
change. We share groceries and leftovers, though it's fine to have
individual food and label it as yours. Food has been averaging
$200/person/month.

One adult is vegan, one is vegetarian, one can't eat tomatoes
or spicy things, and the other two adults are omnivores. Sometimes
I'll try to cook one thing that everyone who'll be at a meal can
eat, but often I find it easier (or tastier) to cook multiple things.

The current group has some involvement with effective altruism and
folk dancing, so if you're interested in at least one of those topics
you'll likely enjoy dinner conversation more! Other common topics:
being a manager in a startup, silly puns, stuff the kids are up to.

We're all vaccinated and fairly covid-chill at this point. We've
hosted indoor events with masks and/or testing.

Someone asked if we expected housemates to do childcare: no, we
don't. One of our housemates did some (paid!) childcare for a while;
if that's something you're interested in we could talk?

We host events 1-2 times a month inside or in the backyard (EA
dinners, music/singing parties.) We'd also be open to yo

...