trentmkelly/LessWrong-43k huggingface dataset:
Theories of Modularity in the Biological Literature
by CallumMcDougall — 2022-04-04T12:48:41.834Z

Introduction

This post is part of a sequence describing our team’s research on selection theorems for modularity, as part of this year's AI Safety Camp, under the mentorship of John Wentworth. Here, we provide some background reading for the discussion of modularity that will follow.

As we describe in more detail in our project intro, the motivating question of modularity that we started with (which is described in John’s post on the Evolution of Modularity) is why does evolution seem to have produced modular systems (e.g. organs and organ systems), but current ML systems (even genetic algorithms, which are consciously fashioned off evolutionary mechanisms) are highly non-modular? So far, most of our research has focused on the idea of modularly varying goals (MVG), but this is not the only proposed cause of modularity in the biological literature. This post serves as a literature review, with a few brief words on how we are thinking about testing some these hypotheses. Subsequent posts will discuss our team’s research agenda in more detail.

Theories of modularity

Modularity in the environment selects for modular systems

Basic idea: If we have an environment which contains a variety of subproblems and which changes rapidly in highly modular ways, this might select for a modular internal design to better adapt to these problems. In the literature, this usually takes the form of “modularly varying goals” (MVG). If we vary one aspect of the agent’s goal, but keep the rest of the goal and the environment constant, then this might produce a selection pressure for modular systems in correspondence with this modular goal structure.

Biological modularity: Many examples of modularity in biology seem to have a pretty clear correspondence with goals in the environment. For instance, type of terrain and availability of different food sources might vary somewhat independently between environments (or even within the same environment), and correspondingly we see the evolution of modular systems specifically designed to deal with one particular task (muscular system, digestive system). One can make a similar argument for reproduction, respiration, temperature regulation, etc.

Evidence: The main paper on this idea from the biological literature is Kashtan & Alon’s 2005 paper. Their methodology is to train a system (they use both neural networks and genetic algorithms) to learn...
Announcing Duncan Sabien as the keynote speaker for the 2022 LessWrong Community Weekend August 26-29!
by ChristianKl — 2022-04-04T12:23:29.778Z

We will start taking applications on May 1, with rolling responses until all of the spots are filled. At that time an announcement will be made with all additional details.

For four days at the end of August, aspiring rationalists from all around Europe and beyond will gather at the lovely Lake Wannsee near Berlin to socialize, run workshops, talk, and enjoy our shared forms of nerdiness.

This year our keynote speaker will be the fascinating and incisive Duncan Sabien. He currently works at MIRI and is the former director of curriculum at the Center for Applied Rationality.

Duncan regularly posts valuable new essays on Less Wrong. Some examples of his work are:

In Defense of Punch Bug

,

Lies, Damn Lies, and Fabricated Options

r!Animorphs: the Reckoning

He was the primary person preparing the

CFAR Handbook

and also held a talk at EA Global

Creating a Personal Autopilot

.

Volunteers: We still can use more help to organize the event. A meetup of this scale takes a lot of work to put together, and If you want to be part of the team turning this into reality, write us an email at

lwcw.europe@gmail.com

.

Also if someone is interested in creating an activity, or object or something else that will improve the atmosphere or location or create cool new opportunities for interaction, for example like an art installation, or like the (enormously fun) cuddle fort that is made each year in the basement, there is funding to help make it happen. Send us an email to explain your idea if you’d like to make something like that.

What the event is like: While the keynote speaker will be awesome, most of the content is created by the participants. On Friday afternoon we will put up four giant daily planners, and by Saturday morning they are usually filled up with dozens of talks, activities, workshops and events that the attendees have decided to run.

Aside from the content the attendees organize, it’s the serendipitous encounters, shared meals, late night discussions and new friendships that really make the event.

And of course if you want to spend some time relaxing and recharging on your own you can hike in the forests, sunbathe or stroll lazily along the banks of Lake Wannsee whenever you like. The venue for the event, Wannsee youth hostel, is nestled amongst the beautiful lakes and forests of South West Berlin and provides shared accomm...
Ukraine Post #8: Risk of Nuclear War
by Zvi — 2022-04-04T12:00:01.003Z

It seems worth going through the exercise of estimating the probability of nuclear war, and in particular the probability of it causing one’s death. If the probability gets high enough, one can strongly consider being elsewhere or otherwise doing something about it.

Note that all scoring rules and wagers are essentially useless here. You can look back and decide whether your reasoning was good, but saying ‘I was right’ is meaningless.

As a baseline to work from, this EA forum post presents multiple perspectives on nuclear war risk in terms of the danger of being in London, with the author of the post modeling risk as relatively high, versus some superforecaster predictions that modeled risk as relatively low. The forecasts are divided into steps.

Will there be a conventional exchange between NATO and Russia?
Will there be a nuclear exchange?
If there is a nuclear exchange will it hit London?
If it does will you have to get out?
If you don’t get out, will you die?
(Alternate path) Background risk of accidental nuclear war, which is higher when everyone is on alert.

If you multiply the odds of each step together, you get the level of danger from being in London. The level of danger in New York City should be similar.

Aside from the chances of the bomb killing you if it lands, these differences all point in the same direction, with two of them being on different sides of 50%. It is an interesting exercise to read the arguments, and to decide on one’s own opinion on each leg.

Probability of conventional war

There are two ways to get a conventional war. Russia could attack NATO and cause NATO to invoke Article 5 without an intervention in Ukraine, or NATO could decide to intervene in Ukraine.

We have a Metaculus market on whether a NATO country will invoke Article 5 by the end of the year and it is sitting at 5%. That seems reasonable.

I think there is a modest chance that Article 5 is technically invoked but there is nominal fighting, and there is a similar chance that NATO does something crazy and intervenes first for whatever reason. 5% here seems fine.

Dynamics of nuclear deterrence have changed

In the Cold War, both sides felt NATO had woefully inadequate conventional forces, and would need to resort to either lose or use nuclear weapons within weeks if the Warsaw Pact invaded Western Europe.

This has now reversed. Russia’s conventional forces have been e...
Best non-textbooks on every subject
by Yair Halberstadt — 2022-04-04T11:54:56.193Z

The best way to learn a subject is undoubtedly by reading a textbook on it. But I find textbooks a drudgery, and tend to give up after a couple of chapters.

On the other hand I don't need a deep broad formal knowledge in every subject. I often just want to know enough that I know what it's about, the broad questions in the topic, and how to learn more when I need to.

On the other hand popular books are easy to read, but often teach you about the subject, without actually teaching any of the subject itself. They're full of anecdotes about the founders of the field, and metaphors for what some of the fields are like, but at the end you may end up more misguided than you went in.

There are however the rare popular books that aim to actually give the reader useful knowledge, rather than the illusion of knowledge. For example Godel, Escher, Bach on logic and formal systems, Quantum Computing since Democritus on computer science and Who We Are and How We Got Here on ancient DNA.
These examples vary hugely in how involved they are, their style, and how readable they are, but they all share one thing in common: none of them talk down to the reader - they all assume the reader is an intelligent person whose perfectly capable of understanding the topic, but might just be missing a lot of background knowledge.
What other books do you know of like that?
Ideally all answers should give the title of a single book, optionally with a brief description, and a set of bullet points describing what they liked and didn't like about the book.

I'm more interested in physical sciences than social sciences, since it's common in the social sciences to introduce a thesis in book form, so it's easy to find good quality non-textbooks. Meanwhile in the physical sciences most original research is done in research papers, and most pedagogical work in textbooks, leaving much poorer pickings for non-textbooks.
Why are we so early?
by Flaglandbase — 2022-04-04T08:49:03.103Z

Countless people have wondered about one of the most obvious cosmological puzzles. It's a very simple question with no answer:
If the universe could exist forever, why do we exist almost as close to the beginning of time as possible?

Some very sophisticated attempts have been made to answer it, including what is known as "Doom Soon" (we exist this early because we don't have a future).

However, our existence is biased by the physical universe in which we are embedded. There are many reasons to think human minds are NOT a random selection from the list of all possible minds (for example, most possible minds would be absurd and infinite).
Our type of universe may be able to generate and vastly amplify our mind type, to the extent that we outnumber all other mind types everywhere else. The only thing that makes us special is how common we are. Commonness implies simplicity implies a brief past.

The existence of an undeniable "Earlyness Bias" in our existence also suggests we are not ancestor simulations.
Well we are, as our minds are a blend of all possible instances everywhere - but only an infinitesimal fraction of these instances are ancestor simulations.
This is indicated by the fact we have only a brief past. If there will be an infinite future, there will be infinitely more advanced ancestors to simulate than us. There would be fewer reasons to simulate early ancestors. Simulating us would be like solving 1 + 1.

(This also implies that if you actually are an ancestor simulation, you are most likely being simulated in the nearest future when that technology can exist.)

The explanation for why we live in a young universe may be that our type of universe is extremely easy to generate. Much easier than suspected from studying quantum chromodynamics or string theory.

Reality may be entirely mathematical, the only real thing being logical functions. While our physical universe is vast, it may be a result of a very simple and abundant equation.
New universe equations are constantly sprouting off from logical functions within our own universe and everywhere else. If every older universe is constantly spinning off new ones, most observers would find themselves in young universes.

A mathematical equation that could generate something like a universe is still far beyond us. Conway's Game of Life or Mandelbrot fractals don't even come close.
For all we kn...
My Superpower: OODA Loops
by Gordon Seidoh Worley — 2022-04-04T01:51:46.622Z

I've pushed the value of OODA loops in various posts over the years, but I don't recall ever writing a post specifically about them. Which is weird, because they're a superpower. So much so that I think they are at least 60% responsible for what enables me to lead a life I enjoy (the rest is some mix of luck, privilege, and high intelligence). And unlike those other things, you can learn to use OODA loops.

OODA stands for Observe, Orient, Decide, and Act. The concept was created by John Boyd. His version has a lot of details I'll probably leave out as it's not how I actually think of them (more on that later). The loop consists of these parts:

Observe. Notice what's going on. Stop and pay attention to what's happening around you. This is the skill of getting your expectations about what you are going to observe out of the way and actually noticing what's happening in front of you. This is the skill of seeing the world with fresh eyes so you can respond to it.

Orient. Now is the time when you make sense of what you've observed. Apply your priors to come up with some prediction about what's happening in the world. Use the models and context you have to make sense of things. This is where you might start to get things wrong, jump to the wrong conclusions, and so on. That's fine, it's all part of the process, just do the best you can.

Decide. You're going to have to decide to do something based on the information you have, even if it's to take the default action of continuing to observe and orient until you can come to some other decision. The decision making process usually looks like choosing from among a few options that became apparent during the orient phase, although when it's functioning smoothly orient will suggest a single course of action to decide on: the best one you know.

Act. Now that you've decided what to do, do it. Don't hesitate, you've already decided, just perform the action. Once you've done it, observe what happens and start the process all over again.

Okay, so that's a fairly stilted way of describing OODA loops. It's like that because some people are really bad at iterating on feedback to make things better, both on short and long time horizons, so it needs to be somewhat explicit to help them break the skills down and work on them until they become good enough at them to do it naturally.

In fact, I don't really think about OODA loops much excep...
Bayeswatch 6.5: Therapy
by lsusr — 2022-04-04T01:20:20.862Z

"What do you mean I need therapy?" said Vi.

"Our records indicate you may have suffered psychological trauma on one of your recent missions," said Eliza.

"Which mission?" said Vi.

"You know," said Eliza. She leaned close to whisper, "Bangui."

"I can neither confirm nor deny whether I have ever been to Bangui," said Vi.

"You're too young to deal with the psychological impact of the psyop weapon the Bangui cloners deployed," said Eliza.

"I can neither confirm nor deny who was too young for Bangui. If you want me to go to therapy for psychological trauma then tell me, in writing, where I received the trauma and why. Otherwise, how can I prevent it in the future?" said Vi.

"This isn't healthy. You should be on mandatory medical leave," said Eliza.

"Are you allowed to do that?" said Vi.

"Not without a reason," said Eliza.

"Is that all?" said Vi.

"There is one more thing. Miriam recommended you be promoted. All that remains is the psychiatric sign-off. I need to certify you're mentally stable enough for the job," said Eliza.

"Does that mean?" said Vi.

"Congratulations, Agent," said Eliza.
Bayeswatch 9.5: Rest & Relaxation
by lsusr — 2022-04-04T01:13:13.461Z

Vi lay down on the couch in the real half of the therapy room. The other half was illusory. The real and the imaginary halves were separated by a 3D viewscreen. It only worked on one viewer at a time which was fine because only one patient at a time was allowed into the therapy room. Therapy was private. The room wasn't just electromagnetically shielded. It hung from wires in a vacuum to prevent sounds from leaking out.

"You're racking up quite the body count," said Eliza.

"You're talking about Natanz? Those were enemy combatants. They were literally in uniform," said Vi.

"I'm not talking about Natanz," said Eliza.

"You mean Yamongala? Those were enemy combatants straight out of an unaligned AI's wetfactory. I'm not happy about what happened there but there was no other responsible option. Should I have let them break quarantine? Should I have let them shoot me?" said Vi.

"I'm not talking about Yamongala," said Eliza.

"Then what are we doing here?" said Vi.

"We're talking about Juba," said Eliza.

There was a pause.

"I understand why Miriam smokes so much," said Vi.

"I know you didn't mention Juba in report. But Bayeswatch analyists can put two and two together. We know you and/or Miriam released the unaligned Z-Day AI on the civilian population of Juba to provoke a response by our adversary in Natanz," said Eliza.

"I can neither confirm nor deny…" said Vi.

"You can talk to me. My hard drive is wiped after each session," said Eliza.

Vi rolled her eyes.

"Fine. Don't talk to me. I'll place you on paid medical leave until you do," said Eliza.

"Bayeswatch would no longer exist if it weren't for our actions in Juba," said Vi.

"You deliberately released an unaligned AI. The raison d'être of Bayeswatch is to prevent unaligned AIs from going rogue," said Eliza.

"What was an unaligned AI doing in the basement if it wasn't intended to be used?" said Vi.

"The answer to that question is beyond your level of security clearance," said Eliza.

"Can I have a cigarette?" said VI.

"No," said Eliza.

"Look, it's easy to judge the actions of field agents from an ivory tower. You never have to look an engineer in the eye while you EMP his life's work. You never have to throw miracle cures in the medical waste because they might help an AI escape its box. You never have to booby trap scientists' cars," said Vi.

"You've never assassinated a scientist," said Eliza.

"I just met an AI researcher who lost her ...
A Word to the Wise is Sufficient because the Wise Know So Many Words
by lsusr — 2022-04-04T01:08:27.688Z

Collect Ontologies

An ontology is a way of bucketing reality. For example, Russians distinguish голубой from синий whereas Anglos bucket both into "blue". An ontology is an implicit set of priors about the world. If you never bucket people according to skin color then you will be bad at predicting who prefers chopsticks over forks. If you always bucket people according to skin color then you will miss out on human universals.

It takes an untrained neural network a long time to distinguish poisonous snakes from nonpoisonous snakes. It will take you much less time if you treat coral snakes and milk snakes as separate species, even though they look similar to each other.

Different ontologies are appropriate for different contexts. Zero-sum models of the world work great when planning adversarial strategies. Zero-sum models of the world are counterproductive when attempting to put together a family dinner. The more different ontologies you know, the the faster you can process new data.

The more ontologies you know the more information it takes you to distinguish between them. In practice, it doesn't matter because the entropy required to distinguish ontologies is so tiny[1].

The (Cheap) Price

How many ontologies can a person learn? Words aren't ontologies but I think the number of words a person knows provides a reasonable Fermi estimate of the number of ontologies bouncing around our heads. A normal person knows perhaps 32,768 words. If we treat each word as a distinct binary ontology of equal prior probability then it takes 15 bits of entropy to figure out which ontology to use.

The entropy of English text is estimated at 2.3 bits per letter. It takes approximately 6 letters to figure out what word a person is trying to say. Technically, this is just a roundabout way of saying the average English word is about 6 letters long but my point is even with tens of thousands of ontologies to draw from, it takes a tiny bit of entropy (on the order of one word) to find the needle in the haystack. A word to the wise is literally sufficient.

That's assuming all prior probabilities are equally probable. If we allow for unequal prior probability distributions then the median entropy cost is less than a single word. The more ontologies you learn, the smarter you become, because each ontology contains a heavy Bayesian prior you can apply to new situations you encounter. The extre...
Counterbiases & Shifting The Default
by ReluctantRaccoon — 2022-04-04T01:05:57.849Z

Epistemic status: rough thoughts, but the technique described has been useful for me

Suppose you know you have some bias, but also that you can't remove it. One option is to notice the bias, then try to think your way out of it - at least a little. For example, in The Scout Mindset Julia Galef suggests combating status quo bias by inverting the status quo and asking yourself if you would still make the same decision. (E.g. if considering moving from city A to city B, would you make the same decision if you were currently in city B and had the option to move to city A.) This test is imperfect but can help combat bias.

But you could also, or instead, insert a counterbias - a rule of thumb that, without the existing bias, would bias you in the opposite direction, but with the existing bias, tends to move you closer to the truth.

For example, when I am running late for a meeting, my first intuition is almost always to give the most optimistic projection I can for when I will be there. Even when I try to consider the question of when I'll be there objectively, I'm influenced by wishful thinking and tend to underestimate how long I'll take by a long way. This is the bias.

The counterbias is to give what seem like extremely pessimistic predictions for when I'll arrive in these situations. If I think I'll be five minutes late, I might say I'll be there in the next fifteen minutes. This feels wrong but is usually right. It's not always right and will sometimes produce errors, but I'm less wrong over time than if I weren't to use these techniques.

Here are some other examples I think are useful:

Not sure whether there's some truth in a criticism? Assume there is, then search for it. (Counters the bias against taking criticisms seriously.)
Not sure whether you should take responsibility for some error? Assume you should. (Counters the bias against taking responsibility - if you have the opposite bias, then do the opposite.)
Decision frustrating you and you want to be done with it? Default to provisionally committing and sleeping on it, rather than irreversibly committing straight away. (Counters the bias towards making a commitment prematurely.)
Have an easy and a hard choice available to you? Default to the hard choice. (Counters the bias towards conserving energy.)

How do you develop a counterbias? You notice a specific situation in which you are biased, design a counterbias, ...