trentmkelly/LessWrong-43k huggingface dataset:
Wise Crowd & Democratic Spirit
by Hristo Zaykov — 2022-09-27T17:45:00.141Z
Link: https://www.hristo.blog/post/wise-crowd-democratic-spirit

“There will be no end to the troubles of states, or of humanity itself, till philosophers become kings in this world, or till those we now call kings and rulers really and truly become philosophers, and political power and philosophy thus come into the same hands.”

- Plato, 'The Republic'

For aeons people have considered the following questions to be the primary problems of organising a society:

How should we make large-scale decisions?
What system of governance should we use?
What/who should have power?

As loose thinking on the subject accumulated into the field of political philosophy, many ideas converged on a fantasy: power should be bestowed upon the wisest, what is optimal should be considered and decided by the best thinkers. In this light, a political system is judged on its tendency to select for the best politicians. A king may not be wise, but the elected can be. And in order to avoid the biggest follies of humans - conflict and self-interest - we should all have equal say. Voilà, democracy.

But there are fundamental misconceptions in that story. The above are actually secondary problems of organising a society. The primary one is not how we should make decisions, but rather how we should move on, once bad decisions have been made.

Moreover, Plato is wrong in implicitly idealising an end to the troubles of states, or of humanity itself. As far as this end exists, it is completely undesirable. Such a situation - with no troubles - could only occur in lifeless stagnation. Life, if it is joyful, vibrant, and creative, is riddled with problems. I’m using the word problem in a wider, Deutschian sense here: a conflict between ideas. This can be culturally counterintuitive because the common meaning of the word fixates on an unwelcome or harmful situation. Some problems are indeed bleak by nature, just as some ideas are, but in the wider sense, the existence of problems is good in principle.

The richer the world of ideas becomes, the more problems arise. Thus solutions create more (better) problems. The true assessment of a political system is contingent on how effectively it allows a society to transition between different problem-solving attempts. In essence: we try something, we fail to some degree, we are unconstrained to try something different.

This is why, in the tier-list of political systems, democracy stands alone. Not because when given group choice ‘th...
Soft skills for meetups
by mingyuan — 2022-09-27T17:26:12.406Z

This post is adapted from a memo I wrote for the 2022 global meetup organizer retreat, and owes some of its content to the organizers who attended.

'Soft skills' is a broad and somewhat fuzzy term, but the OED defines it as "personal attributes that enable someone to interact effectively and harmoniously with other people", and that seems close enough to me. Being able to interact effectively and harmoniously with other people is obviously an important part of running a community, and it can be hard to work on, especially because everyone is starting out with very different strengths and weaknesses.

The most important thing I have to say about this topic is:

If there's something that you don't know how to do, acknowledge that and find a way to work around it, rather than ignoring it / pretending it's not a problem.

For example, a lot of 'soft skills' involve reading non-verbal cues. If this is hard for you, maybe you can work on it, but in the meantime, you still need to be aware of conflicts, tension, and discontentment in your community.

One way to solve this would be to enlist a trusted friend / member of your community who's better at reading people, and make it their job to notice potential problems and bring them to your attention. This way you can still address problems despite your trouble with non-verbal cues. Similarly, if you find it hard to talk to people or ask people for things, you might have a friend do the actual talking in your stead.

Welcoming new members

Engage with first-time attendees

When a new person shows up, speak to them (ask their name, etc) and help them get oriented. Ask if they have any questions, and set expectations for what's going to happen at this event. You might also ask them what brought them there, what they're hoping to get out of it, etc, but just conversing at all is good. When you inevitably have to leave them, make sure you've introduced them into another conversation, rather than just leaving them alone.

Note: This doesn't necessarily just apply to in-person events! The Miami ACX organizer DMs every new person who joins his Facebook group.

Appoint Welcomers

A Welcomer is someone whose job it is to attend to the experience of new members, as described above. By default you as the organizer are in this role, but if your community is large, or if you struggle with shyness or awkwardness, you can delegate it to one o...
Enriching Youtube content recommendations
by Martín Soto — 2022-09-27T16:54:41.958Z

I always watch Youtube while eating at home to not get bored. I'd love for the content I consume to be somewhat useful or intellectually fulfilling. I'd love to hear any of your recommendations for that! Anything that skills me up in Alignment or rationality would be especially nice! I know there are some great podcasts (especially 80k), but I already listen to those in other circumstances (while walking), so I'd prefer recommendations that do significantly engage the visual part of audiovisual content.

Some of the content I've enjoyed/found useful in the past:

High level maths divulgation (this video is the perfect example). I'm a Maths grad, and so most of the medium level stuff is not as exciting.
Understanding Infra-Bayesianism (similar videos for other Alignment agendas very appreciated)
Peter Millican's courses on General Philosophy and Turing (these were a fucking blast)
Other Philosophy content, like Bonevac's course on The Analytic Tradition or Pessin's Philosophy of Mind. I've become gradually less interested in Philosophy, and that's why I don't browse Philosophy Overdose as much, but I still have a this-might-be-useful spot for Philosophy of Mind.
jan Misali (what a weird guy)
I used to really enjoy philosophico-political video essays (ContraPoints, PhilosophyTube, etc.), but I've mainly abhorred them (with the exceptions of Innuendo Studios and CCK Philosophy)
Neel Nanda's Introduction to ML Theory

Come on, throw me a bone here! Otherwise I'll go back to watching EAG talks I guess
The Onion Test for Personal and Institutional Honesty
by chanamessinger — 2022-09-27T15:26:34.567Z

[co-written by Chana Messinger and Andrew Critch, Andrew is the originator of the idea]

You (or your organization or your mission or your family or etc.) pass the “onion test” for honesty if each layer hides but does not mislead about the information hidden within.

When people get to know you better, or rise higher in your organization, they may find out new things, but should not be shocked by the types of information that were hidden. If they are, you messed up in creating the outer layers to describe appropriately the kind-of-thing that might be inside.

Examples

Positive Example:

Outer layer says "I usually treat my health information as private."

Next layer in says: "Here are the specific health problems I have: Gout, diabetes."

Negative example:

Outer layer says: "I usually treat my health info as private."

Next layer in: "I operate a cocaine dealership. Sorry I didn't warn you that I was also private about my illegal activities."

Negative example:

Outer layer says: "Is it ok if I take notes on our conversation?"

Next layer in: "Here’s the group chat where I mocked each point you made to 12 people, some of whom know you”

Positive Example:

Outer layer says "Is it ok if I take notes on our conversation? Also, I’d like to share my unfiltered thoughts about it with some colleagues later."

Next layer in says: "Jake thinks the new emphasis on wood-built buildings won’t last. Seems overconfident."

------------------------------------------------------------------------------------------------

Passing the test is a function both of what you conveyed (explicitly and implicitly) and the expectations of others. If it’s normal to start mocking group chats, then it doesn’t need to be said to avoid shock and surprise. The

illusion of transparency

comes to bite here.

Andrew:

Social friction minimization is the default trend that shapes the outer layers of a person or institution, by eroding away the bits of information that might cause offence, leaving layers of more pungent information underneath. The “onion model” of honesty or integrity is that each layer of your personality or institution should hide but not mislead about the layer underneath it. This usually involves each layer sharing something about the kinds of information that are in the next layer in, like “I generally keep my health information privat...
Book review: “The Heart of the Brain: The Hypothalamus and Its Hormones”
by Steven Byrnes — 2022-09-27T13:20:51.434Z

1. Introduction

1.1 Hypothalamus as “business logic”

In software jargon, there’s a nice term “business logic”, for code like the following (made-up) excerpt from corporate tax filing software (based on

here

):

def attachSupplementalDocuments(file):
if file.state == "California" or file.state == "Texas":
# SR008-04X/I are always required in these states
file.attachDocument("SR008-04X")
file.attachDocument("SR008-04XI")
if file.ledgerAmnt >= 500_000:
# Ledger of 500K or more requires AUTHLDG-1A
file.attachDocument("AUTHLDG-1A")

When you think of “business logic”, think of stuff like that—i.e., parts of source code that more-or-less directly implement specific, real-world, functional requirements.

By contrast, things that are NOT business logic include infrastructure & subroutines & plumbing that are generally useful in many contexts—e.g. code for initializing a database, or code for memory management, or code for performing stochastic gradient descent.

If genomes are the “source code” of brains, then they need to encode “business logic” too—specific calculations to do specific things that help an animal thrive and reproduce in its particular biological niche. For example:

if about_to_starve_to_death:
reduce_sex_drive()
increase_pain_tolerance()
emit_hunger_sensation()
increase_reward_for_eating()
if fertility > 0:
increase_sex_drive()
...

(We could also call these things “innate reactions”.)

Machine Learning people might interject here: Why does that have to be in the genome? Why can’t the brain derive those kinds of rules via a within-lifetime learning algorithm instead? Well, often it does! But:

Some things can’t be learned within a lifetime, because their evolutionary benefits are only apparent in hindsight, perhaps even many generations hence.
Other things can’t be learned within a lifetime, because not doing them, even once, is potentially fatal.
Still other bits of “business logic” involves setting up the animal’s within-lifetime learning algorithms. A particularly important exa

...
My Thoughts on the ML Safety Course
by zeshen — 2022-09-27T13:15:03.000Z

This summary was written as part of

Refine

. The

ML Safety Course

is created by

Dan Hendrycks

at the

Center for AI Safety

. Thanks to Adam Shimi and Thomas Woodside for helpful feedback.

Overview

Background

I recently completed the

ML Safety Course

by watching the videos and browsing through the review questions, and subsequently writing a short

summary

. As an engineer in the upstream oil and gas industry with some experience in dealing with engineering safety, I find the approach of the course of thinking in this framework especially valuable.

This post is meant to be a (perhaps brutally) honest review of the course despite me having no prior working experience in ML. It may end up reflecting my ignorance of the field more than anything else, but I would still consider it as a

productive mistake

. In many cases, if my review seems to be along the lines of ‘this doesn’t seem to be right’, it should be read as ‘this is how a course participant may misinterpret the course contents’. I am also well aware that it is much easier to criticize something useful than actually doing something useful.

For each section of the course, I will give a short summary, describe what I liked, and what I didn’t like. I may be especially brief with the parts about what I liked, and the brevity is no way a reflection about how much I liked it.

Thomas Woodside, who helped with creating parts of the course, has kindly provided feedback to this post. His comments are formatted in italics.

My Initial Expectations of the Course

Having engaged with AI Safety as an outsider, my general impression of the field were:

Predominantly based with AI FOOM scenarios and primarily concerned with abstract concepts like agency.
Even among prosaic AI alignment, it appears that the general research modus operandi is that people would (somewhat randomly) generate ideas that could be applicable to certain classes of AI safety problems. Although the ideas may be interesting and valuable, they tend to be rather narrow and specific problems and may not be scalable.
One of the more common arguments for advocating AI alignment is that failure to align AI systems lead to existential scenarios. From this perspective, a pragmatic approach towards AI safety with the aim of minimizing risks by reducing AI misalignments may not be very useful, since a s

...
Summary of ML Safety Course
by zeshen — 2022-09-27T13:05:39.828Z

This summary was written as part of

Refine

. The

ML Safety Course

is created by

Dan Hendrycks

at the

Center for AI Safety

. Thanks to Linda Linsefors and Chris Scammel for helpful feedback.

Epistemic status: Low effort post intended for my own reference. Not endorsed by the course creators.

I have also written a review of the course here.

Risk Analysis

Risk Decomposition

A risk can be decomposed into its vulnerability, hazard exposure, and hazard (probability and severity). They can be defined as below, with an example in the context of the risk of contracting flu-related health complications.

Hazard: a source of danger with the potential to harm, e.g. flu prevalence and severity
Hazard exposure: extent to which elements (e.g., people, property, systems) are subjected or exposed to hazards, e.g. frequency of contact with people who are possible flu virus carriers
Vulnerability: a factor or process that increases susceptibility to the damaging effects of hazards, e.g. old age and poor health makes someone more vulnerable to illness

Hazards can be reduced by alignment, vulnerability can be reduced via robustness, hazard exposure can be reduced by monitoring. Generally, risks can be reduced by systemic safety and increased by the lack of ability to cope.

Accident Models

The Failure Modes and Effects Analysis (FMEA) aims to identify the “root causes” of failure modes and mitigate high-risk events based on the risk priority. Risks are prioritized according to the “effects” which is made up of its severity, probability of occurrence, and detectability.

The Swiss Cheese Model illustrates the use of multiple layers of protection to mitigate hazards and improve ML safety.

The Bow Tie Model illustrates the use of preventive barriers to prevent initiating hazardous events and protective barriers to minimize the consequences of hazardous events.

All of the above models implicitly assume linear causality, which is not true in complex systems. In complex systems, reductionism does not work, as properties emerge which can hardly be inferred from analyzing the system’s parts in isolation.

The System-Theoretic Accident Model and Processes (STAMP) framework views safety as an emergent property, and captures nonlinear causalities.

Black Swans

Black Swans are events that are deemed as outliers but carry extreme impacts, and are more common i...
Probabilistic reasoning for description and experience
by Q Home — 2022-09-27T10:57:06.217Z

For some time I wanted to apply the idea of probabilistic reasoning (used for predicting things) to describing things, making analogies between things. This is important because your hypotheses (predictions) depend on the way you see the world. If you could combine predicting and describing into a single process, you would unify cognition.

Fuzzy logic and fuzzy sets is one way to do it. The idea is that something can be partially true (e.g. "humans are ethical" is somewhat true) or partially belong to a class (e.g. a dog is somewhat like a human, but not 100%). Note that "fuzzy" and "probable" are different concepts. A dog is somewhat like a human, not "probably" a human. But fuzzy logic isn't enough to unify predicting and describing. Because it doesn't tell how descriptions can depend on each other (unlike probability: probability tells how events can depend on each other). For example, how can the definition of "animal" depend on the definition of "human"? Fuzzy logic doesn't answer such questions.

I have a different principle for unifying probability and description. Here it is:

Properties of objects aren't contained in specific objects. Instead, there's a common pool that contains all possible properties. Objects take their properties from this pool. But the pool isn't infinite. If one object takes 80% of a certain property from the pool, other objects can take only 20% of that property (e.g. "height"). Socialism for properties: it's not your "height", it's our "height".

How can an object "take away" properties of other objects? For example, how can a tall object "steal" height from other objects? Well, imagine there are multiple interpretations of each object. Interpretation of one object affects interpretation of all other objects. It's just a weird axiom. Like a Non-Euclidean geometry. This sounds strange, but this connects probability and description. And this is new.

What do we need this axiom for? In general, for describing things happening simultaneously in your mental state. Because in mental states things tend to mix together. More specifically, my axiom can be used in classification and argumentation. Before showing how to use it I want to explain it a little bit more with some analogies. Core claims of the post:

There are two important types/usages of probability. One of them is the usual probability and the other is my "axiom".
There are 2 main typ

...
A Prince, a Pauper, Power, Panama
by Alok Singh — 2022-09-27T07:10:16.545Z

Instrumental convergence is an issue I think about a lot because it’s so natural. Being rich and powerful almost never works against you.

Take The Prince and the Pauper. It’s asymmetric.

The pauper’s side of the story can only happen because the prince allowed it, but the prince can just wear rags without anyone’s permission. It’s easy to go from prince to pauper, but not the other way around.

More abstractly, one with resources and power can always just leave it all behind and go live in the woods. Going the opposite way is The Story of Civilization.

What is power? I think of it as a large action space, with large orbits in the state space induced by that. A fancy way of saying “it’s having a lot of options”. Hence, the idea that power is the ability to do what you want. An omnipotent one would be maximally powerful, because it can do anything. Largest possible action (and state) space.

So one would want to gain power, pretty much independent of its goals. Other universal instrumental goals are lots of mass-energy under your control and minimizing energy usage for a given level of function. Influence –in the literal sense of matter flowing towards you– and the Free Energy Principle.
Double Asteroid Redirection Test succeeds
by sanxiyn — 2022-09-27T06:37:17.816Z
Link: https://twitter.com/NASA/status/1574539270987173903

Existential risks are important, and asteroid impact is one kind of existential risks. Thankfully, compared to other existential risks, asteroid impact is well funded.

Double Asteroid Redirection Test launched in November 2021, and just (September 2022) impacted Dimorphos, a moon of an asteroid 65803 Didymos. This "double asteroid" system was chosen because we can precisely measure velocity change of eclipsing binary systems.

Let us congratulate NASA and humanity for the job well done!
How would I know if a PhD is the right career path?
by Bob Guran — 2022-09-27T05:49:12.262Z

I'm working on my master's degree, and during a talk with one of my professors, the subject "Would you be interested in pursuing a PhD?" came up. I've given this a lot of thought over the last few months, but I'm at a loss for a response. I believe the issue is that with a bachelor's or master's degree, you have a good idea of what you're getting yourself into and what is expected of you, however with a PhD, it's much more difficult to get information (at least for me / in my country).

I had some work experience before going to university, and I know I don't want to spend the rest of my life doing monotonous office work. During my master's studies, I intend to participate in as many extracurricular events as possible in order to gain a grasp of "department life" or at the very least a sense of what it's like to work in science. I enjoy working with the team and the professor, although I have had more contact with the postdoc so far. With this respect, I have a few questions and your feedback might help to list the pros and cons

Other PhD students complain about the professor's ambiguity about the direction of their research. Isn't a PhD position about charting your own course and conducting research with the assistance of postdocs and professors? If there is a clear way, I believe we won't need PhDs or researchers, right?
What professional options will I have in the sector once I complete my PhD? Will I be considered overqualified?
Can I earn the same pay as someone who has worked in industry for 4-5 years if I opt to do a PhD and then transfer into industry?