trentmkelly/LessWrong-43k huggingface dataset:
Things that can kill you quickly: What everyone should know about first aid
by jasoncrawford — 2022-12-27T16:23:24.831Z
Link: https://jasoncrawford.org/first-aid

There are things that kill you instantly, like a bullet to the head or a fall from twenty stories. First aid can’t help you there. There are also things that kill you relatively slowly, like a bacterial infection. If you have even hours to live, you can get to the emergency room.

But there is a small class of things that will kill you in minutes unless someone comes to the rescue. There isn’t time to get to a hospital, there isn’t even time for help to arrive in an ambulance. There is only time for someone already on the scene to provide emergency treatment that either solves the problem, or stabilizes you until help arrives. Here, first aid can be the difference between life and death.

Not long ago I became a father. Being responsible for the life of someone so helpless and vulnerable spurred me to finally take first aid training, including CPR. Here’s what I learned from that experience, and what I think everyone should know about first aid.

What most of the things that kill you quickly have in common is that oxygen can’t get to your cells. If you are choking, oxygen can’t get in. If your heart stops beating, blood doesn’t flow. If you have a severe wound, you’re losing that blood rapidly. If any link in the respiratory-circulatory chain is broken, your cells are starved for oxygen and you have minutes to live.

The key first aid skills follow from this: CPR manually substitutes for heart and lung action; the Heimlich maneuver expels an object from the airway; a tourniquet stops life-threatening bleeding (on an extremity, at least—if the wound is elsewhere, there is a different technique, known as packing the wound).

The basic skills are remarkably simple. The course that I took was only a few hours of online instruction, followed by about an hour of in-person demonstration and practice with dummy patients. And I went through a lot of the optional material, including things like stroke, fainting, and jellyfish stings. I’m sure I’m nowhere near as good someone with more professional training or experience, but an introductory course is not daunting.

The most important thing I learned is that if you find yourself in an emergency situation, it is better to do almost anything rather than nothing. Again, if someone stops breathing for any reason, they have only minutes to live. They are dead by default, unless someone intervenes. There is very little you can do to the...
Why The Focus on Expected Utility Maximisers?
by DragonGod — 2022-12-27T15:49:36.536Z

Epistemic Status

Unsure[1], partially noticing my own confusion. Hoping Cunningham's Law can help resolve it.

Related Answer

Confusions About Arguments From Expected Utility Maximisation

Some MIRI people (e.g. Rob Bensinger) still highlight EU maximisers as the paradigm case for existentially dangerous AI systems. I'm confused by this for a few reasons:

Not all consequentialist/goal directed systems are expected utility maximisers

E.g. humans

Some recent developments make me sceptical that VNM expected utility are a natural form of generally intelligent systems

Wentworth's subagents provide a model for inexploitable agents that don't maximise a simple unitary utility function

The main requirement for subagents to be a better model than unitary agents is path dependent preferences or hidden state variables
Alternatively, subagents natively admit partial orders over preferences

If I'm not mistaken, utility functions seem to require a (static) total order over preferences

This might be a very unreasonable ask; it does not seem to describe humans, animals, or even existing sophisticated AI systems

I think the strongest implication of Wentworth's subagents is that expected utility maximisation is not the limit or idealised form of agency

Shard Theory suggests that trained agents (via reinforcement learning[2]) form value "shards"

Values are inherently "contextual influences on decision making"

Hence agents do not have a static total order over preferences (what a utility function implies) as what preferences are active depends on the context

Preferences are dynamic (change over time), and the ordering of them is not necessarily total

This explains many of the observed inconsistencies in human decision making

A multitude of value shards do not admit analysis as a simple unitary utility function
Reward is not the optimisation target

Reinforcement learning does not select for reward maximising agents in general

Reward "upweight certain kinds of actions in certain kinds of situations, and therefore reward chisels cognitive grooves into agents"

I'm thus very sceptical that systems optimised via reinforcement learning to be capable in a wide variety of domains/tasks converge towards maximising a simple expected utility function

I am not aware that humanity actually knows training paradigms that select for expected utility maximisers

Our most capable/economically transformative AI s

...
Mere exposure effect: 
Bias in Evaluating AGI X-Risks
by Remmelt — 2022-12-27T14:05:29.563Z

Note on unusual formatting: Sentences are split into lines so you can parse parts precisely.

Also called an 'Availability cascade',
a self-reinforcing process in which
a collective belief gains more and more plausibility
through its increasing repetition in public discourse
(or "repeat something long enough and it will become true").

Insofar as nearly all
of the published literature on risk assessment
assumes the same argument form, outline, logic, etc,
there is also established a kind of "mono-culture".
As with any other sort of mono-culture in nature,
the mere fact of that being that way ensures
that there are additional issues introduced:
a kind of brittleness and fragility.
Anything which impacts
the validity and applicability
of the single argument form,
will therefore also have the undesirable effect
of "undoing" and invalidating far too large
a proportion of the published risk assessments,
in far too many critical areas.
With something as critically important
as a category 1 or 2 extinction risk,
much more diversity of independent
and overlapping argument forms
is needed and called for, particularly
in regard to general context
considerations and assumptions.

- link Wikipedia: Mere exposure effect
- an item on Forrest Landry's compiled list of biases in evaluating extinction risks.
Housing and Transportation Roundup #2
by Zvi — 2022-12-27T13:10:00.979Z

Previously: #1

In my look back I concluded I want to be more focused and long-term oriented. Roundups by default are the opposite of that. This one was already in the pipeline (I have a rather large for-roundups draft to pull from), and on the margin easy to produce while creating a bank to pull from later. Still, I do note for myself the need to shift emphasis.

Building Houses Where People Want to Live

New York City Council Member here:

I mean no, not with that attitude.

Luckily, attitudes differ.

Stephen Hoskins thread on a study by Xiaodi Li, that found that building new luxury apartments in New York City did exactly what economics 101 says creating more housing should do, which is lower the cost of housing.

New restaurants is a strange measure. When more new restaurants open, sometimes it means that it became more worthwhile to open a restaurant relative to other uses of space. Other times, it means that an old business closed down, opening the way for a new restaurant. It is not obvious which effect dominates, so I would hesitate to draw conclusions from such data. In this case, it seems safe to presume that more housing means more residents means more restaurants.

Converting Offices People Don’t Want To Work At Into Houses Where People Want to Live

NYC developer Silverstein Properties seeks to raise $1.5 billion for converting older offices to residential buildings (Bloomberg). It is amazing news if this is commercially viable despite all the regulatory barriers. It would be obviously great if those barriers can be overcome at (in-context) reasonable costs.

DC offers tax incentives to convert old office buildings into residential housing, presumably to help overcome all the expensive regulatory burdens of doing so. A city realizing it actively wants something to happen might lead to subsidies. It is less likely to get pointless obstacles out of your way. The money does help.

This thread and the underlying report, an analysis of what New York needs to do to revitalize itself, also points out among many other things the importance of allowing reasonable conversion of office buildings into residential housing. For places like NYC there is no realistic substitute, and this could be bigger than all the other YIMBY initiatives combined in the medium term.

New York City Mayor Eric Adams goes full YIMBY, at least in theory, ...
Reflections on my 5-month alignment upskilling grant
by Jay Bailey — 2022-12-27T10:51:49.872Z

Five months ago, I received a grant from the Long Term Future Fund to upskill in AI alignment. As of a few days ago, I was invited to Berkeley for two months of full-time alignment research under Owain Evans’s stream in the SERIMATS program. This post is about how I got there.

The post is partially a retrospective for myself, and partially a sketch of the path I took so that others can decide if it’s right for them. This post was written relatively quickly - I’m happy to answer more questions via PM or in the comments.

Summary

I was a software engineer for 3-4 years with little to no ML experience before I was accepted for my grant.
I did a bunch of stuff around fundamental ML maths, understanding RL and transformers, and improving my alignment understanding.
Having tutors, getting feedback on my plan early on, and being able to pivot as I went were all very useful for not getting stuck doing stuff that was no longer useful.
I probably wouldn’t have gotten into SERIMATS without that ability to pivot midway through.
After SERIMATS, I want to finish off the last part of the grant while I find work, then start work as a Research Engineer at an alignment organisation.
If in doubt, put in an application!

My Background

My background is more professional and less academic than most. Until I was 23, I didn’t do much of anything - then I got a Bachelor of Computer Science from a university ranked around 1,000th, with little maths and no intent to study ML at all, let alone alignment. It was known for strong graduate employment though, so I went straight into industry from there. I had 3.5 years of software engineering experience (1.5 at Amazon, 2 as a senior engineer at other jobs) before applying for the LTFF grant. I had no ML experience at the time, besides being halfway through doing the fast.ai course in my spare time.

Not going to lie, seeing how many Top-20 university PhD students I was sharing my cohort with (At least three!) was a tad intimidating - but I made it in the end, so industry experience clearly has a role to play as well.

Grant

The details of the grant are one of the main reasons I wrote this - I’ve been asked for 1:1’s and details on this at least three times in the last six months, and if you get asked something from at least three different people, it might be worth writing it up and sharing it around.

Firstly, the process. Applying...
Institutions Cannot Restrain Dark-Triad AI Exploitation
by Remmelt — 2022-12-27T10:34:34.698Z
Link: https://mflb.com/ai_alignment_1/ai_governance_psr.html

On how governance institutional designs cannot restrain corporate, criminal and political groups of humans who gain increasing power from training and exploiting increasingly more capable – eventually power-seeking – machine learning architectures.

As elaborating on the human-to-human stage of the previous post.
Excerpts compiled below:

> How can governments and companies
> cooperate to reduce misuse risks from AI
> and equitably distribute its benefits?

that people and governments
simply do not have any way
to understand and mitigate
the significant advantage
that companies having and deploying AI
have, in regards to their own capabilities.
that the governance challenges perceived
by many/most regular American people
to be the most likely to be impactful
to individual people "soon"
(ie, within the next decade)
and which where also considered
by them to be "the highest priority,
most significant/important issues", etc,
in regards to AI, machine learning,
and enhanced use of technology general,
were:

> 1. "Preventing AI-assisted surveillance
from violating privacy and civil liberties".

> 2. "Preventing AI from being used
to spread fake and harmful content online".

> 3. "Preventing AI cyber attacks
against governments, companies,
organizations, and individuals".

> 4. "Protecting data privacy".

that the issues listed by the common people,
as being of 'primary concern',
are actually less likely to be personally noticeable
than the use of AI tools by criminals
by businesses engaged in predatory practices,
by cultural/cult leaders (operating in their own interests),
politicians with predominately privatize interests, etc,
to do the kind of sense making (perception)
and action taking (expression)
as necessary to create better honeypots,
more, deeper, and more complex entanglements,
and to implement more effective and efficient
extraction and extortion of resources
at higher frequency, intensity, consequentiality,
at larger scale, more quickly,
invisibly/transparently/covertly, etc,
in more and more ways
that are more and more difficult
to avoid, prevent, mitigate, and heal/restore from

...
Introduction: Bias in Evaluating AGI X-Risks
by Remmelt — 2022-12-27T10:27:30.646Z

The rationality community has a tradition of checking for biases, particularly when it comes to evaluating the non-intuitive risks of general AI.

We thought you might like this list, adapted from a 2015 essay by Forrest Landry[1]. Many names of biases listed may already be familiar for you. If you "boggle" more at the text, you will find curious new connections to evaluating upcoming risks of AI developments.

About Forrest Landry

Forrest is a polymath working on civilisation design and mitigating risks of auto-scaling/catalysing technology (eg. Dark Fire scenario). About 15 years ago, he started researching how to build in deep existential alignment into the internals of AGI, applying his deep understanding of programming, embodied ethics, and meta-physics. Then, Forrest discovered the substrate-needs convergence argument (as distinct from yet much enabled by instrumental convergence). Unfortunately, because of substrate-needs convergence, any approach to aligning any AI at the embedded level turned out unsound in practice (and moreover, inconsistent with physical theory). To inquire further, see this project.

Introduction

Note on unusual formatting: Sentences are split into lines so you can parse parts precisely.

Ideally,
in any individual or group decision making,
there would be some means, processes,
and procedures in place to ensure that
the kinds of distortions and inaccuracies
introduced by individual and collective
psychological and social bias
do not lead to incorrect results,
and thus poor (risk-prone) choices,
with potentially catastrophic outcomes.

While many types of bias
are known to science
and have been observed
to be common to all people
and all social groups, the world over
in all working contexts, regardless
of their background, training, etc,
they are also largely unconscious,
being 'built-in' by long-term
evolutionary processes.

These unconscious cognitive biases,
while they are adaptive for the purposes
of our being able to survive in
non-technological environments,
are also not able to serve us equally well
when attempting to survive our
current technological contexts.
The changes in our
commonly experienced world
continue t...
MDPs and the Bellman Equation, Intuitively Explained
by Jack O'Brien — 2022-12-27T05:50:23.633Z

Imagine you're a selfish taxi driver who takes passengers around between 3 towns. Each trip has a different probability of catastrophe and different earnings potential. You want to find a repeatable path of trips to maximise profit. No, you can’t just greedily pick the best trip from each city - sometimes, you must forego immediate profit to take a more profitable trip later.

This problem can be framed as a Markov Decision Process. This framing is the building block of reinforcement learning, a field which will likely play a key role in shaping AGI systems. And more importantly, an MDP is one of the simplest models of an agent which we can build upon to learn more about agency in general.

Example: Taxi driver

Let’s describe the taxi driver’s situation using a graph called a Markov chain. For now we’ll ignore the risk of bad things happening on any trips, and just compare the payoffs for taking different routes:

Each town in the graph is known as a “state”. From each state, you can take a couple of actions that involve traveling to a different city. Each action has a different payoff to reflect the current market equilibrium price of different trips. For now, we’ll just pretend that the prices don’t shift over the course of the day, so a taxi driver could always get these payoffs from driving. Now to add some uncertainty into the mix:

That graph looks a bit more complicated. Everywhere we previously had a single arrow, there is now a split arrow. That’s because every trip has two possible outcomes. We could either take a normal trip and make money, or we could have a car crash and end up in the hospital, losing money in the process. Each possible outcome of a decision has a payoff and a transition probability - I just omitted these from the graph for clarity’s sake. For concreteness, let’s zoom into the trip from Berkeley to San Francisco.

Deciding to travel from Berkeley to San Francisco results in:

A 95% chance of an uneventful taxi ride, earning 70 reward
A 5% chance of a car accident and a trip to the hospital, earning -1000 reward

We’ve just created a Markov chain model of the real-world taxi driver situation. These graphs are a bit different from the Markov chains I worked with in a university class. Our Markov chains didn’t have any notion of choices between different actions. An MDP can be viewed as a Markov chain describing the world, with the added fun...
How 'Human-Human' dynamics give way to 'Human-AI' and then 'AI-AI' dynamics
by Remmelt — 2022-12-27T03:16:17.377Z
Link: https://mflb.com/ai_alignment_1/single_post_psr.html

A short overarching narrative of how humans in companies will, by default, bootstrap the existence of AGI that proceed to lethally modify the environment beyond human control.
Excerpts compiled below:

where considering the shaping
of proto-AGI/AI systems
in interaction with humans:

as mostly about how 'human to human' interactions
are bootstrapping of 'human to AI' interactions,
and also about how narrow AI becomes general AI.
where for example, consider an easy narrative
of how this might manifest in the overall trends
of how the present inexorably leads to the future.

where considering the shaping
of now near-threshold-AGI systems
in interaction with world:

as mostly about how prior 'human to AI' interactions
have effectively/functionally implemented a bootstrapping
of all kinds of possible 'AI to world' interactions,
and how these 'AI to world' interactions, in turn,
set the context and future for all manner
of 'AI to AI' interactions, etc.
where there are different outcomes/effects:

that there is an overall movement:

a; towards the environmental conditions
needed for artificial machine:

substrate continuance; and;
continued increase
(of total volume of substrate); and;
increase in the rate of increase
(of volume of substrate).

b; away from the environmental conditions
needed for human living.

as described in Three Worlds and No People as Pets.

where considering the non-shaping
of now post-threshold-AGI
in interaction with itself:

where as considered both internally or externally;
where given the complete failure of exogenous controls
(via market incentives; due to economic decoupling)
that they (the AGI/APS) will have started,
and will increasingly (be able to, and will),
more and more shape the world environment
to suit their own needs/process.
that humanity discovers, unfortunately, far too late,
that any type of attempted endogenous control
is also strictly, functionally, structurally,
completely impossible/intractable.

as due to fundamental limits
of/in engineering control (note 4):

cannot simulate.
cannot detect.
cannot correct.

that any attempt to moderate or control AGI/APS
whether by internal or external techniques,
cannot not eventually fail.

where once the AGI/APS systems exist;
that the tenden

...
Nine Points of Collective Insanity
by Remmelt — 2022-12-27T03:14:11.426Z
Link: https://mflb.com/ai_alignment_1/ai_narrative_psr.html

A scenario detailing how humans automate away other humans, as the beginning of the economic decoupling of the AI-hardware economy from the human-wetware economy.
Excerpts compiled below:

Where/first it begins with the customer service people – they are automated away (since that is expensive, and no one wants to do it – everyone hates working with abusive customers anyway).
that then the engineers building the capability to build capability automate themselves away, (but *not* before also developing automation of maximally efficient optimized marketing and sales processes, since those sales/marketing people tend to also be *very* expensive (and opinionated temperamental narcissist artist types too)).
then, when the VC – seeing that the executive team is no longer needed – (since everyone else has already been factored out of the overall company structure) that they (the VC folk) notice that they also have the capability to make fully autonomous corporations (maybe using some of the newer 'Decentralized Autonomous Orgs', with a heavy crypto emphasis).
then that the overall situation is effectively "passive income on steroids for the VC, so much so, to the point that even the VCs themselves, are no longer even having to evaluate startup business plan proposals at all – they already have the perfect evaluative tools, and the ability, with their 'owned' AI, to identify the 'best possible versions' (the most profitable, and extractive versions) of any possible business plan(s),

That the net effect ends up being that fewer and fewer actual people (users) have any degree of significant potential for any sort of combined economic impact (aside from (maybe) a very few world scale global trillionaires), and the total share of the overall world total economy that has anything to do with the real needs of people – since the total value of their 'investment ability' is monotonically decreasing, and at an ever increasing rate.

→ Read link to Forrest Landry's blog for more.

Note: Text is laid out in his precise research note-taking format.