trentmkelly/LessWrong-43k huggingface dataset:
I there a demo of "You can't fetch the coffee if you're dead"?

by Ram Rachum — 2022-11-10T18:41:50.546Z

Hi everyone! My name is Ram Rachum, and this is my first post here :)

I'm an ex-Google software engineer turned MARL researcher. I want to do MARL research that promotes AI safety. You can read more about my research here and sign up for monthly updates.

I had an idea for a project I could do, and I want you to tell me whether it's been done before.

I want to create a demo of Stuart Russell's "You can't fetch the coffee if you're dead" scenario. I'm imagining a MARL environment where agent 1 can "turn on" agent 2 to prepare coffee for agent 1, and then agent 2 at some point understands how to prevent agent 1 from turning it off again. I'd like to get this behavior to emerge using an RL algorithm like PPO. Crucially, the reward function for agent 2 will be completely innocent.

That way we'll have a video of the "You can't fetch the coffee if you're dead" scenario happening, and we could tweak with that setup to see what kind of changes make it less likely or more likely. We could also show that video to laypeople, and it will likely be much easier for them to connect to such a demo rather than to a verbal description of a thought experiment.

Are there any existing demonstrations of this scenario? Any other insights that you have about this idea would be appreciated.
Mastodon Linking Norms
by jefftk — 2022-11-10T15:10:01.667Z

Linking is one of the core activities of the web, and the origin of
its

name.
I link to things when I want to talk about them, and this is very
standard. If you want to link to any of my published posts you don't
need to ask me first: I've already indicated that's fine by putting
something out here for everyone to read.

I recently read a blog post by a
Mastodon server admin that has me confused how this norm interacts
with Mastodon:

On Saturday evening I published a post explaining a couple of things
about Mastodon's history of dealing with toxic nodes on the
network. ... I realised that some people had cross-posted my Mastodon
post into Twitter. Someone else had posted a screenshot of it on
Twitter. Nobody thought to ask if I wanted that.

I'm pretty confused. The post (citation redacted) is linked from
their public profile, has its own public URL, isn't limited to
followers or otherwise behind a permissions wall, and shows up in
search results: it feels like "stuff someone put on the public web" to
me, and a place where I'd expect normal web norms to apply? While I
don't like the culture of sharing things by screenshot, in part
because it's easy to accidentally bypass privacy settings that way,
part of what the poster is objecting to is people sharing a link on
Twitter. And that seems like something I would have expected to be
completely fine to do without asking first?

It's also not clear to me whether the norm the author advocates
applies to boosting a post on Mastodon. On one hand I'd think it
wouldn't, because I haven't seen anyone saying "ask before boosting!"
and the "boost" button doesn't have a consent flow. But on the other
boosting is just makes a new post on your timeline that links back to
the original post: when someone does this manually with a Twitter
client or blog editor is that socially different from doing it
automatically with the Mastodon UI? Especially since the author's
original post was configured not to automatically federate (citation
redacted) and
the ~1.3k people boosting it removed that restriction allowing it to
propagate to nearly everyone's federated timeline.

I realize this is a tricky situation for long-time Mastodon users,
with lots of people coming in who don't know the local culture. As one of them I'd like to follow the
existing norms for the space. It's not clear to me, however, ...
Covid 11/10/22: Into the Background
by Zvi — 2022-11-10T13:40:00.991Z

There was a lot of news this week.

Elon Musk continued to Do Things at Twitter.

America had midterm elections. The polls were roughly accurate, with ‘candidate quality’ mattering more than expected especially on the Republican side.

FTX imploded. For now I am letting Matt Levine handle coverage of what happened.

None of it had anything directly to do with Covid. So this will be short.

Executive Summary

I hear there were midterms.

And that FTX imploded.

Covid death number down a lot, presumably not a real effect.

Let’s run the numbers.

The Numbers

Predictions

Predictions from Last Week: 255k cases (+7%) and 2,600 deaths (-1%)

Results: 242k cases (+1%) and 1,993 deaths (-24%!).

Predictions for Next Week: 250k cases (+4%) and 2,400 deaths (+20%).

That death number is super weird. At first I thought ‘what, they can count votes or they can count deaths but not both at the same time?’ or ‘election day isn’t actually a holiday is it?’ Then the case number came in flat even in the South, although Alabama didn’t report any cases at all (which wasn’t a big enough effect for me to adjust).

Some of the drop is that last week had a spike of about 250 deaths in North Carolina. Still leaves the majority of the gap unexplained. I don’t know. I don’t see how there could have been a large real drop in deaths, and if it was a reporting issue we would have seen a decline in cases. Also, in the regions where we see a decline in deaths, West and South, we don’t see relatively few cases. So the reporting explanations don’t make that much sense here, and it seems unlikely cases actually rose a lot while being underreported or anything like that.

It does raise uncertainty in deaths a lot for next week, and to some extent also for cases, in the usual ‘was this real or was it both fake and in need of reversal’ dilemma. We shall see.

The good news is that there is not much practical impact on decision making, unless this is all hiding a tidal wave of new infections. That is possible. I would still not expect anything like last year’s wave, and for things not go on too long before stabilizing, but the timing between weather and sub-variants would make some sense.

Deaths

Cases

Booster Boosting and Variants

Nothing.

Physical World Modeling

Babies born during lockdown more often miss developmental milestones (study).

I doubt this...
LessWrong Poll on AGI
by Niclas Kupper — 2022-11-10T13:13:57.387Z

This is just a short post to promote this Pol.is poll to loosely aggregate opinions about AGI / AGI x-risk of the LessWrong community. This poll is less structured than a questionnaire and anyone can add statements that others can agree/disagree with. Pol.is then automatically generates a nice visualization of similar stances.

I recently used Pol.is to gather complaints of PhD students at my department and it exceeded my expectations, so I thought why not try it here. If this is a success I might make a follow-up post with any interesting findings. These things can be slow to get started so be sure to come back to vote on new statements that others have made.

Here is the link: https://pol.is/7yvdmar3nj
The optimal angle for a solar boiler is different than for a solar panel
by Yair Halberstadt — 2022-11-10T10:32:47.187Z

I have both photovoltaic panels, and a solar boiler on my roof.

The solar boiler is in front, and the photovoltaic panels behind. You'll notice that they're at very different angles - the panel for the solar boiler is at approximately 55 degrees, whereas the photovoltaic panels are at about 10 - 20 degrees.

Why?

A solar boiler works by running water through a thin black panel, which gets very hot in the sun, and heats up the water. The hot water is then stored in a tank which can keep it hot for up to about a day.

A photovoltaic panel generates electricity from sunlight. In Israel, this electricity is fed directly into the grid, and we get paid 0.48 ILS per KWH. For perspective we pay about 0.51 ILS per KWH, so we get a pretty good price - there's no point storing the electricity in a battery to make sure we use it ourselves.

Both require direct sunlight to work, and both should be directly facing the sun to maximize efficiency.
However the use cases for both are completely different:

For the photovoltaic panels I want to maximize total energy produced. Since the sun moves around over the course of the day, the panels face south to maximize time in direct sunlight. And since the angle of the sun when it's in the south changes throughout the year, you'll want to angle the panels at about the average angle of the sun when it's in the south. This is equivalent to your latitude - in Israel 31 degrees.

There are a few complexities:

You'll want to capture some light the rest of the day, which pushes for the angle to be steeper,
During the winter the sky is cloudy and the sun is weak so you won't be able to capture much energy anyway. This pushes for the panel to be flatter, to optimize for the summer.

In practice it probably roughly evens out. In Israel the difference in efficiency between a completely flat panel and one at the optimal angle is only 10%.

For a solar boiler however my aim is completely different. I can't heat more than a single tanks worth of hot water, which is enough for my family of 4 and a couple of guests to have hot water, as well as plenty left over for washing up and miscellaneous use. During the summer we easily get this.

Thus there's no point optimizing the solar boiler for summer. Instead it has to be at a steep angle so that we still get a decent amount of hot water during the winter. In practice this works quite well, and we usually only need to tu...
What it's like to dissect a cadaver
by Alok Singh — 2022-11-10T06:40:05.776Z

Why

I never thought I was a bio person. But then I overheard Viv talking about MAOIs at a party. I asked her:

> - What are MAOIs?
> - monoamine oxidase inhibitor
> - What does that mean?
> - It prevents reuptake of neurotransmitters.
> - But what *is* a neurotransmitter? What does reuptake actually mean?
> - ...
> - So life uses chiral properties of space to implement things...

Viv had the most important trait of a teacher: patience. I asked the most naive
questions and they answered them. They walked with me, all the way down to the
very beginning, rebuilding my understanding. It was amazing. I wanted to know
more. Roadblock: finding lifeforms to study.

I wondered if non-medical students could watch dissections. You can’t get more information about an object
than by directly interacting with it. The concrete world contains the abstract
one. I even asked my doctor at a physical if she knew of any, and she said to look at community colleges.

After some searching, I found this: Bio
848NV. Forget
viewing the dissection, you’re doing the dissection. 5 hour dissection for $60, free if you just watch. The only bureaucratic hangup is that you must pay by check.

This is why I love the Bay Area: there’s stuff like this and you can just do it. yes it’s weird no they can’t stop you. The boundary between scientist and serial killer is paper thin sometimes.

Takeaways

I’ve done this a few times now. Turns out that there’s way way way too much information to understand it all in one 5 hour session. Each time, we pick out areas and focus on them.

Seeing how everything fits together ‒and how big it is‒ makes understanding at different scales much easier.

There’s a common template to life. Seeing it in you hits different.

Brain has interesting connections to fractals and graph theory.

Maybe pan-psychism isn’t totally wrong.

What & how & why

I tell my friend Leah and she says “This is the most appealing activity that
I’ve ever seen you do”. Dunno whom that says more about.

We arrive and there are 5 people around 3 cadavers. We get aprons and lab coats
and start syringing what’s mostly Downy fabric softener with a syringe. It
prevents decay and smells sickly sweet.

Corpses can last a long time. One of the corpses had been dead for 5 years.

Many random observations

There’s a crazy amount of connective tissue, and it makes a creepy wi...
I Converted Book I of The Sequences Into A Zoomer-Readable Format
by dkirmani — 2022-11-10T02:59:04.236Z

If I (a 19 year old male) texted "www.readthesequences.com" to my roommate, the probable outcome is that he would skim the site for under a minute, text back something like "seems interesting, I'll def check it out sometime", and then proceed to never read another word. I have another friend, one that I would consider a smart guy. He would consistently rank above me in our high school's math team, and he scored in the 1500's (≥3SD) on his SATs. The same dude did not read a single book during the entirety of his high school career.[1]

Attention is one's scarcest resource, and actually reading something longer than a paragraph is a trivial inconvenience, especially for my generation.

What, then, does manage to hold the fickle eyeballs of zoomers like me? Well, TikTok, mostly. However, there is one (very popular) genre of TikTok video worth investigating. In this genre of video, a Reddit post is broken into sub-paragraph chunks of text, and these chunks are sequentially rendered onscreen while a text-to-speech program reads them to the user. The text is overlaid upon a background video, which is either gameplay from the mobile game Subway Surfers, or parkour footage from Minecraft. The background gameplay provides engaging novelty to the user's visual cortex, while the synthetic voice ensures that the user doesn't have to go through the hard work of translating symbols into sounds. Really, it's all quite hypnotizing.

The fact that these videos are often recommended by TikTok's algorithm implies that they are among the most-engaging videos that our civilization produces. Therefore, to reduce the effort-cost of reading the sequences, I gave the TikTok treatment to Book I ("Map and Territory") of Rationality: From AI to Zombies.

(Update: Circa 2023-02-09, all these links are dead. This was in response to an AWS alert notifying me that 85 gigabytes (or more) of data had been transferred out. I really shouldn't have used a public S3 bucket to serve video in the first place, as it exposed me to an unacceptable amount of risk in the form of a denial of wallet attack. I've got a second batch of videos in the works, which I intend to distribute via a more secure mechanism.)

(Update 2: THE VIDEOS ARE HERE, CLICK HERE)

Predictably Wrong

What Do I Mean By “Rationality”?

Feeling Rational

Why Truth? And…

… What’s a Bias, Again?

Availability

Burdensome Details

Planning Fallacy

I

...
Adversarial Priors: Not Paying People to Lie to You
by eva_ — 2022-11-10T02:29:12.708Z

Reply to Desiderata for an Adversarial Prior

Assertion: An Ideal Agent never pays people to lie to them.

This seems sensible, only a very foolish person would knowingly incentivise dishonesty in others, but what does it actually mean in practice?

You can't use unverifiable information obtained from a single person or from a faction of possibly-conspiring people in any way that benefits that person or faction in the hypothetical where the information is false. Otherwise, they're incentivised to give you the unverifiable and false information to motivate you to do that, and so you'd be paying them to lie to you.
You can't use any information, even verifiable information, obtained from a single person or from a faction of actually-conspiring people in any way that harms that person or faction in the hypothetical where it is true. Otherwise they'd just not tell you, and you'd be paying them to dishonestly shut up.

If everyone followed (2), you could freely go around saying the truth no matter what and expect no personal negative consequences. This would maximise public knowledge (your information can still be used to the benefit or detriment of other people), and people would be better off:

Criminals would confess their crimes in detail, in exchange for payment in the form of a reduced sentence if they do get caught so as not to actually incentivise crime. This information can then be used to help protect against, catch, or convict other criminals, it just can't be used against the specific person who confessed it in the first place.
To the extent that criminals want to form groups that can work together, they can formally notify the police of their criminal conspiracies and then any information from a member can't be used against the rest (unless a member formally betrays the conspiracy, in which case their testimony is fair game again).

If everyone followed (1), lying would be pointless because even though everyone believes you, they'll never believe you in a way that corresponds to doing something that benefits you. This condition imposes a lot of much stranger outcomes:

Claims made by salespeople must be ignored unless you've got a way to actually test the claim, and if you can only test it in the future (after you've already bought the product), you're honor-bound to return it and demand a refund if anything claimed about the product (that your decision to buy

...
Is full self-driving an AGI-complete problem?
by kraemahz — 2022-11-10T02:04:48.983Z

I've felt for quite a while that full self-driving (automated driving without human supervision through arbitrary road systems) is a problem that is deceptively hard. Yes, it is possible to map a route and navigate on a road mesh, do lane following, and even obstacle avoidance using current systems. With LIDAR and well-trained avoidance systems things like Waymo can operate in constrained urban environments.

But as soon as the training wheels are off and the environment becomes unconstrained, the entire problem stops being about just whether we can design an agent which has driving capabilities and becomes "can we make a vehicle which can predict agent-agent dynamics?" If we think about the full range of human road behaviors we must consider adversarial attacks on the system such as:

Blocking it from entering a lane
Boxing it in and forcing it off the road into obstacles
Throwing paint/eggs/rocks at its vision systems
Using deceptive tactics (e.g. pretend to be a road worker) to vandalize it and/or steal from its cargo
Intentionally standing in its path to delay it
Making blind turns in front of it
Running into traffic

In addition to agent-agent problems, we must also consider road hazards:

Poorly maintained roads with damaging potholes
Sinkholes which have disabled the road
Eroded road sides with dangerous falls
Road debris from land slides
Road debris from other vehicles

In these situations, a perfectly rule-following automaton behaves well below human level in preventing delay or damage to itself. Do these scenarios require AGI for a level-5 autonomous vehicle to reach human level? Are the benefits from above-average performance in normal traffic enough to offset the risk of subhuman performance in extrema?
What are examples of problems that were caused by intelligence, that couldn’t be solved with intelligence?
by Peter O'Malley — 2022-11-10T02:04:01.347Z

Like everyone, AI safety has been on my mind a lot lately. I was thinking about how most problems that are caused by intelligence in the world to-date seem to have always be solved by or have the potential to be soluble using more intelligence.

While some problems can be massively lethal of course, this doesn’t seem to be the case with the problems that most applied AI safety today seeks to avoid. Rather, the prevalent approach seems to be safety by avoiding even small risks - like not letting a child play for fear it might scratch its knee. In taking this approach, it feels like we’ll almost certainly limit our ability to access and leverage this intelligence to solve many problems.

I was curious are there many good examples from the real-world where this hasn’t been the case in other areas - what are meaningful problems that were caused by intelligence, that couldn’t be solved with more intelligence? Even if that wasn’t necessarily done - for economic reasons, etc.

For example, intelligence caused carbon emissions - we leveraged it to create such huge levels of industry that we emitted far too much - but intelligence will also almost-certainly be leveraged to solve it - via a variety of human-invented solutions and counter-actions.

Are there examples where this wasn’t the case historically?
Desiderata for an Adversarial Prior
by Shmi — 2022-11-09T23:45:16.331Z

Based on the discussion in the comments on https://www.lesswrong.com/posts/6XsZi9aFWdds8bWsy/is-there-any-discussion-on-avoiding-being-dutch-booked-or.

Epistemic status: I am not an expert in the subject matter, so not confident, seems useful to have, I could not find anything discussion online.

As I understand it, Pascal Mugging can either be an honest opportunity for a large gain in utility (e.g. a Powerball ticket?) if the mugger is honest, or an adversarial attempt to exploit the agent (e.g. a Nigerian Prince Advance Fee Scam) if not. Most people have some skills in telling the two apart, usually erring on the side of caution. However, it is not clear how to formalize this approach. Most of the discussion focuses on ignoring super-tiny probabilities, and accepting a utility loss penalty, where a proverbial $10 on a busy sidewalk gets ignored because if it were real someone would have picked it up already, a situation humans navigate quite successfully most of the time.

I suspect that Pascal Mugging detection might be better approached from a different direction: instead of starting from "I know a liar when I see one" and the rule of thumb "Most too-good-to-be-true proposals are just that" and trying to formalize it for the case of "tiny probabilities of vast utilities", one could try to start with a more general logic of "adversarial detection", and then apply it to the special case of Pascal Mugging.

How would one go about telling apart an indifferent interaction, where something like the Solomonoff's universal prior is appropriate, from an adversarial one, where it runs into trouble? Whatever the approach, it is likely to have the following properties:

In a non-adversarial case it reduces to the universal prior.
An attack by a "less-intelligent" adversary is severely penalized, the larger the difference in "intelligence" the harsher.
Conversely, an attack by a "vastly more intelligent" adversary is indistinguishable from the non-adversarial case.

In the Pascal Mugger's case, the really clever one will get your money without you realizing what happened (see point 3 above), the dumb one will go away empty handed, and there is a whole range in between. There are probably some other desiderata that I am missing here.
Chord Notation
by jefftk — 2022-11-09T21:30:01.759Z

When I

want to

write chords I
use a quick

chord
chart notation. There are lots of different ways of doing this,
so I wanted to explain how the particular system I've ended up using
works. Here's an example:

|: Am AmG Am CG Am AmG FG Am :|
|: Am / AmC DE Am / G Am :|

Compare to chords written above standard musical notation:

Each letter normally represents two beats. When there's a different
chord for each beat you write them squished together: "AmG" is two
beats, while "Am G" is four. A slash ("/") means to stay on the
previous chord for another two beats. There's an extra space every
eight beats and a new line every sixteen, so you can keep track of
where you are in the form. I'll also add extra spaces to line things
up vertically to again make it clearer how it fits the form. It
works best in a monospaced font.

Here's another example:

|: A AD A E A AD AE A :|
|: Bm / A / Bm / AE A :|

Sometimes I'll use numbers
instead of chord names, to make it easier to think about what's
happening musically. The previous tune would be:

|: 1 14 1 5 1 14 15 1 :|
|: 2m / 1 / 2m / 15 1 :|

Note that "14" is "1" and "4" on successive beats, not the fourteenth
scale degree.

Comment via: facebook, mastodon