trentmkelly/LessWrong-43k huggingface dataset:
Musings on the Human Objective Function
by Michael Soareverix — 2022-07-15T07:13:19.711Z

I think it's fair to say that most of us would trust a superintelligent human over a superintelligent AI. While humans have done a lot of evil in the world, our objective function as individuals seems to line up pretty well with the objective function of other individuals and our species as a whole.

My attempt in this post is to create a rough approximation of the human objective function and get feedback from the more knowledgeable LessWrong community.

The Human Objective Function (bulletpoints):

-Desire for a

variety

of basic personal pleasure. It is not enough to simply have one type of pleasure; our desires constantly fluctuate. We are hungry, horny, sleepy, and lonely at different times.

In my opinion, this prevents maximization towards a single goal, as we fear AGI would. If an AGI sometimes wants to make paperclips, sometimes wants to make people happy, and sometimes wants to be turned off, it would probably be easier to deal with than an AI solely focused on maximizing paperclips. We also can't predict our desires well in advance, so we typically take the most strides towards them when we feel like it.

-Desire for stability. We don't want the world to change much. Change often makes us afraid.

In the case of AI, this is really nice too. There's probably some ideal world for us, a bit like a videogame or something, but we don't want to be immediately thrown into it now. It would be a lot nicer to ease into it and see if it is really something we're interested in. This slow change gives us a good sense of what we actually want, because it lets our fluctuating desires average out over time. This is important because...

-Desire for change in a positive direction. Despite the painfulness of exercising, the feeling of growing in a positive direction is incredibly motivating. It gives us purpose. There's more to it than just the goal. The journey is often more important than the destination. We like stories where the protagonist grows, and we're annoyed when we're spoiled about an ending. The progress and struggle is meaningful as long as it is in a positive direction.

We can apply this to AI by making it content to be turned off or altered when it makes what we consider to be a mistake. One of the biggest problems is that we probably can't alter a superintelligent machine once it is already running, because it will have some motivation to avoid being altered...
Peter Singer's first published piece on AI 
by Fai — 2022-07-15T06:18:30.921Z
Link: https://link.springer.com/article/10.1007/s43681-022-00187-z/

Hi! I am Tse Yip Fai. This is a linkpost to Peter Singer's first published piece on AI, it's also my first published paper (second author). Since the paper is open access, I won't be doing any copy-and-paste here.

This is a seminal paper resulting from an one-year research. We hope that this paper can break the ground and raise awareness in the field of AI ethics so that they start caring about the impact on animals. This, I believe, is not just relevant to the AI ethics field, but also to the AI alignment field.

If you prefer videos, here's a talk by Professor Singer and myself on the same topic.
Don't use 'infohazard' for collectively destructive info
by Eliezer Yudkowsky — 2022-07-15T05:13:18.642Z
Link: https://www.facebook.com/yudkowsky/posts/pfbid06GssuFTssNLeguuH9Gk4zaqFqngHkXv4RxU7rr5KhbKfy6eogB6Zfn8QDQqoWTQul

Suppose there were a cheap way to make nuclear weapons out of common household materials. Knowing this information is not directly harmful to the median individual like yourself. On average, it may even be beneficial to the average person, to know that secret yourself; either because you could directly conquer places, or because you could sell the info to somebody who could. But society as a whole would rather that nobody knew it than that everybody knew it.

For a more realistic example, consider the DNA sequence for smallpox: I'd definitely rather that nobody knew it, than that people could with some effort find out, even though I don't expect being exposed to that quoted DNA information to harm my own thought processes.

'Infohazard', I think, should be used only to refer to the slightly stranger case of information such that the individual themselves would rather not hear it and would say, "Don't tell me that".

A spoiler for a book you're in the middle of reading is the classic example of information which is anomalously harmful to you personally. You might pay a penny not to hear it, if the ordinary course of events would expose you to it. People enclose info about the book's ending inside spoilers, if on Reddit or Discord, not because they mean to keep that secret to benefit themselves and harm you - as is the more usual case of secrecy, in human interactions - but because they're trying to protect you from info that would harm you to know.

The term 'infohazard' has taken on some strange and mystical connotations by being used to refer to 'individually hazardous info'. I am worried about using the same word to also refer to information being alleged to be dangerous in the much more mundane sense of 'collectively destructive info' - info that helps the average individual hearer, but has net negative externalities if lots of people know it.

This was originally posted by me to Facebook at greater length (linkpost goes to original essay). Best suggestions there (imo) were 'malinfo' and 'sociohazard', and if I had to pick one of those I'd pick 'sociohazard'.

EDIT: I like shminux's 'outfohazard' even better.

EDIT 2: My mind seems to automatically edit this to 'exfohazard' so I guess that's what the word should be.
Upcoming heatwave: advice
by stavros — 2022-07-15T05:03:49.476Z

There's a heatwave coming (or already arrived) in the UK and western Europe. Many of these places are not equipped for dealing with high temperatures and have large at risk populations* - not simply those with preexisting health conditions, but those living in accomodation grossly unsuited for high temperatures and anyone inexperienced with high temperatures who isn't properly aware of the dangers and precautions they need to take.

*(https://www.nhs.uk/live-well/seasonal-health/heatwave-how-to-cope-in-hot-weather/ for more info on who is at risk, and general advice)

Heatwaves are not just 'nice weather', not in the anthropocene, they are life threatening. People will die. I hope by writing this to nudge EAs and their local communities toward safety.

TL;DR: Sleep and hydration* are the two pillars of survival in hot weather. If you are not already, put maximum effort into getting a good night's sleep (advice below).

*(don't neglect electrolytes; consider this an excuse to move Gatorade to the 'healthy (in moderation)' column for a few days)

Sleep

Sleep is one of the most important factors in health. Most people do not get enough sleep and do not practice good sleep hygiene - are already somewhat sleep deprived or at high risk of sleep deprivation.

Too much heat and humidity are massively deletrious to sleep quality. Further, sleep deprivation puts you at greater risk of heat related illness.

What can you do?
Sleep hygiene basics:

Establish and maintain a routine - stop eating several hours before bedtime, turn off the lights at least 2 hours before bedtime (and use programs like f.lux or redshift on your screens to reduce blue light), go to bed at the same time each night*, get up at the same time each morning, eat breakfast at the same time each morning.

*Many people find using alarms/reminders to establish a set bedtime more effective than using them to get up in the morning.

Create a better sleep enviroment - as dark as possible (Ikea sell 'adhesive' blackout blinds, I personally use an eye mask), and as quiet/peaceful as possible (ear plugs / white noise)

Keep your sleeping area clean, tidy and reserved for sleeping, and sex... and reading (but not on your laptop/phone).

Keep pen and paper near your bed; if your mind is overactive - can't stop thinking about something, or think of something you need to remember, write it down.

If your bedroom is extremely unsuitab...
A note about differential technological development
by So8res — 2022-07-15T04:46:53.166Z

Quick note: I occasionally run into arguments of the form "my research advances capabilities, but it advances alignment more than it advances capabilities, so it's good on net". I do not buy this argument, and think that in most such cases, this sort of research does more harm than good. (Cf.

differential technological development

.)

For a simplified version of my model as to why:

Suppose that aligning an AGI requires 1000 person-years of research.

900 of these person-years can be done in parallelizable 5-year chunks (e.g., by 180 people over 5 years — or, more realistically, by 1800 people over 10 years, with 10% of the people doing the job correctly half the time).
The remaining 100 of these person-years factor into four chunks that take 25 serial years apiece (so that you can't get any of those four parts done in less than 25 years).

In this toy model, a critical resource is serial time: if AGI is only 26 years off, then shortening overall timelines by 2 years is a death sentence, even if you're getting all 900 years of the "parallelizable" research done in exchange.

My real model of the research landscape is more complex than this toy picture, but I do in fact expect that serial time is a key resource when it comes to AGI alignment.

The most blatant case of alignment work that seems parallelizable to me is that of "AI psychologizing": we can imagine having enough success building comprehensible minds, and enough success with transparency tools, that with a sufficiently large army of people studying the alien mind, we can develop a pretty good understanding of what and how it's thinking. (I currently doubt we'll get there in practice, but if we did, I could imagine most of the human-years spent on alignment-work being sunk into understanding the first artificial mind we get.)

The most blatant case of alignment work that seems serial to me is work that requires having a theoretical understanding of minds/optimization/whatever, or work that requires having just the right concepts for thinking about minds. Relative to our current state of knowledge, it seems to me that a lot of serial work is plausibly needed in order for us to understand how to safely and reliably aim AGI systems at a goal/task of our choosing.

A bunch of modern alignment work seems to me to sit in some middle-ground. As a rule of thumb, alignment work that is closer to behav...
Inward and outward steelmanning
by Q Home — 2022-07-14T23:32:47.452Z

The usual steelmanning is when you try to find the best possible arguments for a position you don't like. Or the hardest to refute version of an idea you don't like. Let's call this "inward steelmanning". You're mostly fixing internal cogs of arguments and positions.

I want to propose a different idea, "outward steelmanning". In this approach to steelmanning you're imagining a world (similar enough to our own on the surface) where the idea you don't like is true, then you try to see if this world resonates with the real one in any interesting way. You treat arguments and positions as black boxes and focus solely on their implications.

Imagine that you encountered a car with square wheels

Inward steelmanning: "This is an abomination! It doesn't work! But maybe with round wheels it would be beautiful. Or maybe a different vehicle with square wheels could be beautiful."

Outward steelmanning: "This is ugly! It doesn't work! But maybe if I imagine a world where this car works, it will change my standards of beauty. Maybe I will gain some insight about this world that I'm missing."

If you want to be charitable, why not grant your opponent an entire universe with its own set of rules?

Jokes aside, I'll explain the goals of outward steelmanning in more detail along the way.

Thought experiments

Extremism In Thought Experiment Is No Vice

Scott Alexander describes a world where research about morality works like high energy physics: the more extreme (disgusting) situations you imagine, the better you understand the laws of morality. Let's imagine a world where the opposite idea is true.

I imagine a world where you use your moral intuitions to navigate the chaotic space of all possible moral situations. In such world you may need to seek situations where you have the chance to do a good thing, not where you're forced to do disgusting things no matter what. The former situations contain discernible pieces of general moral rules, not the latter ones. Moral intuitions are your torchlight and you shouldn't try to destroy the torch before you found the details of a flashlight.

We can also imagine a "hybrid world", where both approaches need to be combined. You need to look where the logic inferred from disgusting and unfair situations leads you and where the logic inferred from more optimistic situations leads you. Then you make some meta-inference about general moral rules. Or m...
Potato diet: A post mortem and an answer to SMTM's article
by Épiphanie Gédéon — 2022-07-14T23:18:26.691Z

Slime Mold Time Mold (SMTM) has just published the results of their studies on the potato diet. The idea is to eat nothing but potatoes for four weeks, and report how it went.

You can read SMTM's reasoning here, the basic idea is that they've observed some annectdotal data point of some people losing substantive weight while eating nothing but potatoes and want to reproduce the result.

Natália Coelho Mendonça has written in a very detailed and well-written article why it was probably not lithium, SMTM's leading hypothesis of why obesity is so prevalent.

As SMTM's potato diet is not directly linked with lithium, some readers might think that even if SMTM was wrong on that one, the potato diet could be interesting to try.

Having done the diet, I thought I could give a subjective data point into all this. I wanted to be pseudo-anonymous, but this post has been nagging at me ever since finishing the diet.

This post will not be as sourced as Natália's article, I do not intend to explain the mechanism or absence thereof of the potato diet. I am not knowledgeable in nutrition or biology to really have any insights into the mechanisms of this.

This article will serve both as a report of how it went down for me, a postmortem of mistakes I made, and a general impression of the SMTM methodology that results from this.

While I am glad I tried the potato diet, and it gave me great insights into what can affect my energy and mood levels, I feel that my following of it was dangerous in a way SMTM did not account for. I am also finding their methodology has been very poor, and their recent article is loose in the way it draws its conclusions.

There are some things I either completely blundered and that were my fault, or that I calculated the risks of and decided they were worth the gamble. For instance, I asked SMTM if my bipolar anorexia would muddle the data, to which they said that they advise against bipolars taking the diet. Seeing as I am quite atypical in my bipolarity (my manias are never a problem for me, I actually look for ways to trigger them), I decided it was worth a shot.

So I am not trying to hold SMTM responsible for what happened or what might have happened. I am trying to

warn other users here who might be tempted to follow the diet of precautions they might want to take into considerations

narrate my experience as a whole in case there is something interesting f

...
Proposed Orthogonality Theses #2-5
by rjbg — 2022-07-14T22:59:28.079Z

2

The intelligence, reflectivity, and 'Lawful'ness of a mind are mostly independent of its capacity to experience emotion. Minds can be very unintelligent, lack anything like a "utility function" or some critical OODA loop components, possess little or no self awareness, and generally seem completely foreign to considerations of an abstract utility maximizer while still experiencing a wide variety of deep and complex feelings.

There is obviously some necessary core of "minds" that must exist for any particular one to be a "mind" or feel feelings, and it may be possible to say with confidence that e.g. amoeba don't have qualitative experience. What is not currently obvious is that anybody yet has enough information and conceptual understanding to reason conclusively about what those properties are and which minds have them, in practice, without the ability to check their work. People who have important moral conditionals on whether or not {ems, animals, babies} experience suffering, should keep in mind the possibility that their galaxy brain reasoning, whichever way it leads, is wrong.

3

Your opinion on the validity of Orthogonality Thesis #2, or on the larger question of which minds have the tools necessary for subjective experience, is logically independent of your opinion on whether or not those minds have moral value. Capacity-to-suffer is not some universal barometer for moral relevance, even for humans. Just as people can be sadistic or altruistic towards others, people can also have instincts towards others that are completely sideways of these considerations. Humans have been known to save encrypted backups of fictional video game save files to preserve fictional character lives that they are fully aware don't really "exist".

Thus there is not a contradiction, necessarily, in considering a third trimester fetus a life with similar moral weight to a living human, while believing that it doesn't have the ability to feel things. Similarly, many meat-eaters will cheerfully explain to vegetarians their entirely consistent position, that they do in fact believe animals can suffer and do so extensively throughout factory farms, and agree with holocaust analogies, and yet they simply do not care about nonhuman experiences of suffering.

4

The range of emotions a particular mind experiences as a part of a typical natural life, or the range of emotions it can experienc...
Better Quiddler
by jefftk — 2022-07-14T17:40:05.929Z

Quiddler
is a card game, a cross between rummy and scrabble. I've been playing
it on vacation with in-laws, and it suffers from the common drawback
of games where your turn begins with drawing and ends with discarding:
you begin your turn with new information, and so have to do a lot of
thinking while everyone waits for you. This is something more recent
games

have
fixed and we can make the same fix to Quiddler, rotating the
steps.

Current turn:

Draw a card, from either the discard or draw piles

Optionally put down words

Discard a card

Better turn:

Optionally put down words

Discard a card

Draw a card, from either the discard or draw piles

You each have your own personal discard pile to your right:

The reason we need personal discard piles is that if you keep having a
common one then drawing from it is nearly useless: the card on top
will always be the one you just decided to dump. You could instead
have the option to pick up the card the player before you discarded,
which sort of fixes this, but then there's still important information
you receive just as your turn starts. Best is to have the option to
pick up the card discarded by the player who goes after you:
that's the one that's been sitting out for the longest. Each player
has their own discard pile to their right, and when you want to pick
up a face-up card you draw from the discard pile to your left.

The other change we need to make is that at the beginning of each hand everyone choses between drawing a face up card or one from the draw pile. This step can be simultaneous. You could instead deal an extra card to everyone to begin, but getting a little bit of choice about what your initial hand contains reduces the chances of horrible (no vowels or all vowels) hands.
Circumventing interpretability: How to defeat mind-readers
by Lee Sharkey — 2022-07-14T16:59:22.201Z

(Post now available as a pdf: https://arxiv.org/abs/2212.11415 )

TL;DR: Unaligned AI will have a convergent instrumental incentive to make its thoughts difficult for us to interpret. In this article, I discuss many ways that a capable AI might circumvent scalable interpretability methods and suggest a framework for thinking about these risks.

Summary of key points

The framework: Interpretation as information flow

I categorize potential interpretability circumvention methods on three levels:

Interpretability methods consist of interpretability tools and the human interpreter. To circumvent interpretation an AI can prevent information about its thoughts from flowing through either the tool or the interpreter. (See main text for figure)
Within the tool or the human interpreter, there are several information processing steps that an AI might exploit. (See main text for figure):

Exposure evasion: Being where your adversary isn’t looking
Perception evasion: Being hard to detect even when exposed to your adversary
Recognition evasion: Being hard to recognize/understand even though you’ve been perceived
Dazzling: Looking like something that confuses or incapacitates your adversary

Another useful way to categorize interpretability circumvention methods is whether they are passive or active. Most methods that can be passive can also be active, but other methods are strictly active.

Passive interpretability circumvention occurs during the normal course of learning without the AI intending to make itself less interpretable. (e.g. polysemanticity).
Active interpretability circumvention is where the AI intentionally takes actions in order to circumvent our interpretability methods. (e.g. an AI actively deleting its own representations to hide deception)

The list of potential interpretability circumvention methods with a short summary of each

1. Exploiting weaknesses of the interpretability tool

1.1. Evading exposure to the interpretability tool

1.1.1. Thinking in the gaps of our tools: Our interpretability tools might fail to examine everything about an AI's thoughts (e.g. it might not look at nonlinear interactions between neurons, only their linear relations). An AI might use those blindspots for dangerous thought. (Passive or active)
1.1.2. Externalizing representations: The environment can be used for some parts of an AI's computa

... ,