trentmkelly/LessWrong-43k huggingface dataset:
[Event] Weekly Alignment Research Coffee Time
by adamShimi — 2021-05-29T13:26:28.471Z

Just like every Monday now, researchers in AI Alignment are invited for a coffee time, to talk about their research and what they're into.

Here is the link.

And here is the everytimezone time (what matters is the hour, I don't change the day anymore).

Note that the link to the walled garden now only works for AF members. Anyone who wants to come but isn't an AF member needs to go by me. I'll broadly apply the following criteria for admission:

If working in a AI Alignment lab or funded for independent research, automatic admission
If recommended by AF member, automatic admission
Otherwise, to my discretion

I prefer to not allow people who might have been interesting but who I'm not sure will not derail the conversation, because this is supposed to be the place where AI Alignment researchers can talk about their current research without having to explain everything.

See you then!
How to Write Science Fiction and Fantasy - A Short Summary
by Jsevillamol — 2021-05-29T11:47:30.613Z

I've been recently spending my free time writing fiction. In order to improve, I picked up How to Write Science Fiction and Fantasy, by Orson Scott Card.

I found it an interesting read, and introduced me to some concepts that have helped me realize mistakes in my speculative fiction stories. On the other hand, there is a lot of material addressed at professional writers, on what to read first, where to publish and what conferences to attend.

I am writing this summary for people who want to engage with the writing advice - of the publishing and sales advice I didn't pay much attention nor bothered checking which parts were up to date.

Here is the advice I enjoyed the most. Most of it is common sensical. But there is value in separating common sensical good advice from common sensical bad advice.

Show, don't tell

This is commonly repeated advice, but I think I didn't quite internalize it until now - at least as it applies to introducing technology and magic to the reader.

Essentially it boils down to never, ever have expositional dialogue. As long as your rules of magic / science are self consistent, you do not need to explain them in detail. Instead, Orson Scott Card recommends to expose through implication.

Here is one example in the book:

The classic example is Robert Heinlein’s phrase “The door dilated.” No explanation of the technology; the character doesn’t think, “Good heavens! A dilating door!” Instead, the reader is told not only that doors in this place dilate, irising open in all directions at once, but also that the character takes this fact for granted. The implication is that many-perhaps alldoors in this place dilate, and that they have been doing it for long enough that nobody pays attention to it anymore.

Another piece of advice is to give the exposition in pieces. You do not need to fully flesh out the world at the beginning. Better to begin with the action driving the story, and let the exposition trickle in through implication.

Keep your promises

This is fairly basic advice. However I realized I was butchering it.

Basically, Scott Orson Card invites us to think of story telling as making a series of promises. If you mention a new term like a "seed village", the audience expects you to explain what that is before it becomes relevant for the plot.

The most important promise to keep is the promise you make in the first few pages - what the story will be ab...
Two Definitions of Generalization
by ozziegooen — 2021-05-29T04:20:28.115Z

Epistemic Status: Research notes, plus an appeal for the reader to give their best attempt.

The words “abstraction” and “generalization” are often used fairly interchangeably. This is a pity. We only have so many well known words for abstract concepts.

One distinction I like is the one suggested in this Stack Overflow post. The following images are from the top answer:

Object

Abstraction

Generalization

Simple, right?

Well, I think there are two important definitions of generalization that could apply both that merit discussion.

The first, and perhaps the better known, is essentially what's stated on Wikipedia:

generalization (1) (Wikipedia): A generalization is a form of abstraction whereby common properties of specific instances are formulated as general concepts or claims.

This definition is about identifying the similar properties of supersets and removing the other information. So, for the desert example above, generalizing the cake would mean identifying the common properties it shares with other deserts.

However, this definition doesn't work for many of our uses of the term.

When we discuss "general intelligence", we mean something like, "the superset of all narrow intelligences", not the "elements common to all narrow intelligences".
When I write a "highly general function", this could refer to a function that takes in many sorts of things but still manages to treat them all uniquely.
The lists of "generalizations" include statements like, "Most people find church boring."

From these, we could imagine a different definition. Something like,

generalization (2) (Here): A generalization is a pointer to a set of things.

I use the word set here because it's mostly correct, but there could clearly be modifications. It might really be more of a fuzzy set.

One way I could write a general function would be to remove all of the differentiating details between items. This would be generalization(1). But I could also write this function by just moving a complexity into it and adding a big switch statement. This would be generalization(2).

I'm tempted to say that generalization(2) corresponds somewhat to set theory, and generalization(1) more to type theory.

With all that out of the way, we can attempt some more specific generalization(1) definition:

generalization(1)(Wikipedia complexified): A generalization is an abstraction made up some of common properties of a set.

Gen...
The Sanity Waterline Bulges at the Middle
by matejsuchy — 2021-05-29T03:15:01.177Z
Link: https://matejsuchy.substack.com/p/aristotelian-middle

Ideas have consequences. A subset of ideas — we’ll call it ideology — has consequences on material life. Naturally, this means that ideology is subject to a selective pressure.

Which means we need to consider

slack

. Just as with intermediary adaptions in the stylized case of eyespots, there can are frictions associated with adhering to new ideology. In departing from the previous paradigm, one is supported by less wisdom and has a narrower reference class to work from. There will be bumps along the way as you and your fellow innovators figure things out. These frictions require slack.

In a world with too little slack, activation energy is too scarce to spend on innovation. If eyespots cost energy but only payoff after the system settles into a new equilibrium, they won’t reach fixation in a population of energy-constrained worms. If one day a Russian serf had come up with Progressive Belief X that could yield some utility returns in a decade’s time if the community refactored a bit, it wouldn’t matter. They have mouths to feed right now.
On the other hand, if there’s too much slack, the adaption will also fail to reach fixation. Why? Well, even if it happened to do so, by a stroke of luck, it’d be helpless to prevent another adaption from replacing it. Too much slack means too little selective pressure, which means that there’s no mechanism by which any phenotype could characterize an evolutionarily stable equilibrium. In a lab where worms are fed no matter what, eyespots might arise and then fade back away, just as might the latest Western social fad among the bored and comfortable Russian elite.
In order for good adaptions to reach fixation, you need some slack, but not too much. When these conditions don’t hold, you might expect dysfunction.

What does this look like? Take education. The poor have little slack, so they under-educate themselves. There are scores of people with 145+ IQs working manual jobs. Whereas the middle class have some slack, so they get roughly the education they need — you see the greatest dispersion in middle class schools, with a sizable proportion not going to college and some going to very prestigious colleges. With the upper class, almost all go to good schools, and most are simply not equal to the education and eventual career they hold onto. They’re able to hold onto their jobs by dint of their good social networks (slack), and so ...
Against Being Against Growth
by Alex Hollow — 2021-05-29T02:18:37.660Z
Link: https://alexhollow.wordpress.com/against-being-against-growth/

(Cross-posted from my personal website - this is mostly meant for me to link when people argue against growth at me, so it may not be as relevant here, where most people are pro-growth)

It is inconsistent to be against growth or to advocate for social systems because you think they will reduce growth.

The most common formulation of this argument that I hear in the wild goes, essentially “capitalism bad because exponential growth is causing problems, do socialism instead” (ignoring the fact that the USSR grew faster than the US, for many years, and ignoring the benefits that exponential growth has brought). There are other problems with this argument, in my mind – if you plan on improving quality of life for people, how will you do that without growth of resource consumption? – but let’s address the general idea of advocating for a system on the grounds that it will grow slower.

Systems which grow slower than other systems will eventually not exist in any meaningful capacity. This is basic evolutionary theory – if there are two species, and one has kids twice as fast as the other, the fast-reproducing one will eventually take over entirely. The same is usually true of institutions – imagine two ice cream stores, one which cares about profit-maximization at the expense of all else, and the other which cares about X, for any X other than profit-maximization. Both ice cream stores want to become chains, and open a second location. The profit-maximizing ice cream store will maximize profit, and so earn enough money to open a second store faster, and then with its now doubled revenue (simplifying for this example), it will be able to open a third store before its competitor, and so on.

If you want to lower growth, you can’t do it by yourself. If any group in an evolutionary competition (and they’re all evolutionary competitions, in the long run) stops competing to maximize growth, they’ll become irrelevant. If you adopt a social system that intentionally limits growth, you will be out-grown by people who don’t.

That’s not to say that we are stuck maximizing growth. If growth was bad for everyone, you could potentially escape the growth-maximization dynamic with collective action. You could get countries to sign an agreement that would only take effect if enough countries signed, requiring any signatory to take military action against any economy that is growing too fas...
Wrist Update
by jefftk — 2021-05-29T01:30:02.563Z

It's been about a year now

since my wrists
got really bad. Unfortunately, it's no longer limited to just my
wrists: I'm having similar issues with my hands. Last time I wrote
about how my doctor thought that this was likely due to ganglion
cysts, but since then we've done more imaging and that looks like it
probably isn't the problem. After talking to a rheumatologist, it
looks like [1] it's probably a kind of inflammatory arthritis, which is a
chronic autoimmune disease.

So far, the only things that have helped have been minimizing use
and icing (flexible
wrist packs are nice!). I've tried two nonsteroidal
anti-inflammatory drugs (NSAIDs), Ibuprofen and Naproxen Sodium,
without effect. I'm about to start on the immunosuppressant
methotrexate, which I have mixed feelings about, but positive on
balance.

I'm still doing lots of dictation, but I haven't needed to continue
with voice control. When using the computer/phone, as long as I
dictate words, I've been able to still use my hands for navigation.
The biggest problem is when I have substantial tasks that aren't just
reading and writing, such as spreadsheet work or green-field coding. Luckily this
is only a very small portion of my regular work, and I've been doing
fine there, though I'm not sure what dictation will look like once I'm
back in the office. I'm back to playing music, which I'm really happy
about, but no jammer or fast
melody.

[1] My understanding is that the diagnosis is primarily based on the
appearance of my hand and wrist bones under x-ray. My joint spacing is
narrow and my bone density has decreased, especially in my wrist.
Comparing x-rays from 2020 and 2018, it is most apparent in 2020 but
2018 also showed the issues. They ran a lot of blood tests, which
were all normal, including testing negative for rheumatoid factor,
cyclic citrullinated peptide antibodies, sedimentation rate, and
c-reactive protein. No visible swelling, but substantial stiffness in the
morning—I initially didn't report this is a symptom, because I'd
assumed this is just the way hands are in the morning. Since I also
have some persistent dry scaly patches of skin, they thought most
likely psoriatic arthritis. Possibly something I've had since a kid,
since some kinds of juvenile arthritis can cause tendonitis-like
symptoms.

Comment via: facebook
How one uses set theory for alignment problem?
by Valentin2026 — 2021-05-29T00:28:01.832Z

Is there a simple explanation how set theory, mathematical logic etc. can be used for alignment problem? From reading this post I got an impression that it is very important for the research in MIRI, or, at least, was important in 2013. Maybe I simply don't know what the author means by this subjects. When I hear this, I am thinking about Gödel theorem, axiomatic of set theory and such stuff, and I can't imagine how is it related to the alignment. It would be nice to read something where it is explained. Thank you!
Is anyone else frustrated with 'un-informative' post titles?
by Kenny — 2021-05-28T20:11:20.063Z

The latest post that finally impelled me to ask this question:

A problem and three ideas - LessWrong

This isn't specific to that post and, having noticed that it is a cross-post, I can appreciate that this is (at least somewhat) tricky to improve, but a lot of the titles of posts are very un-informative. Even qualifying that the above post is specific to AI or AI alignment still doesn't seem like a significant improvement, or an improvement that's 'good enough'.

I notice that I often completely ignore posts with sufficiently vague titles. Upon (very shallow) introspection, I feel like the titles are pretty literally clickbait, i.e. 'click me to discover what this is about'. That seems like behavior that warrants 'punishing' (to some degree anyways).

One potential solution (mitigation) would be to include (one of) the tags in the post title, or maybe at least in the RSS feed for posts, à la Stack Overflow, e.g.:

google chrome - navigator.clipboard is undefined - Stack Overflow

In the above, the 'actual' title of the question is just "navigator.clipboard is undefined" and it seems like the 'first' tag is automatically included in the page title.
Teaching ML to answer questions honestly instead of predicting human answers
by paulfchristiano — 2021-05-28T17:30:03.304Z

(Note: very much work in progress, unless you want to follow along with my research you'll probably want to wait for an improved/simplified/clarified algorithm.)

In this post I consider the particular problem of models learning “predict how a human would answer questions” instead of “answer questions honestly.” (A special case of the problem from Inaccessible Information.)

I describe a possible three-step approach for learning to answer questions honestly instead:

Change the learning process so that it does not have a strong inductive bias towards “predict human answers,” by allowing the complexity of the honest question-answering to “pay for itself” by constraining the space of possible human-models.
Introduce a bias towards the intended model by using a more complex labeling process to answer questions where a human answers incorrectly.
Be really careful to avoid penalizing honest answers, by only judging comparisons between two answers where we are confident one is better than the other and getting the model to help us.

I don’t know whether this problem is a relatively unimportant special case of alignment, or one of the core difficulties. In any case, my next step will be trying to generate failure stories that definitely cannot be addressed by any of the angles of attack I know so far (including the ones in this post).

I think it’s relatively unlikely that almost anything specific I said here will really hold up over the long term, but I do think I’ve learned something about each of these steps. If the ideas end up being important then you can expect a future post with a simpler algorithm, more confidence that it works, clearer definitions, and working code.

(Thanks to Ajeya Cotra, David Krueger, and Mark Xu for discussions about this post that helped clarify it.)

The problem

Suppose that we train a model to answer questions in natural language about what will happen in the future (“Will Alice take the train home tonight?”) given a data about what’s happened in the recent past (e.g. video recorded over the last few days).

To generate training data we take the video from time t₀ to t₁ and get humans to answer questions about what happened between time t₁ and t₂. Then we do supervised learning on these (video, question, answer) triples.

The intended model has two parts: (i) a model of the world (and inference algorithm), (ii) a tran...
The blue-minimising robot and model splintering
by Stuart_Armstrong — 2021-05-28T15:09:54.516Z

A long time ago, Scott introduced the blue-minimising robot:

Imagine a robot with a turret-mounted camera and laser. Each moment, it is programmed to move forward a certain distance and perform a sweep with its camera. As it sweeps, the robot continuously analyzes the average RGB value of the pixels in the camera image; if the blue component passes a certain threshold, the robot stops, fires its laser at the part of the world corresponding to the blue area in the camera image, and then continues on its way.

Scott then considers holographic projectors and colour-reversing glasses, where the blue robot does not act in a way that actually reduces the amount of blue, and concludes:

[...] the most fundamental explanation is that the mistake began as soon as we started calling it a "blue-minimizing robot". [...] The robot is not maximizing or minimizing anything. It does exactly what it says in its program: find something that appears blue and shoot it with a laser. If its human handlers (or itself) want to interpret that as goal directed behavior, well, that's their problem.

The robot is a behavior-executor, not a utility-maximizer.

That's one characterisation, but what if the robot was a reinforcement-learning agent that was trained in various scenarios where they got rewards for blasting blue objects? Then it would seem that it was designed as a blue minimising utility maximiser; just not designed particularly well.

One approach would be "well, just design it better". But that's akin to saying "well, just perfectly program a friendly AI". In the spirit of model-splintering we could instead ask the algorithm to improve its own reward function as it learns more.

The improving robot

Here is a story of how that could go. Obviously this sort of behaviour would not happen naturally with a reinforcement learning agent; it has to be designed in. The key elements are in bold.

The robot starts with the usual behaviour - see a particular shade of blue, blast with laser, get reward when blue object destroyed.

The robot notices that there are certain features connected with that shade of blue. Specifically, the blue cells are cancer cells, died blue. Now, the robot has no intrinsic notion of "cancer cells", but can detect features - the extent of the objects, how they change, and how they differ from other extensive objects - that allow it to distinguish cancer cells

...
Curated conversations with brilliant rationalists
by spencerg — 2021-05-28T14:23:30.631Z

Since August 2020 I've been recording conversations with brilliant and insightful rationalists, effective altruists (and people adjacent to or otherwise connected somehow to those communities). If you're an avid reader of this site, I suspect you will recognize many of the names of those I've spoken to.

Since I suspect some LessWrong readers will appreciate these conversations, here is a curated list with links, organized by the LessWrong relevant topics we cover in each conversation. All of these conversations can also be found by searching for "Clearer Thinking" in just about any podcast app. If there are other people you'd like to see me record conversations with, please nominate them in the comments! The format is that I invite each guest to bring 4 or 5 "ideas that matter" that they are excited to talk about, and then the aim is to have a fun, intellectual discussion of those ideas.

Rationality

Lines of Retreat and Incomplete Maps with Anna Salamon

What does it mean to leave lines of retreat in social contexts? How can we make sense of the current state of the world? What happens when we run out of map? How does the book Elephant in the Brain apply to the above questions?

Rationality Education and Dating with Jacob Falkovich

What's the best way to teach rationality? How do you communicate rationalist principles to people who aren't already interested in thinking more clearly? What has COVID taught us about how people typically make decisions and think about problems? Where and how can the rationalist community improve? Does rationalism have anything to say about (for example) exercise, spirituality, art, or other parts of the human experience that aren't typically addressed by rationalists? What are some positive aspects of social media (especially Twitter)? What's going on with recent dating trends? Has dating gotten harder in recent years? How many people does it take to make a pencil? Is there a case to be made for anti-antinatalism?

Scout and Soldier Mindsets with Julia Galef

What are "scout" and "soldier" mindsets? How can we have productive disagreements even when one person isn't in scout mindset? Is knowing about good rationality habits sufficient to reason well? When do we naturally tend to be in scout mindset or soldier mindset? When is each mindset beneficial or harmful? Are humans "rationally irrational"? What are the two different types of co

...