trentmkelly/LessWrong-43k huggingface dataset:
Thinking of tool AIs
by Michele Campolo — 2019-11-20T21:47:36.660Z

Preliminary note: the ideas in the post emerged during the Learning-by-doing AI safety workshop at EA Hotel; special thanks to Linda Linsefors, Davide Zagami and Grue_Slinky for giving suggestions and feedback.

As the title anticipates, long-term safety is not the main topic of this post; for the most part, the focus will be on current AI technologies. More specifically: why are we (un)satisfied with them from a safety perspective? In what sense can they be considered tools, or services?

An example worth considering is the YouTube recommendation algorithm. In simple terms, the job of the algorithm is to find the videos that best fit the user and then suggest them. The expected watch time of a video is a variable that heavily influences how a video is ranked, but the objective function is likely to be complicated and probably includes variables such as click-through rate and session time.[1] For the sake of this discussion, it is sufficient to know that the algorithm cares about the time spent by the user watching videos.

From a safety perspective - even without bringing up existential risk - the current objective function is simply wrong: a universe in which humans spend lots of hours per day on YouTube is not something we want. The YT algorithm has the same problem that Facebook had in the past, when it was maximizing click-throughs.[2] This is evidence supporting the thesis that we don't necessarily need AGI to fail: if we keep producing software that optimizes for easily measurable but inadequate targets, we will steer the future towards worse and worse outcomes.

Imagine a scenario in which:

human willpower is weaker than now;

hardware is faster than now, so that the YT algorithm manages to evaluate a larger number of videos per time unit and, as a consequence, gives the user better suggestions.

Because of these modifications, humans could spend almost all day on YT. It is worth noting that, even in this semi-catastrophic case, the behaviour of the AI would be more tool-ish than AGI-like: it would not actively oppose its shutdown, start acquiring new resources, develop an accurate model of itself in order to self-improve, et cetera.

From that perspective, the video recommendation service seems much more dangerous than what we usually indicate with the term tool AI. How can we make the YT algorithm more tool-ish? What is a tool?

Unsurprisingly, it seems we d...
Welcome to The Thinkery
by SebastianG  — 2019-11-20T21:19:38.632Z

The purpose of the group is maintain continuity with rationality ethos, build "groupiness" with like-minded people, and offer support and sounding boards for each others' personal projects.

We normally do discussions and go on walks and such.

This group formally meets 2 - 4 times per year, and anyone who breathes air is welcome to join. We had one member who was a fish, but she moved to Austin.

If you are reading this, you have all the background you need. Come to our events even if you feel awkward or shy.
We have a Discord Channel and an email list which you can join by dming me.
Doxa, Episteme, and Gnosis Revisited
by Gordon Seidoh Worley — 2019-11-20T19:35:39.204Z

Exactly two years to the day I started writing this post I published Map and Territory's most popular post of all time, "Doxa, Episteme, and Gnosis" (also here on LW). In that post I describe a distinction ancient Greek made between three kinds of knowledge we might translate as hearsay, justified belief, and direct experience, respectively, although if I'm being totally honest I'm nowhere close to being a classics scholar so I probably drew a distinction between the three askew to the one ancient Attic Greeks would have made. Historical accuracy aside, the distinction has proven useful over the past couple years to myself and others, so I thought it was worth revisiting in light of all I have learned in the intervening time.

Nuanced Distinctions

To start, I still draw the categories of doxa, episteme, and gnosis roughly the same as I did before. To quote myself:

Doxa is what in English we might call hearsay. It’s the stuff you know because someone told you about it. If you know the Earth is round because you read it in a book, that’s doxa.

Episteme is what we most often mean by “knowledge” in English. It’s the stuff you know because you thought about it and reasoned it out. If you know the Earth is round because you measured shadows at different locations and did the math to prove that the only logical conclusion is that the Earth is round, that’s episteme.

Gnosis has no good equivalent in English, but the closest we come is when people talk about personal experience because gnosis is the stuff you know because you experienced it. If you know the Earth is round because you traveled all the way around it or observed it from space, that’s gnosis.

There's more nuance to it than that, of course. Doxa, for example, also refers to thoughts, beliefs, ideas, propositions, statements, and words in addition to its connotations of hearsay, common belief, and popular opinion. Episteme, to Plato, was the combination of doxa and logos, contrary to my example above where I root episteme in observational evidence, although then again maybe not because "logos" can mean not only "reason", "account", "word", and "speech" but also "ground" or "ultimate cause". And gnosis, despite its connotations in English as a special kind of insightful knowledge about the true nature of existence as a result of its use by Christian mystics, shares the same root or is the root via borrowing of the...
[AN #74]: Separating beneficial AI into competence, alignment, and coping with impacts
by Rohin Shah — 2019-11-20T18:20:01.647Z

Find all Alignment Newsletter resources

here

. In particular, you can

sign up

, or look through this

spreadsheet

of all summaries that have ever been in the newsletter. I'm always happy to hear feedback; you can send it to me by replying to this email.

Audio version

here

(may not be up yet).

Highlights

AI alignment landscape

(Paul Christiano) (summarized by Rohin): This post presents the following decomposition of how to make AI go well:

[

Link

to image below]

Rohin's opinion: Here are a few points about this decomposition that were particularly salient or interesting to me.

First, at the top level, the problem is decomposed into alignment, competence, and coping with the impacts of AI. The "alignment tax" (extra technical cost for safety) is only applied to alignment, and not competence. While there isn't a tax in the "coping" section, I expect that is simply due to a lack of space; I expect that extra work will be needed for this, though it may not be technical. I broadly agree with this perspective: to me, it seems like the major technical problem which differentially increases long-term safety is to figure out how to get powerful AI systems that are trying to do what we want, i.e. they have the right

motivation

(

AN #33

). Such AI systems will hopefully make sure to check with us before taking unusual irreversible actions, making e.g. robustness and reliability less important. Note that

techniques like verification, transparency, and adversarial training

(

AN #43

) may still be needed to ensure that the alignment itself is robust and reliable (see the inner alignment box); the claim is just that robustness and reliability of the AI's capabilities is less important.

Second, strategy and policy work here is divided into two categories: improving our ability to pay technical taxes (extra work that needs to be done to make AI systems better), and improving our ability to handle impacts of AI. Often, generically improving coordination can help with both categories: for example, the

publishing concerns around GPT-2

(

AN #46

) have allowed researchers to develop synthetic text detection (the first category) as well as to coordinate on when not to release models (the second category).

Third, the categorization is relatively agnostic to the details of the AI systems we develop -- these only show up in level 4, where Paul specifies that he...
Affordable Housing Workarounds
by jefftk — 2019-11-20T13:50:01.412Z

After reading some about how affordable housing is actually
implemented, it looks to me like rich people could exploit it to avoid
paying property and inheritance taxes, and generally get around the
means testing requirements.

Affordable housing is about renting or selling homes well below market
price, so if there were a large pool of affordability-restricted
properties there would be a lot of incentive for people to figure out
how to get around the spirit of the restrictions. I'm going to talk
about buying here, but renting has a lot of similarities.

A typical buyer restriction today (Somerville
example)
is something like:

Annual income no more than $71,400 for a household of two (80% AMI).

Non-retirement assets no more than $250k.

Haven't owned a home within 3y ("first-time homebuyer").

No students.

Preference for people who currently live or work in the city.

No legal minimum income, but mortgage lenders will apply ones in practice.

Buyers who meet these restrictions are entered into a lottery, and the
winner gets a 2-bedroom 2.5 bathroom 1,500 square-foot unit for $177k instead
of $1,049k.
Property taxes are also very low, ~$200/y instead of ~9k/y. [1]

These restrictions apply at purchase time: you have to have a
relatively low income and assets to qualify, but then there are no
further restrictions. This makes sense, because otherwise we would be
requiring poor people to stay poor, but it also allows a lot of
potential ways for rich people to 'legally cheat':

Intentionally keep a low income for several years. Three years
at $70k instead of $140k loses you $210k, but you'd save more than
that in property taxes alone long-term.

Arrange for deferred or speculative compensation. Stock that
vests in four years, stock options, start a startup.

Get training that gives you high earning potential, but don't
start your high paying job until after you have the house. This
training is effectively an asset, but it's very hard for the
affordable housing administrators to price it, so it's ignored.

Learn through self-study or apprenticeship to get around the
prohibition on students.

Postpone transfers to your children until after they have
qualified for affordable housing, since the income and assets of
relatives are not considered.

Buy land, take advantage of density bonuses, build a large 100%
affordable fancy building, an

...
Wrinkles
by johnswentworth — 2019-11-19T22:59:30.989Z

Why does our skin form wrinkles as we age?

This post will outline the answer in a few steps:

Under what conditions do materials form wrinkles, in general?
How does the general theory of wrinkles apply to aging human skin?
What underlying factors drive the physiological changes which result in wrinkles?

In the process, we’ll draw on sources from three different fields: mechanical engineering, animation, and physiology.

Why do Materials Wrinkle?

Imagine we have a material with two layers:

A thin, stiff top layer
A thick, elastic bottom layer

We squeeze this material from the sides, so the whole thing compresses.

The two layers want to do different things under compression:

The thin top layer maintains its length but wants to minimize bending, so it wants to bow outward and form an arc
The elastic bottom layer wants to minimize vertical displacement, so it wants to just compress horizontally without any vertical change at all.

Because the two layers are attached, these two objectives trade off, and the end result is waves - aka wrinkles. Longer waves allow the top layer to bend less, so a stiffer top layer yields longer waves. Shorter waves allow the bottom layer to expand/compress less vertically, so a stiffer bottom layer yields shorter waves. The “objectives” can be quantified via the energy associated with bending the top layer or displacing the bottom layer, leading to quantitative predictions of the wavelength - see

this great review paper

for the math.

Engineers do this with a thin metal coating on soft plastic. The two are bound together at high temperature, and then the whole system compresses as it cools. The end result is cool wrinkle patterns:

Other interesting applications include predicting mountain spacing (with crust and mantle as the two layers) and surface texture of dried fruit - see

the review paper

for more info and cool pictures.

The same thing happens in skin.

Skin Layers

For our purposes, skin has three main layers:

The epidermis is a thin, relatively stiff top layer
The SENEB (subepidermal non-echogenic band, also sometimes called subepidermal low-echogenic band, SLEB) is a mysterious age-related layer, mostly absent in youth and growing with age, between the epidermis and dermis - more on this later
The dermis is the thick base layer, containing all the support structure - blood vessels, connective tissue, etc

Both the SENEB and the dermis are re...
How I do research
by TurnTrout — 2019-11-19T20:31:16.832Z

Someone asked me about this, so here are my quick thoughts.

Although I've learned a lot of math over the last year and a half, it still isn't my comparative advantage. What I do instead is,

Find a problem

that seems plausibly important to AI safety (low impact), or a phenomenon that's secretly confusing but not really explored (instrumental convergence). If you're looking for a problem, corrigibility strikes me as another thing that meets these criteria, and is still mysterious.

Think about the problem

Stare at the problem on my own, ignoring any existing thinking as much as possible. Just think about what the problem is, what's confusing about it, what a solution would look like. In retrospect, this has helped me avoid anchoring myself. Also, my prior for existing work is that it's confused and unhelpful, and I can do better by just thinking hard. I think this is pretty reasonable for a field as young as AI alignment, but I wouldn't expect this to be true at all for e.g. physics or abstract algebra. I also think this is likely to be true in any field where philosophy is required, where you need to find the right formalisms instead of working from axioms.

Therefore, when thinking about whether "responsibility for outcomes" has a simple core concept, I nearly instantly concluded it didn't, without spending a second glancing over the surely countless philosophy papers wringing their hands (yup, papers have hands) over this debate. This was the right move. I just trusted my own thinking. Lit reviews are just proxy signals of your having gained comprehension and coming to a well-considered conclusion.

Concrete examples are helpful: at first, thinking about vases in the context of impact measurement was helpful for getting a grip on low impact, even though it was secretly a red herring. I like to be concrete because we actually need solutions - I want to learn more about the relationship between solution specifications and the task at hand.

Make simplifying assumptions wherever possible. Assume a ridiculous amount of stuff, and then pare it down.

Don't formalize your thoughts too early - you'll just get useless mathy sludge out on the other side, the product of your confusion. Don't think for a second that having math representing your thoughts means you've necessarily made progress - for the kind of problems I'm thinking about right now, the math has to sing with the

...
Drawing on Walls
by jefftk — 2019-11-19T15:00:02.091Z

When I

started the

bathroom

project there was a lot of reason to
move quickly: the bathroom wouldn't be usable while I was working on
it, and the back bedroom was full of construction stuff. Once I got
to the stage where the only thing left to do was plaster and paint the
hallway, however, it was less of a priority. So we spent May-November
with unfinished walls.

Since we were going to paint them at some point, one afternoon I thought it
would be fun to draw on them with the kids. We got out the markers and drew
lots of different things. I emphasized that it was only these walls we could
draw on, which is the kind of rule the kids do well with.

A couple days later they drew on the walls again, but this time with crayon.
Crayon, being wax-based, is not a good layer to have under paint. I hadn't
thought to tell them not to use them, and they didn't have a way to know, but I
was annoyed at myself. I got most of it off with hot water and a cloth, and
then when it came to do the plastering I put a skim coat over it.

Later on a friend wanted help preparing for a coding interview, so we
used the wall as a whiteboard:

One thing I hadn't considered was that you need more primer over dark
marker than plain drywall. As with "no crayon under paint" this seems
like it should have been obvious, and something I should have thought
about before letting them draw on a large area of the wall, but it
wasn't and I didn't, so Julia ended up spending longer painting and
priming than we'd been thinking.

And then, the evening after all that painting, Anna took a marker over
to the nice new clean white wall and started drawing. We hadn't told
her that the wall was no longer ok for drawing, and at 3y "now that
the wall is painted drawing isn't ok" is not the sort of thing I
should be expecting her to know on her own. Luckily the marker was
washable, so it wasn't too bad.

Overall this was more work than I was expecting, and probably wasn't
worth it for the fun, at least not the way we did it. If I did it
again I'd make sure they didn't use crayon, and either give them light
colored markers or pick a smaller section of the wall for them to play
with.
Austin meetup notes Nov. 16, 2019: SSC discussion
by jchan — 2019-11-19T13:30:53.446Z

The following is a writeup (pursuant to Mingyuan's proposal) of the discussion at the Austin LW/SSC Meetup on November 16, 2019, at which we discussed six different SlateStarCodex articles. We meet every Saturday at 1:30pm - if you're in the area, come join us!

You are welcome to use the comments below to continue discussing any of the topics raised here. I also welcome meta-level feedback: How do you like this article format? What sorts of meetups lead to interesting writeups?

Disclaimer: I took pains to make it clear before, during, and after the meetup that I was taking notes for posting on LessWrong later. I do not endorse posting meetup writeups without the knowledge and consent of those present!

The Atomic Bomb Considered As Hungarian High School Science Fair Project

There was a Medium post on John von Neumann, which was discussed on Hacker News, which linked to the aforementioned SSC article on why there were lots of smart people in Budapest 1880-1920.

Who was John von Neumann? - One of the founders of computer science, founder of game theory, nuclear strategist. For all his brilliance he's fairly unknown generally. Everyone who knew him said he was an even quicker thinker than Einstein; but why didn't he achieve as much as Einstein? Perhaps because he died of cancer at 53.

Scott Alexander says: {Ashkenazi Jews are smart. Adaptations can have both up- and down-sides (e.g. sickle cell anemia / malaria resistance); likewise some genes cause genetic disorders and also intelligence. These are common in Ashkenazim.}

Jews were forced into finance because Christians weren't allowed to charge interest on loans, but it turned out interest was really useful.

Scott Alexander says: {And why this time period? Because restrictions on Jews only started being lifted just before this period, and they needed a generation or so to pass before they could be successful. And afterward, Nazis happened. Why Hungary and not Germany? Hungary has a "primate city" (Budapest), i.e. a city that's much more prominent than others in its area, so intellectuals will tend to gather there. Germany, by contrast, is less centralized.}

Simulation of idea-sharing and population density - cities are more likely to incubate ideas (Hacker News discussion). Does that mean we'll get more progress if everyone in a certain field gathers in one place? Perhaps. It's helpful to get feedback for your ideas t...
Cybernetic dreams: Beer's pond brain
by Yuxi_Liu — 2019-11-19T12:48:15.512Z

"Cybernetic dreams" is my mini series on ideas from cybernetic research that has yet to fulfill their promise. I think there are many cool ideas in cybernetics research that has been neglected and I hope that this series brings them more attention.

Cybernetics is a somewhat hard to describe style of research in the period of 1940s -- 1970s. It is as much as an aesthetics as it is a research field. The main goals of cybernetics research are to understand how complex systems (especially life, machines, and economic systems) work, how they can be evolved, constructed, fixed, and changed. The main sensibilities of cybernetics are biology, mechanical engineering, and calculus.

Today we discuss Stafford Beer's pond brain.

Stafford Beer

Stafford Beer is a cybernetician that tried to make more efficient economic systems by cybernetic means. Project Cybersyn is his most famous project: making a cybernetic economy system. It will be discussed in a future episode. From Wikipedia:

Stafford Beer was a British consultant in management cybernetics. He also sympathized with the stated ideals of Chilean socialism of maintaining Chile's democratic system and the autonomy of workers instead of imposing a Soviet-style system of top-down command and control. One of its main objectives was to devolve decision-making power within industrial enterprises to their workforce in order to develop self-regulation [homeostasis] of factories.

The cybernetic factory

The ideal factory, according to Beer, should be like an organism that is attempting to maintain a homeostasis. Raw material comes in, product comes out, money flows through. The factory would have sensory organs, a brain, and actuators.

From (Pickering, 2004):

The T- and V-machines are what we would now call neural nets: the T-machine collects data on the state of the factory and its environment and translates them into meaningful form; the V- machine reverses the operation, issuing commands for action in the spaces of buying, production and selling. Between them lies the U-Machine, which is the homeostat, the artificial brain, which seeks to find and maintain a balance between the inner and outer conditions of the firm—trying to keep the firm operating in a liveable segment of phase-space.

By 1960, Beer had at least simulated a cybernetic factory at Templeborough Rolling Mills, a subsidiary of his employer, United Steel... [T

...