trentmkelly/LessWrong-43k huggingface dataset:
Speaking of the efficiency of utopia
by KatjaGrace — 2021-02-03T17:10:15.293Z

I noted that it is probably reasonable for people to be wary of things introducing themselves as ‘efficiency’, since maximization of specific metrics has some tendency to go badly.

On the other hand, ‘efficiency’ doesn’t mean anything about building explicit or quantitative machinery. It just means getting a lot of the desired output per input. So one might wonder why, if these explicit efforts would tend to make things worse for our actual goals, we would pursue efficiency in such ways, and continue to call that ‘efficiency’. For those who think quantitative pursuit of well-defined goals has been a bad strategy overall, shouldn’t ‘efficient daycare’ suggest a daycare where we have used our best intuitions for holistically improving the experience?

I think one reason why not is that you don’t tend to have much evidence that a thing is efficient unless you are in the business of making quantitative measurements. You are more likely to say ‘this daycare is unusually efficient’ if you have been measuring ‘costs of providing legally adequate childcare’ across different centers, than if you have been observing children and workshopping policies for their thriving. If you have been doing the latter, you will probably just call it something else.

It seems to me that we would be better with more specific concepts though, rather than conflating striving with an eye to a defined metric and the doing the best we can do with our materials. I suggest narrow efficiency and inclusive efficiency. An assembly line is narrowly efficient. Utopia is inclusively efficient.
Late-talking kids and "Einstein syndrome"
by Steven Byrnes — 2021-02-03T15:16:05.284Z

(See also my follow-up posts later: More notes from raising a late-talking kid and
Late-talking kid part 3: gestalt language learning.)

I talked late—no words at age 2, ten words at 2¼, lots and lots of words at 2½. Or so I've been told! :-P

My younger kid is also a late talker—maybe 7 words at age like 1½, and still ~7 words now well into age 2. (And not all the same words! He added a couple and dropped a couple.) When my older kid was the age that my younger kid is now, he was chatting away in 10-word sentences. So anyway, as a Responsible Parent, I'm doing some homework about late talking. Here are some of my notes, not in any particular order.

(If you like my speculative neuroscience writing, there's a bit of it in the last section.)

Update later: Yay! Without any interventions, he finally started rapidly increasing his spoken vocabulary just a few months after I wrote this. :-)

Update even later: Ugh, his language progress got much slower after that burst. As I write this, he’s 4, and has a very obvious language delay, and this is a constant source of stress.

Books

I read the 2014 book Late-Talking Children: A Symptom or a Stage? by Stephen M. Camarata. It's excellent, I really trust the guy. I also read the 2002 book The Einstein Syndrome: Bright Children Who Talk Late by Thomas Sowell. Sowell is an economist (!) while Camarata is a professor of speech and language disorders, so predictably Camarata is much more knowledgeable. My understanding is that Sowell's books were valuable at the time for kick-starting public awareness of the existence of bright late-talking kids (more on which below), but now that Camarata's book is out, that's the best resource.

Sowell was not a late talker himself, but Camarata was, and Sowell and Camarata each have a late-talking child.

Camarata has a comprehensive discussion of the full range of conditions that lead to late talking, from autism to apraxia to "nothing in particular, just a passing phase". Speaking of which...

"Einstein Syndrome"

There's a subgroup of late talkers who are smart (when tested non-verbally), often understand spoken language despite not talking, don't have autism or hearing loss or other conditions, and whose speech delay resolves on its own and they wind up settling into the normal range for adult language skills. This turns out to be an interesting bunch—this is what Sowell dubbed "Einstein Syndrome"...
Counterfactual Planning in AGI Systems
by Koen.Holtman — 2021-02-03T13:54:09.325Z

Counterfactual planning is a design approach for creating a range of
safety mechanisms that can be applied in hypothetical future AI
systems which have Artificial General Intelligence.

My new paper
Counterfactual Planning in AGI Systems
introduces this design approach in full.
It also constructs several example AGI safety mechanisms.

The key step in counterfactual planning is to use an AGI machine
learning system to construct a counterfactual world model, designed to
be different from the real world the system is in. A counterfactual
planning agent determines the action that best maximizes expected
utility in this counterfactual planning world, and then performs the
same action in the real world.

Examples of AGI safety mechanisms that can be constructed using
counterfactual planning are:

An agent emergency stop button, where the agent does not have a direct incentive to prevent its stop button from being pressed

A safety interlock that will automatically stop the agent before it undergoes an intelligence explosion

An input terminal that can be used by humans to iteratively improve
the agent's reward function while it runs, where the agent does not have a direct incentive to manipulate this improvement process

A counterfactual oracle.

Counterfactual planning is not a silver bullet that
can solve all AI alignment problems. While it is a technique for
suppressing strong direct incentives, it will not automatically
remove all remaining indirect incentives which can also lead to
unsafe behavior.

This sequence

In this sequence of Alignment Forum posts. I will give a high-level
introduction to counterfactual planning. The sequence uses text and
figures from the paper, but omits most of the detailed mathematical
definitions in the paper.

I have also added some extra text not included in the paper,
observations targeted specifically at long-time LessWrong/Alignment
Forum readers. For example, in LessWrong terminology, the paper
covers subjects like agent foundations, decision theory, and the
embedded agency, but you won't find these terms being mentioned in the
paper.

Use of natural and mathematical language

When writing about AGI systems, one can use either natural language,
mathematical notation, or a combination of both. A natural
language-only text has the advantage of being accessible to a
larger audience. Books like
Superintelligence
and
Human Comp...
Does anyone else sometimes "run out of gas" when trying to think?
by Sunny from QAD — 2021-02-03T12:57:55.188Z

Sometimes when I'm trying to think something through (like a social dilemma, or a chess problem, or a program I'm writing, etc) I have a pretty unpleasant experience that feels to me the way I imagine a car feels when it runs out of gas in the middle of the interstate. My thoughts screech to a halt, and I stop being able to make progress. If I try to force myself, my head fills with mental static. If I try to power through that, it builds into heavy psychological pain, and I burn through willpower insanely quickly. The only solution I know of is to give in and come back to the problem later.

I have two questions about this, and if you can only answer one of them you should feel free to ignore the other.

Does anyone else experience this / Is this a universal human experience?
Do you have any tips for dealing with it? Predicting when it will occur so I can schedule important work at other times, or taking steps that reduce how frequently it happens, or using some special hack to circumvent it when it's already happening?
Distinguishing claims about training vs deployment
by Richard_Ngo — 2021-02-03T11:30:06.636Z

Given the rapid progress in machine learning over the last decade in particular, I think that the core arguments about why AGI might be dangerous should be formulated primarily in terms of concepts from machine learning. One important way to do this is to distinguish between claims about training processes which produce AGIs, versus claims about AGIs themselves, which I’ll call deployment claims. I think many foundational concepts in AI safety are clarified by this distinction. In this post I outline some of them, and state new versions of the orthogonality and instrumental convergence theses which take this distinction into account.

Goal specification

The most important effect of thinking in terms of machine learning concepts is clarity about what it might mean to specify a goal. Early characterisations of how we might specify the goals of AGIs focused on agents which choose between actions on the basis of an objective function hand-coded by humans. Deep Blue is probably the most well-known example of this; AIXI can also be interpreted as doing so. But this isn't how modern machine learning systems work. So my current default picture of how we will specify goals for AGIs is:

At training time, we identify a method for calculating the feedback to give to the agent, which will consist of a mix of human evaluations and automated evaluations. I’ll call this the objective function. I expect that we will use an objective function which rewards the agent for following commands given to it by humans in natural language.
At deployment time, we give the trained agent commands in natural language. The objective function is no longer used; hopefully the agent instead has internalised a motivation/goal to act in ways which humans would approve of, which leads it to follow our commands sensibly and safely.

This breakdown makes the inner alignment problem a very natural concept - it’s simply the case where the agent’s learned motivations don’t correspond to the objective function used during training.[1] It also makes

ambitious

approaches to alignment (in which we try to train an AI to be motivated directly by human values) less appealing: it seems strictly easier to train an agent to obey natural language commands in a common-sense way, in which case we get the benefit of continued flexible control during deployment.[2]

Orthogonality

Consider Bostrom’s orthogonality thesis, whic...
A Critique of Non-Obstruction
by Joe Collman — 2021-02-03T08:45:42.228Z

Epistemic status: either I’m confused, or non-obstruction isn’t what I want.

This is a response to Alex Turner’s

Non-Obstruction: A simple Concept Motivating Corrigibility

. Please read that first, and at least skim Reframing Impact where relevant.
It’s all good stuff.

I may very well be missing something: if not, it strikes me as odd that many smart people seem to have overlooked the below. From an outside-view, the smart money says I'm confused.
Feel free to mentally add “according to my current understanding”, “unless I’m missing something”, “it seems to me” as appropriate.

I’m writing this because:

Non-obstruction seems like an important idea, but I don’t think it works.
I’d like to find out whether/where I’m confused, why the below can't happen etc.
I want to encourage the norm of thoughtful public criticism of important ideas - even where the critic is uncertain, and may have overlooked technical details.
I stubbornly believe I'm correct :)

[ETA after talking to Alex, we agreed that this is a problem. He suggested allowing a small margin would clear up the likely cases, and I agree with this. I.e. you have a condition more like: the case with the AI activated must give at least 98% of the non-activated attainable utility for all counterfactual payoff functions.
So, for example, this should allow you the chance to do a long reflection, even if you're comparing to a baseline which hastily optimizes hard for a narrow target (which may mistakenly exclude the true payoff function). Under likely conditions, thinking for a few thousand years may lose quite a bit of attainable utility, but not 2%.
Ideally, I'd like a function which tells us the required margin on a set S for some notion of "well-behaved" non-obstruction, but I don't yet see how to do this. (not too important whether it'd return 0.01% or 0.001% margin, but where it'd return 80%, something strange would be happening)]

Summary of my claims:

Non-obstruction seems to be useful where our AU landscape is pretty flat by default.
Our AU landscape is probably spikey by default.
Non-obstruction locks in default spike-tops in S, since it can only make Pareto improvements. (modulo an epsilon here or there)
Locking in spike-tops is better than nothing, but we can, and should, do better.

Alex makes a good case for claim 1.

My argument for claim 2 is that if our AU landscape were flat, we wouldn’t be worried:
Our assumption...
Covid cafes
by KatjaGrace — 2021-02-03T08:20:13.096Z

I’m puzzled that during the pandemic so few cafes near me have moved to serving customers outside, by moving their ordering and payment apparatus to the doorway. I’ve seen about five cafes in San Francisco do this (few enough that none are conveniently close).

(I wanted to include a photo, but I actually just can’t find a picture online, such an obscure idea it is, I guess?)

Is this harder than it looks to organize? And even if it is for a small business run by a single person without a spare second all year to think about aerosols or reorganize, I’m still surprised that Starbucks doesn’t have its act together more.

How bad is it to have your customers indoors to order and pickup their drinks? Some Microcovid estimates for San Francisco, right now:
Indoor: about 20 microcovids
Outdoor: about 1 microcovid

So very roughly 20 extra microcovids per visitor.

How many visitors? Seems like a Starbucks restaurant serves maybe 500 customers per day = ~10k extra microcovids per day. A lot, but still 100 days to cause an extra case of covid. So only a few extra cases of covid per year, per cafe. Then maybe a factor of two for people they infect. Should they move their whole ordering counter to the front of the store just to avoid like seven people getting covid per year?

Well, that’s maybe 14% of a death per year per cafe, and maybe one person disabled longer term.

Which seems pretty bad, for an arrangement of furniture. For instance, if some brand of cafe had a characteristic setup of tables such that every year in one in seven of these cafes someone died, and in just about every outlet, someone was badly injured every year, I think their negligence would be their main claim to fame, if they were not straight up shut down by the law. Am I wrong?

We can also ask, how bad is it for someone to get covid, then compare to the costs of rearranging. Let’s very roughly guess: P(death) of 1.7% * estimated statistical value of life of $9M * 15% of life left at average age of US covid death of about 76 * a factor of 2 for disability (a guesstimate I’ve heard, based on way more likely but way less bad than death, though I wonder about this, since it is also affecting people much younger). That gives us $0.05/microcovid.1 So doing it outside seems worth very roughly $1 to each customer, or about $500 per day across customers.

This has ignored spread from each person who gets it to others, ...
Browser Audio Latency
by jefftk — 2021-02-03T02:40:05.922Z

Any time you are working with digital audio, you have to handle
latency: how long does it take from when the audio enters the system
until it leaves again? Special purpose professional-level audio
hardware generally has been optimized for extremely low latency, often
adding as little as 2ms. Extreme low latency, however, has costs:
scheduling must be very predictable, power consumption is higher,
audio processing algorithms can look at fewer samples, and you have
much less recovery time before causing an audible glitch.

Web browsers are in an interesting position, where they have to work
with whatever audio hardware the computer happens to have, and need to
handle a very wide range of use cases.

I've recently been working on two musical projects where I care a lot
about latency, but in different ways:

Bass Whistle JS: a bass
synthesizer that you control by whistling. To feel natural and
responsive, it needs the lowest possible latency.

Bucket Brigade: a
platform for making music with others over the internet. It doesn't
need low latency, but it does need to know precisely how much there is
so it can compensate.

But how much latency is there? We can measure it: send a beep to the
speaker, check how much time passes until we detect the beep in the
samples coming from the microphone (example worklet). On my 2016 15"
MacBook running OSX 11.1, if I do:

var audioCtx = new window.AudioContext();
var micStream =
await navigator.mediaDevices.getUserMedia(
{audio: true});

I measure ~67ms end-to-end latency in Chrome, and ~55ms in Firefox.
Instead of using default settings, though, we could tune for low
latency:

var audioCtx = new window.AudioContext({
latencyHint: 0,
});
var micStream =
await navigator.mediaDevices.getUserMedia({
audio: {
latency: 0,
echoCancellation: false,
noiseSuppression: false,
autoGainControl: false,
}});

Note that we're asking AudioContext for an output latency
of '0' and not interactive. While MDN documents
interactive as "try to use the lowest possible and
reliable latency it can", '0' is even lower. We are also turning off
all of the smart
processing options, which would add latency.

This gives me ~19ms in Chrome and ~14ms in Firefox. Try it yourself.

In the other direction, for Bucket Brigade, I tune for high la...
Reflections on the cryonics sequence
by mingyuan — 2021-02-03T01:17:13.556Z

Three months ago, I had a dream my mom died while cryocrastinating, and I decided to finally start the process. I quickly found that it was ridiculously hard to figure out what the process even was, and I thought, "huh, someone should really write a guide on how to do this." And lo, I just spent the past three months writing that guide. It is 24,000 words long and took up about 90% of my working hours for the past three months.

I knew very little about cryonics before starting the sequence. It took me about five years after first encountering the idea on LessWrong to feel comfortable enough with it that I wanted to sign up. I still think it's an incredibly long shot, and I'm probably just using it the way many people use religion – to stave off my crippling fear of death. I'm just some random person, not an 'official expert' or even someone who's that deeply invested.

But I've already said everything I want to say about cryonics itself. This post is about the other things I've learned.

First, I learned that I have a really great intellectual support community. Most of my questions were fielded by former housemates, coworkers, and erstwhile acquaintances. Mati Roy was super helpful – he's a knowledgeable and committed cryonicist who's embedded in all of the relevant conversations. My mom and sister (both writers and editors in their own right) helped with last-minute proofreading. And Habryka was a real sport all the times I got so confused about life insurance that I was on the verge of tears (which happened periodically throughout the entire three months); he even let me call him about it in the middle of the workday once.

But more importantly for me, I learned that I am capable of figuring things out and building an understanding of new areas. The skill of "research" always seemed to me like a total black box, and I was told repeatedly it was a skill I didn't have – first when I washed out of the GiveWell Research Analyst application process at the first step (right after college), and later when I was put in a role where I was supposed to do research and was repeatedly given frankly hurtful negative feedback that amounted to little more than "you are bad at this, why aren't you better?" I figured that the people around me who were doing "research" knew some magic secret that I didn't, and I felt paralyzed and fell further and further behind.

But it turns out t...
Has there been any work done on the ethics of changing people's minds?
by mcorvin — 2021-02-03T01:03:30.710Z

Suppose that I was able to come up with an extremely compelling argument for the claim "Mint Chip is the most delicious flavor of ice cream." This argument was so compelling that regardless of what ice cream flavor one initially preferred, after reading the argument they would inevitably find that mint chip did in fact taste much better than other competing flavors.

Would it be ethically wrong of me to post this argument in text form on a forum? Or stand on the street corner yelling it into a megaphone at unsuspecting pedestrians? If the argument is truly a rigorous one such that peoples change in attitude comes from instantly realizing a truth about the world, then have they been harmed by being forced to shed a false belief? In the more likely case, suppose that the argument simply made use of all the various rhetorical strategies fallacies that humans tend to be very vulnerable too. Is it ever ethical to use such an argument on another person, with their consent or otherwise?

This might seem like a silly question, but there are a lot of branches here that seem worth pursuing - especially considering that slightly tweaking ice cream preferences ranks close to the bottom of "potentially sinister ways to modify someone else's internal utility function." On a more positive note, if our goal is to move towards a better world and work to alleviate suffering, at some point we will need to come up with effective ways of changing peoples minds. Do we currently have an ethical framework for doing so?
Bayesian inference on 1st order logic
by Daniel Abolafia — 2021-02-03T01:03:28.153Z

This is an excerpt from a longer article I wrote, where I go into much more depth on many of the ideas here.

In his blog post, Probability theory does not extend logic, David Chapman goes through a number of examples which he claims don't make sense, and therefore Bayesian 1st order logic is dead in the water. I'm now going to take up Chapman's challenge to make his examples well posed, i.e. syntactically valid (only one way to blindly parse the math) and with a reasonable interpretation of what is going on.

Through his "Challenge" problems, Chapman wants to demonstrate that arbitrary nesting of probabilities mixed with quantifiers does not parse, and that generalities cannot be inferred from instances (at least not with Bayesian inference). By making his challenge problems well posed, I hope to demonstrate that both are possible in the Bayesian framework.

Quick review of probability theory

For an in-depth review of probability theory, see my blog post: Primer to Probability Theory and Its Philosophy.

First, we need a set of possibilities, Ω, called the sample set (or sample space). A random process chooses a particular sample ω∈Ω. The word "sample" here is not intended to mean a collection of datapoints. I prefer to think of ω as a possible world state.

The triple (Ω,E,P) (or (Ω,E,μ)) is called a probability space, which is a type of measure space.
P (or sometimes μ) is a measure, which is a function mapping subsets of Ω to real numbers between 0 and 1 (inclusive).
E⊆2Ω is the event set, i.e. set of measurable sets of samples ω. By default assume E=2Ω. Don't worry about this detail.
Then P:E→[0,1].

A probability measure is different from a probability mass (or density) function. P is a function of sets. A probability mass function, e.g. F:Ω→[0,1], is a function of samples.

P({ω1,ω2,ω3}) is valid.
P({ω}) is valid.
P(ω) is invalid. You should write F(ω) where F is the probability mass function for P, i.e. F(ω)=P({ω}).

Having two kinds of probability functions is annoying, so we can pull some notational tricks. For example, leave out function call parentheses:

P{ω}=P({ω}).

Another trick is to use random variables:

P(X=x)

expands out to

P{ω∈Ω∣X(ω)=x}

where the contents of P(…) are a boolean expression (i.e. a logical proposition), and bold variables become functions of ω in the set-constructor version.

The bold variables are called random variables, which are (measurable; bu...