trentmkelly/LessWrong-43k huggingface dataset:
Physics has laws, the Universe might not
by Shmi — 2018-06-09T05:33:29.122Z

Inspired by http://backreaction.blogspot.com/2018/06/physicist-concludes-there-are-no-laws.html, which dissed this article: https://www.quantamagazine.org/there-are-no-laws-of-physics-theres-only-the-landscape-20180604.

Epistemic status: very raw, likely discussed elsewhere, though in different terms, but feels like has a kernel of usefulness in it.

What does it mean for the universe to be governed by physical laws? What does the term physical law mean? It means that someone knowing that law can predict with some accuracy the state of the universe at some point in the future from its state at the time of observation. Actually, a qualifier is in order. Can predict the observed state of the universe at some point in the future from its observed state at the time of observation. So

laws => predictability

This is more than a one-directional implication, however. What does it mean for something to be predictable? Again, it means that, by observing the state of the universe at some point in time the observer can make a reasonably accurate prediction of the observed state of the universe at some point in the future. Notice the qualifier "observed" again. How can an observer make this prediction? They must have a model of the observed universe ("map of the territory") inside, and use this model ("trace the map") to predict the observed state of the universe at some point in the future. This model can be very simple, "Raarg hold rock. Raarg let go. Rock fall", or more complicated, "In absence of other forces all objects accelerate downward at 9.81 meters per second squared", or even more abstract, "The stress-energy tensor is proportional to the spacetime curvature." But it is a model nonetheless.

When is a model promoted to the status of a law? When it is useful for more than a single case. When the prediction can be made repeatedly in similar but slightly different circumstances using the same model. There is a lot of complexity hiding under the surface of this "simple" statement, but at the end of the day, models are only useful if they can be reused, and thus become patterns, templates for the observers to predict the universe. Thus we have the implication in the other direction:

predictability => laws

Thus the two terms are equivalent, at least in this framework:

predictability <=> laws

L...
Could we send a message to the distant future?
by paulfchristiano — 2018-06-09T04:27:00.544Z

Suppose that humanity wiped itself out but left behind complex multicellular life. I think there is a good chance that space-faring civilization would emerge again on Earth, and I've argued that it might be worthwhile to try sending them a message.

(In that post I guesstimate that if all went well you might be able to effectively reduce extinction risk by 1/300 by sending a message to the future; I could imagine that costing only ~$10M and leveraging interest from non-EAs, in which case it sounds like a good buy.)

Unfortunately, sending a message 500 million years into the future seems very hard. Most things that aren't buried will get destroyed and the landscape itself is going to be completely transformed (tectonic plates will be rearranged, mountains will appear and disappear, the world will be covered in forms of life that don't exist yet...).

H/T to Dan Kane for criticizing my original post and clarifying the actual nature of the problem.

More precisely my question is whether we can:

Spend $10M-$100M.
Encode 100MB of information.
Wait 500 million years.
Have it be found with probability >25% by a civilization as sophisticated as humanity in 1900.

I'd also be interested in relaxing any of these constraints a little bit, e.g. spending 10x more, sending 10x less data, only lasting 100M years, or only being discoverable in the 21st century.

I'm not very worried about preserving the information---I suspect that if we are willing to bury something, we can preserve 100MB pretty cheaply. My biggest concern is putting the information somewhere that our successors can find it.

My initial suggestion, clarified/improved/named by Jess Riedel, was to:

Bury a small number of expensive "payload" messages (maybe ~100)
Bury a large number of "map" messages (maybe 10,000 - 100,000).
Somehow get people to stumble across a map. Either put a lot of them in places where they might end up being easy to find, or transform the landscape in ways that might remain visible in 500M years.
Use the maps to encode the location of payloads. (With each payload also including much more detailed maps to all the other payloads.)

I now feel like both of these steps are hard:

It seems very difficult to actually put maps in places where they'll be found. For example, my proposal of using a giant+out-of-place+slow-to-weather rock is probably very difficult

...
RFC: Meta-ethical uncertainty in AGI alignment
by Gordon Seidoh Worley — 2018-06-08T20:56:26.527Z

I'm working on writing a paper about an idea I previously outlined for addressing false positives in AI alignment research. This is the first completed draft of one of the subsections arguing for the adoption of a particular, necessary hinge proposition to reason about aligned AGI. I appreciate feedback on this subsection especially regarding if you agree with the line of reasoning and if you think I've ignored anything important that should be addressed here. Thanks!

AGI alignment is typically phrased in terms of aligning AGI with human interests, but this hides some of the complexity of the problem behind determining what "human interests" are. Taking "interests" as a synonym for "values", we can begin to make some progress by treating alignment as at least partially the problem of teaching AGI human values (Soares, 2016). Unfortunately, what constitutes human values is currently unknown since humans may not be aware of the extent of their own values or may not hold reflexively consistent values (Scanlon, 2003). Further complicating matters, humans are not rational, so their values cannot be deduced from their behavior unless some normative assumptions are made (Tversky, 1969), (Armstrong and Mindermann, 2017). This is a special case of Hume's is-ought problem—that axiology cannot be inferred from ontology alone—and it complicates the problem of training AGI on human values (Hume, 1739).

Perhaps some of the difficulty could be circumvented if a few normative assumptions were made, like assuming that rational preferences are always better than irrational preferences or assuming that suffering supervenes on preference satisfaction. This poses an immediate problem for our false positive reduction strategy by introducing additional variables that will necessarily increase the chance of a false positive. Maybe we could avoid making any specific normative assumptions prior to the creation of aligned AGI by expecting the AGI to discover them via a process like Yudkowsky's coherent extrapolated volition (Yudkowsky, 2004). This may avoid the need to make as many assumptions, but still requires making at least one—that moral facts exist to permit the correct choice of normative assumptions—and reveals a deep philosophical problem at the heart of AGI alignment—meta-ethical uncertainty.

Meta-ethical uncertainty stems fr...
Quantum AI Goal
by Gurkenglas — 2018-06-08T16:55:22.610Z

With quantum computing, we may not need to specify a goal for an AI.

Epistemic status: armchair ethics

[Scott Aaronson's thesis, page 3, last paragraph] talks of measuring how real something is by how much work it would take to manifest.

Therefore, we might interpret the utility of a computer in n timelines not as the average utilities of each timeline, but as the utility of the average of the timelines.

Therefore, we might give an AI the goal to gather the universe into a giant quantum computer, and let each timeline be ruled by an AI with a different goal.

If we can make sure that none of these will contain evils such as suffering humans, and at least one of them will be utopic, and we do not feel a difference between 10^40 happy lives and 10^30, this may be enough.
Quantum AI Box
by Gurkenglas — 2018-06-08T16:20:24.962Z

With quantum computing, an AI box needs not guard against hardware exploits.

Epistemic status: armchair computer science

Here's a simple machine model no more powerful than a quantum computer.

It's a computer with extra instructions. It can spend x*t time to, within the machine, split time into x² timelines. It knows which timeline it's in. Timelines cannot interact. If it attempts to act on anything but the memory while there are n timelines, with probability 1/n, all other timelines are deleted; otherwise, its own timeline is deleted. It can delete its timeline if others exist. After t time, all but one random remaining timeline are deleted.

Split into a² timelines. In timeline 1, run the AI. If it somehow manages to affect the world, it only has a 1/a² chance to escape. After it is done running, run an algorithm of your choice to curate the output. Have all other timelines delete themselves.

Author's Note: I previously made a post making much the same point. I was told it's hard to understand. I suppose I tried to write what I would wish to read.

Edit: I revoke this post's discussion of how to curate output without handing the world to the AI.
Effective Altruism as Global Catastrophe Mitigation
by Evan_Gaensbauer — 2018-06-08T04:17:03.925Z

Cross-posted to the Global Risk Research Network and the Effective Altruism Forum

Summary: In this post I posit how all heretofore activities of effective altruism, both of its original three causes and the ones we've seen its inception, essentially fit an evidence-based pattern-recognition system. I delineate the basis of this framework throughout the history of effective altruism and precursor movements as a focus on groups of beneficiary populations whose well-being is as important as anyone else's, but for a variety of reasons these problems have gone neglected or intractable, until the turn of the 21st century when emerging technologies promised the potential to turn the tide. On this basis EA and adjacent movements have become a globally coordinated network to apply emerging technologies to mitigate extant or potential global catastrophes fitting this framework.

I. Introduction

In May 2016, Ozy Brennan wrote on their blog

Missing Cause Areas

, on how what have thus far been the priority focus areas of effective altruism (EA) appear historically contingent. From their post:

Global poverty seems to be the least historically contingent cause area: if Peter Singer’s Famine, Affluence, and Morality didn’t get people to help people in the developing world, Peter Unger or any of the other people working with similar ideas would have. The popularity of animal rights, however, is clearly connected to the fact that Peter Singer, a prominent early effective altruist, wrote Animal Liberation, one of the foundational books of the animal rights movement. He carried over a significant amount of his fanbase from animal rights into effective altruism.

As a cause area, existential risk reduction seems to be solely a product of Eliezer Yudkowsky, a tireless promoter of effective altruism who researches the risks of artificial general intelligence. In my experience, interest in other kinds of existential risk among effective altruists appears to be primarily a product of people who accept Eliezer’s arguments about the importance of the far future and existential risk, but who are skeptical about the importance of the specific issue of artificial general intelligence.

Existential risk reduction, global poverty, and animal rights all seem to me to be important issues. But “global poverty, plus the pet issues of people who got a lot of people into EA” doesn’t seem to m

...
Poker example: (not) deducing someone's preferences
by Stuart_Armstrong — 2018-06-08T03:19:38.144Z

I've shown that it is, theoretically, impossible to deduce the preferences and rationality of an agent by looking at their actions or policy.

That argument is valid, but feels somewhat abstract, talking about "fully anti-rational" agents, and other "obviously ridiculous" preferences.

In this post, I'll present a simple realistic example of human behaviour where their preferences cannot be deduced. The example was developed by Xavier O'rourke.

The motivations and beliefs of a poker player

In this example, Alice is playing Bob at poker, and they are on their last round. Alice might believe that Bob has a better hand, or a worse one. She may be maximising her expected income, or minimising it (why? read on to see). Even under questioning, it is impossible to distinguish an Alice belief in Bob having a worse hand and Alice following a maximising behaviour, from Bob-better-hand-and-Alice-minimising-income. And, similarly, Bob-worse-hand-and-Alice-minimising-income is indistinguishable from Bob-better-hand-and-Alice-maximising-income.

If we want to be specific, imagine the we are observing Alice playing a game of Texas holdem'. Before the river (the final round of betting), everyone has folded besides Alice and Bob. Alice is holding (K♠,10♡), and the board (the five cards both players have in common) is (10♢,10♣,10♠,J♠,Q♠).

Alice is looking at four-of-a-kind in 10's, and can only lose if Bob holds (9♠,8♠), giving him a straight flush. For simplicity, assume Bob has raised, and Alice can only call or fold -- assume she's out of money to re-raise -- and Bob cannot respond to either, so his actions are irrelevant. He has been playing this hand, so far, with great confidence.

Alice can have two heuristic models of Bob's hand. In one model, μ1, she assumes that having specifically (9♠,8♠) is very low, so she almost certainly has the better hand. In a second model, she notes Bob's great confidence, and concludes he is quite likely to have that pair.

What does Alice want? Well, one obvious goal is to maximise money, with reward R$, linear in money. However, it's possible that Alice doesn't care about how much money she's taking home -- she'd prefer to take Bob home instead, her reward is RBob -- and she thinks that putting Bob in a good mood by letting him win at poker will make him more receptive to... ,,,,,,,,,,,,,,,,,,,
The Incoherence of Honesty
by Gordon Seidoh Worley — 2018-06-08T02:28:59.044Z

Some rationalists have what to me feels like an obsession with honesty. That's fine: we can all be obsessed with our own concerns. But I also think it's a rather strange sort of obsession given the nature of truth and our relationship to it even if we assume the framing of Bayesian epistemology. This is a brief case that most thinking about honesty rests on shaky ground (and is therefore strictly incoherent) as presented and only makes sense through the lens of a generous, reconstructive reading.

Let's first be clear about what "honesty" means. The word "honesty" has its origins in "honor" and comes to have its relationship to truth telling by means of conflating together all moral virtues. I'm going to diverge from this history and use "honesty" as a technical term with a precise definition, and I do so because oddly English has the single-word verb "to lie" but has to use a verb phrase to mean its antonym, "to tell the truth". This is unfortunate because "to lie" is the only word of the bunch that is not attempting to make a normative statement about the subject's actions or imply a disposition to either the action or the subject, but it would be an odd circumlocution to talk about antilying, so "honesty" it is.

(As we'll see, though, there's perhaps good reason for this: it's much easier to talk about what it means to lie than what it means to not lie!)

We might like to say a statement is honest if it is a true statement and to conflate honesty with truth as English tries to do, but this is to ignore that a person can be mistaken, and we would not call them a lier for stating what they believed to be the truth, i.e. they did not try to deceive. So instead we could say a statement is honest if it is a true statement about a person's knowledge, but this phrasing has two problems. First, it supposes that truth is a quality a statement can have rather than an assessment of some criteria someone makes about the statement; it would be safer to say that a statement is honest for a particular subject if they believe it to be a true statement about a person's knowledge. But perhaps you are an essentialist (you believe ontology precedes epistemology) and this is not a problem for you. No worries, because the second problem with this definit...
Reflections on Berkeley REACH
by stardust — 2018-06-08T00:02:15.588Z

This post covers my findings so far in the experiment of running the Berkeley Rationality and Effective Altruism Community Hub (REACH). Crossposted on EA Forum and LessWrong.

tl;dr

REACH has been running since March 2018 (around three months)
It’s doing well

Hundreds of people have enjoyed REACH

During the day, there are generally between 3 and 10 people coworking
Regular events draw 10-40 people
Large one-time events draw around 100

It has broad support -- over 100 people have donated significant time (from one afternoon of work up to around 40 hours) and/or money

Patreon

,

one-time donations

, and guest rooms have covered the rent until it was recently raised
Community guests can stay there for relatively low prices for the area

It has been full 75% of the time in May

I’d like you to be involved

Visit and attend events at REACH
Volunteer (see

this doc

for some ways to help)
Host events

Monday, Thursday, and Friday nights are currently available for recurring or one-time events (see

calendar

)

Help bridge our funding gap

Rent has gone up to $6k/month unless we find a new venue
We need to be able to pay a manager

Managing the space takes 10-30 hours per week

Help us find and apply for grants

already planning to re-apply for CEA and BERI grants

Put your own money in the pot if you find the project valuable (see

Patreon

or

Paypal

)
Provide

specific items

to improve the space

Why We Needed a Community Space

What A Physical Space Can Do for Community

Around December 2017, I started thinking that it would be really nice to have a central place where community members could:

Conveniently host events (a function that had been fulfilled by the CFAR office before the switch to badged access)
Cowork with community members during the day
Come for low-key spontaneous socializing
Bring kids to play together
Be a default place to meet up with someone to chat about EA/R things
Crash on a couch or grab a bed and stay for a while

My initial thought was “we should buy a decommissioned church building!” However, I realized that funding such a large purchase would be pretty tricky, and that I could and probably should try out my ideas in a rental first and work toward buying a venue later.

Even Berkeley Can Be Lonely

People often talk about Berkeley as if it’s a magical community hub, but actually many community members feel lonely, depressed and/or anxious (including myself). This has been a theme that has been ...
Beyond Astronomical Waste
by Wei Dai — 2018-06-07T21:04:44.630Z

Faced with the astronomical amount of unclaimed and unused resources in our universe, one's first reaction is probably wonderment and anticipation, but a second reaction may be disappointment that our universe isn't even larger or contains even more resources (such as the ability to support 3^^^3 human lifetimes or perhaps to perform an infinite amount of computation). In a previous post I suggested that the potential amount of astronomical waste in our universe seems small enough that a total utilitarian (or the total utilitarianism part of someone’s moral uncertainty) might reason that since one should have made a deal to trade away power/resources/influence in this universe for power/resources/influence in universes with much larger amounts of available resources, it would be rational to behave as if this deal was actually made. But for various reasons a total utilitarian may not buy that argument, in which case another line of thought is to look for things to care about beyond the potential astronomical waste in our universe, in other words to explore possible sources of expected value that may be much greater than what can be gained by just creating worthwhile lives in this universe.

One example of this is the possibility of escaping, or being deliberately uplifted from, a simulation that we're in, into a much bigger or richer base universe. Or more generally, the possibility of controlling, through our decisions, the outcomes of universes with much greater computational resources than the one we're apparently in. It seems likely that under an assumption such as Tegmark's Mathematical Universe Hypothesis, there are many simulations of our universe running all over the multiverse, including in universes that are much richer than ours in computational resources. If such simulations exist, it also seems likely that we can leave some of them, for example through one of these mechanisms:

Exploiting a flaw in the software or hardware of the computer that is running our simulation (including "natural simulations" where a very large universe happens to contain a simulation of ours without anyone intending this).

Exploiting a flaw in the psychology of agents running the simulation.

Altruism (or other moral/axiological considerations) on the part of the simulators.

Acausal trade.

Other instrumental reasons for the simulators to let out simulated beings, such as w

...
The first AI Safety Camp & onwards
by Remmelt — 2018-06-07T20:13:42.962Z

Co-author: Linda Linsefors

Summary

Last month, 5 teams of up-and-coming researchers gathered to solve concrete problems in AI alignment at our 10-day AI safety research camp in Gran Canaria.

This post describes:

the event format we came up with
our experience & lessons learned in running it in Gran Canaria
how you can contribute to the next camp in Prague on 4-14 October & future editions

The event format

In

February

, we proposed our plans for the AI Safety Camp:

Goals: Efficiently launch aspiring AI safety and strategy researchers into concrete productivity by creating an ‘on-ramp’ for future researchers.

Specifically:

Get people started on and immersed into concrete research work intended to lead to published papers.
Address the bottleneck in AI safety/strategy of few experts being available to train or organize aspiring researchers by efficiently using expert time.
Create a clear path from ‘interested/concerned’ to ‘active researcher’.
Test a new method for bootstrapping talent-constrained research fields.

Note: this does not involve reaching out to or convincing outside researchers – those who are not yet doing work in the field of AI safety – on the imperative of solving alignment problems.

Method: Run an online study group culminating in an intensive in-person research camp. Participants work in teams on tightly-defined research projects on the following topics:

Agent foundations
Machine learning safety
Policy & strategy
Human values

Project ideas are proposed by participants prior to the start of the program. After that, participants split into teams around the most popular research ideas (each participant joins one team, and each team focuses on one research topic).

What the camp isn’t about:

The AI Safety Camp is not about convincing anyone about the importance of AI alignment research. The camp is for people who are already on board with the general ideas, and who want to develop their research skills and/or find like-minded people to collaborate with. Trying to convert people from adjacent research fields is a very different project, which we do not think mixes well with this event.

The AI Safety Camp is not a summer school (unlike the

one coming up this August in Prague

). There are no teachers although teams can correspond with experienced advisors. Participants are expected to have the knowledge needed to do research together. However, it is not required ...