trentmkelly/LessWrong-43k huggingface dataset:
Just Saying What You Mean is Impossible
by Zvi — 2017-06-28T00:12:28.000Z

Response to: Conversation Deliberately Skirts the Border of Incomprehensibility (SlateStarCodex)

“Why don’t people just say what they mean?”

They can’t.

I don’t mean they don’t want to. I don’t mean that they choose not to. I don’t even mean they have been socially conditioned not to. I mean they literally have no way to do the thing.

I don’t even mean that they don’t know how in some sense, that they have not realized the possibility. It is literally impossible to “just” say what you mean. With work, you can mostly just say what you mean, but even that is hard.

This is because humans are not automatically strategic or automatically scientific but by default humans are automatically doing lightning fast implicit probabilistic analysis on social information in the background of every moment of their lives. Most of them would not describe it that way, but that is a thing that is going on.

So let’s take the example that starts the quote in Scott’s article: Bob is watching a television show and Alice asks what he is watching.

My experience says that most of the time, this primarily really is a request for information. Alice wants to know what show Bob is watching.

It also raises the probability that Alice will want to watch the show with Bob, conditional on what show Bob is watching; there is even a chance that Alice knows what show it is, and is using the question to open the conversation for this purpose. Certainly, this would be a reasonable thing to do, were Alice potentially interested in watching with Bob. Alice could also simply be making conversation, and want something else entirely. For all we know, she just wants to make a new friend, or plans to distract him and stab him with a knife.

Alice is providing at least very weak evidence in favor of those possibilities, and many others, while providing evidence against everything else.

That doesn’t mean Alice wanted to send all of those messages, but whichever one Alice had in mind, there was no way for her to send only that message – any other thing she could have said would also have had lots of likelihood ratios attached to it.

Then, in turn, Bob’s response does all of these same things. Telling Alice what the show is gives her that information, which may or may not have been what she wanted. It also indicated th...
Humans are not agents: short vs long term
by Stuart_Armstrong — 2017-06-27T13:04:36.000Z

A putative new idea for AI control; index here.

This is an example of humans not being (idealised) agents.

Imagine a human who has a preference to not live beyond a hundred years. However, they want to live to next year, and it's predictable that every year they are alive, they will have the same desire to survive till the next year.

This human (not a completely implausible example, I hope!) has a contradiction between their long and short term preferences. So which is accurate? It seems we could resolve these preferences in favour of the short term ("live forever") or the long term ("die after a century") preferences.

Now, at this point, maybe we could appeal to meta-preferences - what would the human themselves want, if they could choose? But often these meta-preferences are un- or under-formed, and can be influenced by how the question or debate is framed.

Specifically, suppose we are scheduling this human's agenda. We have the choice of making them meet one of two philosophers (not meeting anyone is not an option). If they meet Professor R. T. Long, he will advise them to follow long term preferences. If instead, they meet Paul Kurtz, he will advise them to pay attention their short term preferences. Whichever one they meet, they will argue for a while and will then settle on the recommended preference resolution. And then they will not change that, whoever they meet subsequently.

Since we are doing the scheduling, we effectively control the human's meta-preferences on this issue. What should we do? And what principles should we use to do so? We are trying to maximise human preferences, but we can also control what they are (and have to control what they are, though our choice of which philosopher they meet first).

It's clear that this can apply to AIs: if they are simultaneously aiding humans as well as learning their preferences, they will have multiple opportunities to do this sort of preference-shaping.
a different perspecive on physics
by Florian_Dietz — 2017-06-26T22:47:03.089Z

(Note: this is anywhere between crackpot and inspiring, based on the people I talked to before. I am not a physicist.)

I have been thinking about a model of physics that is fundamentally different from the ones I have been taught in school and university. It is not a theory, because it does not make predictions. It is a different way of looking at things. I have found that this made a lot of things we normally consider weird a lot easier to understand.

Almost every model of physics I have read of so far is based on the idea that reality consists of stuff inside a coordinate system, and the only question is the dimensionality of the coordinate system. Relativity talks about bending space, but it still treats the existence of space as the norm. But what if there were no dimensions at all?

Rationale

If we assume that the universe is computable, then dimension-based physics, while humanly intuitive, are unnecessarily complicated. To simulate dimension-based physics, one first needs to define real numbers, which is complicated, and requires that numbers be stored with practically infinite precision. Occam's Razor argues against this.

A graph model in contrast would be extremely simple from a computational point of view: a set of nodes, each with a fixed number of attributes, plus a set of connections between the nodes, suffices to express the state of the universe. Most importantly, it would suffice for the attributes of nodes to be simple booleans or natural numbers, which are much easier to compute than real numbers. Additionally, transition functions to advance in time would be easy to define as well as they could just take the form of a set of if-then rules that are applied to each node in turn. (these transition functions roughly correspond to physical laws in more traditional physical theories)

Idea

Model reality as a graph structure. That is to say, reality at a point of time is a set of nodes, a set of connections between those nodes, and a set of attributes for each node. There are rules for evolving this graph over time that might be as simple as those in Conway's game of life, but they lead to very complex results due to the complicated structure of the graph.

Connections between nodes can be created or deleted over time according to transition functions.

What we call particles are actually patterns of attributes on clusters of nodes. These patterns evolve over...
Self-modification as a game theory problem
by cousin_it — 2017-06-26T20:47:54.080Z

In this post I'll try to show a surprising link between two research topics on LW: game-theoretic cooperation between AIs (quining, Loebian cooperation, modal combat, etc) and stable self-modification of AIs (tiling agents, Loebian obstacle, etc).

When you're trying to cooperate with another AI, you need to ensure that its action will fulfill your utility function. And when doing self-modification, you also need to ensure that the successor AI will fulfill your utility function. In both cases, naive utility maximization doesn't work, because you can't fully understand another agent that's as powerful and complex as you. That's a familiar difficulty in game theory, and in self-modification it's known as the Loebian obstacle (fully understandable successors become weaker and weaker).

In general, any AI will be faced with two kinds of situations. In "single player" situations, you're faced with a choice like eating chocolate or not, where you can figure out the outcome of each action. (Most situations covered by UDT are also "single player", involving identical copies of yourself.) Whereas in "multiplayer" situations your action gets combined with the actions of other agents to determine the outcome. Both cooperation and self-modification are "multiplayer" situations, and are hard for the same reason. When someone proposes a self-modification to you, you might as well evaluate it with the same code that you use for game theory contests.

If I'm right, then any good theory for cooperation between AIs will also double as a theory of stable self-modification for a single AI. That means neither problem can be much easier than the other, and in particular self-modification won't be a special case of utility maximization, as some people seem to hope. But on the plus side, we need to solve one problem instead of two, so creating FAI becomes a little bit easier.

The idea came to me while working on this mathy post on IAFF, which translates some game theory ideas into the self-modification world. For example, Loebian cooperation (from the game theory world) might lead to a solution for the Loebian obstacle (from the self-modification world) - two LW ideas with the same name that people didn't think to combine before!
Loebian cooperation in the tiling agents problem
by cousin_it — 2017-06-26T14:52:54.000Z

The tiling agents problem is about formalizing how AIs can create successor AIs that are at least as smart. Here's a toy model I came up with, which is similar to Benya's old model but simpler. A computer program X is asked one of two questions:

Would you like some chocolate?

Here's the source code of another program Y. Do you accept it as your successor?

Program X will be called a "winner" if is says "yes" to chocolate, every program Y accepted as successor by X also says "yes" to chocolate, every program Z accepted as successor by Y also says "yes" to chocolate, and so on. Otherwise X will be called a "loser".

Let's also assume that X has access to a provability oracle for Peano arithmetic (PA), to avoid dealing with logical uncertainty.

Here's some simple winning programs:

Always say "yes" to chocolate. Refuse to accept any successor.

Always say "yes" to chocolate. Accept Y as successor iff Y has the same source code as X.

Always say "yes" to chocolate. Accept Y as successor iff PA proves that Y is a winner.

Unfortunately (1) is too restrictive, (2) is too sensitive to implementation details, and (3) refuses to accept itself as successor for Goedelian reasons. How do we write a more general winning program?

Note that all programs above are analogous to strategies in the Prisoner's Dilemma with visible source code: (1) is defection, (2) is quining cooperation, and (3) is simulating the opponent. But in the PD we also have Loebian cooperation! We could try applying it to our problem naively, like this:

Always say "yes" to chocolate. Accept Y as successor iff PA proves that "X is a winner" implies "Y is a winner".

At first glance that program looks nice. It accepts itself as successor, and also accepts all programs that PA proves to be winners. Unfortunately it also accepts all losers, because the sentence "X accepts Y" becomes provable for Loebian reasons.

But not all is lost! Here's a better way to apply Loebian cooperation to our problem:

Always say "yes" to chocolate. Accept Y as successor iff PA proves that Y can't lose in fewer steps than X.

(More precisely, for all n such that X can't lose in n steps, Y also can't lose in n steps.)

Proof that X is a winner: X can't lose in one step by construction. Therefore Y can't lose in one step, as long as PA is sound. Therefore X can't lose in two steps. Therefore Y can't lose in two steps, as long

...
Open thread, June 26 - July 2, 2017
by Thomas — 2017-06-26T06:12:35.196Z

If it's worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "Make this post available under..." before submitting
A Call for More Policy Analysis
by madhatter — 2017-06-25T14:24:25.100Z

I would like to see more concrete discussion and analysis of AI policy in the EA community, and on this forum in particular.

AI policy would broadly encompass all relevant actors meaningfully influencing the future and impact of AI, which would likely be governments, research labs and institutes, and international organizations.

Some initial thoughts and questions I have on this topic:

1) How do we ensure all research groups with a likely chance of developing AGI know and care about the relevant work in AI safety (which hopefully is satisfactorily worked out by then)?

Some possibilities: trying to make AI safety a common feature of computer science curricula, general community building and more AI safety conferences, more popular culture conveying non-terminatoresque illustrations of the risk.

2) What strategies might be available for laggards in a race scenario to retard progress of leading groups, or to steal their research?

Some possibilities in no particular order: espionage, malware, financial or political pressures, power outages, surveillance of researchers.

3) Will there be clear warning signs?

Not just in general AI progress, but locally near the leading lab. Observable changes in stock price, electricity output, etc.

4) Openness or secrecy?

Thankfully the Future of Life Institute is working on this one. As I understand the consensus is that openness is advisable now, but secrecy may be necessary later. So what mechanisms are available to keep research private?

5) How many players will there be with a significant chance of developing AGI? Which players?

6) Is an arms race scenario likely?

7) What is the most likely speed of takeoff?

8) When and where will AGI be developed?

Personally, I believe the use of forecasting tournaments to get a better sense of when and where AGI will arrive would be a very worthwhile use of our time and resources. After reading Superforecasting by Dan Gardner and Phillip Tetlock I was struck by how effective these tournaments are at singling out those with low Brier scores and using them to get a better-t

...
Effective Altruism : An idea repository
by Onemorenickname — 2017-06-25T00:56:47.575Z

Metainformations :

Cross-post. Steemit.

Epistemic Effort. Much more reasoning behind that post. I'm mostly trying to see if people are interested. If they are, much more writing will ensue.

Epistemic Status. Field : Development of idea repository for a community. Phase : pre-epistemy.

Thesis. There should be an EA ideas repository.

Auxiliary thesis. EA looks more closed than it is.

Personal Introduction

I came to define myself as a non-standard Effective Altruist. I’ve always been interested in Effective Altruism, way before I’ve even heard of EA. When I was younger, I simply thought I was altruist, and that what people did was … noise at best. Basically, naive ways to relieve one’s conscience and perpetuate one’s culture.

Since primary school I thought about global problems and solutions to these problems. So much so that the word “project” internally connotes “project solving some global problems”. As such, EA should have interested me.

However, it didn’t. The main reason was that I saw EA as some other charitists. I’ve always been skeptical toward charity, the reason being “They think too small” and “There are too much funding in standard solutions rather than in finding new ones”.

I think this exemplifies a problem about EA’s communication.

A Communication Problem

Most people I know got to know Effective Altruism through EffectiveAltruism.org.

Because of that website, these people see EA as a closed organization that help people to direct funds to better charities and find better careers.

That was my opinion of EA until I saw the grant offer : a closed organization with already defined solutions wouldn’t fund new ideas. As such, I changed my outlook of EA. I researched a bit more about it, and found an open and diverse community.

But I am busy person, therefore I have to use filters before putting more time in researching about something. I made my impression from :

effectivealtruism.org

The Wikipedia entry. Particularly the Cause Priorities section.

People loosely related to EA. From LessWrong, SlateStarCodex and outside rationalist circles, I asked them (with open-ended questions) about EA, but they confirmed my first impression.

What convinced me of that impression was the website’s content :

The tabs are “About, Blog, Donate, Effec

...
On Dragon Army
by Zvi — 2017-06-24T10:44:36.000Z

Analysis Of: Dragon Army: Theory & Charter (30 Minute Read)

Epistemic Status: Varies all over the map from point to point

Length Status: In theory I suppose it could be longer

This is a long post is long post responding to a almost as long (and several weeks and several controversy cycles old, because life comes at you fast on the internet) post, which includes extensive quoting from the original post and assumes you have already read the original. If you are not interested in a very long analysis of another person’s proposal for a rationalist group house, given that life is short, you can (and probably should) safely skip this one.

Dragon Army is a crazy idea that just might work. It probably won’t, but it might. It might work because it believes in something that has not been tried, and there is a chance that those involved will actually try the thing and see what happens.

Scott made an observation that the responses to the Dragon Army proposal on Less Wrong were mostly constructive criticism, while the responses on Tumblr were mostly expressions of horror. That is exactly the response you would expect from a project with real risks, but also real potential benefits worth taking the risks to get. This updates me strongly in favor of going forward with the project.

As one would expect, the idea as laid out in the charter is far from perfect. There are many modifications that need to be made, both that one could foresee in advance, and that one could not foresee in advance.

My approach is going to be to go through the post and comment on the components of the proposal, then pull back and look at the bigger picture.

In part 1, Duncan makes arguments, then later in part 2 he says the following:

Ultimately, though, what matters is not the problems and solutions themselves so much as the light they shine on my aesthetics (since, in the actual house, it’s those aesthetics that will be used to resolve epistemic gridlock). In other words, it’s not so much those arguments as it is the fact that Duncan finds those arguments compelling.

I agree in one particular case that this is the important (and worrisome) thing, but mostly I disagree and think that we should be engaging with the arguments themselves. This could be because I am as interested in learning about and discussing general things using the proposal as a taking-off point, as I am in the...
Bi-Weekly Rational Feed
by sapphire — 2017-06-24T00:07:03.768Z

===Highly Recommended Articles:

Introducing The Ea Involvement Guide by The Center for Effective Altruism (EA forum) - A huge list of concrete actions you can take to get involved. Every action has a brief description and a link to an article. Each article rates the action on time commitment, duration, familiarity and occupation. Very well put together.

Deep Reinforcement Learning from Human Preferences - An algorithm learns to backflip with 900 bits of feedback from the human evaluator. "One step towards building safe AI systems is to remove the need for humans to write goal functions, since using a simple proxy for a complex goal, or getting the complex goal a bit wrong, can lead to undesirable and even dangerous behavior. In collaboration with DeepMind’s safety team, we’ve developed an algorithm which can infer what humans want by being told which of two proposed behaviors is better."

Build Baby Build by Bryan Caplan - Quote from a paper estimating the high costs of housing restrictions. We should blame the government, especially local government. The top alternate theory is wrong. Which regulations are doing the damage? It's complicated. Functionalists are wrong. State government is our best hope.

The Use And Abuse Of Witchdoctors For Life by Lou (sam[]zdat) - Anti-bullet magic and collective self-defense. Cultural evolution. People don't directly believe in anti-bullet magic, they believe in elders and witch doctors. Seeing like a State. Individual psychology is the foundation. Many psychologically important customs couldn't adapt to the marketplace.

S-risks: Why They Are The Worst Existential Risks by Kaj Sojata (lesswrong) - “S-risk – One where an adverse outcome would bring about severe suffering on a cosmic scale, vastly exceeding all suffering that has existed on Earth so far.” Why we should focus on S-risk. Probability: Artificial sentience, Lack of communication, badly aligned Ai and competitive pressures. Tractability: Relationship with x-risk. Going meta, cooperation. Neglectedness: little attention, people conflate x-risk = s-risk.

Projects Id Like To See by William MacAskill (EA forum) - CEA is giving out £100K grants. General types of applications. EA outreach and Community, Anti-Debates, Prediction Tournaments, Shark Tank Discussions, Research Groups, Specific Skill Building,...
Idea for LessWrong: Video Tutoring
by Adam Zerner — 2017-06-23T21:40:50.118Z

Update 7/9/17: I propose that Learners individually reach out to Teachers, and set up meetings. It seems like the most practical way of getting started, but I am not sure and am definitely open to other ideas. Other notes:

There seems to be agreement that the best way to do this is individualized guidance, rather than lectures and curriculums. Eg. the Teacher "debugging" the Learner. Assuming that approach, it is probably best for the amount of Learners in a session to be small.

Consider that it may make sense for you to act as a Teacher, even if you don't have a super strong grasp of the topic. For example, I know a decent amount about computer science, but don't have a super strong grasp of it. Still, I believe it would be valuable for me to teach computer science to others. I can definitely offer value to people with no CS background. And for people who do have a CS background, there could be value in us taking turns teaching/learning, and debugging each other.

We may not be perfect at this in the beginning, but let's dive in and see what we can do! I think it'd be a good idea to comment on this post with what did/didn't work for you, so we as a group could learn and improve.

I pinned http://lesswrong.com/r/discussion/lw/p69/idea_for_lesswrong_video_tutoring/ to #productivity on the LessWrongers Slack group.

Update 6/28/17: With 14 people currently interested, it does seem that there's enough to get started. However, I'd like to give it a bit more time and see how much overall interest we get.

Idea: we coordinate to teach each other things via video chat.

We (mostly) all like learning. Whether it be for fun, curiosity, a stepping stone towards our goals.

My intuition is that there's a lot of us who also enjoy teaching. I do, personally.

Enjoyment aside, teaching is a good way of solidifying ones knowledge.

Perhaps there would be positive unintended consequences. Eg. socially.

Why video? a) I assume that medium is better for education than simply text. b) Social and motivational benefits, maybe. A downside to video is that some may find it intimidating.

It may be nice to evolve this into a group project where we iteratively figure out how to do a really good job teaching certain topics.

I see the main value in personalization, as opposed to passive lectures/seminars. Those already exist, and are plentiful for most topics. What isn't easily

...