trentmkelly/LessWrong-43k huggingface dataset:
Communicating concepts in value learning
by Manfred — 2015-12-14T03:06:36.193Z

Epistemic status: Trying to air out some thoughts for feedback, we'll see how successfully. May require some machine learning to make sense, and may require my level of ignorance to seem interesting.

Many current proposals for value learning are garden-variety regression (or its close cousin, classification). The agent doing the learning starts out with some model for what human values look like (a utility function over states of the world, or a reward function in a Markov decision process, or an expected utility function over possible actions), and receives training data that tells it the right thing to do in a lot of different situations. And so the agent finds the parameters of the model that minimize some loss function with the data, and Learns Human Values.

All these models of "the right thing to do" I mentioned are called parametric models, because they have some finite template that they update based on the data. Non-parametric models, on the other hand, have to keep a record of the data they've seen - prediction with a non-parametric model often looks like taking some weighted average of nearby known examples (though not always), while a parametric model would (often) fit some curve to the data and predict using that. But we'll get back to this later.

An obvious problem with current proposals is that it's very resource-intensive to communicate a category or concept to the agent. An AI might be able to automatically learn a lot about the world, but if we want to define its preferences, we have to somehow pick out the concept of "good stuff" within the representation of the world learned by the AI. Current proposals for this look like supervised learning, where huge amounts of labeled data are needed to specify "good stuff," and for many proposals I'm concerned that we'll actually end up specifying "stuff that humans can be convinced is good," which is not at all the same. Humans are much better learners than these supervised learning systems - they learn from fewer examples, and have a better grasp of the meaning and structure behind examples. This hints that there are some big improvements to be made in value learning.

This comparison to humans also leads to my vaguer concerns. It seems like the labeled examples are too crucial, and the unlabeled data not crucial enough. We want a value learner to understand concepts based on just a few exampl...
Meetup : San Francisco Meetup: Board Games
by rocurley — 2015-12-14T01:18:24.709Z

Discussion article for the meetup : San Francisco Meetup: Board Games

WHEN:
16 December 2015 06:15:00PM (-0800)

WHERE:
1597 Howard St. San Francisco, CA

THIS IS ON WEDNESDAY!

We'll be meeting to hang out and play games!

You can call 301-458-0764 to be let in.

Discussion article for the meetup : San Francisco Meetup: Board Games
This year's biggest scientific achievements
by Elo — 2015-12-13T05:26:21.822Z

For our solstice event I tried to put together a list of this year’s biggest scientific achievements. They can likely all be looked up with a bit of searching and each one is worthy of a celebration in their own right. But mostly I want to say; we have come a long way this year. And we have a long way to go.

I tried to include science and technology in this list, but really anything world-scale (non-politics or natural disaster) is worthy of celebrating.

Rosetta mission lands on a comet

using young blood to fight old age (rats)

kinghorn human sequencing machines (Sydney relevant)

100,000 genomes project

the world's oldest cave art @ 40,000 years old

tesla battery//released their patents on their electric engines for use by anyone.

Virtual reality (cardboard)

Astronauts growing their own food

Self driving cars

cubesats

Lab grown kidneys successfully implanted into animals

synthetic DNA

Chicken with a reptile face

nearly an altzeimers cure (ultrasound techniques)

DAWN orbits Ceres

Deepdreaming machine learning (and twitch-deepdream)

Prosthetic limbs that transmit feeling back to the user

Autonomous rocket landing pointy end up

Lightsail project

Ion space travel engine

Anti - aging virus injected into the patient 0

Super black substance made

Q-carbon

High temperature superconductor (-70c)

23&me were allowed to open back up

Enchroma colourblindness adjusting glasses

Google releases "Tensor Flow" which whilst its not very good at the moment has the potential to centralize the Deep Learning libraries.

CRISPR's ability to change the germ line.

Deep Dreaming, but also image generation. Faces generated, bedrooms generated and even a toilet in a field. Its clear that within the next few years you will have pictures entirely generated by Neural Nets. (Code: https://github.com/soumith/dcgan.torch).

On the NLP side of deep learning this post, which whilst not using new techniques, sparked a lot of generative work (http://karpathy.github.io/2015/05/21/rnn-effectiveness/). There has also been really interesting work on Question Answering (http://arxiv.org/abs/1506.02075)

Quasipolynomial time algorith

...
Results of a One-Year Longitudinal Study of CFAR Alumni
by Unnamed — 2015-12-12T04:39:46.399Z

By Dan from CFAR

Introduction

When someone comes to a CFAR workshop, and then goes back home, what is different for them one year later? What changes are there to their life, to how they think, to how they act?
CFAR would like to have an answer to this question (as would many other people). One method that we have been using to gather relevant data is a longitudinal study, comparing participants' survey responses from shortly before their workshop with their survey responses approximately one year later. This post summarizes what we have learned thus far, based on data from 135 people who attended workshops from February 2014 to April 2015 and completed both surveys.
The survey questions can be loosely categorized into four broad areas:

Well-being: On the whole, is the participant's life going better than it was before the workshop?

Personality: Have there been changes on personality dimensions which seem likely to be associated with increased rationality?

Behaviors: Have there been increases in rationality-related skills, habits, or other behavioral tendencies?

Productivity: Is the participant working more effectively at their job or other projects?

We chose to measure these four areas because they represent part of what CFAR hopes that its workshops accomplish, they are areas where many workshop participants would like to see changes, and they are relatively tractable to measure on a survey. There are other areas where CFAR would like to have an effect, including people's epistemics and their impact on the world, which were not a focus of this study.
We relied heavily on existing measures which have been validated and used by psychology researchers, especially in the areas of well-being and personality. These measures typically are not a perfect match for what we care about, but we expected them to be sufficiently correlated with what we care about for them to be worth using.
We found significant increases in variables in all 4 areas. A partial summary:
Well-being: increases in happiness and life satisfaction, especially in the work domain (but no significant change in life satisfaction in the social domain)
Personality: increases in general self-efficacy, emotional stability, conscientiousness, and extraversion (but no significant change in growth mindset or openness to experience)
Behaviors: increased rate of acquisition of useful techniques, emotions experie...
Concise Open Problem in Logical Uncertainty
by Scott Garrabrant — 2015-12-12T03:03:35.000Z

The purpose of this post is to provide a short fully mathematically specified conjecture which can be worked on with very little background, but which has an important consequence in logical uncertainty. Not too many man-hours have been put into this question yet, so it is plausible that a MIRIx team could solve this problem.

Let E be a be an envirnoment which is a function from N to {0,1}.

Let M be an algorithm which on input n is given oracle access to E(i) for all i<logn, and which outputs a probability p(n).

Definition: If
limn→∞n∏i=1|p(i)+E(i)−1||E(i)−13|=0,
then M is bad. Similarly, if
limn→∞n∏i=1|p(i)+E(i)−1||E(i)−23|=0,
then M is bad. Otherwise, M is good. (Note that if there is no limit, this does not mean M is bad.)

Conjecture: For every algorithm M, there exists an environment E such that M is bad.

Intuitively, M is slowly seeing bits from E, and much more quickly making predictions about E. If M is bad in the first sense, that means that M it has made much worse predictions than if it just output 2/3. If M is bad in the second sense, that means that M it has made much worse predictions than if it just output 1/3. It seems easy enough to avoid the first failure mode; we just have to switch to output 2/3 if we cross some threshhold where 2/3 has been doing better. Adding the second failure mode makes this strategy stop working, because an evil environment could cause us to lock in to 2/3, and then switch to giving probability 1/3 forever.

We would like an M which can take a countable set of advisors, and do at least as well as all of them, even when there is a delay in the observations. However, I believe that M can't even do at least as well as two constant advisors. The above conjecture implies that M cannot even balance the predictions of two constant advisors, and therefore also cannot balance countably many advisors. Note that a disproof would also be very interesting.
[Link] Introducing OpenAI
by Baughn — 2015-12-11T21:54:47.229Z

From their site:
OpenAI is a non-profit artificial intelligence research company. Our goal is to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return.

The money quote is at the end, literally—$1B in committed funding from some of the usual suspects.
Benford Test with Gaps
by Scott Garrabrant — 2015-12-11T21:40:06.000Z

In this post, I present a new desirable property in Asymptotic Logical Uncertainty. The property is the analog of satisfying the Benford test, but with a collection of undecidable sentences mixed in. I do not know of any algorithm that satisfies this property yet. I give a proof that a similar stronger property is actually impossible to satisfy.

Consider an algorithm M which on input n runs in f(n) time, and outputs a probability interpreted as the probability to assign to ϕn.

We say that a subset of natural numbers S is (p,q)-irreducible pattern if it is possible to quickly determine if n is in S, and the provability/disprovability of elements of S appears (relative to f(n) time) indistinguishable from a coin which outputs "true" with probability p, "false" with probability q, and "undecidable" with probability 1−p−q.

I am purposefully being vague about the exact definition of irreducible pattern here. You could define it similarly to how we defined it before, but I am open to other definitions. It should at least be the case that if the theorem prover was randomly determining whether elements of the sequence are provable, disprovable, or undecidable with the corresponding probabilities, then the resulting sequence should almost surely be a (p,q)-irreducible pattern.

We say that M passes the Benford test with gaps if whenever S is a (p,q)-irreducible pattern, the limit as n∈S goes to infinity of the distance between M(n) and the interval [p,1−q] is 0.

At first, this property may appear weaker than it has to be. One might be tempted to instead require that the probability converges to pp+q. However, this property would be too strong.

To see that this is too strong, consider an environment in which the nth sentence is true with probability 1/3, false with probability 1/3, and otherwise undecidable.

We would need that with probability 1, the probabilities converge to 1/2. In fact, take a random sequence like this, and change finitely many of the undecidable sentences to true, the probabilities would still have to converge to 1/2.

Now, imagine that for the nth sentence, ϕn, you replace it with the sentence "ϕn or M outputs a probability less than 3/5 at least n times." (We assume that ϕ was such that if ϕ is undecidible, and the second condition is false, the new sentence is still undecidable).

If M only outputs probabilities less than 3/5 finitely often, then only fini

...
Weekly LW Meetups
by FrankAdamek — 2015-12-11T16:58:06.295Z

This summary was posted to LW Main on December 4th. The following week's summary is here.

Irregularly scheduled Less Wrong meetups are taking place in:

San Antonio: TAP Discussion: 06 December 2015 02:00PM
San Francisco Meetup: Rationality Diary: 06 December 2015 06:15PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

MIT/Boston Secular Solstice: 11 December 2015 08:00PM
NH Meet-up: 08 December 2015 07:00PM
NYC Solstice: 19 December 2015 05:30PM
Seattle Solstice: 19 December 2015 05:00PM
Sydney Rationality Dojo - December: 06 December 2015 04:00PM
Sydney solstice: 12 December 2015 05:00PM
Tel Aviv Game Night: 08 December 2015 07:00PM
[Vienna] Five Worlds Collide - Vienna: 04 December 2015 08:00PM
Washington, D.C.: Create & Complete: 06 December 2015 03:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Canberra, Columbus, Denver, London, Madison WI, Melbourne, Moscow, Mountain View, New Hampshire, New York, Philadelphia, Research Triangle NC, Seattle, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers and a Slack channel for daily discussion and online meetups on Sunday night US time.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific mailing list in order to be notified when an irre...
Computable Universal Prior
by Ronny Fernandez — 2015-12-11T09:54:24.935Z

Suppose instead of using 2^-K(H) we just use 2^-length(H), does this do something obviously stupid?

Here's what I'm proposing:

Take a programing language with two characters. Assign each program a prior of 2^-length(program). If the program outputs some string, then P(string | program) = 1, else it equals 0. I figure there must be some reason people don't do this already, or else there's a bunch of people doing it. I'd be real happy to find out about either.

Clearly, it isn't a probability distribution, but we can still use it, no?
List of techniques to help you remember names
by Elo — 2015-12-11T00:41:42.149Z

Name are very important. Everyone has one; everyone likes to know when you know their name. Everyone knows them to be a part of social interaction. You can't avoid names (well you can, but it gets tricky). In becoming more awesome at names, here is a bunch of suggestions that can help you.

The following is an incomplete list of some reasonably good techniques to help you remember names. Good luck and put them to good use.

0. Everyone can learn to remember names

in a growth mindset sense, stop thinking you can’t. Stop saying that, everyone is bad at it. Your 0’th task is to actually try harder than that, if you can’t do that - stop reading. Face blindness does exist but most of these will help with that.

1. Decide that names are important.

If you don't think they are important then change your mind. They are. Everyone says they are, everyone responds to their name. It’s a fact of life that being able to be communicated with directly by name will be useful.

2. Make sure you hear the name clearly the first time, and repeat it till you have it.

I tend to shake people's hands, then not let go until they tell me their name, and share them mine clearly (sometimes twice).

3. Repeat their name*

Part of 2, but also – if you repeat it (at least once) you have a higher chance of remembering it. Look them in the eye and say their name. "Nice to meet you Bob". Suddenly your brain got a good picture of their face as well as a good cue as to their name. If you want to supercharge this particular part; “Nice to meet you bob with the hat”, “susan with the glasses”, “john in the dress” works great!

*Repeating a name also has the effect of someone correcting you if you have it wrong. And if you are in a group - allowing other people to learn or remember a name more easily.

4. Associating that name.

Does that name have a meaning as another thing? Mark, Ivy, Jack.

Does that name rhyme with something? Or sound like something? Victoria, IsaBelle, Dusty, Bill, Norris, Jarrod (Jar + Rod), Leopold.

Does someone you already know have that name? Can you make a mental link between this person and the person who's name you already remember. Worst case about being able to remember their name, "oh I have a cousin also called Alexa"-type statements are ha...
Meetup : Less Wrong NH Meetup
by Tuk — 2015-12-10T22:48:18.791Z

Discussion article for the meetup : Less Wrong NH Meetup

WHEN:
05 January 2016 07:00:00PM (-0500)

WHERE:
269 Pearl St Manchester NH 03104

The tenth NH meet-up is Tuesday, 1/5, in Manchester, NH at 7 pm at a private residence. Light refreshments will be provided.

Have you read Rationality: from AI to Zombies, or any of the Sequences on Less Wrong? Maybe you're just a fan of Harry Potter and the Methods of Rationality. Come hang out with us and discuss optimization of whatever it is you want to optimize.

Agenda: New Year's Resolutions

You may want to bring a notebook.

https://www.facebook.com/events/712319028904397/

https://www.facebook.com/groups/695201067251306/

Discussion article for the meetup : Less Wrong NH Meetup
Meetup : Moscow social meetup: Chaos is back!
by Alexander230 — 2015-12-10T18:31:23.571Z

Discussion article for the meetup : Moscow social meetup: Chaos is back!

WHEN:
27 December 2015 02:00:00PM (+0300)

WHERE:
Russia, Moscow, Strelbishensky pereulok, 10

Welcome to the social meetup! Main theme of this meetup is party games; and some of them are party versions of our rationality training games.

The games planned for this meetup:

Tower of Chaos: https://drive.google.com/file/d/0BzyKVqP6n3hKT1BBLUhNLXZScFE/view

FallacyParty: https://drive.google.com/file/d/0BzyKVqP6n3hKdXh5UmpxeExxRlU/view

Training game: funny and slightly rational :-)

Secret quests: a source of chaos for any meetup!

Zendo: "rational" table game about guessing the rule

Twister: classic party game

Address of meetup:

Strelbishensky per., 10, ap.60, 3rd porch, code 60B3112, 5th floor. Nearest metro station: Vystavochnaya. If you have questions, call me 8-905-527-30-82 or write me on e-mail alexander230r@gmail.com (it's better to call if you're searching the way before the meetup).

Discussion article for the meetup : Moscow social meetup: Chaos is back!
Meetup : FallacyMania game in Kocherga club
by Alexander230 — 2015-12-10T17:47:38.762Z

Discussion article for the meetup : FallacyMania game in Kocherga club

WHEN:
23 December 2015 07:40:00PM (+0300)

WHERE:
Moscow, B.Dorogomilovskaya, 5-2

Welcome to FallacyMania: it is a game where you guess logical fallacies in arguments, or practise using logical fallacies yourself (depending on team in which you will be).

Details about the game are here: https://lesswrong-ru.hackpad.com/Fallacymania--neGfMe9MFjH

Come to antikafe "Kocherga", ul.B.Dorogomilovskaya, 5-2. The map is here: http://kocherga-club.ru/#contacts . Nearest metro station is Kievskaya. If you are lost, call Sasha at +7-905-527-30-82.

The game begins at 19:40, the length is 1-1.5 hours.

Discussion article for the meetup : FallacyMania game in Kocherga club
Existence of distributions that are expectation-reflective and know it
by TsviBT — 2015-12-10T07:35:57.000Z

We prove the existence of a probability distribution over a theory T
with the property that for certain definable quantities φ, the
expectation of the value of a function
E[┌φ┐] is accurate, i.e. it equals the
actual expectation of φ; and with the property that it assigns
probability 1 to E behaving this way. This may be useful for
self-verification, by allowing an agent to satisfy a reflective
consistency property and at the same time believe itself or similar
agents to satisfy the same property. Thanks to Sam Eisenstat for
listening to an earlier version of this proof, and pointing out a
significant gap in the argument. The proof presented here has not been
vetted yet.

Problem statement

Given a distribution P coherent over a theory A, and some
real-valued function f on completions of A, we can define the
expectation E[f] of f according to P. Then
we can relax the probabilistic reflection principle by asking that for
some class of functions f, we have that
E[┌E[f]┐]=E[f], where E is a symbol in the language of A meant
to represent E. Note that this notion of
expectation-reflection is weaker than probabilistic reflection, since
our distribution is now permitted to, for example, assign a bunch of
probability mass to over- and under-estimates of E[f], as long as
they balance out.

Christiano asked whether it is possible to have a distribution that
satisfies this reflection principle, and also assigns probability 1 to
the statement that E satisfies this reflection principle. This was not
possible for strong probabilistic reflection, but it turns out to be
possible for expectation reflection, for some choice of the functions
f.

Sketch of the approach

(This is a high level description of what we are doing, so many concepts
will be left vague until later.)

Christiano et
al. applied
Kakutani’s theorem to the space of coherent P. Instead we
will work in the space of expectations over some theory T, where an
expectation over a theory is, roughly speaking, a function from the set
of variables provably defined by that theory, into the intervals proved
to bound each variable. These are essentially interchangeable with
coherent probability distributions over T. The point of doing this
is to make the language simpler, for example reflection statements will
mention a single symbol representing an expectation, rather than a
complicated formula defining the expectation in

...