trentmkelly/LessWrong-43k huggingface dataset:
Three preference frameworks for goal-directed agents
by jessicata — 2015-12-02T00:06:15.000Z

In the previous post, I discussed
three different applications of a value learning model. In this post, I will go
into more detail about value learning for goal-directed agents by discussing
three possible preference
frameworks for goal-directed
agents.

These three proposals probably do not cover all useful goal-directed AI preference
frameworks; they should be taken as three points in a larger design space. Still,
looking at these three should be instructive for coming up with problems that
apply across different AGI architectures. I plan to write an additional post
for each of these three proposals.

Value-learning sovereign

One proposal is to create an agent that infers human values and optimizes for
them in expectation. It does a mixture of gaining information about human
values and optimizing for them, depending on the value of additional information
about human values. The main existing proposal that falls into this category is
the original coherent extrapolated
volition idea.

Depending on the setup, an agent like this might learn either "more terminal" or
"more instrumental" preferences. This is similar to the distinction between
short-distance, medium-distance, and long-distance extrapolation in the coherent
extrapolated volition document. If the agent is set to learn and optimize for
terminal preferences (the kind discussed in the Arbital article on
"value"), then this agent is
quite brittle: if it learns the wrong preferences, then it will
attempt to optimize the universe for these wrong preferences, possibly with
catastrophic results. On the other
hand, as Paul argues,
it looks more tractable for an agent to infer instrumental preferences
instead. I hope to clarify the difference between learning terminal and
instrumental preferences in a future post.

It is my opinion (and the opinion of everyone at MIRI I have talked to about this)
that the process of constructing a value-learning sovereign is probably too error-prone
for this to be the first advanced AGI we build. This opinion may change if new
information shows that this setup is unexpectedly easy compared to the alternatives.

Reduced-impact genie for concrete tasks

There are many somewhat concrete physical tasks that would be useful to have an
AI accomplish. For example, an AI that could efficiently and safely create food
would be quite useful. AIs that accomplish these concrete tasks a

...
Why effective altruists should do Charity Science’s Christmas fundraiser
by tog — 2015-12-01T23:59:14.688Z

"Maybe Christmas", he thought, "doesn't come from a store."

"Maybe Christmas... perhaps... means a little bit more!"

-The Grinch, How the Grinch Stole Christmas! (p.29)

The Donate Your Christmas fundraiser is simple–instead of Christmas gifts, ask for donations to your preferred GiveWell-recommended charity. Of course, you can take it further than that if you’d like and ask colleagues or your social network.

You should Donate Your Christmas because by using the available resources with relatively small time commitments, it’s likely you’ll raise counterfactual funds for your preferred GiveWell recommended charity. In addition, it’s worth considering because it’s an opportunity to potentially spread ideas relating to effective altruism.

It’s likely to raise counterfactual funds for your preferred GiveWell recommended charity

First, I will outline why it’s likely you’ll raise money by briefly looking at seasonal trends, donor motivations, and results from other peer-to-peer campaigns, and then briefly state why it seems likely that part of the funds raised will be counterfactual.

Much evidence suggests people are more likely to give in December. For instance, one survey reported that 40% said they’re more likely to give during the holiday season than would be for the rest of the year. Network For Good’s Digital Giving Index also reports that 31% of annual giving occurred in December.

In addition, evidence on donor motivations indicates that peer-to-peer fundraisers may be successful. For instance, academic research suggests that social ties play a strong causal role in the decision to donate and increases average gift size as well. Complementary to this influence are the many people who give to those who ask. One survey recorded 20% of respondents saying that they simply donate to the charities that ask them.

Moreover, available data indicates that peer-to-peer fundraising pages regularly raise hundreds of dollars. One peer-to-peer fundraising platform reports that the average fundraising page raises $568 and this blog post reports Charity Water’s average peer-to-peer fundraiser totals at $770. These figures match well with Charity Science’s experience from last year’s Christmas Fundraiser which raised an average of $750 CAD per peer-to-peer fundraising page and a medi

...
Too late for the easy solutions
by KatjaGrace — 2015-12-01T23:00:19.000Z

The other day I took a day off, to do whatever I felt like. One appealing idea early in the day was to solve a major problem that has been bothering me for as long as I remember. I won’t go into details, but it seemed like maybe if I just sat down and tried to solve it, I might.

Then I realized that I didn’t really feel like solving it. If I could just sit down and solve it, then why hadn’t I already solved it? What would I say happened, even to myself? Either I had so far failed to do an easy thing—at great cost— or a legitimately hard problem had been magically and unintelligibly solved. Either I would be a bad person, or reality would not make sense. Neither of these seemed appealing at all. Better to suffer a little longer, and solve the problem some hard way, without blame or confusion.

When I noticed this, I realized it was actually a familiar pattern. It would be disconcerting to just casually decide to not have the problem any more, without some more meaningful banishment ritual.

This relates to forgetting to solve problems because you assume that you have tried. Instead of assuming that you have tried because you are reasonable, you implicitly assume you have tried because you are too scared of learning that you didn’t try or that your trying was somehow wrong.

This is related to the general mistake where a task feels like it is meant to take a long time, so you dutifully spend a long time on it. For instance, you think you might be curious about this philosophical problem, and so you talk to your advisor about your curiosity, and arrange to get some funding to pursue it for a little while, and make a schedule, and think about how to go about thinking about it, and try to go to more conferences on the topic. When, if you had just immediately sat down and tried to answer the question, you might have just answered it. This is something that ‘rationality techniques’ such as setting a five minute timer for solving a problem, or trying to solve a problem assuming it is really easy, are meant to eradicate.

It also reminds me a bit of people who sacrifice a lot for some goal, and then are faced with the prospect of everyone achieving the same ends without sacrifice, and are reluctant to accept that fate.

I’m not sure what to do about this, but for now I’m trying out just not making this error, without any fancy debugging.
Stupid Questions, December 2015
by polymathwannabe — 2015-12-01T22:40:30.786Z

This thread is for asking any questions that might seem obvious, tangential, silly or what-have-you. Don't be shy, everyone has holes in their knowledge, though the fewer and the smaller we can make them, the better.

Please be respectful of other people's admitting ignorance and don't mock them for it, as they're doing a noble thing.
Group Rationality Diary, Dec. 1-13, 2015
by polymathwannabe — 2015-12-01T22:39:15.023Z

This is the public group rationality diary for December 1-13, 2015. It's a place to record and chat about it if you have done, or are actively doing, things like:

Established a useful new habit

Obtained new evidence that made you change your mind about some belief

Decided to behave in a different way in some set of situations

Optimized some part of a common routine or cached behavior

Consciously changed your emotions or affect with respect to something

Consciously pursued new valuable information about something that could make a big difference in your life

Learned something new about your beliefs, behavior, or life that surprised you

Tried doing any of the above and failed

Or anything else interesting which you want to share, so that other people can think about it, and perhaps be inspired to take action themselves. Try to include enough details so that everyone can use each other's experiences to learn about what tends to work out, and what doesn't tend to work out.
Crazy Ideas Thread, December 2015
by polymathwannabe — 2015-12-01T22:37:37.931Z

This thread is intended to provide a space for 'crazy' ideas. Ideas that spontaneously come to mind (and feel great), ideas you long wanted to tell but never found the place and time for and also for ideas you think should be obvious and simple - but nobody ever mentions them.

Rules for this thread:

Each crazy idea goes into its own top level comment and may be commented there.

Voting should be based primarily on how original the idea is.

Meta discussion of the thread should go to the top level comment intended for that purpose.
Bragging thread, December 2015
by polymathwannabe — 2015-12-01T22:35:25.519Z

Your job, should you choose to accept it, is to comment on this thread explaining the most awesome thing you've done this month. You may be as blatantly proud of yourself as you feel. You may unabashedly consider yourself the coolest freaking person ever because of that awesome thing you're dying to tell everyone about. This is the place to do just that.

Remember, however, that this isn't any kind of progress thread. Nor is it any kind of proposal thread. This thread is solely for people to talk about the awesome things they have done. Not "will do". Not "are working on". Have already done. This is to cultivate an environment of object level productivity rather than meta-productivity methods.

So, what's the coolest thing you've done this month?

(Previous bragging thread)
December 2015 Media Thread
by ArisKatsaris — 2015-12-01T21:35:41.646Z

This is the monthly thread for posting media of various types that you've found that you enjoy. Post what you're reading, listening to, watching, and your opinion of it. Post recommendations to blogs. Post whatever media you feel like discussing! To see previous recommendations, check out the older threads.

Rules:

Please avoid downvoting recommendations just because you don't personally like the recommended material; remember that liking is a two-place word. If you can point out a specific flaw in a person's recommendation, consider posting a comment to that effect.

If you want to post something that (you know) has been recommended before, but have another recommendation to add, please link to the original, so that the reader has both recommendations.

Please post only under one of the already created subthreads, and never directly under the parent media thread.

Use the "Other Media" thread if you believe the piece of media you want to discuss doesn't fit under any of the established categories.

Use the "Meta" thread if you want to discuss about the monthly media thread itself (e.g. to propose adding/removing/splitting/merging subthreads, or to discuss the type of content properly belonging to each subthread) or for any other question or issue you may have about the thread or the rules.
Take the EA survey, help the EA movement grow and potentially win $250 to your favorite charity
by Peter Wildeford — 2015-12-01T17:07:56.335Z

This year's EA Survey is now ready to be shared! This is a survey of all EAs to learn about the movement and how it can improve. The data collected in the survey is used to help EA groups improve and grow EA. Data is also used to populate the map of EAs, create new EA meetup groups, and create EA Profiles and the EA Donation Registry.

If you are an EA or otherwise familiar with the community, we hope you will take it using this link. All results will be anonymised and made publicly available to members of the EA community. As an added bonus, one random survey taker will be selected to win a $250 donation to their favorite charity.

Take the EA Survey

Please share the survey with others who might be interested using this link rather than the one above: http://bit.ly/1OqsVWo

-

(Cross-posted on Discussion from Main by popular request.)
Meetup : San Antonio: TAP Discussion
by Bryan-san — 2015-12-01T03:15:19.825Z

Discussion article for the meetup : San Antonio: TAP Discussion

WHEN:
06 December 2015 02:00:00PM (-0600)

WHERE:
12651 Vance Jackson Rd #118, San Antonio, TX 78230

Bubble tea, frozen yogurt, and discussion at Yumi Berry! All are welcome! New Meetup to discuss rationality and all things LessWrong and meet the local community. Look for the sign that says Less Wrong!

Discussion article for the meetup : San Antonio: TAP Discussion
Take the EA survey, help the EA movement grow and potentially win $250 to your favorite charity
by Peter Wildeford — 2015-12-01T01:56:20.255Z

This year's EA Survey is now ready to be shared! This is a survey of all EAs to learn about the movement and how it can improve. The data collected in the survey is used to help EA groups improve and grow EA. Data is also used to populate the map of EAs, create new EA meetup groups, and create EA Profiles and the EA Donation Registry.

If you are an EA or otherwise familiar with the community, we hope you will take it using this link. All results will be anonymised and made publicly available to members of the EA community. As an added bonus, one random survey taker will be selected to win a $250 donation to their favorite charity.

Take the EA Survey

Please share the survey with others who might be interested using this link rather than the one above: http://bit.ly/1OqsVWo
Weirdness at the wiki
by NancyLebovitz — 2015-11-30T23:37:44.283Z

Richard Kennaway has posted about an edit war on the wiki. Richard, thank you.

Unfortunately, I've only used the wiki a little, and don't have a feeling for why the edit history for an article is inaccessible. Is the wiki broken or has someone found a way to hack it? Let it be known that hacking the wiki is something I'll ban for.

VoiceofRa, I'd like to know why you deleted Gleb's article. Presumably you have some reason for why you think it was unsatisfactory.

I'm also notifying tech in the hope of finding out what happened to the edit history.
[link] Pedro Domingos: "The Master Algorithm"
by GMHowe — 2015-11-30T22:28:48.575Z

Interesting talk outlining five different approaches to AI.

https://www.youtube.com/watch?v=B8J4uefCQMc

Blurb from the YouTube description:

Machine learning is the automation of discovery, and it is responsible for making our smartphones work, helping Netflix suggest movies for us to watch, and getting presidents elected. But there is a push to use machine learning to do even more—to cure cancer and AIDS and possibly solve every problem humanity has. Domingos is at the very forefront of the search for the Master Algorithm, a universal learner capable of deriving all knowledge—past, present and future—from data. In this book, he lifts the veil on the usually secretive machine learning industry and details the quest for the Master Algorithm, along with the revolutionary implications such a discovery will have on our society.
Pedro Domingos is a Professor of Computer Science and Engineering at the University of Washington, and he is the cofounder of the International Machine Learning Society.
Meetup : San Francisco Meetup: Group Debugging
by rocurley — 2015-11-30T17:29:21.977Z

Discussion article for the meetup : San Francisco Meetup: Group Debugging

WHEN:
30 November 2015 06:15:00PM (-0800)

WHERE:
1597 Howard St. San Francisco, CA

We'll be meeting to try and solve each others problems! As usual, call 301-458-0764 to be let in.

Discussion article for the meetup : San Francisco Meetup: Group Debugging
Open thread, Nov. 30 - Dec. 06, 2015
by MrMind — 2015-11-30T08:05:47.654Z

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.
The value of ambiguous speech
by KevinGrant_duplicate0.2409764628391713 — 2015-11-30T07:58:10.338Z

This was going to be a reply in a discussion between ChristianKl and MattG in another thread about conlangs, but their discussion seemed to have enough significance, independent of the original topic, to deserve a thread of its own. If I'm doing this correctly (this sentence is an after-the-fact update), then you should be able to link to the original comments that inspired this thread here: http://lesswrong.com/r/discussion/lw/n0h/linguistic_mechanisms_for_less_wrong_cognition/cxb2
Is a lack of ambiguity necessary for clear thinking? Are there times when it's better to be ambiguous? This came up in the context of the extent to which a conlang should discourage ambiguity, as a means of encouraging cognitive correctness by its users. It seems to me that something is being taken for granted here, that ambiguity is necessarily an impediment to clear thinking. And I certainly agree that it can be. But if detail or specificity are the opposites of ambiguity, then surely maximal detail or specificity is undesirable when the extra information isn't relevant, so that a conlang would benefit from not requiring users to minimize ambiguity.
Moving away from the concept of conlangs, this opens up some interesting (at least to me) questions. Exactly what does "ambiguity" mean? Is there, for each speech act, an optimal level of ambiguity, and how much can be gained by achieving it? Are there reasons why a certain, minimal degree of ambiguity might be desirable beyond avoiding irrelevant information?
Promoting rationality to a broad audience - feedback on methods
by Gleb_Tsipursky — 2015-11-30T04:52:03.243Z

We at Intentional Insights​, the nonprofit devoted to promoting rationality and effective altruism to a broad audience, are finalizing our Theory of Change (a ToC is meant to convey our goals, assumptions, methods, and metrics). Since there's recently been extensive discussion on LessWrong of our approaches to promoting rationality and effective altruism to a broad audience, one that was quite helpful for helping us update, I'd like to share our Theory of Change with you and ask for your feedback.

Here's the Executive Summary:

The goal of Intentional Insights is to create a world where all rely on research-based strategies to make wise decisions and lead to mutual flourishing.

To achieve this goal, we believe that people need to be motivated to learn and have broadly accessible information about such research-based strategies, and also integrate these strategies into their daily lives through regular practice.

We assume that:

some natural and intuitive human thinking, feeling, and behavior patterns are flawed in ways that undermine wise decisions.

problematic decision making undermines mutual flourishing in a number of life areas.

these flawed thinking, feeling, and behavior patterns can be improved through effective interventions.

we can motivate and teach people to improve their thinking, feeling, and behavior patterns by presenting our content in ways that combine education and entertainment.

Our intervention is helping people improve their patterns of thinking, feeling, and behavior to enable them to make wise decisions and bring about mutual flourishing.

Our outputs, what we do, come in the form of online content such as blog entries, videos, etc., on our channels and in external publications, as well as collaborations with other organizations.

Our metrics of impact are in the form of anecdotal evidence, feedback forms from workshops, and studies we run on our content.

Here is the full version.

I'd appreciate any feedback on the full version from fellow Less Wrongers, on things like c

...
Smarter humans, not artificial intellegence
by wubbles — 2015-11-30T03:48:59.262Z

I'm writing this article to explain some of the facts that have convinced me that increasing average human intelligence through traditional breeding and genetic manipulation is likelier to reduce existential risks in the short and medium term then studying AI risks, while providing all kinds of side benefits.

Intelligence is useful to achieve goals, including avoiding existential risks. Higher intelligence is associated with many diverse life outcomes improving, from health to wealth. Intelligence may have synergistic effects on economic growth, where average levels of intelligence matter more for wealth then individual levels. Intelligence is a polygenetic trait with strong heritability. Sexual selection in the Netherlands has resulted in extreme increases in average height over the past century: sexual selection for intelligence might do the same. People already select partners for intelligence, and egg donors are advertised by SAT score.

AI research seems to be intelligence constrained. Very few of those capable of making a contribution are aware of the problem, or find it interesting. The Berkeley-MIRI seminar has increased the pool of those aware of the problem, but the total number of AI safety researchers remain small. So far very foundational problems remain to be solved. This is likely to take a very long time: it is not unusual for mathematical fields to take centuries to develop. Furthermore, we can work on both strategies at once and observe spillover from one into the other, as the larger intelligence baseline translates into an increase on the right tail of the distribution.

How could we accomplish this? One idea, invented by Robert Heinlein, as far as I know, is to subsidize marriages between people of higher than usual intelligence and their having children. This idea has the benefit of being entirely non-coercive. It is however unclear how much these subsidies would need to be to influence behavior, and given the strong returns to intelligence in life outcomes, unclear that they can further influence behavior.

Another idea would be to conduct genetic studies to find genes which influence genetics, and conduct genome modification. This plan suffers from illegality, lack of knowledge of genetic factors of intelligence, and absence of effective means for genome editing (they tried CRISPR on human embryos: more work is needed). However, ...
Linguistic mechanisms for less wrong cognition
by KevinGrant_duplicate0.2409764628391713 — 2015-11-29T02:40:08.976Z

I'm working on a conlang (constructed language) and would like some input from the Less Wrong community. One of the goals is to investigate the old Sapir-Whorf hypothesis regarding language affecting cognition. Does anyone here have any ideas regarding linguistic mechanisms that would encourage more rational thinking, apart from those that are present in the oft-discussed conlangs e-prime, loglan, and its offshoot lojban? Or perhaps mechanisms that are used in one of those conlangs, but might be buried too deeply for a person such as myself, who only has superficial knowledge about them, to have recognized? Any input is welcomed, from other conlangs to crazy ideas.