trentmkelly/LessWrong-43k huggingface dataset:
Anatomy of Multiversal Utility Functions: Tegmark Level IV
by Squark — 2015-02-07T16:28:45.208Z

Outline: Constructing utility functions that can be evaluated on any possible universe is known to be a confusing problem, since it is not obvious what sort of mathematical object should be the domain and what properties should the function obey. In a sequence of posts, I intend break down the question with respect to Tegmark's multiverse levels and explain the answer on each level, starting with level IV in the current post.

Background

An intelligent agent is often described as an entity whose actions drive the universe towards higher expectation values of a certain function, known as the agent's utility function. Such a description is very useful in contexts such as AGI, FAI, decision theory and more generally any abstract study of intelligence.

Applying the concept of a utility function to agents in the real worlds requires utility functions with a very broad domain. Indeed, since the agent is normally assumed to have only finite information about the universe in which it exists, it should allow for a very large variety of possible realities. If the agent is to make decisions using some sort of utility calculus, it has to be able to evaluate its utility function on each of the realities it can conceive.

Tegmark has conveniently arranged the space of possible realities ("universes") into 4 levels, 3 of which are based on our current understanding of physics. Tegmark's universes are usually presented as co-existing but it is also possible to think of them as the "potential" universes in which our agent can find itself. I am going to traverse Tegmark's multiverse from top to bottom, studying the space of utility functions on each level (which, except for level IV, is always derived from the higher level). The current post addresses Tegmark level IV, leaving the lower levels for follow-ups.

Some of the ideas in this post previously appeared in a post about intelligence metrics, where I explained them much more tersely.

Tegmark Level IV

Tegmark defined this level as the collection of all mathematical models. Since it is not even remotely clear how to define such a beast, I am going to use a different space which (I claim) is conceptually very close. Namely, I am going to consider universes to be infinite binary sequences . I denote the by the space of all such sequences equipped with the product topology. As will become clearer in the following, this s...
How to learn soft skills
by AnnaSalamon — 2015-02-07T05:22:53.790Z

Acquiring some skills is mostly about deliberate, explicit information transfer. For example, one might explicitly learn the capital of Missouri, or the number of miles one can drive before needing an oil change, or how to use the quadratic formula to solve quadratic equations.

For other skills, practitioners' skill rests largely on semi-conscious, non-explicit patterns of perception and action. I have in mind here such skills as:

Managing your emotions and energy levels;
Building strong relationships;
Making robust plans;
Finding angles of attack on a mathematical problem;
Writing persuasively;
Thinking through charged subjects without bias;

and so on. Experts in these skills will often be unable to accurately and explicitly describe how to do what they do, but they will be skilled nonetheless.

I'd like to share some thoughts on how to learn such "soft skills".

Usefulness of non-true stimuli

If you read a chemistry textbook, it makes sense to ask after each sentence: “Is this true?”. If the answer is “no”, “no”, “no”, for a sufficient number of sentences, you should probably abandon that book and look for a better one. Chemistry textbooks are supposed to be made out of statements you can trust -- statements you can add to your file of “trusted explicit claims”, in such a fashion as to make you better at chemistry. When a book fails at this property, its main value is lost.

Not so, IMO, for soft skills.

You can test ideas in your "inner simulator"

Your “inner simulator” is CFAR’s version of the distinction between profession and anticipation. Basically, your “inner simulator” is the part of you that can play movies forward to determine what to anticipate: “Do I have time to turn left before that car reaches me?”; “What will she do, if I approach and say ‘hi’?” (that is: what does my inner movie-player show as the next scene, when I play it a movie in which I walk up to her and say ‘hi’?).

Your inner simulator is probably more accurate than your explicit models in domains where you have a rich experience base, such as social phenomena, and day-to-day physical phenomena. It is probably worse in areas where you have good book-learning behind you (e.g., you may have an accurate conscious model of the bystander effect, but still mismodel this when you anticipate without conscious adjustment). Your inner simulator is al

...
Have you changed your mind recently?
by  — 2015-02-06T18:34:15.111Z

Our beliefs aren't just cargo that we carry around. They become part of our personal identity, so much so that we feel hurt if we see someone attacking our beliefs, even if the attacker isn't speaking to us individually. These "beliefs" are not necessarily grand things like moral frameworks and political doctrines, but can also be as inconsequential as an opinion about a song.

This post is for discussing times when you actually changed your mind about something, detaching from the belief that had wrapped itself around you.

Relevant reading: The Importance of Saying "Oops", Making Beliefs Pay Rent
Non-manipulative oracles
by Stuart_Armstrong — 2015-02-06T17:05:42.000Z

Benja had a post about trying to get predictors to not manipulate you. It involved a predictor that could predict tennis matches, but where the prediction could also manipulate the impact of those matches.

To solve this, Benja imagined the actions of a hypothetical CDT reasoning agent located on the moon, and unable to affect the outcome.

While his general approach is interesting, it seems the specific problem has a much simpler solution: before the AI's message is outputed, it's run through a randomised scrubber that has a tiny chance of erasing it.

Then the predictor would try and maximise expected correctness of its prediction, given that the scrubber erased its output (utility indifference can have a similar effect). In practice the scrubber would almost never trigger, so we would get accurate predictions, unaffected by our reading of them.

Does this seem it'll work?
Weekly LW Meetups
by FrankAdamek — 2015-02-06T15:58:34.344Z

This summary was posted to LW Main on January 30th. The following week's summary is here.

New meetups (or meetups with a hiatus of more than a year) are happening in:

Czech's first Meetup Prague: 16 February 2015 06:00PM

Irregularly scheduled Less Wrong meetups are taking place in:

Bangalore LWers meetup: 14 February 2015 11:00AM
European Community Weekend 2015: 12 June 2015 12:00PM
[Utrecht] Effective Altruism Netherlands, Utrecht - How to live on a low budget: 01 February 2015 02:00PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Austin, TX - Mini Talks - Caffe Medici: 31 January 2026 01:30PM
Brussels February meetup: Words: 14 February 2015 01:00PM
Durham, NC (RTLW) Discussion Meetup: 05 February 2026 07:00PM
London Social - Time and Space - February 1st: 01 February 2015 02:00PM
[Melbourne] February 2015 Rationality Dojo - Introduction to Ethics: 01 February 2015 03:30PM
[Moscow] Regular Moscow meetup: self-esteem in CBT, psycholinguistics, philosophy of mind: 01 February 2015 02:00PM
Sydney Rationality Dojo - How bad statistics can ruin your life: 01 February 2015 04:00PM
Washington, D.C.: Performance of Masculinity: 01 February 2015 03:00PM
Vienna: 21 February 2015 03:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Cambridge UK, Canberra, Columbus, London, Madison WI, Melbourne, Moscow, Mountain View, New York, Philadelphia, Research Triangle NC, Seattle, Sydney, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved t...
Reductionist research strategies and their biases
by PhilGoetz — 2015-02-06T04:11:32.650Z

I read an extract of (Wimsatt 1980) [1] which includes a list of common biases in reductionist research. I suppose most of us are reductionists most of the time, so these may be worth looking at.

This is not an attack on reductionism! If you think reductionism is too sacred for such treatment, you've got a bigger problem than anything on this list.

Here's Wimsatt's list, with some additions from the parts of his 2007 book Re-engineering Philosophy for Limited Beings that I can see on Google books. His lists often lack specific examples, so I came up with my own examples and inserted them in [brackets].

Conceptualization

Descriptive Localization: Describing a relational property as if it were monadic, or a lower-order relational property.

Fitness treated as a property of phenotype (or even of genes) rather than as a property of phenotype and environment.

[This may be equivalent to assuming that you can apply linearization to remove variables from a function. You often do this to analyze the stability of equilibriums. So often it's a useful assumption.]

Meaning Reductionism: Assuming lower-level redescriptions change meanings of scientific terms, while [going back to?] higher-level redescriptions [does] not.

Philosophers (who view themselves as concerned with meaning relations) are inclined to a reductionist bias.

[What he might mean: Modernist theories of meaning begin by studying sentences in isolation, as logic propositions. Take a classic example, "Bachelors are unmarried men," represented as BAUM = forall(X, B(X)=>U(X),M(X)). They show that there are instances #bob where B(#bob) holds, yet affirming U(#bob) or M(#bob) would seem peculiar, and then say therefore not(true(BAUM)), then generalize to say that true(forall(X, P(X)=>Q(X))) is not meaningful in general. But they don't consider that we may use the English word "truth" differently when talking about propositions with quantified or unbound variables (BAUM) than when talking about propositions with only bound variables ("Justin Bieber is a bachelor"). But I don't think this is the main problem with these modernist analyses; compare the similar "birds can fly" / "*penguins can fly" argument.]

Interface Determinism: Assuming that all that counts in analyzing the nature and behavior of a system is what comes or goes across the system-environment interface.

...
Probabilistic Oracle Machines and Nash Equilibria
by jessicata — 2015-02-06T01:14:49.000Z

Read the paper draft here.

I'm working on a a paper with Benja introducing probabilistic oracle machines and showing how they are equivalent with Nash equilibria, as I initially argued in a comment. This variant of the reflective oracles was originally suggested in Paul Christiano's comment. We might end up submitting a later version of this paper to a conference. Any comments or criticism are very welcome!
Meetup : Warsaw February Meetup
by mkf — 2015-02-05T21:16:02.516Z

Discussion article for the meetup : Warsaw February Meetup

WHEN:
21 February 2015 06:00:00PM (+0100)

WHERE:
Warsaw

The next meetup in Warsaw - we are slowly getting regular!

Facebook event: https://www.facebook.com/events/436771293136918
Facebook group for better coordination: https://www.facebook.com/groups/lwwarsaw/

If you missed all the previous meetups, don't make that mistake now.

Discussion article for the meetup : Warsaw February Meetup
MIRI's technical agenda: an annotated bibliography, and other updates
by So8res — 2015-02-05T18:11:05.051Z

This week, I'm pleased to present Aligning Superintelligence with Human Interests: An Annotated Bibliography. This annotated bibliography complements our technical agenda and the six supporting papers that I've released over the past few months. Once you've read the supporting papers, this annotated bibliography will help you figure out what to read next on any given topic, in order to get to the cutting edge.

This annotated bibliography concludes my series of updates on MIRI's technical agenda. To review, I've presented a series of eight papers which sketch out MIRI's research strategy and overview a number of active research areas. Those papers are:

Aligning Superintelligence with Human Interests: A Technical Research Agenda

Formalizing Two Problems of Realistic World Models

Toward Idealized Decision Theory

Questions of Reasoning Under Logical Uncertainty

Vingean Reflection: Reliable Reasoning for Self-Modifying Agents

Corrigibility

The Value Learning Problem

Aligning Superintelligence with Human Interests: An Annotated Bibliography

The entire suite can be found in one place on our new technical agenda page. You're encouraged to link people to that page if they're wondering what sort of technical research MIRI is up to.

I've also updated the research guide to synchronize it with the technical agenda. The new annotated bibliography serves a similar purpose to the research guide, but where the research guide is geared more towards people starting from scratch, the annotated bibliography is geared towards people who already have the background knowledge and want to know which papers to read in which order without any fluff.

Enjoy!
Request for proposals for Musk/FLI grants
by danieldewey — 2015-02-05T17:04:35.652Z

As a follow-on to the recent thread on purchasing research effectively, I thought it'd make sense to post the request for proposals for projects to be funded by Musk's $10M donation. LessWrong's been a place for discussing long-term AI safety and research for quite some time, so I'd be happy to see some applications come out of LW members.

Here's the full Request for Proposals.

If you have questions, feel free to ask them in the comments or to contact me!

Here's the email FLI has been sending around:

Initial proposals (300–1000 words) due March 1, 2015

The Future of Life Institute, based in Cambridge, MA and headed by Max Tegmark (MIT), is seeking proposals for research projects aimed to maximize the future societal benefit of artificial intelligence while avoiding potential hazards. Projects may fall in the fields of computer science, AI, machine learning, public policy, law, ethics, economics, or education and outreach. This 2015 grants competition will award funds totaling $6M USD.

This funding call is limited to research that explicitly focuses not on the standard goal of making AI more capable, but on making AI more robust and/or beneficial; for example, research could focus on making machine learning systems more interpretable, on making high-confidence assertions about AI systems' behavior, or on ensuring that autonomous systems fail gracefully. Funding priority will be given to research aimed at keeping AI robust and beneficial even if it comes to greatly supersede current capabilities, either by explicitly focusing on issues related to advanced future AI or by focusing on near-term problems, the solutions of which are likely to be important first steps toward long-term solutions.

Please do forward this email to any colleagues and mailing lists that you think would be appropriate.

Proposals

Before applying, please read the complete RFP and list of example topics, which can be found online along with the application form:

http://futureoflife.org/grants/large/initial

As explained there, most of the funding is for $100K–$500K project grants, which will each support a small group of collaborators on a focused research project with up to three years duration. For a list of suggested topics, see the complete RFP [1] and the Research Priorities document [2]. Initial proposals, which are intended to require merely a modest amount of preparat

...
Meetup : Another Frankfurt meetup
by Kendra — 2015-02-05T13:10:51.451Z

Discussion article for the meetup : Another Frankfurt meetup

WHEN:
22 February 2015 02:00:00PM (+0100)

WHERE:
Heerstraße, 60488 Frankfurt

I'm posting this for weird_random who unfortunately can't post in the meetup section yet:

"Hi!

We will have a next LessWrong meetup in February!

Topics:
- introduction and small talk
- complete New Year Plans and Projects
- Goal Tracking
- free discussions, hot topics: AI (Artificial Intelligence), hpmor (Fanfiction: "Harry Potter and the methods of rationality")
(If you want to present something, you are welcome!)
- Prediction Game

We will meet in the "LessWrong community apartment" ;)
Please contact tao10tohitw(at)gmail.com for the exact location.
Feel free to bring some food and fun!

Newcomers welcome!
If you want to know us before, we offer a skype-call to introduce at least two of us.
If you have any special needs, please tell us in advance. Then we try our best to help you.
(Contact see above)
If you want to know more about the meetups, join our mailing list: https://groups.google.com/forum/#!forum/less-wrong-frankfurt :)

Kind regards
weird_random

Hi!

Wir haben wieder ein LessWrong Treffen im Februar!

Themen:
- Begrüßungsrunde und plaudern
- Neujahrsvorsätze zu Ende besprechen
- persönliche Ziele festsetzen, auswerten
- freie Diskussion, beliebte Themen zur Zeit: künstliche Intelligenz, Fangeschichte "Harry Potter und die Methode des rationalen Denkens"
(Wenn du etwas präsentieren möchtest, bist du herzlich dazu eingeladen!)
- Vorraussagungsspiel (man muss zu einer Frage ein Antwortwertintervall tippen, dass möglichst in dem vorgegebenen Wahrscheinlichkeitsintervall - meist 80% - liegt)

Wir treffen uns in der "LessWrong-GemeinschaftsWohnung" ;)
Für die genaue Adresse bitte tao10tohitw(at)gmail.com kontaktieren.
Du kannst gerne Essen und Spaß mitbringen!

Neulinge sind willkommen!
Wenn du uns gerne vorher kennen lernen willst, bieten wir dir einen Skype-Anruf an, in dem mindestens mal zwei von uns sich vorstellen.
Wenn du besondere Bedürfnisse hast, gib bitte vorher bescheid. Wir versuchen dann, so gut wie möglich zu helfen.
(Kontakt: s. oben)
Wenn du mehr über die Treffen erfahren möchtest, dann komm in unsere mailing Liste: https://groups.google.com/forum/#!forum/less-wrong-frankfurt :)

Viele Grüße
weird_random"

Discussion article for the

...
Meetup : Canberra Less Wrong Meet Up - Favourite Sequence Post + Discussion
by DanielFilan — 2015-02-05T05:49:29.620Z

Discussion article for the meetup : Canberra Less Wrong Meet Up - Favourite Sequence Post + Discussion

WHEN:
14 February 2015 06:00:00PM (+1100)

WHERE:
108 North Road, Acton, ACT

For this meet up, we want to talk about our favorite LW posts. Please pick one (or more) posts from the 'How to Actually Change Your Mind' sequence
that you found interesting. Tell us why you liked it and possibly lead into a group discussion.

Feel free to pick a post from another sequence if you want and it its totally fine if you haven't read any of the sequences, you can still come and discuss other peoples favourite posts. It would also be awesome if you could post your favourite post on the Facebook event page so everyone has the chance to read them before hand.

As usual Vegan snacks will be provided. Hope to see you there!

General meetup info:

If you use Facebook, please join our group.

Structured meetups are (usually) held on the second Saturday and fourth Friday of each month from 6 pm until late in the CSIT building, room N101.

Discussion article for the meetup : Canberra Less Wrong Meet Up - Favourite Sequence Post + Discussion
Meetup : Dallas, TX
by DubiousTwizzler — 2015-02-05T05:20:20.009Z

Discussion article for the meetup : Dallas, TX

WHEN:
08 February 2015 01:00:00PM (-0600)

WHERE:
14833 Midway Rd Addison, TX 75001

This is the new Dallas meetup. We will be meeting up at Taste of Chicago on Sunday, Feb 8th at 1 PM. I will have a sign that says "Less Wrong" on it.

Last meetup was lots of fun. We decided to try having it at a restaurant this time. Come hang out and eat pizza with us!

Discussion article for the meetup : Dallas, TX
Meetup : HPMOR Wrap Party! - Sydney Edition
by taryneast — 2015-02-05T04:29:05.857Z

Discussion article for the meetup : HPMOR Wrap Party! - Sydney Edition

WHEN:
15 March 2015 06:00:00PM (+1100)

WHERE:
10 Shepherd St, Chippendale NSW 2008

We're holding this at 1800 on the day after Pi Day, because the final chapter comes out at ~0300 local time.

We're holding it at Humanist House
10 Shepherd St, Chippendale NSW 2008

Discussion article for the meetup : HPMOR Wrap Party! - Sydney Edition
Meetup : Sydney Meetup - February
by taryneast — 2015-02-05T04:28:03.949Z

Discussion article for the meetup : Sydney Meetup - February

WHEN:
25 February 2015 06:30:00PM (+1100)

WHERE:
565 George Street, Sydney, Australia 2000

Regular location - City of Sydney RSL The restaurant on Level 2
Regular time (starting about 6:30)
Topic: TBD

Discussion article for the meetup : Sydney Meetup - February
Don't estimate your creative intelligence by your critical intelligence
by PhilGoetz — 2015-02-05T02:41:28.108Z

When I criticize, I'm a genius. I can go through a book of highly-referenced scientific articles and find errors in each of them. Boy, I feel smart. How are these famous people so dumb?

But when I write, I suddenly become stupid. I sometimes spend half a day writing something and then realize at the end, or worse, after posting, that what it says simplifies to something trivial, or that I've made several unsupported assumptions, or claimed things I didn't really know were true. Or I post something, then have to go back every ten minutes to fix some point that I realize is not quite right, sometimes to the point where the whole thing falls apart.

If someone writes an article or expresses an idea that you find mistakes in, that doesn't make you smarter than that person. If you create an equally-ambitious article or idea that no one else finds mistakes in, then you can start congratulating yourself.
Meetup : Washington, D.C.: Fun & Games
by RobinZ — 2015-02-04T21:22:17.946Z

Discussion article for the meetup : Washington, D.C.: Fun & Games

WHEN:
08 February 2015 03:00:00PM (-0500)

WHERE:
Clarendon Metro

Attention: the location of this week's meetup has changed. Details below.

This week, we will be meeting at a private residence in Northern Virginia to hang out, play games, and engage in fun conversation.

The meetup location this week is nearest to Clarendon Metro Station - please contact David Steinberg at david.isaac.steinberg@gmail.com for the address or to coordinate a ride. Our present plan is to pick most people up from Clarendon at 3:00 p.m. (though again, please contact us to confirm that you'd like a ride).

Trains will be running at reduced frequencies on all lines to accommodate track work on the Red, Orange, Yellow, and Green Lines.

Upcoming meetups:

Feb. 15: Book Swap (lend books, borrow books, talk about books)

Feb. 22: What-If Scenario (discuss one or more interesting hypotheticals)

Mar. 1: Fun & Games (w/Nomic) (bring games, play games, converse, socialize, or any combination thereof)

Discussion article for the meetup : Washington, D.C.: Fun & Games