trentmkelly/LessWrong-43k huggingface dataset:
Seeking Advice About Career Paths for Non-USA Citizen
by almkglor — 2016-09-28T00:07:38.417Z

Hi all,

Mostly lurker, I very rarely post, mostly just read the excellent posts here.

I'm a Filipino, which means I am a citizen of the Republic of the Philippines. My annual salary, before taxes, is about $20,000 (USA dollars). I work at an IC development company (12 years at this company), developing the logic parts of LCD display drivers. My understanding is that the median US salary for this kind of job is about $80,000 -> $100,000 a year. This is a fucking worthless third world country, so the government eats up about ~30% of my salary and converts it to lousy service, rich government officials, bad roadworks, long commute times, and a (tiny) chance of being falsely accused of involvement in the drug trade and shot without trial. Thus my take-home pay amounts to about $15,000 a year. China is also murmuring vague threats about war because of the South China Sea (which the local intelligentsia insist on calling the West Philippine Sea); as we all know, the best way to survive a war is not be in one.

This has lead to my deep dissatisfaction with my current job.

I'm also a programmer as a hobby, and have been programming for 23 years (I started at 10 years old on Atari LOGO; I know a bunch of languages from low-level X86 assembly to C to C++ to ECMAScript to Haskell, and am co-author of SRFI-105 and SRFI-110). My understanding is that a USA programmer would *start* at the $20,000-a-year level (?), and that someone with experience can probably get twice that, and a senior one can get $100,000/year.

As we all know, once a third world citizen starts having first world skill level, he starts demanding first world renumeration also.

I've been offered a senior software developer job at a software company, offering approximately $22,000/year; because of various attempts at tax reform it offers a flat 15% income tax, so I can expect about $18,000/year take home pay. I've turned it down with a heavy heart, because seriously, $22,000/year at 15% tax for a senior software developer?

Leaving my current job is something I've been planning on doing, and I intend to do so early next year. The increasing stress (constant overtime, management responsibilities (I'm a tech geek with passable social skills, and exercising my social skills drains me), 1.5-hour commutes) and the low renumeration makes me want to consider m...
New Philosophical Work on Solomonoff Induction
by vallinder — 2016-09-27T11:12:25.757Z

I don't know to what extent MIRI's current research engages with Solomonoff induction, but some of you may find recent work by Tom Sterkenburg to be of interest. Here's the abstract of his paper Solomonoff Prediction and Occam's Razor:

Algorithmic information theory gives an idealised notion of compressibility that is often presented as an objective measure of simplicity. It is suggested at times that Solomonoff prediction, or algorithmic information theory in a predictive setting, can deliver an argument to justify Occam's razor. This article explicates the relevant argument and, by converting it into a Bayesian framework, reveals why it has no such justificatory force. The supposed simplicity concept is better perceived as a specific inductive assumption, the assumption of effectiveness. It is this assumption that is the characterising element of Solomonoff prediction and wherein its philosophical interest lies.
The set of Logical Inductors is not Convex
by Scott Garrabrant — 2016-09-27T09:05:00.000Z

Sam Eisenstat asked the following interesting question: Given two logical inductors over the same deductive process, is every (rational) convex combination of them also a logical inductor? Surprisingly, the answer is no! Here is my counterexample.

We construct two logical inductors over PA, {Pn}, and {P′n}.

Let {Pn} be any logical inductor over PA.

We consider an infinite sequence of sentences ϕn:⇔"Pn(ϕn)<1/2".

Let Pn(ϕn) be computable in f(n) time.

We construct {P′n} as in the paper, but instead of using all traders computable in time polynomial in n, we use all traders computable in time polynomial in f(n) time. Since this also includes all polynomial time traders, {P′n} is a logical inductor.

However, since the truth value of ϕn is computable in f(n) time, if the difference between P′n(ϕn) and the indicator of ϕn did not converge to 0, a trader running in time polynomial in f(n) can easily exploit {P′n}. Thus,

Pn(ϕn)≂n1/2,

and

P′n(ϕn)≂nThmPA(ϕn).

Now, consider the market

{P′′n=Pn+P′n2}.

Observe that

P′′n(ϕn)≂n1+2⋅ThmPA(ϕn)4,

so

|P′′n(ϕn)−1/2|≂n1/4.

Now, consider the trader who exploits {P′′n} by repeatedly either buying a share of ϕn when the price is near 3/4, or selling a share when the price is near 1/4, waiting for that sentence to be resolved, and then repeating. Eventually, in each cycle, this trader will make roughly 1/4 of a share, because eventually the price will always be close enough to either 1/4 or 3/4, and all shares that this trader buys will be true, and all shares that this trader sells will be false.

Thus {P′′n} is not a logical inductor.
Imagine paint as chance you spread across the world
by BenFRayfield — 2016-09-27T02:29:17.898Z

People are not good at statistics. We tend to imagine more or less than 100% chance when dividing it between possibilities. Instead, try this:

If theres something certain to happen but less certain when, imagine a row of a calendar (or however you visualize sequential days) with different amounts of paint on certain days. Maybe its likely in the next few days but a small chance it could take a few weeks. So spread the paint in those proportions. Its always 1.0 total paint. For uncertainty, leave some of the paint blurred and you dont know where it is. As the days pass and the thing still hasnt happened, you move that chance somewhere onto the later days since its still 100% going to happen.

Or if it may not happen, use 2 colors of paint, 1 for all relevant possibilities, and an inner paint touching only part of that outer paint. If its 30% chance to happen, the inner paint touches 30% of the outer paint.

Recursively, we can think about combinations of events, places, things, and chances this way.
Logical Inductors contain Logical Inductors over other complexity classes
by Scott Garrabrant — 2016-09-26T22:17:44.000Z

In the Logical Induction paper, we give a definition of logical inductors over polynomial time traders. It is clear from our definition that our use of polynomial time is rather arbitrary, and we could define e.g. an exponential time logical inductor. However, it may be less clear that actually logical inductors over one complexity class contain logical inductors over other complexity classes within them.

For this post, I will define polynomial logical inductor to be the logical inductors defined in the main paper. I will define exponential logical inductor similarly, except the traders will be allowed to run in exponential time instead of polynomial time.

Theorem: If {Pn} is a polynomial logical inductor over {Dn}, then the subsequence {P22n} is an exponential logical inductor over {D22n}.

Proof Sketch: {P22n} is clearly computable. Assume by way of contradiction that {P22n} is not an exponential logical inductor over {D22n}. Then there exists some exponential time trading strategy which exploits it. This trader runs in time at most c⋅2nc for some c. But then there exists another trader which runs in polynomial time, and on input 22n implements this trader against {Pn}, and output's the constant 0 trading strategy otherwise. This can be done in polynomial time, since there is a polynomial of 22n much larger than c⋅2nc. Note that this trader exploits {Pn}, contradicting the assumption that {Pn} is a polynomial logical inductor over {Dn}.

Clearly, this generalizes to other pairs of complexity classes as well.

Observe that the converse is not necessarily true. In particular {Pn} may not even converge when {P22n} is an exponential logical inductor over {D22n}. However logical inductors trivially contain logical inductors over smaller complexity classes, since logical inductors are themselves logical inductors over all smaller complexity classes.

Combining this insight with that of Universal Inductors, we can now define a singe universal inductor, and use it to get a logical inductor by conditioning on a theory, and also use it to get universal and logical inductors over larger complexity classes, by taking sparse subsequences.
Meetup : Stockholm: When to stop making a decision
by pepe_prime — 2016-09-26T17:14:40.906Z

Discussion article for the meetup : Stockholm: When to stop making a decision

WHEN:
07 October 2016 03:00:00PM (+0200)

WHERE:
Lindstedtsvägen 3, Room 1537

We'll run monthly meetups starting with this one. This talk is the start of a series on decision analysis for personal life. If you want to influence or organize future Stockholm meetups, let me know.

The talk will introduce a bit of notation, but mostly be an informal presentation. After 30 minutes the event will open to audience discussion.

Discussion article for the meetup : Stockholm: When to stop making a decision
We have the technology required to build 3D body scanners for consumer prices
by ChristianKl — 2016-09-26T15:36:20.138Z

Apple's iPhone 7 Plus decided to add another lense to be able to make better pictures. Meanwhile Walabot who started with wanting to build a breast cancer detection technology released a 600$ device that can look 10cm into walls. Thermal imaging also got cheaper.

I think it would be possible to build a 1500$ device that could combine those technologies and also add a laser that can shift color. A device like this could bring medicine forward a lot.
A lot of area's besides medicine could likely also profit from a relatively cheap 3D scanner that can look inside objects.

Developing it would require Musk-level capital investments but I think it would advance medicine a lot if a company would both provide the hardware and develop software to make the best job possible at body scanning.
Learning doesn't solve philosophy of ethics
by Stuart_Armstrong — 2016-09-26T12:11:08.000Z

A putative new idea for AI control; index here.

This post will use the formalism of this post to illustrate some well known philosophical thought experiments and show why learning algorithms are not sufficient to solve them.

Examples

Death and life extension

A human consists of two agents, A100 and A1. The agent A100 is a long-term agent; it has the preference that the human not live longer than a century. The agent A2 is a short term agent; it prefers that the human survive for the coming year.

The human meta preferences M are that A100 and A1 be eventually be brought into compatibility with each other.

By observation and prediction, the AI knows that, under the normal course of events, A1 will never sync with A100: the human will continue to believe that it shouldn't live another hundred years, but will never want to die that year.

The AI can trigger human introspection I in two ways; the first one removed the long term death preference in A100, the second one will remove the short term death-avoidance in A1, at some later point, so that the human will act consistently with its current A100 (and thus die within the century).

Just based on this information, what is the human's preferences?

Total utilitarianism

The human has the preference r that humans not be reduced to a large population of barely-happy individuals. They also have the meta-preference M that individual utility be additive.

The AI can trigger the human's awareness of the repugnant conclusion. And it can do this in a differential or integral fashion, which will cause the human to either reject its current r (and embrace the repugnant conclusion) or reject M (and reject the repugnant conclusion).

Just based on this information, what is the human's preferences?

The malarial drowning child

Peter Singer has an argument about a drowning child and our duty to them.

To model that contradiction in a human, let r contain the preference to save a drowning child in front of them, and a preference not to send money to distant people dying of malaria. Let M contain the desire that the human preferences not be different across different ways of dying or physical distance.

As before, the right presentation on the AI's part, within the usual bounds of how humans reason, can cause the human to emphasise their preferences or their meta-preferences.

Balconies with a view

The human is modelled as two agents A1 (basically

...
Model of human (ir)rationality
by Stuart_Armstrong — 2016-09-26T11:20:37.000Z

A putative new idea for AI control; index here.

This post is just an initial foray into modelling human irrationality, for the purpose of successful value learning. Its purpose is not to be full model, but have enough details that various common situations can be successfully modelled. The important thing is to model humans in ways that humans can understand (as it's our definition which determines what's a bias and what's a preference in humans).

Humans, actions, and joint distributions

The human themselves is simply modelled as their brain (thus various human sense organs can be observed by the AI rather than being part of the description).

Let R be the set of possible reward functions the human may be maximising. Let Hπ be the set of policies the human may be following. We'll assume that Hπ is closed under the taking of mixed strategies.

The AI has a joint probability distribution P over R, Hπ and events in the world. By conditioning on any element r∈R, P defines a map μ from R to probability distributions over Hπ. Since Hπ is closed under the taking of mixed strategies, this means that μ can be seen as a map from R to Hπ.

The map μ and the marginal distribution PR (P restricted to R) define P entirely. Note that μ is what relates human actions to their explanation in terms of the reward R.

Basic properties of P

Here are a few properties P could have:

#. The distribution P is historical if PR is independent of any action the AI takes.
#. An AI's action a overwrites the reward if μ is constant, conditional on a, while PR is still `broad' ("broad" is not fully defined, but PR is certainly broad enough if it assigns non-zero probability to both an r and −r).
#. The distribution P is Q-rational if there exits a prior distribution Q over the universe such that μ maps r∈R to the optimal policy for an r-maximising agent with prior Q.

It's clear that if P is historical, the AI will treat the human's reward function as something it has to discover, and can't influence. An action a that overwrites the reward means that the human's policy is fixed by action a, independently of whatever reward it might have. This is bad because a) the human actions are no longer informative to the AI about their reward, and b) the human actions are likely suboptimal with respect

Note that stratification can be seen as taking a non-historical distribution, and making it historical via coun

...
Open thread, Sep. 26 - Oct. 02, 2016
by MrMind — 2016-09-26T07:41:51.328Z

If it's worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "Make this post available under..." before submitting.
MIRI's 2016 Fundraiser
by So8res — 2016-09-25T16:55:43.899Z

Update December 22: Our donors came together during the fundraiser to get us most of the way to our $750,000 goal. In all, 251 donors contributed $589,248, making this our second-biggest fundraiser to date. Although we fell short of our target by $160,000, we have since made up this shortfall thanks to November/December donors. I’m extremely grateful for this support, and will plan accordingly for more staff growth over the coming year.

As described in our post-fundraiser update, we are still fairly funding-constrained. Donations at this time will have an especially large effect on our 2017–2018 hiring plans and strategy, as we try to assess our future prospects. For some external endorsements of MIRI as a good place to give this winter, see recent evaluations by Daniel Dewey, Nick Beckstead, Owen Cotton-Barratt, and Ben Hoskin.

Our 2016 fundraiser is underway! Unlike in past years, we'll only be running one fundraiser in 2016, from Sep. 16 to Oct. 31. Our progress so far (updated live):

Donate Now

Employer matching and pledges to give later this year also count towards the total. Click here to learn more.

MIRI is a nonprofit research group based in Berkeley, California. We do foundational research in mathematics and computer science that’s aimed at ensuring that smarter-than-human AI systems have a positive impact on the world. 2016 has been a big year for MIRI, and for the wider field of AI alignment research. Our 2016 strategic update in early August reviewed a number of recent developments:

A group of researchers headed by Chris Olah of Google Brain and Dario Amodei of OpenAI published “Concrete problems in AI safety,” a new set of research directions that are likely to bear both on near-term and long-term safety issues.

Dylan Hadfield-Menell, Anca Dragan, Pieter Abbeel, and Stuart Russell published a new value learning framework, “Cooperative inverse reinforcement learning,” with implications for corrigibility.

Laurent Orseau of Google DeepMind and Stuart Armstrong of the Future of Humanity Institute received positive attention from news outlets and from Alphabet executive chairman Eric Schmidt for their new paper “Safely interruptible agents,” partly supported by MIRI.

MIRI ran a three-week AI safety and robustness colloqui

...
The map of natural global catastrophic risks
by turchin — 2016-09-25T13:17:31.061Z

There are many natural global risks. The greatest of these known risks are asteroid impacts and supervolcanos.

Supervolcanos seem to pose the highest risk, as we sit on the ocean of molten iron, oversaturated with dissolved gases, just 3000 km below surface and its energy slowly moving up via hot spots. Many past extinctions are also connected with large eruptions from supervolcanos.

Impacts also pose a significant risk. But, if we project the past rate of large extinctions due to impacts into the future, we will see that they occur only once in several million years. Thus, the likelihood of an asteroid impact in the next century is an order of magnitude of 1 in 100 000. That is negligibly small compared with the risks of AI, nanotech, biotech, etc.

The main natural risk is a meta-risk. Are we able to correctly estimate natural risks rates and project them into the future? And also, could we accidentally unleash natural catastrophe which is long overdue?

There are several reasons for possible underestimation, which are listed in the right column of the map.

1. Anthropic shadow that is survival bias. This is a well-established idea by Bostrom, but the following four ideas are mostly my conclusions from it.

2. It is also the fact that we should find ourselves at the end of period of stability for any important aspect of our environment (atmosphere, sun stability, crust stability, vacuum stability). It is true if the Rare Earth hypothesis is true and our conditions are very unique in the universe.

3. From (2) is following that our environment may be very fragile for human interventions (think about global warming). Its fragility is like fragility of an overblown balloon poked by small needle.

4. Also, human intelligence was best adaptation instrument during the period of intense climate changes, which quickly evolved in an always changing environment. So, it should not be surprising that we find ourselves in a period of instability (think of Toba eruption, Clovis comet, Young drias, Ice ages) and in an unstable environment, as it help general intelligence to evolve.

5. Period of changes are themselves marks of the end of stability periods for many process and are precursors for larger catastrophes. (For example, intermittent ice ages may precede Snow ball Earth, or smaller impacts with comets debris may precede an impact with larger ...
Meetup : LW Melbourne: A Bayesian Guide on How to Read a Scientific Paper
by Chriswaterguy — 2016-09-24T08:57:56.715Z

Discussion article for the meetup : LW Melbourne: A Bayesian Guide on How to Read a Scientific Paper

WHEN:
08 October 2016 02:30:00PM (+1000)

WHERE:
247 Flinders Lane, Melbourne

PLEASE NOTE: START TIME IS 3:30 PM. For some reason it's displaying incorrectly.

The next meetup for Less Wrong Melbourne will be on Saturday 8th October 2016. Turn up at 3:30pm to socialise, and we start at 4pm sharp.

THIS WEEK: A Bayesian Guide on how to read a Scientific Paper - with Mark Harrigan.

This talk will provide a guide on how to successfully read and make sense of a published Scientific Paper in the peer reviewed literature, regardless of your discipline expertise.
It includes a Bayesian guide to assessing the probability that the conclusions of the paper are credible and a step by step guide outlining how to approach reading, understanding and assessing the conclusions.

It won't make you an instant expert in a field where you don't have much experience, nor will it guarantee that you will spot faulty science - but it will improve your skills in understanding how published science is presented and improve your ability to weigh the credibility of its conclusions.

DINNER AFTERWARDS: At about 6:30pm some of us generally head for dinner in the CBD, and you are welcome to join us.

WHEN: Saturday 8th October, 3:30pm to 6:30pm

WHERE: Ross House, 247 Flinders Lane. Level 1, Room 1. (The room will be noted in a sign on the front door.) If you have any trouble finding the venue or getting in, text or call Chris on 0439 471 632. For the greatest convenience, prompt service and general awesomeness, please arrive around 3:30pm.

WHAT'S THIS ABOUT?: The Less Wrong Melbourne Rationality Dojos are self-improvement sessions for those committed to the art of rationality and personal growth. We welcome new members who are interested in exploring rationality. We're "aspiring rationalists" and always open to learn.

Discussion article for the meetup : LW Melbourne: A Bayesian Guide on How to Read a Scientific Paper