trentmkelly/LessWrong-43k huggingface dataset:
New positions and recent hires at the Centre for the Study of Existential Risk (Cambridge, UK)
by Sean_o_h — 2015-10-13T11:11:23.172Z

[Cross-posted from EA Forum. Summary: Four new postdoc positions at the Centre for the Study of Existential Risk: Evaluation of extreme technological risk (philosophy, economics); Extreme risk and the culture of science (philosophy of science); Responsible innovation and extreme technological risk (science & technology studies, sociology, policy, governance); and an academic project manager (cutting across the Centre’s research projects, and playing a central role in Centre development). Please help us to spread the word far and wide in the academic community!]

An inspiring first recruitment round

The Centre for the Study of Existential Risk (Cambridge, UK) has been making excellent progress in building up our research team. Our previous recruitment round was a great success, and we made three exceptional hires. Dr Shahar Avin joined us in September from Google, with a background in the philosophy of science (Cambridge, UK). He is currently fleshing out several potential research projects, which will be refined and finalised following a research visit to FHI later this month. Dr Yang Liu joined us this month from Columbia University, with a background in mathematical logic and philosophical decision theory. Yang will work on problems in decision theory that relate to long-term AI, and will help us to link the excellent work being done at MIRI with relevant expertise and talent within academia. In February 2016, we will be joined by Dr Bonnie Wintle from the Centre of Excellence for Biosecurity Risk Analysis (CEBRA), who will lead our horizon-scanning work in collaboration with Professor Bill Sutherland’s group at Cambridge; among other things, she has worked on IARPA-funded development of automated horizon-scanning tools, and has been involved in the Good Judgement Project.

We are very grateful for the help of the existential risk and EA communities in spreading the word about these positions, and helping us to secure an exceptionally strong field. Additionally, I have now moved on from FHI to be CSER’s full-time Executive Director, and Huw Price is now 50% funded as CSER’s Academic Director (we share him with Cambridge’s Philosophy Faculty, where he remains Bertrand Russell Chair of Philosophy).

Four new positions:

We’re delighted to announce four new positions at the Centre for the Study of Existential Risk; d...
Open thread, Oct. 12 - Oct. 18, 2015
by MrMind — 2015-10-12T06:57:42.147Z

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.
Subsequence Induction (draft)
by Scott Garrabrant — 2015-10-12T06:19:24.000Z

This post is a draft, I have not written up the relevant proof yet, and when I do, I will probably change the notation around. The basic idea is that we do resource bounded Solomonoff induction, but we allow Turing machines to only make predictions on a subset of the bits. If a Turing machine does not make a prediction for a given bit, we treat it as though it assigned whatever probability this algorithm assigned for that bit.

Here, we define another ALU algorithm proposal. This one, we can prove has very nice properties. However, it only works in a very limited domain.

Consider an environment E, which on input n runs in A(n) time, and we want to predict it in g(n) time. The following algorithm only works when A(n)>>g(n)>>A(n−1). This is a very restrictive condition. However, we are working on some ways that might be able to use this algorithm as a subroutine, and generalize it to a larger domain. (So far,, all of our attempts to extend it have failed, but we are still working on it.)

Consider an enumeration T1,T2,… of all Turing machines. For this problem, we will use finite deterministic Turing machines. Assume that the first Turing machine outputs 1/2 immediately for all inputs.

For each N, we define a weight function wN on Turing machines Tn with n≤N as follows.

wN(Tn)=2−nN−1∏i=n|1−E(i)−Tn(i)||1−E(i)−M(i)|, where Tn(i) is the output of Tn on input i if Tn on input i outputs a rational probability in time at most f(i). If Tn does not output in time or does not output a probability, then Tn(i) is defined to equal M(i), which is defined by:

M(N) is the the weighted average of Tn(N) over all n≤N such that Tn on input i outputs a rational probability in time at most f(i), where Tn gets weight wN(Tn).

Note that M(N) is recursively defined, since wN(Tn) is computed using M(i) for values of i less than N. Let f be a function such that M(N) can be computed in time g(n).

M(N) has the following very valuable property. For any Turing machine T, let S(T) be the set of all natural numbers n on which T outputs a probability in time f(n) on input n. Then, there exists a nonzero constant c such that for all n, the product of the probabilities that M assigns to the correct answer on the first n bits of S(T) is at least c times the product of the probabilities that T assigns to the correct answer on the same bits.

This condition implies that M passes the gene

...
General buying considerations?
by Elo — 2015-10-12T05:18:33.774Z

The following is an incomplete list of suggestions for generic considerations that you might like to make when you go out to buy a thing. I have tried to put the list in order; being generic - certain things will be more or less important in different orders.

0. Do I need the thing? Am I just wanting it on a whim (you are allowed to do that, but at least try to not do that for many expensive things that don’t have resale value)? If a month had gone by, would I still be wanting it?

What is the thing? What functionality considerations do you need to make? What does it need to do? If you already had it - what would it be doing? Will it fit in your life?

What is your expected use? Daily? Once-off? Occasional? (no more than 5 times in your predicted future)

What do I want it to do? Does this thing do what I want it to do? (It can be very easy to buy a thing that doesn't quite suit the need because we get distracted between wanting a thing and getting a thing)

Consider your options that avoid buying it:

Can I borrow one from a friend? Or a family member? (some things cannot be borrowed like a wristwatch - no sense borrowing one if it’s an item you wear every day - or other reasons to not borrow a thing)

Can I get one second hand?
Some items are perfectly fine second hand, i.e. books, whereas others are potentially less fine (i.e. cars) where more can go wrong with a second hand one. The point of this inclusion was to encourage you to consider it when you previously would not have. for whichever reason. Books second hand can also be occasionally out of date or damaged; and cars second hand can be excellent purchases.

Is anyone I know also interested in having the thing, and would they be willing to split the cost with me in order to have it on a kind of timeshare, and can we agree on a deprecation schedule such that one of us buys out the other's share in the future, if one of us is moving away or something?

Renting/hiring the thing - as a one off. (works for most power tools, as well as storage space, a boat, all kinds of things...). It is also an option to rent short term while you decide if the thing fits your life. i.e. rent a jetski. If you find you don’t use it enough to warrant a full purchase you only needed to invest a little bit of the final cost; and might b

...
Meetup : San Francisco Meetup: Fermi Estimates
by rocurley — 2015-10-12T03:20:46.684Z

Discussion article for the meetup : San Francisco Meetup: Fermi Estimates

WHEN:
12 October 2015 06:15:00PM (-0700)

WHERE:
1597 Howard St. San Francisco, CA

NOTE: THE LOCATION HAS CHANGED RECENTLY

We'll be meeting to do some fermi estimates. You can call 301-458-0764 to be let in.

Discussion article for the meetup : San Francisco Meetup: Fermi Estimates
Clothing is Hard (A Brief Adventure into my Inefficient Brain)
by k_ebel — 2015-10-12T01:44:56.967Z

This is an active solicitation for suggestions on how to train it differently.

Apparently, this morning I put on my underwear wrong.

Upon noticing that they were on incorrectly, I took them off by turning them inside out on the Z axis (top of head to bottom of feet), and then rotating them 180degreees along the Y axis (belly button to back, travelling through the spine).

I noted the degrees of off-ness on the two axes, intending to remember them for the next time this happens. Yes, this happens often enough that I'll probably use the information again. Sometimes, even clothing is hard.

...

It was only then that I realized that the easier way to understand what happened would be to say that they were 180degrees off on the X axis (L shoulder to R shoulder, by travelling across the back).

------------------------------------------

Ultimately, how this seems to play out is that I get ahead of myself in some rather strange ways. I tend to think of things in motion before I fully understand them in their static forms. In the example above, it would have meant that I was trying to store larger chunks of more complex data, when a simpler notation would have done just as well. I also find that it can distract me from recognizing the context around whatever I'm observing.

I'm only just beginning to be able to identify when that's happening.

Obviously, I want to address this. I just don't know how to go about figuring out what needs to be done. From how to gather more information, to what to do with it.

Ideas?
Meetup : Reading Weekly Meetups
by Acty — 2015-10-12T00:11:43.524Z

Discussion article for the meetup : Reading Weekly Meetups

WHEN:
14 October 2015 05:00:00PM (+0100)

WHERE:
Reading, England

The Reading meetup is looking likely to be a weekly thing, so we'll be meeting in the same place at the same time as last week. We may use the time to discuss a better place and time to hold the meetups, as this doesn't work well for everyone. For now, we're still in the Starbucks next to Vue cinema and the river Kennet, in the Oracle Shopping Centre (Riverside Entrance). I (amongst others) will be there from 5pm, with a whiteboard with LessWrong written on it and the drawing of Clippy from last week.

In the event that introductions and subsequent interesting discussions don't take up all our time, I have "motivation and long term goals" jotted down as a conversation topic. Hopefully anyone who missed the last one will be able to come to this one, so we'll be able to all introduce ourselves and discuss an optimal scheduling as a group. I am a bad meetup organiser for not putting this up sooner, and for that I am sorry. If you couldn't make it to either but you're in the area, please do comment and let us know a time that would be better, and we'll try and take you into account!

https://www.google.co.uk/maps/place/The+Oracle+Shopping+Centre/@51.45308,-0.971488,3a,75y,90t/data=!3m8!1e2!3m6!1s7305926!2e1!3e10!6s%2F%2Fstorage.googleapis.com%2Fstatic.panoramio.com%2Fphotos%2Fsmall%2F7305926.jpg!7i2048!8i1536!4m2!3m1!1s0x48769b15e736aff1:0x91a4b0447e720259!6m1!1e1 <--- As last week, if you're lost, then if the area you're in looks like this, you are in vaguely the right place!

Discussion article for the meetup : Reading Weekly Meetups
Unbounded linear utility functions?
by snarles — 2015-10-11T23:30:14.763Z

The LW community seems to assume, by default, that "unbounded, linear utility functions are reasonable." That is, if you value the existence of 1 swan at 1.5 utilons, then 10 swans should be worth 15, etc.

Yudkowsky in his post on scope insensitivity argues that nonlinearity of personal utility functions is a logical fallacy.

However, unbounded and linearly increasing utility functions lead to conundrums such as Pascal's Mugging. A recent discussion topic on Pascal's Mugging suggests ignoring probabilities that are too small. However, such extreme measures are not necessary if tamer utility functions are used: one images a typical personal utility function to be bounded and nonlinear.

In that recent discussion topic, V_V and I questioned the adoption of such an unbounded, linear utility function. I would argue that nonlinear of utility functions is not a logical fallacy.

To make my case clear, I will clarify my personal interpretation of utilitarianism. Utility functions are mathematical constructs that can be used to model individual or group decision-making. However, it is unrealistic to suppose that every individual actually has an utility function or even a preference ordering; at best, one could find a utility function which approximates the behavior of the individual. This is confirmed by studies demonstrating the inconsistency of human preferences. The decisions made by coordinated groups: e.g. corporate partners, citizens in a democracy, or the entire community of effective altruists could also be more or less well-approximated by a utility function: presumably, the accuracy of the utility function model of decision-making depends on the cohesion of the group. Utilitarianism, as proposed by Bentham and Mills, proposes an ethical framework based on some idealized utility function. Rather than using utility functions to model group decision-making, Bentham and Mills propose to use some utility function to guide decision-making, in the form of an ethical theory. It is important to distinguish these two different use-cases of utility functions, which might be termed descriptive utility and prescriptive utility.

But what is ethics? I hold the hard-nosed position that moral philosophies (including utiliarianism) are human inventions which serve the purpose of facil...
Simulations Map: what is the most probable type of the simulation in which we live?
by turchin — 2015-10-11T05:10:34.535Z

There is a chance that we may be living in a computer simulation created by an AI or a future super-civilization. The goal of the simulations map is to depict an overview of all possible simulations. It will help us to estimate the distribution of other multiple simulations inside it along with their measure and probability. This will help us to estimate the probability that we are in a simulation and – if we are – the kind of simulation it is and how it could end.

Simulation argument

The simulation map is based on Bostrom’s simulation argument. Bostrom showed that that “at least one of the following propositions is true:

(1) the human species is very likely to go extinct before reaching a “posthuman” stage;

(2) any posthuman civilization is extremely unlikely to run a significant number of simulations of their evolutionary history (or variations thereof);

(3) we are almost certainly living in a computer simulation”. http://www.simulation-argument.com/simulation.html

The third proposition is the strongest one, because (1) requires that not only human civilization but almost all other technological civilizations should go extinct before they can begin simulations, because non-human civilizations could model human ones and vice versa. This makes (1) extremely strong universal conjecture and therefore very unlikely to be true. It requires that all possible civilizations will kill themselves before they create AI, but we can hardly even imagine such a universal course. If destruction is down to dangerous physical experiments, some civilizations may live in universes with different physics; if it is down to bioweapons, some civilizations would have enough control to prevent them.

In the same way, (2) requires that all super-civilizations with AI will refrain from creating simulations, which is unlikely.

Feasibly there could be some kind of universal physical law against the creation of simulations, but such a law is impossible, because some kinds of simulations already exist, for example human dreaming. During human dreaming very precise simulations of the real world are created (which can’t be distinguished from the real world from within – that is why lucid dreams are so rare). So, we could conclude that after small genetic manipulations it is possible to create a brain that will be 10 times m...
Simply locate yourself
by So8res — 2015-10-11T03:00:00.000Z
Link: https://mindingourway.com/simply-locate-yourself/

Imagine I offer you the following bet: I'll roll a fair ten-sided die. If it comes up 1-9, you win a million dollars. If it comes up 0, you lose $10,000. (If you're significantly richer or poorer than the median person, adjust the numbers up or down accordingly, such that winning is very great and losing hurts a lot, but is manageable.) Imagine that you take the bet, because those odds are ridiculously in your favor. Now imagine that I roll the die, and you watch it rolling, and rolling, and rolling, until it starts to settle, and then it settles… on 0.

Imagine the sinking feeling you might get, as you see the zero, and realize that you have to give me ten thousand dollars. Maybe you suddenly feel uncomfortable. Maybe you're unwilling to meet my gaze. Maybe you're angry, or slightly sick to your stomach. Maybe some part of you is pushing against reality, trying to deny it, willing the past to change.

Now imagine a second bet. This time, imagine a world that has figured out cloning and cryonics and space travel. The bet works as follows: I put you to sleep, and then I separate you into ten identical copies (none of which have any more claim to being the original than any other), and then I put them all into stasis. Your possessions are replicated ten ways, and the ten yous are put on ten ships to ten different (already-colonized) planets. On nine of those planets, the local you will be placed in a room with blue walls, and given your possessions along with a million extra dollars. On one of those planets, the local you will be placed in a room with red walls, and will have $10,000 removed from their possessions. Then all ten yous will be awoken. Thus, nine copies of you will gain a million dollars, and one copy of you will lose ten thousand dollars.

Imagine that you understand this procedure, and consent to it. You're put to sleep, and split into ten copies, put into stasis, sent to ten planets, and revived from stasis. You wake slowly, and haven't opened your eyes yet. You know that nine yous will wake in a blue room and find themselves rich, and one you will wake in a red room and find themselves poor, and you don't know which you you are. You open your eyes, and the walls are… red.

In one sense, you've lost exactly the same sort of bet as the first bet. But there's a very different way that you might be feeling. In the second bet, instead of feeling a sinking ...
Conflicting advice on altruism
by leplen — 2015-10-11T00:25:10.268Z

As far as I can tell, rather than having a single well-defined set of preferences or utility function, my actions more closely reflect the outcome of a set of competing internal drives. One of my internal drives is strongly oriented towards a utilitarian altruism. While the altruist internal drive doesn't dominate my day-to-day life, compared to the influence of more basic drives like the desires for food, fun, and social validation, I have traditionally been very willing to drop whatever I'm doing and help someone who asks for, or appears to need help. This altruistic drive has an even more significant degree of influence on my long term planning, since my drives for food, fun, etc. are ambivalent between the many possible futures in which they can be well-satisfied.

I'm not totally sure to what extent strong internal drives are genetic or learned or controllable, but I've had a fairly strong impulse towards altruism for well over a decade. Unfortunately, even over fairly long time frames it isn't clear to me that I've been a particularly "effective" altruist. This discussion attempts to understand some of the beliefs and behaviors that contributed to my personal failure/success as an altruist, and may also be helpful to other people looking to engage in or encourage similar prosocial habits.

Game Theory Model

Imagine a perfect altruist competing in a Prisoner's Dilemma style game. The altruist in this model is by definition a pure utilitarian who wants to maximize the average utility, but is completely insensitive to the distribution of the utility.1 A trivial real world example similar to this would be something like picking up litter in a public place. If the options are Pick up (Cooperate) and Litter (Defect) then an altruist might choose to pick up litter even though they themselves don't capture enough of the value to justify the action. Even if you're skeptical that unselfish pure utilitarians exist, the payoff matrix and much of this analysis applies to a broader range of prosocial behaviors where it's difficult for a single actor to capture the value he or she generates.

The prisoner's dilemma payoff matrix for the game in which the altruist is competing looks something like this:

Agent B

Agent A
Cooperate
Defect

Cooperate
2,2
-2,4

Defect
4,-2
-1,-1

Other examples with altered payoff ratios are possible, but this par...
Aumann Agreement Game
by abramdemski — 2015-10-09T17:14:00.225Z

I've written up a rationality game which we played several times at our local LW chapter and had a lot of fun with. The idea is to put Aumann's agreement theorem into practice as a multi-player calibration game, in which players react to the probabilities which other players give (each holding some privileged evidence). If you get very involved, this implies reasoning not only about how well your friends are calibrated, but also how much your friends trust each other's calibration, and how much they trust each other's trust in each other.

You'll need a set of trivia questions to play. We used these.

The write-up includes a helpful scoring table which we have not play-tested yet. We did a plain Bayes loss rather than an adjusted Bayes loss when we played, and calculated things on our phone calculators. This version should feel a lot better, because the numbers are easier to interpret and you get your score right away rather than calculating at the end.