trentmkelly/LessWrong-43k huggingface dataset:
Flashes of Nondecisionmaking
by lionhearted (Sebastian Marshall) — 2014-01-27T14:30:26.937Z

If you crash a bicycle and cut your knee, it bleeds. You can apply pressure to the wound or otherwise aid in clotting it, but you can't fully control the blood. You can't think, "Body! I command you not to bleed!" Nor can you directly say, "I choose not to bleed" through pure will alone.

This is easy enough to understand. We don't have direct control over our blood. We can apply some measure of indirect to it -- taking aspirin might thin the blood, breathing deeply and relaxing might slow the pulse and the flow of blood slightly -- but we do not have direct and instant control over the flow of our blood.

That's our blood. It's quite a personal thing, when you think about it.

At the same time, there's a view that we have full control and choice over our actions in a given situation.

I no longer believe this to be the case.

We can staunch the flow of bleeding through applying pressure, a cloth, perhaps slowing down our pulse and bloodflow through lowering stress and deep breathing. But we can't, in the moment, command or control blood by force of will or mind alone.

Likewise, I'm starting to believe we have lots of indirect control over our patterns of action in our lives, but perhaps less control and command in individual moments.

When a person rolls out of bed, they usually do very similar things each morning. How much control or command do they have -- mentally or analytically or however you want to define it -- over these actions?

Not much, I'd say.

Yet, they have immense indirect control, similar to blood flow. If you normally lay out your clothes the night before, and you lay out running clothes instead of work clothes, and set your alarm for an hour earlier, your chances of running go up a lot. There still may be an element of choice or self-command when you decide to run or not, but it's very possible there wasn't choice or self-command available if you did not rearrange your environment with that sort of indirect pressure.

I had an experience recently that was incredibly distressing. It was strange and very unpleasant at the time, but I'm now thankful for it.

I was at a convenience store when I realized I was in the process of buying some junk food and energy drinks.

My mind recognized this, but seemingly had not so much say on what's going on. My legs were just walking the familiar convenience store aisles near my home, picking up two of...
Meetup : Rationality Vienna
by Ratcourse — 2014-01-27T13:51:02.219Z

Discussion article for the meetup : Rationality Vienna

WHEN:
15 February 2014 03:00:00PM (+0100)

WHERE:
Vienna, Reichsratstraße 17

Manuel will talk about Universal Artificial Intelligence, which is the theory of perfectly rational agents. It claims to once-and-for-all define what rationality, intelligence, success, understanding etc. really are. Enough flamebait for a lively discussion There will be some maths presented for those who care about such things, but the math is not necessary to understand the theory.

Discussion article for the meetup : Rationality Vienna
Skepticism about Probability
by Carinthium — 2014-01-27T09:49:08.814Z

I've raised arguments for philosophical scepticism before, which have mostly been argued against in a Popper-esque manner of arguing that even if we don't know anything with certainty, we can have legitimate knowledge on probabilities.

The problem with this, however, is how you answer a sceptic about the notion of probability having a correlation with reality. Probability depends upon axioms of probability- how are said axioms to be justified? It can't be by definition, or it has no correlation to reality.
Google may be trying to take over the world
by  — 2014-01-27T09:33:57.943Z

So I know we've already seen them buying a bunch of ML and robotics companies, but now they're purchasing Shane Legg's AGI startup. This is after they've acquired Boston Dynamics, several smaller robotics and ML firms, and started their own life-extension firm.

Is it just me, or are they trying to make Accelerando or something closely related actually happen? Given that they're buying up real experts and not just "AI is inevitable" prediction geeks (who shall remain politely unnamed out of respect for their real, original expertise in machine learning), has someone had a polite word with them about not killing all humans by sheer accident?
How to make AIXI-tl incapable of learning
by itaibn0 — 2014-01-27T00:05:35.767Z

Consider a simple game: You are shown a random-looking 512-bit string h. You may then press one of two buttons, labeled '0' and '1'. No matter which button you press, you will then be shown a 256-bit string s such that SHA512(s) = h. In addition, if you pressed '1' you are given 1$.
This game seems pretty simple, right? s and h are irrelevant, and you should simply press '1' all the time (I'm assuming you value recieving money). Well, let's see how AIXI and AIXI-tl fare at this game.
Let's say the machine already played the game many times. Its memory is h0, b0, r0, s0, h1, ..., b_(n-1), r_(n-1), s_(n-1), h_n, where the list is in chronological order, inputs are unbolded while decisions are bolded, and r_i is the reward signal. It is always the case that r_i=b_i and h_i=SHA512(s_i).
First let's look at AIXI. It searches for models that compress and extrapolate this history up to the limit of its planning horizon. One class of such models is this: there is a list s0, ..., s_N of random 256-bits strings and a list b0, ..., b_N of (possibly compressible) bits. The history is SHA512(s0), b0, b0, s0, ..., b_(N-1), s_(N-1), SHA512(s_N), b_N. Here s_i for i<n must match the s_i in its memory, and s_n must be the almost certainly unique value with SHA512(s_n) = h_n. While I don't have a proof, it intuitively seems like this class of models will dominate the machines probability mass, and repeated arg-max should lead to the action of outputting 1. It wouldn't always do this due to exploration/exploitation considerations and due to the incentive to minimize K (b0, ... b_N) built into its prior, but it should do it most of the time. So AIXI seems good.
Now let's consider AIXI-tl. It picks outputs by having provably correct programs assign lower bounds to the expected utility, and picking the one with the best lower bound, where expected utility is as measured by AIXI. This would include accepting the analysis I just made with AIXI if that analysis can be made provably accurate. Here lies a problem: the agent has seen h_n but hasn't seen s_n. Therefore, it can't be certain that there is an s_n with SHA512(s_n)=h_n. Therefore, it can't be certain that the models used for AIXI actually works (this isn't a problem for AIXI since it has infinite computational power and can always determine that there is such an s_n).
There is an ad hoc fix for this for AIXI-tl: Take the same ...
How to make AIXI-tl incapable of learning
by itaibn0 — 2014-01-26T23:57:56.943Z

Consider a simple game: You are shown a random-looking 512-bit string h. You may then press one of two buttons, labeled '0' and '1'. No matter which button you press, you will then be shown a 256-bit string s such that SHA512(s) = h. In addition, if you pressed '1' you are given 1$.
This game seems pretty simple, right? s and h are irrelevant, and you should simply press '1' all the time (I'm assuming you value recieving money). Well, let's see how AIXI and AIXI-tl fare at this game.
Let's say the machine already played the game many times. Its memory is h0, b0, r0, s0, h1, ..., b_(n-1), r_(n-1), s_(n-1), h_n, where the list is in chronological order, inputs are unbolded while decisions are bolded, and r_i is the reward signal. It is always the case that r_i=b_i and h_i=SHA512(s_i).
First let's look at AIXI. It searches for models that compress and extrapolate this history up to the limit of its planning horizon. One class of such models is this: there is a list s0, ..., s_N of random 256-bits strings and a list b0, ..., b_N of (possibly compressible) bits. The history is SHA512(s0), b0, b0, s0, ..., b_(N-1), s_(N-1), SHA512(s_N), b_N. Here s_i for i<n must match the s_i in its memory, and s_n must be the almost certainly unique value with SHA512(s_n) = h_n. While I don't have a proof, it intuitively seems like this class of models will dominate the machine's probability mass, and repeated arg-max should lead to the action of outputting 1. It wouldn't always do this due to exploration/exploitation considerations and due to the incentive to minimize K (b0, ... b_N) built into its prior, but it should do it most of the time. So AIXI seems good.
Now let's consider AIXI-tl. It picks outputs by having provably correct programs assign lower bounds to the expected utility of these outputs, where the expected utility is defined as it is in AIXI. This would include accepting the analysis I just made with AIXI if that analysis can be made provably accurate. Here lies a problem: the agent has seen h_n but hasn't seen s_n. Therefore, it can't be certain that there is an s_n with SHA512(s_n)=h_n. Therefore, it can't be certain that the models used for AIXI actually works (this isn't a problem for AIXI since it has infinite computational power and can always determine that there is such an s_n).
There is an ad hoc fix for this for AIXI-tl: Take the same model as before, but...
Volunteering programmer hours / discussing how to improve LessWrong's software 
by ChrisHallquist — 2014-01-26T22:47:01.960Z

There's been some discussion of how to improve the structure of LessWrong at the site software level - for example adding subreddits or modifying how main and discussion work. One roadblock to this that's been mentioned is a shortage of programmer hours. I'd like to volunteer mine.

I recently finished a course on web development in which, among other things, I build a Reddit clone using Ruby on Rails and Backbone.js. It's been several months since I've written any Python, and I'm somewhat wary of the time required to get familiar with the LessWrong codebase, but I think think the time would be worth it for me: it could potentially improve LessWrong a lot and would let me tick off my "have contributed to an open source project" box.

Of course, before any of that happens, there needs to be some agreement on what changes we think would be a good idea. So... discuss.
EDIT: For context, it's been suggested that part of the benefit of subforums is it could defuse debates over "what topics are appropriate for LessWrong." We could even have an "off-topic" subforum, a common feature of online discussion forums - I think bringing the format of LessWrong more into line with what's standard on other websites could help newbies be less confused here.
Foundations of Probability
by Manfred — 2014-01-26T19:29:42.378Z

Beginning of: Logical Uncertainty sequence

Suppose that we are designing a robot. In order for this robot to reason about the outside world, it will need to use probabilities.

Our robot can then use its knowledge to acquire cookies, which we have programmed it to value. For example, we might wager a cookie with the robot on the motion of a certain stock price.

In the coming sequence, I'd like to add a new capability to our robot. It has to do with how the robot handles very hard math problems. If we ask "what's the last digit of the 3^^^3'th prime number?", our robot should at some point give up, before the sun explodes and the point becomes moot.

If there are math problems our robot can't solve, what should it do if we offer it a bet about the last digit of the 3^^^3'th prime? It's going to have to approximate - robots need to make lots of approximations, even for simple tasks like finding the strategy that maximizes cookies.

Intuitively, it seems like if we can't find the real answer, the last digit is equally likely to be 1, 3, 7 or 9; our robot should take bets as if it assigned those digits equal probability. But to assign some probability to the wrong answer is logically equivalent to assigning probability to 0=1. When we learn more, it will become clear that this is a problem - we aren't ready to upgrade our robot yet.

Let's begin with a review of the foundations of probability.

What I call foundations of probability are arguments for why our robot should ever want to use probabilities. I will cover four of them, ranging from the worldly ("make bets in the following way or you lose money") to the ethereal ("here's a really elegant set of axioms"). To use the word "probability" to describe the subject of such disparate arguments can seem odd, but keep in mind the naive definition of probability as that number that's 1/6 for a fair die rolling 6 and 30% for clear weather tomorrow.

Dutch Books

The concretest of concrete foundations is the Dutch book arguments. A Dutch book is a collection of bets that is certain to lose you money. If you violate the rules of probability, you'll agree to these certain-loss bets (or not take a certain-win bet).

For example, if you think that each side of the coin has a 55% chance of showing up, then you'll pay $1 for a bet that pays out $0.98 if the coin lands heads and $0.98 if the coin lands tails. If taking be...
How to become a PC?
by DataPacRat — 2014-01-26T18:49:45.644Z

"Cryonics has a 95% chance of failure, by my estimation; it would be downright /embarrassing/ to die on the day before real immortality is discovered. Thus, I want to improve my general health and longevity."

That thought has gotten me through three weeks of gradually increasing exercise and diet improvement (I'm eating an apple right now) - but my enthusiasm is starting to flag. So I'm looking for new thoughts that will help me keep going, and keep improving. A few possibilities that I've thought of:

Pride: "If I'm so smart, then I should be able to do /better/ than those other people who don't even know about Bayesian updates, let alone the existence of akrasia..."

Sloth: "If I stop now, it's going to be /so much/ harder and more painful to start up again, instead of just keeping on keeping on..."

Desire: "I already like hiking and camping - if I keep this up, I'll be able to carry enough weight to finally take that long trip I've occasionally considered..."

Curiosity: "I'm as geeky a nerd as you can find. I wonder how far I can hack my own body?"

Pride again: "I already keep a hiker's first-aid kit in my pocket, and make other preparations for events that happen rarely. How stupid do I have to be not to put at least that much effort into making my everyday life easier?"

Does anyone have any experience in such self-motivation? Does this set of mental tricks seem like a sufficiently viable approach? Are there any other approaches that seem worth a shot?
Productivity as a function of ability in theoretical fields
by Stefan_Schubert — 2014-01-26T13:16:15.873Z

I argued in this post that the differences in capability between different researchers are vast (Kaj Sotala provided me with some interesting empirical evidence that backs up this claim). Einstein's contributions to physics or John von Neumann's contributions to mathematics (and a number of other disciplines) are arguably at least hundreds of times greater than that of an average physicist or mathematician.

At the same time, Yudkowsky argues that "in the space of brain designs" the difference between the village idiot and Einstein is tiny. Their brains are extremely similar, with the exception of some "minor genetic tweaks". Hence we get the following picture:

The picture I am painting is rather something like this:

It would seem that these pictures are incompatible - something that would be a problem for my picture, since I think that Yudkowsky's picture is right. So how can they both be true? The answer is, obviously, that they are measuring different things. The first is measuring something like difference in brain design that is relevant for intelligence. The second is rather measuring the difference in capability to come up with physical theories that are of use for mankind. Here the village idiot is on par with the chimp and the mouse - all of whom have no such capability whatsover. The average physicist has some such capability, but it's just a fraction of Einstein's.

Why is this? Well it is not because the village idiot has no capability at all to come up with physical theories. In fact, a primitive physical theory that is quite useful is hard-wired into our brains. Rather, the reason is that the village idiot has no capability to come up with a physical theory that is not already well-known.

Problems in theoretical physics and mathematics are typically problems that are so complex that they are hard to solve for some of the world's smartest people. This means that unless you're quite smart, your chances of contributing anything at at all to these disciplines is very slim. But, if you are but a tiny bit smarter than everyone else, you'll be able to spot solutions to problem after problem that others have struggled with - these problems being problems precisely because they were hard to solve for people with a certain level of intelligence. Thus we get something like the following relationship between cognitive ability, in Yudkowsky

...
Applying reinforcement learning theory to reduce felt temporal distance
by Kaj_Sotala — 2014-01-26T09:17:29.099Z

(cross-posted from my blog)

It is a basic principle of reinforcement learning to distinguish between reward and value, where the reward of a state is the immediate, intrinsic desirability of the state, whereas the value of the state is proportional to the rewards of the other states that you can reach from that state.

For example, suppose that I’m playing a competitive game of chess, and in addition to winning I happen to like capturing my opponent’s pieces, even when it doesn’t contribute to winning. I assign a reward of 10 points to winning, -10 to losing, 0 to a stalemate, and 1 point to each piece that I capture in the game. Now my opponent offers me a chance to capture one of his pawns, an action that would give me one point worth of reward. But when I look at the situation more closely, I see that it’s a trap: if I did capture the piece, I would be forced into a set of moves that would inevitably result in my defeat. So the value, or long-term reward, of that state is actually something close to -9.
Once I realize this, I also realize that making that move is almost exactly equivalent to agreeing to resign in exchange for my opponent letting me capture one of his pieces. My defeat won’t be instant, but by making that move, I would nonetheless be choosing to lose.
Now consider a dilemma that I might be faced with when coming home late some evening. I have no food at home, but I’m feeling exhausted and don’t want to bother with going to the store, and I’ve already eaten today anyway. But I also know that if I wake up with no food in the house, then I will quickly end up with low energy, which makes it harder to go to the store, which means my energy levels will drop further, and so on until I’ll finally get something to eat much later, after wasting a long time in an uncomfortable state.
Typically, temporal discounting means that I’m aware of this in the evening, but nonetheless skip the visit to the store. The penalty from not going feels remote, whereas the discomfort of going feels close, and that ends up dominating my decision-making. Besides, I can always hope that the next morning will be an exception, and I’ll actually get myself to go to the store right from the moment when I wake up!
And I haven’t tried this out for very long, but it feels like explicitly framing the different act...
Solomonoff induction without perfect memory
by Squark — 2014-01-25T11:55:44.072Z

In this post I construct a variant of Solomonoff induction which allows for agents with imperfect memory.

Solomonoff induction is a mathematical formalization of Occam's razor, and is supposed to be a "master equation" of epistemic rationality. In order words, forming rational expectations about the future is supposed to be tantamount to computing Solomonoff induction (approximately, since it's incomputable).

Solomonoff induction operates by assigning probabilities to continuations of a given finite sequence . In the simplest formalism, the sequence elements are bits. represents past observations and represents a possible future.

A rational agent is supposed to use to evaluate the probability of . There are two problems with this. One is that is uncomputable whereas has a limited amount of computing resources in its disposal. For now, we are going to ignore this. Another problem is that doesn't have direct access to . can only estimate from 's memory. Moreover, this estimation depends on knowledge has about reality which is supposed to be generated by Solomonoff induction itself. In other words, inferring the future from the past only makes sense in the approximation of perfect memory, in general we need to be able to infer both the past and the future from the present.

The bit-sequence formalism is not well-adapted to this, since the present state of has to contain all of 's memory about the past so it has to be more than a single bit. I suggest solving this by considering sequences of natural numbers instead of bit-sequences . Generalizing regular Solomonoff induction to natural numbers is straight-forward: the random program computes convergent sequences of lower bounds for the transition probabilities

Now we want a new induction procedure whose input is a single natural number representing the state of 's consciousness which allows assigning probabilities to sequences of natural numbers containing . We achieve this by assigning the following probabilities to Solomonoff hypotheses :

Here, is a normalization factor, stands for expectation value and is defined by

where is the infinite sequence of natural numbers "gener...
Meetup : Washington DC fun and games meetup
by rocurley — 2014-01-25T06:49:27.698Z

Discussion article for the meetup : Washington DC fun and games meetup

WHEN:
26 January 2014 03:00:00PM (-0500)

WHERE:
National Portrait Gallery, Washington, DC 20001, USA

We'll be meeting to hang out and play games.
(Sorry for the late notice, I forgot that the Learn-To-Code thing wasn't actually the meetup.)

Discussion article for the meetup : Washington DC fun and games meetup
Meetup : Urbana-Champaign: Probability Discussion
by Manfred — 2014-01-25T03:16:09.989Z

Discussion article for the meetup : Urbana-Champaign: Probability Discussion

WHEN:
26 January 2014 02:00:00PM (-0600)

WHERE:
412 W. Elm St, Urbana, IL

Let's have a meetup! I'm writing an article on the foundations of probability - why having probabilities is a good idea for agents. So that would be a good thing to talk about. And then, other things!

Discussion article for the meetup : Urbana-Champaign: Probability Discussion