trentmkelly/LessWrong-43k huggingface dataset:
Testing Rationality Apps for Science
by BayesianMind — 2018-12-24T10:46:20.229Z

TLDR: Do you want to contribute to science by trying out new productivity apps? Then send a mail to re-experiments-on@tuebingen.mpg.de.

Longer version: Do you want to achieve your personal goals more effectively, become more focused and productive at work, improve your planning skills, or learn how to make better decisions?

Then we have good news for you: You can be the first to try out our brand-new productivity and self-improvement apps by participating in our research.

We regularly conduct online studies and are looking for participants willing to try out our productivity apps. You would be amongst the first to test our newly developed innovative applications that foster personal growth and/or productivity.

We are the Max Planck Research Group for Rationality Enhancement at the MPI for Intelligent Systems in Tübingen. Our scientific mission is to lay the cognitive and technological foundations for helping people become more effective. To this end, we investigate how people learn how to think, develop cognitively-informed machine learning methods for discovering rational heuristics, and build cognitive tutors that help people learn faster and cognitive prostheses that assist people in goal setting and goal achievement.

If you would like to contribute to our research and participate in our upcoming studies, please subscribe to our mailing list by sending an email to re-experiments-on@tuebingen.mpg.de.

If you can think of other people or communities who might be interested in using our apps and participating in our research, then please feel to spread the word by sharing or forwarding this message to them.

Many thanks in advance for your support!
State Machines and the Strange Case of Mutating API
by Martin Sustrik — 2018-12-24T05:50:00.599Z

State machines are widely used to implement network protocols, or, generally, objects that have to react to external events.

Consider TCP state machine:

During its lifetime TCP socket moves throught different states in the diagram. When you start connecting it's in SYN SENT state, when the initial handshake is over, it's in ESTABLISHED state and so on.

And here's an interesting observation: The API of the socket changes as you move from one state to another. For example, it doesn't make sense to receive data while you are still connecting. But once you are connected, receiving data is all right.

To give a more illustrative example, have a look at SOCKS5 protocol. It's basically a TCP or UDP proxy protocol. It's used, for example, by Tor. It starts with authentication phase, supporting different kinds of authentication. Then it moves to connection establishment phase. Once again there are different ways to connect. You can connect to an IPv4 address, to a IPv6 address or to a hostname. Finally, the state machine moves to one of the working states. This can be a TCP connection or an UDP connection.

Note how API changes between the states. In CLOSED state you can call functions such as connect_unauthenticated or connect_password. In AUTHENTICATED state you can call connect_tcp, bind_tcp or open_udp. In TCP ESTABLISHED you can do normal stream socket operations, while in UDP ESTABLISHED you can do datagram operations.

This requirement of mutating API is at odds with how the state machines are normally implemented: There's a single object representing the connection during it's entire lifetime. Therefore, a single object must support different APIs.

What it leads to is code like this:

void Socks5::connect_tcp(Addr addr) { if(state != AUTHENTICATED) throw "Cannot connect is this state."; ... }

Which, let's be frank, is just an implementation of dynamically typed language on top of statically-typed one.

In other words, by implementing state machines this way we are giving up proper type checking. While compiler would be perfectly able to warn us if connect_tcp was called in CLOSED state, we give up on the possibility and we check the constraint at runtime.

This sounds like bad coding style, but it turns out that the programming languages we use fail to provide tools to handle this kind of scenarios. It's not networ...
[Video] Why Not Just: Think of AGI Like a Corporation? (Robert Miles)
by habryka — 2018-12-23T21:49:06.438Z
Link: https://www.youtube.com/watch?v=L5pUA3LsEaw

Robert Miles has been creating AI-Alignment related videos for a while now, but I found this one particularly good.

Here is the automatically generated Youtube transcript. Obviously it's not very good, but at least it makes the post searchable (In case Robert reads this and has a transcript for the video lying around, I would love to replace this with one that has proper capitalization and punctuation marks and other luxuries):

hi so I sometimes see people saying

things like okay so your argument is

that at some point in the future we're

going to develop intelligent agents that

are able to reason about the world in

general and take actions in the world to

achieve their goals

these agents might have superhuman

intelligence that allows them to be very

good at achieving their goals and this

is a problem because they might have

different goals from us but don't we

kind of have that already corporations

can be thought of as super intelligent

agents they're able to think about the

world in general and they can outperform

individual humans across a range of

cognitive tasks and they have goals

namely maximizing profits or shareholder

value or whatever and those goals aren't

the same as the overall goals of

humanity so corporations are a kind of

misaligned super intelligence the people

who say this having established the

metaphor at this point tend to diverge

mostly along political lines some say

corporations are therefore a clear

threat to human values and goals in the

same way that misaligned super

intelligences are and they need to be

much more tightly controlled if not

destroyed all together others say

corporations are like misaligned super

intelligences but corporations have been

instrumental in the huge increases in

human wealth and well-being that we've

seen over the last couple of centuries

with pretty minor negative side effects

overall if that's the effect of

misaligned super intelligences I don't

see why we should be concerned about AI

and others say corporations certainly

have their problems but we seem to have

developed systems that keep them under

control well enough that they're able to

create value and do useful things

without literally killing everyone so

perhaps we can learn something about how

to control or align super intelligences

by looking at how we handle corporations

so we're gonna let the first to fight

amongst themselves and we'll talk ...
Boston Solstice 2018 Retrospective
by jefftk — 2018-12-23T20:04:40.244Z

Last night we hosted the Boston Secular Solstice, which I wrote about planning a couple months ago. It felt like it went really well! Several people told me it was their favorite one we've had in Boston.

The past few years we've been at the MIT Chapel. It's a really nice space for this sort of event, but you need more people for it to feel right. Comfortably full is a much better feeling than spread thin. This year we decided to have it at my house, and I was thinking we might have ~20 people.

A few days before I was looking at a fb event with 36 "yes" and 61 "interested", plus several other people who had told me they were coming. I tried to figure out how to configure our space for maximum capacity:

This meant a lot of moving things around, but I think mostly worked. I set up boards across a few chairs as a way to get a bit denser seating than you can with just chairs alone, and I set up a futon without the frame to get people in front a bit lower:

Panoramic view:

We ended up having ~40 people, which was a lot. There were empty seats, but only ones that had basically no ability to see the projector. At the beginning I told people how to follow along on their phones if they couldn't see well, but I'm not sure whether that was useful.

To get the projector image high enough on the wall without blocking the view of people behind it, I put up a temporary shelf:

It's back down now, though I still need to spackle the holes. Helpful things about having your own place!

The overall structure we followed was about 20min of people arriving and getting settled, with snacks, and then a first half of relatively light things. We took a break for intermission, with a dessert potluck, and then when we came back we did progressively darker things culminating in a reading of Beyond the Reach of God and singing Brighter than Today.

We ended with a few more songs and one more reading, and then people hung out for ~2hr until I pushed the stragglers out so we could go to bed.

The program was:

Welcome
All sing: Lean on Me (mp3)
Introduction to Why Did the Sun Shine?
All sing: Why Did the Sun Shine? (mp3)
Reading: What I Learned About Risk from Parenting, adapted
All sing: Uplift (mp3)
Introduction to Her Mysteries
All sing: Her Mysteries (mp3)
Reading: Pale Blue Dot
All sing: Chasing Patterns (mp3)
Introduction to Why Does the Sun Shine?
All sing: Why D

...
Why Don't Creators Switch to their Own Platforms?
by Jacob Falkovich — 2018-12-23T04:46:47.047Z

Almost every content creator rationalists follow owns their platform: podcasters like Sam Harris and the Julia Galef, bloggers like Scott (and myself), all the nerdy webcomics. And yet, outside the rationalsphere every creator seems engaged in an endless fight against censorship and harassment by the platforms that are supposed to enable them: Facebook, Twitter, Tumblr, Patreon... So why do they stay on those platforms? Other than Sam Harris giving Patreon the middle finger, no one else seems to do much except protest platforms on the platforms themselves.

This questions really came up for me after reading the saga of Pewdiepie and YouTube. Currently, pewdiepie.com redirects to his YouTube page, where he posts videos protesting YouTube. This is crazy. The technology that YouTube provides was hard to build when YouTube started a decade and a half ago, but surely today it's not a huge challenge. PDP has 20 billion total views. He doesn't need traffic from the algorithm suggesting his videos, everyone else is trying to game the algorithm to get redirected by PDP! Switching to his own platform would allow him to capture a higher percentage of revenue, be immune to any kind of censorship, and make him a legend if he starts an exodus from YouTube. He can host all the other non-PC comedians on his own platform. How is that not worth losing a bit of traffic as viewers readjust?
Cognitive Bias of AI Researchers?
by Mindey — 2018-12-22T09:20:52.045Z

I find it inconvenient that many AI discussions circle around "agents", “environments” and “goals”. These are non-mathematical words, and by using this vocabulary we are anthropomorphizng natural world's phenomena.
While an "agent" may well be a set of interacting processes, that produce emergent phenomena (like volition, cognition, action), it is not a fundamental and pragmatic mathematical concept. The truly fundamental pragmatic mathematical concepts may be:
(1) states: a set of possible world states (each being a sets of conditions).
(2) processes: a set of world processes progressing towards some of those states.
If so, how could we avoid that anthropomorphic cognitive bias in our papers, and discussions?

Would the (1), (2) terms be a good alternative for our discussions, and to express the ideas in most AI research papers? E.g., Bob is a process, and Alice is processes,... and they collectively are progressing towards some desired state convergent state, defined by process addition.

What fundamental concepts would you consider to be a better alternative to talk formally about the AI domain?
Spaghetti Towers
by eukaryote — 2018-12-22T05:29:47.551Z
Link: https://eukaryotewritesblog.com/2018/12/21/spaghetti-towers/

Here’s a pattern I’d like to be able to talk about. It might be known under a certain name somewhere, but if it is, I don’t know it. I call it a Spaghetti Tower. It shows up in large complex systems that are built haphazardly.

Someone or something builds the first Part A.

Later, someone wants to put a second Part B on top of Part A, either out of convenience (a common function, just somewhere to put it) or as a refinement to Part A.

Now, suppose you want to tweak Part A. If you do that, you might break Part B, since it interacts with bits of Part A. So you might instead build Part C on top of the previous ones.

And by the time your system looks like this, it’s much harder to tell what changes you can make to an earlier part without crashing some component, so you’re basically relegated to throwing another part on top of the pile.

I call these spaghetti towers for two reasons: One, because they tend to quickly take on circuitous knotty tangled structures, like what programmers call “spaghetti code”. (Part of the problem with spaghetti code is that it can lead to spaghetti towers.)

Especially since they’re usually interwoven in multiple dimensions, and thus look more like this:

“Can you just straighten out the yellow one without touching any of the others? Thanks.”

Second, because shortsightedness in the design process is a crucial part of spaghetti machines. In order to design a spaghetti system, you throw spaghetti against a wall and see if it sticks. Then, when you want to add another part, you throw more spaghetti until it sticks to that spaghetti. And later, you throw more spaghetti. So it goes. And if you decide that you want to tweak the bottom layer to make it a little more useful – which you might want to do because, say, it was built out of spaghetti – without damaging the next layers of gummy partially-dried spaghetti, well then, good luck.

Note that all systems have load-bearing, structural pieces. This does not make them spaghetti towers. The distinction about spaghetti towers is that they have a lot of shoddily-built structural components that are completely unintentional. A bridge has major load-bearing components – they’re pretty obvious, strong, elegant, and efficiently support the rest of the structure. A spaghetti tower is more like this.

Image from the always-delightful r/DiWHY.

(The motto of the spaghetti tower is “Sure, it works fine, as long...
Boundaries enable positive material-informational feedback loops
by jessicata — 2018-12-22T02:46:48.938Z

(Cross-posted from my blog)

[epistemic status: obvious once considered, I think]

If you want to get big things done, you almost certainly need positive feedback loops. Unless you can already do all the necessary things, you need to do/make things that allow you to do/make more things in the future. This dynamic can be found in RPG and economy-management games, and in some actual economic systems, such as industrializing economies.

Material, information, and economy

Some goods that can be used in a positive feedback loop, such as software and inventions, are informational. Once produced, they can be used indefinitely in the future. In economic terms, they are nonrivalous.

Other goods are material, such as manufactured goods and energy. They can't be copied cheaply. In economic terms, they are rivalrous.

In practice, any long-lasting positive feedback loop contains both informational and material goods, as production of information requires a physical substrate. While ensuring that informational goods can be used in the future is an organization and communication problem (a subject beyond the scope of this post), the problem of ensuring that material goods can be used in the future is additionally a security problem.

An important question to ask is: why haven't material-informational positive feedback loops already taken over the world? Why don't we have so much stuff by now that providing for people's material needs (such as food and housing) is trivial?

To some extent, material-informational positive feedback loops have taken over the world, but they seem much slower than one would naively expect. See cost disease. As an example of cost disease, the average cost of a new house in the USA has quadrupled over a 60-year period (adjusted for inflation!), whereas models of capitalism based on economy-management games such as Factorio (or, more academically, according to the labor/capital based economic models of classical economists such as David Ricardo) would suggest that houses would be plentiful by now. (And no, this isn't just because of land prices; it costs about $300K to build a house in the US in 2018)

Security and boundaries

I've already kind of answered this question by saying that ensuring that material goods can be used in the future is a security problem. If you use one of your material goods to produce another material good, and someone takes

...
Anthropic probabilities and cost functions
by Stuart_Armstrong — 2018-12-21T17:54:20.921Z

I've claimed that anthropic probabilities like SIA and SSA don't actually exist - or, more properly, that you need to include some details of preferences in order to get any anthropic probabilities, and thus that anthropic issues should be approached from the perspective of decision theory.

What do I mean by this? Well, informally, what are probabilities? If I said that X (a very visible event) would happen with a probability 10%, then I would expect to see events like X happen about a tenth of the time.

This makes a lot of sense. Why can't it be transposed into anthropic situations? Well, the big problem is the "I" in "I would expect". Who is this "I" - me, my copies, some weighted average of us all?

In non-anthropic situations, we can formalise "I would expect to see" with a cost function. Let me choose a number p(X) to be whatever I want; then, if X doesn't happen I pay a cost of (p(X))2, while if it does happen, I pay a cost of (p(X)−1)2 (this is exactly equal to −(p(X)−IX)2, for IX the indicator function of X).

Then, for this cost function, I minimize my losses by setting "p(X)" to be equal to my subjective opinion of the probability of X (note there are many eliciting cost functions we could have used, not just the quadratic loss, but the results are the same in for all of them).

In the informal setting, we didn't know how to deal with "I" when expecting future outcomes. In the formal setting, we don't know how to aggregate the cost when multiple copies could all have to pay the cost.

There are two natural methods of aggregation: the first is to keep −(p(X)−IX)2, as above, as the cost for every copy. Thus each copy has the average cost of all the copies (this also allows us to generalise to situations where different copies would see different things). In this case, the probability that develops from this is SSA.

Alternatively, we could add up all the costs, giving a total cost of −n(p(X)−IX)2 if there were n copies (this also generalises to situations where different copies see different things). In this case, the probability that develops from this is SIA.

So probability might be an estimate of what I expect to see, or a cost-minimiser for errors of prediction, but anthropic probabilities differ depending on how one extends "I" and "cost" to situations of multiple copies.
Standing on a pile of corpses
by Jsevillamol — 2018-12-21T10:36:50.454Z

[In the darkest day of 2018, it is proper to think about the darkness that surrounds us]

When we think about the history of humanity, we focus on its highlights.

Galileo discovering the moons of Jupiter. Edward Jenner developing the first pox vaccine. Emmy Noether setting the mathematical foundation of modern physics.

We stand on the shoulders of giants, we say, that have elevated us over the clouds so we can see the stars above.

I believe a more apt metaphor is that we stand on a pile of nameless corpses.

Because for each great human that made it to the history books, a million have been forgotten. And each of them, knowingly or unknowingly, it is part of our legacy.

And most of them are dead.

100 billion humans have been born and died, most in undignified circumstances of sickness and age, and most by no choice of their own. They are now part of the pile of corpses.

7 billion people remain alive. And even though our lives are much better than those of the people from 1000 years ago, from 100 years ago and even from 10 years ago, we still suffer.

Many of our living kin live in sickness and hunger. Even amongst the most fortunate we wrestle with mental illness and accidents and the plights of aging.

We do not stand proud, but afraid and resigned to become yet another layer of the pile of corpses.

For not a single human lives free of the tyranny of death. We can choose to embrace it early, but we cannot still postpone it, not for much.

And so the pile of corpses grows, too fast for us to properly mourn the fallen.

There is a glimmer of hope, that the pile of corpses will stop growing as we come of age as a civilization. That we will stop the non consensual suffering we experience, and death will be no more for those who want to defy it.

That we will have, for the first time in history, time to breathe and reflect and remember all the nameless people who came before. To properly ponder without having to constantly struggle over our survival, and leaving behind extreme suffering.

To finally give a proper burial to the pile of corpses we stand on, and decide what to do with our piece of the universe.

But we are not guaranteed this happy ending - the book of humanity might suddenly end, without us having a chance to dictate the final words.

The worst we have endured is not a good predictor of the worst that it is to come, and for all we know the horrors ahead may be the ones tha...
Systems Engineering and the META Program
by ryan_b — 2018-12-20T20:19:25.819Z

I periodically look for information on systems engineering. This time I came across a powerpoint presentation from the MIT Open Courseware course Fundamentals of Systems Engineering. Professor de Weck, who taught the course, had done some research on state-of-the-art methods developed as part of DARPA's META Program.

A few years ago DARPA wrapped up the program, designed to speed up delivery of cyber-electro-mechanical systems (war machines) by 5x. Since the parent program Adaptive Vehicle Make seems to have concluded without producing a vehicle, I infer the META Program lost its funding at the same time.

The work it produced appears to be adjacent to our interests along several dimensions though, so I thought I would bring it to the community's attention. The pitch for the program, taken from the abstract of de Weck's paper:

The method claims to achieve this speedup by a combination of three main mechanisms:

1. The deliberate use of layers of abstraction. High-level functional requirements are used to explore architectures immediately rather than waiting for downstream level 2,3,4 ... requirements to be defined.

2. The development and use of an extensive and trusted component (C2M2L) model library. Rather than designing all components from scratch, the META process allows importing component models directly from a library in order to quickly compose functional designs.

3. The ability to find emergent behaviors and problems ahead of time during virtual Verification and Validation (V&V) and generating designs that are correct-by-construction allows a more streamlined design process and avoids costly design iterations that often lead to expensive design changes.

Which is to say they very carefully architect the system, use known-to-be-good components, and employ formal verification to catch problems early. In the paper a simulation of the META workflow successfully achieved a 4.4x development speedup compared to the same project's actual development using traditional methods.

There are a bunch of individual directions explored which are of interest. Some that struck me were:

A metric for complexity.
A metric for adaptability.
Stuff for quantitative verification methods.
Stuff for complexity reduction and verification.

It looks like the repository for the program's outputs is here. A missile engineer try...