trentmkelly/LessWrong-43k huggingface dataset:
Welcome to Efektywny Altruizm Warszawa [Edit With Your Details]
by mtrzesim — 2018-04-23T19:37:49.765Z

(The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)

What kind of events does your group usually run? What does it usually do?

How frequently does your group organize events or meet?

Who would be a good fit for you group?

Should they have any particular skills or have done some specific background reading?
Welcome to Warsaw Rationality Dojo [Edit With Your Details]
by mtrzesim — 2018-04-23T19:34:08.638Z

(The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)

What kind of events does your group usually run? What does it usually do?

How frequently does your group organize events or meet?

Who would be a good fit for you group?

Should they have any particular skills or have done some specific background reading?
Welcome to Rationality Zurich [Edit With Your Details]
by Felix Denker — 2018-04-23T16:11:58.896Z

(The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)

What kind of events does your group usually run? What does it usually do?

How frequently does your group organize events or meet?

Who would be a good fit for you group?

Should they have any particular skills or have done some specific background reading?
The Alignment Newsletter #3: 04/23/18
by Rohin Shah — 2018-04-23T16:00:32.988Z

Highlights

Incomplete Contracting and AI Alignment

(Dylan Hadfield-Menell et al): This paper explores an analogy between AI alignment and incomplete contracting. In human society, we often encounter principal-agent problems, where we want to align the incentives of the agent with those of the principal. In theory, we can do this with a "complete" contract, that is an enforceable contract that fully specifies the optimal behavior in every possible situation. Obviously in practice we cannot write such contracts, and so we end up using incomplete contracts instead. Similarly, in AI alignment, in theory we could perfectly align an AI with humans by imbuing it with the true human utility function, but in practice this is impossible -- we cannot consider every possible situation that could come up. The difference between the behavior implied by the reward function we write down and the utility function we actually want leads to misalignment. The paper then talks about several ideas from incomplete contracting and their analogues in AI alignment. The main conclusion is that our AI systems will have to learn and use a "common sense" understanding of what society will and will not sanction, since that is what enables humans to solve principal-agent problems (to the extent that we can).

My opinion: I'm excited to see what feels like quite a strong connection to an existing field of research. I especially liked the section about building in "common sense" (Section 5).

Understanding Iterated Distillation and Amplification: Claims and Oversight

(William_S): The post introduces a distinction between flavors of iterated distillation and amplification -- whether the overseer is low bandwidth or high bandwidth. Let's think of IDA as building a deliberation tree out of some basic overseer. In the high bandwidth case, we can think of the overseer as a human who can think about a problem for 15 minutes, without access to the problem's context. However, there could be "attacks" on such overseers. In order to solve this problem, we can instead use low-bandwidth overseers, who only look at a sentence or two of text, and verify through testing that there are no attacks on such overseers. However, it seems much less clear that such an overseer would be able to reach high levels of capability.

My opinion: This is an excellent post th...
Welcome to Munich LW/SCC Meetup [Edit With Your Details]
by Lukas — 2018-04-23T08:40:07.312Z

(The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)

What kind of events does your group usually run? What does it usually do?

How frequently does your group organize events or meet?

Who would be a good fit for you group?

Should they have any particular skills or have done some specific background reading?
Hold On To The Curiosity
by Ben Pace — 2018-04-23T07:32:01.960Z

I.

Recently, an excited friend was telling me the story behind why we care about the mean, median and mode.

They explained that a straightforward idea for what you might want in an ‘average’ number, is something that minimises how far it is from all the other numbers in the dataset - so if your numbers are 1, 2 and 3, you want a number x such that the sum of the distance to each datapoint is as small as possible. It turns out this number is 2.

However, if your numbers are 1, 2, and 4, the number that minimises the distance from all of them is also 2.

Huh?

When my friend told me this, the two other people I was with sort of said “Okay”. I said “What? No! I don’t believe you! It has to change when the data does - it’s a linear sum, so it has to change! It’s like you’re saying the sum of 1, 2 and 3 is the same as the sum of 1, 2 and 4. This is just wrong." Suffice to say, my friend’s claim wasn’t predicted by my understanding of math.

Now, did I really not believe my friend? The other two people with us were certainly fine with it. Isn’t this just bayesianism? That’s how the old joke goes:

Math teacher: Now I’m going to prove to you that X is true.

Bayesian: You just did.

Actually, no. You taught me a detail to memorise, but my models didn’t improve. I won’t be able to improve how I use averages, because I don’t understand how it fits in with everything else I understand - it doesn’t fit with the models I use everywhere else in math.

I mean, I could’ve nodded along. It’s only one fact, after all. But if I’m going to remember it in the long term, it should connect to my other models and be reinforced. The alternative is to be stored in the brain with all those other memorised facts that students learn for exams and forget immediately after.

If you’re trying to build new models of a domain, it’s important to choose to speak from the confusion, not from the rest of yourself. Don’t have conversations about whether you believe a thing. Instead talk about whether you understand it.

(The problem above was the definition of the median, and an explanation of the math for the curious can be found in this comment.)

II.

It can be really hard to feel your models. Qiaochu Yuan’s method of learning involves rampi...
The Spousal Chain of Succession
by pku — 2018-04-23T00:10:48.839Z
Link: https://shakeddown.wordpress.com/2018/03/01/the-spousal-chain-of-succession/

I told a bunch of people I'd post this idea here when I was writing it, then procrastinated on that. So here it is in all its glory. (Epistemic Status: Silly. Read at your own risk of wasting time).
Symbiosis - An Intentional Community For Radical Self-Improvement
by Matt Goldenberg — 2018-04-22T23:15:06.832Z

The vision for Symbiosis is an intentional group house in the Bay Area where high performers commit to radical life experimentation that will fundamentally shift who they are.

We will create a recursively self-improving household that supports these high performers with a platform to experiment with radical lifestyle design and mutual identity change.

As a household, we’ll help members change their self identity and belief system (first person self-improvement), their habits and actions (second-person self-improvement), and their expressive qualities (third-person self-improvement). All in a manner congruent with the person they want to be and the outcomes they want to achieve.

Who are we?

Matt Goldenberg

The founder of Verity and lifelong self-improvement obsessive. Working to create better forms of trust and collaboration with algorithmic reputation.

Conor White-Sullivan

Founder of Roam. Working to create more explicit, shared models of knowledge that aren't bottlenecked by working memory.

Who are we looking for?

We’re looking for existing high performers who know they could be pushed to even higher levels through large scale lifestyle experiments.

Traits

Doer: What have you accomplished in the past?
Improver: How have you shifted who you are?
Team Player: How have you interacted with previous communities?

Skills

Change Worker: Do you have experience as a coach, therapist, or mentor in helping others improve? Do you have knowledge in a specific changework modality such as NLP, Rationality, or Men’s Work?
Community Organizer: Do you have specific experience organizing communities in the past and bringing people closer together?
Logistical Coordinator: Do you have experience dealing with complicated logistics and planning around conflicting schedules?

The Pillars of Symbiosis

The goal for Symbiosis is to start with as few ingredients as possible and limiting the amount of assumptions we make. We’ll start with a base that will set the house in a certain direction, and give it the ability to recursively self-improve over time. This will create a success spiral that will continue to build off of the success of previous improvements.

To this end, we’ve created 5 pillars for the house. These pillars themselves will likely change and evolve over time, but will still serve as a foundational base from which the more radical experiments will flow.

Pillar #1 - Goals

The first ...
Computational Morality (Part 2) - on the Need for a Dynamic League Table of Proposed Solutions
by David Cooper — 2018-04-22T22:35:31.116Z

AGI will, over the course of the next few years, be put to work at national intelligence agencies and in military devices. Some countries will be more careful than others in ensuring that their AGI is safe and moral, but they all need all the help they can get in getting this right (or as close to that as possible). For that reason, there needs to be a place they can look up to find out what is the current state of the art in machine ethics. Such a place should maintain a league table of proposed solutions listed in order of how well they perform, and any issues with them should be listed under each entry to invite people to try to resolve them, thereby enabling any advances to lead to that proposal moving up the league table, or down if a new fault with it is uncovered.

I have been unable to find a league table if that kind thus far. All I see is a mass of different approaches scattered about with no clear answers. I set out my own proposed solution in the opening part of this series ( https://www.lesswrong.com/posts/Lug4n6RyG7nJSH2k9/computational-morality ), and I want to see it placed into a proper league table with all the other proposed solutions. People would then be invited to provide objections to show where they think it fails, and these would be listed in order with the most serious objections at the top. People would also be free to try to show that those objections aren't valid (so any that are shown to be wrong would be moved to a list of failed objections), and they'd also be free to add new objections. The same would happen with all the other proposals, and over the course of time we would see them all migrate slowly to their proper places.

The biggest problem with such a league table might be the issue of who gets to decide when a claimed fault has been invalidated (or whether it applies in the first place), so it would all depend on fair judging, but any kind of bias that's introduced would soon lead to a rival site offering a better alternative table which would gain greater credibility and take over as the go-to place for everyone involved in this business, so it would be in the interests of the people in charge to make sure that they are being fully fair.

But would it really be worth the bother?

Well, judging from my experience as an AGI system builder, yes: I'm trying to find out what the latest thinking is on machine et...
On exact mathematical formulae
by daozaich — 2018-04-22T19:41:46.117Z

This is inspired by the review on "Linear Algebra done right". I decided to do a top-level post, because it hits a misconception that is pretty common.

The starting point of this post is this quote from "Linear Algebra done right":

Remarkably, mathematicians have proved that no formula exists for the zeros of polynomials of degree 5 or higher. But computers and calculators can use clever numerical methods to find good approximations to the zeros of any polynomial, even when exact zeros cannot be found.

For example, no one will ever be able to give an exact formula for a zero of the polynomial p defined by p(x)=x5−5x4−6x3+17x2+4x−7.

The authors misrepresent an important point that is understood by most mathematicians, but not properly understood by many laypeople.

What does it mean to solve a problem? What does it mean to have an exact formula for the solution of a problem?

The answers to both are a social convention that has historically changed and is expected to continue to evolve in the future.

Back in the days, people only considered rational numbers, ie fractions. Oh, but what about the positive solution to x2=2? Ok, we can't express this as a rational number (important theorem). Because these kinds of problems occured quite often, the mathematical community arrived at the consensus that √2, or more generally √r for nonnegative r should be considered an explicit solution. Amazingly, this allows us to express the solution to any quadratic equation 0=ax2+bx+c explicitly, with our expanded notion of "explicit". From an algebraic viewpoint it was natural to bless the positive solution to xn=a as an "explicit formula" next; historically it was a more contentious thing, because greek geometry wanted numbers to be constructible using a ruler and compass only. "Doubling the cube", ie expressing the positive solution to x3=2 as a geometric construction was a famous old problem (proven impossible in 1837, after having been a very prominent mathematical research problems for more than 2000 years).

Now, this obviously says not a lot about the cube root of 2, but says a lot about "constructible with ruler and compass".

In other words: "Explicit solutions" are a messy historical map to mathematical territory, nothing more.

The same holds if you ask for explicit formulas for zeros of polynomials after... ,,,,,,,,,,
The First Rung: Insights from 'Linear Algebra Done Right'
by TurnTrout — 2018-04-22T05:23:49.024Z

Foreword

Linear algebra, my old flame - how I missed you. At my undergraduate institution, linear algebra was my introduction to proof-based mathematics. There are people who shake hands, and there are people who shake hands. You know the type - you grasp their hand, and they clamp down and pull you in, agitating so wildly you fear for the structural integrity of your joints. My first experience with proofs was an encounter of the latter variety.

I received my first homework grade, and I was not pleased with my performance. I promptly went to the library and vowed not to leave until I learned how to write proofs adequately. The hours passed, and, (thankfully for my stomach), I got it. I didn't let up all semester. Immediately before the final exam, I was doing pushups in the hallway, high-fiving my friends, and watching the 'Michael Jordan Top 50 All Time Plays' video while visualizing myself doing that to the test. Do that to the test I did indeed.

This time around, the appropriately-acronymized LADR is the first step on my journey to attain a professional-grade mathematical skillset.

Tight Feedback Loops

In a (possibly maniacal) effort to ensure both mastery of the material and the maturation of my proof skillset, I did nearly1 every one of the 561 exercises provided. I skipped problems only when I was confident I wouldn't learn anything, or calculus I didn't remember was required (and the payoff didn't seem worth the time spent relearning it now in a shallow manner, as opposed to thoroughly learning more calculus later). If I could sketch a solid proof in my head, I wouldn't write anything down. Even in the latter case, I checked my answers using this site (additional solutions may be found here, although be warned that not all of them are correct).

I also sometimes elected to give myself small hints after being stuck on a problem for a while; the idea was to keep things at the difficulty sweet spot. Specifically, I'd spend 10-20 minutes working on a problem by myself; if I wasn't getting anywhere, I'd find a hint and then backpropagate the correct mental motion instead of what I had been trying to do. I think that focusing on where you were going wrong and what insight you should have had, in what direction you should have looked, is more efficient than just reading solutions.

Over time, I needed fewer hints,... ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
The Case Against Education: Foundations
by Zvi — 2018-04-21T14:30:00.454Z

An Introduction: The Case Against Education

We’ll now focus more on the book and its data, reasoning and arguments, and related questions. There’s a lot to address, starting with the foundations of Bryan’s analysis. Rather than write one novella-long post, I’ll be going through my highlights and talking about related issues, and post sections as I go.

IV

Is our children learning?

Well, what’d ya know?

Bryan has the data: Not much. You?

Here are some data points about adults:

Most people who take high school algebra and geometry forget about half of what they learn within five years and forget almost everything within twenty-five years. Only people who continue on to calculus retain most of their algebra and geometry.

Yikes. We can argue about the value of geometry and calculus, but a practical proficiency in basic algebra is kind of important to functioning in the world.

Possible conclusions from this, aside from ‘Americans can’t do basic algebra, set your prices and marketing campaigns appropriately and maybe learn Mandarin’ include:

A: We don’t spend enough on algebra! Spend more time and money!

B: Don’t spend more time and money, we already spend a ton. It’s that we are horrible at teaching math. Stop it. Work smart, not hard. Maybe treat math as an adventure and not a series of memorized rules and tables.

(There’s an utterly devestating Scott Aaaronson post about what music education would look like if it was structured like math education, but my Google-fu couldn’t find it, someone link to it in the comments and I’ll edit in the link)

C: Don’t spend more time and money on algebra and geometry. Make calculus mandatory. Study shows that will make people remember algebra!

D: People actually implement things like C (and also A) all the time, so maybe drop mandatory algebra and geometry, and focus on teaching statistical literacy instead.

E: Maybe before we teach anyone algebra we should see if they know how to multiply.

Good idea. How about adults’ basic arithmetic?

Barely half know that saving $.05 per gallon on 140 gallons of oil equals $7.00.

Whoops.

You can either complain that this means our failing schools need even more funding and to imprison our kids even longer, since the right response to something not working despite ludicrous investments of resources is to invest ...
Rationality and Spirituality - Summary and Open Thread
by Chris_Leong — 2018-04-21T02:37:29.679Z

I've noticed that there has recently been a lot of interest in spirituality, ritual and meaning within the rationality community. I could highlight the discussions of Jordan Peterson (SSC, SSC Follow Up, Put A Num On It), Valentine's recent posts (Fake Frameworks, Kensho, The Intelligent Social Web, Mythic Mode, Kaj Sotala's Response), discussions on Meditation (Mastering the Core Teachings of the Buddha, Gupta on Enlightenment, Five Years After Starting Mindfulness Meditation) and a few others (Funeral Ritual, Open-Source Monasticism). I thought I would encourage this trend making this post as this seems like an interesting direction for exploration and I suspect more progress occurs when multiple people develop an interest in a topic at the same time than when they all explore individual directions.

This is an open thread, so you can post whatever you want about this topic, but here's a few possible discussion directions:

Have you made any attempt to explore any kind of spirituality in the broadest sense? If so, what have you learned?
In the past, attempts to engage with post-rationality didn't seem to be particularly successful. Perhaps now is a better time?
What are the core rationality posts or resources on this topic? What are some resources from outside the rationality community that we may find helpful?

(I'm too lazy, but the links above could form the basis of a Wiki post)
My confusions with Paul's Agenda
by Vaniver — 2018-04-20T17:24:13.466Z

Paul put out a call for probable problems with his agenda, which prompted this post detailing my confusions with it. This isn’t confidently asserting that the plan is doomed, and it’s not obvious to me that any of what follows is a novel objection. But it seems like it’s worth broadcasting the quiet voices of doubt when they appear, and worth trying to be an independent source of judgment. I’ll start by talking about the proposal of Paul’s that I think I understand best, the approval-directed agent, and then talk about IDA, and then finish by pointing at what I suspect is a major underlying crux.

Approval-Directed Agents

While many approaches to alignment view the agent as having some goal (which is only part of what we want it to do) and then restrictions (which prevent it from destroying the other things we want), approval-directed agents try to point directly at the goal of doing the thing that we want it to do as part of a broader ecosystem. If I say “Buy me a plane ticket for my trip to Austin,” the agent decides how much computing resources to allocate the task based on what it thinks I would approve of, and then it spends those resources based on what it thinks I would approve of, and so on, and eventually I trust the conclusion the system comes to and the tradeoffs it made along the way because my judgment has been approximated at every step.

The core insight here seems to be that the question “what would the architect approve of this subsystem doing?” remains a well-formed query for all subsystems, and we could in theory train all of them separately using standard machine learning techniques, or train them jointly with some sort of parameter sharing, or so on. Any unaligned behavior--the agent deciding to angrily tweet at airline companies in order to get a discount, despite me not wanting it to do that--can be traced back to a training failure of some subsystem, and a training failure of the meta-system that decides how to explore. It seems to me like the overall number of mistakes depends most on the training of that meta-system, and if it gets well-trained during the small-stakes high-involvement education period, then the system can learn new domains without many (or, potentially, any) catastrophic failures.

It seems like a core benefit of this sort of approach is that it replaces the motivation system (take action that argmaxes some score) with a...