trentmkelly/LessWrong-43k huggingface dataset:
Swiss Political System: More than You ever Wanted to Know (II.)
by Martin Sustrik — 2020-07-22T13:50:04.033Z

Previous part

In the previous part of this article we've looked in detail at the Swiss system of direct democracy.

It's hard not to conclude that direct democracy is the reason why Swiss society works so well. However, one then remembers the direct democracy in California and is no longer so sure. Not that the Californian direct democracy is necessarily bad, but it shows no spectacular results either.

One possible argument to explain the discrepancy could be that direct democracy needs time to mature. Which it certainly does. We've already seen how the Swiss popular initiative needed hundred years to evolve to its current form. However, Swiss direct democracy dates back to the end of 19th century, while Californian direct democracy became fully established in 1911. Difference of mere 22 years.

Another possible explanation of why Swiss direct democracy works better is that California is constrained by being part of the United States. And there is no direct democracy on the federal level in the US. Maybe the dysfunction trickles down from above and the local direct democratic institutions are not able to cope with it.

I have no idea whether that's a plausible explanation, however, we may be wiser after giving a closer look to the second pillar of the Swiss political system, the feature that is often called "federalism" but which can, in my opinion, be interpreted in much wider sense as decentralization at all levels of the society.

Cantons

Switzerland has once been a loose confederacy of independent states and the unification in 1848, to a significant extent, preserved that independence.

While there is a central government, its powers are rather limited. So-called "principle of subsidiarity" applies: The areas managed by the federal government are explicitly listed in the constitution and anything that is not included in the list automatically falls under the jurisdiction of the cantons. Even in the areas that the federal government is responsible for, it often does the decision making, but leaves the execution to the cantons. So, for example, although the Department of Justice and Police exists at the federal level, the police forces themselves operate at the cantonal level, in larger cities even at the municipal level.

Each canton is a small state on its own. It has its own constitution, its own parliament and its own government.

Even the political system v

...
If you are signed up for cryonics with life insurance, how much life insurance did you get and over what term? 
by Eli Tyre — 2020-07-22T08:13:38.931Z

Hey all,

I'm been cryo-crastinating, but I'm trying to push through and actually get signed up in the next two months.

A next step is that I need to buy life insurance. I'm a little bit at a loss, because I'm not sure how much I should get. It seems like I want some margin of error above the minimum amount required by the cryonics org that I'm signing up with, but I'm not sure how much is a reasonable margin of error.

If you have life insurance for cryonics, how much did you get, and why? What is the term of your policy, and why did you select that length of time?

Also, for those of you who paid / are planning to pay out of pocket, why did you opt to do that instead?

Thanks!
The Basic Double Crux pattern
by Eli Tyre — 2020-07-22T06:41:28.130Z

I’ve spent a lot of time developing tools and frameworks for bridging “intractable” disagreements. I’m also the person affiliated with CFAR who has taught Double Crux the most, and done the most work with it.

People often express to me something to the effect,

The important thing about Double Crux is all the low level habits of mind: being curious, being open to changing your mind, paraphrasing to check that you’ve understood, operationalizing, etc. The ‘Double Crux’ framework, itself is not very important.

I half agree with that sentiment. I do think that the low level cognitive and conversational patterns are the most important thing, and at Double Crux trainings that I have run, most of the time is spent focusing on specific exercises to instill those low level TAPs.

However, I don’t think that the only value of the Double Crux schema is in training those low level habits. Double cruxes are extremely powerful machines that allow one to identify, if not the most efficient conversational path, then at least a very high-efficiency conversational path. Effectively navigating down a chain of Double Cruxes, resolving a disagreement at the bottom of the chain, iterating back to the top, and finding that you've actually changed you mind, is like magic. So I’m sad when people write it off as useless.

This post represents my current best attempt to encapsulate the minimum viable Double Crux (TM) skill: a basic conversational pattern for steering toward and finding double cruxes. I’m going to try and outline the basic Double Crux pattern, the series of 4 moves that makes Double Crux work, and give a (simple, silly) example of that pattern in action.

Again, this is only a basic schema. These four moves are not (always) sufficient for making a Double Crux conversation work. That does depend on a number of other mental habits and TAPs. And real conversations can throw up all kinds of problems that require more specific techniques and strategies. But this pattern is, according to me, at the core of the Double Crux formalism.

The pattern

For simplicity, I have described this in the form of a 3-person Double Crux conversation, with two participants and a facilitator. Of course, one can execute these same moves in a 2 person conversation, as one of the participants. But that additional complexity is hard to manage for beginners, so I usually have people practice in a facilitation ...
Research agenda for AI safety and a better civilization
by agilecaveman — 2020-07-22T06:35:36.400Z

This agenda is various ideas that occurred to me throughout the years of reading up on AI progress and AI safety. One big idea is that AI is simultaneously software, applied mathematics as well as philosophy and ideology. AI being philosophy and ideology means the big issues in theoretical AI have a corresponding ideological issue in non-AI. For example, the idea of what utility function ought we optimize is a very related to question such as what a “good” society is, how should we measure progress. Question of why AI is not reliable are very dependent on questions of why software is not reliable or why we have yet to come with a process to create bug-free software after all these years of trying. To fully “solve” AGI means fully solving ethics, politics and economics, for starters. This may not sound possible, but “sounding possible” isn’t that important compared to doing what’s actually needed. Good AI research has dual use as good philosophy and deep theoretical understanding of society. So, solving a number of problems in this research agenda would basically lay a new mathematical, conceptual and philosophical foundation for a new way of doing software, reasoning about science and general ways of knowing things.

Now given that the scope of the problem is large, it’s not hard to come up with some issues, which likely can help progress in AI. However, it’s a bit harder to narrow done the ones which are most likely to be helpful in terms of the resulting final or intermediate produce. Many of these seem somewhat under-studied to me and most of these are of a theme around “how can we gain a reliable understanding of this?” rather than “here is what will win a particular benchmark.”

Improving and understanding current implementations.

1. Functional programming / Category theory + AI

The joke is, all dysfunctional AI should be banned. What is dysfunctional? AI not written in functional languages. Jokes aside, it is a travesty of epic proportions that we settled on Python and not Haskell for our AI libraries. Most of what we need to do is lots of data processing, which lends itself no-side effect functions. Not to mention, there is a lot of iterative work that happens in trying parts of data science experiments repeatedly, which makes caching ever more important. If yo...
More Right
by Adam Zerner — 2020-07-22T03:36:54.007Z

Before I came across LessWrong, I felt like I was the only sane person in the world.1 This excerpt from HPMOR illustrates those feelings well:

Harry didn't quite know how to describe in words the sense of kinship he felt with Professor Quirrell, except to say that the Defense Professor was the only clear-thinking person Harry had met in the wizarding world. Sooner or later everyone else started playing Quidditch, or not putting protective shells on their time machines, or thinking that Death was their friend. It didn't matter how good their intentions were. Sooner or later, and usually sooner, they demonstrated that something deep inside their brain was confused. Everyone except Professor Quirrell. It was a bond that went beyond anything of debts owed, or even anything of personal liking, that the two of them were alone in the wizarding world.

Here's an example. I have a friend who I study poker with. When we analyze hands, we dive deep into probabilities and expected values and game theory. We've done this hundreds and hundreds of times, and he's really good at doing it. He also seems quite sane overall.

The other day I tried something different with him. I told him that I estimate that the risk of dying from the coronavirus costs $6/hr in EV for someone playing live poker and asked what his estimate is. It's just probabilities and expected values, not radically different from when we analyze a poker hand together. I also said that as you value life more (yours and other peoples), it becomes more and more costly, and that I personally am hoping for some sort of life extension (anti-aging, cryonics, AI takeoff, etc).

His response was that too long of a life is a curse, because you'll eventually learn all you can learn, and because you stop the evolution of your consciousness when you unnaturally extend this life. Instead, you should be moving or cycling to whatever is next.

It caught me off guard. I genuinely wasn't expecting that from him. But alas:

Sooner or later, and usually sooner, they demonstrated that something deep inside their brain was confused.

This has been discussed many times before.

Outside The Laboratory:

Now what are we to think of a scientist who seems competent inside the laboratory, but who, outside the laboratory, believes in a spirit world? We ask why, and the scientist says something along the lines of: "Well, no one really kn

...
[not ongoing] Thoughts on Proportional voting methods
by Jameson Quinn — 2020-07-22T02:46:27.490Z

About this post

This post was a work-in-progress, I (Jameson) updated it over the course of weeks/months, and the reposted it when it got substantial updates. I have now decided to leave it as it is for historical interest, as the next step would be a ground-up rewrite. I'm not sure when I'll have the time to finish that ground-up rewrite but I'd say there's a >80% chance something like that will happen within the next year; that is, before September 2021.

Status

I think I'm going to stop making large edits to the article as it stands. I'll add a longer "post-mortem" at the end. In sum: I think that the process of writing this article has helped me sharpen my thinking and make several breakthroughs, but that making an article that best conveys these insights to others requires rewriting from scratch. I still think that it would be worthwhile to eventually write the unfinished parts of my initial outline, but that attaching them to this thinking-it-through prologue would not do them justice.

Epistemic status

This is largely a work of political philosophy with a mathematical bent. That is to say, it is working through the implications of assumptions; more specifically, it is an attempt to find a way to estimate the implications of certain normative assumptions in certain real-world situations. As such, it rests on a larger context of thought about those assumptions — whether they are realistic, whether they are desirable — which I cannot possibly fully explore here.

But leaving aside those connections to wider ideas and taking the assumptions I make here as axioms, how confident am I of the train of thought that is contained within this essay? Reasonably so. At various times I gesture towards or sketch an idea of a mathematical proof for a claim, but I do not and have not worked out those proofs fully, so I'm aware that they may not "compile". Still, as somebody who has a PhD and so has the experience of following an idea more of the way to the ground, I would be pretty surprised (<10% subjective chance) if there were some mistake here so large that it would invalidate the main ideas (rather than just requiring minor adjustments). I would of course be a lot less surprised if someone found a better or simpler way to reach the same goals I'm pursuing here.

Last 3 updates

V0.9.1: Another terminology tweak. New terms: Average Voter Effectiveness, Average Voter Choic...
[Preprint] The Computational Limits of Deep Learning
by Gordon Seidoh Worley — 2020-07-21T21:25:56.989Z
Link: https://arxiv.org/abs/2007.05558

"The Computational Limits of Deep Learning" by Neil C. Thompson, Kristjan Greenewald, Keeheon Lee, and Gabriel F. Manso

Links:

PDF version
General Audience Summary on VentureBeat

NB: This is a preprint and not peer-reviewed or accepted for publication as best I can tell, so more than usual you'll have to make your own judgements about the quality of the results.

Abstract:

Deep learning's recent history has been one of achievement: from triumphing over humans in the game of Go to world-leading performance in image recognition, voice recognition, translation, and other tasks. But this progress has come with a voracious appetite for computing power. This article reports on the computational demands of Deep Learning applications in five prominent application areas and shows that progress in all five is strongly reliant on increases in computing power. Extrapolating forward this reliance reveals that progress along current lines is rapidly becoming economically, technically, and environmentally unsustainable. Thus, continued progress in these applications will require dramatically more computationally-efficient methods, which will either have to come from changes to deep learning or from moving to other machine learning methods.

A few additional details: they look at papers in ML to see how much compute was required to get results, and extrapolate the trend lines to suggest we're nearing the limits of what is economically feasible to do under the current regime. They believe this implies we'll have to get more efficient if we want to see continued progress, such as by having more specialized and efficient hardware or by improving algorithms. My takeaway is that they believe most of the low hanging fruit in ML gains has already been picked, and additional gains in capabilities will not come as easily as past gains.

The straightforward implications for safety are that, if this is true, we are less near x-risk territory than it might appear we are if you were to only look at the "numerator" of the trend lines (what we can do) without consider the "denominator" of them (how much it costs). Not that we are necessary dramatically far from x-risk territory with ML, mind you, only that it's not obviously very near term since the economic realities of deploying this technology will soon shift to naturally slow immediate progress without significant effort or innovation.
Fresh Bread
by Zvi — 2020-07-21T20:40:00.994Z

Learning to bake bread was a top quarantine activity for many. For too long, I neglected to post the method my wife and I have been using for years. It’s time to fix that.

This recipe is quick and easy, it’s flexible, and it’s absolutely delicious.

This doesn’t mean there isn’t room for other preferences, or for variety, but this is pretty awesome.

The core comes from the book The New Artisinal Bread in Five Minutes a Day.

The book provides lots of useful variants, tweaks and methods one can use to make things great. And they’ve certainly earned your money! But you can get most of the benefits from the core recipe, and tweak from there.

Here’s how it works.

Ingredients:

3 cups water

6.5 cups bread flour (can substitute all-purpose flour and it’s fine)

1.5 tablespoons self-activating yeast

1.5 tablespoons kosher salt

How to make the dough:

1. Put the water wherever you want to store the dough.

2. Add yeast and salt, and mix together.

3. Then add the flour, and mix again. No need to kneed, only to mix until it’s roughly one texture.

4. Cover with saran wrap and let rise for two hours.

5. Put it in the refrigerator and use as needed. It should be good for at least two weeks.

Because the timing is so flexible, you can always get hot fresh bread. You can make the exact amount you want, exactly when you want it. Don’t drop the ball on this. It’s a huge deal. Fresh and hot is where it is at.

When the time comes to make the bread:

1. Preheat oven to 450.

2. Grease pan with flour and butter and/or wax paper. Good wax paper is a big win, but bad wax paper only makes things worse.

3. Make desired shape of dough. Remember that it doesn’t have to be that high, it will rise a lot.

4. (Optional) You can let the bread rise again before baking, and it’s slightly better if you do that for a half hour to an hour, but you don’t need to. We mostly skip it.

5. (Optional) If desired, place flour over loaf and make slash marks on the top to get more crisp. This totally is not actually necessary. We mostly skip it.

6. Place in oven.

7. (Optional) Have a pan on the rack below the bread, and dump a cup of warm water onto that pan to generate steam. Again, slightly useful, not actually necessary. We mostly skip it.

8. Bake bread for 30 minutes or until it’s about as done as you’d like. Ovens and preferences vary.

Serve as desired, preferably with some combination Kerrygold Butter, Sea Salt, fresh groun

...
Competition: Amplify Rohin’s Prediction on AGI researchers & Safety Concerns
by stuhlmueller — 2020-07-21T20:06:09.194Z

EDIT: The competition is now closed, thanks to everyone who participated! Rohin’s posterior distribution is here, and winners are in this comment.

In this competition, we (Ought) want to amplify Rohin Shah’s forecast for the question: When will a majority of AGI researchers agree with safety concerns? Rohin has provided a prior distribution based on what he currently believes, and we want others to:

Try to update Rohin’s thinking via comments (for example, comments including reasoning, distributions, and information sources). If you don’t want your comment to be considered for the competition, label it ‘aside’
Predict what his posterior distribution for the question will be after he has read all the comments and reasoning in this thread

The competition will close on Friday July 31st. To participate in this competition, create your prediction on Elicit, click ‘Save Snapshot to URL,’ and post the snapshot link in a comment on this post. You can provide your reasoning in the ‘Notes’ section of Elicit or in your LessWrong comment. You should have a low bar for making predictions – they don’t have to be perfect.

Here is Rohin’s prior distribution on the question. His reasoning for the prior is in this comment. Rohin spent ~30 minutes creating this distribution based on the beliefs and evidence he already has. He will spend 2-5 hours generating a posterior distribution.

Click here to create your distribution

We will award two $200 prizes, in the form of Amazon gift cards:

Most accurate prediction: We will award $200 to the most accurate prediction of Rohin’s posterior distribution submitted through an Elicit snapshot. This will be determined by estimating KL divergence between Rohin’s final distribution and others’ distributions. If you post more than one snapshot, either your most recent snapshot or the one you identify as your final submission will be evaluated.
Update to thinking: Rohin will rank each comment from 0 to 5 depending on how much the reasoning updated his thinking. We will randomly select one comment in proportion to how many points are assigned (so, a comment rated 5 would be 5 times more likely to receive the prize than a comment rated 1), and the poster of this comment will receive the $200 prize.

Motivation

This project is similar in spirit to a...
Alignment As A Bottleneck To Usefulness Of GPT-3
by johnswentworth — 2020-07-21T20:02:36.030Z

So there’s this thing where GPT-3 is able to do addition, it has the internal model to do addition, but it takes a little poking and prodding to actually get it to do addition. “Few-shot learning”, as

the paper

calls it. Rather than prompting the model with

Q: What is 48 + 76? A:

… instead prompt it with

Q: What is 48 + 76? A: 124

Q: What is 34 + 53? A: 87

Q: What is 29 + 86? A:

The same applies to lots of other tasks: arithmetic, anagrams and spelling correction, translation, assorted benchmarks, etc. To get GPT-3 to do the thing we want, it helps to give it a few examples, so it can “figure out what we’re asking for”.

This is an alignment problem. Indeed, I think of it as the quintessential alignment problem: to

translate what-a-human-wants into a specification usable by an AI

. The hard part is not to build a system which can do the thing we want, the hard part is to specify the thing we want in such a way that the system actually does it.

The GPT family of models are trained to mimic human writing. So the prototypical “alignment problem” on GPT is prompt design: write a prompt such that actual human writing which started with that prompt would likely contain the thing you actually want. Assuming that GPT has a sufficiently powerful and accurate model of human writing, it should then generate the thing you want.

Viewed through that frame, “few-shot learning” just designs a prompt by listing some examples of what we want - e.g. listing some addition problems and their answers. Call me picky, but that seems like a rather primitive way to design a prompt. Surely we can do better?

Indeed, people are already noticing clever ways to get better results out of GPT-3 - e.g. TurnTrout recommends

conditioning on writing by smart people

, and the

right prompt

makes the system complain about nonsense rather than generating further nonsense in response. I expect we’ll see many such insights over the next month or so.

Capabilities vs Alignment as Bottleneck to Value

I said that the alignment problem on GPT is prompt design: write a prompt such that actual human writing which started with that prompt would likely contain the thing you actually want. Important point: this is worded to be agnostic to the details GPT algorithm itself; it’s mainly about predictive power. If we’ve designed a good prompt, the current generation of GPT might still be unable to solve the problem - e.g. GPT-3 ...