trentmkelly/LessWrong-43k huggingface dataset:
Squiggle: An Overview
by ozziegooen — 2020-11-24T03:00:32.872Z

Overview

I've spent a fair bit of time over the last several years iterating on a text-based probability distribution editor (the 5 to 10 input editor in Guesstimate and Foretold). Recently I've added some programming language functionality to it, and have decided to refocus it as a domain-specific language.

The language is currently called Squiggle. Squiggle is made for expressing distributions and functions that return distributions. I hope that it can be used one day for submitting complex predictions on Foretold and other platforms.

Right now Squiggle is very much a research endeavor. I'm making significant sacrifices for stability and deployment in order to test out exciting possible features. If it were being developed in a tech company, it would be in the "research" or "labs" division.

You can mess with the current version of Squiggle here. Consider it in pre-alpha stage. If you do try it out, please do contact me with questions and concerns. It is still fairly buggy and undocumented.

I expect to spend a lot of time on Squiggle in the next several months or years. I'm curious to get feedback from the community. In the short term I'd like to get high-level feedback, in the longer term I'd appreciate user testing. If you have thoughts or would care to just have a call and chat, please reach out! We (The Quantified Uncertainty Research Institute) have some funding now, so I'm also interested in contractors or hires if someone is a really great fit.

Squiggle was previously introduced in a short talk that was transcribed here, and Nuño Sempere wrote a post about using it here.

Note: the code for this has developed since my time on Guesstimate. With Guesstimate, I had one cofounder, Matthew McDermott. During the last two years, I've had a lot of help from a handful of programmers and enthusiasts. Many thanks to Sebastian Kosch and Nuño Sempere, who both contributed. I'll refer to this vague collective as "we" throughout this post.

Video Demo

A Quick Tour

The syntax is forked from Guesstimate and Foretold.

A simple normal distribution

normal(5,2)

You may notice that unlike Guesstimate, the distribution is nearly perfectly smooth. It's this way because it doesn't use sampling for (many) functions where it doesn't need to.

Lognormal shorthand

5 to 10

This results in a lognormal distribution with 5 to 10 being the 5th and 95th confidence intervals respe...
Commentary on AGI Safety from First Principles
by Richard_Ngo — 2020-11-23T21:37:31.214Z

My AGI safety from first principles report (which is now online here) was originally circulated as a google doc. Since there was a lot of good discussion in comments on the original document, I thought it would be worthwhile putting some of it online, and have copied out most of the substantive comment threads here. Many thanks to all of the contributors for their insightful points, and to Habryka for helping with formatting. Note that in some cases comments may refer to parts of the report that didn't make it into the public version.

Discussion on the whole report

Will MacAskill

Thanks so much for writing this! Huge +1 to more foundational work in this area.

My overall biggest worry with your argument is just whether it's spending a lot of time defending something that's not really where the controversy lies. (This is true for me; I don't know if I'm idiosyncratic.) Distinguish two claims one could argue for:

Claim 1: At some point in the future, assuming continued tech progress, history will have primarily become the story of AI systems doing things. The goals of those AI systems, or the emergent path that results from interactions among these systems, will probably not be what you reading this document want to happen.

I find claim 1 pretty uncontroversial. And I do think that this alone is enough for far more of the world to be thinking about AI than currently is.

But it feels like at least for longtermist EAs trying to prioritise among causes (or for non-longtermists deciding how much to prioritise safety vs speed on AI), the action is much more on a more substantial claim like:

Claim 2: Claim 1 is true, and the point in time at which the transition from a human-driven world to an AI-driven world is in our lifetime, and the transition will be fast, and we can meaningfully affect how this transition goes with very long-lasting impacts, and (on the classic formulations at least) the transition will be to a single AI agent with more power than all other agents combined, and what we should try to do in response to all this is ensure that the AI systems that get built have goals that are the same as the goals of those who design the AI systems.

Each of the new sub-claims in claim 2, I find (highly) controversial. And you talk a little bit about some of these sub-claims, but it's not the focus.

Interested if you think that's an unfair characterisation. Perhaps you...
Straight-edge Warning Against Physical Intimacy
by Raphaëll — 2020-11-23T21:35:50.535Z

I’m someone who doesn’t do drugs or drink alcohol for a variety of reasons, the most important to me being so that I’m as “true to myself” as possible. By that, I mean that I do not want to do actions that I wouldn’t do sober (and might regret or might not identify with), or having my appreciation of people or things altered (ie. finding a person annoying just because of my state of mind, or finding a movie way funnier than I would normally do, and so on) or misremembering them. I want to say that I appreciate something and recommend it genuinely, and not just because I was intoxicated at the time I experienced it.

There is a movement for people who do not consume psychotropic drugs for self-control and “authenticity”: straight-edge people. So their motivations vary and some are drastically opposed to my values, but at its core these people share similar goals to me. This is relevant to know only in that one of the things some of them do avoid is promiscuous sex (and sometimes more specifically premarital sex, but I don’t see any strong reason backing this). I brushed it off at first, not seeing how sex could really go against my quest of being true to myself. But it was still on the back of my mind.

I stumbled upon a post on a polyamory group on Facebook asking people if oxytocin (which is produced when having all kinds of physical contacts, like hugging or kissing) could lead to some bad trips or other negative effects similar to drugs.

First things first, people in the comments debunked the idea I had (and that others might have) that hormones produced from physical affection are very different from the effects of drugs since they’re generated naturally, and therefore non-threatening (I know, naive, but eh). Turns out that their mechanisms are closer than what I thought:

“Yes, oxytocin and other hormones released during hugs/sex have a similar effect to drugs. Actually, most drugs don’t add anything into your body but will simply increase/activate or affect hormones naturally present, which will give you an altered state of mind. It’s therefore exactly the same for hugs/sex and other practices which release these hormones (there’s a pretty long list, and it’s not just for oxytocin). It’s even possible to rely on some practices without substances involved to activate a trance state similar to the experience of taking MD or LSD.
As with substance uses, we can l...
Using false but instrumentally rational beliefs for your career?
by Tim Liptrot — 2020-11-23T19:18:08.825Z

The truth

The job market for recent PhD graduates is tough. Simply, there are many more degree holders than available tenure track positions. People who went through the market describe it as capricious, random, unfair and difficult. New professors often lament highly qualified colleagues who found no position. If your dissertation is on Witte and no Witte-teaching positions are available, tough luck. This evidence suggests that the market is inneficient and that my personal behavior has a small impact on the outcome.

But OTOH, there are reasons to believe that a candidates ability has a large influence. Firstly, the profile of the desired candidate is clear beforehand: deep drive for knowledge, record of successful publication, agreeableness toward colleagues, and teaching skills. Secondly, there is an abundance of clear signals of these attributes; quality of publications, research influence and size of network are all clear and hard to fake. Thirdly, candidate ability has high variance (the best researchers greatly outperform). Therefore hiring committees have tools and incentives to select high-quality researchers.

The truth likely lies between these two positions. Underperforming PhD's rarely advance and many strong CV's fail due to unpredictable changes in demand.

The Instrumental Truth

But perhaps believing the truth is against my interest. If I believe that my work is important to outcomes, I will work harder. If I believe my work has little importance, I may become lazy or seek side-hustles. Should I convince myself that my publication count and thesis quality are more important than is true?

I'd love people's thoughts on this. Also link to great essays and blog posts about instrumental rationality!

Appendix

The best strategy is to remember that academia is a silly place. If the market is unkind don't hesitate to go elsewhere. Not-academia has more money and other good stuff! Be careful not to murder-pill yourself into thinking academia is the whole world.
Syntax, semantics, and symbol grounding, simplified
by Stuart_Armstrong — 2020-11-23T16:12:11.678Z

tl;dr: I argue that the symbol grounding problem is not an abstract philosophical problem, but a practical issue that it is useful and necessary for algorithms to solve, empirically, to at least some extent.

Intro: the Linear A encyclopaedia

Consider E, a detailed encyclopaedia of the world:

The inside pages are written, for some inexplicable reason, in Linear A, an untranslated Minoan script:

We also have access to R, a sort of Linear A Rosetta Stone, with the same texts in Linear A and in English. Just like the original Rosetta Stone, this is sufficient to translate the language.

The amount of information in E is much higher than the very brief R. And yet E on its own is pretty useless, while E+R together unlock a huge amount of useful information: a full encyclopaedia of the world. What's going on here?

Grounded information

What's going on is that E contains a large amount of grounded information: information about the actual world. But when the meaning of Linear A was lost that grounding was lost too: it became just a series of symbols with a certain pattern to it. Its entropy (in the information-theory sense) did not change; but now it became essentially a random sequence with that particular entropy and pattern.

Adding in R restores that grounding, and so transforms E back into a collection of useful information about the world; again, without changing its information content in the entropy sense.

This is the symbol grounding problem, not seen as an isosteric philosophical issue, but a practical learning problem.

Writing the encyclopaedia

To put this in symbols, let W be the world, let FW be a set of features in the world, and QW be a probability distribution over these features (see this post for details of the terminology).

An agent A has their own feature set FA, in Linear A, which corresponds to the features of the world. The correspondence is given by the map gA:FW→FA. The agent also knows QW.

That A then sits down to write the encyclopaedia E; in it, they put a lot of their knowledge of QW, using the terms in FA to do so.

Thus any agent B that knows gA, the grounding of A's symbols, can use E to figure out a lot about QW and hence about the world.

What if agent B doesn't know Linear A, but does know English? Then they have their own symbol grounding gB:FW→FB. The encyclopaedia E is almost useless to them. It still has the same amount of information, but t...
Continuing the takeoffs debate
by Richard_Ngo — 2020-11-23T15:58:48.189Z

Here’s an intuitively compelling argument: only a few million years after diverging from chimpanzees, humans became much more capable, at a rate that was very rapid compared with previous progress. This supports the idea that AIs will, at some point, also start becoming more capable at a very rapid rate. Paul Christiano has made an influential response; the goal of this post is to evaluate and critique it. Note that the arguments discussed in this post are quite speculative and uncertain, and also cover only a small proportion of the factors which should influence our views on takeoff speeds - so in the process of writing it I’ve made only a small update towards very fast takeoff. Also, given that Paul’s vision of a continuous takeoff occurs much faster than any mainstream view, I expect that even totally resolving this debate would have relatively few implications for AI safety work. So it's probably more useful to compare both Paul and Eliezer's scenarios against more mainstream views, than against each other. Nevertheless, it’s disappointing that such an influential argument has received so little engagement, so I wanted to use this post to explore some of the uncertainties around the issue.

I’ll call Paul’s argument the changing selection pressures argument, and quote it here at length:

Chimpanzees evolution is not primarily selecting for making and using technology, for doing science, or for facilitating cultural accumulation. The task faced by a chimp is largely independent of the abilities that give humans such a huge fitness advantage. It’s not completely independent - the overlap is the only reason that evolution eventually produces humans - but it’s different enough that we should not be surprised if there are simple changes to chimps that would make them much better at designing technology or doing science or accumulating culture.

Relatedly, evolution changes what it is optimizing for over evolutionary time: as a creature and its environment change, the returns to different skills can change, and they can potentially change very quickly. So it seems easy for evolution to shift from “not caring about X” to “caring about X,” but nothing analogous will happen for AI projects. (In fact a similar thing often does happen while optimizing something with SGD, but it doesn’t happen at the level of the ML community as a whole.)

If we step back from skills

...
Retrospective: November 10-day virtual meditation retreat
by Kaj_Sotala — 2020-11-23T15:00:07.011Z

So yesterday I finished a 10-day virtual meditation retreat taught by Tucker Peck and Upasaka Upali.

Several people have asked me what it was like, so here are some highlights.

First, a “virtual” retreat means that you spend 10 days doing pretty much nothing but meditation, and also don’t talk to anyone except the teachers, who hold daily lectures and once-every-two-days personal interviews over Zoom. Also, when you sit down to meditate, you are encouraged to do it in front of a camera, so that you can see everyone else who is meditating and they can also see you.

At times it was great, such as when I was mostly just doing concentration meditation and focusing on my breath, and then suddenly memories of playing XCOM: Enemy Unknown together with a friend came up and I just felt a strong sense of connectedness and loving-kindness towards her, even though I hadn’t even been doing loving-kindness practice.

At other times I was figuratively clawing my eyes out of boredom and a desire to just be back on social media and able to talk to people.

In retrospect, it feels odd that the boredom was sometimes so strong as to make it impossible to meditate, since if I hadn’t been bored I could simply have meditated, and I was bored because I couldn’t get the meditation to work… it now feels like what was actually going was some desire to be in control, and that clinging onto the desire to be on social media and check my messages was a way of asserting a sense of control. Or something like that. Something to look into, anyway. In any case, it was a good opportunity to investigate the nature of discomfort, and I got quite a bit of that done.

Things that felt like significant shifts, or at least interesting experiences:

* I went into the retreat with the thought of wanting to give The Mind Illuminated -style concentration meditation another try, since it had worked well for me before, but I had eventually ran into various roadblocks with it. Over the last few years, every now and then I have tried it for a bit, maybe gotten a bit of initial success, and then had it stop working again.

What I noticed this time was that following the breath felt hard because it would bring up unpleasant sensations in my belly – sensations which pretty much only pop up when I’m doing meditation, so have to be psychogenic. So this time I decided to investigate those sensations. Shifting my attention on t...
The Mutant Game - Rounds 11 to 30
by lsusr — 2020-11-23T09:20:17.475Z

This game continues from the alternate timeline here where I made two mistakes in the game engine.

Bots were passed their own previous move and told it was their opponent's previous move.
Bots were always given 0 as the round index instead of the correct positive integer.

CloneBots

Multiple people have noted that CliqueZviBot is outperforming the other CloneBots. This is due to how the CloneBot code interacts with the bugs in the my engine.

The CloneBots still cooperate, but they do so imperfectly. All CloneBot pairings result in 200-300 splits instead of 250-250 splits. The CloneBots use source code parity combined with round number parity to determine who wins the 200-300 split. Therefore if CloneBotA and CloneBotB get a 200-300 split in favor of CloneBotB then they will always get a 200-300 split in favor of CloneBotB.

Rounds 11-30

Round 11

Looking at the obituary I suspect that CooperateBot may not last much longer.

― Prediction by Larks after seeing the results from Rounds 1 to 10

Larks' CooperateBot died on round 11.

Round 12

PasswordBot from Team Multics died along with "Why can't we all just get along" from Chaos Army and an NPC.

Round 13

No casualties.

Round 14

6 bots died.

BeauBot, OscillatingTwoThreeBot, RandomOrGreedyBot and SimplePatternFinderBot from Chaos Army
"Definitely Not Collusion Bot" from Team Multics. Multicore's fodder has been consumed. Team Multics contains only the MimicBot from here on.
1 NPC

Round 15

5 bots died.

Silly Invert Bot 2, AttemptAtFair, Insum's CooperateBot, MeasureBot and "Random-start-turn-taking" from Chaos Army
1 NPC

MeasureBot had succeeded in infecting AbstractSpyTreeBot's move method and replacing it with return 0. AbstractSpyTreeBot ought perform better with MeasureBot out of the game.

Round 16

3 bots from Chaos Army died

"Silly Counter Invert Bot"
LiamGoddard
"Pure TFT"

Rounds 17-22

4 NPCs died

Round 23

BendBot and Copoperater [sic] died. BendBot belonged to Zvi. CliqueZviBot does not actually belong to Zvi. It is named after Zvi's strategy from the original name.

Round 24

No casualties.

Round 25

Copybot Deluxe died.

Round 26

RaterBot died. RaterBot performed semantic analysis on its opponents' source code. This may have contributed to breaking the symmetry of the clones.

Round 27

No casualties.

Round 28

Empiricist died. Empiricist was the most complicated bot I agreed to write the code for. The bot is exemplar of a precise, well-written spec of a clev...
Mark Manson and Evidence-Based Personal Development
by Chris_Leong — 2020-11-23T05:13:48.056Z

Most personal development is notoriously unreliable.

Mark Manson, a popular personal development author, is making an effort to make his advice more scientific. To this end, he has started labelling his blog posts based on their evidence base. He writes that he has: "put together a team of Psychologists with MSc’s and PhD’s to help me research, outline and fact-check the content here on the site". He divides his blog posts into four categories:

Evidence-based: for posts that recommend actions based on academic research, where it has been checked that studies have replicated and have decent sample sizes, ect.
Fact-checked: for posts that have been fact checked and are descriptive (he only uses evidence-based to describe posts that recommend actions)
Theory: Rooted in academic theories that might be highly theoretical or not yet confirmed
Opinion: Articles based purely off his own personal opinion

Anyway, just thought that I'd mention this here, because it's great to see an example of a popular author shifting more this direction and adopting their own (less nerdy) version of epistemic statuses.
Changing the AI race payoff matrix
by Gurkenglas — 2020-11-22T22:25:18.355Z

Suppose that AI capability research is done, but AI safety research is ongoing. Any of the major players can launch an AI at the press of a button to win the cosmos. The longer everyone waits, the lower the chance that the cosmos is paperclips. The default is that someone will press the button once they prefer their chance at an intact cosmos to risking the race going on further. This unfortunate situation could be helped by the fact that pressing the button need not be obvious by the other players. So suppose that the winner decides to lay low and smite whoever presses the button thereafter*. Then other people would have an incentive not to press the button that goes up over time!

Let paperclip probability p(t):=e^-t decay exponentially. Let t' be the last time at which the one other player wouldn't press the button. What mixed button-pressing strategy do we employ to make the get-smitten probability shore up the fading paperclip probability? At time t>=t', we press the button with probability density -p'(t)=e^-t. Then the probability that our strategy ever causes paperclips is .5*e^-2t'.

*He could also just figure out what everyone else would do in any situation and reward accordingly as a strategy against one-boxers, or copy the planet ten times over as a strategy against thirders, but this variant should work against your average human. (Turns out a large amount of strategies become available once you're omnipotent. Suggest more.)