trentmkelly/LessWrong-43k huggingface dataset:
Work on Bayesian fitting of AI trends of performance?
by Jsevillamol — 2021-07-19T18:45:19.148Z

I remember seeing a report on trends of performance in AI. It would be quite useful for me to find it again. As I remember it, the author looked at things like ImageNet top5 accuracy and extrapolated how it increased over time.
I can't remember the author, though I think it was sponsored by OpenPhil. Anyone knows what I am talking about?
I'd also be interested in related work about predicting trends of performance in technology more in general :)

EDIT: I found the report I was thinking about (which had nothing to do with OpenPhil oops). Still interested in other work in forecasting trends of performance if anyone has any recommendations!
Is the argument that AI is an xrisk valid?
by MACannon — 2021-07-19T13:20:57.379Z
Link: https://onlinelibrary.wiley.com/doi/10.1111/rati.12320

Hi folks,

My supervisor and I co-authored a philosophy paper on the argument that AI represents an existential risk. That paper has just been published in Ratio. We figured LessWrong would be able to catch things in it which we might have missed and, either way, hope it might provoke a conversation.

We reconstructed what we take to be the argument for how AI becomes an xrisk as follows:

The "Singularity" Claim: Artificial Superintelligence is possible and would be out of human control.
The Orthogonality Thesis: More or less any less of intelligence is compatible with more or less any final goal. (as per Bostrom's 2014 definition)

From the conjuction of these two presmises, we can conclude that ASI is possible, it might have a goal, instrumental or final, which is at odds with human existence, and, given the ASI would be out of our control, that the ASI is an xrisk.

We then suggested that each premise seems to assume a different interpretation of 'intelligence", namely:

The "Singularity" claim assumes general intelligence
The Orthogonality Thesis assumes instrumental intelligence

If this is the case, then the premises cannot be joined together in the original argument, aka the argument is invalid.

We note that this does not mean that AI or ASI is not an xrisk, only that the the current argument to that end, as we have reconstructed it, is invalid.

Eagerly, earnestly, and gratefully looking forward to any responses.
Preparing for ambition
by Adam Zerner — 2021-07-19T06:13:10.477Z

When I was in college, I decided that I wanted to spend my life starting startups. It seemed like the perfect "career path". If things really work out for you, the reward is billions of dollars. Awesome. But here's the really cool part: even if you "fail", like that Seinfeld episode, things will still even out for you.

The consolation prize is a life working as a programmer. What does that life look like? Well, it's pretty cushy. You make good money, are treated relatively well, have the option of working remotely, and get to do something that is intellectually interesting. Not bad. And if you play your cards right, you'll be able to retire early.

So back when I was in college beginning my startup journey, this was my mindset. Don't get me wrong, I did really, really, really want to make those billions of dollars and use the money to change the world. That is definitely where the crosshairs were aiming. But at the same time, I recognized that if that didn't work out for me, life would still be pretty great.

I like to think of this as the Gravy Mindset. In the context of mashed potatoes, the gravy is "extra". Mashed potatoes are perfectly satisfying by themselves, but pouring gravy on top makes them extra good. In the context of this "career path" of starting startups, a 9-5 job as a programmer is the potato, and succeeding with a startup is the gravy. If we were to throw some random numbers around, we can say that the potato by itself is an 8/10, but if you add some gravy to it it brings it up to a 10/10.

This is how things started for me. I had this Gravy Mindset. But somewhere along the lines, things changed. Instead of it being an 8/10 and a 10/10, it became a 2/10 and a 4/10. I was so insanely fixated on the gravy that the potato started tasting pretty gross without it. And when I was fortunate enough to get a taste of some gravy, it just felt like, "Yeah, so? This is how it's supposed to be."

In the language of DHH, I had become poisoned by ambition.

In the right dose, ambition works wonders. It inspires you to achieve more, to stretch yourself beyond the comfort zone, and provides the motivation to keep going when the going gets tough. Rightfully so, ambition is universally revered.

But ambition also has a dark, addictive side that’s rarely talked about.

...

That’s exactly the danger of what too much ambition can do: Narrow the range of acceptable outcomes to

...
Compositionality: SQL and Subways
by DanB — 2021-07-19T05:37:53.097Z

Programmers like to talk about a concept called compositionality,
which is the idea that programs and code should
consist of small, modular components that fit together cleanly.
This property is considered almost a holy grail of software design,
but there's not a lot of clarity about what it means in concrete terms.
In this post I want to offer a working definition for compositionality,
and offer some examples of sofware - and infrastructure! - that meets the definition.
I'll also look at some structures that fall short,
and try to drill down into why they failed.
According to my definition, a system of tools is composeable if it meets the following conditions:

Parsimony: there is a small number of tools.

Efficiency: the tools are highly optimized (often this is possible because there is only a small number of them).

Interoperability: the output of one tool can be used as the input to another tool without a lot of additional work. More generally,
the tools fit together cleanly.

Practicality: Users can combine the tools together to perform tasks that are useful and significant in the real world.

A system that has these properties creates huge value for its users.
The system is easy to learn,
since there are just a small number of basic operations.
But these primitives can be composed together
to create an exponentially large set of complex operations,
which will be efficient because they are made up of efficient components.
Finally, if the Practicality requirement is met,
the large set of possible operations will overlap substantially with thet set of tasks that people actually want to perform.

Discussions of compositionality often start by mentioning the
Unix Philosophy of software design,
and I'll follow that precedent.
This philosophy emphasizes building small, modular programs,
that can be connected together easily.
Unix offers an elegant tool for chaining programs together called the Pipe,
which causes the output of one program to be used as the input of the next program
(in between stages, the data is stored in an OS-managed buffer that the programs don't need to know about).
This technique allows the user to create powerful applications
just by typing a few commands together at a prompt.
For example,
several years ago I had a job at an ad-tech company,
where we had huge logs of tab-separated data that represented
the billions of ads the company bought...
Happy paths and the planning fallacy
by Adam Zerner — 2021-07-18T23:26:30.920Z

The other day at work I was talking with my boss about a project I am working on. He wanted an update on what tasks are remaining. I spent some time walking him through this. At the end, he asked, "So just to confirm, these six tasks are all that is remaining for this project?".

I wasn't sure how to respond. It didn't feel true that those were the only six tasks that were remaining. But if it wasn't true, then what are the other remaining tasks? I didn't have an answer to that, so I was confused.

Then it hit me. Those six were the only known tasks that were remaining. But there very well might be unknown tasks that are left to do. So I told him this, and that we should expect the unexpected.

He responded by saying that he agrees, but also rephrased his question. He asked me to confirm that if we're talking about the happy path, that the six tasks I outlined are the only ones on the happy path.

I don't see a problem with talking about happy paths. Well, at least not in theory. In theory, it might be useful to have a sense of what that best case scenario path forward looks like. But in practice, I worry that, somewhere along the way, happy paths will get converted to expected paths.

That's a little tangential to the main point I want to make in this post though. The main point I want to make in this post is that the terminology of happy path, expected path, and unhappy path seems like it'd be really helpful for thinking about and fighting against the planning fallacy.

What is the planning fallacy? Well, using this new terminology, the planning fallacy is our tendency to confuse the happy path with the expected path. When people think about how long something will take − what the expected path is − they tend to think mostly about what the happy path looks like. And therefore, they tend to underestimate how long things will take. Or how many resources they will require.

For example, consider students predicting how long their homework will take:

Buehler et al. asked their students for estimates of when they (the students) thought they would complete their personal academic projects. Specifically, the researchers asked for estimated times by which the students thought it was 50%, 75%, and 99% probable their personal projects would be done. Would you care to guess how many students finished on or before their estimated 50%, 75%, and 99% probability levels?

13% of subje

...
One Study, Many Results (Matt Clancy)
by aarongertler — 2021-07-18T23:10:24.588Z
Link: https://mattsclancy.substack.com/p/one-study-many-results

I didn't see this post, its author, or the study involved elsewhere on LW, so I'm crossposting the content. Let me know if this is redundant, and I'll take it down.

Summary

This post looks at cases where teams of researchers all began with the same data, then used it to answer a question — and got a bunch of different answers, based on their different approaches to statistical testing, "judgment calls", etc.

This shows the difficulty of doing good replication work even without publication bias; none of the teams here had any special incentive to come up with a certain result, and they all seemed to be doing their best to really answer the question.

Also, I'll copy the conclusion of the post and put it here:

More broadly, I take away three things from this literature:

Failures to replicate are to be expected, given the state of our methodological technology, even in the best circumstances, even if there’s no publication bias.
Form your ideas based on suites of papers, or entire literatures, not primarily on individual studies.
There is plenty of randomness in the research process for publication bias to exploit. More on that in the future.

The post

Science is commonly understood as being a lot more certain than it is. In popular science books and articles, an extremely common approach is to pair a deep dive into one study with an illustrative anecdote. The implication is that’s enough: the study discovered something deep, and the anecdote made the discovery accessible. Or take the coverage of science in the popular press (and even the academic press): most coverage of science revolves around highlighting the results of a single new (cool) study. Again, the implication is that one study is enough to know something new. This isn’t universal, and I think coverage has become more cautious and nuanced in some outlets during the era of covid-19, but it’s common enough that for many people “believe science” is a sincere mantra, as if science made pronouncements in the same way religions do.

But that’s not the way it works. Single studies - especially in the social sciences - are not certain. In the 2010s, it has become clear that a lot of studies (maybe the majority) do not replicate. The failure of studies to replicate is often blamed (

not without evidence

) on a bias towards publishing new and exciting results. Consciously or subconsciously, that leads scientists ...
Book Review: The Signal and the Noise
by Sam Enright — 2021-07-18T22:05:44.093Z

This is a compressed version of a blog post I wrote on my personal website.

The Signal and the Noise: The Art and Science of Prediction

was written by Nate Silver, a political analyst who is most famous for creating the election forecasting website

fivethirtyeight.com

. The Signal and the Noise is one of the small number of popular books about forecasting, hence why I thought this write-up would be useful.

Weather forecasting

Humans don’t have a good track record predicting the outcomes of complex systems. But one domain where we have excelled is weather forecasting. Weather forecasts are amazingly accurate relative to the complexity involved. In the mid-70s, the US National Weather Service was off by about 6 degrees (Fahrenheit) when trying to forecast three days in advance. This isn’t much more accurate than what you get if you look at long-term averages – as in, what temperature is most likely in this region at this time of year, not taking into account any specific information. Now, the average miss is 3.5 degrees. This is actually slightly less of an improvement than I would have guessed, although to reduce the error in a forecast by a factor of two requires way more than twice the effort, since errors can compound.

I was surprised to learn how large a role humans still play in weather forecasting. A human expert assessing many computer-generated forecasts is often better than any of the forecasts are by themselves. Humans make precipitation forecasts 25% more accurate than computers alone and temperature forecasts 10% more accurate. Moreover, the accuracy added by humans has not significantly changed over time, so humans have been getting better at the same rate as the machines! If you’re wondering why the weather forecasts you use don’t feel very accurate, it’s in part because weather services are private companies that tend to exaggerate forecasts for appeal; you won’t see this inaccuracy in government forecasts. In particular, meteorologists are known to have a “wet bias” – they forecast rain more often than it actually occurs.

There have been some pretty tremendous positive externalities of commercial weather forecasting, most notably in creating sophisticated early warning systems for extreme weather. The ability to predict typhoons in India and Bangladesh has probably saved many thousands of lives. Silver has a few stories in here ab...
Any taxonomies of conscious experience?
by JohnDavidBustard — 2021-07-18T18:28:50.444Z

I have some expertise in machine learning and AI. I broadly believe that human minds are similar to modern AI algorithms such as deep learning and reinforcement learning. I also believe that it is likely that consciousness is present wherever algorithms are executing (a form of panpsychism). I am trying to create theories about how AI algorithms could generate conscious experiences. For example, it may be the case that when an AI is in a situation where it believes that many actions it could take will lead to an improvement in its situation it might feel happiness. If it feels that most choices will lead to a worse situation and it is searching for the least worst option, it might feel fear and sadness. I am trying to find existing research that might give me a taxonomy of conscious experiences (ideally with associated experimental data e.g. surveys etc.) that I could use to define a scope of experiences that I could then try to map onto the execution of machine learning algorithms. Ideally I am looking for taxonomies that are quite comprehensive, I have found other taxonomies very useful in the past for similar goals, such as Wordnet, ConceptNet, TimeUse surveys, DSM (psychology diagnosis) etc.

I have a very limited understanding of phenomenology and believe that its goals in understanding conscious experience may be relevant but I am concerned that it is not a subject that is presented in a systematic textbook style format that I am looking for. I would be very grateful for any suggestions as to where I might find any systematic overview that I might be able to use. Perhaps from teaching materials or something from Wikipedia or any other source that attempts this kind of broad systematic taxonomy.
A model of decision-making in the brain (the short version)
by Steven Byrnes — 2021-07-18T14:39:35.338Z

(UPDATE: For a revised-and-improved version of this post, see this later post I wrote (especially Section 6.2.2).)

I’ve been playing around with a hypothesis about how decision-making is laid out in the human brain. Regular readers of my posts have seen this multiple times before; in particular I discussed it in great detail in

Big Picture of Phasic Dopamine

. But that’s a long post covering many topics, and recently I've found that I keep wanting to talk about (and solicit feedback about) just this one thing.

So here it is in brief, stripped of all the details, justifications, uncertainties, references, etc. Email me or comment at the lesswrong crosspost if you want to discuss, tell me I’m wrong, etc.

Neuroscience abbreviations: dlPFC = dorsolateral prefrontal cortex; DS, VS = dorsal & ventral striatum; mPFC = medial prefrontal cortex.

The model, in four steps

Step 1 (top-left of the diagram): there are parts of the cortex (dorsolateral prefrontal cortex, hippocampus) that propose thoughts, plans, etc. You can think of these as having some attractor dynamics or whatever, such that you start with a bubbly soup of partially-formed mutually-incompatible sub-thoughts, and then they quickly settle into a stable, fully-formed thought. The striatum intervenes during this process, making a quick rough guess at how promising the different pieces look, suppressing the less promising bits and enhancing the more promising bits, so that when you get a fully-formed thought, it’s likelier to be fairly promising.

Step 2 (top-right of the diagram): once you have a stable fully-formed thought, various other parts of the brain (mainly medial prefrontal cortex, anterior cingulate cortex, ventral striatum, amygdala, hippocampus (sorta)) “assess” that thought according to maybe dozens-to-hundreds of genetically-hardcoded criteria like “If I'm gonna do this plan, how appropriate would it be to cringe? To salivate? To release cortisol? To laugh? How much salt would I wind up eating? How much umami?” Etc. etc. And they send this scorecard down to the hypothalamus and brainstem.

Step 3 (bottom of the diagram): Finally, we're at the hypothalamus & brainstem. They look at the scorecard from the previous step, and they combine that information with other information streams that they have access to, like metabolic status information—if I'm hungry, a plan that will involve eating gets e...
The Mountaineer's Fallacy
by Alex Power — 2021-07-17T23:45:54.154Z

I'm sure this has a name, but I can't remember it. So I have given it a new name. The Mountaineer's Fallacy.

The Mountaineer's Fallacy is the suggestion that climbing Mount Everest is a good start on a trip to the moon.

In one sense, you are making progress on a key metric: distance from the moon. But in another, more accurate sense, you are wasting a lot of time and effort on something that has no chance of helping you get to the moon.
Reflecting on building my own tools from scratch and 'inventing on principle'
by Amir Bolous — 2021-07-17T18:43:38.772Z

This is a linkpost for https://amirbolous.com/posts/build

Introduction

Tools and Products

Dogma

Building is Empowering

Closing Thoughts

Introduction

For the past couple of months, I've built personal tools completely from scratch. I built a way to organize and record my thoughts. I built a programming language to understand Lisp. I built a tool to generate cards for family and friends. I built a web framework from scratch. I used that web framework to ship a couple of projects.

The weird thing is that, until recently, I didn't know that I could do this. I didn't even realize this was an option or move I could make. I didn't know that if I had an opinion for how things should be, I had the right to push my view of the world into reality by doing something about it. Bret Victor has an incredible talk called "Inventing on Principle" where he touches on just this.

Several months later of trying to do this, I can see exactly what he meant. Here's a story/reflection on this:

Tools and Products

Every now and then, a new tool or technology becomes mainstream and garners a passionate cult of followers advocating for it. For example, this happened with Roam. This happened with Notion. This has been happening with tools and technologies long before we invented them. Often, when I would try these mainstream tools, some of them stuck. However, more often than not, I was left confused and somewhat disappointed.

That's not to say that these tools or technologies are not great products. Not at all. They're clearly good products because many people use them and recommend them to others.

Dogma

I realize now that reason these products or technologies didn't stick was because of a misaligned dogma. This is the mental model I now keep in mind: any product or tool has the potential to be great if it can solve a high frequency, high intensity problem for at least one person. Sometimes, it can even be a low frequency, high intensity problem or vice versa. For example, Uber solved the problem of moving from one place to another conveniently. React helped tackle building fast, flexible, and rich websites.

However, any technology or tool also imposes an expensive cost. It carries a dogma for how things should be. A set of implicit (e.g. react component patterns) or explicit (e.g you can't call an Uber if no driver is nearby) rules that are imposed on how you use the tool/technology and ...