trentmkelly/LessWrong-43k huggingface dataset:
AI and Efficiency
by DragonGod — 2020-07-27T20:58:43.625Z
Link: https://openai.com/blog/ai-and-efficiency/

The amount of compute required to train neural networks to SOTA (state of the art) on some tasks has been decreasing at an exponential rate. Efficiency of training AI models seems to be growing faster than Moore's law.

From June 2012 to May 2019, the amount of compute needed to train a neural network to AlexNet level of performance fell 44x (halving every 16 months). In contrast, Moore's Law would have provided an 11x doubling in available compute.

There are other examples of exponential gains in efficiency. From the article:

We saw a similar rate of training efficiency improvement for ResNet-50 level performance on ImageNet (17-month doubling time).716 We saw faster rates of improvement over shorter timescales in Translation, Go, and Dota 2:

Within translation, the Transformer22 surpassed seq2seq23 performance on English to French translation on WMT’14 with 61x less training compute 3 years later.

We estimate AlphaZero24 took 8x less compute to get to AlphaGoZero25 level performance 1 year later.

OpenAI Five Rerun required 5x less training compute to surpass OpenAI Five26 (which beat the world champions, OG) 3 months later.
Why You Might Want a New Way to Visualize Biases
by dtm — 2020-07-27T17:30:45.065Z

A few days ago I made a post called A New Way to Visualize Biases, and the community seems to have been generally skeptical as to the value of the diagram. I don't think the original post made clear enough the types of use cases I was primarily interested in, and I know I haven't perfected the model yet, but the primary reason I posted was to get some clarity and ideas from LW on how I might do that, which some commenters offered.

But I do want to defend the potential value of a framework like this. If I had just been trying to understand biases in isolation, I probably never would have seen a reason to develop a diagram like this either, but I came to it from a different angle.

Most people are familiar with the idea that our thoughts and reasoning play a large role in our emotions and behaviors. Cognitive behavioral therapy (CBT), the most empirically effective therapeutic treatment for depression and other disorders to date, is based on the idea that cognition mediates our emotions, and biased and distorted thoughts are very often responsible for maladaptive emotions and mood disorders.

David Burns, author of Feeling Good says "Our research reveals the unexpected: depression is not an emotional disorder at all! The sudden change in the way you feel is of no more causal relevance than a runny nose is when you have a cold. Every bad feeling you have is the result of your distorted negative thinking." You can dispute whether this analysis oversimplifies emotion or depression, but it is clear that many negative and harmful emotions are the result of cognitive biases.

One of the most effective techniques for eliminating persistently negative emotional cycles is called cognitive restructuring. If you wanted to make use of cognitive restructuring, you might keep a log of your emotions and the thoughts and circumstances which may have preceded or triggered them, and mapping out the chain to identify which psychological algorithms are systematically leading to these emotional spirals. Then you could use the Socratic method to attempt to spot and remove the biased thoughts which triggered your emotions.

Here is a simple example of a chain of automatic psychological algorithms resulting in maladaptive emotions:

So if you read the original post and generally understand how I'm trying to use the diagrams, you will see that I have added nodes for emo...
Are we in an AI overhang?
by Andy Jones — 2020-07-27T12:48:05.196Z

Over on Developmental Stages of GPTs, orthonormal mentions

it at least reduces the chance of a hardware overhang.

An overhang is when you have had the ability to build transformative AI for quite some time, but you haven't because no-one's realised it's possible. Then someone does and surprise! It's a lot more capable than everyone expected.

I am worried we're in an overhang right now. I think we right now have the ability to build an orders-of-magnitude more powerful system than we already have, and I think GPT-3 is the trigger for 100x larger projects at Google, Facebook and the like, with timelines measured in months.

Investment Bounds

GPT-3 is the first AI system that has obvious, immediate, transformative economic value. While much hay has been made about how much more expensive it is than a typical AI research project, in the wider context of megacorp investment, its costs are insignificant.

GPT-3 has been estimated to cost $5m in compute to train, and - looking at the author list and OpenAI's overall size - maybe another $10m in labour.

Google, Amazon and Microsoft each spend about $20bn/year on R&D and another $20bn each on capital expenditure. Very roughly, it totals to $100bn/year. Against this budget, dropping $1bn or more on scaling GPT up by another factor of 100x is entirely plausible right now. All that's necessary is that tech executives stop thinking of natural language processing as cutesy blue-sky research and start thinking in terms of quarters-till-profitability.

A concrete example is Waymo, which is raising $2bn investment rounds - and that's for a technology with a much longer road to market.

Compute Cost

The other side of the equation is compute cost. The $5m GPT-3 training cost estimate comes from using V100s at $10k/unit and 30 TFLOPS, which is the performance without tensor cores being considered. Amortized over a year, this gives you about $1000/PFLOPS-day.

However, this cost is driven up an order of magnitude by NVIDIA's monopolistic cloud contracts, while performance will be higher when taking tensor cores into account. The current hardware floor is nearer to the RTX 2080 TI's $1k/unit for 125 tensor-core TFLOPS, and that gives you $25/PFLOPS-day. This roughly aligns with AI Impacts’ current estimates, and offers another >10x speedup to our model.

I strongly suspect other bott...
Generalizing the Power-Seeking Theorems
by TurnTrout — 2020-07-27T00:28:25.677Z

Previously: Seeking Power is Often Provably Instrumentally Convergent in MDPs.

Circa 2021, the above post was revamped to supersede this one, so I recommend just reading that instead.

Thanks to Rohin Shah, Michael Dennis, Josh Turner, and Evan Hubinger for comments.

The original post contained proof sketches for non-IID reward function distributions. I think the actual non-IID theorems look different than I thought, and so I've removed the proof sketches in the meantime.

It sure seems like gaining power over the environment is instrumentally convergent (optimal for a wide range of agent goals). You can turn this into math and prove things about it. Given some distribution over agent goals, we want to be able to formally describe how optimal action tends to flow through the future.

Does gaining money tend to be optimal? Avoiding shutdown? When? How do we know?

Optimal Farsighted Agents Tend to Seek Power proved that, when you distribute reward fairly and evenly across states (IID), it's instrumentally convergent to gain access to lots of final states (which are absorbing, in that the agent keeps on experiencing the final state). The theorems apply when you don't discount the future (you're "infinitely farsighted").

Most reward functions for the Pac-Man game incentivize not dying immediately, so that the agent can loop around higher-scoring configurations.

Many ways of scoring Tic-Tac-Toe game states incentivize not losing immediately, in order to choose the highest-scoring final configuration.

"All states have self-loops, left hidden to reduce clutter.
In AI: A Modern Approach (3e), the agent starts at 1 and receives reward for reaching 3. The optimal policy for this reward function avoids 2, and one might suspect that avoiding 2 is instrumentally convergent. However, a skeptic might provide a reward function for which navigating to 2 is optimal, and then argue that "instrumental convergence'' is subjective and that there is no reasonable basis for concluding that 2 is generally avoided.
We can do better... for any way of independently and identically distributing reward over states, 1011 of reward functions have farsighted optimal policies which avoid 2. If we complicate the MDP with additional terminal states, this number further approaches 1.
If we suppose that the agent wil

...
Change the world a little bit
by DirectedEvolution — 2020-07-26T22:35:05.546Z

Working directly in the medium

I'm in the middle of building the largest and most complex piece of software I've ever worked on. It's a stepping-stone tool for high schoolers and undergraduates to start reading scientific literature.

The basic features are all in my head. As I code, I build whatever gives me the most bang for my buck. So far, I've been able to create a pretty sophisticated working prototype without a to-do list or any other detailed plan.

Very often, I have to stop and skill up before I can build a feature. I might just read an article or post on Stack Overflow. For bigger learning projects, I take a Coursera course on some concept I'm unfamiliar with. I just plow through it and keep getting things done.

When you're working directly in your medium, the project itself tells you what to do next. It reveals itself step by step, moment by moment.

It's the same when I'm teaching music. I never plan lessons. The right thing to do reveals itself to me from moment to moment.

Useful planning and meta-level thinking

Useful planning is generally obvious. Make a list before you go grocery shopping. Talk out what your needs and desires are when house-buying with your spouse. Check the personality traits of the dog breeds you're considering. You have an obvious question, you look up the answer, and you move on.

Try before you buy. Do some independent research before you apply to a PhD program. Hang out a lot before you have sex. Take the car for a test drive.

Bad planning

It's the finger pointing at the moon, not the moon itself.

It's a long to-do list that doesn't translate into action. A spreadsheet where you gather information in order to forget about it. A long chain of thought culminating in an epiphany that goes nowhere. An argument about an issue that you never work on directly.

Bad planning like a belief in telepathy. It makes you feel like your private thoughts can change the world. The quintessential example? A college humanities essay that gets read by the student, the professor, and nobody else, but which the student remains proud of for the rest of their life.

The mechanism is the mission

If you have a big dream, set yourself up to work directly in a medium that would help you realize it. If you want to build technology, learn to code, build electronics, machine metal. If you want to do work in the life sciences, set up a little home lab. If you want to sp...
Developmental Stages of GPTs
by orthonormal — 2020-07-26T22:03:19.588Z

Epistemic Status: I only know as much as anyone else in my reference class (I build ML models, I can grok the GPT papers, and I don't work for OpenAI or a similar lab). But I think my thesis is original.

Related:

Gwern on GPT-3

For the last several years, I've gone around saying that I'm worried about

transformative AI

, an AI capable of making an Industrial Revolution sized impact (the concept is agnostic on whether it has to be AGI or self-improving), because I think we might be one or two cognitive breakthroughs away from building one.

GPT-3 has made me move up my timelines, because it makes me think we might need zero more cognitive breakthroughs, just more refinement / efficiency / computing power: basically, GPT-6 or GPT-7 might do it. My reason for thinking this is comparing GPT-3 to GPT-2, and reflecting on what the differences say about the "missing pieces" for transformative AI.

My Thesis:

The difference between GPT-2 and GPT-3 has made me suspect that there's a legitimate comparison to be made between the scale of a network architecture like the GPTs, and some analogue of "developmental stages" of the resulting network. Furthermore, it's plausible to me that the functions needed to be a transformative AI are covered by a moderate number of such developmental stages, without requiring additional structure. Thus GPT-N would be a transformative AI, for some not-too-large N, and we need to redouble our efforts on ways to align such AIs.

The thesis doesn't strongly imply that we'll reach transformative AI via GPT-N especially soon; I have wide uncertainty, even given the thesis, about how large we should expect N to be, and whether the scaling of training and of computation slows down progress before then. But it's also plausible to me now that the timeline is only a few years, and that no fundamentally different approach will succeed before then. And that scares me.

Architecture and Scaling

GPT

,

GPT-2

, and

GPT-3

use nearly the same architecture; each paper says as much, with a sentence or two about minor improvements to the individual transformers. Model size (and the amount of training computation) is really the only difference.

GPT took 1 petaflop/s-day to train 117M parameters, GPT-2 took 10 petaflop/s-days to train 1.5B parameters, and the largest version of GPT-3 took 3,000 petaflop/s-days to train 175B parameters. By contrast, AlphaStar seems to hav...
The problem of Pop Analogy
by thesilvermen — 2020-07-26T21:44:43.066Z

There is a very good article pinned to the start of Less Wrong at the time I am writing this, where the author starts by pointing out that human beings ‘run on corrupted hardware.’

This is a tech analogy, which might be meant to point at that homo sapiens have not evolved to live in the environment that they currently inhabit, with many first and second-order problems derived. On that level, the analogy is solid. But it is part of a deeper struggle, the problem of language, when married to the perils of modernity.

Let us give this some context. A computer/smartphone/any digital device/program has both hardware, which encompass broadly its mechanical components, and software, which encompass the OS of 1010100111 that tell it what to do. If the computer/smartphone/any digital device/program does not give the desired output for the correct input, either the program or the mechanical components of the thing are most likely broken and malfunctioning in some way.

As I type this, I am a human being, sitting and breathing a nitrogen/oxygen mix with a body that has evolved for hunting things with fire and sharp sticks. The body reminds me of my own nature, with sharp pains in my torso when I sit in the chair for too long. The being I am was also evolved to live in a communal setting, with a tribe of others, and I am stabbed with loneliness as I tap at a mechanical contraption to make hieroglyphics on false paper and transmit them digitally on a device so that other scared, lonely hominids might see the same things that I have typed.

My hardware is working just fine. The input I am giving it is suited for some being other than the thing that I am, and the nature I have been given. I do not have hardware or software, there is no difference given in homo sapiens between mind and body.

The thing that I am comes before the time and group into which I have been born.
Jam is obsolete
by jefftk — 2020-07-26T14:00:07.179Z

Jam is a very natural way to preserve fruit. The fruit is ripe only a
small fraction of the year, you'd like to be able to eat it all year
long, so you cook it with enough sugar that it won't go bad for a long
time. But now that we have freezers, jam is obsolete.

When you make jam, you change the taste and texture of the fruit in
several ways:

Squishing the fruit makes it into a more consistent texture,
suitable for spreading.

Adding sugar makes the jam sweeter, and also much less healthy.

Heat changes the flavor of the fruit, With more volatile flavors
evaporating.

Heat evaporates some of the water, thickening it (reducing).

Heat releases pectin, or you add additional pectin, which gels the
jam.

Some people make freezer jam, which skips the heated steps, and gives
you a flavor much closer to fresh fruit. Because it hasn't been cooked
thoroughly, you need to store it in the freezer, but since freezers
are a thing now, that's not a problem.

We can go a step further, though, and just use frozen fruit. Take
frozen raspberries, defrost them, and squish them up with a fork:

on eggy waffles, with nutella

This takes about a minute for defrosting, 10s for squishing, and
is far superior to jam. Not so sweet, better flavor, and it's
cheaper [1] and healthier as well!

Being able to store traditional jam in the refrigerator once you've
opened it is an advantage, but I think it's a temporary one. Over
time, I expect households will shift towards more freezer space
relative to refrigerator space, as the quality of frozen food
continues to rise. We got a chest freezer soon after we bought our
house, and I'm glad we did!

[1] Our grocery store has raspberry jam for $2.75/lb, frozen
raspberries for $4.39/lb, and jam is ~40% fruit. Though since we have
the freezer space like raspberries a lot I've been buying them online,
30lb
at $3.08/lb shipped. A shipment fills 6-8 one-gallon ziplocs
depending on whether it ends up mostly whole or mostly pieces.

Comment via: facebook
How I use Google Forms in my Workflow
by Harri Besceli — 2020-07-26T12:17:28.138Z

I use google forms* a lot and they are a key part of my workflow. I have different forms to review my day, add to-dos and track my habits and typically fill out 3 different forms per day.

This post gives examples of forms I use and why I like them.

*[I actually use airtable forms rather than google forms. They are functionally pretty similar - I just said 'google forms' in the title as I expect it's the kind people are most familiar with.]

This is an airtable base with templates for some of the forms I use.

Forms I Use

Meta Form

This is the one I find most useful and I fill it out ~5 x per day, taking a minutes or so each time. It is composed of just two text boxes 'Thinking' and 'Next Step'. Situations I'll use the form.

Deciding what to do: I don't know what to do next. I open the form, list options in and decide what to do.
Noticing aversion: I feel averse to doing something. I open the form, write down what I'm averse to and why, and what to do about it.
Planning a task: I don't know how to go about doing something. I open the form, make a plan for the thing, and decide on the next step.

One of the main reasons is that it allows me to fully switch tasks from eg. 'trying to do something' to 'planning how to do it'. If I don't switch tasks properly I can end up half-heartedly trying to do something whilst having a vague feeling that there's a better way of doing it. Also, just having the form makes me more likely to 'go meta' and think strategically about what I'm doing.

Task Inbox

I use a form as my capture method a la Getting Things Done - basically a method to capture anything that I might want to do at some point. Once it's captured in the form, I know that I'll review this later, and so don't need to worry about it until then. Forms are fast, which feels important for my willingness to use the capture system.

Time Tracking

I use a form to track my time and prefer this to time tracking software like toggl and clockify. The main reason is that forms are easy to customise. In the past I've experimented with writing out a plan for each block of work time, and retrospectively rating my productivity after completing a block. I've found this much easier to implement in a form.

Why I Like Forms

I use forms in cases where I want to either ...
Godel in second-order logic?
by Abhimanyu Pallavi Sudhir — 2020-07-26T07:16:26.995Z

I'm following the HAE sequence [Godel's Completeness and Incompleteness Theorem], and I can see what it means for first-order logic to be incomplete.

I don't understand how Godel's incompleteness theorem can still apply to second-order logic -- doesn't PA with second-order logic allow us to uniquely pin down a model?

But Godel's sentence can still be constructed, so second-order logic must still be incomplete -- I just don't understand how we can have models with a "weird idea of what it means to be a proof" if we don't have non-standard numbers.

I asked on Math Stackexchange and was told that second-order logic "doesn't have a computable deduction system". Unfortunately, I don't really understand what this means.
A Natural Explanation of Nash Equilibria
by Dr. Birdbrain — 2020-07-25T23:40:38.031Z

(This article is cross-posted from my blog)

Imagine a world in which agents may interact, but may not coordinate in an enforceable way. Agents may communicate, but there is no mechanism to hold another agent accountable to commitments reached while communicating. In this world, all conflicts of interest between agents would resolve to a Nash Equilibrium.

A Nash Equilibrium, sometimes described as a "self-enforcing law", is an outcome from which no agent has an incentive to unilaterally deviate.

I will talk about the Prisoner's Dilemma, an illustration of a simple scenario that is famous because its Nash Equilibrium is /bad/. If the players could coordinate to escape the pull of the force that pulls them towards the Nash Equilibrium, they would be better off.

Scott Alexander's excellent work Meditations on Moloch has demonstrated that the world is full of negative Nash Equilibria, but there are also those that yield positive real-world consequences. This essay concludes by describing how free markets under perfect competition yield a Nash Equilibrium that is positive from the point of view of consumers.

Motivating example: Prisoner's Dilemma

Two criminals are detained by the police and interrogated separately. The police do not have strong evidence to convict the criminals of the full crime, and at most can put each criminal behind bars for one year, so they need the criminals to snitch on each other to prosecute the full extent of the crime. To incentivize snitching, the officers offer the prisoners a deal. If one of the prisoners "defects" from his criminal partnership and snitches, while the other continues to "cooperate" in crime, the police will offer the snitch the opportunity to walk free, but the prisoner that does not snitch will serve nine years in prison. If both criminals defect from the criminal partnership, both will serve six years in prison.

We can write down the penalties in a convenient table form. The rows represent the action of Prisoner/Player 1, and the columns represent the action of Prisoner/Player 2. The first number is the "payoff" given to Player 1 and the second is the "payoff" to Player 2. The payoffs are represented as negative numbers because the players "lose" years of their life rotting in prison.

Player 1 Coop	Player 1 Defect
Player 2 Coop	-1, -1	-9, 0
Player 2 Defect	0, -9	-6, -6

Time travel analogy

Suppose that the players play a sing...