trentmkelly/LessWrong-43k huggingface dataset:
A Correspondence Theorem in the Maximum Entropy Framework
by johnswentworth — 2020-11-11T22:46:38.732Z

Classical mechanics didn't work any less well once we discovered quantum, Galilean relativity and Newtonian gravity didn't work any less well once we discovered special and general relativity, etc. This is the correspondence principle, aka Egan's Law: in general, to the extent that old models match reality, new models must reproduce the old.

This sounds like it should be a Theorem, not just a Law - a "correspondence theorem".

This post presents such a theorem within one particular framework - maximum entropy. It's not the theorem I'd eventually like - even my correspondence theorem from last month has more substance to it. But this one is simple, intuitive, and both its strong points and shortcomings help to illustrate what I want out of a correspondence theorem. (In fact, I wrote down this one before the other correspondence theorem, and the shortcomings of this theorem partially inspired that one.)

Background: Principle of Maximum Entropy

By "maximum entropy framework", I mean roughly the technique used by Jaynes - see e.g. the widget problem in Logic of Science, page 440. If you've seen the principle of maximum entropy in the context of statistical mechanics, it's the same thing. I expect a lot of people who are otherwise familiar with maximum entropy distributions are not familiar with this particular framework (especially outside of a physics context), so a bit of background is in order.

We'll start with the widget problem: a company produces red, green and blue widgets, and for inventory purposes we wish to predict how many of each color will be sold on a given day. Based on sales aggregated over the past year, the average sales per day are roughly 40 green, 20 red, and 10 blue. Given only this information, what distribution should we assign to tomorrow's sales?

In a high-school statistics class, the answer would be "insufficient information" - the averages are not enough info to figure out the whole distribution. But this isn't a high-school statistics class. We're Bayesians, we've received relevant information, we have to update somehow.

Jaynes argues that the canonically-correct way to do this is maximum entropy: we find the model M which has the highest possible entropy, subject to the constraints E[#green|M]=40, E[#red|M]=20, and E[#blue|M]=10. A couple ways to interpret this idea:

The classical argument from stat mech: the vast m

...
[Link] Digital Democracy Is Within Reach
by Alexei — 2020-11-11T22:12:41.438Z
Link: https://your-undivided-attention.simplecast.com/episodes/the-listening-society-yZ1PBlPF

EPISODE SUMMARY

Imagine a world where every country has a digital minister and technologically-enabled legislative bodies. Votes are completely transparent and audio and video of all conversations between lawmakers and lobbyists are available to the public immediately. Conspiracy theories are acted upon within two hours and replaced by humorous videos that clarify the truth. Imagine that expressing outrage about your local political environment turned into a participatory process where you were invited to solve that problem and even entered into a face to face group workshop. Does that sound impossible? It’s ambitious and optimistic, but that's everything that our guest this episode, Audrey Tang, digital minister of Taiwan, has been working on in her own country for many years. Audrey’s path into public service began in 2014 with her participation in the Sunflower Movement, a student-led protest in Taiwan’s parliamentary building, and she’s been building on that experience ever since, leading her country into a future of truly participatory digital democracy.
Learning Normativity: A Research Agenda
by abramdemski — 2020-11-11T21:59:41.053Z

(Related to Inaccessible Information, Learning the Prior, and Better Priors as a Safety Problem. Builds on several of my alternate alignment ideas.)

I want to talk about something which I'll call learning normativity. What is normativity? Normativity is correct behavior. I mean something related to the fuzzy concept humans convey with the word "should". I think it has several interesting features:

Norms are the result of a complex negotiation between humans, so they shouldn't necessarily be thought of as the result of maximizing some set of values. This distinguishes learning normativity from value learning.
A lot of information about norms is present in the empirical distribution of what people actually do, but you can't learn norms just by learning human behavior. This distinguishes it from imitation learning.
It's often possible to provide a lot of information in the form of "good/bad" feedback. This feedback should be interpreted more like approval-directed learning rather than RL. However, approval should not be treated as a gold standard.
Similarly, it's often possible to provide a lot of information in the form of rules, but rules are not necessarily 100% true; they are just very likely to apply in typical cases.
In general, it's possible to get very rich types of feedback, but very sparse: humans get all sorts of feedback, including not only instruction on how to act, but also how to think.
Any one piece of feedback is suspect. Teachers can make mistakes, instructions can be wrong, demonstrations can be imperfect, dictionaries can contain spelling errors, reward signals can be corrupt, and so on.

Example: Language Learning

A major motivating example for me is how language learning works in humans. There is clearly, to some degree, a "right way" and a "wrong way" to use a language. I'll call this correct usage.

One notable feature of language learning is that we don't always speak, or write, in correct usage. This means that a child learning language has to distinguish between mistakes (such as typos) and correct usage. (Humans do sometimes learn to imitate mistakes, but we have a notion of not doing so. This is unlike GPT systems learning to imitate the empirical distribution of human text.)

This means we're largely doing something like unsupervised learning, but with a notion of "correct"/"incorrect" data. We're doing something like throwing data o...
CHAI Internship Application
by martinfukui — 2020-11-11T21:10:43.990Z

Hi everyone,

I'm the Assistant Director at CHAI and as some of you may know, CHAI is currently accepting applications for our 2021 internship program.

Early deadline is 11/23 for applicants who require an earlier response from us. Our normal deadline is 12/13.

You can find more information and the application itself here

Please e-mail me at chai-admin@berkeley.edu if you have any questions!
How can we lobby to get a vaccine distributed faster?
by Tim Liptrot — 2020-11-11T21:01:16.373Z

Fellow Rats,

Tyler Cowen has argued that we can release vaccines now without compromising phase III trials through randomization. We could thus benefit from the expected value of innoculating more people earlier and of getting an answer sooner. He has proposed two mechanisms.

Randomly distribute treatments and placebos to at risk groups like bus drivers. This seems like a great idea, since bus drivers are in unusual danger and need.

Use a "tie-breaker" design, which is a hybrid of regression discontinuity and randomized control trial. Basically you want to treat some subset of the population, but want an impact assessment. So you randomize only near the cutoff for service. So we could vaccinate some of the most at-risk persons and randomize at the liminal cases, achieving an optimal tradeoff between present benefits and information. The abstract of Owen and Varians article is below.

For some reason, the US is currently implementing neither ideas. Our approach is to stockpile lots of vaccines and wait for a greenlight from a single conventional information-only trial. Cowen is mostly being ignored.

Lobbying

We can lobby the government and Pfizer to change this. The US gov has plenty of avenues for lobbying to force discussion on these ideas. Here are a few, off the top of my head.

Tweet at public health experts in the style of 1 day sooner

Call our senators, complain about FDA regulations

Call our representatives, complain

Go to our representatives office and demand to speak to the staff. Show him the paper. Demand a meeting.

Call local television stations. Read the paper in detail and prepare a speech. Build publicity

Tweet at Donald Trump directly

Call the FDA

Call think tanks affiliated with party leadership

I am uncertain which body needs to approve such a policy change. It could be mandatable from the white house, require legislation, be mandatable from the FDA, or be entirely under the pharma companies control. The easiest body to pressure is the FDA because they answer to our legislators.

Appendix A:
Motivated by customer loyalty plans and scholarship programs, we study tie-breaker designs which are hybrids of randomized controlled trials (RCTs) and regression discontinuity designs (RDDs). We quantify the statistical efficiency of a tie-breaker design in which a proportion Δ of observed subjects are in the RCT. In a two line regression, statistic...
Time in Cartesian Frames
by Scott Garrabrant — 2020-11-11T20:25:19.400Z

This is the twelfth and final post in the Cartesian Frames sequence. Read the first post here.

Up until now, we have (in the examples) mostly considered agents making a single choice, rather than acting repeatedly over time.

The actions, environments, and worlds we've considered might be extended over time. For example, imagine a prisoner's dilemma where "cooperating" requires pushing a button every day for five years.

However, our way of discussing Cartesian frames so far would treat "push the button every day for five years" as an atomic action, a single element a∈A.

Now, will begin discussing how to use Cartesian frames to explicitly represent agents passing through time. Let us start with a basic example.

1. Partial Observability

Consider a process where two players, Yosef and Zoe, collaboratively choose a three-digit binary number. Yosef first chooses the first digit, then Zoe chooses the second digit, then Yosef chooses the third digit. The world will be represented by the three-digit number. The Cartesian frame from the perspective of Yosef looks like this:

C0=⎛⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜⎝000010000010001011001011000011000011001010001010100110110100101111111101100111111100101110110101⎞⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟⎠.

Here, C0=(A0,E0,⋅0) is a Cartesian frame over W0={000, 001, 010, 011, 100, 101, 110, 111}.

The four possible environments from left to right represent Zoe choosing 0, Zoe choosing 1, Zoe copying the first digit, and Zoe negating the first digit.

The eight possible agents can be broken up into two groups of four. In the top four possible agents, Yosef chooses 0 for the first digit, while in the bottom four, he chooses 1. Within each group, the four possible agents represent Yosef choosing 0 for the third digit, choosing 1 for the third digit, copying the second digit, and negating the second digit.

Consider the three partitions W1, W2, and W3 of W0 representing the first, second and third digits respectively. Wi={w0i,w1i}, where w01={000, 001, 010, 011}, w11={100, 101, 110, 111}, w02={000, 001, 100, 101}, w12={010, 011, 110, 111}, w03={000, 010, 100, 110}, and w13={001, 011, 101, 111}.

Clearly, by the definition of observables, W2&...
[AN #125]: Neural network scaling laws across multiple modalities
by Rohin Shah — 2020-11-11T18:20:04.504Z

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

Please note that while I work at DeepMind, this newsletter represents my personal views and not those of my employer.

SECTIONS

HIGHLIGHTS

TECHNICAL AI ALIGNMENT

MESA OPTIMIZATION

FORECASTING

OTHER PROGRESS IN AI

REINFORCEMENT LEARNING

HIGHLIGHTS

Scaling Laws for Autoregressive Generative Modeling (Tom Henighan, Jared Kaplan, Mor Katz et al) (summarized by Asya): This paper looks at scaling laws for generative Transformer models of images (predicting pixels or parts of image encodings), videos (predicting frames of image encodings), multimodal image <-> text (predicting captions based on images or images based on captions), and mathematical problem solving (predicting answers to auto-generated questions about algebra, arithmetic, calculus, comparisons, integer properties, measurement, polynomials, and probability). The authors find that:

- Cross-entropy loss as a function of compute follows a power law + constant in all these data modalities (just as it does in language (AN #87)). Information theoretically, this can be interpreted as scaling a 'reducible loss' which estimates the KL divergence between the true and model distributions, and an 'irreducible loss' which estimates the entropy of the true data distribution.

- Performance on ImageNet classification fine-tuned from their generative image model also follows such a power law, whereas ImageNet classification trained from scratch actually gets worse with sufficiently large model sizes. Interestingly, this classification power law continues even past model sizes where the generative cross-entropy loss starts bending as a result of irreducible loss. The authors conclude that approaching the irreducible loss for some dataset does not necessarily indicate diminishing returns for representation quality or semantic content.

- Optimal model size as a function of compute follows a power law with an exponent very close to ~0.7 for all data modalities they've studied so far. This implies that in the current compute regime, as compute budgets grow, it's best t...
Think in Terms of Actions, not Attributes
by Collisteru — 2020-11-11T15:57:19.476Z

How do we model the actions of those around us? This problem has been crucial for human survival since before the Holocene, so the brain has developed clever tricks to solve it. Personal attributes emerged as one of these tricks, as reflected in language: Bob is reliable, while Sally's a perfectionist. When these attributes are extended to groups, they become stereotypes: politicians are liars, but priests are honest.

This allows us to categorize people and then predict the actions of any person with a certain attribute. Rather than ask, "What would Bob do?" we ask, "What would a reliable person do?" It becomes unnecessary to examine individuals.

This type of reasoning creates problems. On a small scale, it can cause prejudice, rash judgement, and hurt feelings. On a large scale, it can lead to demographic conflict, systemic bias, and social ruin. We can do better! How can we reliably predict the others' actions without relying on stereotypes?

The answer is to look directly at the data—previous actions. While we already use past actions to justify given attributes, I propose cutting out the middleman and using previous actions to directly predict future ones. Rather than trusting Bob to protect a keepsake because he's reliable, trust him because he has always returned borrowed items quickly. Instead of calling a public figure a liar, show that past claims have been false 15% of the time, and make a prediction.

When using past actions to predict future ones, use actions that are as closely related as possible to what you want to predict. For example, the fact that Bob return books quickly doesn't necessarily mean he will come to your meeting on time, even though both would traditionally indicate reliability.

We can extend this beyond people to replace many more vague attributes with specific observations. One example is saying, "Mathematics involves extreme precision and abstraction" instead of "Mathematics is difficult." It's important to understand that attributes are nothing more than a shortcut: a more accurate option is always available. This is the same idea underpinning both E-prime and the writing rule show, don't tell.

Once thinking in this way becomes the default, mental impressions of both people and situations become more specific, more actionable, and simply more accurate.
Why those who care about catastrophic and existential risk should care about autonomous weapons 
by aaguirre — 2020-11-11T15:22:15.625Z

(crossposted to EA forum here.)

Although I have not seen the argument made in any detail or in writing, I and the Future of Life Institute (FLI) have gathered the strong impression that parts of the effective altruism ecosystem are skeptical of the importance of the issue of autonomous weapons systems. This post explains why we think those interested in avoiding catastrophic and existential risk, especially risk stemming from emerging technologies, may want to have this issue higher on their list of concerns.

We will first define some terminology and do some disambiguation, as there are many classes of autonomous weapons that are often conflated; all classes have some issues of concern, but some are much more problematic than others. We then detail three basic motivations for research, advocacy, coordination, and policymaking around the issue:

Governance of autonomous weapon systems is a dry-run, and precedent, for governance of AGI. In the short term, AI-enabled weapons systems will share many of the technical weaknesses and shortcomings of other AI systems, but like general AI also raise safety concerns that are likely to increase rather than decrease with capability advances. The stakes are intrinsically high (literally life-or-death), and the context is an inevitably adversarial one involving states and major corporations. The sort of global coordination amongst potentially adversarial parties that will be required for governance of transformative/general AI systems will not arise from nowhere, and autonomous weapons offer an invaluable precedent and arena in which to build experience, capability, and best practices.

Some classes of lethal autonomous weapon systems constitute scalable weapons of mass destruction (which may also have a much lower threshold for first use or accidental escalation), and hence a nascent catastrophic risk.

By increasing the probability of the initiation and/or escalation of armed conflict, including catastrophic global armed conflict and/or nuclear war, autonomous weapons represent a very high expected cost that overwhelmingly offsets any gain in life from substituting autonomous weapons for humans in armed conflict.

Classes of autonomous weapons

Because many things with very different characteristics could fall under the rubric of “autonomous weapon systems” (AWSs) it is worth distinguishing and classifying them. First, l...
The (Unofficial) Less Wrong Comment Challenge
by adamShimi — 2020-11-11T14:18:48.340Z

Feedback is important. It's true in Machine Learning, and it's true in writing. So when you write a cool new post on LW, you generally hope for feedback. Positive feedback, fingers crossed, but good negative feedback is also very nice for improving your writing chops and your ideas. Yet getting meaningful feedback can prove difficult on LW. This Challenge aims at starting a community trend to solve this problem.

First, I need to frame the issue. How do you get feedback on LW? Through two mechanisms: karma and comments.

High karma (let's say above 50) show that people find your post valuable. Maybe they even find your post clear and well-written; they might even agree with you. It would be cool to have comments, but you still have valuable feedback without any.
Low karma (The "Low Karma" filter is for less than -10, but I'll count any negative karma) show that people actively disagree with either your idea or your way of presenting it. That's a useful signal, but one that really warrants comment to understand what people disliked enough to downvote you.
Comments (let's say at least 3 comments/comment threads from 3 different authors, all concerning the actual subject of the post instead of some other thing mentioned in the post), regardless of the karma, show an involvement of readers. Once again, they can be positive or negative; what really matters is that they interact with the subject of the post.

Okay. But what happens when none of these criteria is met? Posts with karma between 0 and 49 and less than 3 comments are not that rare: looking at last week's frontpage posts (from Monday to Sunday my time, CET), there are 18 such posts out of 47, which is about 38%. Not a majority, but still more than a third. And I'm pretty sure many of the posts that didn't make the cut could benefit from more feedback.

This is why I'm proposing the (Unofficial) Less Wrong Comment Challenge: committing to read and comment all posts in a given category, or by a given author, or with a given tag, in a specified period. The simplest example would be something like: read and comment every new frontpage post on LW for the rest of November. But the parameters allow for tweaks. For example, maybe you're free this week, but not after. And maybe you don't feel like you have the level for reading Alignment Forum posts. In that context, you could challenge yourself to read and comment every ...
What could one do with truly unlimited computational power?
by Yitz — 2020-11-11T10:03:03.891Z

One day, you find an unmarked cardboard box sitting on your front porch.

On top of the box is a single note:

“Open me.”

You take the box inside and open it, to find what appears to be a small black laptop nestled between old newspaper clippings. There are no identifying features to the laptop, other than its sleek blackness and small size. When you open the laptop, instead of being greeted with any sort of familiar welcome screen, there is simply text, displayed in white over a black background. The text says the following:

“Welcome to the next stage of the human experiment.

I have have been watching your kind for a while now, and I believe that you are now ready. I have decided to entrust this machine to your care. Do not attempt to figure out how this machine performs its calculations, as uninformed tampering may result in the deletion of your universe.

The item you are now holding is what your mathematicians would call a Universal Turing Machine, similar in many ways to your typical computer. This machine, however, is significantly different from any other currently existent on Earth.

After reading this message, press any button to reveal two input fields stacked on top of each other, and one output field at the bottom. All input fields can be fed any required data from the internet, or can be entered into directly through the keyboard. Input lengths of any finite size are acceptable.

The top input field will accept and is capable of automatically running the intended code of any language or format that is capable of being run on a standard Universal Turing Machine.

That input will then be translated into the necessary binary code to be computable by the internal “black box” computer.

The bottom input field must be fed a finite natural number of any length, which will determine the number of computations per second. Do not worry about exceeding the speed of light, or of going past any other finite limit; the computation itself is performed in a “bubble universe” with different physical laws than your own, and is capable of computing at absolutely any positive finite speed relative to your universe.

The output field will display the output of your calculations, if any exist, after exactly one minute of computation relative to you.

Do with this machine as you will.

Wishing you the best,

God.”

Your finger hovers over the keyboard.

What will you do with this marv...