trentmkelly/LessWrong-43k huggingface dataset:
Consistent Glomarization should be feasible
by Stuart_Armstrong — 2020-05-04T10:06:55.928Z

Glomarization is responding that one "can neither confirm nor deny" something. It has to be done consistently, to avoid problems like:

LAWYER:Did you ever sleep with him in New York?WITNESS:I refuse to answer that question.LAWYER:Did you ever sleep with him in Chicago?WITNESS:I refuse to answer that question.LAWYER:Did you ever sleep with him in Miami?WITNESS:No

There's been a lot of talk on this list about honesty and meta-honesty. Glomarization is a minor part of this, I know, but there is a claim that:

Genuinely consistent Glomarization (i.e., consistently saying "I cannot confirm or deny" whether or not there's anything to conceal) does not work in principle because there are too many counterfactual selves who might want to conceal something.

Consistently Glomarizing with "OR"

The problem with Glomarizing in everyday life is:

FRIEND:Did you get to sleep early last night?ME:I can neither confirm nor deny...

So either you Glomarize very rarely (thus making it obvious when you've been up to something questionable), or you do it all the time (making you a terrible friend who refuses to share basic details of their life in a simple conversation).

A somewhat better answer would be:

FRIEND:Did you get to sleep early last night?ME:Either I did sleep early, or I don't want to tell you about what I did.

Now, that's still socially awkward in many situations, but you can at least do it consistently to every query, while sharing most of your life in a conversational way. Note that this is superior to "or I did something I don't want to tell you about", because if you said that, then they merely needed to find out that you hadn't slept early to conclude you were up to something dubious.

With close friends or rationalist groups, you might agree in advance that there's a "or I don't want to tell you about what I did" attached to every statement about your life, or have a short abbreviation equivalent to that.

Other alternatives could be:

FRIEND:Did you get to sleep early last night?ME:I rolled a D100 this morning; if it came up a 1, or if I did somethingI want to conceal, I may lie to questions about yesterday.My answer is: I did sleep early, thanks for asking.

Not the easiest and most conventional way of phrasing things, but in terms of socially sharing harmless information, it gets the job done (and is probably more honest than most people's answers to these questions).
Scott Garrabrant's problem on recovering Brouwer as a corollary of Lawvere
by Rupert — 2020-05-04T10:01:56.305Z

Edit: Pre-print available here https://arxiv.org/abs/2005.01563

Also link to earlier post here https://www.alignmentforum.org/posts/5bd75cc58225bf06703753a9/formal-open-problem-in-decision-theory?fbclid=IwAR2u-8RVfnfRtEo51vag5vYWAcfkoHDJAr6aydVGqYVmEq0oxnjwR0Ee34E

Scott Garrabrant has raised the question of whether there is any topological space A for which the exponential topology on [0,1]A exists and such that there is a continuous surjection g:A→[0,1]A. In cited pre-print above I claim to resolve this question negatively in ZFC (using techniques of inner model theory, Skolem hull arguments, and some Reverse Mathematics results presented by Stephen Simpson in "Subsystems of Second Order Arithmetic") but also to give an example of a pair of spaces A′,A′′ with the same carrier set, topology on A′ finer than that on A′′, and a continuous surjection g:A′→[0,1]A′′ with the following properties. Take an open subset of [0,1], take pre-image in [0,1]A′′×A′′ under evaluation map, take projection onto first factor and then pre-image under surjection g, and view this both as a subset of A′ and A′′ (which have the same carrier set), this is required to be always open relative to both topologies. If this can be achieved then Brouwer fixed point theorem is recoverable as a corollary with "same method of proof" as Lawvere. Results along these lines appear to be essentially "the best one can do".

My first example of such a pair of spaces are spaces of cardinality 2ℵ1 which may lead one to think that practical applications are doubtful. But there are techniques to produce a broad class of related examples by refining things both along the hierarchy of descriptive complexity and indeed even computational complexity. The details of this are outlined in the final section of the paper.

One example of a result that can be obtained is as follows. Let α be the least countable ordinal such that there is no polynomial-time computable recursive well-ordering of length α. (I think it's probably ω2 or something, not sure, but this could be figured out. EDIT: Alex Mennen has clarified I was mistaken about this and the ordinal is in fact ωCK1, which perhaps makes this particular result of le...
Named Distributions as Artifacts
by George3d6 — 2020-05-04T08:54:13.616Z
Link: https://blog.cerebralab.com/Named_Distributions_as_Artifacts

On confusing priors with models

Being Abraham de Moivre and being born in the 17th century must have been a really sad state of affairs. For one, you have the bubonic plague thing going on, but even worse for de Moivre, you don't have computers and sensors for automated data collection.

As someone interested in complex-real world processes in the 17th century, you must collect all of your observations manually, or waste your tiny fortune on paying unreliable workers to collect it for you. This is made even more complicated by the whole "dangerous to go outside because of the bubonic plague" thing.

As some interested in chance and randomness in the 17th century, you've got very few models of "random" to go by. Sampling a uniform random variable is done by literally rolling dice or tossing a coin. Dice which are imperfectly crafted, as to not be quite uniform. A die (or coin) which limits the possibilities of your random variable between 2 and 20 (most likely 12, but I'm not discounting the existence of medieval d20s). You can order custom dice, you can roll multiple dice and combine the results into a number (e.g. 4d10 are enough to sample uniformly from between 1 and 10,000), but it becomes increasingly tedious to generate larger numbers.

Even more horribly, once you tediously gather all this data, your analysis of it is limited to pen and paper, you are limited to 2 plotting dimensions, maybe 3 if you are a particularly good artist. Every time you want to visualize 100 samples a different way you have to carefully draw a coordinate system and tediously draw 100 points. Every time you want to check if an equation fits your 100 samples, you must calculate the equation 100 times... and then you must compute the difference between the equation results and your samples, and then manually use those to compute your error function.

I invite you to pick two random dice and throw them until you have a vague experimental proof that the CLT holds, and then think about the fact that this must have been de Moivrea's day to day life studying proto-probability.

So it's by no accident, misfortune, or ill-thinking that de Moivre came up with the normal distribution [trademarked by Gauss] and the central limit theorem (CLT).

A primitive precursor of the CLT is the idea that when two or more independent random va...
How uniform is the neocortex?
by zhukeepa — 2020-05-04T02:16:50.650Z

How uniform is the neocortex?

The neocortex is the part of the human brain responsible for higher-order functions like sensory perception, cognition, and language, and has been hypothesized to be uniformly composed of general-purpose data-processing modules. What does the currently available evidence suggest about this hypothesis?

"How uniform is the neocortex?” is one of the background variables in my framework for AGI timelines. My aim for this post is not to present a complete argument for some view on this variable, so much as it is to:

present some considerations I’ve encountered that shed light on this variable

invite a collaborative effort among readers to shed further light on this variable (e.g. by leaving comments about considerations I haven’t included, or pointing out mistakes in my analyses)

There’s a long list of different regions in the neocortex, each of which appears to be responsible for something totally different. One interpretation is that these cortical regions are doing fundamentally different things, and that we acquired the capacities to do all these different things over hundreds of millions of years of evolution.

A radically different perspective, first put forth by Vernon Mountcastle in 1978, hypothesizes that the neocortex is implementing a single general-purpose data processing algorithm all throughout. From the popular neuroscience book On Intelligence, by Jeff Hawkins[1]:

[...] Mountcastle points out that the neocortex is remarkably uniform in appearance and structure. The regions of cortex that handle auditory input look like the regions that handle touch, which look like the regions that control muscles, which look like Broca's language area, which look like practically every other region of the cortex. Mountcastle suggests that since these regions all look the same, perhaps they are actually performing the same basic operation! He proposes that the cortex uses the same computational tool to accomplish everything it does.

[...]

Mountcastle [...] shows that despite the differences, the neocortex is remarkably uniform. The same layers, cell types, and connections exist throughout. [...] The differences are often so subtle that trained anatomists can't agree on them. Therefore, Mountcastle argues, all regions of the cortex are performing the same operation. The thing that makes the vision area visual and the motor area mo

...
Hate Crime Statistics
by jefftk — 2020-05-03T22:30:03.706Z

It's generally hard to tell whether long-term problems are getting
better or worse from looking at news coverage or hearing about
problems. Each gives you some information, but it's hard to tell how
it compares to what was happening before. Quantitative data can help
give some perspective, however, and I decided to look some into US

hate crime
statistics.

Since 1992 local police have reported crime information to the FBI
under the Uniform
Crime Reports program, and then the FBI collates them into an
annual report. There's some risk of underreporting, or changing
reporting levels over time, but this seemed really hard to look into
and I decided to just using the numbers as presented in the reports.
I couldn't find an annual table, so I went through the 27 published
reports and pulled out the number of incidents from each report's
Table 1 (raw
data).

There were ~30% more people in the US in 2018, though, so we should
look at per-capita numbers instead:

The report breaks down crimes by the motivating bias. The five most
common, anti-Black (34%), anti-Jewish (13%), anti-White (12%),
anti-Gay (10%), and anti-Hispanic (6%) account for
75% of the total incidents:

The spike in 2001 is coded as "Anti-Other Ethnicity/National Origin"
but it's probably anti-Arab with 9/11.

Instead of normalizing by the total population, however, for specific
biases it would be clearer to normalize by the population of the
affected group. For example, how many anti-Hispanic hate crimes are
there compared to the number of Hispanic Americans? Here's what that
looks like for the same five most common categories:

This is very sensitive to the size of the target group, however.
I've taken the number of Black and Hispanic Americans from the Census
(pretty good), the number of Jewish Americans from the Statistical
Abstract of the US (ok), and the number of Gay Americans from the General
Social Survey (not so good, and other sources can give pretty
different numbers).

Statistics are fundamentally a summary, and they can't be our only
window to the world. But for questions like, "is this better or worse
than it was ten years ago" they're a very powerful tool.

Comment via: facebook
Biological evolution as an analogy for the market economy
by Snorkelfarsan — 2020-05-03T18:51:00.636Z

Both evolution and the free market are powerful forces in our universe. They can create both immense suffering and unimaginable success. We make moralistic judgments about the outcomes, but they are amoral.

By approaching economic policy in much the same way we approach biology I believe we can begin to resolve a lot of our ideological grievances.

Evolution

Evolution through natural selection is the means by which simpler organisms evolve into more complex ones (generally over long periods of time) through many, small incremental changes (mutations).

Although the mutations are random, the selective pressure put on any given organism by its environment allows the organisms “lucky” enough to have gotten beneficial mutations to copy these on to the next generation. Since these mutations confer some sort of adaptive advantage over other organisms without that mutation the “good” genes will, on average, get passed on to successive generations where new mutations may arise, repeating the loop.

One organism is not very different from its immediate ancestor. But over many generations, big differences emerge and the complexity will often (but not always) increase.

Misconceptions about evolution

There are many common misconceptions and myths about evolution. Here are three that are relevant to this article:

First of all, the term “survival of the fittest” as understood by most people is a little misleading. It invariably makes us think of fitness as physical prowess. But it has more to do with the relative fitness of an organism vis-à-vis its environment. How well the organism is able to adapt to external selective pressures (such as from other organisms).

An organism can have fitness both in the immediate sense of being able to outrun a predator — or indeed being fast enough to catch prey. And on evolutionary time scales as in a species having the fitness to adapt to a changing climate, and migrating to a new habitat.

Second, a common misinterpretation of evolution is that it represents a progression from simple to advanced or worse to better. But the results of the current evolution of say, Homo Sapiens, is not the end-goal. On the contrary, it is a non-linear never-ending process of changes. One state of that process is not necessarily better or “more evolved” than a previous state. It is just ...
On Negative Feedback and Simulacra
by Zvi — 2020-05-03T17:00:00.558Z

Response to (Elizabeth @ LessWrong): Negative Feedback and Simulacra

Requires/Assumes (Compass Rose): Simulacra and Subjectivity

Epistemic Status: Exploring, thinking out loud, taking a break from Covid-19 stuff, etc. Long post is long because I didn’t have the time or insight to make it shorter. Later I hope to write shorter versions.

Simulacrum levels are very important. If you haven’t read the Compass Rose post above, please do so, even if you don’t read the rest of this post. At a minimum, read part 2 of Elizabeth’s post above, which attempts to summarize the central point.

Elizabeth’s post provides good relatively clean examples of a common problem related to simulacrum levels.

The common problem – in fact, the most common problem – is that there is a desirable action X that you wish to take, or an undesirable action Y that you wish to avoid. Unfortunately, taking action X, or avoiding action Y, has undesirable (at least to someone, in some sense) consequence Z.

So far, that is every non-trivial decision ever made. The distinction here is that consequence Z takes the form of conveying true information. Where that true information is, with respect to some aspect or attribute of some person or group, a lower than expected opinion. And that’s terrible.

Usually, sharing true information is the opposite of terrible. By default sharing true information (assuming it is new and useful/interesting, or it builds useful common knowledge) is good.

What this is not, is a law of nature that says that sharing true and relevant information is automatically and everywhere always good, or is always good for good people or justice. It sometimes isn’t.

I’ve made clear in the past, most notably in Blackmail and Privacy, that I strongly believe that the sharing of the wrong information, at the wrong time, to the wrong person, is in fact sometimes be terrible. Even if everyone is (relatively) well-meaning, sometimes terrible results follow from the sharing of true and useful information. And if someone goes looking for damaging information, and goes looking for damaging formats and times and places to share that information, that you should expect such information to do harm.

It’s worth emphasizing how weak this claim is. All it’s saying is, that sharing information has consequences, sometimes some of those consequences are bad, and sometimes the net consequences w

...
What is a “Good” Prediction?
by  — 2020-05-03T16:51:46.802Z

[My inside view is pretty confident in this; my outside view is very not. Cross-posted from Grand, Unified, Crazy.]

Zvi’s post on Evaluating Predictions in Hindsight is a great walk through some practical, concrete methods of evaluating predictions. This post aims to be a somewhat more theoretical/philosophical take on the related idea of what makes a prediction “good”.

Intuitively, when we ask whether some past prediction was “good” or not, we tend to look at what actually happened. If I predicted that the sun will rise with very high probability, and the sun actually rose, that was a good prediction, right? There is an instrumental sense in which this is true, but also an epistemic sense in which it is not. If the sun was extremely unlikely to rise, then in a sense my prediction was wrong – I just got lucky instead. We can formally divide this distinction as follows:

Instrumentally, a prediction was good if believing it guided us to better behaviour. Usually this means it assigned a majority probability to the thing that actually happened regardless of how likely it really was.
Epistemically, a prediction was good only if it matched the underlying true probability of the event in question.

But what do we mean by “true probability”? If you believe the universe has fundamental randomness in it then this idea of “true probability” is probably pretty intuitive. There is some probability of an event happening baked into the underlying reality, and like any knowledge, our prediction is good if it matches that underlying reality. If this feels weird because you have a more deterministic bent, then I would remind you that every system seems random from the inside.

For a more concrete example, consider betting on a sports match between two teams. From a theoretical, instrumental perspective there is one optimal bet: 100% on the team that actually wins. But in reality, it is impossible to perfectly predict who will win; either that information literally doesn’t exist, or it exists in a way which we cannot access. So we have to treat reality itself as having a spread: there is some metaphysically real probability that team A will win, and some metaphysically real probability that team B will win. The bet with the best expected outcome is the one that matches those real probabilities.

While this def...
Book Review: Narconomics
by ketchupduck — 2020-05-03T01:19:05.750Z

I have often heard that the war on drugs is a failure, but the reason why varies.

Sometimes, the war on drugs is a failure like the war in Iraq is a failure: because it is very hard to nation-build, to create a society that does not produce or consume drugs.

Sometimes, the war on drugs is a failure like the war in Vietnam was a failure: because the opponents, drug cartels, don’t fight like we do.

Sometimes, the war on drugs is a failure like WWI was a failure: we can win the battles but we don’t know how to write a lasting peace treaty.

Tom Wainwright’s Narconomics ropes in elements of the above, but offers another overarching story. The war on cocaine is a failure like the war on alchemy would be a failure.

If you knew how to perform alchemy, to turn dirt from your backyard into gold with just an incantation, would you stop just because the government forbade using the incantation? Cocaine isn’t gold, but only because it's twice as valuable as gold in the US. $385 worth of coca plant leaves in Columbia are worth $122,000 once turned into a kilogram of cocaine and shipped to the US. That’s a 30,000% increase in value. Not exactly alchemy, but close.

To put it another way, if drug cartels lost a half-million dollar plane for each shipment of cocaine to the US, it would increase the final price of cocaine by one percent.

The economics of cocaine, like that of alchemy, are very lucrative. As long as that stands, drug cartels will do anything to keep growing and smuggling cocaine. The war on cocaine is bound to fail for as long as cocaine is in high demand.

Wainwright’s recommendation is simple. To drive profits from cocaine down and criminal entities out, we should legalize cocaine.

Burning coca leaves won’t win the war

Here is an Economics 101 problem: In the year 20XX, a mysterious almond blight kills half of all almond plants worldwide. What happens to the price of almonds?

Easy. Constant demand + restricted supply → increased price.

Now here’s a variation on the same problem. In the year datetime.now().year, the Columbian government, with encouragement from the US, lays waste to about half the coca crops in the country. What happens to the price of cocaine?

Constant demand + restricted supply → ¯\_(ツ)_/¯ nothing much, apparently.

The reason is that coca farmers operate ...
How does iterated amplification exceed human abilities?
by riceissa — 2020-05-02T23:44:31.036Z

When I first started learning about IDA, I thought that agents trained using IDA would be human-level after the first stage, i.e. that Distill(H) would be human-level. As I've written about before, Paul later clarified this, so my new understanding is that after the first stage, the distilled agent will be super-human in some respects and infra-human in others, but wouldn't be "basically human" in any sense.

But IDA is aiming to eventually be super-human in almost every way (because it's aiming to be competitive with unaligned AGI), so that raises some new questions:

If IDA isn't going to be human-level after the first stage, then at what stage does IDA become at-least-human-level in almost every way?

What exactly is the limitation that prevents the first stage of IDA from being human-level in almost every way?

When IDA eventually does become at-least-human-level in almost every way, how is the limitation from (2) avoided?

That brings me to Evans et al., which contains a description of IDA in section 0. The way IDA is set up in this paper leads me to believe that the answer to (2) above is that the human overseer cannot provide a sufficient number of demonstrations for the most difficult tasks. For example, maybe the human can provide enough demonstrations for the agent to learn to answer very simple questions (tasks in T0 in the paper) but it's too time-consuming for the human to answer enough complicated questions (say, in T100). My understanding is that IDA gets around this by having an amplified system that is itself automated (i.e. does not involve humans in a major way, so cannot be bottlenecked on the slowness of humans); this allows the amplified system to provide a sufficient number of demonstrations for the distillation step to work.

So in the above view, the answer to (2) is that the limitation is the number of demonstrations the human can provide, and the answer to (3) is that the human can seed the IDA process with sufficient demonstrations of easy tasks, after which the (automated) amplified system can provide sufficient demonstrations of the harder tasks. The answer to (1) is kind of vague: it's just the smallest n for which ⋃ni=0Ti contains almost all tasks a human can do.

But the above view seems to conflict with what's in the IDA post and the IDA paper. In both of those, the amplified system is described as a human doing the decompositions (

...