trentmkelly/LessWrong-43k huggingface dataset:
The RAIN Framework for Informational Effectiveness
by ozziegooen — 2019-02-13T12:54:20.297Z

[Epistemic status: Medium-Uncertainty. I've only spent a few days thinking about this, but it seems to fit well for some specific problem so far.]

The following describes one possible framework for understanding the usefulness of different sources of information. It's particularly meant to help value source types such as books, academic articles, blog posts, online comments, and mathematical models. I think it could be a useful starting point but would guess that there are better alternatives upon further deliberation.

The framework factors are robustness, importance, novelty, and accessibility.

Simple Use Cases

There are multiple things this could be useful for, I'm sure most of which I haven't yet considered. For a start, I would hope that it could be used when discussing options on either writing information for others or deciding what materials to encourage.

Some possible discussion quotes relating to this framework

"These blog posts are quite novel, but I think they aren't very robust."

"This video may not be very dense, but it is highly accessible."

"This paper has a lot of equations but they don't seem useful to the point. It's both inaccessible and not robust."

"I think that you can change your paper to make it more accessible without sacrificing any robustness."

Informational Effectiveness

People use informational resources (books, videos, etc), in part, to learn information. There are many important attributes of such resources that will impact the quality and magnitude of such learning. Here I wrap these in the total term "information effectiveness."

Information effectiveness, when most narrowly estimated, is context-specific to an agent or group of readers or writers. It is specific to a set of topics; for instance, a particular article by George about politics may be considered ineffective on the topic of politics, but highly effective in it's revealed information about George's beliefs.

Informational effectiveness could be judged for any quantity of information; an entire book, a "per-page average", a "per-bit average", or similar.

In this document, we focus on "reader information effectiveness", which seeks to understand the effectiveness of information to readers. Similar frameworks could be made for writers; for instance, they may have goals such as persuading readers of specific claims or generating status.

To give a simple example, if you were to

...
Layers of Expertise and the Curse of Curiosity
by Gyrodiot — 2019-02-12T23:41:45.980Z

Epistemic status: oversimplification of a process I'm confident about; meant as proof of concept.

Related to: Double-Dipping in Dunning-Kruger

Expertise comes in different, mostly independent layers. To illustrate them, I will describe the rough process of a curious mind discovering a field of study.

Discovery

In the beginning, the Rookie knows nothing. They have no way to tell what's true or false in the field. Anything they say about it will probably be nonsense, or at best, not better than chance.

Consider a child discovering astronomy. They know the Sun and the Moon move in the sky, that other planets and stars exist, but they wonder about the mysterious domain of space. They open a book, or watch a few videos, and their first discoveries are illuminating. The Moon goes around the Earth which goes around the Sun, the other stars are very very far. Everything makes sense, because beginner material is designed to make sense.

The basic facts are overwhelming. They feel so valuable and wondrous that they have to be shared with other children. They know nothing! The knowledge gap is so large that the enlightened child is viewed as an Expert, and for a while the little explorer does feel like one.

However, the child is still a Rookie. They start talking about how planets go in perfect circles around the Sun, that there's nothing but interstellar space beyond Pluto except maybe comets, because introductory material is fuzzy on the details. The child may be overconfident, until someone more educated points out the mistake. Then Curiosity kicks in.

Learning iteration

When discovering gaps in their knowledge, one with a curious mind will strive to fill them. They will seek new material, kind teachers, and if they're lucky they'll learn more and more. This is the first layer of expertise: accumulation of true facts. Repeatedly, they will be confronted with their own ignorance: for each new shard of knowledge they reveal, dozens appear still shrouded. Every time they think they've exhausted the field, an unexpected complexity will show them wrong.

At some point they will internalize the pattern: the field is deep, and full of more details than they can learn in a lifetime. They will be cautious about their learning process, acknowledging they may be wrong, that their models of reality aren't perfect, that they don't know all there is to...
Nuances with ascription universality
by evhub — 2019-02-12T23:38:24.731Z

Part of this post is based on points made by Scott Garrabrant and Abram Demski at a recent AI safety workshop put on by Alex Zhu.

Paul Christiano recently published a sequence of posts on a concept he calls "ascription universality." I highly recommend reading the full sequence of posts, though I will do my best to summarize the basic concept here. As I see it, ascription universality can be seen as a formalization of transparency/honesty: a system is ascription universal if, relative to our current epistemic state, its explicit beliefs contain just as much information as any other way of ascribing beliefs to it.

Of particular interest to this post is the last part—what does "any other way of ascribing beliefs" mean? Paul points this out as one of the most informal steps in his definition of ascription universality. In this post, I want to try and point out one way in which I think this process could go wrong.

First, one of the most obvious ways in which a system could fail to be ascription universal is if it is doing some sort of internal cognition which is at odds with its stated beliefs—if an AI system is attempting to deceive its programmers, for example. Such a system would fail to be ascription universal because there would be a way of ascribing beliefs to the internal cognition it is hiding from its programmers that would give the programmers information they currently don't have and can't get purely from the system's explicit output.

However, consider the following alternative system: take the system from the previous example and "memoize" it, replacing it with a simple lookup table that always outputs what the system from the previous example would have output. Is this system ascription universal? From a moral standpoint, we certainly don't want it to be, since it's still taking the same deceptive actions. But if your concept of "ascribing beliefs" only considers ways of looking at the system's internal cognition, you might think that this system is ascription universal and thus erroneously conclude that it is safe.

Is this likely to ever be an actual problem for a real-world AI system? I argue yes. Certainly, we don't expect our AI systems to be massive lookup tables. However, there is a sense in which, if we are only looking at an agent's internal cognition to determine whe...
Learning preferences by looking at the world
by Rohin Shah — 2019-02-12T22:25:16.905Z
Link: https://bair.berkeley.edu/blog/2019/02/11/learning_preferences/

We’ve written up a

blog post

about our recent

paper

that I’ve been linking to but haven’t really announced or explained. The key idea is that since we’ve optimized the world towards our preferences, we can infer these preferences just from the state of the world. We present an algorithm called Reward Learning by Simulating the Past (RLSP) that can do this in simple environments, but my primary goal is simply to show that there is a lot to be gained by inferring preferences from the world state.

The rest of this post assumes that you’ve read at least the non-technical part of the linked

blog post

. This post is entirely my own and may not reflect the views of my coauthors.

Other sources of intuition

The story in the blog post is that when you look at the state of the world, you can figure out what humans have put effort into, and thus what they care about. There are other intuition pumps that you can use as well:

The world state is “surprisingly” ordered and low-entropy. Anywhere you see such order, you can bet that a human was responsible for it, and that the human cared about it.
If you look across the world, you’ll see many patterns recurring again and again -- vases are usually intact, glasses are usually upright, and laptops are usually on desks. Patterns that wouldn’t have happened without humans are likely something humans care about.

How can a single state do so much?

You might be wondering how a single state could possibly contain so much information. And you would be correct to wonder that. This method depends very crucially on the assumption of known dynamics (i.e. a model of “how the world works”) and a good featurization.

Known dynamics. This is what allows you to simulate the past, and figure out what “must have happened”. Using the dynamics, the robot can figure out that breaking a vase is irreversible, and that Alice must have taken special care to avoid doing so. This is also what allows us to distinguish between effects caused by humans (which we care about) and effects caused by the environment (which we don’t care about).

If you take away the knowledge of dynamics, much of the oomph of this method is gone. You could still look for and preserve repetitions in the state -- maybe there are a lot of intact vases and no broken vases, so you try to keep vases intact. But this might also lead you to making sure that nobody puts warning signs near cliffs, s...
Functional silence: communication that minimizes change of receiver's beliefs
by chaosmage — 2019-02-12T21:32:27.015Z

I just thought of this concept and I'd like to know if it is new or useful.

"Functional silence" is communication intended to carry as little information as possible. It intends for the beliefs of listeners to be changed as little as possible.

This is usually the same as regular silence. The difference is in cases where regular silence would convey the wrong message, such as when asked a question that one cannot politely refuse to answer, when unwantedly prompted to give a statement or when padding a short message because more words are expected. In these cases, a range of techniques is commonly used:

Boilerplate statements. This is prepared words that have little or no relation to the subject at hand but are unobjectionable and usually express or imply an expectation of mutual respect or a mission statement.
Distraction / subject change. This will usually be towards a superficially related subject. For example, many politicians when asked to give a prediction will instead talk about what they want to happen. Failing that, humor is powerful here. A switch to a shiny subject or an applause light works much better than a switch to a boring subject.
Incomprehensibility. Use of lots of complicated words and allusions in quick succession in order to overload the listener's comprehension. Delivered with a friendly-seeming air of expectation that the listener obviously gets all of this, so the listener cannot ask without losing face. A classic salesman's technique.
Repetition. Saying the same thing again in different words. This is most vacuous if it repeats something already said in the same conversation, but anything the listener already knows the speaker has said will do.
Tautology. Statements that boil down to "if A is the case, then A is the case" without any explicit statement on whether A is the case. Sounds dumb when boiled down like this, but this is shockingly effective at convincing people that actual information has been shared.
The median response. Expresses the most average (hence predictable, boring) thing that average people say in this situation. This does transport a bit of information, but will probably at least not be remembered.

These seem different and there are surely others I've forgotten, but I'd like to call all of them "functional silence" in order to emphasize how they all do the same thing. ...
Arguments for moral indefinability
by Richard_Ngo — 2019-02-12T10:40:01.226Z

Epistemic status: I endorse the core intuitions behind this post, but am only moderately confident in the specific claims made.

Moral indefinability is the term I use for the idea that there is no ethical theory which provides acceptable solutions to all moral dilemmas, and which also has the theoretical virtues (such as simplicity, precision and non-arbitrariness) that we currently desire. I think this is an important and true perspective on ethics, and in this post will explain why I hold it, with the caveat that I'm focusing more on airing these ideas than constructing a watertight argument.

Here’s another way of explaining moral indefinability: let’s think of ethical theories as procedures which, in response to a moral claim, either endorse it, reject it, or do neither. Moral philosophy is an attempt to find the theory whose answers best match our intuitions about what answers ethical theories should give us (e.g. don’t cause unnecessary suffering), and whose procedure for generating answers best matches our meta-level intuitions about what ethical theories should look like (e.g. they should consistently apply impartial principles rather than using ad-hoc, selfish or random criteria). None of these desiderata are fixed in stone, though - in particular, we sometimes change our intuitions when it’s clear that the only theories which match those intuitions violate our meta-level intuitions. My claim is that eventually we will also need to change our meta-level intuitions in important ways, because it will become clear that the only theories which match them violate key object-level intuitions. In particular, this might lead us to accept theories which occasionally evince properties such as:

Incompleteness: for some claim A, the theory neither endorses nor rejects either A or ~A, even though we believe that the choice between A and ~A is morally important.
Vagueness: the theory endorses an imprecise claim A, but rejects every way of making it precise.
Contradiction: the theory endorses both A and ~A (note that this is a somewhat provocative way of framing this property, since we can always add arbitrary ad-hoc exceptions to remove the contradictions. So perhaps a better term is arbitrariness of scope: when we have both a strong argument for A and a strong argument for ~A, the theory can specify in which situations each conclusion should apply, based on crite

...
Art: A Rationalist's Take?
by schrodingart — 2019-02-12T05:07:41.856Z

Hello LWers,

I am in my 20s, and I like to collect art.

Admittedly, I am a lurker on the LW forums and can proudly say I have been for a very long time now. I've refrained from commenting simply because I have not felt that I have all that much to contribute, but rather, find that I have much to learn still.

In my time lurking, I have come across so many spectacular posts, which in turn, have expanded my way of interpreting real-world scenarios. Realising there exists a general proclivity for thoughtful discussion in this lesser known corner of the internet, I've only remained hesistant to post this because of the potential to dampen things around here with discussions about personal matters.

It is my intention to take advantage of the many bright minds attracted to this forum that do actually take an interest in fine art, then perhaps, better understand how a team of rationalists would see through my goals and execute a plan of action, if in my shoes, eager to succeed.

To be honest, I don't exactly know what brought about my passion for collecting art. Growing up I was always interested in collecting things like coins, rocks, negatives, etc. I always dreamt of having a collection of something for others to appreciate and learn more about.

Today, I find this ambition takes the form of a bias more than I'd like for it to; this is due to my involvement in art-centric projects that would stand to benefit me. A COI is what I see it all as.

The concept of arbitrage in the art world is nothing new. However, the idea that we can use modern technologies and mainstream platforms to find, interpret, attribute, even invest in art is life-changing. The possibility for us to be our own advisors, and for scientists to step-in where specialists and connosseiurs were once highly-valued makes the 'art specialist/committee' less useful in the field of art authentication.

It was rarely [if ever] the likely case that you could purchase a painting by Monet, Cézanne, or Delacroix for 1/1,000,000th of it's value 150 years ago. The actual value of their work then in comparison to their value today was much different of course.

150 years later however, and one can argue such a possibility (under the unlikeliest of circumstances) is quite possible. But, how does one find these rare opportunities? Even more crucial to understand, can this be done wit...
Language, the Key to Everything
by chris82179 — 2019-02-12T05:06:03.763Z

The one thing that gets in the way time and time again of productive discussions in philosophy is misunderstanding. A lot of the misunderstanding that I see is due to different levels of understanding different word meanings. Language can be very confusing with the same words having different meaning, particularly when people don’t utilize them in a way that shows understanding that there are different meanings. So while someone may use a word in a proper sense, they may not necessarily understand that there are different senses of that same word that can have an entirely different meaning and both might be logically correct in the context of the discussion. When that occurs, it is imperative the author differentiate to avoid confusion. On the other hand, there are instances of a word being conveyed that really only has one sense that it must have logically been referred to, otherwise in context the author doesn’t make sense. This burden, rests upon the reader in order to understand. While it can be nice for the author to mention the definition, that can get tedious. This can lead us to a slippery slope of defining every word used.

Philosophy requires in depth thinking and master of language in order to comprehend it due to the nature of philosophy that by nature, attempts to have us arrive to the deepest level of understanding that we can. Those who know the senses of words and have mastered the senses of words will understand that there are rules to play by, rules to communicate effectively and rules to understand information to communicate effectively. I have presented two of those reasons above, I hope the readers take the time to take that to heart so that they do understand. However taking it to heart is not merely enough. Understanding language is a matter of intelligence – in that it can be very difficult if you do not process information quickly. There are over a million words in the English language. Most adults use 20,000 – 35,000 words. Each one of those words typically has multiple senses in of itself. Remember, a word is just a symbol for the larger meaning of it, which definitions only hope to convey the meaning accurately. It can be very difficult to communicate effectively but in Philosophy is extremely important.

Meaning comes from within the conveyors mind, it is a construct of a person’s understanding of not only the concept a word is ref...
Would I think for ten thousand years?
by Stuart_Armstrong — 2019-02-11T19:37:53.591Z

Some AI safety ideas delegate key decisions to our idealised selves. This is sometimes phrased as "allowing versions of yourself to think for ten thousand years", or similar sentiments.

Occasionally, when I've objected to these ideas, it's been pointed out that any attempt to construct a safe AI design would involve a lot of thinking, so therefore there can't be anything wrong with delegating this thinking to an algorithm or an algorithmic version of myself.

But there is a tension between "more thinking" in the sense of "solve specific problems" and in the sense of "change your own values".

An unrestricted "do whatever a copy of Stuart Armstrong would have done after he thought about morality for ten thousand years" seems to positively beg for value drift (worsened by the difficulty in defining what we mean by "a copy of Stuart Armstrong [...] thought [...] for ten thousand years").

A more narrow "have ten copies of Stuart think about these ten theorems for a subjective week each and give me a proof or counter-example" seems much safer.

In between those two extremes, how do we assess the degree of value drift and its potential importance to the question being asked? Ideally, we'd have a theory of human values to help distinguish the cases. Even without that, we can use some common sense on issues like length of thought, nature of problem, bandwidth of output, and so on.
"Normative assumptions" need not be complex
by Stuart_Armstrong — 2019-02-11T19:03:38.493Z

I've shown that, even with simplicity priors, we can't figure out the preferences or rationality of a potentially irrational agent (such as a human H).

But we can get around that issue with 'normative assumptions'. These can allow us to zero in on a 'reasonable' reward function RH.

We should however note that:

Even if RH is highly complex, a normative assumption need not be complex to single it out.

This post gives an example of that for general agents, and discusses how a similar idea might apply to the human situation.

Formalism

An agent takes actions (A) and gets observations (O), and together these form histories, with H the set of histories (I won't present all the details of the formalism here). The policies Π={π:H→A} are maps from histories to actions. The reward functions R={R:H→R} are maps from histories H to real numbers), and the planners P={p:R→Π} are maps from reward functions to policies.

By observing an agent, we can deduce (part of) their policy π. Then a reward-planner pair (p,R) is compatible with π if p(R)=π. Further observations cannot distinguish between different compatible pairs.

Then a normative assumption α is something that distinguishes between compatible pairs. It could be a prior on P×R, or an assumption of full rationality (which removes all-but-the-rational planner from P), or something that takes in more details about the agent or the situation.

Assumptions that use a lot of information

Assume that the agent's algorithm π is written in some code, as Cπ, and that α will have access to this. Then suppose that α scans Cπ, looking for the following: an object CR that takes a history as an input and has a real number as an output, an object Cp that takes CR and a history as inputs, and outputs an action, and a guarantee that Cπ chooses actions by running Cp on CR and the input history.

The α need not be very complex to do that job. Because of rice's theorem and obfuscated code, it will be impossible for α to check those facts in general. But, for many examples of Cπ, it will be able to check that those things hold. In that case, let α return R; otherwise, let it return the trivial 0 reward.

So, for a large set S of possible algorithms, α can return a reasonable reward function estimate. Even if the complexity of Cπ and R is much, much higher than the complexity of α itself, there are still examples of these where α can successfully id

...
Meetup schedule
by Bae's Theorem — 2019-02-11T18:47:07.641Z

Our SSC meetup is every Sunday, 12:30pm, at Panera Bread for lunch/coffee.
10606 Shawnee Mission Pkwy

We also have a monthly game at Pawn and Pint (a board game pub, anyone younger than 21 must be accompanied by a legal guardian. Cover fee is waived if you say you're with the Kansas City Rationalists).
First Wednesday of each month, 6pm, 613 Walnut St, Kansas City, MO 64106

We also just began a Rationality Dojo, following the 'Hammertime' sequence.
Every Tuesday, 6pm, Central Resource Library.