trentmkelly/LessWrong-43k huggingface dataset:
Wikipedia Edit War Update
by jefftk — 2020-05-22T11:30:02.800Z

A few months ago I

stumbled into an edit war on
Wikipedia. I noticed that Wikipedia's page on

Jacy Reese was
being, essentially, guarded from having any mention that he previously
went by his full name. There was a pattern where someone would notice
this information was missing, add it, and then it would be reverted
soon after.

The main user guarding the page was Bodole, and
someone pointed me yesterday to where they've been banned
from editing Jacy's page for three months. The discussion there
was another interesting window into how Wikipedia handles disputes, so
after reading it I thought it would be interesting to review:

User Drmies edited the page to remove a list of articles Jacy
had published ("rm linkfarm. we list books, not articles", link).
Drmies is an experienced editor, making a routine cleanup.

User Bodole reverts the change ("Many BLP list articles. Please
discuss on talk page if you think this should be an exception.", link).

Drmies reverts the revert ("It's the other way around. what you
are doing is promoting this person by linking a set of articles. if
you have secondary sources that prove these articles are worth
noticing, that's a different matter", link)

Bodole reverts the reversion of the revert ("You are edit
warring. Please stop. Discuss on the talk page if you insist. See the
WP:BRD
cycle", link)
and puts a warning (link)
on Drimes' talk page.

Drimes responds there with "Aw boohoo" (link)

Drmies reverts the reversion of the reversion of the revert
("see talk page", link)
and marks the page as being subject to Wikipedia:Conflict
of Interest (link).
It looks to me like Drmies thinks Bodole may either be Jacy or someone
closely connected with him. Drmies removes biographical information
from the page ("this 'Sentience Institute' has an article--why this
biography is bloated with content about some poll, verified only with
links to websites, is not clear", link)

Discussion moves to the talk page

Drmies is clearly quite unhappy with apparent promotional
editing ("we are not here to produce link dumps for resume-style lists
of publications", "The article itself is way too fluffy anyway; it
used to be a lot worse, thanks in part to edits like this one by the
creator, Utsill, and this one, by Reckston. A quick look at the
references show a plethora of primary links and references to
non-notable outfits", "The

...
The Devil You Know
by Slimepriestess — 2020-05-21T21:50:01.425Z

A short story set in the EVE Universe.

The air was thick with smoke and shit. It smelled like oil and rot, rust and decay and desperation. Between the market and the river lay the slums. Between the tank farms and the market stalls rose a shantytown of ill fortunes and shattered dreams. Young people came to the city from all around the countryside, but the city wasn’t a place where poor men became rich, it was a place where rich men became richer and poor men became criminals if they were lucky and corpses if they weren’t.

Metz chewed on the end of his cigarette, breaking the bulb in the end as he stood in the entrance to the nondescript hovel. The only barrier from the outside was a grimey sheet hung across the hole in the building. The beat down old building reflected upon the structure’s beat down old inhabitant. Metz lit his cigarette and leaned against the frame. A show of force, they couldn’t touch him, not Metz Jerindold. The people of Kor’ali knew better than that, if those Krullefor thugs hadn’t realized it, so much the worse for them.

“Metz please, be reasonable,” the hobbled old man who ran the ghetto was trying to tell him, “We can’t pay protection to two gangs, the Krullefor are everywhere, they won’t take no for an answer!”

Metz smiled and shook his head, taking a drag of his cigarette. He was a tall and lanky man, attractive in his own bushwhacked and weatherbeaten way, with dark hair and darker eyes. “You’ve known me for a long time Pardin,” he said to the man.

“Since you barely came up to my waist,” Pardin confirmed.

“Did I ever give you the impression that this was an optional arrangement that we had here? That you could simply choose to do otherwise if you so wished?” Metz sneered at the ramshackle dwelling, stepping further into the dimly lit interior, looking like he’d stepped in something foul.

“The Krullefor…” the old man started.

“We’ll deal with them like we dealt with everyone else,” Metz said simply, “The Krullefor, the CBD, the capsuleers sticking their noses where they don’t belong. None of them will be here forever. They’re vultures circling a fresh carcass, but in time, they’ll wander off when they get bored, when it’s no longer in their interests to care what happens to this place. But us? we’ll still be here, and where will you be then?”

“The council decided…” Pardin started, but Metz cut him off.

“I’m well aware of what the c...
Reflective Complaints
by Raemon — 2020-05-21T21:11:48.842Z

Frequently, I'll be having an argument with someone. And I'll think "Grr! They are doing Obnoxious Behavior X!" or "Arg, they aren't doing Obviously Good Behavior Y!".

Then I complain at them.

And... sometimes, they make the exact same complaint about me.

And then I think about it, and it turns out to be true.

Another portion of the time, they don't complain back at me, but the argument goes into circles and doesn't resolve, and we both feel frustrated for awhile. And later, independently, I realize "Oh, I was also failing to do Good Thing X, or doing Bad Thing Y."

Often, "Good Thing X" and "Bad Thing Y" amount to some kind of "not listening", or "not doing enough interpretive labor." It seems to me that I'm explaining something reasonable, and they're not understanding it because of some obvious bias, which should be apparent to them.

But, in order for them to notice that, from inside the situation, they'd have to run the check of:

TRIGGER: Notice that the other person isn't convinced by my argument
ACTION: Hmm, check if I might be mistaken in some way. If I were deeply confused about this, how would I know?

And, typically, I haven't actually been making that same check, for myself. Or, I've done, it but in a kinda superficial way.

Often, adversarial conversational moves beget more adversarial conversational moves. If someone is talking over me, I'm likely to respond by talking over them. If someone seems to be ignoring my arguments, I'm more likely to ignore their arguments. This means, by the time a complaint arises to my conscious thought, there's a decent chance there's been some kind of escalation cycle where I started doing the same thing, even if they started it. (And, maybe, I started it?)

This has led me to a general habit:

TRIGGER: Notice that I'm about to complain to someone about a thing they're doing
ACTION: Check if I'm doing that thing too.

I've mentioned this concept before in passing, but it seemed important enough to warrant it's own top-level post.
What are objects that have made your life better?
by Mark Xu — 2020-05-21T20:59:27.653Z

If I wanted to buy things to make my life better, what objects would I buy? What objects do you own that spark joy every time you use them?

I'm looking for statements like "X are the best wireless, noise-canceling headphones" or "Y is the best shirt" with appropriate reasons, if they can be provided.

You can think of buying this object as a weak version of one shot life improvements.
Wrapping and Centering
by jefftk — 2020-05-21T14:30:02.911Z

In the early days of the web screens weren't very large, and people
normally wanted text to run from edge to edge. If you code up HTML by
hand,

with no styling,
this is still what you'll get. As screens got larger, though, this
could lead to lines that were too wide for easy reading. Sites have
typically handled this by adding

width or

max-width styling to force wrapping, and then usually
centering this column

as well.

As an example of this evolution, lets follow example.com
with the Wayback Machine.
Here's the earliest capture, from January 2002. No max-width, no
centering:

Around March 2002 they simplified the page dramatically (reduce
serving costs?) but it's still not centered or limited:

In January 2011 they made it redirect to an IANA page with wrapped and
centered text:

In July 2013 they stopped redirecting and switched back to a simple
page with centered wrapped text:

Most sites that are still served full-width today are ones that are no
longer updated, like the Original
HTTP Specification:

Or Sergey Brin's
Academic Page:

The only major site I can think of that runs completely unwrapped is
Wikipedia:

Mobile Wikipedia, however, is wrapped, even on desktop:

Once a site is wrapping, they need to decide where their text should
fall. The two main options are "left-aligned" and "centered". All
the sites above have chosen to center. Search engines are the main
type of site I see today that doesn't center:

I've always been somewhat retrogrouch,
but after spending some time with a wide monitor I've now I've come
around on centering like I did on wrapping years ago. Yesterday I
switched my main pages from left-aligned:

To centered:

Pages here that are not part of my sorry
excuse for a CMS will still be left-aligned (example), at least for now.
How does publishing a paper work?
by Liron — 2020-05-21T12:14:17.589Z

While I’ve never published a research paper and have no plans to do so, I realized I don’t even know how the process works. These are the bits and pieces I think I know (probably wrong about some):

Papers are annoying 2-column pdfs

Getting a paper published takes a lot of work beyond the research itself

When a paper has multiple collaborators or a student/professor relationship, there’s an awkward political negotiation about whose name is included and whose name goes first, last, or in the middle of the list

There are multiple journals you can submit to and maybe none will accept you, or maybe you’ll get multiple offers and then I don’t know if you have to pick one at most

When you submit a paper to a journal, the journal sends it out to your peers who submit anonymous feedback before publishing, which seems like more trouble than it’s worth these days because the peers might be slow or unfair, or be playing a zero-sum game competing with you

Many academic conferences have their own associated journals which you can submit papers to and in some cases getting accepted to that conference-journal means you get to give a talk at that conference

Paid-access journals currently have a monopoly on high-status research publication, and academia is stuck in this local maximum that's hard to dislodge without a coordinated effort to agree on how to publish in a high-status place that isn't a paid journal, and in the meantime the journals get to rent-seek in a way that tragically/comically undermines the ideal of academic research not being a capitalist enterprise

arXiv is a place where you can upload papers for free and people can download them for free, thereby bypassing the journals to some degree

Sci-hub lets anyone illegally download pirated papers that normally require access to a paid journal

Publishing papers is a valuable thing to do because it gives the content of the paper and its author(s) a certain social legitimacy, and allows future research to frictionlessly cite your findings

Can someone confirm or correct my impressions, and elaborate on any other interesting parts?
Comparing reward learning/reward tampering formalisms
by Stuart_Armstrong — 2020-05-21T12:03:54.968Z

Contrasting formalisms

Here I'll contrast the approach we're using in using in Pitfalls of Learning a Reward Online (summarised here), with that used by Tom Everitt and Marcu Hutter in the conceptually similar Reward Tampering Problems and Solutions in Reinforcement Learning. In the following, histories hi are sequences of actions a and observations o; thus hi=a1o1a2o2…aioi. The agent's policy is given by π, the environment is given by μ.

Then the causal graph for the "Pitfalls" approach is, in plate notation (which basically means that, for every value of j from 1 to n, the graph inside the rectangle is true):

The R is the set of reward functions (mapping "complete" histories hn of length n to real numbers), the ρ tells you which reward is correct, conditional on complete histories, and r is the final reward.

In order to move to the reward tampering formalism, we'll have to generalise the R and ρ, just a bit. We'll allow R to take partial histories - hj shorter than hn - and return a reward. Similarly, we'll generalise ρ to a conditional distribution on R, conditional on all histories hj, not just on complete histories.

This leads to the following graph:

This graph is now general enough to include reward tampering formalism.

States, data, and actions

In reward tampering formalism, "observations" (oj) decompose into two pieces: states (Sj) and data (Dj). The idea is that data informs you about the reward function, while states get put into the reward function to get the actual reward.

So we can model this as this causal graph (adapted from graph 10b, page 22; this is a slight generalisation, as I haven't assumed Markovian conditions):

Inside the rectangle, the histories split into data (D1:j), states (S1:j), and actions (a1:j). The reward function is defined by the data only, while the reward comes from this reward function and from the states only - actions don't directly affect these (though they can indirectly affect them by deciding what states and data come up, of course). Note that in the reward tampering paper, the authors don't distinguish explicitly between Rj and rj, but they seem to do so implicitly.

Finally, ΘR∗ is the "user's reward function", which the agent is estimating via D1:j; this connects to the data only.

Almost all of the probability distributions at each node are "natural" ones that are easy to understand. For example, there are arrows into rj

...
Orthogonality
by lsusr — 2020-05-21T03:17:36.365Z

TL;DR: I used to think the best way to get really good at skill si was to specialize by investing lots of time ti into si. I was wrong. Investing lots of time ti into si works only as a first-order approximation. Once ti becomes large, investing in some other tj≠i produces greater real-world performance pi than continued investment in ti.

I like to think of intelligence as a vector s={s1,s2,…,sn} where each si≥0 is a skill level in a different skill. I think of general intelligence g as the Euclidean norm g=√∑ni=1s2i.

I use the Euclidean norm instead of the straight sum ∑ni=1si because generality of experience equals generality of transference. Suppose you are exposed to a novel situation requiring skill sn+1=0. You have no experience at sn+1 so you must borrow from your most similar skill. The wider a variety of skills you have, more similar your most similar skill will be to sn+1.

The best way to increase your general intelligence g is to invest time tw into your weakest skill sw. If your invested time ts for your strongest skill is already high then investments in tw can also increase the real world performance of your strongest skill faster than investments in ts.

Suppose you want to increase p1, your real world performance at s1. ∂p1∂t1>0. Investing time t1 into s1 always results in increasing p1. But eventually you will hit diminishing returns. For every ϵ>0 there exists a δ∈R such that if t>δ then ∂s1∂t1<ϵ.

limt1→∞∂p1∂t1=limt1→∞∂s1∂t1=0

Here's where things get interesting. "All non-trivial abstractions, to some degree, are leaky" and a system is only as secure as its weakest link; cracking a system tends to happen on an overlooked layer of abstraction. All real world applications of skill are non-trivial abstractions. Therefore performance in one skill occasionally leaks over to improve performance of adjacent skills. Your real-world performance at p1 leaks over from adjacent skills sj≠1∈s on rungs above and below s1 on the ladder of abstraction.

These adjacent skills increase your real world performance on p1 by a quantity independent of s1. Since limt1→∞∂s1∂t1=0, there will inevitably come a time when increasing t1 increases s1 less than increasing ti≠1.

It follows that quantity of avocations correlates positively with winning Nobel Prizes, despite the time these hobbies take away from one's specialization.

When I want to improve my ...
Is there a way to replicate photosynthesis artificially?
by basu_enterprise — 2020-05-20T19:45:03.547Z

Humans have imitated nature since prehistoric times. Aircraft, submarines and even cars are designed in the image of wild animals. So why haven't we imitated plants yet?

The reason this question is significant is due to the increasing discordance between the existence of humanity and nature. Try as we may, the coronavirus pandemic clearly showed us just how much the national economy of many countries depends on the sale and consumption of non-renewable resources like petroleum.

Since we can neither reduce the consumption of such resources nor kill off the majority of the human population, we should try our best to transform this planet to sustain the ever-increasing human population.

The first step for such an endeavour is unavoidably the imitation of photosynthesis. It harnesses the inexhaustible supply of solar energy to convert carbon dioxide into glucose and oxygen, both of them extremely important for humans. An artificial contraption will have many advantages over its natural counterpart as it can run 24/7 and be less susceptible to external factors like disease or parasites. The productivity of such a unit can be scaled infinitely once the prototypes have been optimised.

We already have the necessary tools to create such technology. We know the composition of chlorophyll. We are sufficiently experienced in creating super thin structure and even microscopic circuits. Even though I do not have a degree in Botany or Chemistry, I believe that with sufficient stimulus, human researchers can develop such a technology rapidly.

So the only question remains is: Why haven't the developed countries of this planet invested in such a project already and what will it take for the scientific community to focus on this topic?
Comment on "Endogenous Epistemic Factionalization"
by Zack_M_Davis — 2020-05-20T18:04:53.857Z

In "Endogenous Epistemic Factionalization" (due in a forthcoming issue of the philosophy-of-science journal Synthese), James Owen Weatherall and Cailin O'Connor propose a possible answer to the question of why people form factions that disagree on multiple subjects.

The existence of persistent disagreements is already kind of a puzzle from a Bayesian perspective. There's only one reality. If everyone is honestly trying to get the right answer and we can all talk to each other, then we should converge on the right answer (or an answer that is less wrong given the evidence we have). The fact that we can't do it is, or should be, an embarrassment to our species. And the existence of correlated persistent disagreements—when not only do I say "top" when you say "bottom" even after we've gone over all the arguments for whether it is in fact the case that top or bottom, but furthermore, the fact that I said "top" lets you predict that I'll probably say "cold" rather than "hot" even before we go over the arguments for that, is an atrocity. (Not hyperbole. Thousands of people are dying horrible suffocation deaths because we can't figure out the optimal response to a new kind of coronavirus.)

Correlations between beliefs are often attributed to ideology or tribalism: if I believe that Markets Are the Answer, I'm likely to propose Market-based solutions to all sorts of seemingly-unrelated social problems, and if I'm loyal to the Green tribe, I'm likely to selectively censor my thoughts in order to fit the Green party line. But ideology can't explain correlated disagreements on unrelated topics that the content of the ideology is silent on, and tribalism can't explain correlated disagreements on narrow, technical topics that aren't tribal shibboleths.

In this paper, Weatherall and O'Connor exhibit a toy model that proposes a simple mechanism that can explain correlated disagreement: if agents disbelieve in evidence presented by those with sufficiently dissimilar beliefs, factions emerge, even though everyone is honestly reporting their observations and updating on what they are told (to the extent that they believe it). The paper didn't seem to provide source code for the simulations it describes, so I followed along in Python. (Replication!)

In each round of the model, our little Bayesian agents choose between repeatedly performing one of two actions, A or B, that can "suc

...
[AN #100]: What might go wrong if you learn a reward function while acting
by Rohin Shah — 2020-05-20T17:30:02.608Z

Newsletter #100 (!!)

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

HIGHLIGHTS

Pitfalls of learning a reward function online (Stuart Armstrong et al) (summarized by Rohin): It can be dangerous to learn the metric that you are trying to optimize: if you don't set it up correctly, you may end up incentivizing the agent to "update in a particular direction" in the metric learning for the sake of future optimization (a point previously made in Towards Interactive Inverse Reinforcement Learning). This paper analyzes the problems that can arise when an agent simultaneously learns a reward function, and optimizes that reward function.

The agent may have an incentive to "rig" the reward learning process, such that it finds a reward that is easy to optimize. For example, consider a student Sandra who must figure out the deadline and evaluation criteria for a project from a teacher Trisha. Sandra expects that if she asks Trisha when the deadline is, she will say that the deadline is later this week. So, Sandra might cleverly ask, "Is the project due next week, or the week after", to which Trisha might respond "next week". In this way, Sandra can rig the deadline-learning process in order to obtain a more favorable deadline.

Worse, in such scenarios the need to rig the learning process can destroy value for every reward function you are considering. For example, let's suppose that if Trisha couldn't be manipulated, Sandra's optimal policy would be to start the project today, regardless of when the actual deadline is. However, given that Trisha can be manipulated, Sandra will spend today manipulating Trisha into setting a later deadline -- an action that seems clearly suboptimal from the perspective of any fixed deadline. The paper describes this as sacrificing reward with certainty.

To avoid such situations, we need unriggable learning processes, that is, ones where at all times, the expected final learned reward (deadline) is independent of the agent's (Sandra's) policy. This unriggability property is nearly equivalent to the property of uninfluenc...