skysight-inc/HackerNews huggingface dataset:
This is timely - I wrote an ed guide today and this post is a good opportunity to share it. I've been working on a remote server via ILO-in-a-browser with broken console definitions and a screen that's too small for the interface. It's a gift - I've wanted to be fluent in ed for a long time and spent the time to get there today.

If anyone has something similar for console-mode sam usage I'd love to know that too.

Anyway - the guide:

Ed doesn't push any of the fancy cruft that modern
editors do - full screen views of the text in question
or fiddly mouse interfaces.

It is line based, scriptable, it works *everywhere*
that unix runs, and the default configuration is
predictable. A true power tool from the days of yore
when men were men and each tool did one simple thing
extremely well.

How to get into ed and create some text:
<pre>
$ touch my_filename
$ ed my_filename
0
a
Here is some text
next line
more
yet more
The dot on the next line indicates I'm done inserting text. Then save and exit.
.
w
q
$
</pre>

Basic commands:
* a: append to the first line
* p: print the current line
* n: print the current line with a line number (handy)
* %n: list all lines
* 1n: list line one
* 1,5n: line lines one to five
* 5,$n: list lines five until the end
* 2d: delete line two
* 3,4j: merge lines 3 and 4
* 2i: insert text to a new line before the current line 2
* P: enable prompt
* %l: - that's a lower-case letter after 'k' not the
number one. Print all lines, including end-of-line
markers (useful in dos2unix settings among others).

Regular expressions (this is where it gets exciting):
* 2s/next/another/: change the word 'next' on line
two do 'another' (don't forget that trailing slash)
* g/e/: find all the lines containing the letter 'e'
* g/e/n: find all the lines containing the letter 'e'
and then print them out with their line number next
to them.
* %s/e/q/: change the first instance of the letter 'e'
to 'q' on all lines
* %s/e/q/g: change all instances of the letter 'e' to
'q' on all lines
* 1,5s/e/q/3: change the third instance of the letter
'e' to 'q' on all lines
* %v/for/n: print out the lines that do NOT contain
the word 'for'

Other magic:
* 1,4y: cut ('yank') lines 1 to 4
* 5x: paste to after line 5
* 1,3t6: Transfers lines one to three to a position
after line 6.
Why do you say that scientists to day are dull and don't have a sense of a mission? I really feel, since I never see any real evidence, that these sort of claims are just self-serving.

Your example of the physics professor is fine, in that you actually provided an example, but I'm not sure that is a fair evaluation. I think his question is fairly reasonable. The field of physics is so large and the amount of knowledge and sophistication required in both mathematics and physics to actually "discover laws of the universe" that it would be very difficult to achieve such a level by simply independent study.

Sure, there people like Srinivasa Ramanujan but they are very, very rare. I think there is too much made about autodidactism. There are many autodidacts. However, how many autodidacts take their learning to such a rigorous degree? Now, compare that with how many people read a few books on physics and are very interested in it but never come close to that level of sophistication.

I just don't think we can say "See, academics aren't even interested in learning about the World. They are glorified bureaucrats that can solve differential equations."

I think that is a bridge too far.
It has little to do with credit. Writing a book/survey paper can get you great recognition if you finish it and do a good job.

The reason scientists write papers is because science is fun, but writing is boring (compare: coding vs documentation). Research papers represent the bare minimum level of documentation you can get away with but still keep your job.
Pssst...read it here: http://design.caltech.edu/Misc/pirsig.html
Nit picker.

:)

I was watching the original version of The Day the Earth Stood Still, and there was one scene that stood out as marking this film as truly from another time.

Klaatu is talking to the boy, asking him about who the famous scientists are. And the boy knows! And acts as if everyone in American knows!

I gather there are many interesting contemporary scientists, but they simply get no attention in modern culture.
"OGG file for Firefox 3.5, and one MP4 file for everything else (Flash / Safari / iPhone)." I think this illustrates the problem with FF's choice of codecs-- you could do this all with one encode!

Also, the iPhone's supported h.264 profile lacks major features that other platforms (ie Flash) support. Serving one encode downsampled to the iPhone's limited profile is serving to the lowest common denominator. Serving a smaller iPhone-only copy and a second copy with B-frames and more efficient encoding would not only be smaller but would look a lot better too!

Does anyone know if there's any plans to support <video> for other codecs?
And if meditation is too artsy for you...
I used to work on doing tricks with my soccer ball for a while after work everyday. They weren't easy, so I basically didn't think about anything else and essentially held one state of mind the whole time -- pretty close to meditation, I'd say. Headstands work too, until they become easy ... then you can move onto handstands! :D
Books used to contain original research and theories. (Think Stanley Milgram's book Obedience.) It's only in the last 30 years that science books tend to just be collections of other people's papers rewritten for the lay audience. Because modern academia rewards people for churning out large amounts of derivative work but not for spending time coming up with groundbreaking theories, pretty much the only time scientists write books is when they get 600k advances (like Gilbert).
Tolerance and shared vision/values are definitely important. But that only comes after there is an equal desire to be in a long term relationship. For some people that desire wanes (for whatever reasons), and that's why these lasting connections (professionally and privately) are rare.
"Whoever decided to use the semicolon to end something should just be taken out and have his colon semified."

Thanks Erik, I didn't know you, but you made my life happier for a whole minute :-D
Ask YC: Favorite email parsing library?
Any recommendations on a good library for parsing emails in mbox format? I'm pretty agnostic as to language (I don't know Python, but if the library is that good, I will learn it). I mostly care that it takes care of all the language encoding business for me. I've been using the Perl Mail::Box suite, and language encodings with that is just a mess.

Thanks everyone!
Thanks for the reply. 9vx looks promising.

I'm surprised that acceptance of the browser is so global, but particularly in a community like yours. HTML is dirty, HTML+CSS+javascript is dirty. Plan 9 folks know about tool simplicity, and about nice documents. I've always half expected something to come from your team that says "no - this is all wrong. The browsable internet should be like this" that gives the middle finger to the HTML world, and which can easily be replicated as a firefox plugin and then go on to take over the world.

I guess I'm saying that I think you took the wrong path by attempting to follow the browser - you should have challenged it, but in a way that didn't rely on plan9-specific user interface innovations [1].

I've had several attempts at it but don't hang around in circles where I get to talk about the problems regularly face-to-face with other people who get it, and that's what a problem like that requires.

The site you've pointed me to is a real improvement over the online docs that were around two years ago. It answers the sort of questions a newcomer asks.

--

[1] I think sam would have gone further if the console tool had been the core with the gui as a separate app that leveraged the core. Console sam should be to ed as vim is to vi, but it's trying to do two things at once, and the gui editor - which I'm sure is very good for people who know it well - hampered the progress of the console app.
you want to avoid a bogey because it starts a vicious cycle. assuming you are at par, a bogey requires a birdie just to get back to par. going for that birdie risks another bogie that would require another birdie to balance, and so on. playing in this highly volatile manner you run a higher risk of having to make great shots in order to end up with an average score. it's more in your favor to wait until you make a great shot (say a great second shot on a par 4 that leaves you two feet from the hole) and then capitalize on the "free" consequence (the two foot birdie putt) that actually takes you above average than to be forced to sink a 50 foot birdie just to stay at average (because you missed par on the previous hole).
Erik Naggum's wonderful rant about XML
Doctorow Says Google & Amazon Stifle Progress
Is there something special about this? I imagine you'd find equivalent content on dozens of pages by searching for "vi commands".
Benford's Law Anomalies In The 2009 Iranian Presidential Election
I was a theoretical physicist for 13 years, and struggled a lot with this question. I found it very useful to develop several different styles for reading mathematics and physics. Mostly I did this in the context of reading papers, not books, but the comments below are easily adapted to books.

One unusual but very useful style was to set a goal like reading 15 papers in 3 hours. I use the term "reading" here in an unusual way. Of course, I don't mean understanding everything in the papers. Instead, I'd do something like this: for each paper, I had 12 minutes to read it. The goal was to produce a 3-point written LaTeX summary of the most important material I could extract: usually questions, open problems, results, new techniques, or connections I hadn't seen previously. When time was up, it was onto the next paper. A week later, I'd make a revision pass over the material, typically it would take an hour or so.

I found this a great way of rapidly getting an overview of a field, understanding what was important, what was not, what the interesting questions were, and so on. In particular, it really helped identify the most important papers, for a deeper read.

For deeper reads of important papers or sections of books I would take days, weeks or months. Giving lectures about the material and writing LaTeX lecture notes helped a lot.

Other ideas I found useful:

- Often, when struggling with a book or paper, it's not you that's the problem, it's the author. Finding another source can quickly clear stuff up.

- The more you make this a social activity, the better off you'll be. I organize lecture courses, write notes, blog the notes, and so on. E.g. http://michaelnielsen.org/blog/?p=252 (on Yang-Mills theories) and http://michaelnielsen.org/blog/?page_id=503 (links to some of my notes on distributed computing).

- On being stuck: if you feel like you're learning things, keep doing whatever you're doing, but if you feel stuck, try another approach. Early on, I'd sometimes get stuck on a book or a paper for a week. It was only later that I realized that I mostly got stuck when either (a) it was an insubstantive point; or (b) the book was badly written; or (c) I was reading something written at the wrong level for me. In any case, remaining stuck was rarely the right thing to do.

- Have a go at proving theorems / solving problems yourself, before reading the solution. You'll learn a lot more.

- Most material isn't worth spending a lot of time on. It's better to spend an hour each seriously reviewing 10 quantum texts, and finding one that's good, and will repay hundreds of hours of study, than it is to spend 10 hours ploughing through the first quantum text that looks okay when you browse through it in the library. Understanding mathematics deeply takes a lot of time. That means effort spent in identifying high quality material is often repaid far more than with (say) a novel or lighter non-fiction.
"Like it or not, there's no guarantee of job security even for the plumbers and electricians and carpenters."

Well, to the extent that there is no guarantee of anything.

What make you think that improvements in technology in the next 30 years will enable machines to crawl under sinks or behind furniture; make estimates about corrosion, or wood grain, or wiring quality; manipulate small, complex items in tight spaces; explain the options (fix now but risk breakage later; replace now, but pay larger cost upfront) and answer questions, and do all this in a timely and friendly manner?

There is a large human factor to much manual labor that is hard to replicate with machines or remote operation.

Most everyone I know who ends up dealing with a non-local telephone support person hates it. Trying to get a decent explanation of why I should have my plumbing replaced rather than repaired from someone with poor localized language sounds skills sounds like a nightmare.

A better bet would be for, say, North Americans to operate remote repair devices for locations in North America (and likewise for other language-unified locales).

Perhaps a local, unskilled person operates a device to feed information back to a pool of technicians in some other location; you get the human element, but centralized technical smarts.

I believe something like that happens now with medical imaging. Technicians run the scanning machines, send the images off to who-knows-where for skilled readers to interpret, the results are moderated by a more-skilled supervisor, and sent back to the lab or hospital.

Still, I get the feeling that fixing bikes is part engineering, part art, and transmitting sufficient information over the wire for remote viewing may a hard problem. Like cooking, some things are just better handled by real live people.
d	Delete text. (see explanation above)

And where is the explanation?
It's illegal to use Internet Explorer without a valid Windows license, even through Wine.
Nearly every Naggum post on comp.lang.lisp was an attempt to hurt other people; he was a master of emotional cruelty. I was never a victim of it myself, as far as I can remember, but it repelled me from the forum.
I love that article. It has been a favorite of mine for quite a while.
Some of the "problem" might be that things are a bit more entreprenurial than previously. Now, if you've got a wild idea (that just might be right) and you've got the ego to go with it, you can start a company and profit from it. Way more $$ in that than academia.

Similarly, there's more $$ in working in industry with a PhD rather than academia. And the tenure/grant game requires those in academia to have political and social skills that might be off-putting to the smart/driven set being talked about.

Finally, it's possible we're just not that interested in scientists/engineers (as in the 50's and 60's). Better to know what Lindsay Lohan's up to... There's like an hour of entertainment news EVERY DAY, and maybe a science article or two per quarter.

Followup: Rephrasing my Lohan comment - there might be all sorts of interesting scientists out there, but how would any of us know unless they're in the field? Rock stars, nowadays, aren't big-name scientists like they used to be. They're just rock stars...
Business Processes and the human factor
A murderer typically only kills one person, causing loss to a few dozen. Naggum didn't, as far as I know, kill anybody; he merely turned a leading-edge technical community into a kind of twisted cult of personal abuse.

Every person alive today is a little bit poorer because of the poor state of Lisp implementations, which can largely be attributed to Naggum's vicious dominance on comp.lang.lisp, and so for a long time Lisp lacked any kind of widely-used forum for technical discussion among sane people.
I am implying that. Microsoft's motivation behind IE is to exploit their dominant position to crowd out the rest of the competition. As I explain in my first post, compelling Windows licenses for testing (and, by the way, many are on the line, I think) is just a small part of the overall plan (possibly the only part that's working really well right now).

Microsoft wants IE to be different because they hope that eventually companies will decide it's not worth the cost to support or utilize the technologies that have smaller marketshare.

For a while, this worked with IE and there were large swaths of the web inaccessible through other browsers. Recently, it hasn't been working as well and in fact has been backfiring as frustration with Internet Explorer mounts and developers shy away from supporting it, or imbue it with lesser features, which is why IE is much more compatible than it used to be, but still not _quite_ there.

This strategy still works well in other areas and has been part of Microsoft's philosophy for many years -- see the Halloween memos for more.

Microsoft really has no incentive to continue Internet Explorer _except_ control. They leverage it to try to make Windows a requirement to browse the web, which would render OS X, Linux, and any other potential competitors comparatively useless for desktops. That's the mission and goal of Internet Explorer.
International Editions have saved me a ton of money. I found out about this when I started seeing big C++ books (the really thick ones) around the lab with a big sticker on it that read "not for sale in the USA" (or similar). These are the same book, though usually a bit more cheaply made, but at sometimes 10-20% of the price.
I think his question is fairly reasonable. The field of physics is so large and the amount of knowledge and sophistication required in both mathematics and physics to actually "discover laws of the universe" that it would be very difficult to achieve such a level by simply independent study.

I wanted to disagree with your sentence, but now I see it is not grammatically valid. I'll try to guess what you meant and disagree with that.

I heard this argument many times. It's basically, "There's too much physics to do physics!" Just because there's a lot of physical knowledge out there, doesn't mean that you have to be a super-genius to do fundamental work. I don't know exactly how to explain this, but I think I can equate it to the internet business, which is probably more familiar to most HN readers.

What people say about science can also be said about the internet business. There are so many companies working on internet products, so many start-ups and so much money being poured into this, that surely you have to be a genius with a lot of money and connections to do something that will make a difference!

But we know that's not true. And I claim it's not true for science either.

--

it would be very difficult to achieve such a level by simply independent study [...] I think there is too much made about autodidactism.

I think you're dissing autodidactism too much. What do you think about people teaching themselves to program?

Also, just because you're studying independently, doesn't mean you don't consult with other people. I routinely consult with grad students.

Also, when you're doing research, you're being autodidactic by definition.

--

Sure, there people like Srinivasa Ramanujan but they are very, very rare. [...] how many autodidacts take their learning to such a rigorous degree?

How many HN readers actually found a successful start-up, get rich, and live the rest of their lives leisurely working on interesting problems? Probably not too many. But it's not a reason to give up.

--

I just don't think we can say "See, academics aren't even interested in learning about the World. They are glorified bureaucrats that can solve differential equations."

It's a generalization. There are still many fine scientists. But when trying to compare now to the past, this is what now looks like.
I found the last sentence sad, though not unexpected:

The TSA says disciplinary action has been taken against
one of its employees for inappropriate tone and language.

Of course, tone and language aren't the crux of the problem here - stomping of a passenger's rights. That wasn't acknowledged (according to the article).
Well, I don't think he would mind. He would mind if someone said something stupid about it ;)
That's also the kind of thinking that made the internet and the Web workable systems, while Hyper-G, OSI, SNA, DECNET, Xanadu, and so on ended up in the dustbin of history. Yeah, it's too bad that at some point someone has to build quirksmode.org and write the HTML5 standard, but that's better than not having widely-used web standards to deal with in the first place.
Sortable table rows with jQuery.
Bottom up view of the brain (edge video)
john bargh studies the brain from its most primitive subliminal core up to modern consciousness. this is a long breezy video - stream of sub-consciousness :) but insightful nonetheless.
This is utter bullshit.
How the macintosh got it's distintive look
With all due respect to Hawking - I don't think he's anywhere near as interesting as Feynman or Erdos.

Feynman traveled to Brazil to learn Bongo drums on whim, he worked in a Strip Club, mastered the art of the pickup, broke into TS safes at the Manhattan Project, was a well known painter under a pseudonym, etc.

Erdos was basically homeless, and would show up at mathematician's doors unannounced looking for a roof - and while under their roof, would collaborate on some of the most ground breaking work of the century. He was a heavy drug user, active libertarian, etc.

I'm sure Hawking can compete in terms of intellect or work (I have no idea - I'm not capable of judging), but he is nowhere near as interesting.
Textbooks are indispensable for really getting the material down, but I've found it much easier to read lectures.

Feynman has great examples of this type of work: "The Character of Physical Law" or "The Feynman Lectures on Computation" are both excellent. I've found that these books walk somewhere between textbook and popular science book. Easy to read but all the while teaching you the material.

I've found this type of work to be the crucial link between pop. science and a textbook. You learn enough that you can find your footing when you're in a textbook and and also the inspiration to actually read it.
The iPhone 3G S: finally, iPhone hardware that's worthy of iPhone software.
Here's my list of distributed papers, I think you're missing quite a few good ones:

http://bytepawn.com/readings-in-distributed-systems/

But, more importantly, I'm curious about the 15/3-routine. How often do you do this? Recently I subscribed to the arxiv RSS (astro-ph.co, gr-qc), there's lots of papers uploaded daily, but most don't look very interesting, so 15 interesting ones would be ~1 weeks worth for me.
http://www.open-voip.com/blogs/blog1/2009/06/20/erik-naggum-...
No, it demonstrates that they are likely to get 1000s of e-mail every day from random people claiming that their servers are down, or they send out corrupted emails or whatever, when in fact the problem is squarely on the users end. Processing all these to catch the odd legitimate one has a very unfavourable cost/benefit ration.

Yeah, it is a legitimate complaint, and it'd be quite simple to fix. I want them to make their AJAX degrade more gracefully, so the site will work on my cell in Opera mobile.

I would be nice with a "I know what I'm talking about" certificate that would bypass front-line support for those who really do understand what they're talking about. A secret handshake or something :)
A Chat with Fred Wilson [video]
Jeff Hawkins: Inside the Mind of a Reluctant Entrepreneur