skysight-inc/HackerNews huggingface dataset:
Ask HN: how would you graph news comments and/or website crosslinking?
I've been wondering for a while if there's a way to analyze and graph comment spam, on blogs or particularly on news websites.

Not the 'great deals on shoes http...' variety, but more astro-turfing, where a small group of people who are fanatical about a certain issue leave comments on any relevant news story/discussion forum they find. Often they simply pasting the same comment over and over (even on multiple stories at the same site) until they get banned, or they might post the same generic comment on lots of different news sites but change it up each day so their contributions on any individual site look varied and unique.

The task is quite similar to plagiarism detection or lexical analysis, but I'm not aware of good tools for doing that on a really wide scale. I'm looking for similarities both across the universe of news websites and blogs, and over time, especially in response to specific news events where an issue yields a lot of coverage and the astro-turfers go into overtime.

In a related task, there are a relatively small number of websites focused on an issue, and there would be a great deal overlap between them, they would be constantly citing each other and so on, partly to ensure that when a mainstream news outlet covers 'their' issue one or other of them is sure to be quoted as an authoritative source. Of course, there would be a great deal of outward linking to various news stories from articles/forums on these subject-specific websites.

This is a bit like recursively graphing blog track/pingbacks, or retweets, or citations in academic papers or legal judgments. Unfortunately most of the analytic tools I've looked at either do geomaps or just graph statistical aggregates as bars. What I'm really interested in is looking at the patterns of propagation - both to see how they evolve, and to identify major vectors leading back to sources. This seems like something best handled by De Bruijn or directed graphs, but my knowledge of graph theory and related tools is pretty limited.

I'm interested in the content of the subject material rather than product development, so tools/services > APIs/frameworks > theory/research - although I'd be grateful for any and all suggestions. Thanks for your interest.
Nice. Specific commit: http://android.git.kernel.org/?p=platform/libcore.git;a=comm...
Ah, no. Actually, I have. I think our eyes are wide open, thank you very much.
Hello there,

I would also create a landing page for that domain name mentioning that it is for sale. You could then redirect them to your sedo listing or you could place a simple contact form for them to contact you.

Good luck!

Karl
I'm a fan of minimalism, but I'd tend toward something of a middle ground.

e.g. in the case of old digg, I'd go for something that added color to headlines to more clearly separate articles, de-emphasized the category/posted-by line, and narrowed the summary column.

too-wide blocks of text, too-close bunches of paragraphs and too-strict hit-boxes for buttons are pretty common on 'minimalist' sites and those drive me absolutely crazy anymore.

Too-small fonts also used to bother me, but browsers have largely resolved that. So long as minimalist design plays well with dynamic font scaling, that doesn't bother me anymore.
That is clearly webspam. We only indexed five pages from that site, but there's no reason for that site to show up at all. Thanks for mentioning it. It won't be in our index for much longer.
I'm in the exact same boat.

Funny thing though - even if Google deprecated Reader, it would most likely still be like Google Notebook which has been end-of-lifed, but still completely usable years later.
The Road to Better Authorization
Facebook raises $1.5 Billion at $50 Billion Valuation
Can I get my nickel, please? I was promised one every time the phrase "they're selling an /experience/" was uttered.
His comments amount roughly to: "I am too a real developer," and lots of "maybe this code was shipped," but nothing concrete.

edit: Thought the blog post was removed from Techmeme, but was mistaken.
Sadly, while sensational articles like Engadget’s and Mueller’s will get splashed all over the web and lavished with thousands of views and hundreds of comments, the boring truth will rate no such attention.

I think this speaks to the poor quality of tech journalism. A part of the problem are poorly informed consumers.
Excellent 101: How to Configure Analytics for Your Small Business Website
Needs tuning; every 2.2-magnitude earthquake seems like noise in this context, for instance.
Two 25-year-old American entrepreneurs launch a Groupon clone…in France?
> "Since corn and soy-based foods are one of the leading causes of obesity"

In the US, yes. There are many cultures that have been heavily soy-based, for example, for centuries without health issues.

Furthermore, there are even more developing countries on Earth that rely on corn as a staple food. The diversion of corn supplies to create fuel over food is widely regarded as a main cause of the last food price crisis, which impacted poor countries disastrously (compared to the rather mild price hikes seen in this country). There is also the issue of land being used to grow other non-corn food crops being diverted to corn-fuel farming, thus further reducing food production.

If the entire world was just the US, I'd be inclined to agree with you, but we don't exist in a vacuum.
HN is a good example of how doing one thing very right (no clutter) can overshadow doing many things wrong/unsatisfactorily (no built in search, no instructions on how to use certain features...). Also Applies to a lot of Apple products
No, they don't become stupid. But their view of the uses of computers changes. Everyone reading this post is in the top 5% in terms of computer skill. The majority of the friends of everyone reading this post are in the top 25% in terms of computer skill. It gives a very warped perspective of the "average computer user". You know that guy who's still getting the hang of EBay and Amazon and who Googles URLs instead of typing them? He's the average computer user[1]. Which means half of all computer users are less capable than him.

For instance, consider the sentence "Angry Birds is lots of fun - you should get it from their website, they take paypal"[2]. To HNers that's incredibly straightforward and simple - (1) Google "Angry Birds", (2) make sure it's a legit site, (3) look for the "Buy" button, (4) check out, (5) enter your paypal information, and (6) install the game. To me (and most HNers, I'd imagine), the hardest part of that is "what was my PayPal password again?", but to the average computer user, that's a confusing 30 minutes, part of which will be spent on an ornithology site. It's for that average user that things like the Mac App Store and locked-down phones were designed.

The point of this long, meandering post is that the reason techies have issues analyzing Apple's strategy is because they mis-over-estimate the skill of the average computer user, for the totally understandable reason that they're surrounded by people with well-above-average computer skills.

[1] - Probably hyperbole, but not by much.
[2] - I've actually never played Angry Birds.
A lot of technical people love these kinds of UIs, but they can be unwelcoming to the average user on the web who might not be as technical or might be intimidated by the amount and presentation of the content, especially if it's mostly text.
I live in Canada and none of my friends heard of Groupon.
Python and Ruby both ship source with the product. Most (all?) Android builds (and most other compiled binaries) don't include the unit tests.
Open source projects always ship test code (when they have some).
Not really a choice there - people use craigslist because that's where the people are.
Has anyone thought of paying something like Y Combinator $20,000 instead of going to college? It seems the value extracted is enormous...
Why? Why do all companies have to operate the way you want them to and why do you have to buy their things?
Apple's Giving its Customers a New Kind of Screw
I think he is a strong advocate of getting rid of SW patents. But I think he thinks companies like IBM hide behind the GPL for somethings to get community favor and then use their patents in other endeavors (like Hercules).

I'm speculating, but I think he views the problem as being patents, and thinks the effort around the GPL to be a distraction and one that fundamentally doesn't solve the patent issue. Again, only speculating, but I think he'd like to see a real patent nuclear attack which will push the industry to move to get rid of patents.
I have an 07 Honda and the dealer (and I believe the owner's manual) specifically instructed against any ethanol. Unfortunately, I'm also supposed to avoid anything above 87 octane... Basically, I just have to go around Iowa.
How Should a White House Quora Work?
This isn't quite the same thing as installing open-source code onto a cellular phone.
Just a side note about those weird screws.

When dealing with consumer wifi gear, the FCC required that we use non-standard antenna connectors, for instance - to prevent people from just putting whatever they wanted on there.

This wasn't specifically to prevent tampering, but to make it evident that, from our point of view as a company, we do not intend or encourage people to just open it up and mess with it.
When I think of notable portmanteaus e.g. chortle, chillax they don't often fit with the formula of matching whole phonemes at the end of the first word and the beginning of the second (Chortle uses the first phoneme, the middle phoneme of word two, then back to word one to finish).

That said, the werdmerge discovered concept of a chillaxative made me chortle a bit.
While the examples (HN and old Digg) are minimal, I also find them "un-refined". I prefer minimal and refined.

Like Delicious. Simple color palette, lots of white space, and no extraneous stuff like gradients, rounded corners, drop shadows, etc.
As a newbie in the blogging culture in Silicon Valley, I honestly can't agree more. I want to see more of this style of reporting and I try pretty hard to do it myself.

I guess I'm what you'd call a "classically trained" journalist. So maybe I'm just feeling a little pretentious or annoyed at the state of things out here. But 99 percent of the time it feels like people are too damn lazy to pick up the phone and call their sources — or even shoot them an email and ask for a quick comment. I almost fell into that trap when I first joined the technorati out here but have since reverted to my own personal instincts of actually talking to people for stories (I know, what a concept, right?)

For a new face in silicon valley it's honestly extremely discouraging and frustrating — not only seeing it in other blogs but from my co-workers from time to time at VentureBeat as well. It's even more discouraging to be encouraged by my editors from time to time to simply bust out a story without any additional reporting. I feel pretty proud that I have not done a straight reblog without additional reporting in more than a month when it's merited.

Naturally, some stories don't demand that kind of additional reporting. So there's a place for analysis, commentary, and the like. I'll be damned if I end up as an MG Siegler-type reporter that injects opinion/praise/vitriol in every single article I write, though. I don't really see that opinion changing any time soon, either.

(Edit for quick background: former Reuters reporter, graduated in 2010.)
YC always seemed like a plan to reorganize corporations.

In Capitalism Hits the Fan lecture http://www.reddit.com/r/Documentaries/comments/f35gs/capital... the professor concludes that startups are really communism, because the workers own the "means of production" and directly benefit from the profits made. So capitalism doesn't deserve credit for much of our technological progress.

Startups are also the only solution he knows for fixing what's wrong with the US economy. Bailouts, tax-breaks, regulation and deregulation won't help. The conclusion in his lecture sounds funny and simplistic but 500 startup incubators agrees with his basic premise.
Sure, it's a means.

So is scraping - a means to sell big lists of e-mail addresses to people who buy them for many thousands of dollars, with little regard to where they came from - only that they receive e-mail.
G-WAN - High-performing HTTP server, scriptable using ANSI C
But electric cars don't inherently save us from the detrimental aspects of burning fossil fuels. You have to change the source of the energy that we are putting on the grid as well.

You're right - in the short term an electric car is a coal/oil car. The advantage of abstracting out the energy source from the car by using electricity is that it can get cleaner over time. A car that gets 30 mpg today will get no more than that 10 years from now. An electric car that gets 300 miles on the equivalent of 20 gallons of gas/coal/oil can get more efficient over time as the grid gets upgraded to phase out fossil fuels. It is far easier to upgrade the 5000 or so power plants in the US than constantly upgrade the few hundred million cars on the road from gas to ethanol to hydrogen to mystery fuel.
"I would have written a shorter letter but didn't have time." - Blaise Pascal
The core of game theory is just the min-max result from von Neumann and Morgenstern. Actually, just as mathematics, linear programming is a slightly stronger subject, and the min-max theorem is a simple consequence of standard results in linear programming.

This von Neumann - Morgenstern result is for, say, the game of Paper, Scissors and Rock. Closely related is the J. Nash result on bi-matrix games where, basically, the difference is two payoff matrices instead of just one.

Beyond those cases there is the case of games such as chess and checkers. Here it is easy to prove that if players play as best as possible, then the outcome is determined just by who moves first. So, the player who moves first may get always a win, a tie, or a loss, depending on the game. To determine what a player should do, the calculation is a simple tree traversal; the practical problem for games as complicated as chess is that the tree is too big for current computers. As some games illustrate, there can be fast, simple algorithms to determine the moves with no tree traversal at all, but so far apparently no one knows such a simple algorithm for a game such as chess.

What else in game theory did you have in mind?
They definitely sell experience. The average user mostly wants[1] (1) email, (2) Facebook, (3) iTunes, (4) web surfing, and they don't really care about how the box on their desk gets their emails, whether it's open source or closed source, locked down or modifiable, or even if it's electronic or just really fast carrier pigeons.

Like the article said, the average user wants toast, not a toaster.

[1] - it's starting to look like we can add internet video to this list.
Review my start-up: Make your Girlfriend Happy (Valentine's coming up)
http://www.makeyourgirlfriendhappy.com

I put this as a comment because it is a resubmission from a few months ago (a lot has changed!).
New alleged evidence of Android infringement isn't a smoking gun
It might, if it wasn't actually a problem in all forms of journalism. The bleeding lede wins, and most of the people who come across an incorrect story won't even hear about the correction.
Ryan's post is good, he explains things more thoroughly.

Very important:
"It's not clear how the zip file got included in the AOSP, but it's obvious that it wasn't intended to be there and isn't actually used by Android in any capacity."
Definitely agree with you. I would add that choice is quite often made by the technology leadership which of course is a corollary to the technology choices already made. One last point, I will make is that cost factors in at some point. The interesting thing about cost, though, is it is often seen through two different prisms: upfront cost (which may may Macs more expensive) and total cost of ownership (which may be the Windows PC.)
http://www.makeyourgirlfriendhappy.com
OK, to the two people who have downmodded my post, please clarify. This is a serious question: I don't understand why people who don't buy Apple products complain about Apple. It's not like they completely control the marketplace -- so what's the point (other than whining)?
App makers should heed this message: You are in the experience business.
Finding Cash for Your Start Up – Part 1
Running SproutCore with Passenger or Unicorn
A brief article explaining how to move from SproutCore's built in sc-server to a more flexible approach.
I think the two are orthogonal. People who are driven to create art will keep doing it, growing, learning from other artists they come across; they might seek out more formal training as part of their journey or not.
Setting Up And Testing Devise - Delicious Clone
There is a google tech talk on the book which is really good for those curious to find out more.
Well it's about time someone tried it. Computers have been failing the 99% rest of the market for a long time now. I'm a GPL advocate, but I'd argue the problem Apple is tackling is more important.
Note however that deleting the files don't suddenly make the copyright infringement claims invalid.
> "That some of our abilities are inherent. They just have to be set free."

Heavily disagreed. I'm a street photographer myself (self-trained), and Maier's work has been inspiring both from an artistic and motivational perspective.

These abilities aren't "inherent". They take years and years of practice to hone, and a constant criticism of your own work (from yourself or otherwise).

I don't think any photographer ever picked up a camera, figured out how to work it, and then just started taking good work. Bear in mind also that I don't know a single street photographer who's able to shoot anything close to >5% keepers, much less show-ers.

When you view art like this, keep in mind that not only is it a highly selected slice of the artist's entire body of work, but also you are looking at what is most likely their top work at the prime of their abilities. What you don't see are the years of producing forgettable, boring, and downright bad work to get there.
min deftnly
I agree.

Some people who would have made excellent entrepreneurs will never become one because their existing lives are satisfactory. However, they may still develop many of the same skills as an entrepreneur while rising up through the ranks of a corporation, which puts them in an excellent position to write business literature.

Others will never become one because their lives are hard and making the jump to starting a business would only make them harder.

For everyone else in the murky middle, the question remains. "Should I become an entrepreneur?" And if the answer is 'yes', falsestprophet's question follows: "Will I become an entrepreneur?"

Entrepreneurs are people who are risk-averse enough to try something new. The successful ones minimize risk by making calculated decisions that make sense.

Every entrepreneur has a unique story to tell, but not so unique that you can't imagine yourself fitting in to the stories of some entrepreneurs.
Getting further along in the process of re-inventing SQL, I see.
Hello there,

http://www.djangoproject.com/

The Django project is one of the best frameworks for Python Programming. If you would like to contribute to the visit this link:

http://docs.djangoproject.com/en/dev/internals/contributing/...

This page explains how you can contribute to the project.

You can also contribute to few popular desktop Python programs such as Miro and Blender.

Good luck!

Karl
I concur. If anything, they provide tremendous innovation that then can be copied in more open ways. If not for the iPhone, there's no way the rest of the cell phone marketplace would look like it does right now.
Ken Auletta got outstanding sources at Google because he had already climbed his way to the top of the journalistic food chain. If he hadn't, his sources would not have spoken with him. See also: why can't all students with decent grades get into Harvard, why can't all soccer teams starting with my child's under-8 squad win the World Cup, why Warren Buffet returns Barack Obama's phone calls but not yours, etc. People at the top of one hierarchy want access to the tops of external hierarchies they deal with.
Facebook Raises $1.5 Billion, Now Worth $50 Billion
They're using thousands of disposable email adresses that they will never check again, it won't make a difference.
Yea, how many tech journalists know how source code control systems works?
I don't mean to derail the thread, and the content of this news is great. But to respond to your post, that's only true in the loosest, vaguest sense of the word communism.

First of all, it's only true in digital start-ups, because once you make the product you can copy it for free forever. In any other start-up you're still paying someone to do the other work.

In every start-up you're also counting on the idea of "property rights", or the right to keep the money you earn, the right to your software, etc. You're still looking to make a profit that's more than the work you actually did, or else you wouldn't be working. In communism, you would be thanked for your contribution and still make as much as a doctor, at most, because that would be "fair". I could go on...
PS - A more limited but equally handy thing would be suggestions for tools to graph individual websites. For example, imagine visualizing HN or the comments on an individual HN thread in tree (or other?) form without the text, though allowing access to the text via clicking on individual nodes. I'm interested in studying the structure and temporal development of the thread in particular.
Why is is that every MongoDB article on HN gets to the font page with 25-odd points but almost no comments? I see them several times a week.
XMLRPC for Pharo
This is so true! It happens to on nearly every tech article that the author just pushed the buttons and turned a three paragraphs info in a murder-tech-blog-murder-looking-article.
The bad thing about it is that you can't more out of those articles as you could from those "original" three paragraphs.

It's not that article size or letter count matters to all of us. I just want to get informed the right things. No long history telling, cause mostly I know it cause I hang around in this area everyday and for more I have Wikipedia. No long opinions and suggestions, cause most time I don't care or I disagree. No long mambo-cambo which let's me ask "Wait. What did I just read".

More of those short, quality and informing articles please.
@ Matt, wondering if you can give any insight on how the new algo will affect syndicated news sites? Will it grade them as duplicate content and devalue them? And if so, will google news be held to the same crucible of truth?
ready.js now with offline compiler
Google's Plan to Combat Search Spam
I don't see how seeing this is incompatible with the revelation that Apple sells experience.
We're flattered, Addy.
http://eng.wealthfront.com/2010/10/jquery-right-way.html