skysight-inc/HackerNews huggingface dataset:
Traffic sites == affiliate sites. @ aebn they have an entire traffic department that does nothing but throw up traffic (or affiliate) sites and drive traffic to their theaters. Also the pure affiliate model has been in decline for some time now. I also personally own about 10 traffic sites and each one has appox 10 affiliate offers on it. the term "Traffic site" has nothing to do with how the user gets to the site e.g.: organic search vs clicking link on 3rd party page. It simply refers to the purpose of the page which is to drive traffic to another site.
I still don't understand why Google hasn't done the exact same test with a control variable. Run the exact same test again on a domain that isn't google.com.
Madeleine: In-memory Object Prevalence in Ruby
You can sign up and ask for help to sharpen your skills :) - you wont be listed to help anyone until you click 'help a fellow hacker'
Virtualization in the trenches with VMware, Part 1: Basics and benefits
Am I the only one getting tired of hearing Google whine about this? In my view it's simple, just do something about it or shut up.
Claiming that Bing is copying outlier results is still a fair claim that Bing is copying results, especially considering that outlier queries are hard in general for search engines to get right: http://portal.acm.org/citation.cfm?id=1277939
Great point.

In one of the earlier threads there was discussion of MSR being in a bubble, and this is a good example. Did the people publishing that paper realize it could look like copying Google's results? Almost certainly not; as a researcher, this would look to me like I was showing that the approach worked across the three biggest search engines. Researchers are supposed to do stuff like that! But in this other non-research context, with billions of dollars at stake and Microsoft's track record of being the evil empire, it can look very different ...

[disclaimer: I was in MSR from 1999-2005 and then did competitive strategy for MS in 2006-2007. so that influences my perspective.]
People will go to google and type babylist. (Or even search for babylist.com.)
$10.00 or $2.70, it's a moot point to the issue at hand. The baker has lost something.

If I have the same pattern of bits on my hard drive that a movie studio has on theirs, the movie studio has lost $0.00.

That's the fundamental difference between piracy and theft. If you want to assert that we, as a society, have made an agreement to limit people's freedom to configure their bits as they see fit in order to encourage content creation, that's fine. But make that argument, don't try to conflate configuring my hard drive in a specific way with depriving you of an object you own.
Ask HN: Inexperienced developer trying to get into serious development
I'm a recent graduate with an ok CS degree and some experience with crappy PHP jobs, which is all I can get so far. I'm pretty good with Java, alright with C, and I'm learning Python, and I know my theory too. Every day at work I feel like I'm not using my skillset, and my paycheck reflects that feeling.

I'd love to get some opinions and ideas on where I could go from here. I've tried (and failed) to get Java jobs through job websites, I suspect because I have zero commercial or EE experience.
Arabian Knightz Rap the Revolution, MideastTunes Burns the Track
Funny and I see your point. However, the catch is that treating them as merely documents rather than human beings and also thinking you could whip up a codable algorithm to rank them would be exactly the sort of mistake I'd most expect Google to make.

To give just the easiest and most recent example of how that approach breaks: the increasing deluge of content farms and SEO spam in Google results. Many folks consider that non-optimal but Google defends it because their data "tells them" it's optimal. Now picture that approach with people. Rank things by some measurement and only things that measure well by that metric will rank high, shutting you out of great opportunities. The false negative, etc. Related anti-pattern is the concept of Local Maxima.
A little offtopic but from my brief understanding of redis is that it manages it's own memory using LRU? Why not let the operating system handle this? It seems redundant to handle memory with LRU when Linux, etc use LRU or ARC or something similar.
That's a good point, I'm downgrading my opinion from "suspicious" to "curious".
Since you're doing a custom Rails app, may I point you at this:

https://github.com/ryanb/railscasts/blob/master/app/controll...

https://github.com/ryanb/railscasts/blob/master/app/views/ep...

That's all you'd need to do. Oh, and if you submit your feed to http://feedburner.com/, you can hand out their URL instead, and you'll get stats and stuff.
I was toying with doing a thesis in this area, focusing on player agency. Have a look Facade.
Detexify needs help
If this joint toolbar comes to fruition, which it never will, the crowd will decide the obvious winner for practically all queries. At that point what will be the difference between Bing and Google?
As a former classifieds phone operator, please note:

"ad free", not "adds free".

Please. I beg you.
Best for their customers in what time frame? They might get better results in the short term, but it's much less clear what the long term effects are.

What if it slows down innovation in search quality and moves effort into marketing, shiny baubles? After all, why bother spending resources on researching, implementing and evaluating the algorithms when your competitors will within a couple of months gain the most of the improvements, automatically with no effort.

You complain about search relevance sucking. Why are you so eager to support a practice that has a high chance of causing it to stagnate?
Spot on. It took Stanford roughly two years to move off their /8.
Unless the resume has a .gmail address on it. ;-)
> My take on this is quite the opposite. If MS thinks they're doing nothing wrong, they shouldn't stop. They should do what is best for their customers.

Does this mean anyone should feel free to ignore robots.txt if they think it will be better for their customers?
You know someone's a programmer when they even talk in recursion.
Many years ago I was working on a project that required development in Lisp, C and PostScript. If sort of agree that jumping from Lisp to PostScript and vice-versa was somehow easier that jumping from either into the infix world of C.
Your actual argument depends on the sense of the word "take" that you use.

(a) It is a well-established norm in any modern society that it's immoral to take (move) something without permission.

(b) It is not well-established -- or at least, pirates would like to say that it's controversial -- that it's immoral to take (copy) something without permission.

(c) It's even less well-established that it's immoral to take (purchase) something with permission and then crack the DRM or produce further copies to share with other people.

Nobody disagrees with you about (a). But you're extrapolating that view to (b) and (c), and that part of the argument seems to hinge upon an equivocation of the different senses of "take".

Content producers have a shaky argument because they want to use the "moving" sense of the word "take" to argue that (b) is immoral, while using something like the "copying" sense of the word "take" to argue that (c) is immoral. (If ownership of the movie was actually moved when you purchased it, it's none of their business what you do with the DVD you now own!) Don't make the same mistake of equivocation. This is not an argument that piracy is OK. I'm just trying to point out that there are good and bad ways to argue against piracy.
Google should do more testing. For example: repeat the same tests (different nonsense search terms and results just to be sure) where none of the data sent to/from the browser contains any google specific terminology. If the Bing toolbar isn't special-casing google then I think the test results should be the same.
what control are you suggesting? I read a suggestion elsewhere of using only Firefox for some queries and seeing if they show up, which is silly, of couse.

without a possible mechanism of action there's no point of using something as a control; I might as well wait to see if the files on my disabled usb drive show up on bing. they were "gaming the algorithms" because they suspected that the algorithms existed, and it wouldn't have worked if there was nothing to game.
Google Gets Record 75,000 Job Applications In One Week for 6,000 Openings
I don't think this is really a common pro-piracy argument. Most of the arguments I see stop at "it costs them nothing for me to copy it" and completely ignore the lost sale side of things. Those that address the lost sale aspect just say that they wouldn't have payed for it either way.

Personally, I sit somewhere in between. I consider piracy immoral but not as bad as actual theft.
I'm posting this on a throw away account, not my regular user name for obvious reasons.

I used to work for one of these baby companies. It was an interesting challenge coding wise due to the large volume of data they dealt with.

When I first joined what they didn't tell me is that all e-mails about miscarriages and deceased babies would be forwarded to my work inbox. It was then my job to add the flag to the user's account on the database, that way the marketing e-mails and so on would stop. Users could do this themselves, but often they wouldn't find the menu option to do so as it's hidden away, and it's harder to find when you're stressed, so instead they e-mailed us. What shocked me was the sheer volume of e-mails like this I had to deal with from understandably distraught users.

What's worse is that sometimes you'd get an e-mail from a mother who has given birth to a still-born baby or had a baby with SIDS, asking for things like a specialist photographer, or other service who can provide the sensitivity and care they require.

When you here statistics like this it's one thing, but to get e-mails from people, knowing each one relates to a real deceased infant every day it slowly wears you down.

That's one job that haunts me when I least expect it. A lot more people have miscarriages than I first expected.
Harvard(HBS) changes Course
Top Chef's Recipe for Gender Bias
Except everyone knows what they're paying, with $x discount.
How to Choose the Perfect Blue by HAL 9000
As a result of centralizing the servers, everything in the stadium now operates on one system and one network

Awesome! So you mean there is now one single Point of Failure?

Brilliant.
I can't wait til I turn 66 knowing that by my next birthday I'll become 11 again. ;)
Ok, so you're not writing "mysql_query()" in the middle of your templates, but you are letting them decide what data they need. Your view layer is pulling data down from the model layer. And the whole DOM-Object "intertwining" thing would worry me: how do you test it, for a start.
robots.txt. a more specific example might make for better discussion.
This is a good quick overview of RDF for programmers: http://rdfabout.com/quickintro.xpd
I'm working on a text indexing/retrieval program, like locate (http://www.openbsd.org/cgi-bin/man.cgi?query=locate) but for content and not just filenames, and with an index <=2% the size of the indexed data.

It's very nearly together (integrating individually working parts now), and is currently ~1,500 lines (according to sloccount).

Adding support for indexing Unicode text, more configuration, composite search queries (A and B near C and not D), etc. will no doubt make the source expand a bit, but it's still pretty small.

If you're interested in trying it out once it's ready, contact info is in my profile. I'm shooting for within a week or two for a beta vulgaris. (Requires Unix. ANSI C, strung together with sh and/or awk to avoid dependencies.)
You can increase your site's rankings by installing the Bing Toolbar amongst a group of people and have said group search google for your target keyword and click through on your result.

But you can just skip the toolbar altogether. Just go to Bing and do the search and click your site. Likewise, if you go to Google and do the same, you'll increase your ranking on Google. There is nothing new here, except if you install the toolbar and go to Google you can increase it on Bing also.

This is something people have been doing for a while.

Google is able and willing to manually screw around with search results in a seemingly easy manner.

Google can do this even if Bing stops this current practice. Just go to Bing, do a search, go the last page and click a link.

Fundamentally, with respect to fraud, nothing has really changed.
The technology throughout the 3 million square-foot facility is connected by 250 miles of fiber optic cable, 6 million feet of copper wiring and 70 wiring closets

Argh. Why the pointless adjacent use of two different measures of length? 250 miles of fiber and 1100 miles of copper wiring would have been more communicative, IMO.
So I tried that, while the UI looks nice, it certainly falls flat in the actual functioning component of the UI.

For example I have to click on the search button next to the greyed out text to enter anything, and for Seattle the first suggestion is the seaplan airport. Again in Toronto it suggested the city center airport, which while a nice airport only has mostly domestic flights and even then only one commercial airline. It also required me to enter a seperate flight back (when I tried clicking for a return date it suggested another one way flight from Seattle). That being said, it may work better in Russia, I noticed the prices returned were in rubbles.
I remember back when people hand-crafted directories for the web. They'd put a lot of work into updating their directory and then in just a couple of days an automated spider would look at their site and a search engine would use their link structure to point everybody directly to the best pages without having to go through the directory. The search engine had no idea of what evidence the directory used. Was it fair? Was it ethical? Was it legal?
No, I must of misrepresented my thoughts. I stated that that was one way, not the way. It seems that with the size of Google today, and their notoriously strict hiring guidelines; based upon my observations, Google hires high-profile developers or they are acquired through a startup.
I think software industry is slowly headed this way: that there will come a point when you cannot let a member of the public use something based on your code without you being a licensed SE.
Get your landing pages fixed for free (worth $1495)
...all to deliver an entertainment product. Amazing.
Yep that's me.
I've been working for Manwin for the past 3 years and a half. Worked on almost every projects they've done.
For the past 2 years, i've been involved specifically on the tube sites, Pornhub.com and other ones.
I am currently Lead Developer, and like my job, like others stated in this thread, it's a great occasion to work on high traffic websites, solving scalability issues, and a lot of interesting stuff.
Either bring a lawsuit, or be quiet. You aren't going to shame Microsoft into changing their current policy, and even if it does improve results by only 1/1000, as long as they are legally within their right to do it, why wouldn't they?

It's laughable to think this "controversy" is putting a black eye on to Bing in any way shape or form, except among perhaps the most technical and anal retentive among the population.
Because Google has a symbiotic relationship with the entities whose data it's grabbing. It uses that data to drive traffic or attention to the originators and owners of that data. Everyone wants Google to use their data to drive them traffic and those who do not use robots.txt. Bing is being parasitic to Google in this situation by generating no value for them.
You're right in that its not a startup. I'm just kind of having fun and playing up a few responses from the original thread. The million dollar question here is this: is it repeatable? If so, then we've got a startup.
يوتيوب العشق الممنوع
Set up some other dynamic sites (not indexed or linked anywhere) with random nonsense words in their URL strings, and a single link to some unrelated site. Click the link. Repeat, wait two weeks.

Same as what google did, just not on google.com.

Of course, if Bing has other "relevance" indicators (ala BingPageRank) then this might not work because it will rule the nonsense control site irrelevant linkspam, and not put it in (even Google's experiment only got a 9% success rate.) It would be better if someone like Facebook injected the test nonsense strings in their URLs.

Something of this sort would at least show they tried to differentiate between "scrapes all clickstream data" and "uses Google's search results".
At that point what will be the difference between Bing and Google?

Speed of innovation. It's not our job to make it easy for them to have diffentiation. I just want great results. The more parity we give them with respect to data, the harder they'll have work to innovate in order to differentiate. AFAICT, that benefits us the users.
Well, as I said before -- there are lots of caveats. But I hope that professor loses his bid to introduce another sound-bite into the statistics world, because it's not correct to blindly say that the data are "worthless" simply because they're voluntary responses. We make useful inferences from voluntary response data all the time (c.f. every political poll ever conducted, where people can choose not to respond). The correct question is: does the voluntary response process result in a systematic bias that will affect the conclusion in question? But that's neither short, nor catchy.

In this case, there's a strong argument to be made that the samples could skew young (if for no other reason than because younger people frequent sites like HN more than older people). It's also quite possible that the demographics for HN and for slashdot are different in a way that would make the HN audience skew younger. But the voluntary response process itself? I think it's a stretch to claim that it'll cause a measurable bias in age distributions, across two different surveys, spaced a decade apart. You'd have to argue that younger people in 2011 are more likely to answer a poll than younger people in 1998 were to answer a poll. Possible? Sure. The most likely source of error here? Nah.

In other words, if you're going to pick (and you should!), pick on the BIG, OBVIOUS differences between the data sets...not the little ones.
I have an HP 35s which I found to be a lovely calculator which got me started with RPN. Unfortunately, I wasn't able to use it at school for my final exams since they had a list of specifically approved calculators (which the 35s would never have made since it's programmable). I switched to the Casio fx-82AU Plus which graphically displays your calculation, meaning you don't have to keep track of the inside-to-out expansion of RPN or the mountains of parentheses you usually require in algebraic mode. They're both good systems, but I can see the graphical display being more useable for most people (despite being awfully slow).
> He said there is a potential for violation if DOT
> and the public were misled by "engineering-quality work"

Could be a problem here. If I make a complete balls-up of a project would the public be misled into thinking that the government had done it - even if I wasn't a government employee?
I wish HN would give me the option of bubbling up old threads that got new comments.

And you are right about my example with the deep stack. I probably did not remember correctly. The variable array was probably something with passing around an address plus length on the stack---while Lispers prefer that as one object.
My question is... did Google install the Bing toolbar and then performing these searches on Google and click the links? If not, is Bing suggesting that users searched for those convoluted strings, in IE, with the Bing toolbar, on Google's website, and then clicked those links?

I hope this doesn't sound hyperbolic, that is really my understanding of the issue. Sorry, I'm not a big SEO guy, it's very possible my understanding or comprehension here is just flawed.
Hey, I'm new to Django, can someone explain what bcrypt is and why it is important?
Easy to catch and filter. But yes, in theory, this will improve results. The proportion of users clicking the 10th link on a page is much lower than those who click the 1st. If users clicking the 10th > 1st, then there is an assumed ranking problem. So clicks help fix ranking.
This is true. If you plan to keep doing this, then suddenly, it is a startup.
The post keeps getting downvoted... I'm not sure why. All the older people I know have very small monitors and considering the state of resolution across computers today, and the width of that website with that huge font, I feel like my assertion is rather valid.
"Google created an artificial scenario where the ONLY input was Google search behavior and lo, the search results are exactly the same."

Cutts addresses this:

"As we said in our blog post, the whole reason we ran this test was because we thought this practice was happening for lots and lots of different queries, not simply rare queries."
Those other entities can opt-out of being indexed.
So, Bing's Copying Off Google: What Now?
That's not how the AFP felt about Google News when it launched:
http://en.wikipedia.org/wiki/Google_News#News_agencies

or how some publishers felt about Google Books when it launched:
http://en.wikipedia.org/wiki/Google_Books#Copyright_infringe...
If you can make it more powerful and still as simple to use as ExtJS grid, I'd consider paying, but only if the cells could use the same kind of fantastic templating and CSS options that you can in ExtJS.
What is being data mined is a bit more than a user broadcasting their own preference on the correct result. The user is broadcasting a URL which is selected based on two factors:

- the user's preference

- Google's ranking algorithm.

Had Google not ranked that URL, the user wouldn't be broadcasting it. If there was some way to extract the factor of the user's preference of URLs as a signal without the factor of Google's ranking algorithm, you would be more spot on. Unfortunately this isn't possible. Also, in most cases the ranking algorithm component of this broadcasted data is the more useful factor of the two.
Well put. Further, I think Google has made a mistake drawing such a line in the sand here, and potentially missing out on a cool feature.
The AFP could have opted out if they wanted to. Copyright and fair use laws are not something we should be dragging into this. Your examples have nothing to do with the argument at hand between Google and Bing.
Anyone know how big nginx is? I wouldnt at all be surprised if its under 2k LOC.
Brilliant essay on how to find interesting problems to work on
I was considering the case when you go to NetFlix, and they recomment the movie "Social Network" to you, and you click on it. Surely the url will have sufficient info to claim that its a recommendation.

Now bing implements a movie recommendation service, which also magically recommends for you "Social Network", based on clicktracking of Netflix.

IMHO, what Bing is doing with Google is exactly the same scenario as above (extract click data from a competitor's service, and surface the exact same data in their own product).
Agreed! I shouldn't need an M.D. to stitch you up!
If Google think they have other evidence, let's see it. The burden of proof is on the accuser, I think.

(c/p from other thread.)