skysight-inc/HackerNews huggingface dataset:
Yeah, I think you have a good understanding of what we do. It gives a lot of control over when, in which order and how you group your data needs, and allows you to "drop" promise objects in the output. You can create dependencies between promise objects too (what we call "datum" in my example), which creates a web of relationships between these objects that lives in parallel to the realm of their DOM placement.

A typical use case for that kind of dependency is advertisements on deviantART. We serve different ad inventory depending on whether or not there is mature content on the page. Our ad datum objects thus depend on all the image/thumbnail datum objects being resolved first. That dependency between datum object classes is defined in the code and guarantees that we never serve the wrong inventory.

Before datex we had to be really careful never to output an image after the ad code had been called. And of course we have a banner at the very top of the website, above any image in the DOM. That meant we had to use templating, output buffering and all sorts of convoluted ways to make sure that we generated the ad code after all images had been output, then inserted the ad back at the top of the page where it belonged.

Now we can just render the page in logical top-to-bottom order, merging the ad datum object into the datex stream at the very beginning of the page generation, then move on and merge image datums as they come. We know that the dependency will always be respected; the ad datums will see their data populated after all the image datums have been processed.
Google Adwords Tactics for Startups & Launches
The Mac Inventor's Gift Before Dying: An Immortal Design Lesson For His Son
I agree with you. Particularly, why not just stop using those clicks and reduce the negative coverage and perception of this seems like a very biased suggestion.

All of the negative perception seems to come from Google deliberately inserting outlier data. For MS to back down on that basis immediately makes them look like they were doing much more than what is proven.
WikiLeaks Cables Shine Light on ACTA History
You sir, are a machine. Steve started working on this 3 feet behind me a little less than 2 hours ago.
taking = gaining possession of [item]

taking doesn't have to be negative and it's not a word associated with either side of the discussion, it's just a word. I can take $10 from my mum to pay for dinner, or take a magazine from a shop (after paying for it), it's just the opposite of give. You can take with permission, that's why I stated take without.
Those examples to me are like finding a maggot in a piece of meat. Maybe that's the only one, and if you just remove it the rest is ok, but it's going to make you pretty suspicious especially if it already smells.

The reality is that it's almost impossible for google to measure how much their results influence bing. After all, they are both trying to rank the same internet. The best they can do is prove the obvious cases and let bing do the explaining.
I'm certain they never stopped trying to be innovative, but Bing is able to use that click stream data to reap the benefits of Google's innovations. Google is on the right side of this argument even if it makes them seem whiny to some people.
These things usually start quietly. Truth is MySpace has probably been shopped around for quite a while. They didn't find any takers so only now do they publicly announce it in the hope that someone, anyone steps up.

DST?? How about Saudi Prince Alwaleed bin Talal? He led the group that invested in Citibank and surely MySpace isn't that big a white elephant.
"voluntary response data are worthless"

There are ways of correcting for non-responders though. For example, there is the Monitoring The Future study that asks high school kids about their history of drug use. Almost everyone who is in school that day answers the questions, but that still leaves everyone who was absent that day. So what they do is they ask the kids the number of days of school they have missed so far this year, and then they correlate the number of missed days with drug usage. Using this information they can correct for every kid who was absent, because they know how many absentees there were and can figure out the distribution of how many of those kids have skipped one day of school so far, two days of school so far, etc. And then they can assign drug usage rates accordingly.
It's funny. The main reason I'd want to work at Google is to have access to their scale of infrastructure.

OTOH, they'd have to be insane to just give random new hires free access to their infrastructure.

So, if you've got some computing problems that would benefit from scale or data, maybe the best route is to get a proof-of-concept working on EC2, and then try to collaborate with Googlers.

This is distinct from the startup->Gacquisitionoogle route, since I'm more interested in projects than companies (except my on co.)
If I tell a designer to recreate Amazon.com's front page as a Photoshop PSD, they could probably do it in under an hour.

If I tell a designer to create the front page of a feature-packed, online store where every tiny decision has a significant impact on sales figures, and the process has to be run past (and work for) countless people within the organisation - no one is doing that in under an hour!

In this case, there's every chance that the actual decision making process was "resize logo or part of logo to fit a small square", but there's also a chance that other options were considered, a couple of meetings were held to get it approved, etc.
Great site and thanks for the background info. As a developer, I'm always in need of good design. I love the site design and the story of how you found the designer. I hadn't heard of Forrst. If anyone has a spare invite to kick my way, I applied for membership.
How is that telling, exactly?

The top 4 results (including one which has the domain name of the query) are the same but in a different order. The rest of the results appear totally different.

Even if one random query you ran returned an identical first 10 results, exactly what would that prove? Statistically significant how?
It took him two hours to monkey patch a few methods?
I'm planning to build a jQuery Grid, and have the following points in mind.

- Consistency: Same or better as ExtJs grid.

- Flexibility: You can change anything, and I'll provide a documentation about the code structure so you can hack in.

- Light-Weight: It'll be very small and just one file. Yes, one JavaScript file, no CSS or other JavaScript files. (+ jQuery library)

- Server Side Integration: Easy sever side integration for some language (like PHP, ASP.net, Ruby...) you just put a line to create a grid from a mySql table.

Any other points you think it'll make it succeed? Also how much would you pay for it? (at least give me a range)?
I agree...

So just between us, is this an indication that Bing uses Google strongly or doesn't use it strongly?
I wonder if you would be ok if Bing did the same thing to amazon. That is, imagine they used toolbar/IE logs to infer that people went to amazon, searched for LCD TVs and then purchased model X. Then they could boost pages about X in search, or implement a "bestselling" feature. After all, they are "just" using clickstream data here. Similarly, they could track Netflix, and so on.

IMHO, there's a loophole in Bing's argument that the user click data is free for them to use. What about the site - do they have no ability to consent/dissent to being tracked?

Maybe robots.txt needs to be updated for the toolbar.
Just watched the video. It's evident that Bing uses clicks as one of its signals. So a possible SEO tactic now would be to spam clicks on your link?
An interview with Sean Gourley, founder of Quid.com [video]
Take a look at some of the really cool stuff at the last few SciPy conferences:

http://conference.scipy.org/proceedings/SciPy2009/
http://conference.scipy.org/scipy2010/schedule.html

also, consider joining the mailing lists I listed above (numpy-discussion, scipy-user). Other smaller lists are: pystatsmodels, lists for Cython: www.cython.org and the associated Sage project.
How likely is it that completely different ranking algorithms would turn up the solarviews.com and ruthannzaroff.com results on the first page?
Yeah, I mean you could make the argument that it's just one piece of MS research among thousands of others, etc etc, but the fact that they've sunk resource into literally spying on Google (et al) is just too much. It could still turn out that Bing don't use that technique... but it's looking a lot less likely. Especially given this whole thing started based on Google noticing their spelling corrections popping up on Bing, and stealing Google's spelling corrections is exactly what this paper was about.
And that's a shame, i think we could learn a lot from Post Mortem posts of projects, especially if the project fails.
It's also a good habit for yourself, to look back at the project, and analyse it, the mistakes that were made, why it didn't work, what would you do differently if you were to start now.
When i was a freelancer, and doing my own little flash games, i was almost always do a small report for myself on why it did/didn't work, always a good process to avoid making the same mistakes again and again.
SQL formatting style
>gave them fair warning.

Fair warning they're going to put out a hit piece? Or a "We will put this out if you don't do X, Y and Z".

That sort of fair warning runs awfully close to "extortion" in the layman's use of the word.

I don't KNOW either party or the facts. One or the other could have acted horribly here. But "fair warning" with a demand for value is pretty good definition of extortion.

I'm not really excited about a business relationship with either side of this spat.
> The alleged violator clearly started with the conclusion, and put
> together some official looking research to support it

So he was biased for his own argument? As much as people try to be unbiased, I think that it's probably impossible for humans to be completely impartial. If the arguments in the recommendation are so weak/faulty then the DOT engineers should be able to rip it to shreds with a counter-argument. I'm having a hard time caring about the fact that the guy put together a 'professional-looking' analysis without having a professional license. He didn't represent himself as a professional engineer, and he was putting together information for to present his viewpoint to his local representatives.
I think you misunderstood the article, the point of that technique is to greatly reduce the amount of database calls. We don't "pepper the templates with database calls", to the contrary, we avoid doing them on the spot like a bad implementation would. We treat the data needs for these objects all at once, as late as possible (so we can group the data fetching needs of as many objects as possible), greatly reducing the amount of DB calls needed.

We don't scan the HMTL either, this system doesn't have templates, this is another point of the technique. The data structure holding the output contains both DOM and objects precisely to avoid a templating syntax, which would be costly to scan.

The tree that contains intertwined objects and DOM is only traversed once, when it's echoed. The data resolution is done in passes, but doesn't require traversing the "output" tree.
I don't want to get into a detailed argument here, but there are popular, consistent moral philosophies which hold that (1) physical property rights exist a priori to the state (Locke-style) and so are immoral to violate at all times, and (2) intellectual property rights are a social construction which only have moral import when defined by a legal framework and enforced by a reasonably just government.

In other words, many philosophers do draw a morally relevant distinction between these categories of rights.
A "good society" is a society where corporations can run a legal extortion racket by creating artificial scarcity of the cultural products of said society and take draconian measures to protect said racket based on outdated legislation that was originally intended to achieve exactly the opposite?

No, that's not a "good society". That is a sick and corrupt society.
Matt Cutts should know better. Being part of '1000 signals' does not mean all signals are weighted evenly. It does not even mean the signals are weighted the same across all query types. This is machine learning - the actual weighting is learned and dynamic (always shifting) and not controlled. And there is absolutely no reason for Microsoft to take out a particular signal just because Google asked. There needs to be proof of unethical behavior, of which there is none.

The Chrome and Gmail EULA's are as bad as the IE8 Suggested Sites bit mentioned in this article. GMail basically reads your email to provide contextually relevant ads. Does my mom know that? No.

Not a plug, but I blogged about this @ http://roshank.posterous.com/google-versus-bing-no-one-is-be... . I believe this should be a discussion on ethics - and feel it is ethical for a company to do whatever it wants with data contained in its own software application.
Can you design a tank? Well you could win $10,000
I left a few minutes after he started. He very well might've been banging his head against other (client-related) issues within those two hours.
I just went to check out InDinero and linked up my personal accounts to see how it worked. I have to say it was so nice, easy clean, etc. that I think I might start using it to monitor my personal accounts, it's 10x better than my banks online banking pages.
The Android 3.0 Fragments API
I agree-- and doing so could make some pretty serious waves across both academia and industry. You should join the discussion on the mailing lists: in particular pystatsmodels and the numpy/scipy lists. Any input would be much appreciated.
I'd start with the fact that they both have the keywords of the search string in the page title and in the URL string.

Look, I'm not saying it's impossible, I'm just saying that two different algorithms can yield the same results without one having used the other as an "indicator". To trot out a tired phrase, correlation does not imply causation (and I'd argue you don't even have correlation at this point.)
I like the honesty about employee salary, keep it up! CNN knows you guys are onto something!
I do like your contribution to the discussion. The the endearments did not add anything, though.
Absolutely on all of those sites. And Wikipedia. Why would I not want them to? The only reason I could think of that I would not want them to is if I thought it would create worse search links as a result.

Although being able to personallize the search queries would be incredible. So when I search for "Movie XYZ" -- it can also look at my clickstream and see that I spend a lot of time in Netflix and Netflix has that movie, and they can point me to that movie directly in my search results ("You can stream Movie XYZ now!") -- kind of like a built in Clicker -- I'd love it.

But the problem is search relevance sucks. I'm in my freakin web browser all day, yet it contributes nothing to my search relevance. WTF!?

But again, key to me is opt-in. If anything, the one takeaway I would take if I were MS is to make it more apparent. But besides that. Everything you said, and more, please. And from Google too -- Bing shouldn't be the only one to benefit.
Money spent on stuff like that can act as a deterrant for the future.. So it might be about break-even.
Meh. That is a surprisingly subjective and arguable statement, when you consider there are useful purposes for code besides running on your current dev system. For instance I frequently find code on the net that is rotten from the POV of "it runs on the latest git head of the dependencies" but completely and utterly fresh and relevant in the "how would I approach a solution in this language with this general design paradigm" POV. Other forms of "rotten" include software written to $HEAD, only to have those features never incorporated in an official release. The list goes on and the differences between "rotten" and "doesn't work for my special case" grow very indistinct.
My two favorite takeaways from this blog post are:

1) You can increase your site's rankings by installing the Bing Toolbar amongst a group of people and have said group search google for your target keyword and click through on your result.

2) Google is able and willing to manually screw around with search results in a seemingly easy manner.

As to the issue of Bing slurping up google's data for their own purposes: Scraping the web is a double edged sword. They aren't doing anything illegal. If anything, this whole ordeal has made both Search Engines look like a bunch of teenagers fighting over a who stole who's boyfriend.
You don't have RSS feed on your website?
Are there seriously 0-10 year old HN readers?
The design and marketing need quite a lot of work. The button on the landing screen should be shown as more of a screenshot of the project rather than what appears to be a demo of how it works. I clicked several of these buttons, including the browser buttons before realizing it wasn't interactive.

Then, while setting up my buttons there is a pop up when you rollover "edit" that reads "click here…" I never found out what it said after "click here," because I would try to click there when it popped up, then it would disappear. This is entirely confusing to have an edit button, then when they try to use it, you distract them with fluff.

Overal the interface needs to be treated with a similar concept as the business. Make it easier and dumb it down, don't try to add playfulness and uniqueness where it isn't needed.
I wrote EasyEXIF: http://code.google.com/p/easyexif/

About 120 lines of C++ [see 1] that parses basic EXIF information out of a JPEG image. I found all the other EXIF parsing tools and libraries a little too heavyweight for something as simple as getting the date and time a picture was taken, or the f/stop or exposure time. It only uses string.h for memcpy and memset, and no other headers.

[1] http://code.google.com/p/easyexif/source/browse/trunk/exif.c... (note: Google's source view screws up my whitespace)
Agreed. And his example of how his mom could not understand what the IE disclosure really meant in terms of information sharing is silly. Your typical mom would also does not understand the true extent of data collection from using Google's own products. Does that make it wrong?
No one is saying he isn't smart.
Trust comes from two things fundamentally: capacity and compassion.
If someone is smart but they've made a deal with folks who are trying to rob you, they are not trustworthy.

Jon just proved that the only thing that matters to (admittedly bright) Cramer is Cramer.

Viewers should have kicked his ass out by ignoring his show.
I always assumed SharpDevelop supported VS plugins.
In high school and college I used a HP48GX first and then a HP49G. Now I'm using Emacs Calc.
That's certainly possible, but I personally find sarcasm very difficult to detect on the web. I generally avoid it when communicating online and via text.
You've hit the nail right on the head. I've had the same idea and articulated some of the problems here: http://news.ycombinator.com/item?id=2176962.
Ron Conway / SV Angel's entire portfolio
Perhaps you can convert some customers from the snow-shirts?
It only cost 585 pounds to add a favicon to a government website?

I'm surprised it wasn't more.

Working wit the government is not cheap. It is not easy. It is not something that lends itself to lean development shops. That development work is supporting a bevy of sales and process people to deal with government stuff. 585 pounds is cheap
>"one of the many things you learn not to do in engineering school."

Practicing engineers, however, figure out how to do what the client wants within the limits of the law. I usually don't link to youtube videos, but this pretty much sums up the real world: http://www.youtube.com/watch?v=P9BUyWVg1xI
> Not sure twitter is the best medium for that, but hey, it's what happened..

I'd say that if you've got 1.3k followers, and the company you are slamming has 100k followers, Twitter is definitely not the best medium to take your fight public in.
> they are both trying to rank the same internet.

Yeah, this was a point I wanted to make in one of my blog posts. If Google wanted to claim some kind of ownership over SERPs, that would be ridiculous because you could easily argue that there is only one correct set of "ten links ordered from most relevant to least" for each query. Eventually all search engines will converge on the same ten results for every query.

I don't think Google ever made such a claim of ownership, but it was taken that way by some.
I totally agree with this. There's not really any good reason why Microsoft should piggyback off of Google's search results. The fact that they're able to observe Google's results by observing user clicks in a browser rather than by simply harvesting results directly off of Google's website confuses the matter, but ultimately it's irrelevant. This is just a sneaky way for Microsoft to observe a user's interaction with an information service (Google), to see what information the user obtained, and then offer the same information themselves.
The socks idea is interesting, but the blurb says the company estimates it will save the average customer 12 hours a year buying socks. Really? 12 hours per year?
Bravo.

This is, by far, the best treatment of the issue I've seen anywhere, on account of how it's the only level-headed one apart from Nate Silver's. There's a little needless inflation at the beginning with the screenshot comparison. It does have the obvious and expected slant. On the other hand, there's no enormous hyperbole or anything like that. There are no vacuous statements. The word "copying" appears a few times, but that's at least descriptive, and there's no use of other loaded terms like "cheating", "stealing", and "unethical".

Also, for what it's worth, it moved me from 80/20 certain that nothing fishy is going on to 50/50: the spell correction paper shows to a certainty that this has been considered. I'm sure Googlers know as well as I do that not everything in a research paper finds its way into the product, and this one in particular may have been nixed by management types for this very reason. It's still awfully suspicious.

I still think the original experiment completely fails to demonstrate anything unethical, and I still think the original info release was both hyperbolic and needlessly inflammatory. It does demonstrate a need for some more information, which seems to be all this post is asking for. If it had looked more like this post, I think the 'net could have been spared a lot of controversy. Maybe Matt Cutts should be writing these things, though far be it from me to decide that.
I would argue that users own their clickstream data and are free to give it away. (The question then becomes whether Microsoft makes sufficiently clear exactly which data users are giving to them.) Clickstream data is basically a list of URLs, I don’t think (parts of) such lists should controlled by website owners.

I’m not sure, though, whether I have overseen something that would make it useful or fair to give website owners control over clickstream data. I’m honestly open for suggestions.

But that’s only part of the problem. Even if everyone were to agree that collecting clickstream data from users with their consent is ok, we still don’t know how those who collect the data are allowed to use it. That’s certainly not an easy question.
I absolutely agree. I was merely pointing out that Pasco tweeting a screenshot that was essentially a threat does nothing to support his side of the story. All it shows is that he knew it was coming: He had received a fair amount of warning from Casantas. Not that I think it's fair to publicly air grievances in a manner that is frankly libelous.

Anyhow, I think this whole thing is tasteless and obnoxious.