skysight-inc/HackerNews huggingface dataset:
A lean algorithm will comfortably execute every tick even in a busy market. Of course if it can't be processed before the next tick comes in, subsequent executions will run late, though we leave this up to the trader and hope to just provide the most performant execution environment we can.

We find no issues running scripts of reasonably high complexity. Of course if you have a particularly heavy algorithm it may be better suited to running at interval close.

With regard to legal position, you're absolutely right. While we're in beta you can connect to live broker accounts, though we are blocking order requests from actually being sent. We want to be water tight both technically and legally before we full launch with regulatory authorization, lifting this restriction and allowing real money to be risked.

As with brokers, we just want to support as many as we can. FXCM were a natural first choice for integration as they have excellent support for API solutions and have the most fully featured FIX implementation we've come across. Of course it's up to the trader to establish if any conflict of interest scenario exists between themselves and the broker.
An App Store for Algorithms
The mapquest one is actually useable. Almost as good as google.
Inside a Chinese Bitcoin Mine
Do the BSc + premed and evaluate the medical thing more deeply in your third year--or graduate, work for a couple years, and then decide. Work can provide a lot of perspective that's difficult to get otherwise (assuming you're still in high school). It's tough to plan your life that far in advance; you might develop more specific interests as time goes on (plus expectations of what the industry will look like may change over the next few years too). Even if your plans change considerably, EE + premed is a solid background, so transitioning to a different field wouldn't be a huge pain.

1. Automation will have a big impact on medicine, but it will be an ongoing process, not something that happens overnight. If you're doing a joint MD/PhD(EE) program, then you'll presumably at least have some technical qualifications that would make you more appealing to a company that is working towards the automation of healthcare.

2. You might be able to start a business on the side, but not as a resident unless residencies become immensely less stressful and time-consuming. Part-time work as an oncologist may be hard to negotiate early in your medical career, but I am not a doctor (though I did investigate similar questions a few years back, and this was the impression I got from speaking with doctors and MD/PhDs). Financial feasibility... as an MD/PhD, you shouldn't have much in the way of personal debt (since the PhD covers the MD tuition as well), and I wouldn't expect you to be dipping too close to the poverty line.

Realistically, it depends a lot on whether you have a specific business idea in mind and what your drive looks like. Those aren't things that you can plan very well 20 years in advance.
How to Attach CD-ROM and Install ISO for AWS EC2 Instances
The worst feature of the Apple TV remote is that you lose them so easily (because they're tiny). Amazon Fire TV's RF remote (which is somewhat larger) has the same problem, and it's far harder/more expensive to replace.

Note that the AppleTV can be programmed to recognize almost any remote (and adds extra buttons). I agree completely about the IR receiver and also the Apps (especially if you have more than one device -- switching devices is a total pain).
Stats on OpenStreetMap data growth in response to ebola
I'm working on a startup that has web scraping at its core. The vision is a bit larger and includes fusing data from various sources in a probabilistic way (e.g. the same people, products, or companies found on different sides with ambiguous names and information. This is based on the research I've doen at uni). However, I found that there are no web crawling frameworks out there that allow for large-scale and continuous crawling of changing data. So the first step has become to actually write such a system myself, and perhaps even open source it.

In terms of use cases, here are some I've come across:

- Product pricing data: Many companies collect pricing data from e-commerce sites. Latency and temporal trends are important here. Believe it or not, there are still profitable companies out there that hire people to manually scrape websites and input data into a database.

- Various analyses based on job listing data: Similar to what you do by looking at which websites contain certain widgets, you can start understanding job listing (using NLP) to find out which technologies are used by which companies. Several startups doing this. Great data for bizdev and sales. You can also use job data to understand technology hiring trends, understand the long-term strategies of competitor's, or us them as a signal for the health of a company.

- News data + NLP: Crawling news data and understanding facts mentioned in news (using Natural Language Processing) in real-time is used in many industries. Finance, M&A, etc.

- People data: Crawl public LinkedIn and Twitter profiles to understand when people are switching jobs/careers, etc.

- Real-estate data: Understand pricing trends and merge information from similar listings found on various real estate listing websites.

- Merging signals and information from different sources: For example, crawl company websites, Crunchbase, news articles related to the company, LinkedIn profile's of employees and combine all the information found in various source to arrive at meaningful structured representation. Not limited to companies, you can probably think of other use cases.

In general, I think there is a lot of untapped potential and useful data in combining the capabilities of large-scale web scraping, Natural Language Processing, and information fusion / entity resolution.

Getting changing data with low latency (and exposing it as a stream) is still very difficult, and there are lots of interesting use cases as well.

Hope this helps. Also, feel free to send me an email (in my profile) if you want to have a chat or exchange more ideas. Seems like we're working on similar things.
No, Google will display the language you normally look at. Also if I go to china, i see the chinese writing as well as english. OSM its just chinese. HOWEVER, when someone with their language set to Mandarin, they can view US cities and see it in mandarin, or even arabic cities.

and for the record, yes, google maps is way better. Not only is the data more accurate, it uses consistent and easy to differentiate symbology.
Isn't NAT anti-economics? Sure, it helps reuse a limited set of IP addresses, but it also introduces unnecessary layers which doesn't seem cost-effective beyond a certain scale.
Such a pity the audio and video quality is so mediocre. Python (David Sanders Julia presentation at iPython conference) did it much better. :)
That's been tried before. App.net was that way. The downside is it would have to be a paid service, because how else would you fund something like that?
Subsidies, taxes and regulation. Public awareness campaigns education and so on. All of these can change the way people approach a problem like this.

The beginning of course is articles like this one.
Have double-standards? We can, and we do. Does that make it right? Most would say no, including me.

It's actual quite hypocritical, if you think about it. Putting a double-standard in place to fix a previous double-standard. But, I suppose, with the "power" of the state, most people think they can play social-architect and fix all the world's ills...from their armchair. Standards being only one "weapon" in their arsenal.
New Facebook messanger has only one star on the app store
Would love it if you posted a blog article sometime about the technical details.
if you could publish a price list for items sold at major grocery chains, I am sure that many people could use it (bonus if it includes aisle numbers)
What about https://www.kimonolabs.com/ ? Makes it pretty easy to collect data and presents in a structured format (JSON).
Have you thought at all about the ethics of meddling in people's lives? [1]

This "service" seems contrary to basic notions of privacy and freedom.

[1] http://www.davidhume.org/texts/suis.html
I do a lot of this kind of mapping too. You can drag and drop a gpx file onto iD and it will show on the map to make tracing easy.
The head states that the network is reliable, but then goes on to list lots of cases where the network fails.
How unfortunate. http://en.wikipedia.org/wiki/Prolapse
A quick search reveals:

https://play.google.com/store/apps/details?id=com.auto.like....

https://addons.mozilla.org/en-us/firefox/addon/autolikefbsta...
Yes the greatest advantage SquareUp has is that they already have a list of clients that they can advertise this to. Clients that might be using their SquareUp credit card processing system.
This could potentially increase the 'cost-to-company' for each employee. Also, it could well be that an employee is worth a lot to Infosys, Cognizant, or other off-shoring companies who make up a bulk of the H1B applications, and they may be willing to spend enough to be the highest bidder.
> Also does everyone else think it's creepy when your friends stop using facebook and old "likes" pop up on your feed to make it look like that user hasn't left?

A friend of mine died last year. Her FB account is still active. From time to time I get old "likes" of hers. It is a bit creepy.
Ask HN: Community for pair programming/tutoring?
I am looking for a site for one-on-one tutoring (paid) or free pair programming on certain projects. Maybe this is a good startup idea but I don't have the time/knowledge to implement it.

Basically I have a bunch of projects I would like to do (or figure out how feasible they would be) but am having a hard time getting started. I am willing and want to do most of the programming myself but there are some aspects that I am completely clueless about and being able to discuss the project with more experienced developers would be really helpful.

The pair programming could be project based or just open channels about different languages or project types (native, web, plugin, etc.)

Does anything like this exist or does anyone have good resources for getting started?
Just wanted to throw more weight behind this comment. This book is excellent and is super easy to follow. It's one of those rare technical books that actually does an amazing job explaining things.
From Zero to App in less than 5 minutes
Microsoft technology turns shaky videos into smooth time lapses
Appointment reminder doesnt seem much like a competition as they do more than just reminders. They also setup a site and make it easy for people to select a time that is free.
Against “contrib”
Maybe the title shouldn't propose a question that is controversial (and obviously click-bait), and then have the bulk of the article be about something completely different.

You're right, I should have read more of the article to realize that it's basically about nothing (someone looking for, and fixing bugs in a library he uses, which everyone who cares does).
Thank you for your reply. Would you be so kind to elaborate on what you do for a living as well as in what capacity you work with MDs? Frey and Osborne (2013) published a paper that indicates how likely certain professions are to computerization. You can find it here:

http://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Futu....

However they didn't elaborate on the various medical specialties. Do you happen to have any information on that?

From what I hear MD/PhDs do research about 10-20% of their time, while they spend the remainder on treating patients. I'm afraid very few hospitals/practices would be willing to let me work only 3 days a week so I can work on a tech company on the side. I'd think maybe I could do some consulting but that's it; I don't think there would be time to pursue outright entrepreneurship.
The "local" drivers don't certainly all live in the Hamptons either. I worked at an ISP doing tech support while I was in high school, and one of the other guys also drove a cab in the Hamptons. This was 30-50 miles west of the Hamptons; yet still solidly Long Island (50 miles east of Manhattan) not the city.
I don't think this is a biological imperative at all, rather I think it's just that experienced parents worry less. The first child teaches them that they can mess up a bit and the kid will turn out just fine. The second kid teaches that even more. Each successive kid gets less obsessive care not for biological reasons, but because the parents have become more confident in their parenting and more aware of how indestructible little kids can be (obviously not literally, but humans are quite resilient).
You'll see quite a bit more traffic if you get over 3,000 votes. Seeing the 3k or the 4k threshold is a quality signal for reddit's more casual browsers.
The most confusing part to me is that the 'fast paths' in the API are not obvious, especially if you need to support older GL versions (GL2, GLES2). For instance:

- Using Vertex Array Objects should be fast since it's only a single call to change the vertex data binding. But in many GL implementations they are slower then using the old way calling VertexAttribPointer and EnableVertexAttribArray

- What's the fastest way to update dynamic vertex data? Buffer orphaning? Doing double buffering yourself? BufferData or BufferSubData? It's likely that the answer is different dependent on driver or GPU

Resource binding can introduce subtle errors by screwing up the GL state (for instance to create and initialize a resource you need to 'make it current', should the creation function restore the previous binding, or keep the GL state 'dirty'?)

Some call parameters are overloaded with completely different meaning depending on GL state. For instance the 'const void* pointer' argument in VertexAttribPointer is actually an integer offset and not a pointer if an array buffer is bound.

The most important problem which the OpenGL NG initiative tries to solve is validation overhead. In current OpenGL, an app can change every state at every time in the frame, and after each change (or rather before each draw call) the entire GL state must be validated. Most games (or 3D apps in general) only create a relatively small set of 'pre-compiled' state at startup and then just switch between these state presets. With this pattern, all expensive validation can happen once at startup, and not during the render loop.
Forgot to answer the questions directly and editing does not work:

1) No, definitely not.

2) Yes, lot's of research minded doctors/phd's already do. From producing transplant transfusion fluids to pedicle screw insertion simulators or acetabular cup reconstruction prosthesis after hemipelvectomies. During residency its almost impossible though.
I love Nest's response where they suggested buying a Dropcam to fix the problem. That's hilarious.
What It Will Take to Win Google's Million-Dollar Electric Power Prize
> A profit-maximizing company is only going to hire a H1B if they are cheaper than a US citizen per unit work.

Not true. What if the company can't find a qualified candidate? Then what they pay will likely be higher than the average worker in that industry. That's how salaries go up.
Interesting. Effectively it's base62 encoding with a fixed salt. I think saying "encrypt" and "decrypt" is sort of misleading, even though you call the non-cryptographic nature of the hashing in the README files.

For my own products I do something a little different. Instead, when I create a record I generate a random number (with Ruby's SecureRandom module) and store the base32 encoding in the database. With the universe set at 1bil, this reliably generates a random 6 character string that I can safely show to the customer.

Edit: base62, not base82
The head quotes a famous list and then goes on to list where the original quote was proven right.
I regularly scrape financial data - historical prices, live quotes, company information, quarterly reports, etc...
Thanks a lot for the answer! :D
They seem to auction off spectrum pretty efficiently.
For some businesses, that's going to be paid for in the first day or two if it's able to reduce no-shows.
I like Sausalito (pop 8,000), but I've come to think it should be covered in high-density housing, with a BART stop. Same for the vast areas of the Penninsula owned by Stanford. If you drive from AT&T park southward, keeping close to the water, you will see an enormous amount of unused land. Drive through Palo Alto and you'll see McMansions with huge lawns where there should be apartment buildings. Drive up San Bruno Mountain and look around, and you'll see entire swathes of Visitacion Valley that are just empty wasteland. Colma is an entire city made of graveyards.

What's missing is any kind of regional government capable of coming up with a housing and transit plan and finding the money to fund it. The result is no new homes in an area that is seeing a huge influx of workers, most of them in tech. Since those workers earn a lot of money, they price out other people.

We know how to build out our cities, but for some reason that is politically impossible in the Bay Area. It's a pretty despicable abrogation of responsibility.
Same here. I decided to remove myself from FB altogether (inspired by 99 days of freedom).

It's been 3 weeks so far[1] and it's amazing how after the first day Facebook started sending me two emails per day, along the lines of "you are missing important posts from your friends" and "you have 8 pending notifications". It instantly reminded me of Zynga's tactics to try to suck you into a game and paying for content.

[1] I made ONE exception- my close friend was missing for 24 hrs, left his wallet and phone at home, his parents freaked out so I reached out to all his friends on Facebook. This tells me I only need one feature Facebook has to offer - access to my own social network. Somehow we conditioned ourselves to only share Facebook information but not phone numbers or emails or any other means of communication. If Facebook suddenly went away I would lose 3/4 of my contacts. That's a scary thought.
I think the problem isn't the conversion of rent controlled apartments, it's that there are rent controlled apartments.

Rent control artificially limits supply and drives up prices for everyone else. This is one of the few things economists can agree on regardless of political affiliation. This isn't Airbnb's fault.
I've had three primary uses of web scraping. The hard part for me has never been speed. Getting the results structured is somewhere between easy and hideously complicated.

1. Reformatting and content archival (lag times of hours to days are no prob).

As an example, I put together a site to archive comments of a ridiculously prolific commenter on a site I follow. I needed the content of his comments, as well as the tree structure to shake out all the irrelevant comments leaving only the necessary context. Real time isn't an issue. Up until recently it ran on a weekly cron job. Now it's daily.

2. Aggregating and structuring data from disparate sources (real time can make you money).

I work in real estate. Leasing websites are shitty and the information companies are expensive and also kinda shitty. Where possible we scrape the websites for building availability but a lot of time that data is buried in PDFs. For a lot of business domains, being able to scrape data in a structured way from PDFs would be killer if you could do it! I guarantee the industries chollida1 mentioned want the hell out of this too. We enter the PDFs manually. :(

Updates go in monthly cycles, timeliness isn't a huge issue. Lag times of ~3-5 business days are just fine especially for the things that need to be manually entered.

This is exactly the sort of scraping that Pricenomics is doing [1]. They charge $2k/site/month. Hopefully y'all are making that much.

3. Bespoke, one shot versions of #2.

One shot data imports, typically to initially populate a database. I've done a ton of these and I hate them. An example is a farmer's market project I worked on. We got our hands on a shitty national database of farmers markets, I ended up writing a custom parser that worked in ~85% of cases and we manually cleaned up the rest. The thing that sucks about one shot scrape jobs from bad sources is that it almost always means manual cleanup. It's just not worth it to write code that works 100% when it will only be used once.

Make any part of structuring scraped data easier and you guys are awesome!

[1] http://priceonomics.com/data-services/
Yes, that's the point of the H-1B visa? It's supposed to be used for people with rare and unusual skillsets, the kind of people who command very high salaries; not for offshoring.

The H-1B is a non-immigrant visa in the United States... [that]
allows U.S. employers to temporarily employ foreign workers in
specialty occupations.

The regulations define a "specialty occupation" as requiring
theoretical and practical application of a body of highly
specialized knowledge in a field of human endeavor including
but not limited to biotechnology, chemistry, architecture, engineering[...]
I work as a research analyst for a Canadian provincial opposition party. Most government data is in terrible HTML tables, often dynamically generated, and almost none of it is in an easily machine readable format. I spend a lot of time downloading PDF files of data and converting them to JSON formats.

I have two main recurring scrapes:

- political donations. Every donation to a political party in my province above ~$300 is posted publicly on a gov't website (in a PDF). I use the data to run machine learning algorithms to predict who is most likely to want to donate to my party.

- public service expenses. My province has a "sunshine list" which publishes the salaries and contracts for all senior government officials. We grab it weekly (as once someone quits the gov't, their data disappears).

One tool that you could consider building is an easily accessible expense website, where people can enter the name of a public official and see all their expenses, including a summary of the total amount spent. There have been a number of massive expenses here in Canada related to this [1, 2].

[1] http://news.nationalpost.com/tag/alison-redford/
[2] http://en.wikipedia.org/wiki/Canadian_Senate_expenses_scanda...
Theoretically you could design a tiered system with a separate quota allocated for companies with <10 employees to bid for (or some other easily enforceable definition of start-up or small business).
Scaling with Go: YouTube's Vitess
Economics' guidance, like evolution by mutation and natural selection, is a random walk with a lot of local optima.
There's been a campaign aimed at the new permissions they ask for, especially recording audio without asking. People protest by installing the app, leaving a 1-star review, then uninstalling it.
You did not address the first part of his comment,

> You think it would be wonderful because of the way that you felt, not the way your friend felt.

Why not?
Mac+ARM: potential perf implications
The only thing the service does is alert the friends of someone who appears to be about to end their own life, but who, for some reason, is posting about it on a public forum.

I can assure you that the service does no meddling whatsoever.

Someone who is swept by the depths of depression, and takes a last moment to emerge long enough to ask their friends for help -- well, I think that it is only fair to counterbalance the feed algorithms that try to keep the rest of us blissfully ignorant, and let them have the support that they ask for.
meh...if you could do an experiment by dropping a foreigner - that has never been to North America - right into a Canadian city and then asked what country they are in... I would suspect a high percentage would say the U.S. The differences between the countries are more noticeable to Americans/Canadians.

Of course.. the U.S. has far more cultural influence worldwide so most foreigners hear and see more about it than Canada.