skysight-inc/HackerNews huggingface dataset:
I do a lot of scraping for my day job. We have a business intelligence team that will build us reports that we need from the data that we have. However I find that this process is so incredibly slow and sometimes we only need to compile the data for a one-off project. I used to use vb.net for this as that's what I started learning programming with. Now I use python/requests/bs4 for all my scraping scripts.

I've started working on a new website that will use data scraped from several vbulletin forums. I've found that even 2 vbulletin forums running the same version may have completely different html to work with. I'm assuming that it's the templates they are using that changes it so much.

I'm setting up the process so that the webscraping happens from different locations than the server were the site is hosted. The scraping scripts upload to the webserver via an api I've built for this. Mostly did this because for now I'm just using a free pythonanywhere account and their firewall would block all of this without a paid account. And then also none of these sites would see the scraping traffic coming from my website, etc...
Yes, if only this existed for turing, we could have thrown him in jail for not wanting to live in our bigoted society.

You hit the nail on the head, most people want a tool like this so every now and then they can stop being a shitty person for a few days before they go back to the navel gazing: "No one had talked to him in months, we had no idea he was depressed."
True. But maybe if you create an O(n^4) algorithm, and the site has an O(n log n) algorithm, then it could take far less time sending it over. But that's all conjecture..
>Then why not limit H1Bs to US companies (HQ in the US and not subsidiaries of international companies) only. Solves your stated problem of how the H1B system is currently misused without favoring big corporations over startup companies.

Because I'm not favoring big corporations. I'm favoring workers.
Oh they're definitely migrating away from it, because it was pretty terrible. Whether it was terrible because it was a bad implementation, or because it was a bad idea, or because Unity is an editing tool and its users expect a more "retained mode" approach that enables them to build UIs without writing code, is an interesting discussion, but Unity is through with immediate mode UIs, and trying to bury it. Their new stuff is very nice, and leverages many things about Unity's editor and graphical capabilities, but it's certainly not minimal.

http://blogs.unity3d.com/2014/05/28/overview-of-the-new-ui-s...
Feedback:

You have to get many more screen shots showing actual use on your website. I'm not seeing any. Hard to say much w/o seeing that. Can't even pass it along to anyone who I might think is interested without more of that.
I guess AirBNB is also responsible for the vast number of high-paying jobs in the SF vicinity and the utter lack of housing development.[2]

[1] https://news.ycombinator.com/item?id=8120070

[2] http://www.newgeography.com/content/003899-plan-bay-area-tel...
Agreed, it's not Facebook's job to retain evidence for all their users' lawsuits.
I actually created something really similar, except with more options to fine tune the encoding/decoding process:

https://github.com/mikecao/hashkit
The article mentions that Amsterdam has recently allowed short-term rental. I live in Amsterdam. While it's true that this law exists, it was created to set rules surrounding the practice. It doesn't exactly encourage it, in fact the law limits short term rental:

only allowed by the main occupant
less than 2 months per year
you must pay tourist tax
you must have permission from home ownership association
your house must be rated safe by the fire department
maximum 4 people
no disturbances of any kind

The list (in Dutch): http://www.amsterdam.nl/wonen-leefomgeving/wonen/informatie/...

In practice hardly anyone complies with the full list. All rent-controlled apartments are excluded from short term rental anyway, as are most free-market rental apartments.

edit: removed unrelated rant :)
This looks like a useful tool.

Assuming for the moment that this is very accurate.

What then?

Ann discovers that Bob is suicidal. What is Ann supposed to do then? (My suggestions are to ask Bob if he intends to die; and to help Bob access medical help. (Emergency help if he says he intends to die soon)).

Perhaps some set of accurate, international, flowcharts showing what you're supposed to do if a colleague / friend / relative / etc is suicidal would help. (In UK: Contact their GP; or an ambulance if suicide is in progress;)

You also seem to be ignoring ages, which is tricky. What do your ethics team say about reporting suicidality and people under 18?
That's indeed what we're using, we built everything internally, including our mobile API.
I'm pretty sure they kick you off the uber system anywhere if your rating goes below 4.7
Even light bulbs deserve a well-designed landingpage
A lot of those homeowners are tech millionnaires
Right now we support SSL out of the box for *.aerobaticapp.com domains. SSL for custom domains is on the roadmap.
Having worked with FIX, I feel for you.
i would pay to have that on a video player fast forward feature.
Thanks for the feedback, we're definitely going to add some more screenshots. In the mean time we've updated our landing page with a short introduction video (which was also already posted in the comments here).
How to Build Customer Feedback into Your Development Cycle
I just looked up Dayboard's name to share with a friend. It's surprisingly nowhere to be seen on the page, which means you're doing your marketing right - from the ground roots.

I really appreciate this app even as a single user. I do not get the benefits of the network effect, but it's amazingly focused. Thank you for making the free portions of it - I'm more than happy to donate as well if you have a link.
This is why I oppose the luxury shuttles. If you want to work in Palo Alto, Sausalito, or Mountain View, I think you should live there.

That would require those cities allowing enough housing development to support the growth.
Is the app synced with the user's calendar, to know when the meeting ends? That would be useful.
More on generating responsiveness, as I understand it.

Natural gas plants can be "spun up" in about 15 minutes. This is because they run as directly-fired turbines, so there's no time required to build up a head of steam.

Standby diesel generators can be fired up even more quickly (a couple of minutes), though these are expensive and somewhat less efficient.

Hydroelectric power also has an activation response measured in single-digit minutes, or even seconds -- open a sluice gate, synchronize to the grid, and you're on. A fair amount of California's hydro power is used specifically to match against current demand and larger blocks of power bought from elsewhere.

Geothermal is also fairly responsive, though it's got a limited total output. For California, The Geysers (among the largest geothermal generating plants in the world) has a total capacity of 725 MW (this is down from the peak largely due to groundwater exhaustion, with various re-injection schemes proposed and partially adopted).

http://www.calpine.com/media/Geysers_Factsheet.pdf

Coal (virtually unused in California) and nuclear (one plant, Diablo Canyon) have very long ramp-up periods, but can be operated as "spinning reserve" by wasting their unused thermal output. This is particularly expensive for
Sure, if time wasn't a factor. However, if you were to like green first, the algorithm might show you this "destroy non-green" community before giving you the opportunity to like other colours.

Similarly, purely out of chance you might be given the opportunity to like the "destroy non-green" community before the "destroy non-blue" community.

Maybe what we can take from the fact that the writer ended up shown/liking both left and right views is that the algorithm doesn't give a very strong weighting to negative correlation; i.e. even though people who like left-wing pages (presumedly) don't like as many right-wing pages; facebook still gives users the opportunity to like both. And possibly will not necessarily bias on one side of apparently conflicting interests when choosing what to present in the news feed.
> use them as forgotten password hashes

Please don't.

The author tries to trick you into believing it is secure by including a salt. However the resulting key space of the "hash" gives you the same security as a 5-6 character alphanumeric password. A motivated attacker could enumerate all possibilities in a few hours.
JavaScript Idiosyncrasies
Wouldn't surprise me at all! Happy customer since 2012....
I wonder how much this thing costs? It's missing a special sauce dispenser. I'll admit my stomach did grumble when looking at the gallery. I appreciated a toasted bun.
If time allows i would recommend attending a hackathon and diving into the groups primary skill set be it rails, php, swift, etc.
I think the focus on immediate, actionable feedback is great, but I get really nervous when I see screenshots with graphs. Does peer to peer feedback happen often enough to merit such an analytical approach? Getting continuous feedback is a great goal, but it strikes me as a human (or process) problem, not a technical one.
The creators already have the option of making their algorithms open source.
Just because it's using a web service doesn't mean the algorithm itself isn't running efficiently on hardware.

I imagine you ship off the data, let them run the algorithm, and they return the results. This has no effect on the Big-O efficiency of the algorithm; the only additional cost is a simple roundtrip of the data.

I'm failing to see how that's so different from ex. running an intense algorithm in a worker thread.
ML would be a huge step forward, this just matches a couple keywords.
Legality of scraping is a subtle issue - I wrote up my take on it here: https://blog.scraperwiki.com/2012/04/is-scraping-legal/
Not exactly sure what you mean by dynamic user profiles routes. With Angular or Ember routing and HTML5 pushState the app developer can create whatever routes they like. Please feel free to follow up with us orville@aerobatic.io
An Australian Startup Is Offering One Lucky Intern A $100,000 Job
Mobile friendly design as well!
I don't really care about super high performance - I need a small NUC that I can cool passively with reasonable performance for typical business stuff I do on the desktop. I understand the difference between mobile i5 vs i7 is almost none even on Haswell (both are dual-core with hyperthreading unlike desktop versions). I already have a 17W IvyBridge i7-3517U in one of my notebooks so I know what I am talking about. I also can't wait for 19W Kaveri for another tiny desktop setup...

EDIT: i7 details
If it is encryption and decryption why is it called 'hash'?
Dan Savage coined the term "gold star pedophile" for these types of people (http://www.thestranger.com/seattle/SavageLove?oid=12927907). I'm really glad that we (as a society) are beginning to discuss this problem rather than sweep it under the rug by locking people up. It truly is a mental health issue, and we need to ask the question of whether or not we want to lock people up for trying to do the right thing by seeking proper treatment.
> alert the friends of someone who appears

That's meddling all right.
I agree with you that rent control fails in all the corrosive ways you describe.
This issue you're running into is why I think trying to think about free will in that context is a category error. The concept doesn't make sense when it's something that depends on randomness or determinism. Instead, I think it's useful to think about how we use it in actual practice, which is generally something like, "is a person more or less self-directed" where self-directed means that their actions are consistent with their observed character and expressed desires. That's not particularly satisfying if you're raised in a culture that puts a lot of value on free will as a concept, but I think it may be the best we can do.
You can share vertices between primitives if all vertex components are the same and indexing is used.

That's the same on all 3D APIs though. I think the only exception was the Wii where you could have positions, texcoords, normals, etc... in different arrays of different sizes and index them with multiple indices per vertex. I can't image that this was efficient though...
Joined Twitter. Followed 1,000 people. Got suspended. Doing better now
Not having used Twitter before, I thought I’d give it a try to promote a new app that I’ve been working on called Lysterr. A colleague said that I should look into one of those automated bots to get more followers. I thought about it and figured that a similar result could be had by spending a few hours following people manually. So, this is how I spend my entire Friday night. Things were great and I quickly had 40, then 50 then 60 followers. I went to sleep that night having followed over 1,000 and felt sort of weird about the whole experience. I mean, I just followed anyone and everyone who showed up under suggestions. The next morning, in addition to my inbox being blown up, I saw a message from Twitter saying that my account had been suspended. This seemed odd since while setting up the Twitter account, the service prompted me to follow as many people as possible. I get it though. I was clearly overdoing it. So, I clicked off their check boxes saying that I would be good from now on and was back in business within the hour. I’m definitely going to respect their terms and conditions moving forward. Besides, I spent my entire Saturday unfollowing the ‘interesting’ and somewhat questionable accounts I had followed the night before. Now all is well and I feel like I can move on with life as a responsible Tweeter.

Thanks Twitter and sorry that I started off so aggressively.

If you are curious about our app and what we do, check us out at www.lysterr.com or follow us on Twitter @LysterrApp
I scrape the Hacker News website to get links and numbers of points. These allow me to produce my "top 10" lists.
Show HN: Stellar-backed Twitter upvote bot
Here I thought someone was dispelling the myth that you could exchange a Intel processor with an ARM one and get "like super low energy consumption", and look, "it has the same GHz and much more cores!", and have the same or better performance.

But of course this article instead repeats that flawed idea, right down to the GHz numbers. As it turns out, no, you won't magically beat the leading semiconductor company in the world at their own game using 1/5th of the power.

(In fact, today Intel presented details on their upcoming 14nm process:

http://www.anandtech.com/show/8367/intels-14nm-technology-in...

This is how you advance power and performance simultaneously)
I feel like this is a technical solution for a bigger people problem: managers not communicating effectively with their subordinates.

Soon you won't ever have to talk to your boss! Thanks computers!

That being said, I think this is a good product to sell to companies that choose to ignore the real "problem".
NODE.JS Developer, Realtime Platform Engineering at UBER
UBER's Realtime Engineering team is looking for a talented Node.js developers to build, scale, and support our mission critical infrastructure. Apply here: https://www.uber.com/jobs/4807

Our small team of engineers is responsible for the most complex systems at Uber with the most stringent uptime requirements.

Things you'll do:
-Design and build APIs to support a seamless mobile experience
-Evolve the core dispatching infrastructure that matches supply and demand and powers millions of trips a week
-Innovate in the areas of distributed computing, network programming, machine learning, and more
-Contribute to a realtime platform that is flexible, robust, performant, and scalable
-Become an expert of the Node.js ecosystem
The UVP for this should be: "The One Minute Manager" for socially awkward millennials
MIT professor risks his life to demonstrate the law of conservation of energy
what you ask is impossible. unless you do:

1. get the screen height
2. draw a block with ascii art with that height.
3. when done put height*line breaks to clean the screen.

or do you want something to handle termcap for you? `dialog` i think does that
If the city wants to target the poor, why not pay housing subsidies based on income?

If you force owners to rent to people at rates way under what they should be getting, you're just making the owners pay that subsidy. When those owners happen to own multiple units, and new ones open up...they pass those subsidies on to the new renters.
Golang patterns for serving on-demand, generated content
Show HN: Spotify Desktop Notifications in .NET
I still think we'll see a hybrid Mac at some point. You'll be able to run apps in either x64 or ARM mode, depending on what performance they require. Apps will be able to delegate tasks to run on the high-performance core or GPU if necessary, or just run entirely in Intel mode if they are a game. For a video editor or photoshop, you can get away with running the majority of the GUI parts on a low-end chip.
I scrape for Google Playstore for app data, top ranked apps, etc. Unlike iTunes, they don't have a public API/feed. So, in order to get the data I require, I scrape playstore on a pretty regular basis.
Ask HN: Solid Affordable Monitor for Office Work
I'm going to build a standing desk at home since my main desk will be occupied when both my wife and I have telework days. While my 42" Sharp LCD HDTV has been ok to use while working, the stand it's on is pretty low and it's uncomfortable sitting on the ground. My days of gaming are on the sharp decline so I really just need something that works well for editing documents and reading text on. Any suggestions are welcome. ($200 or less would be ideal, but open to more expensive if worthwhile).

Thanks!
You're storing 6-char base-32 values, so 1073741824 possibilities, but you've got 50-50 chance of collision after 38582 values - that's not a lot, the birthday paradox bites again! Using much larger random values or hashes gives you a better safety margin.
There are programs that make housing available to people based on an income test. My girlfriend's sister and husband (a schoolteacher) have a nice place that I (a filthy capitalist) could never afford on the open market.

Unfortunately it's the stories of people who don't need the help getting away with murder (financially speaking) that make the whole system seem arbitrary and unfair.
>I was thinking of a way to destroy my facebook account because you can't actually delete it

"Maybe that was true in the past, but today when you delete your data it is gone. Trust me, I wrote it myself. The law enforcement guidelines that have been circulating recently corroborate this."[0]

"If you don't think you'll use Facebook again, you can request to have your account permanently deleted. Please keep in mind that you won't be able to reactivate your account or retrieve anything you've added. Before you do this, you may want to download a copy of your info from Facebook. Then, if you'd like your account permanently deleted with no option for recovery, log into your account and let us know."[1]

[0] https://news.ycombinator.com/item?id=3320240

[1] https://www.facebook.com/help/224562897555674

https://www.facebook.com/help/delete_account
Unbiased Randomization with the Fisher-Yates Shuffle
Hockey news. Hockey news. Hockey news.
I'm sure they have a history of all your actions. They could easily revert your profile to a time before you went to town liking irrelevant stuff.
I just started getting into scaping (mostly been using import.io) mostly because it is a complement to what I really care about - data visualization. I've gotten a ton of interest for my side project and despite that I haven't opened the beta I'm still worried that it won't be as lucrative as creating some niche reporting services for various verticals (real estate, auto, etc). Essentially data that is very tabular and not hierarchical or qualitative. You can think of my work as pivot charts on crack. If someone already pre-compiled this data, I'd much rather pay for it that do it myself. My value add is the analysis/viz done on top of the data. If you want to chat, feel free to email me, contact info is in profile.
Agreed. Managers not getting feedback from their subordinates is also a problem.

Everyone has things they could improve in, and being frank about that is a hard personal problem which no tech can overcome.
I find it interesting that this article was yanked from Hacker News just as quickly as I saw it. I clicked on the link, wanted to go to the discussion and had to find it through search in spite of its high vote count and recent submission.
Nvidia  Tegra K1: First Mobile Chip With Hardware-Accelerated OpenCL
> this reliably generates a random 6 character string that I can safely show to the customer

Are you sure about that? You might want to consider a smaller alphabet, with vowels and ambiguous letters/numerals (0/O, 1/l/I) omitted. That has two advantages: you won't accidentally generate an identifier containing meaningful words (notably profanity), and the identifiers become easier to unambiguously transcribe and interpret. And you'll still reliably get a 6-character identifier as long as you have at least 32 symbols to use.
Linked article: [0]

I'm guessing the pieces were original works used just for the study, the article doesn't state what was used.

[0] http://news.stanford.edu/news/2006/may31/brainwave-053106.ht...
There are services that cover at least part of what you mentioned. These effectively provide you a tool to visually build a scraper and then they automate the scraping in the background, creating an API or spreadsheet of the data.

Import.io is one example, and I think there's another more recent YC-backed one. However, I tried using import.io a little while back but without much joy.
If You Only Do One Thing – Part 3
Is metal that much of a factor to OpenGL?

It's competition, but for a much much smaller piece of open gl's pie.

OpenGL code will run on billions of mobile devices, but metal will only run on iOS which is an ever-shrinking share of total smartphone sales each year.

Anyone who picks metal over opengl is picking iOS only--period. Any crossplatform game that doesn't want to write the entire game multiple times (once for apple, once for crossplatform, or heck onece for every platform) will simply not pick metal or any OS-dependent API.
I scrape Google to save search results.

http://www.labnol.org/internet/google-web-scraping/28450/
An App Like Yo Could Turn Out to Be Bigger Than Twitter
What It's Really Like To Have Your Startup Featured On Big News Outlets
My barber uses Appointy.

http://appointy.com/
We've come a long way from the time that the HP technical support person asked Steve Strassmann "Why would you ever need to point to something that you've drawn in 3D?"

http://www.art.net/~hopkins/Don/unix-haters/x-windows/disast...
Black Hat USA 2014 – Archives
I'm thinking back to some image processing processes I've implemented that had multiple algorithms. Passing images to a service like this back and forth would slow down those processes greatly. YMMV.
Does your friend pay hotel/occupancy or visitor taxes on the rentals? I'm guessing not, if they're opposed to following rental tenancy laws. I guess a whole new reason to dislike the "AirBnB effect" on housing in America.