skysight-inc/HackerNews huggingface dataset:
also -- room for improving solar energy generation - including proliferation of more electric appliances [electric central heat to replace gas/oil]

Boats with solar panels running on wind+electric [this is a fantasy of mine that I have drawings with, alas it is unlikely I have 10 million to build my dream boat, but that does not mean I cannot make sketches :-) ]
Just make them available to everybody.
HTTP/1.1 pipelining works beautifully for requesting web pages or other files in bulk from a single domain.

I would imagine "bots" like Googlebot make use of it.

I have been using it daily for 15 years with no problems. I mainly use netcat. This is how I digest large quantities of information as text not possible to do using a browser.

Most websites allow pipelining. Usual max is 100 requests per connection.

But there are rare exceptions where pipelining is disabled. For example, users cannot request pages of results for 100 different queries from Google in a single connection.

Efficient. But not allowed. Always wondered why.

Advertising-filled web pages opened in a "modern browser" are expected to auto-request files from many third party domains.
I will call these "conglomerate" pages for lack of a better term.

Not sure HTTP/1.1 pipelining works very well for that. Hence HTTP/2.

HTTP/1.1 benefits users like me. HTTP/2 benefits advertisers, Google, and other companies in the web advertising racket, but not sure how it would benefit users except to serve them advertising and conglomerate pages more efficiently.

As has been posted to HN before the number of outgoing connections that "modern browsers" make to third party servers upon loading a single webpage is staggering. At least, it looks staggering if you have been using the web since the early days, before web advertising grew to its current proportions.
Aspect-Oriented Software Association

On the "article" page, sixth bullet, follow the <Programming> Conference link[1]. On the conference page, find the copyright at the bottom, follow the AOSA link[2]. Finally, hover on AOSA in the first paragraph.

Finding this information should not have required such meandering.

1 - http://programming-conference.org/
2 - http://programming-conference.org/steering/
Just to add fuel to the RSS notion: there's been a minor exodus away from Twitter since the election, so for some of us that's no longer an option.
Electronic nose to detect fruit ripening
I'm probably gonna be downvoted for this, but what the heck. I watched the Warren Buffet documentary on HBO with my 11 year old daughter. Now, I've raised my daughter to have an entrepreneurial mindset and to always look at ways to add value to the world. The major points of the documentary that stood out to her were that he has lots of money. He owns pieces of companies who do well, and in turn he does well. His wife didn't feel that was an ideal way to live her life and left to provide value to actual people.

I didn't really know what to say, because she is right. Warren Buffet is good at picking companies who are undervalued and making money from that. That's a skill I guess. But do you want a country filled with Warren Buffets? Is there any reason to value this kind of parasitic success? The guy isn't really much different than the mafia or a loan shark. He loans money to companies he thinks will succeed, and wants interest paid back to him. At least Bill Gates provided value (and he was predatory in doing so) and his work changed the world. Aside from his eventual philanthropy, it's hard to see how Warren Buffet has changed the world.

Maybe I'm missing something. But how do you inspire a young person to be like Warren Buffet. And more importantly, should you?
I think the point is to arrive at answers without revealing how certain formulas (formulae?) are calculated? Or something?
As I understand it, Convox is closely adapted to AWS. Which, like any closely adapted software, can be great or terrible, according to circumstances.

Disclosure: I work for Pivotal on Cloud Foundry, a competing system.
I think this could be solved by having the [dupe] tag link back to the original. I've seen a few posts marked as dupe and it's sometimes difficult to trace down the original.
The trouble with the car analogy is that internet-connected devices need constant security updates, whereas cars don't. A car built ten years ago is as safe as it ever was, minus whatever problems arise from wear and tear. A device running iPhone OS 1.0 will probably not survive long on the internet of 2017.
It's not about encrypting files (encryption at rest): you can do the secure multi-party computations over your private LAN, and the encrypted network traffic will never leak to anybody (ie. no way to cryptanalyze anything in the future).

It's about encrypting the computation between the parties: this kind of technology is being used to prevent satellite collisions on secret orbits, for example.
Ask HN: Didn't want to be a manager, do you always have to go up?
My boss wanted to promote me last year to manage the team I started. I'm good at the technical parts of my job, I can communicate with lots of different types of people about it, and have set up the team for success. I can do people/project management and end up doing a lot of it by proxy, but I very much dislike it. It is not my core strength. I was offered to manage the team when it grew and said I'd rather not, so I was promoted to the parallel seniority on the IC track instead and we hired someone. The manager we hired left recently, and I was again pushed into a manager role. I relented. It was a mistake, and I hate the position I'm in now. I was perfectly happy has an IC and I didn't want more responsibility.

2 questions:

1) What is the best course of action to go back to being an IC? Do I need to leave my current company and how does that look?
2) Do you have to continually keep getting more responsibility? I think I provide (as per my stellar reviews) more than enough value as an IC, so why does everyone push towards something that is clearly not my interest or strength, where I just end up failing a lot more and am now much less value to the company.

"Rising to the level of your incompetency" comes to mind and how do you escape that?
Stack Overflow might not be the best comparison; it doesn't look like this app lets a user post questions or indeed interact with a community of any kind. It's more of a consolidation of resources designed to help immigrants.

That being said, this does look potentially useful.

A couple questions: who's curating the newsfeed? Where does the data in the FAQ section come from?
Ask HN: How to you advertise yourself as a freelance developer
Specifically if your not in a huge city like San Francisco.
"routinely hire people with a 6-months probationary period just to fire them the week before"

That's not due to labor law. That's due to the boss being a complete asshole.
Discovering New Ways to Containerize with CoreOS Rkt
Precisely this.

Things that aren't tested can pretty much be counted on to fail a non-trivial percentage of the time. If it's going to be business critical, then it needs to be tested. (Lesson hopefully learned here)

By the same token, things which aren't monitored can also be counted upon to fail a non-trivial percentage of the time and even worse, go unnoticed. (Another teachable point here)

...but people being what they are tend to learn these things the hard way unless we've got concrete, real-world incidents like this for them to learn from.
Several great points (normit is the basis for the Gaussian copula, which was used to great effect to model the CDOs (collateralised debt obligations) that blew up in the GFC (global financial crisis)); but it would have been possible to raise them while being less dismissive...
C2FO | Research Engineer | Seattle | ONSITE, Full time, Visa considered | https://c2fo.com

The C2FO lab is charged with developing tools for small businesses to help them win the fight against their own unpredictable cash flow using networks instead of traditional financial intermediaries, e.g., banks.

Today we work mainly in Go and JavaScript and put roughly equal emphasis on writing papers and writing software. Team members need to be able to define their own areas of interest, write their own supporting documents, and lead their own coding projects while welcoming help and input from others. Simultaneous with the largely independent effort is a need to demonstrate to stakeholders outside the team why your work is valuable.

Our interview process consists of an initial conference call to assess mutual interest, then one or more onsite interviews in Pioneer Square.

C2FO as an organization is headquartered in Kansas City and has some great investors on board including Mithril Capital Management, Temasek, and Union Square Ventures. We have a separate, larger engineering team that we collaborate with and that publishes open source at https://github.com/c2fo.

The position includes an excellent compensation package (commensurate with experience), equity ownership, Health Benefits, 401K and more.

If interested, please email pete@c2fo.com.

EDIT: added benefits blurb
Ask HN: Presentation – tracing visualization revealis thread affinity issue
I'm trying to find a presentation and accompanying paper. A group of Google engineers demonstrated thread tracing. It revealed latency outliers that were caused by the Linux kernel trying to keep threads scheduled on the same core. Some threads were stuck waiting on a core that was currently occupied (by disk i/o iirc) when others were available.
Yewno | FrontEnd, BackEnd | Redwood City, CA | ONSITE

Fresh off our Series A ($16.5M), we're looking to expand our engineering team with great talent. At Yewno we're moving beyond keyword search and allowing users to explore concepts derived from a vast wealth of vetted and trusted knowledge from the world's leading academic and financial publishers.

Along the way we're solving really cool problems in data storage, ingestion, analysis and visualization.
Our front end is a modern React/Redux/Babel/D3/Webpack stack and our backend runs on Python, Go & Nodejs.

Feel free to send me an email at {joey}{at}{yewno.com} for more details. I'm on the front end engineering team and I'd be happy to tell me you more about what we do.
It's easier said than done in some companies where stake holders are always pushing for new stuff.

Unless those who fund what you're doing understand why disaster recovery is vital, you're going to see this.

Ideally you want devops in such a state that you create new lower environments that mirror production, complete with state/backup restoration, that's run automatically every week.
The German Passivhaus standard just begs for factory prefab construction. I suspect a major factor is how one can reduce the cost of making the structure airtight.
San Francisco | Full-time | C++/C#/Xamarin

Genomics it's good stuff.
We made some tools that are better than what academics/gov/big companies make.

Desktop applications, not web or "cloud"-- b/c you make video games and movies with desktop applications, so maybe when lives depend on it skip the web browser.

Anyways you'll get paid market-rate and get to work with fun technologies.

You can email me: mo_r [at] me.com
time python -c "import ujson;
for i in xrange(0, 1000000):
json.loads('{\"data\": \"hello, world\"}')"

0.291s

Well, that was an easy way to reduce 7 seconds to 0.3.
this. just a simple rsync and docker-compose up can be all you need in most cases.
Given India's populations, seems like more potential for mobile journalism there
As a new (American) grad I briefly looked into working in Japan. The companies I was interested in were posting for Americans specifically, but they were still hiring people on as salarymen. The rates, even COLA-adjusted, were shockingly low. Like, top-flight companies offering bottom 10% salaries in American terms.

That whole experience has come to make way more sense as I've learned about the contractor divide, and the expectation that salaried workers will "start at the bottom".
Well I don't know what your friends, social or professional circle is, but it does happen.

What I describe is not out of a management textbook. This is really really elementary stuff if you want to sustain any serious operation.

Let me explain:
so you're saying a customer is on fire. Let's say that a query generated by that customer for some reason it creates a spike in your DB. Let's say that it almost DOSes your system. You know it doesn't happen locally, with your dummy/fixture data and it doesn't happen with other users/customers. So some particular configuration of state(your DB) is creating this problem. What are you gonna do my friend? Retry the query on production while you have a debugger on the webapp process? Please... be serious.

Remember, it's only once you have to setup this kind of infrastructure(and you can scale it up or down depending on your resources). And then you can do business like a normal person. What you describe is like throwing my sanity out of the window every time there's a severe(or not) support issue ticket. It doesn't scale!
Device-To-Device Communication in 5G Cellular Networks (2014)
The impression I got when I originally read this article was that:

1) Trump outsmarted Clinton (and the presumed technology advantage she inherited from the Obama campaign apparatus) with psychometric local targeted propaganda / communication

2) Some of that communication may have been deliberately targeted at discouraging democratic voters by putting negative articles about Clinton in their social media feed.

This is interesting in the context that Trump only won by a 70K voter advantage split over three states.
Reuters' position on covering Trump
If you have a fleet you might want to check out NixOS and Hydra. There is someone building nix pkgs for ARM/RPi not sure what is the current status... Just keep in mind that the learning curve is steep for NixOS, but the entire thing can be automated.

Broadly speaking if you want security you need to compile and distribute your own packages.
Is paying for you own cancer treatment morally defensible? You could be paying for someone else's cancer treatment.
Not a literal gun, but the requirement to have money to pay rent and buy food is a very symbolic gun.
did you have experience beforehand in using emacs? I've been using spacemacs for a couple of weeks, but I'm having trouble getting past the most basic usage, and partially it's because I feel like I don't have much experience with emacs.
This sort of thing is common in many countries around the world. When the state doesn't take care of a need the people will.
How planning a weekend trip changed our perception of Slack and WhatsApp
Is it possible to sensibly generalize the concept of median to "higher orders"?

E.g., I can imagine the difference between the 25th and 75th percentile to be descriptive of spread (like standard deviation), but those numbers seem arbitrary.
Amazingly easy distributed data processing
Using Datomic? Here’s How to Move from PostgreSQL to DynamoDB Local
By using C for the hot paths (e.g. HTTP parsing) and python for business logic you can get code that is both faster and easier to maintain than, say, Java.
If you use a C library and have hardly any Python code running it obviously goes faster. That misses the point I was making. At some point you'll have to write some Python code, and that will lower the number of requests dramatically. If you're talking millions of requests per second, you will have maybe 10s of thousands of CPU cycles per request. That doesn't buy you much in Python. Case in point: parsing a trivial json string in Python eats up your budget.

Of course, most websites aren't dealing with anywhere near that volume of requests, and then Python is fine.

Pypy is interesting, but last time I looked it didn't support C extensions and hence can't be used with the poster's library.
we use Nomad[0], we pretty much use Hashicorp's entire stack (consul, vault and nomad). Vault has been fabulous for secret(s), authentication, etc. Consul for service discovery and Nomad for job running/deployment. We have a mix of static binaries that we run and docker containers. Most of our new stuff is all docker containers. We use Jenkins as our CI/CD, that just run nomad jobs and confirm their successful deployment.

Cost management is easy, all the projects are open-source and since we can spin Nomad up against any cloud provider or internal machine hosts, depending on what's the cheapest at the time. It's pretty easy to wrap your head around Nomad and make it do what you need.

0: https://www.nomadproject.io/
What does it take to become a bestselling author? $3 and 5 minutes
I get the analogy, but it's a bit confusing since there sort of is a Stack Overflow for US immigration:

http://expatriates.stackexchange.com/questions/tagged/usa+im...

This seems more like a knowledge base for US Immigration (particular since it doesn't look like you can post your own questions).
That's precisely what's so nefarious about this widely accepted set of terms: your company will be fine with you loosely interpreting your employment contract, up and until you become successful.
Boston Dynamics “nightmare inducing” wheeled robot, presentation video close-up
Find the perfect brains to pick
The problem with that is when you start getting too many "all OK" messages.

Getting a message must be mostly a surprise. One daily message is OK for me, but if so it may be good to tell about more than a backup. A daily digest of "everything is up and running" may indeed be something very nice to have.

And how do you ensure a team will always have someone surprised not to get a daily message? Maybe sending for may people, or requiring some acknowledge, and creating a warning over the warning channel if there isn't one. (And you certainly aren't going to send those positive messages over the warning channel, because it requires a completely different kind of action.)
Matador Cloud (https://matador.cloud/) uses nixops to manage NixOS machines.
Bigleaf Networks | Beaverton, OR (Portland suburb) | ONSITE | http://www.bigleaf.net

Bigleaf is an SD-WAN platform that provides reliability and performance for Cloud applications over commodity broadband. We're a small team but we've got an established business with hundreds of paying mid-market customers, and we're growing quickly.

Our interview process entails some initial email discussions, 1-2 in-person or phone-based interviews (no crazy technical algorithm memorization tests), and often a brief (~1 hr) coding challenge for you to do from home.

We're hiring for the following technical roles right now:
* Front-end Developer
* Sr. Software Engineer (Linux networking focus)
* Network Operations Engineer
* Network Integration Engineer

Check out more details here: http://www.bigleaf.net/careers and feel free to email me (Founder and CEO) at joelm@bigleaf.net (no recruiters please).
That's a fascinating account, thanks. Certainly not the first place I've heard of that restructures to avoid horizontal or vertical integration, and then funnels everything like it was one company.
I work here (in solutions engineering and professional services) and it's the best job I've ever had. Strong trajectory, great management, smart peers. On top of that, we're solving a real problem: matching skills to jobs which -- just look at this thread -- is such an incredible hodgepodge right now.

Anyway, feel free to ask me any questions.
A million requests per second with Python.

Spoiler: technically, it's not "with Python".
https://cloud.docker.com/

Surprised it wasn't already in the long list of suggestions. Have been using the tool when it was called Tutum. The guys behind Docker bought Tutum and renamed it to Docker Cloud. It's currently set up to redeploy services when I push an image to my repositories. Really loving the simplicity, even tough it's got some quirks.

You can now link your bitbucket or github repositories. Let it build your containers and deploy it to production. This way you can build an easy CI/CD pipeline.
If you already have the job and they try to make you sign something like this, don't quit, but say no. You may not be prepared to quit over such a clause, but it's also likely that they're not prepared to fire you over it.

They will probably tell you that you must sign it, or else terrible consequences will ensue. If those consequences involve the company, brush it off, because that's not your problem. If they involve you, then see if they're actually willing and able to carry them out before you make a decision.

I've heard many stories of attempted unilateral changes in employment terms like this where employees refused to sign. In some cases, they got mildly harassed for a while and then it petered out. In some cases, nobody even followed up. In some cases, they were able to negotiate the language to everybody's satisfaction. I haven't heard of anybody getting fired for it. (I'm sure it has happened, it just doesn't seem all that common.)
Detecting Strict Aliasing Violations in the Wild
That was part of the intention of the second question. I don't have an answer for myself, unfortunately. I'm searching, of course.
ezhome | Fullstack & iOS/Mobile Engineers | Fulltime | Fully Remote

ezhome is a Series B funded home services startup founded by two serial entrepreneurs (founders of oDesk and PatternInsight) who are passionate about creating a global/remote workforce. We're building a company that will become "The Amazon of Home Services" and are looking for phenomenal talent to join our team.

Here's more about us: https://www.ezhome.com/about
Here's more about our remote teams: https://www.ezhome.com/careers/remote
Angel List: https://angel.co/ezhome

Apply here:
Senior Software Engineer - http://grnh.se/wev5f61
Senior iOS/Mobile Engineer - http://grnh.se/32sehk1

Or email Liz at Recruiting@ezhome.com
Malauzai | Front End Engineer | Austin | Onsite

Malauzai is a FinTech company providing mobile and internet banking solutions for community banks and credit unions. We are looking to hire a Front End Engineer to join our Web team.

Our team uses a fully Javascript stack: React, Redux, and Node. We encourage functional programming practices. We welcome people who can push the product forward and bring their own improvements and ideas to the table. We don't just want to build an internet banking solution, we want to build the best internet banking solution.

We are a high visibility team with a quickly growing product in a fast moving company - there is plenty of opportunity to make your mark.

Full job description - http://www.malauzai.com/front-end-web-engineer-scripting-foc...

Instructions for applying as well as other open positions (iOS, Android, Ruby) - http://www.malauzai.com/about-malauzai/careers/