skysight-inc/HackerNews huggingface dataset:
I only know your side of the story, so I know this assessment won't be entirely accurate. However, based on what you have written, it sounds like you were a pretty good manager who just got caught between your employees and your bosses. I especially like how you recognize your team members' intelligence, and tried to advocate for them with senior management. You likely made some mistakes along the way, but every manager on earth makes many mistakes.

Sometimes even the greatest skippers can't keep a boat from sinking. It's helpful to consider how you could have made the situation better or worse. But, based on what I've read, you're putting too much blame on your own shoulders. Rather, remember that it took three people for them to replace you.
http://mjg59.dreamwidth.org/31714.html
Windows 7 is a solid OS. Also, built my own rig and own a copy of Windows CS4. If Adobe started making a Linux version I'd switch to Ubuntu.
Used to use Win7, but upgraded to Crunchbang Linux :)

I'd dual boot if I could, but my primary computer is a laptop and it's too much effort.
So, this is like RunDeck (http://rundeck.org/), only hosted.
How common is 600% turnover in management?

I know turnover in engineering is usually high (if people are moving around every 18 months to 2 years)... but 600%?
I think you're making an assumption: that pknight is correctly understanding your point and disagreeing with it out of a sense of entitlement to a date. I think he is misunderstanding your point, partially due to the word "tutorial", which to me indicates something ephemeral (something I'd want to have a date on), or the word "content", which can be interpreted any way. By failing to appreciate the cause of his disagreement, you exacerbate it with true-but-misplaced statements like "I don't owe it to you".
Well, I work for a registrar. I know practically all the tricks around their silly rules at this point and was more wondering which particular set of tricks you used. :-)
Its amazing how well traffic gets rerouted now days
Side effect of JS setinterval, it doesn't work when you leave the tab. Made sure to put a limit on the buffer to make sure the page doesn't use too much memory.
I think human readability was something antirez really wanted, and having explicit line delimeters would make it easy to visually parse in a telnet session or building/debugging a client.
When a story is good and hasn't had significant attention yet, a small number of reposts is
ok. We want good stories to have multiple cracks at the bat.
Ask HN: designing for interactive querying of large-scale high dimensional data
Hello Folks,

I am looking to get some inputs on architectural design on data storage and processing for something new I am working on.

I have large scale datasets (read a few hundred TB) which are used to generate data to power analytics. For the most part the analytics are usually data aggregations along different dimensions which are computed in an offline fashion (think Hive/Pig/Hadoop) and then fed-into/stored in an RDBMS or an in-memory data-store. This allows for fairly low-latency/highly responsive 'online interactions' either via a web-page or an API.

However the new features that are being contemplated involves allowing users to slice and dice the data along various new dimensions. Previously since the dimensionality was low, the batch jobs would compute rollups of the cartesian products of various combinations of these dimensions. The 'online interactions' could then be reduced to a mere lookup operation. This no longer seems viable with increased dimensionality and high cardinality.

So to get to the question, does anyone have any suggestions on new/interesting data architectures/data models/solutions that would be suited for slicing and dicing large-scale datasets along different dimensions while keep interactivity high/latencies low.

Looking forward to hearing from the resident data gurus. TIA.

PS:

1) HN n00b here. Apologies if I am doing anything wrong.
2) In the comments, there is an example provided (here: https://news.ycombinator.com/item?id=7770551).
Aircraft operational environments tend to include far fewer navigational hazards (though drone proliferation could change this).

Aircraft final-approach zones are very tightly controlled. Any idiot, or even a dumb lump of wood, can float through the water.

Not all that long ago: rowing through a local harbor (a transport mode in which one faces sternwards and periodically checks over the shoulder for hazards) I managed to spot a large timber likely broken from a pier. Perhaps 0.3-0.6 meters across and 7-8 meters long, with an iron spike or bolt some 30 cm long protruding from one end. I passed only 3-4 meters from it. A week or so later another boat in our fleet struck a piece of driftwood and suffered a hull puncture (fortunately just above waterline).

Timber, lumber, telephone poles, and other hazards are frequent in water. They can be significant hazards not only for small craft but for larger vessels with fiberglass, wood, or even steel hulls. It's rare for an aircraft to encounter a tree at 10,000m.

Yes, human error is a significant factor in many disasters, but human response is also frequently a factor in surviving disasters. This is particularly the case where automated or navigational inputs turn out to be erroneous.

The Air France 447 incident is an interesting case study in this: a design flaw (pitot tube) lead to an operational obstruction (pitot-tube icing) resulting in loss of navigational data (airspeed), causing automated navigational systems to disengage (autopilot disconnect) and fly-by-wire systems to reconfigure to alternate-law mode, crew failures in supplying inputs, following loss-of-airspeed-indication procedures, and responding to flight-path deviation, lack of proper crew response to stall, attempted dual control of aircraft by both co-pilot and captain (Marc Dubois). Compounding all of this was the question of what instrumentation and alerts to trust (airspeed, stall, altitude). Something was lying, but what wasn't clear.

https://en.wikipedia.org/wiki/Air_France_Flight_447

And all of that in skies with inclement weather but no specific fixed or floating navigational hazards to worry about.
An evolution-based theory is that we are genetically programmed to ensure our offspring's survival. This means that whatever tools, knowledge and experience parents have picked up along the way are improved and applied to raising their children. Parents will therefore observe and learn from other parents and their own research for healthier foods, better education tracks, both leadership and teamwork development activities, and whatever else we are convinced will help.

That drive then gets co-mingled with the need for acceptance and competitiveness (also evolution-based). The soccer mom satisfies all those drives concurrently: Improved survival rate for offspring, need for acceptance by a group, competitiveness for resources.

Thanks to evolution, we are pretty much spring-loaded to drive ourselves crazy busy in our 21st century world.
Mac OSX, made the change 3 years and 2 laptops ago.
Example/Fabricated scenarios below

A simplistic view of the old scenario:
Assume something like Google Analytics that tracks different websites traffic data. Previously a user could filter various website stats(unique users, time spent by users on a site, number of pageviews by user et al), by gender, by age, by device. So for example, interact with the data using a filter that picks all males(3), that are in an age bucket(assume 5 different age buckets) and that use a specific device category(assume 15 different device catgories). Here the cartesian product becomes 3(gender) X 5(age buckets) X 15 which would make it a maximum of 225 data-points per day, per website that is tracked.

And now the new scenario:
Besides the number of dimensions going up i.e. more multipliers in the cartesian product above, the way a filter works is being tweaked as well. So previously where it was an AND across different dimensions(eg. all males AND 25-34 AND Windows Tablet users), now it is an AND between dimensions and an OR within a dimension(eg. all males AND [25-34 OR 35-44 OR 45-54] AND [Windows Tablet OR Windows Desktop] AND [Some fourth dimension option 1 OR Some fourth dimension option 2]... ). This makes it a combinatorial explosion i.e. its roughly 3! * 5! * 15! (even for the old scheme).
If you have a good hackerspace close by, ask some of the core people there for leads and ideas. You might be able to prototype stuff for cheap.
I'm not arguing against mobile, just... well... exactly what you say. It will never enter the pro market unless changes are made to the app and security models that allow a user to admin their own device. It won't matter how powerful mobile devices get, or if they gain the ability to attach to external monitors, mice, keyboards, etc. Nothing technical prevents that.
You had your dev team do data entry. No wonder they resented you.

Wouldn't it be cheaper and better for morale to hire a couple of temps to do the data entry?
What's the difference between your service and Hoodie, GoInstant and Firebase?
Yuppy – Python Programming for the Privileged Class
Yuppy is a small Python library that integrates seamlessly with your application to promote data integrity by supporting common object-oriented language features. It intends to provide fully integrated support for interfaces, abstract classes and methods, final classes and methods, and type hinting in a manner that preserves much of the dynamic nature of Python. Yuppy can improve the integrity of your data and the stability of your code without comprimising usability. It is easy to use and is intentionally designed to fit with the Python development culture, not circumvent it.
What really always rubbed me up the wrong way was approaching me to come up with the question "OK, how long then?". Because "managers" tend to ask this when neither scope nor current state of the project are visible, and are then getting offended when one points out to them that it's an impossible question. They then usually proceed to ask you to just make something, clearly demonstrating that they don't want to improve the disaster state of the project or care for your opinion in any shape or form, but instead prefer some randomly made up number.

So whenever I hear this question in that very particular tone, I already plan my exit strategy because I know it's going to be a train wreck.
Those answers are great, but they're also very high level and general.

One of the best pieces of advice, badly paraphrased below, I've heard from a military context.

"Any time you instruct a subordinate, you must be
prepared to deliver the same instruction every single
time they perform that action, and expect it to be
performed in that way until otherwise instructed."

This is a warning about micromanagement, flippant decisions and how to delegate. For example, if you tell someone off-hand not to bother you with X, be prepared to never be bothered with X again. If you tell someone how to shine their shoes, be prepared to tell them how to shine their shoes every single day.

Again, this is an a military context where orders flow downhill, but the same applies in other areas of business. An experienced manager knows where they need to set the boundaries within which their staff operate, with as much autonomy and initiative as possible. An inexperienced manager doesn't understand how to balance this equation.

PS if anyone has a better formulation of the above, please share =D
May be, but many (most?) carnivores first kill their prey before eating them. There are exceptions of course, like the Python.

Either way, it is horrible to watch.
I'm a long time Linux desktop user, since the days of Debian Slink and Windowmaker, but I dig OS X. It's slick. I wish there was a desktop on Linux that could strike that balance between features, usability and stability. That being said, I'm also cheap. If Mac's were more competitively priced, I would probably own one, but paying that much over market value for what essentially is a "pretty" PC that can run OS X has always been an issue for me. I just chug along on my aging but sturdy Thinkpad with Linux. I think Apple could change the game if they decoupled OS X from the hardware. Who would pay for Window 8 when you could just install OS X instead?
That's just one of the sources, though, isn't it?
GoPro has all the characteristics of a successful long-term brand. I recently bought one for my son. I could not find a better product in that category. Many of his friends have one (or more). It produces great video. It is simple to use. They are nearly indestructible.

Established camera makers don't seem to understand how to compete, despite the GoPro being a pretty simple camera made with an off-the-shelf camera SoC that comes with firmware for many of the key features.

It pleases me that the valuation is more than double that of Twitch. A glimmer of hope for life outdoors.
This is very interesting. Linking those that are suspicious of the MMR vaccine with people who used to think the world is flat, people who just stubbornly "don't want to be right," people who are dumb? Hmm, ok I'll bite.

The writer is acting like the debate has happened and is over, that the reader already knows this truth. This tone should drive suspicion of the actual goal of the writer. Or am I jumping to uneducated conclusions too?

As a reader I want to be educated but in this piece the writer doesn't tell me why the MMR vaccine is safe. I don't know if the MMR vaccine does cause autism but I'd like to at least hear evidence why it doesn't.
Reminds me of this article, which was also here on HN ages ago: http://opinionator.blogs.nytimes.com/2012/06/30/the-busy-tra...

"Busyness" is a social defense against other people making us do things we don't really want to do and a mechanism for coping with feeling guilty about things we know we should do, but don't. My intuition is that people have had to project themselves as busier and busier over the last couple decades as technology has made us more efficient in order for this strategy to continue working.

If you're always "so busy!", your boss is less likely to give you more work; your spouse is less likely to ask you to do more around the house; you're not going to feel as guilty about only seeing grandma once a year or refusing to help your friend move; you can feel a little better about yourself when a friend lands an awesome new job or plans a cool vacation and you haven't really done anything. When you haven't seen someone in a long time and they ask how things are, you can give them the impression that things must be even better than the last time you saw them, because you've clearly been so busy improving your lot in life. Well, that's the idea, at least.

After I read that article I decided to try to be more honest with myself and the people I know about how busy I really was (which is often "not very" - I, like the author, am the "laziest ambitious person I know."). It's a very hard habit to break, and it forces you to be more honest about the things you really want to spend your time on and the things you're avoiding.
We do have a lot more leisure, but it is distributed at the tails of our lifespan, the times of lowest productivity. We sure don't start full-time work at the age that kids on the farm do. And even a century ago, most kids outside of big cities did live on a farm. On the other side of a career, many of us will spend a decade or two in retirement and good health.

But inbetween... well, the typical young upper-middle-class couple (as usual, the real point of fascination here) has committed themselves to huge expenditures of interactive time with their kids. That's really where the time went.

Today, highly-educated parents bring their kids home from soccer or tae kwon do practice, pick up dinner on the way home and read to each of them for a school-mandated half-hour after helping them with homework. This is crazy different from a few generations ago. Ask even your grandparents how much time your parents spent in a "play pen" as toddlers, or roamed their neighborhood after school when they were older. Now there's a competitive cultural expectation that you need to invest in your children's development daily, so they become socially self-realized and not economic roadkill.
It would have been easier to select which OS's are not a part of my computer ecosystem.

Laptop is OS X(but dual-boot into Win7 as needed), work workstation is Win7, work servers are Linux, HTPC is a chomeOS (box), phone, tablet and 2nd HTPC are Android.

I'm a workaholic with ADD. I'm using any number of the above devices at the same time whether at the office or on the couch.
Organisms are not the only things that face evolutionary pressures. If bosses spend enough energy on trying to employ only the hardest workers, the end result is work hours increase until as many people quit over hours as join over compensation.
The "safety" seems to only apply to the drone ship, and not to other craft that will encounter these essentially blind moving vessels. I think that there already enough hazards on the sea without adding drone ships to the mix.

ALL vessels in the future may need AIS transponders.

http://en.wikipedia.org/wiki/Automatic_Identification_System
I still can't understand why so many hackers/developers use OS X. Where I work doing web development, Linux machines account for less than 10% of the workstations, and the vast majority use OS X. When I tried OS X at one point I found it horrible. The most irritating thing was the keyboard which could not be mapped in the "standard" PC way (i.e. Ctrl Super Alt in the bottom left). I could get some programs to respect custom key bindings, but others would not. Besides that, there is no standard package manager for OS X and nearly all of the built-in libraries are terribly out-of-date. Worse yet from a security/privacy/philosophical viewpoint, OS X is proprietary software. We have no idea what risks or hidden back-doors exist inside.

I've been using Linux for many years now and I'm still investing time daily sharpening my tools. It's an investment that continues to pay exponentially increasing rewards. Nearly all of the remote computing resources I use in my job are running Linux. That probably is true for most HN users. Is Linux really that much more difficult to use on the desktop? I find getting all my laptop hardware running smoothly is one of the easiest system administration tasks I face. And I only face that task once in a blue moon when I upgrade laptops.
Show HN: Can I Wear Shorts Today?
A real world application of some interesting ML concepts. You can check out the Git repo here: https://github.com/thatjpcsguy/shorts
I was probably a bit negative with that reply (though I do think "X is our best engineer, let's have them manage others to make them more like X!" is far too common).

I'm enjoying the new (now 2 year old) role but there are also negatives -- I have less visibility into the direction of the group/company and "stuff happens" that I would've known was coming at the old role. I'm definitely getting to stay hands on, and satisfying the "big picture" itch with involvement in architectural and project planning discussions.

I think if you're considering that sort of move just make sure you find the management challenges interesting and be willing to invest as much time and effort into getting good at the new job as you have at your "individual contributor" role. Have a good relationship with some existing managers and directors and talk honestly about the role with them if at all possible.

I don't regret the short move to management or the move back, and I'd consider either in the future. It's important to understand that they are usually almost completely different jobs though.
This is probably just me, but the biggest reason I use Linux is the package manager and repositories (Arch's package manager and wiki are amazing). Anything other than that, Windows or OS X, feels like it's missing something and I have to work around it.

People are saying they have to spend a lot of time configuring Linux but in my experience that's not the case (obviously once it's installed in the case of Arch, which for me is a worthwhile investment).

I do admire the polish of OS X's desktop though.
Can you explain this? Is it because you don't want your enterprise customers to see the pricing, or because the pricing will be dependent on the customer, or something else?
Unless the people writing legislation are the ones doing it so they never pass the bill to change term limits.
Facebook should go for it! Then, they would own Go Pro to capture stuff, and Oculus to VR-display the same stuff.
COGS. HERO3 and HERO3+ cameras have lower margins. They have the fancy Ambarella 4k chips in them and volumes are still relatively low so chip prices are high without the scale. COGS is already coming down though, dropped 10% in Q1. Though they don't break it out, they must make insane margins on the mounts and accessories.
Back around 81 or 82 I was a principal in a software that marketed a software implementation if IEEE 754 and associated libraries, developed by Rick James, formerly of CDC, and Bill Gibbons, formerly of HP. It had a small market among compiler writers because no hardware supported it other than the 8087. Even so it was faster than the native 8087, but dont hold my feet to the fire if my memory is faulty. It was a very hairy specification to implement, because it included everything, even the kitchen sink. It was the kind of floating point that an astrophysicist would love. (I had several of those for roommates way back when.) Even by the 90s, I dont think any hardware manufacturer had implemented it in all its glory. If someone told me that no one has done do yet, I would not be surprised.
Hi, I've been a lurker here for some time now. I rarely ever comment on stories because, well, I have better things to do. But now, I really need to broaden my perspective. I just don't get it.

*Disclaimer : I am young (23), broke but happy.

This past year, I spent four months in Asia travelling. I rock-climbed for a while, tried bungee, canyon swing, diving, trekking and the motorcycle. I was able to afford all that on my student's salary working and studying for my degree. I met amazing people and I'm just excited about everything since.

There's been a lot of discussion lately about the 80 hours work weeks and quite frankly, I just don't get it! I did work 66 hours work weeks this past fall for my honours thesis. I started getting anxiety attacks. It wasn't healthy.

I don't mean to make a point. I want to hear some. I want to understand how some of you do it, and, more importantly why. I keep thinking that "Hey I was able to have this experience with less than 20k a year. Imagine what I could do with the salary of a `real` job! (Or freelancing)"

Why do you do it?
The reason airplane food tastes bad.
I would be interested to see how they'll deal with pirates.
X.Org Server Commit Activity Continues Declining
> the dev team was the fallback for doing data entry for weeks on end

Wow, what? It sounds like you were doing the right things, but this is hard to comprehend. Wouldn't temps cost $12/hour for this? No wonder they were angry.
Microsoft is pushing really hard. I see commercials everywhere. In this other HN post, it says that Microsoft has lost $2 billion on the Surface.

https://news.ycombinator.com/item?id=7770452

They are certainly putting up a fight. Anything could happen in the future but I think getting excited about a million units shipped is a bit premature.
Good point, I will have to swap out some testing libs but should be possible.
Java actually is pretty easy to decompile and thus to reverse.
Unless the remote piloting can be handed off to the local expert who could be watching from the dock based control room.