skysight-inc/HackerNews huggingface dataset:
Couple of thoughts and questions:

- The difference between discredited and threatening is academic, since both lead to the same mindset and course of action

- A large swing towards Trump was noticeable amongst minority voters, the poor, and whites without a degree[0], groups who would be expected to be harmed by competition from illegal immigration

- A large swing towards Hillary was observable in the rich, retirees, and whites with college degrees, groups which are generally believed to be well-to-do

- If fairness feels like losing, and losing feels like losing, then how can we tell which is which?

- If white people implicitly pursue identity politics even when they try not to, will making them aware of this lead to them laying aside identity, or actively pursuing it?

- If race-blindness/"I don't see colour" is a myth, is it a useful myth for the purposes of causing people to get along?

- Is it possible for any action taken against a group that is winning to be unfair? If yes, when might this occur? If no, are you really sure that winning justifies literally any action against you?

Note that I'm not American, so I'm looking at this with an outsider's perspective.

[0] http://www.nytimes.com/interactive/2016/11/08/us/politics/el...
fear.
> I've been thinking about this space a lot

Me too, for better or for worse.

As for the issues, there are many. Just quickly a few:

* Data provider has an FTP server, most files are automatically generated, some are hand-named (with inconsistencies). How do you handle (without a lot of effort) a list of exceptions along with the regular files?

* Data provider has a good strict XML schema, but the relevant information for a single item is spread across three files, inside a tar archive. Since the there are 500k files inside the archive, you best not want to extract it, but process it on the fly.

* Data provider chooses layout that saves every item in a single XML file, inside 2-3 levels of directories. There are 20M of them. Unzipping the archive alone takes more than a day with default system settings and the usual tools. How do you process these things fast?

There are more subtle issues as well:

* FFFD regularly occurs in natural language strings. Can you correct these strings?

* File has .csv ending, looks like CSV on first glance, but all the standard RFC compliant parsers choke on it.

* XML file that elements, that have RTF tags embedded in it. You need to parse the RTF in the elements, because there is relevant information there, that you need to add to the transformed version.

* Date issues. Inconsistent formats and almost-valid dates.

* Combine data, coming from an API with data fetched from ten different servers to produce a transformed version with a legacy command line application (that might be slow, so you have to split your data first and parallelize the work, combine it and make sure it's complete).

I am thinking about a longer article or even a short book about these kind of data handling and quality questions and what ways there are to address them. Would you read a book like this and what topic would be the most pressing or relevant?
Ask HN: How many messaging apps do you use?
I use Slack, Intercom, WhatsApp, Facebook Messenger, Social Media DM's, Kik, WeChat, Hipchat, Telegram, Snapchat, Instagram Stories, and 5 different emails.

Total - 20+

What about you??
Thanks a lot - interesting read!
I broadly agree that Tesla may take the lead. That said, I know they've announced autopilot being capable of handling fog and various weather conditions and have separately demoed their full autonomy, but I haven't seen anything combining the two.
Noting a stereotype is a "cultural dig"? Why don't you take offense at the other stereotypes preceding those (ie "socially challenged, type-first-think-later hacker")?
Dell UltraSharp UP3218K: the first 8K monitor
Thanks for the additional context - though it's disappointing to hear. Seems the point of teaching this kind of thing should be to encourage people to explore their own interpretations. Having a fixed path for the student to travel seems counterproductive.
Uber however caters to a specific class of people and they don't seem to be the people who can only afford public transport. This may not be an issue in America, but i think if cities change their public transit planning to accomodate the class of people who can afford uber, they will be disadvantaging those who really need it.
Algorithm and Blues: Why Hillary’s Moneyball Strategy Failed
Although I think it is obvious. I mention it anyway. The founders fee that goes to the developers is meant to fund the development for the next four years.

If you don't like that, there is already the Zclassic fork wihout the founders fee.
I feel like people keep underestimating how difficult it is to build self-driving hardware and software. You can build a prototype in a couple of month with a small team of people as comma.ai showed, but getting from there to something that is at least as reliable as a human driver is still an open problem.

Building the car itself, creating a ridesharing network, finding parters, swaying consumer perception and even regulatory challenges are going to be walk in park in comparison.

Google/Waymo is well positioned to be the first to solve this problem, since they have been working on this for a decade and because Google has a big advantage in Machine Learning and Computer Vision research. On the other hand Tesla already has cars on the road collecting data.
I think I'll enjoy doing this. As a teacher with less free time, these tasks seem short and will keep my brain "puzzling" over code without the stress of a high priority project looming in the background.

So far I've got a time of around 1 second on the first challenge. It's nice to be able to practice my profiling and see how low I can get it!
Do you know why devs is no unresponsive on mailing list and gitgub? There are many pull request to fix bugs and they are simply ignoring them.
Software industry is driven by plenty of workers where work hours equal to output, maybe not development but then in the software industry in many cases development isn't the bulk of the workforce.

Sure everyone can be an engineer, I've seen companies that give a title of "customer services engineer" to their phone banks, but at the end support staff which unless you work at a startup can be as high as 80% of the work force does need to be present for as many hours as possible, even 24/7 in many cases.
> You don't need Adobe Cc on Linux

What? Even if you wanted to, you can't use Adobe CC on Linux. Doesn't run. So these aren't alternatives, they are your only options. It's not that "Linux gets no love", it's that Linux simply doesn't apply in this case, because nobody who's currently using Adobe CC is on Linux.
It's not part of the European union AND it's not a country.
"It's like today's social science. It's an extension of moral and political philosophy. But moral and political philosophy are fairly pointless and get bogged down by fairly meaningless arguments about the meaning of "self" or "altruism", while social science attempts to make people's lives better."

That's not an error in the subject, that's an error in how the field is being practiced. There is also plenty of social and political philosophy that had nothing to do with what you said bogs it down. Writing off an entire field for a few bad pieces would be like saying program design is useless because look at how many people use bad design.

Additionally, a field's effect on people's lives doesn't have anything to do with its level of complication, which was your original claim.

I see no evidence for the idea that any philosophy is simple in any way.

The idea that you are trying to flaunt your credentials downthread but really fail to address any points relating to most of what philosophy studies and the questions it asks (ones I am sure you don't have the answer to, which would seem odd for someone who finds the subject to be uncomplicated.) is why many commentators don't think you understand what philosophy about, myself included. You're taking an incredibly objective approach with a subject that isn't. If you're looking to say that philosophy does little to nothing for society, there's an argument to be had there and valid arguments on both sides. I think a decent number of philosophers would even agree. That still doesn't make the subject uncomplicated. Your original posts seem incredibly ignorant of the subject you studied.
>Colonel Alon was a decorated pilot who had flown dozens of missions in Israel’s 1948 War of Independence and helped found the country’s formidable air force.

Ahem, coming from a multi-generation lineage of aviators, I can read between the lines here enough to smell a Raven [1] cum General. This isn't to discount the circumstances of extrajudicial execution, mind you. It's simply that one side's hero is another's score to settle.

[1] https://en.wikipedia.org/wiki/Raven_Forward_Air_Controllers
Those elements will make your HTML invalid, it's not how a framework should work, sacrificing validity for functionality.

It would better work as a CSS framework with individual classes. Or, you can use the new Web Components API or just make a React component library.

The website needs some work. The colors on the website hurt my eyes, and things aren't aligned correctly.

This will perform differently on almost every browser, can you let us know what browsers are supported on the page?
PNG also explicitly specifies image dimensions, so there's no need for that at all.
> Stuff flying apart, though, is rare now; most systems drain the energy out of a system when they detect that happening. It's physically wrong, but looks less awful.

Most games are using position based dynamics now (see Nvidia PhysX) so there isn't even a variational principle to arrive at the system from an energy. Just a constraint satisfaction problem with a weak notion of momentum coupled in.
"Where an ideology was identified and recorded, just over half, or around 54%, of referrals related to Islamist extremism. Nearly one in 10 referrals was linked to far-right extremism."

So <10% are "far right" what makes up the other nearly 40%?
Hint: python -c 'help(max)'
That's a tough order. The closest I've seen is a 25 year old AS/400 working in the corner.
I've wanting an LG OLED55C6P for a monitor. It's a 55" 4K curved TV with HDR. I figure it's like 4 27" 1080 monitors without the borders in the middle. It's also vertically taller than one of those monitors in portrait orientation while still having tons of space to the side of said portrait. As I'm old enough to need progressive lenses, I don't really think there's any value in having higher pixel density than 4K at that size.
Coding is fun. Debugging sucks though.
Why hello there!

Are you guys looking to hire programmers with a few years of experience? Or are you looking to hire Tesla's lead programmers. The difference is important, because I've done things from both "server side" and "vehicle side" but I'm 3 years out of college, not an esteemed lead engineer who has been at Tesla and Toyota and working on cars my whole life. :)

I'm not trying to patronize you but I couldn't really tell. Usually newer companies advertise on here as a tiny dev team looking for extremely experienced programmers with a very specific set of skills, so I almost never apply to anything. This one sounds interesting enough, but yeah, I'm curious what your level of expectations are.

Thanks!
With Only 44 CS Grads, Is Harvey Mudd's CS Program's Success a Tad Overhyped?
Higher highs and lower lows. It makes your life before it seem flat, for better and worse.
Any body else here know what I mean?

Yes, I've read them all. Truly changed the way I look at and think about things in all sorts of contexts.
Origins of Python's “Functional” Features (2009)
It seems wrong and honestly disingenious to dismiss the term fake news and its narrative just because you've heard it once before.

Thats an anecdote, and we've had things like batboy and fake news about hillary clinton since the 90s.

Also the huge spike on google trends right after the election is undoubtedly important and meaningful.
I did it where I correlated the recommendations across all of the entrepreneurial podcasts I listen to:

http://blog.startopz.com/recommended-books-entrepreneurial-p...
With a date like that, it should be out of copyright. But I can't find it in any of the usual sources. I'll keep an eye out, though.

Thanks!
I ought to have distaste for them? No.

Sure I'd reform the program, but I am glad that they can come here and so are they. I would be upset if they couldn't anymore and so would they.

Sorry to say, I don't think reforming the program with applicants in mind is on the table in this administration.
This is what an "ETL" (Extract-Transform-Load) tool is for. Something like FME Server [1] would handle the first two points and the last point well.

For unzipping something that crazy, I'm interested in your solution - I think I'd have to write a custom zip library and use a RAMdisk or similar.

1: https://www.safe.com/fme/fme-server/
testing is a good idea, and I've seen it done in other "road warrior" type scripts in an unsafe way via `curl -s6 https://canhazip.com`. I may not have the time to contribute for a while, but what I would contribute is mostly laid out in the openvpn docs for enabling ipv6[0].

most of what you need to do with ipv6 is analagous to what you'd do with ipv4 like remembering to uncomment net.ipv6.conf.all.forwarding=1 in /etc/sysctl.conf ( just as you would with net.ipv4.ip_forward). use tun-ipv6 instead of tun, and server-ipv6 in addition to server. ip6tables rules in addition to iptables rules. etc etc. although that may be it -- can't think of anything else of the top of my head. I'll look more at your code and see if there's anything I can help with as time allows- ping me if I forget :)

0. https://community.openvpn.net/openvpn/wiki/IPv6
"I didn't say anything like that."

No one said that you said anything like that. Of course you didn't. But what you said necessarily implied that.

"We're talking past each other."

No, you willfully ignored and misrepresented all his points.

"Oh, but you can't pedal it on paths I don't provide unless you disable the safety."

That's a grossly dishonest misrepresentation the situation with Rust.
I completely agree with you on the difficulty of truly anonymizing data.

I do wonder if there any holes in this idea though: What if they released the data the number of cars per hour per segment of road (a segment meaning a continuous piece of road between two intersections) rounded to the nearest multiple of 10?
>> I'm on the lookout now for a 55" or larger 4K with a curved screen as I think that would help on both fronts.

Check out the LG OLED55C6P. As the name indicates 55" old Curved. Go to the store and sit right in front of it as though it's on your desk. I really want that at home and at work. Oh, and there's a 65CP6 as well...
You have identified a chain of events. Here is a list of questions I have about how that chain of events occurs. I am asking these in good faith, not being snarky, just in case it seemed otherwise.

How would the AI learn that more processing power equals becoming better at Go?

How would the AI learn to program?

How would the AI acquire the creativity necessary to develop novel programs?

Why would a Go AI have the ability to modify its own code?

How would it learn to use such an ability to escape its container?

How would it identify escaping its container as a goal?

How did this computer become connected to the internet?

How would it learn to communicate over a network?

How would it learn that taking over computers would give it more processing power?

Even biological human-level intelligences have difficulty learning such skills and planning so far in advance.
They help in a discussion with someone intellectually honest.
Where exactly is all this data being stored and transferred? Nowhere. The car doesn't have the storage and it doesn't have the bandwidth and all indications are it isn't transferring anything that amounts to actual images of the many many cameras it has.

Sure, they can push a beta algorithm to cars and record high-level decision making between human & algo, verifying it's not totally out of whack. But that's hardly something that is going as training data into the models.
Please don't post like this here. Accusations of shillage are not part of a civil conversation.
"Is that a bad thing?"

Regardless, it's a complete mispresentation of Rust, which is all that zero has to offer.
It sucks until you figure out the problem, then it's euphoric.
I don't think it's mutually exclusive - I simply don't trust Uber to act in good faith for anything they do. They've yet to demonstrate that they're deserving of trust that they're going to make that data open.
20% went to Republicans

I think that source has issues. The percentages total up to 135% for Santa Clara County. I'm 112% sure that has an error.
Very interesting. I worry about a selection bias, though. Uber riders are wealthier than average, so the trips won't necessarily reflect all the commutes of those in the city. I hope this doesn't lead to just an improvement in bus routes, road construction, traffic, etc. between nicer sections of the city.
Interesting how it can be dated based on the examples. Digg, CSS Zen Garden, and Silverlight.
Same data in Clojure:

[{:appt/start #inst "01-02-2014T9:15:00Z" :appt/end #inst "01-02-2014T9:15:00Z" :appt/description "See Anderson"}
{:appt/start #inst "01-02-2014T10:45:00Z" :appt/end #inst "01-02-2014T11:00:00Z" :appt/description "See Lundstrom"}
{:appt/start #inst "01-02-2014T13:15:00Z" :appt/end #inst "01-02-2014T16:00:00Z" :appt/description "Attend Y Committee Meeting"}
...]

Let's point out some important differences:

1) Clojure prefers maps over cons cells. This means that I always know exactly what I'm looking at. I don't have to guess that the second times are the end-times...I know because it' named :appt/end.

2) We don't overload data types. Have a date? Use a date type. In the CL example we see symbols and numbers sometimes used for descriptions, sometimes for times, etc. Here in Clojure we have #inst ... which creates an actual DateTime object. Now I no longer wonder what I'm looking at, I know it's a date.

3) Clojure prefers data over DSLs. What this calendar example shows is some sort of domain specific language that not only has to be parsed by a program, but it also has to be parsed by a human.

4) Got a meeting description, use a actual string....what's up with the use of symbols as strings (I never understood that about CL).

5) From the perspective of a data modeler...in the CL example if you want a meeting to last over midnight or for longer than a day, it looks like you're sunk.

But thanks for the example, it's fun to see how data modeling is done in other languages.
N900 is coming back:

https://neo900.org/
We detached this subthread from https://news.ycombinator.com/item?id=13351477 and marked it off-topic.
"the condescending comments from beginners"

You like to make stuff up.

"If you can't see them, it might be because you're aligned with that point of view."

Or it might not. It might be that you're just being abusive and dishonest.
Same here, but Ansible forces a basic knowledge of Python if you want to use it really well.
Yep. This article seems like yet another of MIT's attempts to aggrandize its "scientific reputation" in every field, even when the 'insights' gained are obvious or non-novel.
Depending on your threat model it's perfectly fine. It won't work against a nation-state-level adversary, but I don't get the feeling it's meant to. Against opportunistic passive sniffing or active MitM in cafés and such it's adequate.
That's a hilariously appropriate typo in the spelling of his last name.
The concept seems interesting. I' am trying it
> it's been known for decades that academia leaned left

Yes, but the question here is different: Has the proportion of conservatives diminished significantly? (tl;dr: Among 4-year colleges and universities in the U.S., according to one study, there has been a big change in New England but only a little elsewhere.)

And thanks for adding some actual facts to that discussion! Some thoughts on it, and a summary so that everyone doesn't have to read it:

The analysis is from Samuel Abrams, an apparently conservative[0] political science professor at Sarah Lawrence and the Hoover Institution. In looking up Abrams, I found an article that addresses his study plus other useful research and in a less biased way (the Boston Magazine article is titled "How Liberal Professors Are Ruining College" and mostly is anecdotes about persecution):

https://www.insidehighered.com/news/2016/07/05/new-analysis-...

the Higher Education Research Institute at the University of California, Los Angeles, ... conducts periodic surveys of faculty members and asks them, among other things, about their political views. Recent surveys have shown that faculty members are moving more to the left than has been the case in the past.

Abrams analysis is of the above data, and shows that the ratios are holding mostly constant in most of the country but in New England the ratio of liberal-to-conservative has shot up from around 5:1 to 28:1:

http://cdn1.bostonmagazine.com/wp-content/uploads/2016/12/li...

He found that region was the most significant factor, even controlling for institution type of discipline of faculty members.

Also of interest: The HERI data are on four-year college and university faculty members, and much research suggests that community college faculty members are more centrist than are their four-year colleagues. Remember that very many people attend community colleges, regardless of their relative prominence.

Still open is how much of that change is due to the country moving right and the definition of conservative changing; for example, much of Ronald Reagan's politics would be anathema to current "conservative" leaders. But again, many thanks; the research definitely informs the conversation.

----

[0] Abrams is identified as centrist in one article and conservative in another. The Hoover Institution is conservative, AFAIK.
It often helps to answer "I've seen something with this title before. Is this the thing I know already, or something new?" without clicking the link.
I benefited from this repost.
https://github.com/artursapek/mondrian
Thanks for posting this - I had been running into the same issues above.
The problem is that this drug is used worldwide as excuse to put in jail a lot of people also. Health issues aren't the only issues here to adress. This boy has now a bullseye painted publicly on his back and will be targeted in a few years by several kind of predators.

On the other hand, this "just put them to sleep all day" is a particularly outrageous "solution" for autistic children, that often are very smart and sensitive people and deserve to be respected and loved, as any other children.
Yes, that's ETL. Classic ETL dealt with databases, the modern variant has relaxed this constraint.

As for the zip: We simply "unzip -p" and stream process it carefully (with a custom program reading XML and transforming it). Cuts processing time from hours (extracting the zip and creating all directories, then visiting each file) to minutes (read from a single file).
That's not remotely related to what the comment you replied to was talking about. They're talking about groups of people that neither have nor want their own children.
Fun, too.
The guy's a troll.
> use all that bandwidth for an ultrawide display

Responding from my Dell u3417w, I have to agree that ultrawide is a great form factor for a monitor. The PPI of my monitor is about that of an Apple Thunderbolt display, so I do miss the "retina" experience of my Macbook Pro driving the monitor, but overall having so much horizontal real estate is glorious. I think my dream, too, would be a monitor with these dimensions but a higher PPI.
Can we stop making references to the trolley problem with respect to self-driving cars?