skysight-inc/HackerNews huggingface dataset:
One might say the same thing about browsers - as Flash bloated in scope over time, so do browsers now.

Firefox clocks in at a healthy ~50 MB, as does Google Chrome, for the offline installers - Chrome expands to ~360MB app size, and Firefox to ~180MB (both current versions on OS X 10.11). I won't even get started on the RAM usage - a typical 10-tab session can easily munch happily through 4 GB RAM and more (especially when there are lots of ads).

It's a pity that a 3-year-old netbook (or cellphone) is basically unusable because browsers creep up so much in featuritis :(
You can still use the old site for now https://sites.google.com/site/stevenhalim/
Multicast fails before bandwidth starts becoming an issue. Collisions and congestion are fact of life and do not necessarily mean that links are full (you may just have lots of packets stuck close together in time).

Most TCP implementations handles it well and Linux has well written guides on how to tune parameters when the defaults don't match well to your specific situation well. Multicast just fails catastrophically in arcane ways that depend upon your specific hardware, network topology and mix of your firmware version on your switch/routers. It makes it very hard to "design" a product around that kind of fragile system.
When I was first getting into philosophy for the first year or so I was convinced I was brilliant because I was coming up with all these great ideas that people loved to talk about but then I realized that it was all addressed (largely by the existentialists and postmodernists, but also the German idealists and Plato) decades, centuries or millennia ago. Now when I have what seems to be a unique idea I spend a while googling different formulations of it to make sure it's truly original.

This isn't to say that philosophy has already asked and answered every possible question, just that the most obvious versions of these (obvious for the kinds of people who would bother doing this in the first place, i.e. relatively intelligent people) probably have good discussions going on and that the philosophy of those questions is more about joining that discussion and adding your perspective (if it's new) than about formulating original paradigms and meta-narratives for solving them, because that's been done already for a lot of questions.
I think this isn't necessarily about open source, it's about tracking users in order to manipulate them with relevant ads. A company could use a fully open-source stack and still cross that creepy threshold with user tracking no problem.

A few years back I was using a paid Google Apps account (which IIRC excludes you from some tracking?) and FastMail simultaneously. Both were excellent from a technical perspective, but FastMail was just better. They make all their money from selling their service to end users and I trust this model more. So I switched to using FastMail exclusively.

I use free and open source software and strongly prefer it - but I don't see it as some kind of 'automagic' solution to tracking and privacy concerns. Buying the product, and not being the product, is what is important to me.

Also don't forget to self-host what you can, for example it's not that difficult or expensive to set up Nextcloud on a VPS for file and contacts/calendar syncing. And don't be a cheap bastard with the FOSS you use, make donations to developers of useful tools. You're going to pay for all this one way or another.
> The consequences of Zuckerberg’s inaction are now clear. On December 4, a North Carolina man brought a loaded weapon into Washington, D.C.’s Comet Ping Pong to investigate a conspiracy theory called #Pizzagate in which Comet is the hub of a child-trafficking ring organized by Democratic Party operatives. The story was popularized by fake news.

In the days after the election, I too believed in social media's major role the unexpected results. However, after more reflection, the problem seems far beyond facile dissemination of fake news: a large portion of the population are either post-truth thinkers, or, simply never learned how to fact check their news sources. If someone is going to be swayed to elect a different candidate because of a story that popped up in their news feed, there's not a whole lot we can do besides ensure that the next generation is better educated in healthy skepticism and critical thinking.
Not my industry, but in my area. I'm still looking for a good modular house that can be setup reasonably quickly, low cost, and can survive North Dakota winter and summer. Something suitable for a single person or a couple.
> play "fractal" like tones

What does that sound like?
Very good point, there's definitely a growing market for high-bandwidth GPGPU solutions, neural networks is probably just the start.
Long time reader, first time caller...er commenter. I might be biased since I work for them, but at www.stages.co we've JUST unveiled a headphone and a tabletop product (that lets you use any existing headphones) that I think this thread would find relevant. Directional listening, noice cancellation, noise reduction, high quality audio, user configurable admittance criteria - we have it all.
I'm doing the same thing. Been working on it for a year now and I'm solo.

Not going to lie, it's fucking hard sometimes. It's definitely not for everyone. Some days I want to give up and some days when things come together I feel that it's all worth it. That's the life you have to be prepared for - drastic ups and downs.

Since I'm bootstrapping the company I just don't have the money to hire full-time people yet. So what I did is bring on a few advisors who can help out in exchange for equity. Mainly in the areas that I'm not very strong skill wise. If anything they are someone to talk to. It can get lonely being solo.

As far as meetups and presenting it, that's usually better after you have something. If you already have something and have traction, people are often more interested in joining you rather than competing. Before then talk to prospective customers and make sure people actually want what you have.

Good luck!
I'm not very familiar with the open source options since after many years of coding this by hand, I work with what I know. I am a developer that works with data, not a Data Scientist, so I don't really know the lingo and whatever hipstery terms people are using these days. I will answer to the best of my ability, though (mostly for my sake, who knows if this will be useful):

Cleaning:

Open Refine seems to be the best product in this category. I haven't used anything but my own tools to do this before, so I can't really offer any advice.

Warehousing:

My understanding is that this is just a fancy way to talk about a database with a schema designed for analytics. There are many open source databases which do this very well, the one I use being Cassandra (and/or KairosDB), though it is also likely the one that is hardest to use. For a beginner, you might want to refer to this SO answer: http://stackoverflow.com/questions/8816429/is-there-a-powerf...

Data processing/collection:

This is something that is incredibly dependent on the data sources, so I likely can't tell you anything that will help. Most of my data sources I've worked with have been internally sourced log files, messages from ZMQ, or CSV data - you might be working with something far different though, since there are lots of public data sets and such which are common. Ideally, this would be integrated into the tools that you are using to clean the data, but I don't know if that exists.

Handling input from many different sources at different rates is not a very hard problem to solve if your system is build correctly - you could for example run a daemon for each data source which will populate the database when there is new data available, then send a message off to the processing engine, which will integrate the data into whatever reports you are running.

Specifically for a use case of a hedge fund, the reports could be triggered by a message which is sent when the new data is available, and processing could be done in parallel in Lambda or similar dependent on need to get a nearly instant return, enabling nearly real-time reporting.
Urban Crop Circle: “Ghost Roundabout” Designed to Confuse and Slow Drivers
That makes sense. I think there's opportunity for generative ML to eventually help here. An open dataset of (images, description) -> 3d model would go a long way. Check out this paper on using GANs to generate voxel-based models: http://3dgan.csail.mit.edu/

I've been studying and working with GANs for about a year now. They are still very exciting, and I'd love to try to expand my codebase to new types of data.

Additionally, there are some recent techniques that haven't been tried with voxel-based renderings.

Perhaps there is another algorithm that can help go from voxel -> polygons as well.

I think with the right tech, time, and execution this could be a matter of:

1. Take a picture

2. Generate until you get the 3d model you want
Or as this is otherwise known, "spying on your customers."
dang: that wasn't my intention, but rereading what I wrote, I must admit you're right. The story upset me and I overreacted. Please feel free to delete/hide my comment. I added a note to prevent upvoting for now.
Yep, tried on the Vive and my experience mirrored what I've seen since. There was another paint/modeling program I played with and it was similar. I think it was Sketchfab.

I'm just not impressed with most of my VR experiences to the point that they made a real impact on my thinking. Maybe it's cause I'm an AR developer.
It seems worth noting that as web browsers have removed the ability to disable JavaScript they have added options to disable web fonts. I flipped this switch in Firefox for Android because it makes pages load noticeably faster and conserves data. Lots of sites seem to host web fonts on third-party CDNs which require a separate TCP connection: a big deal on high-latency mobile.

Blocking web fonts usually does not noticeably degrade a site's usability or functionality. Icon fonts are the exception. I've learned that a question mark in the upper-right corner is usually a site's menu, except when it's the search function. I don't think that this level of breakage is comparable to what happens when JavaScript is disabled, since many sites just break completely without scripting.
I think Intel has been trying to do this for a while with their realsense technology and accompanying cameras.

http://www.intel.in/content/www/in/en/architecture-and-techn...
Splunk provides this kind of solution and it's pretty amazing :)
Named after the Russian word for "arrow", right?
The more anti-consumer way would be to just pipe every URL back home in real time.
I'd never delete an exchange with so exemplary a recovery :)
Finance, specifically fixed income:

Regulations will unbundle research from liquidity provision starting sometime in 2018 (moving regulatory target, but theme seems clear). Sellside will have lower incentive to pay large research groups; buyside will have to pay explicit fees for research advisory. There is a big opportunity in providing platforms, with macro and market data live, where researchers can interact with capital managers, given that said researchers will likely find themselves bereft of their current distribution networks (bank sales forces) sometime in the next few years.

Equities already have a hybrid form of this where buyside earns "credits" to be allocated at the end of each year to research providers. But fixed income is at least twice as big as equities, is much more opaque because is essentially unlisted (mainly "OTC" = "over the counter" ie only those in the know), and therefore much more susceptible to disruption.
I immediately went to their web site, and looked for their facilities near me. There are two, and a third in the region.

I hope I remember not to go there in a crisis.
Is this idiomatic Brainfuck?
Visualising the Riemann zeta function and analytic continuation
Cool. Do you have a sample of some music you did, so I can figure out how to work with it easily? Thanks.
Hey, amos here, main developer of wharf/butler, here's a quick technical summary so you don't have to do the digging yourself:

- File formats are streams of protobuf messages - efficient serialization, easy to parse from a bunch of programming languages. Most files (patches, signatures) are composed of an uncompressed header, and a brotli-compressed stream (in the reference implementation, compression format are pluggable) of other messages.

- The main diff method is based on rsync. It's slightly tuned, in that: it operates over the hashes of all files (which means rename tracking is seamless - the reference implementation detects that and handles it efficiently), and it takes into account partial blocks (at the end of files, smaller than the block size)

- The reference implementation is quite modular Go, which is nice for portability, and, like elisee mentioned, used in production at itch.io. We assume most things are streaming (so that, for example, you can apply a patch while downloading it, no temporary writes to disk needed), we actually use a virtual file system for all downloads and updates.

- The reference implementation contains support for block-based (4MB default) file delivery, which is useful for a verify/heal process (figure out which parts are missing/have been corrupted and correct them)

- The wharf repo contains the basis of a second diff method, based on rsync - for a secondary patch optimization step. The bsdiff algorithm is well-commented with references to the original paper, and there's an opt-in parallel bsdiff codepath (as in multi-core suffix sorting, not just bsdiff operating on chunks)

- A few other companies (including well-known gaming actors) have started reaching out / using parts of wharf for their own usage, I'll happily name names as soon as it's all become more public :)

I'd be happy to answer any questions!
HDD is the new tape.
I love earasers. Best I've used so far. Super comfortable, tiny, and discreet. No one even knows you're wearing them, and they work so well for concerts, without making everything muffled. I use them for everything with noise at this point,mowing the lawn, snowblowing, etc.
Twinax DAC's cost about that much, assuming you don't need a long run. Hell, I bought two Mellanox Connect-X 2 cards + a twinax cable for my home (direct connection between my FreeNAS box and my TD340 running oVirt) for under $100.
Intel Xeon Phi Knights Landing – X86 Code Compatibility – It Boots Windows
They are parallel because they don't intersect.
Having a "meaning" that you have given to yourself, that you truly believe in, and that is used to guide your life as a "meaning" should is, I think the bedrock of happiness, productivity and mental stability.

Certainly, in a world where everything needs to be questioned and nothing faithfully believed, it helps to have a meaningful central anchor on which to moor your self and your actions.

It is amazing to me that so many people go about their lives and couldn't reliably answer basic questions like, "why are you still doing this(life)?", "what is your purpose?", "how do you justify the value of your continued existence?" Alas, you can't ask those questions too often as they have a tendency to induce some dark thought patterns. However, having a well-constructed sense of meaning/purpose hardens you against those questions and also the many other problematic questions in life and it would do many well to answer them.
Check out Striim

http://www.striim.com/

It's a native streaming platform so your data will be cleansed, processed, scanned for outliers event-by-event rather than in batches. We have dozens of streaming connectors IT/Enterprise/Web data sources. We also support initial load for your firehose data. For unstructured data, we have support for RegEx based parsers.

Shoot me a message if you have any more questions. We have many big name users in Aerospace, Banking, Device manufacturing, and Logistics industries.
It depends on the network. If the network is 100-1000 nodes and you can guarantee that the switches and routers you use to build it (and upgrade in future) implement it correctly and won't break/introduce bugs in future firmware release then yes.

In the kind of environments I worked in, this was a non-starter.

edit: also "limit multicast messages only to ports that have actually joined a group" part is usually implemented by bloom-filters that I talked about above. They work well as long as there are not many groups and not many members.
It seems like a lot of anxiety issues, including this case, arise from the following internal monologue: "I am not as good as my peers at skill A. My peers are aware that I'm lacking in this area and judge me negatively for it."

From what I've seen (in this thread and in real life), people mostly combat this internal monologue in a couple ways that ignore reality (at times):

Counter Belief #1) I'm not as bad at skill A as I thought. (e.g. look at all these examples where I did well at skill A)

Problem: If the reality of the situation is that you aren't actually lacking in skill, this works. However, if your peers legitimately outclass you, you're just tricking yourself -- which presumably won't work for long.

Counter Belief #2) My peers are aware that I lack skill A, but are not negatively judging me for it. (e.g. what evidence do i have that they're judging me? they're probably just thinking about themselves all day)

Problem: This works when your peers aren't actively negatively judging you. But, how are we to determine if that's the case? Lack of evidence isn't proof at all -- especially in a work environment, social norms are that they won't express negative criticism to you (even if you ask for feedback). Certainly, there are environments where your peers ARE indeed actively negatively judging you in which case this counter belief does not work.

What is the right way to cope in a situation where reality is:
1) you ARE outclassed by your peers
2) your peers ARE judging you for it

One solution seems to be to get better at the skill than your peers. However, this isn't foolproof if you don't have the resources to do so.

The more foolproof solution seems to be:
1) learn to stop comparing yourself to your peers. your self-esteem should not depend on the relative amount of skill you possess to your peers
2) learn to stop caring what your peers think of you. your self-esteem should not be dependent on outward signals like this.

However, this seems REALLY hard to execute. How does one develop these sets of beliefs? Are there other logical solutions?
Time to go play outside.
I'm guessing these aren't really consumer drives. :(

Might as well ask here. My 2TB Plex hard drive is getting full. What's the best solution if I want say 10TB to store my media for Plex but very hands off?

I don't want to weld any contraption just plug something in and have it be visible by Windows 10.
I'd argue it depends on the methodology by which they're implementing the technology. They could be using the directionality to amplify the "target" signal and identify the signals that the wearer desires to mute and then run a fast algorithm to reduce that sound.
Please don't editorialize in the headlines.

And no, there's nothing even close to an Enron situation with Magic Leap. Misleading investors is no where near the same as repeatedly pushing and encouraging your employees to reinvest retirement funds into the company at the same time that you're instigating wide-spread fraud with that same money.
Abovitz publicly said just a few months ago that the production line was ramping up and they were getting ready to start.

http://www.businessinsider.com/magic-leap-production-begins-...

IMO that's not a "long game" announcement by any means.
Global CDN with a fixed monthly cost and a capped bandwidth. For example, pay 100$/month and get up to 1Gb/s, traffic above this limit is dropped.

Providers today offer uncapped bandwidth, some, like Amazon, without any cost cap, some, like KeyCDN, allow to set a cost cap, but take the resources off-line when the limit is exceeded.
Oh, so you mean like a big box that could fit XYZ items inside it and capture something like 10,000 per day?

Maybe but I actually think that's the wrong approach.

I mean, to me it's kind of hard to believe nobody's tried making a "conveyor belt" like process inside a closed system

Yea they have - kinda. None of it works well or fast enough though. We put up a patent for one a year ago before I thought there was a better way to do it. The manpower required to move items onto/off of a line is a big part of the problem.
Google just released File-By-File patching for zip files -- sounds like it might be close:

http://android-developers.blogspot.com.au/2016/12/saving-dat...
20 Funny Gifts for Her- Gift Ideas for a Woman Who Likes Funny Gifts
Processor monitoring with Turbo Boost, c-states, hyper threading topology, ips ...
ASCII UI & kernel module designed for Linux and accuracy
You will need some kind of NAS, I suppose, or you can use something like stablebit drivepool.
A good post of David's to start with is "Developing Ethical, Social, and Cognitive Competence":

https://vividness.live/2015/10/12/developing-ethical-social-...
Yeah, I'm sure, I've been doing audiometry tests yearly and the doctor says it's quite good.
Hijacking Healthcare: Ransomware Attacks vs. Health Record Security
Kickstarter separates video games from software. They're #1 in that section when sorted by funding.

https://www.kickstarter.com/discover/categories/technology/s...
I also feel like we should perhaps be grateful that it's Google that's powerful here. I know they are know longer the company that 100% embraces "Don't be evil" but I still believe they are trying. It's just that this amount of success, their market(s) and political pressure will steer any company towards "profit > privacy". If you accept this, Google might be one of the better companies to experience this from as some of the other contenders like Yahoo, Microsoft seem to go through this process more rapidly and without much consideration.

To sum up my thoughts: while I don't like the direction they seem to be going i'm glad they're the one leading.
A lot of stuff.

SHA256 is very slow, and that's no surprise. It's cryptographic after all.

Here's a small list of usecases for non-cryptographic hash functions:

- Checksums and error correction codes, as long as there is no way to maliciously use this.

- Hash tables. These always use non-cryptographic hash functions.

- Bloom filters.

- Heuristic fingerprinting. They're not strong enough to be used for normal data fingerprints, but they can be used as a way to decide if two buffers are "probably equal" or "certainly not equal".

Hash tables are the main one. Cryptographic hash functions are almost never used in them. SipHash is a popular choice, but it is not cryptographic. That is a misunderstanding: It's a MAC function.
No, it's not a cryptographic hash function. It's a MAC function.

The paper clearly states that it is not collision resistant.
Please don't. DJB2 is a poor hash function. It's similar to FNV: Entropy only moves upwards, so flipping higher bits doesn't affect lower bits.

In other words, you risk mapping `n` and `-n` to the same value under some modulus.