

Video Processing at Dropbox - lowe
https://tech.dropbox.com/2014/02/video-processing-at-dropbox/

======
9999
Interesting article.  I have a few comments/questions:

HLS support on Android is incredibly spotty.  Do you mean in-app HLS support? 
The Android Chrome browser doesn't support it at all, and only a select few
versions of the regular Android browser support it.  Then again, there's not a
great alternative. 

Why offer the multiple bitrates on the HLS stream at all?  You know what the
client's bandwidth is, why perform three separate live transcodes?  Is this
because you're delivering the stream to an iOS app and Apple requires that?  In
Safari or a UIWebkitView based app you probably wouldn't have to do that... Or
is it to compensate for potential bandwidth fluctuations? 

You mentioned the mpeg-ts segmenter in more recent versions of ffmpeg but also
mentioned that it has unacceptably high latency.  I have not found this to be
the case so long as you set the segment times individually and force new
keyframes at given times (-force_key_frames and -segment_times flags,
otherwise the segmenter ignores your segment time option and just creates TS
chunks at whatever frequency it wants to). 

Pre-transcoding the first few seconds is a really great approach that I had
never considered before.  Very cool. 

~~~
nairteashop
> HLS support on Android is incredibly spotty

I assume you mean spotty in Android pre-4.0?  I'd worked on HLS streaming for
Android recently and didn't have issues with ICS and later.  But then again
with Android you can't possibly test on all OS/device combinations, so would
love to know if there are things I should be watching out for. 

> Why offer the multiple bitrates on the HLS stream at all?  You know what the
> client's bandwidth is

If you're watching video over a cellular network, it's very difficult to use a
single bandwidth since it can fluctuate a lot.  So the best practice (as
dropbox is doing) is to offer multiple streams at different bandwidths to the
client, and allow the client to pick the best one for the following segment
based on current network conditions. 

~~~
mackey
Even post 4.0 it isn't super reliable.  Basic playback of HLS works, but we
have had issue seeking, pausing for long periods of time and resuming, etc.  I
wouldn't be surprised if things like WebVTT(Subtitles), ID3 tags and other
advances features also don't work well. 

4.4 is supposed to be much better but I don't work on Android anymore, so I am
not really sure. 

------
__david__
Interesting.  I recently implemented a very similar transcoding server[1] for
StreamToMe[2].  I used node and was pleasantly surprised at how easy it was to
parse and segment MPEG Transport Streams[3].  It's good to know I was on the
right track...

[1] [https://github.com/caldwell/stm.js](https://github.com/caldwell/stm.js)

[2]
[http://zqueue.com/streamtome/index.html](http://zqueue.com/streamtome/index.html)

[3]
[https://github.com/caldwell/stm.js/blob/master/mpeg2ts.js](https://github.com/caldwell/stm.js/blob/master/mpeg2ts.js)

------
plg
Does it bother anyone else that dropbox is "processing" files at all?  whether
video or otherwise? 

~~~
morganvachon
I was under the impression that Dropbox can't "see" your files in the sense
that they wouldn't be able to tell what was inside a document or photo; it's
all supposed to be encrypted.  Given this new information, it appears that not
only can they see what you have uploaded, they can actively process, encode,
decode, etc. 

Not exactly what I'm looking for in an online file storage company.  I realize
they've never outright said "we're 100% secure end to end" but they've
indicated in the past that they have no interest in looking at their users'
files.  Now they are showing interest.  Count me out. 

~~~
JoshTriplett
> I was under the impression that Dropbox can't "see" your files in the sense
> that they wouldn't be able to tell what was inside a document or photo; it's
> all supposed to be encrypted. 

No, that was never true.  In particular, that would break Dropbox's model for
deduplication, as well as their web clients, and various other things.  If
you're looking for an end-to-end encrypted solution, Dropbox was never it. 

~~~
nwh
To expand on that, mega.co.nz does attempt to do encryption end deduplication. 
Each file uploaded is encrypted with it's own hash, and the hash is stored in
the users encrypted account.  The idea is that if two users upload the same
file they will have the same hash and be deduplicated.  It's not particularly
secure or private in their implementation, but it can certainly be done. 

------
supercoder
Everytime I've ever tried to play an mp4 video from the iPhone app it's never
worked.  Maybe it would be better to just simplify the setup. 

------
nodata
Watching videos on dropbox.com with my head turned sideways is uncomfortable. 

Please could you use the rotate flag for videos? 

~~~
pbacci
Hello there, Pierpaolo from Dropbox here. 

Indeed, we do try to extract the rotation flags from the original and apply
them when we stream the video.  One possible issue is that different devices
seem to apply different criteria to determine the rotation.  For instance, the
iPhone seems to detect the rotation only at the beginning of the video capture
so if you rotate your device after you started capturing, you end up with
content that has a messed up rotation. 

Having said that, there is a very good chance that you are simply hitting some
bug that I'd love to fix :) If you are willing to share the problematic video,
you can hit me up at pierpaolo@dropbox.com and we can go from there. 

~~~
nodata
Thanks for your reply. 

You are right - on the iPhone it does things correctly unless I rotate the
phone again. 

On my camera ffmpeg says it never sets the flag.  So not your fault :)

------
carson
It would be awesome to see the 1st and 3rd tools open sourced. 

~~~
pbacci
Hello Carson, Pierpaolo from Dropbox here. 

we are considering open sourcing the segmenter tool (3rd).  As you can imagine
it's very much tailored for our pipeline and as the internal tool built in
ffmpeg matures, we think our solution will become of marginal value.  I keep
debating myself if/when to bite the bullet and try out the ffmpeg one. 

The first tool is quite similar to the one that mau posted
([https://github.com/danielgtaylor/qtfaststart](https://github.com/danielgtaylor/qtfaststart)). 
Given that we forked quite some time back, I'd suggest to start from that one
since it has probably bug fixes on top of our version. 

------
afhof
libfaac is a safe aac default, but aacplus works MUCH better at the the 8 - 16
kbps range, which seams to be a stated goal of this page.  It turns on SBR+PS
and has ffmpeg support. 

If any Dropbox people are reading this, can you comment on what criteria you
used to pick the audio codec? 

~~~
pbacci
Hello there, Pierpaolo from Dropbox here. 

You correctly guessed that we picked libfaac as a safe option for
compatibility.  I never tried out aacplus and sounds indeed interesting so
we'll likely try it out soonish.  Our target rate is a bit higher than those
you mention though.  For audio, we target 32kbps at low quality layers and
96kbps at higher qualities.  Couple of questions for you: \- how does aacplus
works at these rates? \-  what compatibility issues can we expect if we were to
try it out? 

~~~
otterley
HE-AAC (AAC+) sounds surprisingly fantastic at 32kbps. 

------
higherpurpose
It's 2014.  Does Dropbox still offer only 2 GB of storage?