TAMPER-EVIDENT STORAGE AND PROVISIONING OF MEDIA STREAMS

The present disclosure relates to systems and methods for tamper-evident storage of a media stream, and for serving tamper-evident media. The tamper-evident media is stored using a data structure comprising a payload field and an associated metadata field. A portion of a media stream is obtained from a media source. A set of parameters, to be stored in the metadata field of the data structure, is determined. A progenitor parameter to associate with the set of parameters is obtained, the progenitor parameter based on a hash of at least part of a previous data structure storing a previous portion of the media stream. The progenitor parameter is incorporated into the set of parameters, and the set of parameters is stored in the metadata field of the data structure. The portion of the media stream is stored in the payload field of the data structure.

FIELD

The present disclosure relates generally to digital media, and more specifically to securing digital media.

BACKGROUND

Media recording devices are used in a wide variety of contexts, and it is often desirable to store recorded media for long periods of time to enable later retrieval thereof. Media may be stored in a variety of formats and using any suitable type of storage medium, and indeed may be transferred between media, storage locations, and the like, throughout its lifespan.

In the context of surveillance video or other surveillance media, a particular recording may not appear useful or important at the time it is acquired, but may prove crucial at a later time, whether as part of an investigation, as evidence in a proceeding, or the like. For surveillance media to be admissible, for instance as part of a legal action, authorities may require that a so-called “chain of custody” for the media be demonstrated, whereby the party presenting the media can prove who produced the media, how it was produced, and that it has not been altered until the moment it is presented as evidence, as well as demonstrate that the media has not been altered in any way.

As such, approaches for ensuring that media can be stored in a way to prevent and/or indicate evidence of tampering may be useful.

SUMMARY

The following presents a simplified summary of one or more implementations in accordance with aspects of the present disclosure in order to provide a basic understanding of such implementations, without limiting the embodiments presented within the present disclosure. While existing approaches for media storage, secure and otherwise, are suitable for their purposes, in some cases they may lack suitable security mechanisms, or apply incomplete or circumventable security mechanisms. Conversely, in some cases the security mechanisms used are so elaborate as to impede the responsiveness or speed of the system, or require complex data storage approaches which may limit their usability. As a result, improved approaches for securing digital media may be of interest.

In particular, approaches to securing digital media which are tamper-evident—that is to say, digital media which will display evidence of tampering should such tampering occur—may be of interest in certain particular contexts, such as with surveillance video or similar media. For example, a party wishing to present or enter into evidence a particular piece of digital media may need to prove a chain of custody for the piece of digital media. This may involve demonstrating the source of the piece of digital media, how it was produced, and demonstrating that the piece of digital media has not been altered in any way.

To this end, the present disclosure provides, inter alia, methods, systems, devices, and computer-readable media for tamper-evident storage of a media stream, or other types of digital media.

The operational context for the tamper-evident storage may, in some cases, be a cloud-based environment, wherein the media stream, in whole or in part, is stored in a cloud-based storage system, though other operational contexts are also considered, including local storage environments, as well as so-called “hybrid” environments (which involve both local and cloud systems). Additionally, it should be understood that, while in certain cases, part or all of the disclosed methods, systems, devices, and computer-readable media may operate in a local (i.e., non-cloud) context, when at least part does operate in a cloud context, or when an output is destined for a cloud-based system (storage or otherwise) at some point downstream, the relevant method, system, device, and/or computer-readable medium may still be said to operate in a cloud context.

To store the media stream in a tamper-safe capacity, a data structure which comprises a payload field and an associated metadata filed is used. A portion of the media stream, which can be of any suitable length or size, is obtained from a media source. Then, a set of parameters associated with the portion of the media stream are determined; the set of parameters are to be stored in the metadata field of the data structure, and the portion of the media stream are to be stored in the payload field of the data structure. A progenitor parameter is obtained, which is based on a hash of at least part of a previous data structure storing a previous portion of the media stream, which has already been rendered tamper-evident. Then, the progenitor parameter is incorporated into the set of parameters, which is then stored in the metadata field of the data structure. The portion of the media stream is stored in the payload field of the data structure, such that both the set of parameters and the portion of the media stream, to which the set of parameters is associated, are stored in the same data structure.

In accordance with a broad aspect, there is provided a method for tamper-evident storage of a media stream using a data structure comprising a payload field and an associated metadata field. The method comprises: obtaining a portion of the media stream from a media source; determining a set of parameters to be stored in the metadata field of the data structure, the set of parameters associated with the portion of the media stream to be stored in the payload field of the data structure; obtaining a progenitor parameter to associate with the set of parameters, the progenitor parameter based on a hash of at least part of a previous data structure storing a previous portion of the media stream, the previous data structure having previously been rendered tamper-evident; incorporating the progenitor parameter into the set of parameters and storing the set of parameters in the metadata field of the data structure; and storing the portion of the media stream in the payload field of the data structure.

In at least some embodiments according to any one or more of the previous embodiments, said determining the set of parameters associated with the portion of the media stream comprises generating at least an identifying hash of the portion of the media stream.

In at least some embodiments according to any one or more of the previous embodiments, the method comprises generating a subsequent progenitor parameter based on the set of parameters, including the identifying hash of the portion of the media stream, after said incorporating the progenitor parameter therein, the subsequent progenitor parameter for use in rendering a subsequent portion of the media stream tamper-evident.

In at least some embodiments according to any one or more of the previous embodiments, said generating the identifying hash of the portion of the media stream is performed incrementally as the portion of the media stream is obtained from the media source.

In at least some embodiments according to any one or more of the previous embodiments, said obtaining the progenitor parameter comprises obtaining the progenitor parameter being based on a hash of a previous set of parameters associated with the previous portion of the media stream and stored in the previous data structure.

In at least some embodiments according to any one or more of the previous embodiments, said obtaining the progenitor parameter being based on a hash of a previous set of parameters associated with the previous portion of the media stream comprises the progenitor parameter being based on a hash of a metadata field of the previous data structure storing the previous portion of the media stream.

In at least some embodiments according to any one or more of the previous embodiments, said obtaining the progenitor parameter being based on a hash of a previous set of parameters associated with the previous portion of the media stream comprises the progenitor parameter being based on a hash of the previous portion of the media stream present in the previous set of parameters associated with the previous portion of the media stream.

In at least some embodiments according to any one or more of the previous embodiments, the method comprises, after said incorporating the progenitor parameter into the set of parameters and said storing the set of parameters in the metadata field of the data structure, generating a subsequent progenitor parameter based on the metadata field of the data structure for use in rendering a subsequent portion of the media stream tamper-evident.

In at least some embodiments according to any one or more of the previous embodiments, said determining the set of parameters comprises obtaining a digital signature based on the portion of the media stream and a cryptographic key associated with one of the media source, a media archiver performing said storing the set of parameters and the portion of the media stream, and a user authorizing the tamper-evident storage of the media stream.

In at least some embodiments according to any one or more of the previous embodiments, said storing the set of parameters in the metadata field of the data structure comprises: storing an indirection in the metadata field; and storing the set of parameters in a file separate from the data structure, wherein the indirection points to the file to render the set of parameters accessible via the metadata field.

In at least some embodiments according to any one or more of the previous embodiments, said storing the set of parameters in the metadata field comprises: applying a compression algorithm to the set of parameters to produce a compressed set of parameters; and storing the compressed set of parameters in the metadata field.

In at least some embodiments according to any one or more of the previous embodiments, the method comprises storing a copy of the set of parameters in the payload field in association with the portion of the media stream.

In at least some embodiments according to any one or more of the previous embodiments, the data structure is configured for storing multiple portions of the media stream, wherein said storing the set of parameters in the metadata field of the data structure comprises storing the set of parameters in the metadata field alongside a previous set of parameters associated with the previous portion of the media stream, and wherein said storing of the portion of the media stream comprises storing the portion of the media stream in the payload field alongside the previous portion of the media stream.

In at least some embodiments according to any one or more of the previous embodiments, the method comprises storing a copy of the set of parameters in the payload field in association with the portion of the media stream alongside a copy of the previous set of parameters also stored in the payload field, in association with the previous portion of the media stream.

In at least some embodiments according to any one or more of the previous embodiments, said obtaining the portion of the media stream from the media source comprises acquiring the portion of the media stream as original media.

In accordance with another broad aspect, there is provided a system for tamper-evident storage of a media stream in a cloud-based environment using a data structure comprising a payload field and an associated metadata field. The system comprises: a processor; and a non-transitory computer-readable medium storing thereon instructions. The instructions are executable by the processor for causing the system to perform: obtaining a portion of the media stream from a media source; determining a set of parameters to be stored in the metadata field of the data structure, the set of parameters associated with the portion of the media stream to be stored in the payload field of the data structure; obtaining a progenitor parameter to associate with the set of parameters, the progenitor parameter based on a hash of at least part of a previous data structure storing a previous portion of the media stream, the previous data structure having previously been rendered tamper-evident; incorporating the progenitor parameter into the set of parameters and storing the set of parameters in the metadata field of the data structure; and storing the portion of the media stream in the payload field of the data structure.

In accordance with a further broad aspect, there is provided a method for serving tamper-evident media, the media having previously been stored using at least one data structure comprising a payload field and an associated metadata field. The method comprises: obtaining a request to serve a portion of a media stream; determining, based on the request and at least one set of parameters associated with the media stream, a particular data structure of the at least one data structure, the at least one set of parameters having previously been stored in the metadata field of the at least one data structure; obtaining, from the payload field of the particular data structure, the portion of the media stream to be served; and providing the portion of the media stream.

In at least some embodiments according to any one or more of the previous embodiments, determining the particular data structure to be served comprises consulting a seek table stored in the metadata field.

In at least some embodiments according to any one or more of the previous embodiments, said determining the particular data structure to be served comprises consulting an index generated based on the at least one set of parameters having previously been stored in the metadata field of the at least one data structure.

In at least some embodiments according to any one or more of the previous embodiments, the method comprises, following said consulting the index generated based on the at least one set of parameters, validating the particular data structure to be served based on consulting the at least one set of parameters having previously been stored in the metadata field of the at least one data structure.

In at least some embodiments according to any one or more of the previous embodiments, a plurality of portions of the media stream are stored in a common payload field of a single data structure, wherein the method comprises determining, based on the request and a plurality of sets of parameters associated with respective ones of the plurality of portions of the media stream, a particular one of the portions of the media stream to be provided.

In at least some embodiments according to any one or more of the previous embodiments, the method comprises validating that the portion of the media stream is tamper-free.

In at least some embodiments according to any one or more of the previous embodiments, said validating that the portion of the media stream is tamper-free comprises comparing a progenitor parameter stored as part of the set of parameters in the metadata field, the progenitor parameter based on a hash of at least part of a previous data structure storing a previous portion of the media stream, with a newly generated progenitor parameter generated based on the at least part of the previous data structure, the previous data structure having previously been rendered tamper-evident.

In at least some embodiments according to any one or more of the previous embodiments, the method comprises comparing a plurality of progenitor parameters for each of a plurality of data structures storing the media with a plurality of newly generated progenitor parameters, wherein the plurality of progenitor parameters comprises the progenitor parameters, and wherein the plurality of data structures comprises the at least one data structure.

In at least some embodiments according to any one or more of the previous embodiments, said validating that the portion of the media stream is tamper-free comprises verifying a signature associated with the portion of the media stream.

In at least some embodiments according to any one or more of the previous embodiments, said validating that the portion of the media stream is tamper-free comprises comparing a hash of the portion of the media stream stored as part of the set of parameters in the metadata field with a newly generated hash of the media stream.

In accordance with another broad aspect, there is provided a system serving tamper-evident media, the media having previously been stored using at least one data structure comprising a payload field and an associated metadata field. The system comprises: a processor; and a non-transitory computer-readable medium storing thereon instructions. The instructions are executable by the processor for causing the system to perform: obtaining a request to serve a portion of a media stream; determining, based on the request and at least one set of parameters associated with the media stream, a particular data structure of the at least one data structure, the at least one set of parameters having previously been stored in the metadata field of the at least one data structure; obtaining, from the payload field of the particular data structure, the portion of the media stream to be served; and providing the portion of the media stream.

Additional details and information regarding one or more embodiments, including those described in the preceding paragraphs, are set forth in the accompanying drawings and the description below. Other features and aspects will be apparent from the description and drawings, as well as from the claims.

DETAILED DESCRIPTION

The present disclosure relates to, inter alia, methods, systems, devices, and computer-readable media for tamper-evident storage of a data stream, such as a media stream, or of other types of digital media and to various related methods. For clarity, reference to and discussion of digital media in the present disclosure should be understood as encompassing various manners and form of digital media. The digital media considered herein includes, inter alia, images, video media, audio media, haptic media, and other types of digital media, as appropriate, and may be stored in various formats, depending on the type of digital media. In certain embodiments, the digital media may be stored in standards-compliant formats, whereas in other embodiments, the digital media may be stored in proprietary or custom formats. Additionally, in some embodiments, the digital media may be a media stream, that is to say, a sequence of data elements produced over time, for instance in a continuous fashion, and in some cases substantially in real-time.

With reference toFIG.1, there is illustrated a media streaming system100. The media streaming system100is composed of one or more media sources, illustrated inFIG.1as a camera110and a microphone120, and a media storage server130. The camera110and the microphone120, collectively referred to as the media sources105, are configured to produce digital media, for instance in the form of media streams150. Although the present disclosure focuses primarily on embodiments of the media streaming system100in which the media sources105produce digital media, it should be understood that embodiments in which analog media is produced and then converted to digital media are also considered. Additionally, although only two media sources105are illustrated inFIG.1, it should be understood that the media streaming system100may include any number of media sources105of any suitable type, including media sources105which produce images, video media, audio media, haptic media, and the like. The media sources105may include media sources which produce original media, such as the camera110and the microphone120, which record or otherwise acquire digital media by way of suitable sensors. The media sources105may additionally, or alternatively, include media sources which playback or otherwise provide previously recorded media, such as hard drives, solid state drives, network-attached storage devices, cloud or other network-based storage systems, media center computers, general purpose computers, or the like.

In some embodiments, the media streaming system100may include one or more additional elements, for example one or more devices which mediate the connection between the media sources105and the media storage server130: for instance, a media archiver135is coupled to the microphone120and to the media storage server130. It should be noted, however, that other types of devices may be used to mediate the connection between the media sources105and the media storage server130, for instance a gateway, a bridge device, or the like (reference herein to the media archiver135should not be construed as limiting or excluding other types of mediating devices). As will be described in greater detail hereinbelow, some media sources105are capable of generating a media stream150which is already tamper-evident, whereas some other media sources105are not. In cases in which a media source is not capable of generating a tamper-evident media stream150, the media source105(e.g., the microphone120) will generate a media stream150which is intercepted by the media archiver135. The media archiver135can then transform the media stream150so as to render it tamper-evident, as will be described hereinbelow. Alternatively, the media source105may generate a media stream150and transmit it to the media storage server130, which can render it tamper-evident. Other configurations are also considered, including a first media source105sending a media stream150to a second media source105, which renders the media stream150of the first media source105tamper-evident before passing the media stream150on to the media storage server130for storage. Reference herein to embodiments and examples which discuss the media stream150being rendered tamper-evident by the media source105, or by the media storage server130, do not exclude cases in which an intermediate device, for instance the media archiver135, are used to perform the requisite operations to render the media stream150tamper-evident.

The media storage server130may be any suitable type of device for storing the media streams150provided by the media sources105. The media storage server130may be accessible to the media sources105over one or more local connections, e.g., via one or more Ethernet cables, one or more local routers, and the like, over one or more remote connections spanning broader networks, e.g., over the Internet, or any suitable combination thereof. In some embodiments, the media storage server130may be part of a cloud-based storage service, such as Microsoft® Azure®, Amazon® AWS®, or a similar cloud-based storage service offered by another provider. In some embodiments, the media storage server130is an abstraction of several layers of storage, which may include local storage, networked storage, cloud storage, or any suitable combination thereof. In some embodiments, the media storage server130is implemented as distributed storage, for instance as a collection or one or more of distributed servers, or in any other suitable fashion. The media storage server130may store the media obtained from the media sources105-provided as the media streams150—for any number of reasons, including for later playback, for performing analytics or other forms of analysis thereon, for safekeeping of evidence, or the like. As will be described in greater detail hereinbelow, the media storage server130may provide stored media, for instance a stored copy of the media obtained from the media sources105as part of the media streams150, to a user or other system, for instance in response to a user request, on the basis of a provisioning schedule, or the like.

To simplify the following discussion, reference will be made to a single media source105providing the media stream150to the media storage server130. The media stream150may be in any suitable format, and may be converted between formats at different stages, for instance when received at the media storage server130, as part of the output by the media source105, or the like. The media stream150may be composed of a number of portions, which may be of a various length, size, and/or time duration, depending on the implementation. In some cases, each portion of the media stream150may represent a single packet, a group of packets composing a single frame, a group of packets composing multiple frames, for instance the frames between successive keyframes, or the like. Thus, when reference is made herein to a portion of the media stream150, it should be understood that the “portion” may encompass any suitable part, or all, of a particular media stream.

Additionally, in some embodiments, the data rendered tamper-evident may include or consist of data not conventionally understood as being media data. This may include data streams which are a stream of events data, a stream of changelog data, a stream of sensor data, or of any other suitable type of data. For example, the data stream may be produced by an access controller, which catalogs events relating to access cards being read, access being granted or refused, and the changing status of a door or similar resource controlled by the access controller. By way of another example, the data stream may be produced by a computing device which monitors changes to one or more documents or repositories of documents. Accordingly, in the embodiments and/or examples described herein, the term “media stream” may be replaced with the term “data stream” where appropriate, and references to particular types of media streams, including video streams, audio streams, and the like, may be replaced by references to streams of other types of data, as appropriate.

With reference toFIG.2, an example implementation of a blockchain200is illustrated. For the purposes of the present disclosure, the blockchain200is shown as being composed of a series of blocks210,220,230,240,250(though it should be understood that the blockchain200may include any number of blocks). Each block210-250can store any suitable amount of data in a payload portion, indicated here by212,222,232,242,252. In many embodiments, the payload portions212,222,232,242,252of the blocks210,220,230,240,250of the blockchain200are used to store data relating to transactions, whether the transactions are exchanges of so-called “cryptocurrencies” or more conventional types of assets. In some other embodiments, though, the payload portion is used to store information which a user wishes to render immutable, for instance a contract, a deed or other official record, and the like. Transactions, as well as other information stored in the payload portion212,222,232,242,252of the blocks210,220,230,240,250of the blockchain200, are typically recorded after validation of a signature of the user or entity storing the information in the blocks210,220,230,240,250. Different signature approaches may be used depending on the implementation; in many embodiments, a public key cryptography scheme, using public-private key pairs, is employed. A user wishing to record a transaction (or other information) on the blockchain200signs the transaction with their private encryption key. Before accepting the transaction, the entities responsible for the blockchain200will validate the signed transaction using the user's public encryption key; only once the transaction is validated is the transaction accepted and incorporated into the blockchain200. It should be understood that other applications for digital signatures within the blockchain200are also considered.

To link each block210-250with the preceding block in the chain, each block210-250also includes a progenitor field, indicated here by224,234,244,254(although not illustrated here, block210would also include a progenitor field). The progenitor field224of block220includes therein a numerical representation216of the previous block in the chain, in this case, block210. The remaining blocks230-250similarly contain a numerical representation, indicated here as numerical representations226,236,246, to be stored in their respective progenitor fields234-254.

Using block220as an example, the numerical representation216is produced using a particular function or transformation, applied to the block210, and is then stored in progenitor field224. In some embodiments, the numerical representation216is produced by a hash of block210, by a hash of part of block210, by a digitally-signed hash of all or part of block210, or may be produced using any other suitable function or transformation applied to the block210. The numerical representation216identifies the block210and its contents-including the data stored in the payload portion212and the data stored in the progenitor field of the block210-sufficiently uniquely as to ensure that any change in the block210or its contents would also result in a change in any newly generated numerical representation216, using the same function or transformation. Because the numerical representation216for the block210is stored in the block220, and thus outside the block210, any tampering with the block210could easily be detected by comparing a newly generated numerical representation216with the numerical representation216stored in the progenitor field224of the block220. In this fashion, any block210-250in the blockchain200can be used to verify the integrity of all preceding blocks of the blockchain200.

With reference toFIG.3, a data architecture300for storing digital media within the media storage server130is illustrated. The data architecture300is composed of a number of data structures310,320,330, each of which includes a payload field312,322,332and a metadata field314,324,334. The data architecture300borrows from the approach illustrated inFIG.2to allow for tamper-evident storage of the media streams150using, inter alia, successive progenitor blocks to form a blockchain, as well as other tamper-evidencing features, which will be described in greater detail hereinbelow.

In one example implementation, the data architecture300is implemented at least in part on a Microsoft® Azure® cloud-based platform, which provides data structures known as Blobs: each Blob can store an arbitrary amount of data in its payload field, and has a metadata field with a fixed size (e.g., 8 KB), in which properties and custom information about the Blob can be stored in the form of name-value pairs. In another example implementation, the data architecture300is implemented at least in part on an AWS® Simple Storage Service® cloud-based platform, which provides data structures known as buckets: each bucket can store an arbitrary amount of data in its payload field, and has a metadata field with a fixed size (e.g., 2 KB), in which properties and custom information about the bucket can be stored in the form of name-value pairs. In a further example implementation, the data architecture300is implemented using a file system (e.g., ext4 or similar) which supports the storage of extended file attributes (sometimes referred to as xattr) in conjunction with files on a storage medium. Storing properties and other custom information, for instance as name-value pairs, within the extended file attributes effectively allows the extended file attributes to be used as metadata for the files. The storage size available within the extended file attributes may vary from one file system to another; in the case of ext4, the space available in the extended file attributes may be 4 KiB. In a still further example implementation, the data architecture300is implemented using forks, for instance the alternate data stream (ADS) functionality of the NTFS™ file system (typically used with the Windows® operating system), whereby multiple data contents can be associated with a single file. The metadata field314,324,334of the data structures310,320,330can be stored via a fork, so as to be associated with the file data in the main fork or stream, which represents the payload field312,322,332. In this fashion, the data architecture300may be implemented via cloud-based platforms, or by way of locally-operated storage, or by any suitable combination thereof. In a yet further implementation, the particular storage mechanism (file structure, cloud platform, etc.) may not have an implicit system for storing a payload in association with metadata. However, the data architecture300may be implemented by way of separate files stored within the storage mechanism, with one file acting as the payload field, and another file acting as the metadata field. The separate files may be associated to one another in any suitable fashion, so as to link the file acting as the payload field with the file acting as the metadata field.

In some embodiments, the media source105is configured for packaging the media provided to the media storage server130in accordance with the data architecture300, so that the media stream150is composed of data structures similar in form to the data structures310,320,330of the data architecture300. Put differently, the media storage server130may receive properly formatted data structures from the media source105, which can be stored in the data architecture300as—is, thereby ensuring that the tamper-evident mechanisms present in the data structures310,320,330(discussed in greater detail hereinbelow) are enforced from the edge of the media streaming system100. In some cases, the media source105receives part or all of the information to be included in the media stream150from other sources, and packages the additional information, alongside, for example, original media produced by the media source105, into properly formatted data structures for storage in the data architecture300. This may facilitate interplay with media sources105which are not configured to generate some of the data necessary for implementation of the data architecture300, but which can be configured to forward provided metadata as part of the media stream150.

In some other embodiments, the data architecture300may be generated within the media storage server130, or an intermediary device (e.g., the media archiver135), in a fashion that is transparent to the end-user and the media source105. That is to say, the media source105may not be aware that the media stream150it is providing to the media storage server130will be stored within the data architecture300, and need not package the contents of the media stream150in any particular way to facilitate or enable the contents of the media stream150to be stored therein. Similarly, users of the media streaming system100may not be aware that the digital media being provided by the media sources105is being stored by the media storage server130within the data architecture300. For example, a user installs one or more cameras110, microphones120, or other media sources105and couple the media sources105to the media storage server130without any particular configuration settings relating to the storage of digital media produced by the media sources105. Upon receipt of the media stream150, the media storage server130performs the necessary conversions and/or transformations to cause the digital media of the media stream150to be stored within the data architecture300. In still further embodiments, the media source105and the media storage server130may collaborate to format the digital media, and to produce the data to be stored in the metadata fields314,324,334, in accordance with the data architecture300.

With reference toFIG.4, example implementations of the data structures310,320of the data architecture300are illustrated: the data structures310,320are used to store portions of the media stream150in a tamper-evident manner. Although only the data structures310,320are illustrated inFIG.4, it should be understood that the data architecture300may include any number of data structures. It should also be understood that the data structures310,320, as well as any others, could be substantially identical in form, or may vary from one to the next. The data structure310stores, in the payload field312thereof, a media file412, which represents a portion of the media stream150. The media file412may be a substantially identical copy to the digital media as provided by the media stream150, or may be a repackaging or conversion thereof. For example, in some embodiments, the media stream150provides video media from the camera110in a raw format or similar format. The media storage server130converts the raw format video media to a compressed format, for example H.264, H.265, or a similar compression standard, which may be stored in an MP4 container, thereby forming the media file412. The media file412is then stored in the payload field312of the data structure310. The media file412can be delimited in any suitable fashion, and thus may represent any suitable portion of the media stream150. In some cases, the media file412is delimited based on being a certain size (i.e., a certain number of bytes). In some other cases, the media file412is delimited based on representing a certain time duration (i.e., a certain number of seconds, minutes, or hours) of the media stream150. Other approaches for delimiting the media file412are also considered. For instance, the media file412may be delimited based on key frames (also known as “intra-frames” or “i-frames”), based on when the media stream150changes to a different scene, or based on when other changes in the media stream take place (different origin, different requestor, different subject, etc.).

The data structure310also stores, in the metadata field314thereof, a set of parameters414. The set of parameters414can include any suitable number of parameters which are associated with the portion of the media stream150stored as the media file412, which describe, relate to, and/or secure the portion of the media stream150as stored in the media file412. Some of the parameters which compose the set of parameters414may be determined or otherwise provided by the media source105which provides the media stream150, and some of the parameters may be determined by the media storage server130, for instance upon receipt of the media stream150. As will be described in greater detail hereinbelow, the set of parameters414includes therein parameters which facilitate detection of tampering of the media file412, of other media files in certain others of the data structures in the data architecture300, and of the set of parameters414itself. The set of parameters414may also include parameters which facilitate identification of the source of the media file412, which facilitate playback of the media file412, which identify the media stream150, the portion thereof stored as the media file412, the entity responsible for the media source105, and the like. In some embodiments, the set of parameters414is stored as a number of name/value pairs, that is to say, as combinations of a name of a parameter and a value for said parameter (e.g., “length:: 0h20m30s”, “source:: camera_001”, etc.), though other approaches for storing the parameters in the set of parameters414are also considered. The set of parameters414may be assembled by the media storage server130at any suitable time: in some embodiments, the set of parameters414may be fully assembled at or before the time at which the media file412is stored in the payload field312, and in some other embodiments, the set of parameters414may only be fully assembled at some time after the media filed412is stored in the payload field312.

In some embodiments, additional information may also be stored in the payload field312and/or in the metadata filed314. For instance, in some embodiments, the payload field312also stores a copy416of the set of parameters414. By storing the set of parameters414in the payload field312, in the form of the copy416, a party wishing to perform playback of the portion of the media stream150stored as the media file412may, by downloading the payload field312of the data structure310and not the metadata field314, nevertheless acquire both the media file412and the set of parameters414. In this fashion, a single operation or interaction with the data architecture300allows the party to obtain both the media file412and the set of parameters414, thereby reducing the number of interactions with the data architecture300required to acquire the desired data. This approach may be of interest in cloud-based environment which limit the number of interactions that can be performed on stored data, or which adjust the cost of hosting data based on how frequently the data is the subject of interactions. In some other cloud-based environments, both the media file412and the set of parameters414may be acquired in a single request to the media storage server130, with the request specifying that the data in the payload field312and in the metadata field314be returned.

In some embodiments, the set of parameters of a particular portion of the media stream150may be stored indirectly in the metadata field314. For example, the data structure320stores another portion of the media stream150as media file422in its payload field322, but a set of parameters424relating to the portion of the media stream150stored as media file422may be stored outside the data structure320. Instead, the metadata field324of the data structure320stores an indirection426-sometimes referred to as a shortcut or a symbolic link (symlink)—which points to the location at which the set of parameters424is stored. The set of parameters424may be stored indirectly in the metadata field324in situations in which the set of parameters424would occupy more storage space (i.e., more bytes) than is available in the metadata field324, in which writing the set of parameters424to the metadata field324(or reading the set of parameters424from the metadata field324) would be too slow of an operation, or in other appropriate situations. Another approach that may be employed in certain embodiments, for instance when a particular set of parameters would occupy more storage space than is available in a relevant metadata field, is to apply a compression algorithm to the particular set of parameters to produce a compressed set of parameters. Thus, for instance, the set of parameters414stored in the metadata field314may be compressed via a compression algorithm prior to being stored in the metadata field314. In such cases, the copy416of the set of parameters414may be uncompressed, or may also be compressed, depending on the particulars of the implementation.

As noted hereinabove, the media storage server130may also serve the digital media stored therein for playback, transfer, or the like, whether to a third-party computing device requesting digital media from the media storage server130, or at the media storage server130itself, which may be configured for receiving user input to request playback of certain media. Requests to the media storage server130to provide digital media stored therein may be sent from any suitable device, including internally to the media storage server130, for instance based on pre-established schedules or the like, and the request may take any suitable form. Upon receipt of a request to provide digital media, the media storage server130can search the metadata fields of the various data structures310,320,330, to locate the appropriate digital media to server to the requestor. By way of an example, when a particular request specifies that video from a particular camera is desired (e.g., from a camera identified as “camera_012”), the media storage server130can use camera identifiers stored in the metadata fields314,324,334to identify the appropriate one of the data structures310,320,330from which to acquire the digital media to provide to the requestor. By way of another example, when a particular request specifies that video taken at a particular moment in time is desired (e.g., from 9 PM on a particular Tuesday), the media storage server130can use timestamps or recording times stored in the metadata fields314,324,334to identify the appropriate one of the data structures310,320,330from which to acquire the digital media to provide to the requestor. By way of a further example, when a particular request specifies a particular location (e.g., as latitude/longitude coordinates, by a civil address, or the like), the media storage server130can use location data stored in the metadata fields314,324,334, in some cases in cross-reference with maps or similar geographical information, to identify cameras near the specified location, and in turn to identify the appropriate one of the data structures310,320,330from which to acquire the digital media to provide to the requestor.

When providing the media to the requestor, the media storage server130provides the data stored in the relevant payload field312,322,332to the requestor. In some embodiments, the media storage server130also provides the data stored in the related metadata field314,324,334. In some other embodiments, for instance the embodiment of data structure310(in which the copy416of the set of parameters414is also stored in the payload field), the media storage server130can provide both the media file412and the set of parameters414(as present in the copy416) by supplying the payload field312to the requestor. In this fashion, the number of operations performed on the data architecture300can be reduced by storing the copy416in the payload field312.

With reference toFIG.5A, an example of the set of parameters414is illustrated as being composed of a number of categories: headers510, a file hash520, a signature530, and in some embodiments a file length525. It should be understood, however, that the division of the set of parameters414shown inFIG.5Ais for the purposes of illustration, and does not necessarily mean that the set of parameters414, as physically stored within the metadata field314, are subdivided into specific categories or the like. Indeed, the set of parameters414may include additional or fewer categories of parameters, and the parameters themselves may be stored within the metadata field314in any suitable fashion, depending on the implementation.

The headers510are composed of various parameters which provide identifying information about the media file412and information to facilitate playback of the media file412. For example, the headers510may include any one or more of a unique identifier for the media stream150, a unique identifier for the source of the media stream150(e.g., the relevant media source105), a unique identifier for groups or families of media streams150which may be related to one another, one or more timestamps associated with the portion of the media stream150which is stored as the media file412, a length, size, or duration of the media file412, and the like. By way of another example, the headers510include the location (e.g., timestamps) of one or more keyframes of the media file412, to facilitate playback of and seeking within the media file412. Additionally, the headers include at least a progenitor parameter512; as described in relation toFIG.2, a progenitor in a blockchain is a numerical representation of the previous block in the chain. In this fashion, the progenitor parameter512is based on the previous data structure of the data architecture300. Thus, the progenitor parameter for the data structure330would be based on at least part of the data structure320, the progenitor parameter for the data structure320would be based on at least part of the data structure310, and the progenitor parameter512for the data structure310would be based on at least part of some previous data structure in the data architecture300. It should be noted that each subsequent data structure of the data architecture300stores a subsequent portion of the media stream150. As a result, the progenitor parameter for a particular data structure is based on at least part of a previous data structure in the data architecture300, which stores a previous portion of the media stream150.

The progenitor parameter512can be any suitable numerical representation of the previous data structure, including being based on a hash of part or all of the previous data structure, using any suitable hashing algorithm. In some embodiments, the progenitor parameter512is obtained by hashing the previous data structure as a whole. For example, the progenitor parameter for the data structure320is obtained by hashing the data structure310as a whole. In some other embodiments, the progenitor parameter512is obtained by hashing a particular portion of the previous data structure. By way of an example, the progenitor parameter512is obtained by hashing the metadata field of the previous data structure. By way of another example, the progenitor parameter512is obtained by hashing a subset of the metadata field of the previous data structure, for instance the header of the previous data structure. Other approaches for generating the progenitor parameter512are also considered.

Approaches in which the progenitor parameters, including the progenitor parameter512, are obtained by hashing part or all of the data in the metadata field314may, in certain cases, provide certain benefits over approaches in which the progenitor parameters are obtained by hashing larger portions of the data structures310,320,330. Because the metadata field of any given data structure310,320,330is likely to be of a smaller size than the payload field, hashing the metadata field may be a less computationally- and time-intensive operation, thus simplifying the task of generating the progenitor parameter. This may be of particular interest when rewinding, browsing, or validating the blockchain of the data architecture300: rather than recomputing progenitor parameters by hashing large amounts of data present in the payload fields312,322,332, only the data of the metadata fields314,324,334is used to recompute the progenitor parameters. Additionally, in cases where the media files412,422,432are moved off-chain (i.e., deleted from the payload fields312,322,332, for instance to reduce storage costs), the metadata fields314,324,324can be retained to keep the blockchain intact, and thus suitable for ensuring that the contents of the media stream150previously stored in the data architecture300are free from tampering. In cases in which the duration of the media stream150to be stored in the payload fields312,322,332is not predetermined, certain parameters to be stored in the metadata field314, as part of the set of parameters414, may be unknown (e.g., the length or size of the media file412). If the subsequent progenitor parameter for the data structure310is based only on data stored in the metadata field314, it may be computationally feasible to re-update the sets of parameters of the data structures310,320,330of the data architecture300at a time after storing the portions of the media stream150, since the computationally intensive hashing of the media files (e.g., the media file412) has already been performed and stored in the file hash520. This may also facilitate linear storage of multiple media files412, since the sets of parameters414are stored separately in the metadata field(s)314.

Of course, once the data structure310has the media file412and the set of parameters414stored therein, a subsequent progenitor parameter, for instance for the data structure320, may be generated in accordance with any of the previously-described embodiments. Notably, in embodiments in which the subsequent progenitor is generated based on the set of parameters414stored in the metadata field314, the subsequent progenitor is based on the file hash520, the progenitor512, the signature530, and in some embodiments, the file length525. As a result, the subsequent progenitor can be used to detect tampering in both the media file412(due to changes in the media file412affecting the file hash520, and therefore the subsequent progenitor), in the set of parameters414(due to changes in the set of parameters affecting the subsequent progenitor), and attempted tampering with the data architecture300as a whole (due to the inability to produce new signatures using the same cryptographic key as the one used to produce the signatures530).

The set of parameters414also includes the file hash520, which is a hash of part or all of the media file412(for simplicity, the foregoing discussion will refer to the file hash520as being a hash of the media file412, though this should not be understood as limiting). By storing a hash of the media file412within the set of parameters414, any tampering with the media file412would be detectable by comparing a new hash of the media file412with the file hash520stored in the metadata field314as part of the set of parameters414. In some embodiments, the file hash520may be an incremental hash (e.g., obtained using the IncrementalHash class of the .NET™ API by Microsoft®): as the media file412is stored in the payload field312, concurrently with the media stream150being received at the media storage server130, the stored portions of the media file412may be hashed to progressively generate the file hash520in increments. In this fashion, the hashing of the media file412can begin prior to the totality of the media file being stored in the payload field312, which may reduce the amount of time required to complete generating the set of parameters414. In some embodiments, the particular hashing function used may be selected to avoid certain potential vulnerabilities or attack vectors, for instance a length extension attack, or the like.

In some embodiments, the set of parameters414also includes the file length525. Illustrated here as a separate element within the set of parameters414, in some embodiments the file length525may be included as part of the headers510, whereas in some other embodiments, the file length525is stored within the metadata field314separately from the headers510. The file length525is a value of the length of the media file412, as stored within the payload field312. The file length525may represent the length of the media file412in any suitable fashion, using any suitable numerical representation, as appropriate. In some cases, the presence of the file length525may be used to assess what proportion of the media stream150was successfully saved into the media file412, for instance in cases in which transfer of the media stream150to the media storage server130is interrupted.

The set of parameters414also includes the signature530, which is a digital signature produced using a cryptographic key or other cryptographic secret of an authority which certifies the authenticity of the media stream150and/or the media file412. The signature530may be produced using any suitable approach, depending on the implementation. In some embodiments, the signature530is produced by applying the cryptographic key to a hash of some or all of the remaining data stored as part of the set of parameters414in the metadata field314. For example, a hash of the headers510, the file hash520, and, in some embodiments, the file length525, may be signed using the aforementioned cryptographic key. This approach may, as described hereinabove, assist in rendering the set of parameters414tamper-evident, as changes to the set of parameters414would be detectable via a comparison to the signature530, and the signature530would not be reproducible without the cryptographic key used to generate it.

In some other embodiments, the signature530may additionally, or alternatively, be obtained by encrypting a hash of the media file412with a cryptographic key. The hash of the media file412used to produce the signature530may be the same hash stored as the file hash520, or may be a different hash. For instance, the signature530may generated using a hash of a portion of the media file412, whereas the file hash520may be a hash of the whole of the media file412. Other approaches are also considered.

In some embodiments, the signature530is produced using the private encryption key of the media source105which produced the media stream150, for instance embodiments in which the media source105is provided with a strong identity. The signature530is provided to the media storage server130concurrently with the media stream150, so that the media storage server130can store portions of the media stream150as media files412, and store the concurrently-provided signatures from the media source105as the signature530. In some other embodiments, the signature530is produced using the private encryption key of the media storage server130or of an archiver operating separately from or as part of the media storage server130, for instance upon receipt of the media stream150by the media storage server or the archiver. In some further embodiments, the media source105and/or the media storage server130are configured for being operated by specific users, for instance by logging into the media source105or the media storage server130, by providing the media source105or the media storage server130with a physical security token (e.g., a Yubikey™), or the like, and the signature530may be produced using a private encryption key associated with the user, for instance a private encryption key stored on the user's Yubikey. Other approaches for generating the signature530, using any other suitable asymmetric encryption approach, are also considered.

As mentioned hereinabove, the file hash520may be generated incrementally, as the media file412is stored in the payload field312. In some embodiments, other parts or all of the set of parameters414may also be generated as the media file is stored in the payload field312. Put differently, the set of parameters414, in whole or in part, may be generated over multiple iterations during the acquisition of the media stream150which is stored as the media file412. For example, some or all of the headers510may be generated early in the acquisition process of the media stream150, for instance substantially concurrently with receiving the first or first few packets of the media stream150. Others of the headers510may be changed over the course of receiving the media stream150, for instance elements within the headers510relating to the length, duration, etc. of the media file412. By way of another example, the file length525and the signature530may be recomputed periodically during the course of receiving the media file412. Thus, the file length525may be re-evaluated and modified as the media stream150is being received and stored as the media file412, and the signature530may be re-generated periodically in response to changes in the headers510, the file hash520, the file length525, and the like.

Thus, in some embodiments, the various elements of the set of parameters414-including the header510, the file hash520, the file length525, and the signature530-collaborate to render the media file412, as well as the data structure310, tamper-evident in at least the following ways. First, tampering with the media file412directly can be detected based on the file hash520and/or the file length525stored in the metadata field314: if the media file412is changed, a newly generated hash of the media file412would differ from the file hash520, and/or the length of the media file412would not match the value stored in file length525, thereby evidencing the tampering. Second, tampering with the headers510and/or the file hash520can be detected based on the signature530stored in the metadata field314: because the signature530is generated by applying a cryptographic key to a hash of the headers510and the file hash520, if the headers510and/or the file hash520are changed, a newly generated signature generated in the same manner would differ from the signature530, thereby evidencing the tampering.

Additionally, depending on the implementation, it may be difficult or impossible to obtain the cryptographic key used to produce the original signature530, serving as a further defense against tampering with the headers510and/or the file hash520. Third, tampering with any part of data structures preceding a particular one of the data structures, (e.g., preceding to the data structure310) can be detected based on the progenitor parameter512stored in the headers510: if one or more of the preceding data structures are changed, a newly generated progenitor parameter would differ from the progenitor512, thereby evidencing the tampering. Similarly, tampering with the data structure310can be detected using the progenitor parameter stored in the data structure320. Moreover, attempted tampering with the whole of the blockchain formed by the metadata fields of the data structures of the data architecture300can be detected by validating the signatures530, which cannot be replicated without access to the private key(s) of the media source(s)105which produced the media stored in the data architecture300.

Validation of media stored in the data architecture300can therefore be performed on these basis, namely by producing new hashes, signatures, progenitors, or the like, and comparing with those already stored as part of the data architecture300. Additionally, in cases in which the tamper-evident mechanism involves a digital signature, a certificate associated with the signing authority can be used to validate the signature. For example, a certificate associated with the media source105can be used to validate the signature530of the data structure310. The certificate may be provided alongside the media field412when served to a requesting entity, so that the requesting entity may itself also validate that the signature530is valid. In certain cases, because the sets of parameters are stored in the metadata fields314,324,334of the data structures310,320,330, separately from the media stored in the payload fields312,322,332, the metadata field314,324,334can be extracted from the data architecture300(or copied therefrom) to be provided to a third-party or other external part to audit or otherwise validate the data architecture300. Moreover, the data architecture300as a whole can be validated based on the blockchain formed by the metadata fields of the data structures of the data architecture300.

In some embodiments, validation of media files (e.g., the media file412) can be performed upon ingestion by the media storage server130. For example, the media source105can produce the media stream150as a plurality of data structures310,320,330, which are then sent to the media storage server130. Upon receipt, the media storage server can perform one or more of the above-described validations on the media stream, prior to storing the data structures310,320,330within the data architecture300. This may be referred to as performing “in-line” validation. In some other embodiments, similar validations of the media file412can be performed after ingestion, at any suitable time.

With additional reference toFIG.5B, a non-limiting example of the headers550is illustrated, and is described in relation to the data structure310. The headers550include a streamId552, which is a universally-unique identifier (UUID) which identifies the stream to which the data structure310belongs. The headers550also include a source554, which is a UUID representing the physical source of the information in the data structure310(e.g., a UUID associated with the CMOS sensor or a microphone of the camera110, a UUID associated with the transducer of the microphone120, etc.). In this fashion, the source554represents the edge-most producer of the data architecture300. Depending on the particular application, the context in which the source554is used will vary. In some embodiments, additional information relating to the physical source of the information in the data structure310may be stored in a separate data architecture. For instance, configuration data relating to the media source105and/or the physical sources of the media stream150may be stored in a separate data architecture, which is also assigned the same UUID as that used for the source554in the data architecture300, relating to the media produced by the media source105and/or the physical sources of the media stream150. The headers550also include a parent556, which serves to indicate a logical hierarchy between different data architectures. The inclusion of the parent556gives data architectures a recursive nature, and a given data architecture will generally have a single parent556during its lifetime. For example, a first data architecture stores an original or “clean” version of a video file, and a second data architecture stores a modified version of the video files, in which a privacy blurring effect is applied to the persons depicted in the video. In this example, the second data architecture will indicate the first data architecture as its parent556, and the first data architecture will indicate a null value in the parent556. The headers550also include a home558, which is a UUID representing an authority on the state of a set of data architectures. In certain embodiments, the home558indicates which of the data structures in the data architecture300is currently at the head of a group of data architectures, and can be used to identify the latest data structure in the group of data architectures. The headers550also include a family560, which is a UUID indicating the family to which the data architecture300belongs. In some embodiments, families are an administrative hierarchy. The headers550also include a metadata562, which is a UUID identifying the stream of metadata (i.e., a separate data architecture which stores only metadata) associated with the data architecture300. The headers550also include a progenitor564, which is a hash of the data structure preceding the data structure310in the data architecture300. For example, the hash may be a SHA256 hash. The headers550also include a signatory566, which is a certificate thumbprint for the certificate containing the signing key for the data structure310. The certificate itself can be found through any suitable approach, for instance based on relevant metadata. For example, the signatory566may be generated using SHA256, SHA1, or any other suitable approach. The headers550also include a time of record (ToR)568, which is a timestamp at which the data structure310was created, which may follow any suitable timekeeping standard. The headers550also include a time of fact (ToF)570, which is a timestamp representing a start time for the portion of the media stream stored in the media file412. The headers550also include a duration of fact (DoF)572, which is a value indicating the duration between the start time for the portion of the media stream150stored in the media file412, as indicated by the ToF570, and the end time for the portion of the media stream stored in the media file412. Put differently the DoF572is an indication of the length of the portion of the media stream150stored in the media file412. The headers550also include ancestors574, which is a list of data structure identifiers (e.g., hashes of previous data structures within the data architecture300, which may need to be re-merged into the blockchain of the data architecture300) used for resolving potential forks in the chain of data structures310,320,330. Additionally, the headers550may include one or more extension headers576, which may include any additional information necessary for properly storing and/or performing playback of the media file412, for instance as described previously in relation toFIG.4.

With reference toFIG.6A, another embodiment of a data architecture600for tamper-evident storage of media is illustrated as including a data structure610composed of a payload field612and a metadata field614. It should be noted that the data architecture600may include any suitable number of data structures, thoughFIG.6Aillustrates only the data structure610. In certain cloud environments, storing the same amount of data in a single data structure (e.g., a single Microsoft® Azure® blob) may be more cost-effective, result in faster access times, and/or facilitate streaming-based playback of the data vis-à-vis storing the data in multiple data structures. To this end, the payload field612is configured for storing multiple portions of the media stream150therein, illustrated here as media files622,632,642,652. Each media file622,632,642,652may be of any suitable length, size, or duration, and successively stored ones of the media files622,632,642,652may be temporally subsequent to one another. In some embodiments, the media files622,632,642,652may be of substantially similar length, size, or duration, or may of varying length, size, or duration, as appropriate. Additionally, the metadata field614is used for storing sets of parameters associated with each of the media files622,632,642,652: the metadata field614may store the sets of parameters in a concatenated fashion, in a compressed fashion, or the like.

In some embodiments, the payload field612itself may implement an internal data structure for storing therein copies of the sets of parameters624,634,644,654for each of the media files622,632,642,652. The copies of the sets of parameters624,634,644,654may be stored, as part of the internal data structure, in conjunction with their respective media file622,632,642,652, for instance so that the set of parameters624is disposed logically ahead of the media file622to which it relates, and so on. In some embodiments, in addition to storing the copies of the sets of parameters624,634,644,654ahead of the media files622,632,642,652, copies of the sets of parameters624,634,644,654may also be disposed logically following the media files. Additionally, in situations in which part of the media stream150is not received by the media storage server130, for instance due to poor connectivity, unavailability of the media storage server130, or the like, the presence of the copies of the sets of parameters624,634,644,654prior to the media files622,632,642,652may serve as a backup in the event that the copies of the sets of parameters624,634,644,654stored following the media files622,632,642,652are not received by the media storage server130. It should be understood that the sets of parameters stored in the metadata field614are substantially identical to the copies of the sets of parameters624,634,644,654stored in the payload field612. For example, the sets of parameters stored in the metadata field614may be stored in a compressed format, whereas the copies of the sets of parameters624,634,644,654stored in the payload field612may be stored in an uncompressed format, even though the actual sets of parameters stored both the payload field612and the metadata field614are identical.

With additional reference toFIG.6B, in some embodiments, the metadata field614stores, in addition to the sets of parameters624,634,644,654, file length values626,636,646,656(which may replace, or be stored in addition to, any file lengths525which may already be part of the sets of parameters624,634,644,654). In some embodiments, including the embodiment illustrated inFIG.6B, the metadata field614stores concatenations of the sets of parameters624,634,644,654and respective file length values626,636,646,656in an array or similar structure. For example, the set of parameters624is concatenated with the file length value626, the set of parameters634is concatenated with the file length value636, and so on, as different entries in the array stored in the metadata field614. For instance, if the file length525is already present within the sets of parameters624,634,644,654, the file length values626,636,646,656can be obtained from the sets of parameters624,634,644,654and then concatenated thereto. The file lengths525may then be removed from the sets of parameters624,634,644,654, or the sets of parameters624,634,644,654may be left intact, as appropriate.

By including the file length values626,636,646,656in the metadata field614, the media storage server130can look to the metadata field614to determine which of the media files622,632,642,652to serve to a requestor who specifies a particular portion of a requested media stream. Put differently, if a user requests that the media storage sever130provide a media stream at a time of 00:13:37 (i.e., 13 minutes and 37 seconds), the media storage server130can use the file length values626,636,646,656to determine which of the media files622,632,642,652should be served to the user to provide the user with the media stream at the requested time. In this fashion, the metadata field614can be used as a seek table to facilitate locating and streaming relevant media to requestors. In some embodiments, the metadata field614can include additional information, such as the location of one or more keyframes within the media files622,632,642,652, or the like, to facilitate playback of the media files622,632,642,652.

With reference toFIG.7, there is illustrated a schematic diagram of an example computing device700. As depicted, the computing device700includes at least one processor710, a memory720, and program instructions730stored within the memory720, as well as input and output interfaces (I/O interfaces)702and704, respectively. For simplicity, only one computing device700is shown; the media sources105, the media storage server130, and/or any other computing devices included as part of other elements of the present disclosure (e.g., subcomponents or sensors of the camera110) may be embodied by one or more implementations of the computing device700, which may be the same or different types of devices. The components of the computing device700may be connected in various ways including directly coupled, indirectly coupled via a network, and distributed over a wide geographic area and connected via a network, for instance via a cloud computing implementation.

The I/O interfaces702,704may include one or more media interfaces, via which removable media or other data sources may be coupled, one or more network interfaces, or any other suitable type of interface. The I/O interfaces702,704of the computing device700may additionally, in some embodiments, provide interconnection functionality to one or more input devices, such as a keyboard, mouse, camera, touch screen and a microphone, or with one or more output devices such as a display screen and a speaker, for instance devices via which a user may interact with the media storage server130. In embodiments in which the I/O interfaces702,704include one or more network interfaces, the network interface(s) of the computing device700may enable the computing device700to communicate with other components, to exchange data with other components, to access and connect to network resources, to serve applications, and perform other computing applications by connecting to a network (or multiple networks) capable of carrying data including the Internet, Ethernet, plain old telephone service (POTS) line, public switch telephone network (PSTN), integrated services digital network (ISDN), digital subscriber line (DSL), coaxial cable, fiber optics, satellite, mobile, wireless (e.g. Wi-Fi, WiMAX), SS7 signaling network, fixed line, local area network, wide area network, and others, including any combination of these.

The processor710may be, for example, any type of general-purpose microprocessor or microcontroller, a digital signal processing (DSP) processor, an integrated circuit, a field programmable gate array (FPGA), a reconfigurable processor, a programmable read-only memory (PROM), or any combination thereof. The processor710may be configured for executing the instructions730stored within the memory720. The memory720may include a suitable combination of any type of computer memory that is located either internally or externally such as, for example, random-access memory (RAM), read-only memory (ROM), compact disc read-only memory (CDROM), electro-optical memory, magneto-optical memory, erasable programmable read-only memory (EPROM), and electrically erasable programmable read-only memory (EEPROM), Ferroelectric RAM (FRAM) or the like.

In certain embodiments, the computing device700is operable to register and authenticate users (using a login, unique identifier, and password for example) prior to providing access to applications, a local network, network resources, other networks, and network security devices. The computing device700may serve one user or multiple users.

For example, and without limitation, the computing device700may be a server, network appliance, set-top box, embedded device, computer expansion module, personal computer, laptop, personal data assistant, cellular telephone, smartphone device, UMPC tablets, video display terminal, gaming console, electronic reading device, and wireless hypermedia device or any other computing device capable of being configured to carry out the methods and/or implementing the systems described herein.

With reference toFIG.8, there is illustrated a flowchart of a method800for tamper-evident storage of a media stream, for instance the media stream150. In some embodiments, the method800may be implemented in a cloud-based environment. The method800may be performed using data structures, for instance the data structures310,320,330of the data architecture300, which comprise a payload field and an associated metadata field. In some embodiments, the method800is performed, in whole or in part, by one of the media sources105, for instance the camera110, the microphone120, or another media device, including devices which produce original media. In some other embodiments, the method800is performed, in whole or in part, by the media storage server130, or by a utility operating thereon or in concert therewith, for instance a media archiver, which may operate on the media storage server130or on another device, for instance coupled to the media source105and to the media storage server130. In some further embodiments, the method800is performed collaboratively by the media source105, the media storage server130(and/or any utility operating thereon, for instance a media archiver), and/or by any other entity forming part of the media streaming system100. It should be understood that the steps described hereinbelow forming part of the method800need not be performed in any particular order, unless otherwise specified.

As part of step810, the method800comprises obtaining a portion of the media stream from a media source, for instance a part of the media stream150from one of the media sources105. In some embodiments, obtaining the media source is performed by acquiring the portion of the media stream as original media, for instance via a sensor or transducer of the media source105. The obtained media may be any suitable type of media, including images, video media, audio media, haptic media, or any suitable combination thereof.

As part of step820, the method800comprises determining a set of parameters to be stored in the metadata field of the data structure, for instance the set of parameters414to be stored in the metadata field314of the data structure310. The set of parameters414is associated with the portion of the media stream, which is to be stored in the payload field of the data structure310, for instance the payload field312. In some embodiments, determining the set of parameters414includes generating an identifying hash of the portion of the media stream150, for instance the file hash520. In some embodiments, the file hash520may be generated via incremental hashing. In some embodiments, determining the set of parameters414includes obtaining a digital signature, for instance the signature530, based on the portion of the media stream and on a cryptographic key. The signature530may be obtained using any suitable signing scheme, and which may vary based on the implementation, and may involve signing any suitable portion of the media filed412, for instance part or all of the file hash520. In some cases, the signature530is obtained using a cryptographic key associated with the media source105, for instance a private encryption key of the camera110or the microphone120. In some other cases, the signature530is obtained using a cryptographic key associated with the media storage server130or another entity associated therewith, for instance a media archiver, a gateway or bridge device, and/or a utility operating on the media storage server130. In some further cases, the signature530is obtained using a cryptographic key associated with a user, for instance as stored in a Yubikey™ or other physical security token. Other approaches for obtaining the signature530are also considered.

As part of step830, the method800comprises obtaining a progenitor parameter to associated with the set of parameters414, for instance the progenitor parameter512. The progenitor parameter512is based on a hash of at least part of a previous data structure which stores a previous portion of the media stream, and which was previous rendered tamper-evident. In some embodiments, the progenitor parameter512is based on a hash of a previous set of parameters associated with the previous portion of the media stream, for instance as stored in a payload field of the previous data structure. For instance, the progenitor parameters512may be based on a hash of the payload field of the previous data structure, which may include an identifying hash of the previous portion of the media stream, as stored in the payload field of the previous data structure.

As part of step840, the method800comprises incorporating the progenitor parameter512into the set of parameters414and storing the set of parameters414in the metadata field314of the data structure310. In some embodiments, the set of parameters414is compressed, using any suitable compression algorithm, prior to being stored in the metadata field314as a compressed set of parameters. In some embodiments, storing the set of parameters414in the metadata field314of the data structure310includes indirectly storing the set of parameters414, for example by storing an indirection in the metadata field314, which points to another file or data location, at which the set of parameters414are stored. The indirection may be of any suitable format, and may depend on the particulars of the file system used to implement the data architecture300.

As part of step850, the method800comprises storing the portion of the media stream150in the payload field312of the data structure310. In some embodiments, as part of step850, one or more copies of the set of parameters414is also stored in the payload field312, for instance as copy416. In some embodiments, multiple portions of the media stream150may be stored in a common payload field, for example the media files622,632,642,652stored in the payload field612. In these embodiments, multiple sets of parameters may also be stored in a common metadata field, for example the sets of parameters624,634,644,654stored in the metadata field614; storing the multiple sets of parameters624,634,644,654may take place as part of step830. In some cases, copies of the sets of parameters may also be stored in the payload field612.

As part of step860, the method800comprises generating a subsequent progenitor parameter based on the set of parameters414, which, in some embodiments, includes an identifying hash of the portion of the media stream150, for instance the file hash520. The generating of the subsequent progenitor parameter is performed after step840, as part of which the progenitor parameter512is incorporated into the set parameters414and may be performed before step850, concurrently therewith, or thereafter, as appropriate. As a result, the subsequent progenitor includes therein the progenitor parameter512, so that tampering with the progenitor parameter512would be detectable based on the subsequent progenitor parameter. The subsequent progenitor parameter may then be used in rendering a subsequent portion of the media stream150tamper-evident, for instance the portion of the media stream150stored as media file422in the data structure320. It should be understood that different parts of the steps840,850, and860, can be performed in any suitable order, as appropriate. For instance,

As previously mentioned, in some embodiments, part or all of the method800is implemented by the media sources105, for instance the camera110. In one example implementation, the camera110in question is capable of running custom applications of various types in addition to any standard video and/or image collection functionality. To this end, the camera110provides hooks or other interfaces which allow for a custom application to run on and/or interact with the standard functionality of the camera110. The hooks may include a “pull” hook, which enables the custom application to obtain, from the camera110, original media produced by the camera110. For instance, the custom application pulls the original media produced by the camera110, which may be in MP4 format, and formats the media in accordance with the data architecture300. The hooks may also include a “push” hook, which enables the custom application to supply, to the camera110, additional information about the media produced by the camera110. The additional information may be generated by the custom application, or obtained by the custom application, for instance from a web server or similar remote system (e.g., the media storage server130). For instance, the custom application pushes parameters from the set of parameters relating to the media to the camera110, so that the camera can bundle the parameters with the original media in accordance with the data architecture300. In this fashion, the custom application running on the camera110can assist the camera in producing the media stream150in accordance with the data architecture300, so that the media storage server130can take the data obtained from the media stream150, which is already formatted in the appropriate format to be stored with the data architecture300.

With reference toFIG.9, there is illustrated a media streaming system900, which comprises a client device905, a streaming server910, a trusted delivery service (TDS) server920, and the media storage server130. In some embodiments, the media streaming system900may be an extension of the media streaming system100, insofar as the media storage server130may concurrently receive media to store, in the form of the media stream150, and also serve media to external devices, such as the client device905. Although illustrated here as separate servers, it should be understood that the streaming server910, the TDS server920, as well as the media storage server130need not be separate physical servers. Rather, one or more of the streaming server910, the TDS server920, and the media storage server130may be implemented by one or more common physical servers. Additionally, or in the alternative, one or more of the streaming server910, the TDS server920, and the media storage server130may be implemented by a plurality of physical servers, in a distributed server arrangement, or the like, as appropriate.

The client device905may be any suitable type of device via which a user can request access to media stored in the media storage server130. The client device905may be a conventional desktop computer, a laptop computer, a portable electronic device (e.g., a smartphone), or any other suitable type of computing device. The client device905can request media from the media storage server130in any suitable fashion: the client device905may specify the name or other identifier of a particular media source105, a particular date and time, a location of interest, and the like.

The request for media from the media storage server130is received at a streaming server910. The streaming server910is configured for interpreting the request from the client device905, and for providing media to the client device905in a format suitable for the client device905. To this end, the streaming server910may employ a suitable streaming protocol, for instance HTTP Live Streaming (HLS), Dynamic Adaptive Streaming over HTTP (also known as MPEG-DASH), or the like, to serve video to the client device905. In some embodiments, upon receipt of the request from the client device905, or at a suitable time thereafter, the streaming server910provides the client device905with a playlist composed of one or more universal resource indicators (URIs) which point to the streaming server910, for instance to obfuscate the actual source of media (e.g., the media storage server130).

The streaming server910consults the TDS server920to identify the appropriate portions of the data architecture300from which to obtain the media requested by the client device905. To this end, the TDS server920maintains an index922, which is based on the sets of parameters stored in the metadata fields314,324,334of the data structures310,320,330of the data architecture300. Although referred to as an “index”, it should be understood that the index922may include other information than what would conventionally be understood to be contained in an index. For example, the index922may include information relating to the different media sources105, to the times and places from which media was acquired, and the like. In some cases, the index922includes copies of parts or all of the information stored in the metadata fields of the data structures310,320,330, that is to say, copies of the sets of parameters for the media files of the data structures310,320,330. Using the index922, the streaming server910can identify the appropriate one of the data structures310,320,330from which to obtain the media to be provided to the client device905without needing to query the metadata fields314,324,334of the data structures310,320,330in response to the request from the client device905. In some embodiments, the streaming server910and/or the TDS server920may nevertheless query the metadata fields314,324,334of the data structures310,320,330, for instance to validate the index922, to account for the index922being incomplete, or in cases in which the index922is not yet available.

Based on the index922, the TDS server920accesses the appropriate data structure(s) of the data architecture300, stored in the media storage server130, and provides the media to the streaming server910, for instance part or all of the media file412. Alternatively, the streaming server910can identify the appropriate data structure(s) of the data architecture300, stored in the media storage server130, to obtain the media, for instance part or all of the media file412. The streaming server910then serves the media file412to the client device905, for instance after converting or otherwise transforming the media file412to facilitate the transfer and/or streaming thereof. In some embodiments, the TDS server920can also provide, or the streaming server910can obtain, part or all of the set of parameters414, associated with the media file412, to the client device905. For example, once the TDS server920identifies the appropriate data structure(s), the TDS can obtain both the media file412and the set of parameters414, and provide both the streaming server910. The streaming server910may format the media file412and the set of parameters414by placing the set of parameters414in the header of one or more streams provided to the client device905. In cases in which the client device905is configured for interpreting or otherwise making use of the set of parameters414, the client device905may use the set of parameters to facilitate playback of the media file412, to obtain identifying information about the media file412, to validate one or more aspects of the media file412, and the like. In cases in which the client device905is not configured for interpreting or otherwise making use of the set of parameters414, the header of the stream provided from the streaming server910to the client device905can be ignored by the client device905.

In some embodiments, the media provided to the client device905is encrypted. In some embodiments, the encryption was performed at the time of acquisition, for instance by the media source105. In some other embodiments, the encryption was performed at the time of storage in the media storage server130, for instance by the media storage server130, or by another device antecedent to the media storage server130. In some further embodiments, the encryption is performed as part of the media delivery process (i.e., to the client device905), for instance by the media storage server130, the TDS server920, the streaming server910, or any other suitable device. In still further embodiments, the encryption is performed at any other suitable time, and by any other suitable entity. For example, the encryption may be a conventional encryption performed as part of the process of generating the media file412, for instance generating the one or more .MP4 files which compose the media file412.

In order to perform playback or otherwise use the provided media, the client device905is provided with the necessary decryption mechanisms (e.g., one or more cryptographic keys or “decryption keys”) for decrypting the media it obtains from the streaming server910. In some embodiments, the media file412is encrypted symmetrically, for example in cases in which the media file412includes video media. To enable the client device905to perform playback of the media file412, the client device905is provided with the requisite symmetric encryption key (i.e., the same symmetric encryption key which was used to encrypt the media file412). The symmetric encryption key can be provided to the client device905in any suitable fashion, for example as part of serving the media file to the client device905, included in a header or other metadata portion of the packets providing the media file412to the client device905, or the like. In some cases, the symmetric encryption key used is a symmetric encryption key generated by or otherwise associated with the media source105which produced the media stream150which was thereafter stored as the media file412. The symmetric encryption key may then be stored in a secure repository (e.g., Azure® Key Vault™), which may form part of the media storage server130, or may be a separate server, as appropriate. For instance, the media source105may periodically generate new symmetric encryption keys for encrypting the media stream150, and transfer, alongside the media stream150or separate therefrom, the symmetric encryption keys for storage, whether in the media storage server130or in a separate repository. Other approaches are also considered. For example, for other types of media, the media stream150and the media file412may be encrypted asymmetrically, and depending on the implementation, different encryption keys may be used. For example, the media stream150may be encrypted by the media source105using a public encryption key of the media storage server130, or of another entity involved in the media storage process (e.g., the135). Upon request to serve a portion of the media stream150, the media storage server130may use its private encryption key to decrypt the relevant media file412, then use the public encryption key of the client device905to re-encrypt the media file412, so that only the client device905can decrypt the media file412. Other approaches are also considered.

In some embodiments, the symmetric encryption key can itself be encrypted prior to being provided to the client device905, for instance to ensure that unauthorized parties cannot decrypt the media file412. For example, the symmetric encryption key can be encrypted asymmetrically, for instance using the public encryption key of the media storage server130(or of another entity involved in the media storage process, e.g., the135), prior to or as part of storage of the media stream150within the media storage server130. In this fashion, only the media storage server130can decrypt the symmetric encryption key used to encrypt the media stream150. Then, when the client device905requests a particular portion of the media stream150, the media storage server130can decrypt the relevant symmetric encryption key, using its own private encryption key, to obtain the decrypted symmetric encryption key. The media storage server130can then provide the symmetric encryption key as—is, or re-encrypt it, for instance using the public encryption key of the client device905, so that only the client device905, using its private encryption key, can decrypt the symmetric encryption key needed to then decrypt the media file412. Other approaches are also considered.

In some embodiments, the TDS server920may also be responsible for managing the flow of media streams150from media sources105to the media storage server130. To this end, the TDS server920may be coupled to one or more of the media sources105for providing instructions thereto, or may send commands to the media sources105via the media storage server130. The media storage server130, which may be composed of a number of virtual servers, may be a distributed server, or the like, may need to manage a total storage capacity of the media storage server130, a maximum number of concurrently received media streams150, or the like. The TDS server920can issue instructions to different ones of the media sources105indicating particular targets for the media streams150produced by the media sources105, for instance to facilitate load balancing on the media storage server130, to account for differing tiers or importance levels of the media streams150produced by the media sources105, or the like.

With reference toFIG.10, there is illustrated a method1000for serving tamper-evident media to a user or other entity requesting the media. In some embodiments, the method1000may be implemented in a cloud-based environment. The method1000may be performed using data structures, for instance the data structures310,320,330of the data architecture300, which comprise a payload field and an associated metadata field. It should be understood that the steps described hereinbelow forming part of the method1000need not be performed in any particular order, unless otherwise specified.

As part of step1010, the method1000comprises obtaining a request to serve a portion of a media stream. The request may come from a user or other party, for instance from the client device905, and may provide various information specifying the portion of the media stream desired. For example, the user may request a video stream from a particular camera (e.g., camera 012) at a particular time (e.g., 2:00:00 PM on Jun. 16, 2015).

As part of step1020, the method1000comprises determining, based on the request and at least one set of parameters associated with the media stream, a particular data structure of the at least one data structure, the at least one set of parameters having previously been stored in the metadata field of the at least one data structure. The particular one of the data structures312,322,332is determined based on the portion of the media stream150stored in its payload field being relevant to the request of the user (e.g., containing the requested media). In some embodiments, this determination is made based on an index generated from the metadata fields of the data structure(s), for instance the index922. In some other embodiments, the sets of parameters stored in the data structure310,320,330are used to determine a particular one of the data structures310,320,330. In embodiments in which the metadata fields include a seek table or similar data, determining the particular data structure to be served may include consulting the seek table, which may be stored in the metadata field, and/or which may be replicated in the index922. For example, in embodiments in which a plurality of portions of the media stream are stored in a common payload field of a single data structure (e.g., the media files622,632,642,652in the data structure610), the seek table present in the metadata field614and/or replicated in the index922may be used to find the appropriate data structure and/or the appropriate one of the media files622,632,642,652. As a result, determining the appropriate one of the multiple portions of the media stream to provide is based on the request obtained at step910, and on the multiple sets of parameters (e.g., stored in the metadata field614or as copies within the payload field612), each of which is associated with respective ones of the media files622,632,642,652.

As part of step1030, the method1000comprises obtaining, from the payload field of the particular data structure, the portion of the media stream to be served. For example, if the data structure310is determined to be the particular data structure, the media file412is obtained from the payload field312of the data structure310, for instance by the TDS server920.

In some embodiments, as part of step1040, the method1000comprises validating that the portion of the media stream is tamper-free. In some embodiments, validating that the portion of the media stream is tamper-free is performed by comparing a progenitor parameter stored as part of the set of parameters in the metadata field, the progenitor parameter based on a hash of at least part of a previous data structure storing a previous portion of the media stream, with a newly generated progenitor parameter generated based on the at least part of the previous data structure, the previous data structure having previously been rendered tamper-evident. In some other embodiments, validating that the portion of the media stream is tamper-free is performed by verifying a signature associated with the portion of the media stream, for instance the signature530. Validation of the signature530can be performed by using a certificate associated with the signature530, for instance a certificate belonging to the entity which produced the signature530. Other approaches to validating the portion of the media stream are also considered, including those described hereinabove, and can be performed by the streaming server910, by the media storage server130, by the client device905, or by any other suitable device. If the portion of the media stream is found to have been tampered with, or if other evidence of tampering is found to have occurred (e.g., with the set of parameters, or with other information stored in the metadata field), an alert can be raised, a notification can be issued to the requestor, and in some cases, the method1000may stop without providing the requested media to the requestor.

As part of step1050, the method1000comprises providing the portion of the media stream, for example in the form of providing the media file412, for instance by the streaming server910. The media file412may be provided to the user or relevant party over a network, or may be caused to be presented on a display associated, for instance, with the media storage server130. Other approaches for providing the portion of the media stream are also considered. In some embodiments, including in embodiments in which a plurality of portions of the media stream are stored in a common payload field of a single data structure (e.g., the media files622,632,642,652in the data structure610), a copy of the relevant set of parameters for the particular data structure may also be provided with the portion of the media stream to be served, which may be obtained from the payload field. For example, the relevant set of parameters may be stored in a header field of a stream used to provide the relevant media file to the client device905.

Although the present disclosure focuses primarily on systems and methods for storage of tamper-evident media and provisioning of the same, the embodiments described herein may also be applied to other fields of endeavor. For instance, other types of data may be similarly stored and rendered tamper-evident.

The embodiments of the methods, systems, devices, and computer-readable media described herein may be implemented in a combination of both hardware and software. These embodiments may be implemented on programmable computers, each computer including at least one processor, a data storage system (including volatile memory or non-volatile memory or other data storage elements or a combination thereof), and at least one communication interface.

The foregoing discussion provides many example embodiments. Although each embodiment represents a single combination of inventive elements, other examples may include all possible combinations of the disclosed elements. Thus, if one embodiment comprises elements A, B, and C, and a second embodiment comprises elements B and D, other remaining combinations of A, B, C, or D, may also be used.

Although the embodiments have been described in detail, it should be understood that various changes, substitutions, and alterations can be made herein without departing from the scope as defined by the appended claims.