Dynamic media content previews

A sandboxed process is initiated in response to a request for a preview of dynamic media content. A stream of frames is generated in a preview specific format by a plug-in that executes within the sandboxed process. The stream of frames is provided as the preview of the dynamic media content, where the preview is viewable on a display screen.

FIELD

Embodiments of the invention relate to streaming media and, more particularly, to providing secure previews of streaming media and digital media content.

BACKGROUND

As computer users become increasingly sophisticated, there is an increasing desire for efficiency in the computer programs that people use. For example, many computer systems have some type of file management system that manages hundreds, or even thousands, of files, to which a user may need access. In many traditional file management systems, it is difficult to ascertain the contents of a particular file without actually opening the file, using the necessary application associated with the file type for the file. Oftentimes, the only useful information that the user can obtain about a file before opening it is the file name. Many users would like to be able to preview the contents of a file before actually committing resources to opening the full application and then the file. Some file management systems, such as the Finder in Mac OS X, offered by Apple Inc. of Cupertino, Calif., provides functionality for previewing files before opening them. Other operating systems and file management systems may also offer some static previewing functionality.

For example, using Apple's Finder, a user can preview documents, such a PDFs, text files, word processing documents, and spreadsheets, etc. Finder also allows a user to preview certain video content that has a native file type recognized by the Mac OS.

The Quick Look daemon in the Mac OS incorporates a variety of plug-ins, each of which accepts a particular data type from a client process and converts it into a preview in one of a set of standard format, such as PDF and HTML. The standard format previews are then displayed by the client process using the Quick Look framework. The advantage of this structure is that it avoids a client application crash being caused by a third party plug-in crashing. However, this model does not work particularly well for dynamic media, such as streaming media, because converting dynamic media associated with a non-native file type into a suitable standard format, takes too much processing time to be useful or reliable.

SUMMARY OF THE DESCRIPTION

When a client process receives a request for a preview of dynamic media content, the request is passed to a sandboxed process. The sandboxed process is isolated from the client process, meaning that if the sandboxed process crashes, the client process remains functional and operational. Within the sandboxed process, the type of dynamic media content is determined. The content type may be determined based on, for example, a uniform-type identifier (UTI). Once the content type has been determined, the sandboxed process retrieves a streaming plug-in to convert the dynamic media content to a media type that the client process can use to display a preview of the dynamic media.

The plug-in is used to convert the dynamic media content according to the protocol associated with the plug-in. In some embodiments, the plug-in generates both an audio component and a video component (at least for the dynamic media content that has both audio and video). As each frame of dynamic content is generated, it is passed to the client process for display as a streaming preview. In some embodiments, the plug-in provides one or more controls to the client process that allows a user to control limited aspects of the preview of the dynamic media content (e.g., “stop,” “play,” and “pause,” etc).

DETAILED DESCRIPTION

As provided herein, various methods, apparatuses and systems enable previews of dynamic media content. When a user selects a file (e.g., from a file management system) and requests a preview of that file, embodiments described herein facilitate conversion of the dynamic media into a format that is suitable for previewing the dynamic media content.

FIG. 1is a block diagram, according to various embodiments. As shown, a file management system (e.g., Finder, Windows Explorer, etc.), or other program (e.g., client application, Web browser, etc.), includes a client process102. Client process102is where the user-level interaction takes place with the particular program. When a user requests a preview of file110, an application programming interface (API)112sends a request116to preview daemon104. If preview daemon104is not already running, then it is initiated. In various embodiments, preview daemon104is a sandboxed process. A sandboxed process is one that is separate from other running processes, or programs. The sandboxed process, or simply sandbox, typically provides a tightly controlled set of resources for guest programs to run in, such as scratch space on disk and memory. In various embodiments, network access, the ability to inspect the host system, or read from input devices, are usually disallowed, or heavily restricted, in the sandbox.

By sending request116to preview daemon104, any manipulation, or conversion, of file110that causes any errors/problems with preview daemon104(e.g., crashing, hanging, unacceptable latency, etc.), will not affect client process102. In other words, if preview daemon104crashes, client process102will continue to run unaffected. For the user, this means that any errors in generating a preview of a file will not prevent the user from using the underlying program, such as the file management system. This can be particularly important if client process102is associated with a fundamental user program such as a file management system (e.g., Apple Finder, Windows Explorer, etc.). In some embodiments, if the sandboxed process crashes, the client process may display an indication that the requested preview is not current available.

In various embodiments, a file type module118initially receives request116from client process102. File type module118determines the file type associated with file110. As discussed previously, the file type may be determined using a uniform-type identifier, or UTI, which is a string defined by Apple Inc. that uniquely identifies the type of a class of items. Thus, UTIs are used to identify the type of files and folders, clipboard data, bundles, aliases, symlinks, and streaming data. If file type module118recognizes the file type of file110as a native file type, then file type module118retrieves a known plug-in120associated with the native file type. For files that contain static media, such as word processing documents, PDFs, HTML files, etc., the retrieved plug-in can handle the file request and convert it into a file preview122that is sent back to client process102for display as a preview. In certain embodiments, plug-in120is directly capable of converting dynamic media content (audio/video files, interactive 3D animation files, etc.), as described in more detail below. The file preview122that is sent to client process102is handled by API112, which includes various display bundles114. Each display bundle includes the resources necessary to display a preview of one or more specific file types. Thus, if file preview122is a preview of a PDF, then a PDF display bundle from the group of display bundles114takes the PDF preview and prepares it for display on display130.

Further embodiments described herein are capable of providing previews for dynamic media content (e.g., streaming media, video, 3-D animation, etc.). For example, if file110includes dynamic media content, request116might be sent to preview daemon104and file type module118determines the file type, based on, for example, the UTI associated with the file110. Other identifiers could be used in different embodiments. If the dynamic media is of a native file type and recognized by the file type module118, then a corresponding plug-in120is retrieved to convert the file110and provide a preview. If, however, file110includes dynamic media content of a non-native file type, then file type module118retrieves a designated plug-in120to handle the non-native conversion of file110.

In certain embodiments, the designated plug-in120is responsible for retrieving an additional plug-in (stream plug-in124) to handle the dynamic media content. The designated plug-in may also retrieve a copy of the file that needs to be converted. In some embodiments, this file is the same file that is received as a part of request116. However, in other embodiments, the file may be a different copy of the same file to be used in converting to a streaming preview.

In addition, another sandbox process106is opened, or initiated, for the stream plug-in124to convert file110into a streaming preview126. In this way, streaming preview daemon106is isolated from preview daemon104and from client process102. Thus, if during the conversion of the file to streaming preview the stream plug-in124crashes, process106, preview daemon104and client process102remain unaffected. In this way, a user does not lose any functionality in the client application (e.g., the file management system) and, moreover, the user can still preview other files, specifically those that do not require streaming previews without any interruption.

In various embodiments, stream preview126is a series of frames converted from the original file110. The frames associated with stream preview126may include limited user functionality. In other words, if a full application (e.g., a media player) were opened to display the dynamic media content in file110, the user would have the application's full range of user-interfaced options to control and/or manipulate the file. For example, if a user wanted to view a video file and opened a video file viewing application (e.g., Quicktime offered by Apple Inc., Windows Media Player offered by Microsoft Corporation, etc.), the user would have the ability to stop, pause, play, adjust the volume, re-size the window, and/or perform various other operations on the file (e.g., save, delete, copy, etc.) from within the context of the opened application. In contrast, streaming preview126offers limited control to the user, as it is specifically designed to be a preview. For example, streaming preview126may only provide control over stopping and playing the content in the preview.

Streaming preview126is received and handled by API112and, more particularly, by one of the designated display bundles114. In the case of streaming previews, one or more display bundles114includes the resources necessary to display streaming preview126. Thus, a streaming preview display bundle from the group of display bundles114takes streaming preview126and prepares it for display on display130.

In addition to generating stream preview126, stream plug-in124may also generate and provide a set of one or more stream controls128to client process102. Stream controls128include one or more commands that allow a user to control various aspects of streaming preview126. Thus, as discussed previously, stream controls128might include commands to stop, pause, and/or play the content in the streaming preview. More commands or fewer commands may be included in certain embodiments. In some embodiments, stream controls128may not provide the user with the full range of functionality available in the native application associated with the file.

FIG. 2is a block diagram illustrating various embodiments associated with generating a streaming preview of dynamic media content. Similar toFIG. 1,FIG. 2shows a client process202and a streaming preview daemon206. A stream plug-in220is used to convert file208into the streaming preview. The stream plug-in220runs within the sandboxed streaming preview daemon206. The stream plug-in220generates the streaming preview on a frame-by-frame basis and, in certain embodiments, separately generates a video component and an audio component. Thus, file208is converted into an audio component212and a video component210. The audio component212is added to the stream preview (or, simply, stream)214and the video component210is added to the stream on a frame-by-frame basis as each frame is generated. So, for example, as soon as frame1is converted and generated into a preview-specific format, it is added to the stream and sent to client process202, without waiting for frame2to be converted and generated. Likewise, frame2may be immediately added to stream214as soon as it is converted and generated. This process continues for each frame that is converted and generated.

As discussed previously, stream plug-in220provides stream controls, in various embodiments, to the client process. As shown inFIG. 2, stream controls216run within client process202, and allow a user to control certain aspects of stream214. As described herein, stream controls216provide limited control over the stream214and do not provide the full range of control that would be available if the file208were opened in a native application.

FIG. 3is a flow diagram illustrating various embodiments for providing previews of dynamic media content. A request for a preview of dynamic media is received310. In response, at least one sandboxed process is initiated320. A single sandboxed process may already be running, in which case a second sandboxed process might be initiated to separate the streaming media preview from other previews being generated. The content type of the dynamic media is determined330. The content type may be determined based on a uniform-type identifier (UTI), or other form of file type identification.

Based on the content type, a stream plug-in is retrieved340to convert the dynamic media into a preview-specific format. As used herein, a “stream” plug-in refers to any plug-in specifically designated to convert dynamic media content (e.g., video, audio, interactive 3D animation, etc.) into streaming media. If the content type of the dynamic media is a native file type, or, in other words, a recognized file type, then it may be possible to quickly and conveniently convert that file into a preview. However, if the file type is determined to be a non-native file type, then the stream plug-in needs to be capable of converting the original file from one format to a preview-specific format that is compatible with dynamic previewing.

The retrieved plug-in is provided350to the sandboxed process and the preview is generated360. In situations where the file is a video file containing both audio and video data, the plug-in may generate separate audio and video components for the preview. If the original file contains only audio, then only an audio component might be generated. If the original file contains only video without audio, then only a video component might be generated. The streaming media plug-in can also support various files that contain interactive 3D animation, such as COLLADA files. Once the preview components have been generated the stream preview is provided370for display to a user and controls for the dynamic preview are also provided380. Such controls allow a user to control limited functionality associated with the dynamic preview (for example, “stop,” “play,” “pause”). For 3-D animated files, the dynamic preview may include similar controls, such as the ability to stop, pause, or play the animated file. However, in certain embodiments, the provided user controls are limited, as compared to the controls available when a native application is opened to play the dynamic media content.

It should be noted that in alternate embodiments the steps described inFIG. 3can be rearranged and performed in a different order that the one shown. Also, it should be noted that more process steps, or fewer process steps, than those described inFIG. 3, can be used to accomplish the same dynamic media preview functionality contemplated by the various embodiments described herein.

According to various embodiments,FIG. 4also represents a form of the system that can be used to implement the described details shown inFIGS. 1-3. In particular, it should be noted that display410can be used to display dynamic media content as does display130ofFIG. 1. Embodiments that facilitate dynamic media content previews as described herein may included as instructions422(in the system ofFIG. 4) that are stored, for example, in drive unit418or main memory404and can be executed by processor402.

The computer system400may further include a network interface device416. The computer system400also may include a display unit410(e.g., a liquid crystal display (LCD), light emitting diode (LED) display, a cathode ray tube (CRT)), and an input device412(e.g., a keyboard and/or mouse, etc.).

The secondary memory418may include a machine-readable storage medium (or more specifically a computer-readable storage medium)424on which is stored one or more sets of instructions (e.g., software422) embodying any one or more of the methodologies or functions described herein. The software422may also reside, completely or at least partially, within the main memory404and/or within the processing device402during execution thereof by the computer system400, the main memory404and the processing device402also constituting machine-readable storage media. The software422may further be transmitted or received over a network420via the network interface device416. In various embodiments, web content requested by a user can be retrieved (e.g., from the World Wide Web) via network420using network interface device416.

While the machine-readable storage medium424is shown in an exemplary embodiment to be a single medium, the terms “machine-readable storage medium” or “computer-readable storage medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions. The terms “machine-readable storage medium” or “computer-readable storage medium” shall also be taken to include any medium that is capable of storing or encoding a set of instructions for execution by the machine/computer and that cause the machine/computer to perform any one or more of the methodologies of the present invention. The terms “machine readable storage medium” or “computer-readable storage medium” shall accordingly be taken to include, but not be limited to, solid-state memories, and optical and magnetic media.

Various components described herein may be a means for performing the functions described herein. Each component described herein includes software, hardware, or a combination of these. The operations and functions described herein can be implemented as software modules, hardware modules, special-purpose hardware (e.g., application specific hardware, application specific integrated circuits (ASICs), digital signal processors (DSPs), etc.), embedded controllers, hardwired circuitry, etc.

Aside from what is described herein, various modifications may be made to the disclosed embodiments and implementations of the invention without departing from their scope. Therefore, the illustrations and examples herein should be construed in an illustrative, and not a restrictive sense.