Patent Publication Number: US-6216267-B1

Title: Media capture and compression communication system using holographic optical classification, voice recognition and neural network decision processing

Description:
FIELD OF THE INVENTION 
     The present invention generally relates to the field of media communications systems, and particularly to an audio/video system utilizing optical processing. 
     BACKGROUND OF THE INVENTION 
     Traditional semiconductor based digital electronic processors are typically serial devices processing data in a serial manner, i.e. a first operation is performed on a first set of data before a second set of data is fetched and operated upon. Although advents in semiconductor based vocoders and processor architectures, such as predictive branching and higher microprocessor speed, have provided systems capable of performing increasingly faster operations, the fundamental serial structure of semiconductor processing systems inherent in the device technology (e.g., von Neumann architecture) have limited the speed at which complex processing algorithms such as video signal processing and compression may be performed with a general purpose semiconductor based processor. Further, although specialized semiconductor processors have been developed having architectures optimized for signal processing algorithms (e.g., digital signal processors, Harvard architecture), semiconductor devices still exhibit a signal processing limit. These problems become apparent when it is desired to transmit video or audio over a limited bandwidth channel. Since video and audio signals many times have a greater bandwidth than the channel over which it is desired to transmit, compression algorithms are utilized to reduce the bandwidth such that transmission over the lower bandwidth channel may be achieved. However, video and some audio compression algorithms require large amounts of processing power. Using traditional semiconductor processors to perform the compression algorithms in real-time or near real-time many times requires too much processing time to accomplish high quality image or audio compression. Therefore, at best, semiconductor processors only provide lossy video compression (where some information quality and content is sacrificed) as real time compression is approached. 
     However, optical processors using holographic image processing/classification techniques are capable of processing information in parallel such that much more complex image processing algorithms such as compression, correlations, and transform decompositions may be processed in much shorter amount of time than with traditional semiconductor processors. Such optically implemented image classification and processing algorithms may provide optimized transmission of video, image and graphics file signals over lower bandwidth channels. Furthermore, by combining an optical processor with a media communication system, voice command recognition, control and neural network signal processing may be implemented with the system by taking advantage of the parallel processing and data classification provided by an optical processing system. Thus, there lies a need for a media processing and transmission system that utilizes optical processing to provide faster and more optimized transmission of video and audio signals over lower bandwidth communications channels and that further utilizes optical processing to implement voice command recognition and neural network decision processing. 
     SUMMARY OF THE INVENTION 
     The present invention is directed to a system for capturing and processing video and audio for optimized transmission over a lower bandwidth communication channel. In one embodiment, the system includes a digital processing system for receiving a video and an audio signal from a media source or sensor wherein the video and audio signals are representative of the media delivered or captured by the media source or sensor, at least one or more optical processing systems coupled to the digital processing system for performing signal classification and processing algorithms on the video and audio signals, and a transceiver for transmitting the combined signals over a lower bandwidth communications channel wherein the signal processing algorithms and classification performed by the optical processing system optimizes the video and audio signals for transmission over the lower bandwidth communications channel. In a further embodiment, the system provides optical processing for voice recognition and neural network processing. 
     The present system level invention is further directed to a method for transmitting audio and video signals over a lower bandwidth communications channel. In one embodiment, the method includes steps for capturing audio and images and providing the information as a multimedia signal, optimizing the signal for transmission over the lower bandwidth communications channel by performing separate compression algorithms on the video and audio signals with an optical processing system, and transmitting the optimized multimedia signal over the lower bandwidth communications channel. 
     It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed. 
     The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate an embodiment of the invention and together with the general description, serve to explain the principles of the invention. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     The numerous advantages of the present invention may be better understood by those skilled in the art by reference to the accompanying figures in which: 
     FIG. 1 is a block diagram of a video and audio multimedia system capable of optically processing a video signal and an audio signal in accordance with the present invention; 
     FIG. 2 is a block diagram of the hardware of a computer processing system operable to tangibly embody a digital processing system of the video and audio multimedia system of the present invention; and 
     FIG. 3 is a block diagram of a holographic optical processing system for optically processing a video signal and an audio signal in accordance with the present invention. 
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     Reference will now be made in detail to the presently preferred embodiment of the invention, an example of which is illustrated in the accompanying drawings. 
     Referring now to FIG. 1, a block diagram of a video and audio multimedia system capable of optically processing a video source and an audio source will be discussed. The multimedia system  100  receives video images or files captured with a video transducer  110  of a media source  108  and captures audio signals with an audio transducer  112 . Video transducer  110  may include, for example, a charge coupled device (CCD) array and related hardware for capturing and buffering frames of video. Audio transducer may comprise, for example, a microphone and preamplifier for capturing and amplifying sounds and providing the sound as a signal representative of the sound at a predetermined signal level. The images and sounds captured by video transducer  110  and audio transducer  112  of media source  108  are provided to a digital processing system  114  that may be, for example, an electronic computer system (such as shown and described with respect to FIG.  2 ). An optical processing system  116  couples to digital processing system  116  for performing signal processing algorithms utilizing an optical signal processing apparatus. 
     In operation, multimedia system  100  receives a video signal and an audio signal from media source  110  that are provided to digital processing system  114  as a multimedia signal intended to be transmitted to a remote device or location with a transceiver  120  coupled to digital processing system  114 . Typically, the bandwidth of the channel  122  over which the image is to be transmitted is too narrow for real-time transmission of a complete, full bandwidth video and audio multimedia signal. The video and audio signals are processed by an optical processing system  116  coupled to digital processing system  114  prior to transmission such that the video and audio signals are optimized for transmission over the limited bandwidth channel  122 . Optical processing system  116  implements a library of definable and programmable signal transforms and holographic image correlators for implementing a wide range of signal decomposition functions including Fourier transforms, Hartley transforms, discrete valued transforms (e.g., z-transforms), transform inversions, signal compression, filtering, etc. A data storage device  118  may be coupled to digital processing system  114  for storing video and audio signal data during processing as required, or for longer term storage of the video and audio signals. 
     After optical system  116  performs the desired signal processing, the video and audio signals may be transmitted as a processed multimedia signal via channel  122  to be received by a second transceiver  124  disposed at a remote location. The received multimedia signal is processed by a second digital processing system  126  coupled to transceiver  124  for reconstructing the video and audio signals from the received multimedia signal. A second optical processing system  128  coupled to digital processing system  126  implements algorithms for reconstructing the video and audio signals (e.g., inverse transforms) from the received multimedia signal. Upon reconstruction of the video and audio signals, the video signal may then be displayed on display  134 , and the audio signal may be reproduced with an audio transducer  132  (e.g., an amplifier and speaker). A data storage device  130  coupled to digital processing system  126  may be used for storing the video and audio signals during processing as required, or for longer term storage of the video and audio signals. As required by the particular application in which multimedia system  100  is utilized, the definable and programmable transforms or correlations performed by optical processing systems  116  and  128  may be optimally selected for the particular utilization of media source  108 . For example, a first optical processing algorithm and system may be selected for processing of landscapes, a second algorithm or system may be selected for processing buildings, a third algorithm or system may be selected for processing vehicles, a fourth algorithm or system may be selected for processing images of human beings, a fifth algorithm may be selected for processing conversations, and so on. In one particular embodiment, either of optical processing system  116  or optical processing system  128 , alone or in combination, is configured to perform a compression algorithm on the video or audio signals as received and delivered through a Transmission Control Protocol/Internet Protocol (TCP/IP) packet communications network. For example, media source  108  may receive video or audio signals, images, graphics files or multimedia files from a TCP/IP packet communications network (e.g., the Internet). 
     Referring now to FIG. 2, a computer hardware system operable to tangibly embody a digital processing system of a multimedia system of the present invention will be discussed. The computer system  200  may be utilized for either digital processing system  114  or digital processing system  126  and generally includes a central bus  218  for transferring data among the components of computer system  200 . A clock  210  provides a timing reference signal to the components of computer system  200  via bus  218  and to a central processing unit  212 . Central processing unit  212  is utilized for interpreting and executing instructions and for performing calculations for computer system  200 . Central processing unit  212  may be a special purpose processor such as a digital signal processor. A random access memory (RAM) device  214  couples to bus  218  and to central processing unit  212  for operating as memory for central processing unit  212  and for other devices coupled to bus  218 . A read-only memory device (ROM)  216  is coupled to the components of computer system  200  via bus  218  for operating as memory for storing instructions or data that are normally intended to be read but not to be altered except under specific circumstances (e.g., when the instructions or data are desired to be updated). ROM device  216  typically stores instructions for performing basic input and output functions for computer system  200  and for loading an operating system into RAM device  214 . 
     An input device controller  220  is coupled to bus  218  for allowing an input device  222  to provide input signals into computer system  200 . Input device  222  may be a keyboard, mouse, joystick, trackpad or trackball, microphone, modem, or a similar input device. Further, input device  222  may be a graphical or tactile input device such as a touch pad for inputting data with a finger or a stylus such. Such a graphical or tactile input device  222  may be overlaid upon a screen of a display device  226  for correlating the coordinates of a tactile input with information displayed on display  226 . Display  226  is controlled by a video controller  224  that provides a video signal received via bus  218  to display  226 . Display  226  may be any type of display or monitor suitable for displaying information generated by computer system  200  such as cathode ray tube (CRT), a liquid crystal display (LCD), gas or plasma display, or a field emission display panel. Preferably, display  226  is a flat-panel display having a depth being shallower than its width. A peripheral bus controller  228  couples peripheral devices to central bus  218  of computer system  200  via a peripheral bus  228 . Peripheral bus  230  is preferably in compliance with a standard bus architecture such as an Electrical Industries Association Recommended Standard 232 (RS-232) standard, an Institute of Electrical and Electronics Engineers (IEEE) 1394 serial bus standard, a Peripheral Component Interconnect (PCI) standard, or a Universal Serial Bus (USB) standard, etc. A mass storage device controller  232  controls a mass storage device  234  for storing large quantities of data or information, such as a quantity of information larger than the capacity of RAM device  214 . Mass storage device  232  is typically non-volatile memory and may be a disk drive such as a hard disk drive, floppy disk drive, optical disk drive, floptical disk drive, etc. 
     Referring now to FIG. 3, an optical processing system for optically processing video and audio signals in accordance with the present invention will be discussed. The optical processing system of FIG. 3 may be utilized as one or both of optical processing systems  116  and  128  discussed with respect to FIG.  2 . Optical processing system  300  may be utilized to perform a correlation algorithm or the like type of algorithm (e.g., convolution, cross-correlation, etc.). Digital processing system  114  provides the audio signals  330  and the video signals  332  to be processed by optical processing system  300 . An audio scan signal  318  to be processed in conjunction with audio signals  330  is coupled to a spatial light modulator (SLM)  314  for modulating the light beam output of a laser  310  impinging upon SLM  314 . Audio scan signal  318  may be, for example, a codebook of voice phonemes for decoding phonemes in audio signal  330 . Likewise, digital processing system  114  provides video signals  332  to a second SLM  316  for modulating the light beam output of a second laser  312 . A video scan signal  320  is provided to SLM  316  such for processing in conjunction with video signals  332 . For example, video scan signal may be a codebook of images to be correlated with images present in video signals  332 . Optical processing system  300  may be thereby utilized to perform correlation processes between audio signals  330  and audio scan signal  318 , and between video signals  332  and video scan signal  320 . Other types of optical processing algorithms may also be implemented by optical processing system  300 . Optical processing system  300  may be utilized to perform an autocorrelation, for example, to remove undesirable noise from audio signal  330  or video signals  332 . Further, optical processing system  300  may be utilized to perform a convolution algorithm, for example, in order to perform digital filtering on audio signals  330  or video signals  332 . Additional signal processing techniques (e.g., compression, signal transforms, etc.) may also be implemented by optical processing system  300 . 
     The modulated laser beams are applied to a lens system  322  for directing and focusing the laser beams through a photorefractive (PR) crystal  324  and thereby impinge upon a detector  328 . The modulated light beams from lasers  310  and  312  impinge upon photorefractive crystal  324  at different angles of incidence, thereby forming a holographic grating within the crystal. Further, PR crystal  324  may contain data stored holographically that may be further utilized in a signal processing algorithm. For example, a correlation may be performed on audio signals  330  or video signals  332  and the data stored in PR crystal  324 . A third light beam from a third laser  326  is applied to PR crystal  324  for reading out the holographic output onto detector  328  as an output signal such that an appropriate signal processing algorithm on audio signals  330  or video signals  332   318  is obtained. Detector  328  may be a charge-coupled device (CCD) for converting optical signals processed by optical processing system into a digital signal readable by digital processing systems  114  or  126  for further signal processing in a discrete-time domain, discrete-amplitude range. 
     It will be seen that multimedia processing system  100  may be utilized to process audio signals and video signals received by media source  108 . Audio transducer  112  may include a microphone and preamplifier for capturing an audio signal. The audio signal may be incorporated as part of the video signal captured by video transducer  110 , for example when capturing the image of a person speaking such that the contents of the speech is captured and processed, and the audio and video signals are combined into a multimedia signal (data signal). Additionally, audio transducer  112  may capture voice commands from a user of multimedia system  100  such that the user may operate and control multimedia system  100  through spoken commands and utterances with voice recognition algorithms. Thus, optical processing system  116  may be utilized to process voice commands (i.e. audio instructions) received via audio transducer  112 . Optical processing system  116  is capable of processing audio command signal faster than a semiconductor based processor since optical processing may be performed in parallel rather than serially as with semiconductor processors. For example, a book of commands may be stored in PR crystal  324 , and a correlation preformed on audio signals  330  to determine which of the commands holographically stored in PR crystal  324  corresponds to the spoken command represented by audio signals  330 . Furthermore, optical processing system  116  may be utilized to implement neural network decision processing algorithms for controlling the operation of multimedia system  100 , for pattern recognition and image and speech analysis. Since multiple data sets may be stored in PR crystal  324  as holographic images, signal processing on all of the data set images stored in PR crystal may be performed simultaneously such that a neural network computing system may be implemented. 
     It is believed that the video capture and compression system using holographic optical correlation, signal transform decomposition, voice recognition and neural network decision processing of the present invention and many of its attendant advantages will be understood by the foregoing description, and it will be apparent that various changes may be made in the form, construction and arrangement of the components thereof without departing from the scope and spirit of the invention or without sacrificing all of its material advantages. The form herein before described being merely an explanatory embodiment thereof. It is the intention of the following claims to encompass and include such changes.