Patent Publication Number: US-2019177004-A1

Title: Integrated Imaging System for a Connected Aircraft

Description:
BACKGROUND 
     Certain seats within the passenger compartments of commercial aircraft may be designated for the use of flight attendants and crewmembers. Once the aircraft reaches a safe cruising altitude or flight segment, crewmembers may monitor any passenger security and safety issues firsthand as they move throughout the cabin. For example, Richard Reid was thwarted in his effort to detonate explosives concealed in his shoes aboard American Flight 63 when he was detected by cabin crew. However, during taxi, takeoff and landing (TTL) procedures cabin crew must monitor the safety of the cabin and passengers from assigned seats, in a safely seated and restrained (e.g., belted) position. Federal aviation guidelines provide that each such assigned seat provide a direct view of the cabin area for which the occupying crewmember is responsible. In practice, this means that each crewmember must have (from the seated and belted position) direct visual contact with the cabin area (and a minimum percentage of the passengers) and main aisles, such that the crew are aware of any emerging needs or issues relative to passenger safety. 
     While cabin crew should be proximate to an emergency exit should evacuation or other emergency procedures supervised by crewmembers be necessary, direct view can be a critical factor in the success or failure of such procedures. For example, in the 1985 British Airtours accident at Manchester Airport (caused by engine failure during an aborted takeoff, which resulted in catastrophic fire both outside and inside the aircraft) the TTL direct-view positions of forward crew members were obstructed by galley bulkheads. This frustrated the crew&#39;s ability to monitor cabin conditions and evacuate the aircraft, contributing to extensive casualties due to smoke inhalation by passengers unable to rapidly evacuate through limited exits (some of which were blocked by smoke and/or fire). In addition, contemporary and next-generation commercial cabin interiors may provide for additional seating classes or amenities and consequently additional partitions or bulkheads, either of which may frustrate direct view requirements by obstructing crew sightlines. For example, economy-class cabins may be further partitioned into standard-economy and premium-economy zones. In some cases, business-class or first-class seats may be further partitioned or enclosed for enhanced privacy. The height of said partitions, bulkheads, and walls may directly frustrate the sightlines of cabin crew in a seated and belted position, even if the partitions are only temporary in nature, e.g., curtains deployed to separate premium and economy seats. While these curtains may be opened during TTL phases, enhanced-privacy zones and compartments, however, may incorporate opaque floor-to-ceiling partitions rather than curtains, and thus it may not be possible to improve visibility. This is especially true if cabin crew seats are positioned at the front of the cabin (e.g., to maximize direct view of the cabin and aisles in a generally aft direction) and such premium or enhanced-privacy seating is positioned directly aft of the cabin crew seats (e.g., between the crew seats and the economy cabin proper). 
     SUMMARY 
     In one aspect, embodiments of the inventive concepts disclosed herein are directed to an integrated imaging system for a connected aircraft. The system includes cameras mounted within the cabin interior of the aircraft. Each camera may have a particular field of view (FOV) which may include one or more defined zones within the aircraft. Each camera may capture an image stream including seating, aisles, and passengers within the camera&#39;s FOV. The system includes processors for receiving the image streams and assembling enhanced image streams, e.g., by stitching together or composing image streams from within the same zone or from different zones throughout the aircraft, or by overlaying the image streams with relevant environmental data. The enhanced or composite image streams may be transmitted wirelessly or sent via cable or other physical link to fixed-mount or mobile display devices (e.g., tablets, smartphones) for viewing by cabin crew. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       Implementations of the inventive concepts disclosed herein may be better understood when consideration is given to the following detailed description thereof. Such description makes reference to the included drawings, which are not necessarily to scale, and in which some features may be exaggerated and some features may be omitted or may be represented schematically in the interest of clarity. Like reference numerals in the drawings may represent and refer to the same or similar element, feature, or function. In the drawings: 
         FIG. 1  illustrates an exemplary embodiment of a system according to the inventive concepts disclosed herein; and 
         FIG. 2A  illustrates the system of  FIG. 1 ; 
         FIG. 2B  illustrates the system of  FIG. 2A ; 
         FIG. 3A  is a diagrammatic illustration of the system of  FIG. 2B ; 
         FIG. 3B  is a diagrammatic illustration of the system of  FIG. 3A ; 
         FIG. 4  illustrates a mobile device of the system of  FIG. 3B ; and 
         FIG. 5  illustrates the system of  FIG. 3B . 
     
    
    
     DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS 
     Before explaining at least one embodiment of the inventive concepts disclosed herein in detail, it is to be understood that the inventive concepts are not limited in their application to the details of construction and the arrangement of the components or steps or methodologies set forth in the following description or illustrated in the drawings. In the following detailed description of embodiments of the instant inventive concepts, numerous specific details are set forth in order to provide a more thorough understanding of the inventive concepts. However, it will be apparent to one of ordinary skill in the art having the benefit of the instant disclosure that the inventive concepts disclosed herein may be practiced without these specific details. In other instances, well-known features may not be described in detail to avoid unnecessarily complicating the instant disclosure. The inventive concepts disclosed herein are capable of other embodiments or of being practiced or carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein is for the purpose of description and should not be regarded as limiting. 
     As used herein a letter following a reference numeral is intended to reference an embodiment of the feature or element that may be similar, but not necessarily identical, to a previously described element or feature bearing the same reference numeral (e.g., 1, 1a, 1b). Such shorthand notations are used for purposes of convenience only, and should not be construed to limit the inventive concepts disclosed herein in any way unless expressly stated to the contrary. 
     Further, unless expressly stated to the contrary, “or” refers to an inclusive or and not to an exclusive or. For example, a condition A or B is satisfied by anyone of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present). 
     In addition, use of the “a” or “an” are employed to describe elements and components of embodiments of the instant inventive concepts. This is done merely for convenience and to give a general sense of the inventive concepts, and “a’ and “an” are intended to include one or at least one and the singular also includes the plural unless it is obvious that it is meant otherwise. 
     Finally, as used herein any reference to “one embodiment,” or “some embodiments” means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the inventive concepts disclosed herein. The appearances of the phrase “in some embodiments” in various places in the specification are not necessarily all referring to the same embodiment, and embodiments of the inventive concepts disclosed may include one or more of the features expressly described or inherently present herein, or any combination of sub-combination of two or more such features, along with any other features which may not necessarily be expressly described or inherently present in the instant disclosure. 
     Broadly, embodiments of the inventive concepts disclosed herein are directed to an integrated imaging system for enhancing direct view capability aboard a connected aircraft. It is an objective of the disclosed system to maximize the proportions of passengers, seats, aisles, and other relevant factors visible by cabin crew from a seated and belted position (rather than merely meeting minimum direct-view requirements) without adding crewmembers or removing revenue-generating seating. In addition, the system may enhance the direct view capability of each individual crewmember by providing access to views and perspectives not within their particular sightlines. Further, the system may account for additional partitions and bulkheads which may obstruct physical sightlines, and provide equal direct view capability to both sides of the aircraft. Finally, the system may provide for centralized, remote direct view assistance via ground-based control facilities. 
     Referring to  FIG. 1 , an exemplary embodiment of an integrated imaging system  100  (IIS) according to the inventive concepts disclosed herein may include one or more cameras  102 ,  104 ,  106  mounted within various zones (e.g., first class zone  108 , business class zone  110 , economy class zone  112 ) of the interior cabin of an aircraft  114 . The aircraft  114  may include, for example, commercial aircraft of any size subject to regulations requiring cabin crew ( 116 ) to maintain a minimum proportion of passengers ( 118 ), aircraft seats ( 120 ) and/or aisles ( 122 ) in direct view from seated, belted positions (e.g., in designated cabin crew seats) during taxi, takeoff and landing (TTL) procedures. 
     During TTL procedures, when the cabin crew  116  are seated and belted in their designated cabin crew seats, the cameras  102 ,  104 ,  106  may provide views of the interior cabin (e.g., passengers  118 , aircraft seats  120 , and aisles  122 ) visible to the cabin crew  116  via monitors ( 124 ) mounted in fixed location proximate to the cabin crew seats (e.g., a forward bulkhead  126 ). Fixed-mount monitors  124  may be mounted proximate to any cabin crew seat wherein a crewmember may be stationed during TTL procedures, or within the cockpit (not shown) for display to the command crew. For example, the cameras  102 ,  104 , and  106  may capture image streams respectively corresponding to a first class zone  108 , a business class zone  110 , and an economy class zone  112 . Each member of the cabin crew ( 116 ) may observe the image stream corresponding to a particular zone, or the image streams may be centrally processed into enhanced video streams accessible to all cabin crew, regardless of their positions within the aircraft  114 . The IIS  100  may provide for the display of captured image streams via mobile devices  128  (e.g., tablets, smartphones, or other portable computing or communications devices) held by members of the cabin crew  116 . For example, a crewmember may access the captured or enhanced image streams through his/her mobile device  128  via a wireless connection, and may be able to manipulate the displayed image via the mobile device  128  (e.g., scrolling through streams from multiple cameras  102 ,  104 ,  106 ; selecting a particular image stream or corresponding zone to watch; panning or zooming a particular camera; enlarging a displayed image; accessing passenger details or additional environmental data through a displayed image). The cabin crew ( 116 ) or command crew may activate the IIS  100  for direct view, or engage a direct-view mode of the IIS, via the fixed-mount monitors  124  or mobile devices  128 , or via any other appropriate command and control interface of the aircraft  114 . 
     Referring now to  FIG. 2A , the system  100   a  may be implemented and may function similarly to the IIS  100  of  FIG. 1 , except that the IIS  100   a  may orient the cameras  102 ,  104 ,  106  so as to maximize the coverage of the respective field of view  102   a ,  104   a ,  106   a  (FOV) of each camera. For example, one or more of the cameras  102 ,  104 ,  106  may be oriented in a generally lateral direction (e.g., transverse, orthogonal, or otherwise at an angle to the longitudinal (roll) axis of the aircraft  114 ), greatly increasing the portions of each zone ( 108 ,  110 ,  112 ;  FIG. 1 ) in direct view by the cabin crew  116   a - c  compared to the respective fields of view  130   a - c  directly visible by each crewmember from a seated and belted position (e.g., looking in a generally longitudinal direction with minimal head movement) and visible only by that particular crewmember. Furthermore, via fixed-mount monitors  124  and/or connected mobile devices ( 128 ,  FIG. 1 ), the crewmember  116   a  (seated and belted in first-class zone  108 ) may have direct-view access to the image streams of zones  110 ,  112  captured by the cameras  104 ,  106  in addition to the image stream of the first-class-zone captured by the camera  102 . The cameras  102 ,  104 ,  106  may be partially or fully controllable by command crew or cabin crewmembers  116   a - c  (e.g., by panning a camera to shift its field of view or zooming the camera to more closely examine a passenger  118  captured in an image stream). Each camera  102 ,  104 ,  106  may be associated with a default configuration and orientation certified for direct view operations during TTL flight segments. For example, when direct view operations are initiated (e.g., when the IIS  100   a  is activated or direct-view mode engaged), the cameras  102 ,  104 ,  106  may revert to their certified configurations and orientations (e.g., direction, focus, settings) throughout the TTL phase (or until the direct-view mode is deactivated). Similarly, the IIS  100   a  may be connected to an interior lighting system of the aircraft  114 , such as the main cabin lighting system or a supplemental LED lighting system. For example, when direct view operations are initiated or the IIS  100   a  activated, cabin interior lighting may be optimized (e.g., by adjusting the brightness and/or orientation of one or more cabin lighting elements) to emphasize image clarity. For example, cabin lighting may be optimized (either manually or automatically) for maximum clarity when viewed by the human eye, in order to facilitate direct view by cabin crewmembers  116   a - c  or prevent interference with the cameras  102 ,  104 ,  106  (e.g., by preventing frequencies associated with LED cabin lighting from interfering with, or “whiting out”, video capture or display equipment). Alternatively, the cabin lighting, or cabin lighting elements associated with a particular zone ( 108 ,  110 ,  112 ) or FOV ( 102   a ,  104   a ,  106   a ) may optimize brightness or orientation for maximum clarity when viewed by an associated camera  102 ,  104 ,  106 . 
     The IIS  100   a  may include cameras mounted within, and capturing direct views of, other interior areas or exterior surfaces of the aircraft. For example, the IIS  100   a  may capture (and display to the cabin crew  116   a - c  via fixed-mount monitors  124  and/or mobile devices  128 ) image streams of cargo compartments, galley areas, crew rest areas, and other remote areas of the aircraft (e.g., remote areas of the main cabin, cargo compartments, rest areas, and other parts of the aircraft interior not directly visible to the cabin crew). Similarly, the IIS  100   a  may include cameras mounted to exterior surfaces of the aircraft to monitor, e.g., cargo doors or control surfaces. Interior cameras, such as the camera  106 , may be positioned and oriented so as to capture, through one or more windows ( 114   a ) of the aircraft  114 , an image stream including an engine  132  of the aircraft. In this way the camera  106  may assist in rapid detection of a failure of the engine  132  during takeoff (e.g., the engine failure associated with the aforementioned Manchester Airport accident). In addition, if the image stream generated by the camera  106  indicates that a large number of passengers within the FOV  106   a  are looking outside their windows in the direction of the engine  132 , this may indicate a potential problem with the engine  132 , even if the potential problem is not directly visible by the camera  106 . One or more of the cameras  102 ,  104 ,  106  may include an infrared (IR)-spectrum (NIR, SWIR, LWIR) thermographic imager for capturing thermal signatures (in addition to visible-light images) of exterior surfaces, cargo compartments, and other areas of the aircraft  114  (e.g., an anomalous thermal signature including the engine  132  may indicate a potential problem). 
     Referring now to  FIG. 2B , the IIS  100   b  may be implemented and may function similarly to the IIS  100   a  of  FIG. 2A , except that the IIS  100   b  may be a high privacy system including one or more deployable cameras  134 . For example, the deployable camera  134  may, during TTL procedures when direct view of passengers  118  and seating areas  120  is required (e.g., when the IIS  100   b  is activated or direct-view mode engaged), deploy into an active position or configuration ( 134   a ; e.g., from a Passenger Service Unit  136  (PSU) or similar overhead/interior fixture or structure) in an obvious and conspicuous fashion for direct viewing of passengers and seating areas in enhanced privacy sections of the aircraft  114  ( FIG. 2A ; e.g., the first-class zone  108  ( FIG. 1 )). When TTL procedures are complete (e.g., the IIS  100   b  is deactivated or direct-view mode disengaged), the deployable camera  134  may conspicuously retract into an inactive position or configuration (e.g., into the PSU  136 ) so as to reinforce to occupying passengers ( 118 ) the enhanced privacy associated with the first-class zone  108 . The deployable camera  134 , similarly to the cameras  102 ,  104 ,  106  ( FIG. 1 ) may be oriented with a generally downward FOV ( 134   b ) relative to the horizontal. This may increase the proportion of passengers  118  and seating areas  120  within direct view of the camera  134 . By contrast, the crewmember  116   a , at the front of the first-class zone  108 , may have at best an obstructed direct view ( 130   a ) from a seated position over bulkheads and partitions ( 126 ,  138 ) associated with enhanced-privacy seating ( 120 ). 
     Referring to  FIG. 3A , the IIS  100   c  may be implemented and may function similarly to the IIS  100   b  of  FIG. 2B , except that the IIS  100   c  may include processors ( 140 ) connected to each camera ( 102 ,  104 ,  106 .  134 ) physically or wirelessly. The processors  140  may receive raw image streams ( 142 ) from each camera  102 ,  104 ,  106 ,  134  for further processing; for example, the processors  140  may generate enhanced image streams ( 144 ) by combining one or more raw image streams  142  captured by one or more cameras  102 ,  104 ,  106 ,  134  or corresponding to one or more zones ( FIG. 1 ;  108 ,  110 ,  112 ) of the aircraft  114  ( FIG. 1 ). Enhanced image streams  144  generated by the processors  140  may generate virtual-reality or augmented-reality environments, in which one or more raw image streams  142  may be combined into a composite presentation corresponding to the aircraft  114  as a whole (or to one or more zones  108 ,  110 ,  112  thereof) and navigable by cabin crew ( 116 ,  FIG. 1 ) via a fixed-mount monitor  124  or mobile device  128 , e.g., crewmembers may scroll through multiple image streams or choose from a selection of image streams. Enhanced image streams  144  including virtual-reality or augmented-reality environments may be integrated with environmental data  146 ) stored by the IIS  100   c  (e.g., passenger data, seating data, baggage data, three-dimensional imagery corresponding to the aircraft  114 ). The IIS  100   c  may include, in addition to fixed-mount display units  124  and mobile devices  128 , transceivers  148  wirelessly connecting the IIS  100   c  to a ground-based control facility ( 150 ) for remote direct viewing of the passenger cabin in real time or near real time, to reduce the workload on the cabin crew  116 . The IIS  100   c  may include recording devices for recording and storing raw image streams captured by each camera  102 ,  104 ,  106  as well as any enhanced image streams or augmented/virtual reality environments generated from the raw image streams by the IIS. Recording devices may be incorporated into the IIS  100   c  aboard the aircraft, or the image streams may be forwarded to the ground-based control facility  150  for remote recording and storage. One or more of the cameras  102 ,  104 ,  106 ,  134  of the IIS  100   c  may be partially or fully controllable by the cabin crew  116  based on control input ( 152 ) entered through the fixed-mount display units  124  or mobile devices  128 . 
     Referring now to  FIG. 3B , the IIS  100   d  may be implemented and may function similarly to the IIS  100   c  of  FIG. 3A , except that the IIS  100   d  may combine multiple image streams ( 142   a - d ) from cameras  102 ,  104 ,  106   a - b  mounted within zones Z 1 , Z 2 , Z 3  ( 108 ,  110 ,  112 ) of the aircraft  114 . The cabin crew ( 116 ,  FIG. 1 ) may access, via mobile devices  128   a - b , composite enhanced image streams  144   a - b  incorporating multiple direct views of the passenger cabin, either composite image streams  144   b  incorporating multiple views (captured by cameras  106   a - b ) from within a single zone Z 3  ( 112 ) or composite image streams  144   a  incorporating multiple views (captured by cameras  102 ,  104 ) from different zones Z 1  ( 108 ) and Z 2  ( 110 ). 
     Referring to  FIG. 4 , the mobile device  128   c  may be implemented and may function similarly to the mobile devices  128  ( FIG. 1 ) and  128   a - b  ( FIG. 3B ) except that the mobile device  128   c  (or a fixed-mount display unit  124 ,  FIG. 1 ) may display an enhanced image stream  144   c  incorporating an augmented reality environment corresponding to the aircraft  114  ( FIG. 3B ). For example, the enhanced image stream  144   c  may provide direct view of passengers ( 118 ) and passenger seats ( 120 ) overlaid with tabs ( 154 ) corresponding to each occupied (or unoccupied) seat. A member of the cabin crew ( 116 ,  FIG. 1 ) may access additional environmental data ( 146   a ) about the seat  120  and/or its occupying passenger  118  (e.g., the passenger&#39;s name, corresponding ID photo, connection information, checked bags, dietary or other special needs, health considerations) by clicking, tapping, or otherwise interacting with the tab  154 . The enhanced image stream  144   c  may indicate ( 156 ) the locations of additional cameras within the aircraft  114 , so that the crewmember may toggle between enhanced image streams, e.g., by tapping or clicking a camera indicator  156 . Further, the enhanced image stream  144   c  may be linked to onboard sensors configured to provide additional occupant data, e.g., weight sensors in a seat to indicate whether or not the seat or module is occupied, harness sensors to indicate whether or not a seatbelt or security harness is fastened, or thermal imagers configured to determine whether an enhanced privacy suite, remote area or compartment is occupied (e.g., without necessarily providing a visual image of the occupying passenger). Based on the accumulated passenger data, for example, the IIS  100   d  ( FIG. 3B ) may determine to a sufficiently high degree of confidence that a given enhanced privacy compartment, remote area or compartment is occupied by a passenger who has fastened their seatbelt. 
     Referring to  FIG. 5 , the IIS  100   e  may be implemented and may function similarly to the IIS  100   d  of  FIG. 3B , except that the IIS  100   e  may include one or more cameras ( 158 ) incorporated into a Passenger Service Unit  136  (PSU) or into its supporting structure or rails. For example, the IIS  100   e  may be integrated into, or communicative with, a “smart PSU” system whereby the camera  158  and other cameras of the IIS  100   e  are controllable to capture image streams ( 142 ,  FIG. 3A ) of overhead bins ( 160 ) aboard the aircraft  114  ( FIG. 3B ). For example, the camera  158  may capture raw image streams ( 142 ,  FIG. 3A ), and the IIS  100   e  may assemble composite or enhanced image streams ( 144 ,  FIG. 3A ) of overhead bins  160  opposite the camera  158  (including the contents of said overhead bins, depending upon whether the overhead bins are open or closed). 
     It is to be understood that embodiments of the methods according to the inventive concepts disclosed herein may include one or more of the steps described herein. Further, such steps may be carried out in any desired order and two or more of the steps may be carried out simultaneously with one another. Two or more of the steps disclosed herein may be combined in a single step, and in some embodiments, one or more of the steps may be carried out as two or more sub-steps. Further, other steps or sub-steps may be carried in addition to, or as substitutes to one or more of the steps disclosed herein. 
     From the above description, it is clear that the inventive concepts disclosed herein are well adapted to carry out the objects and to attain the advantages mentioned herein as well as those inherent in the inventive concepts disclosed herein. While presently preferred embodiments of the inventive concepts disclosed herein have been described for purposes of this disclosure, it will be understood that numerous changes may be made which will readily suggest themselves to those skilled in the art and which are accomplished within the broad scope and coverage of the inventive concepts disclosed and claimed herein.