Patent Description:
Centre of gravity calculation is a safety critical task that is performed prior to departure of an aircraft. According to the value of the centre of gravity, pilots trim horizontal stabilizers in order to compensate the pitch moment generated by a forward or aft centre of gravity position. A centre of gravity outside the aircraft safety envelope can lead to dramatic consequences, which include tail strike, pitch-up during climb, stall and crash.

In a passenger aircraft, determining the total weight and distribution of the occupants, and of the hand baggage, is required in order to compute the centre of gravity of the aircraft. Conventional methods include flight attendants counting passengers manually and a rough weight estimate for adults and children is calculated. The estimate may change according to the season in order to, for example, consider the weight of the clothes worn by the passengers. Once the passenger distribution is known, the weight distribution in predetermined cabin zones is calculated and then entered manually into the Flight Management System. The Flight Management System then computes the centre of gravity of the aircraft. This can lead to inaccurate and time consuming processes. Moreover, the weight of the hand baggage is usually not considered. Normally the weight of the hand baggage is included in the overall passenger weight estimate or is simply neglected prior to flight. <CIT> discloses an aircraft weight and balance measurement system and method, which comprises an identification means which enables each person and/or object boarding an aircraft to be tracked, and their weight to be known or estimated. A processing system receives data from the identification means and produces an operational weight and balance calculation for the aircraft.

In one aspect, there is provided a system for calculating weight and distribution on an aircraft, as provided in claim <NUM>. The system comprises an aircraft, at least one cabin camera configured to view the cabin of the aircraft, an image collector configured to collect images from the at least one cabin camera and a processor. The processor is configured to continuously perform the following steps: a) detecting passengers and/or hand baggage from the images collected from the image collector, b) estimating positions of the passengers and/or hand baggage from the images collected from the image collector, and c) estimating weight of the passengers and/or hand baggage from the images collected from the image collector. The processor also calculates a real-time weight distribution of the aircraft based on the estimates provided in steps a) - c). The real-time weight distribution of the aircraft is determined pre-flight, during flight and on landing and the real-time weight distribution of the aircraft is continuously updated to a pilot and/or a Flight Management System.

The system may further include a first module configured to determine passenger weight distribution based on the images collected from the image collector, and a second module configured to determine hand baggage weight distribution based on the images collected from the image collector. The system may further comprise a third module configured to calculate cabin weight distribution based on the data from the first and second modules.

The first module may be configured to determine biometric information of the detected passengers and wherein the passenger weight distribution may be based on the biometric information, and the position, of the passenger.

The second module may be configured to determine the size and shape of detected hand baggage, and wherein the hand baggage weight distribution may be based on the size and shape of the detected hand baggage, and the position, of the hand baggage.

The image collector may be hardware or software.

In another aspect, there is provided a method of continuously calculating weight distribution on an aircraft, as provided in claim <NUM>. The method comprises collecting images from at least one cabin camera that is configured to view the cabin of the aircraft, detecting passengers and/or hand baggage from the images collected, estimating positions of the passengers and/or hand baggage from the collected images, estimating weight of the passengers and/or hand baggage from the images collected, and calculating a real-time weight distribution of the aircraft based on the estimated weight and estimated positions of the passengers and/or hand baggage. The real-time weight distribution of the aircraft is determined pre-flight, during flight and on landing and the real-time weight distribution of the aircraft is continuously updated to a pilot and/or a Flight Management System.

There may be provided a first module for detecting passengers, estimating positions of the passengers and estimating weight of the passengers. There may also be provided a second module for detecting hand baggage, estimating positions of the hand baggage and estimating the weight of the hand baggage. There may also be provided a third module for calculating the weight distribution of the aircraft based on the data from the first and second modules.

The first module may be configured to determine biometric information of the passengers.

The second module may be configured to determine the size and shape of detected hand baggage.

An example of a system <NUM> for calculating weight and distribution of an aircraft is shown in <FIG>. The system <NUM> includes one or more imaging apparatus, such as a first cabin camera 100a, a second cabin camera 100b and an nth cabin camera 100n, n being any number greater than two. The amount of cabin cameras, 100a-100n, is determined by the size of the aircraft and any number that is necessary to have a field of view of the entire cabin is envisaged in the system <NUM>. The cabin cameras 100a-100n ensure that the entire cabin is monitored for passengers and cabin baggage in order to effectively calculate the centre of gravity of the aircraft by calculating the weight and distribution of the passengers and cabin baggage. The cabin cameras 100a-100n monitor the entirety of the cabin whilst grounded before take-off, during flight and on landing of the aircraft.

As shown in <FIG>, system <NUM> includes an imaging collector <NUM> that collects the images from the cabin cameras 100a-100n. The imaging collector <NUM> may be hardware and/or software within the system <NUM>. For example, the image collector <NUM> may be hardware, as an independent device, or software, running either in an independent processor or a processor within the system, as mentioned below. When the cabin cameras 100a-100n are in operation (e.g. before take-off, in flight and/or on landing), it is envisaged that the imaging collector <NUM> continuously receives real-time images from the cabin cameras 100a-100n. The images are then fed to a processor (not shown) and the processor may include a first module <NUM> and a second module <NUM>. The first module <NUM> is a passenger weight distribution module. The first module <NUM> determines passenger weight distribution by reviewing the images collected by the imaging collector <NUM> in order to detect the passengers within the cabin, estimate the passenger positions and to estimate the weight of the passengers. As an example, a first cabin camera may be located to view rows <NUM>-<NUM> on an aircraft, and a second cabin camera may be located to view rows <NUM>-<NUM>. As passengers enter the field of view, the image collector <NUM> collects the images from the first and second cabin cameras. The first module <NUM> then detects passengers from the images collected from the image collector <NUM>, determines the passenger positions and calculates an estimated weight of the passengers for both sections of the aircraft (e.g. rows <NUM>-<NUM> and rows <NUM>-<NUM>). Of course, there may be more than <NUM> rows of seats and the amount of images collected will depend on the amount of cabin cameras that enable a full view of the passenger cabin.

In the instance where there are overlapping passengers, the system <NUM> compensates for this by comparing the estimated positions of the passengers in the cabin. When the position of the passengers in the cabin is estimated from the cabin cameras 100a-100n, the same position of the same passengers can be recognised by the system <NUM> in the cabin, and compensations can be made in order to remove the overlapping image of the passenger. This therefore results in an adjustment to compensate for overlapping passengers within the view of the cabin cameras 100a-100n.

As mentioned, the first module <NUM> can detect passenger objects in the images collected by the image collector <NUM>. An object detection model, such as MobileNet SSD could be used, as an example. Of course, other detection models could be used. Once the passenger has been detected by the first module <NUM>, the position of the passenger can be estimated. As the position of the cameras, and the layout of the cabin, is known by the processor, the first module <NUM> can match detected passengers to the position of the cabin (e.g. specific seats). On take-off and landing, the passengers are located in their seats, and, therefore, a static frame image process may be used to determine the position of the passengers at their seats via pixel association. It may be necessary to also know the passenger positions during flight - i.e., to compute the passenger locations all the time. As the position of passengers is to be detected continuously (e.g. during flight), a dynamic algorithm may be used in order to detect the passengers and update as the passengers move around the cabin.

The first module <NUM> can then estimate the passenger weight. In order to do this, the first module <NUM> may determine some biometric information from the images collected by the image collector <NUM>. As an example, the first module <NUM> may be able to determine whether a detected passenger is an adult or a child, and estimate a weight based on the determination. Other additional examples, that may be used in conjunction, could be height estimation via pixel detection, or otherwise; complexion estimation via posture detection; weight estimation based on face landmark extraction; clothing weight estimation based on skin detection, which consists of assessing the amount of skin that is detected for each passenger by the first module <NUM>. Based on one or more of these estimation techniques, the first module <NUM> can then provide an estimation of the weights of the passengers and their positions.

The second module <NUM> can detect hand baggage objects in the images collected by the image collector <NUM>. As mentioned above, and as an example, an object detection model, such as MobileNet SSD be used. Of course, other detection models could be used. Once the hand baggage has been detected by the second module <NUM>, the position of the hand baggage can be estimated. As the position of the cameras, and the layout of the cabin, is known by the processor, the second module <NUM> can match detected hand baggage to the position of the cabin (e.g. the position in the overhead locker, or under the seat in front of their seat). On take-off and landing, the passengers are located in their seats, and, usually, the hand baggage is placed under the seat in front of them or in the overhead locker. Therefore, hand baggage positions may be gathered statically or dynamically. In the static approach, the cabin cameras 100a-100n have the overhead lockers in full view and a static frame image process may be used to determine the position of the hand baggage via pixel association. As the position of hand baggage is to be detected continuously (e.g. during flight), a dynamic algorithm may be used in order to detect the hand baggage and update once the hand baggage has been stored.

The second module <NUM> can then estimate the hand baggage weight. In order to do this, the second module <NUM> may determine the size and shape of the hand baggage to determine an estimate of the weight. Further, the second module <NUM> may be able to use statistical analysis and regulations for baggage allowed in the cabin (e.g. based on the airline) and provide an average weight estimate. A computer vision algorithm could then compute the weight based on the statistical analysis and the hand baggage detected in the images. Based on one or more of these estimation techniques, the second module <NUM> can then provide an estimation of the weights of the hand baggage and their positions.

The data of the passenger weights and positions from the first module <NUM>, and the data of the hand baggage weights and positions from the second module <NUM> is then fed to a third module <NUM>. The third module <NUM> is a cabin weight distribution module. The third module <NUM> calculates a cabin weight distribution based on the data from the first and second modules <NUM>, <NUM>. The cabin weight distribution can then be used to calculate a centre of gravity for the aircraft. These values are then fed to the Flight Management System such that the centre of gravity of the aircraft may be updated, and the pilot may be informed of a change in the weight distribution. As shown in <FIG>, the processor loops back to the image collector <NUM> for continuous updating throughout the flight (including on take-off and landing) such that the pilot or the Flight Management System are updated of the weight distribution at all times.

An example of a method of using the system <NUM> of <FIG> is shown in <FIG>. As can be seen in <FIG>, the method includes:.

The steps above are also repeated throughout the entire flight (including take-off and landing) for up-to-date weight distribution calculations.

An example of passenger weight estimation is shown in <FIG>. The example shown in this figure could apply to many of the estimations discussed above in relation to <FIG>, but in the particular example shown in <FIG>, identification of summer or winter clothes is made to produce a weight estimate. At step <NUM>, the images are collected from the image collector <NUM> (shown in <FIG>). The processor then identifies passengers as discussed above. The processor determines if the passenger is an adult at step <NUM> and then determines if the passenger is wearing winter clothes after analysing the image at step <NUM>. Calculations are used at step <NUM> to determine if the passenger detected is an adult. If the calculations determine that the detected passenger is not an adult, the processor moves to step 304b and determines if the passenger is wearing winter clothes. If the passenger (i.e., a child passenger) is identified to not be wearing winter clothes, the processor identifies that the passenger is wearing summer clothes at step 306b and calculates a weight estimate accordingly. If the passenger is identified as wearing winter clothes, the processor at step 306a estimates a weight based on the passenger wearing winter clothes. If the calculations determine that the detected passenger is an adult, the processor moves to step 304a and determines if the passenger is wearing winter clothes. If the passenger (i.e., an adult passenger) is identified to not be wearing winter clothes, the processor identifies that the passenger is wearing summer clothes at step 305b and calculates a weight estimate accordingly. If the passenger is identified as wearing winter clothes, the processor at step 305a estimates a weight based on the passenger wearing winter clothes. Therefore, the passengers weight, whether a child or an adult, can be estimated using the steps shown in <FIG>.

Although identifying passengers wearing winter or summer clothes is described and shown in <FIG>, it is envisaged that similar processes may be used to determine the height of the passenger, size and shape of the baggage etc.. The passenger weight estimation is used in conjunction with the determination of the position of the passengers and/or baggage, as discussed above.

The systems and methods described above are therefore used to determine passenger and hand baggage weight distribution to provide an overall weight distribution of the aircraft. The overall weight distribution of the aircraft is determined in real-time before flight, during flight and during landing of the aircraft. The system and method provide an accurate weight distribution to a pilot or Flight Management System during the entire process of boarding, seating, flight, landing and disembarkation of the aircraft.

Claim 1:
A system (<NUM>) for
calculating weight and distribution on an aircraft, the system comprising:
an aircraft;
at least one cabin camera (100a-100n) configured to view the cabin of the aircraft;
an image collector (<NUM>) configured to collect images from the at least one cabin camera (100a-100n);
a processor configured to continuously perform the following steps:
a) detecting passengers and/or hand baggage from the images collected from the image collector;
b) estimating positions of the passengers and/or hand baggage from the images collected from the image collector;
c) estimating weight of the passengers and/or hand baggage from the images collected from the image collector;
d) calculating a real-time weight distribution of the aircraft based on the estimates provided in steps a) - c); and
characterized in that
the real-time weight distribution of the aircraft is determined pre-flight, during flight and on landing; and in that
the real-time weight distribution of the aircraft is continuously updated to a pilot and/or a Flight Management System.