Abstract:
A plurality of sensors of various types each initiating a marker signal as it produces a &#34;frame&#34; of information sensed, such as a camera film frame for example. A computer in the navigation system of the reconnaissance craft receives the marker signals and generates respective successive frame count numbers which are fed back to the proper respective sensors and recorded directly on the individual frames to whch they pertain while the frames are still being taken or sensed by the sensors. At essentially the same time as the frame count number is being processed, the computer outputs a block of information to be stored and recorded for that one frame (sensor product), this information including the identifying frame count number, the sensor which took it, the time of occurrence, and the geographical location of the event as provided by the navigation system. Thus, each frame of intelligence gathered and later processed on the ground is simply correlated with the recorded block of desired data concerning it by a ground interpreter. An additional feature is a process of including photographic and location data of certain known ground sites during the same reconnaissance flight, providing a comparison capability to enable correction of any nagivation system biases and thereby obtain very precise geographic coordinate location of the sensor products.

Description:
This application is a continuation in part of our application Ser. No. 624,116, filed Oct. 20, 1975, now abandoned. 
    
    
     BACKGROUND OF THE INVENTION 
     The invention herein described was made in the course of a contract with the United States Air Force. The present invention relates to multiple-sensor airborne reconnaissance systems, and more particularly, to a novel and rapid means for correlating the discrete products of the sensors and determining their geographic locations. The &#34;sensors&#34; are any of cameras, magnetic recording devices, a radar set, or other type of detector, all capable of simultaneous operation in the same system. 
     In conventional photo-reconnaissance systems where camera photos of the ground are taken from an aircraft flying along an area of interest, for example, the photo interpreter on the ground after the flight requires many hours to positively establish the location of the first picture or frame, and many additional hours for each following frame in succession, one by one. Where many photos are taken, especially from a plurality of cameras or other types of sensors, this large amount of time consumed is obviously unsatisfactory. The problem is that there seems to be no system or method using a common basic reference for correlation of the sensor events or outputs as they occur, and no fast accurate method of obtaining corresponding location or other data for the sensor products after they have been developed or processed, on the ground. 
     Accordingly, it is an object of the present invention to provide a sensor correlation system wherein, as soon as the collected and recorded data is available for inspection, any one selected photo frame or sensor product of interest can be immediately checked in a single simple step to determine its precise geographic location. Conversely, it is another object of this invention to provide a sensor correlation system wherein it can be determined, by a single simple scanning of a list, which sensor products (if any) cover any desired geographic locations of interest. In other words, the exact sensor product(s) for those locations can be identified and then picked out from all the collected data for examination. 
     A further object is to provide a sensor correlation system wherein no time-consuming step-by-step procedure for each individual successive information frame beginning with the first one taken, is required. 
     BRIEF SUMMARY OF THE INVENTION 
     Briefly, our invention comprises a correlation means and method wherein each sensor generates a marker signal for each frame of information which it takes or records and sends the signal to a computer associated with the navigation system of a reconnaissance vehicle. The computer produces a frame count number to identify each such frame in successive order of its occurrence, the frame numbers being in ascending numerical order for each sensor. The frame count number is sent back to the individual sensor which generated the corresponding marker signal and is recorded on that particular frame during the time it is being made or exposed. At the same time, the computer gathers and stores one line or block of information in a storage memory, this block containing the frame count number, the sensor number where the individual frame is to be found, the time of occurrence, the geographic location as read from the navigation system, and other data as desired, such as the altitude and attitude of an aircraft containing the present invention. For each following frame of information or reading from a sensor, the same procedure takes place. The stored or recorded blocks of information are later reduced or printed out to give a listing of all sensor activities arranged in the order of their occurrence and identified by the reference frame count number for each particular sensor. The intelligence content of each and every frame listed can thus be found immediately by going to that frame number of the designated sensor. 
     In addition, a method of getting extremely accurate coordinate location of each output frame is to proceed over a straight line course, for example, in the same area as where the reconnaissance data is being gathered, taking one or a few photos of a first location and a second, distant, location which are absolutely known in longitude and latitude beforehand, with the same system as described above, then comparing the listed results from this present system with the known information of the two locations to determine the error or bias of the navigation system, and then applying the proper difference correction to each line of recorded location as gathered. 
    
    
     This invention may be more fully understood by reference to the following description of a preferred embodiment and mode of operation read in conjunction with the accompanying illustrative drawings. 
     BRIEF DESCRIPTION OF DRAWINGS 
     FIG. 1 is a functional block diagram of a sensor system incorporating the present invention. 
     FIG. 2 is an overall component block diagram of the system. 
     FIG. 3 is a block diagram of a camera used as a first sensor. 
     FIG. 4 is a diagram of a magnetic recording device used as a second sensor. 
     FIG. 5 is a diagram of a radar device used as a third sensor. 
     FIG. 6 is a block diagram of the signal processor of FIG. 2. 
     FIG. 7 is a block diagram showing typical components in the computer. 
     FIG. 8 is a plan diagram illustrating a correction procedure for obtaining more precise data from the sensor system. 
     FIG. 9 is a block diagram illustrating the functions of the computer of FIG. 7, and 
     FIGS. 10 and 11 are flow charts illustrating the programming of the computer of FIG. 7. 
    
    
     DETAILED DESCRIPTION OF INVENTION 
     Referring first to FIG. 1 for an overall view of the system, three sensors 1, 2 and 3 are shown. As mentioned before, more sensors may be used. The embodiment to be described as an example accommodates seven sensors. Each sensor generates a marker signal when an event is being sensed by that sensor and sends it via a marker line 4, 5 or 6 to a digital computer 7. Assuming sensor 1 is a camera, it produces an output frame 9. The computer processes the event (frame) marker signal, computes and stores the precise event time, the aircraft&#39;s position from a navigation system 10, and other data. In addition, computer 7 simultaneously stores and transmits a frame count number 11 for each marker signal back to the respective sensor on the proper frame count line 12 to be impressed on the corresponding output frame 9. Various navigation parameters along with the corresponding frame count numbers 11 are recorded on a magnetic tape 14 for use by intelligence interpreters on the ground. As can be seen, a print-out of the tape 14 has recorded on it a number of lines of information, each line containing the frame count number 11, sensor number, and all the identifying information about a sensor event, and this line refers to the photographic or other intelligence content of the respective sensor number and frame number thereof. The frame count number 11 also entered on the photographic frame itself positively and quickly identifies the information listed on the tape 14. 
     In turn, the next output event (frame) 15 of the first sensor 1 is similarly handled, together with the other sensors 2 and 3 which may be simultaneously operating and collecting their own sensed information on second sensor frame 16, etc. The point is that the simple basic correlating means is the frame count number 11, which is started by the event at the individual sensor, carried to the computer 7, given its number, and sent back to the sensor in time to be actually indicated in readable form on the intelligence-containing frame itself of that sensor and that sensor only. 
     Referring now to FIG. 2, each sensor 1, 2, -n actually sends its marker signals on marker lines 4, 5, -N to a signal processor 17 which in turn sends it on to the computer 7. The computer generates and sends back to the signal processor 17 the sensor event number (frame count number) 11 in preferably serial form on a frame count line 19, a three-bit sensor number in parallel form on sensor number code lines 20, 21 and 22, and a correlating clock pulse signal on clock signal line 23. The clock signal, as is conventional in digital computers, is formed from a clock pulse generator 28 (FIG. 7) in the computer 7, which is common technology. From the processor 17, the frame count number 11 in serial form is sent to the proper sensor on either frame count lead 25, 26 or 27, together with the clock pulse signal on clock signal leads 29, 29a and 29b. For the purpose of our invention, however, it is not mandatory that the frame count number be in serial format. 
     The computer 7 further sends all the data to be recorded for one event to a tape recorder 30 (in preferably serial form) on recording line 31, together with the same clock pulse signal on clock signal conductor 32. A chronometer 34 provides the correct time in pulse form to the navigation system 10 on time line 35. 
     FIG. 3 shows the detailed operation of this invention starting with the structure of a sensor where the marker signals first originate. In this drawing the sensor 1 is an airborne photographic camera which may be started or stopped at any time by means of a first on-off switch 40 controlling a camera motor 41. As the camera runs, a series of film frames such as the frame 42 is produced from a scene on the ground, for example. Somewhere near the start of exposure of each frame, a mechanical connection 44 from the motorized film drive triggers a marker pulse generator 45 which produces a marker pulse on output marker line 4. This pulse is transmitted through the signal processor 17 to the computer 7, starting the train of operation resulting in the proper frame number 11 for the frame now being taken to be serially entered on frame count lead 25 to a holding register 46 in the sensor package 1. Clock pulses on clock signal lead 29 read out the frame number to a decoder 47 comprising well known control logic. This decoder 47 supplies voltages to the proper elements of a display unit 49 comprising a plurality of light-emitting diodes, for example, which are thus illuminated to indicate the frame count number. Optical means (not shown) produces an image 50 of the count number directly on a chosen portion of the film frame 42, such as in a lower corner for example. In order to synchronize the time of the exposure of the display unit 49 with the film, another mechanically operated interconnect 51 triggers a trigger generator 52 which releases the output of the decoder 47 at a chosen time and position portion of the film exposure period. This may be near the center of the frame length, at the end, or between frames, for example. Therefore, one marker pulse from the sensor 1 and one entry of the frame count number 11 occur for each film frame taken, the marker pulses and the entries both being produced one frame period apart during camera operation. Obviously the details of this camera sensor can be modified as desired. 
     FIG. 4 shows details of the sensor 2 constituting a magnetic recording device, for example, which records events in sequential mode. It is illustrated as comprising a second on-off switch 55, a conventional tape transport 56, recording head 57 and magnetic recording medium 59 such as tape. Magnetic data being sensed on input line 60 is thus recorded by this sensor. 
     This magnetic sensor 2 is arranged to function essentially equivalent to the camera sensor 1. Driving means 61 from the tape transport 56 controls a trigger generator 52a to cause a marker pulse generator 45a to emit a marker pulse on marker line 5 for each chosen fixed length or amount of magnetic medium 59, which amount is designated as a &#34;frame&#34; for purposes of this invention. Frame count lead 26 brings in from the computer 7 the proper frame count number for this sensor and enters it in a register 46a. It is clocked from there into a decoder 47a which drives a magnetic pulse circuit 62 (including an additional recording head) and thus records the frame count number onto the magnetic medium 59. The trigger generator 52a times the release of the frame number from the decoder 47a to the magnetic circuit 62. Instead of the additional recording head and magnetic pulse circuit 62, a summing junction or multiplexing device (not shown) might be provided in the main recording circuitry so that the existing recording head 57 may be used to impress the frame number on the medium 59. In this case, if recorder speed and frequency response are compatible with the serial output rate of the processor 17, the holding register 46a may be unnecessary, and only the trigger decoding logic 47a is required to initiate frame count number recording. 
     FIG. 5 shows a third sensor as being a radar device. There are several ways by which to implement a radar set to fit the present invention, one example only being illustrated herein. Radar data being sensed is fed to a receiver 65 and the final desired information displayed on a cathode ray tube 66. An oscilloscope camera 67 photographs the CRT 66 as often as desired, each camera picture being a frame to be correlated in this sytem. Again, the camera 67 drives a mechanical link 69 to energize another marker pulse generator 45b each time a frame is exposed, as in the other sensors 1 and 2. The frame count number sent back by computer 7 is entered in a holding register 46b and clocked into a decoder 47b. Another mechanically operated trigger generator 52b subsequently empties the decoder 47b to apply the frame count number 11 to the photographed frame. The actual display is here accomplished on the CRT by a Z-axis modulator 70 operating from the decoder 47b and synchronized with the CRT deflection sweeps to properly position the frame number 11 on the CRT face as desired. Alternatively, an L.E.D. display might of course be coupled into the optical path to the oscilloscope camera 67 if display on the CRT face is not desired. 
     Other sensors, either more of the same type as hereinbefore described or other kinds may be adapted to be added to or substituted in the correlation system in accordance with the present invention. There is ample time from one sensor frame to the corresponding point in the next frame of the same sensor for the computer and other processing circuitry to receive each marker pulse and send back the frame count number to the sensor and imprint it on that one sensor frame. 
     The signal processor 17 is further shown in FIG. 6. Here, the incoming marker pulses on lines 4, 5, N are each applied to a respective marker amplifier 74, 74a and 74b before going from the processor 17 to the computer 7 on their own separate marker pulse input lines 75. The clock pulses from the computer 7 on clock signal line 23 are fed in parallel to a number of clock signal amplifiers 76 equal to the number of sensors used in the system, and the identical signal appears on clock signal leads 29, 29a and 29b from the outputs of each respective sensor used. 
     The frame count line 19 from the computer 7 enters the processor 17 and connects via gate input leads 77, 77a and 77b in parallel to a number of AND gates 79 equal to the number of sensors. The sensor number code lines 20, 21 and 22 enter the signal processor 17 from computer 7 and feed to a sensor select logic circuit 80. The output of this conventional type logic circuit energizes just one of a number of gate control leads 81, 81a and 81b, depending upon the binary coded input on the three-bit sensor code lines 20, 21 and 22. These three code lines will obviously provide enough capacity for seven sensors. More sensors could of course be accommodated by including a fourth code line to the logic 80. It is thus seen that the serial frame count number will appear on the output lead of only one AND gate 79 and hence will go only to the sensor to which it belongs, i.e., the one from which the last marker pulse came. 
     The digital computer 7 within the navigation system 10 contains information relative to real time, aircraft position (latitude, longitude), aircraft altitude, attitude and the like at all times. This information may be transmitted or fed out in the form of digital serial words, along with the usual controlling clock pulse signal, to the tape recorder 30, upon command. 
     Receipt of a marker pulse from a sensor by the computer initiates, essentially in parallel, two basic sequences of events. 
     I. the computer generates the frame count number 11 for the particular sensor number. The frame count number generally starts with number 0001 and for each subsequent event (frame) for that particular sensor, the previous computer memory stored number is increased by 0001. The frame count number is fed to the signal processor 17 in serial format on frame count line 19, along with the correlation clock signals on clock signal line 23. At the same time, the sensor number designation is supplied from the computer in parallel format on the three sensor number code lines 20, 21 and 22, which three-bit code allows for unambiguous selection of any one of up to seven sensors. One additional code line will provide for 15 sensors, and so on. 
     Ii. the computer initiates the process of &#34;outputting&#34; to the recorder 30 on the recording line 31 the various information in preferably serial format mentioned before in regard to FIG. 1, along with the correlation clock pulse signal on clock signal conductor 32. 
     If more than one sensor marker pulse occurs simultaneously, computer 7 processes the signals in a priority sequence, i.e., one after another, in an anti-coincidence detector which is part of the select logic block 84 in FIG. 7. 
     This computer 7 may be any general purpose digital computer programmed by well known techniques to provide the above required functions. FIG. 7 illustrates the general computer make-up. Various different manners of implementing the computer for this job are of course suitable in the actual practice of this invention. 
     The following will explain one way the computer could be implemented to enable operation of the present invention. The navigation system computer 7 in the vehicle in which this invention is incorporated is programmed to compute navigation and other data, as is customary. The system of the invention employs such computer which will be adapted (further programmed) to function as required. FIG. 9 shows the computer memory 24 divided into sections A, B and C, for example, which can be memory portions or channels of any common digital computer. These sections will be briefly explained so that the following flow charts of softwave operation in FIGS. 10 and 11 will be clarified. 
     Memory section C contains the main navigation program which is continuously executing instructions at a rate determined by the computer clock pulse generator 28. This main program was initially started at a very precise time according to Coordinated Universal Time (C.U.T.) as determined by external input from a chronometer synchronized with radio station WWV for example. During program execution, data from the aircraft data computer and from the navigation system are received and manipulated by this program to compute the craft&#39;s location as precisely as possible in latitude and longitude. 
     The main program in section C is recycled or repeated at a rate R while being internally interrupted periodically at a rate which is a multiple of R, i.e., faster. One reason for these regular interruptions is to increment the initially read time (C.U.T.) to keep an accurate updating of time, since the outside chronometer was read only at the start, and from then on, time is kept by a precise crystal control within the system which is read each second, for example. During cycling of the program in section C, data necessary to the system is being outputted from C to recorder 30 (see FIG. 2) at a certain known constant rate compatible with recorder speed capability. This program also functions to read the outputs of a real time counter 90 (FIG. 7) at precisely the times which the latitude and longitude are calculated and outputted to the recorder 30. The real time counter 90 counts at a rate faster than the above program interrupt, and by using this rate in conjunction with computer updated time, C.U.T. time T can be refined by shorter units (ΔT) to achieve greater resolution. 
     Memory section B contains programs which are executed only on demand by section C through subroutine call or the above program interrupt, or on demand from section A when an external interrupt preempts operation from C, such as may be called for in the operation of the present invention. 
     Memory section A comprises a series of storage locations which are hard-wired to respond to stimuli from external sources such as the sensor marker signals entering the computer on the marker pulse input lines 75. The storage locations contain a single program instruction which executes a &#34;branch&#34; or &#34;jump&#34; to the proper program in section B. As an example, when a marker pulse from sensor 2 on marker line 5 is activated, an external interrupt is generated which causes the main program in section C to cease operation at a step &#34;z&#34; for example. Control is then transferred to section A, memory location 2. The instruction residing in location 2 is executed and a &#34;jump&#34; to the program for sensor 2 activity in section B takes place. This latter program is executed uninterrupted to its completion, and control is then returned to C at step &#34;z + 1&#34; and normal operation resumed until again interrupted by some internal or external source. 
     The present invention may be performed by programming the memory sections B and C as shown in FIGS. 10 and 11, respectively. 
     FIG. 10 shows the program path taken upon receipt of a sensor marker signal for sensor number 2 as an example. The main program in section C is briefly interrupted and control is transferred to section A via hardward functions. The software now executes the &#34;branch&#34; to section B. The software program in B manipulates data for an essentially real time output to sensor 2 of the identifying frame count number. Time (T + ΔT) is saved for outputting to recorder 30 (along with the frame count number) in a program imbedded in memory section C. The program in C has an iteration rate faster than the sensors can operate. Therefore, there can be no ambiguity of data presented on the recorder 30 when subsequently (on the ground usually) correlating a frame count number in the recorder with the same frame count number on the sensor intelligence frame. As shown in FIG. 10, the software instructions in section B have control over setting the three-bit sensor number code for computer output on code lines 20, 21 and 22, and over transmitting the frame count number from the computer on count line 19. 
     FIG. 11 illustrates the segment of the large program in section C which is triggered by the particular program executed in section B (FIG. 10). This program in C finishes the processing of the sensor activity and initializes parameters that will be used in section B at the time of the next sensor event. 
     While most general purpose computers can be programmed to perform the functions of digital computer 7, the preferred embodiment of the invention has been implemented with Northrop Computer Model NDC1070, manufactured by Northrop Corporation, Hawthorne, California. A program which implements the functions set forth in the flow charts of FIGS. 10 and 11 in the aforementioned Northrop computer, is set forth on the following pages as follows: 
     On pages 35 and 36 a summary of the assembler instructions is set forth, while on page 18 the repertoire for the program is set forth. The sensor code library for the program is set forth on pages 24 and 25. The program for a single sensor (sensor 2) is set forth on pages 26-34. The same program can be used for any of the other sensors. The various portions of the program are appropriately identified. 
     Thus, positive and rapid post-mission correlation of a photograph, for example, with its identification information from the tape recorder 30, for example, can easily be accomplished. As has been described, both the photograph (or other sensor output frame) and the recorded block of information carry the same frame count number, and the remainder of the recorded block gives the precise time of the frame event plus its geographical location and such other data as chosen to be included. 
     It will be noted that a variation of this system could be employed wherein, although the particular sensor from which each marker pulse comes must be identified, the frame count numbers could be assigned in successive order as they occur in any sensor instead of in each sensor. Thus, there would be no repetition of actual frame count numbers from the system, and numerical gaps could exist between frame count numbers of consecutive intelligence frames of any one sensor. In other words, frame count No. 0001 could be from sensor III, No. 0002 from sensor II, Nos. 0003-0004 from sensor I, No. 0005 from sensor IV, Nos. 0006-0010 from Sensor II, etc., for example, just as they happened to occur in actual time. Such variation is deemed to lie within the scope of the appended claims. 
     The accuracy of the geographic location of the items recorded on film and the like by the sensors can be improved by the use of an additional procedure, involving the correction of navigation system errors. During the flight as the aircraft gathers the various sensor data, the aircraft would fly over two fairly well spaced and geographically known points normally located at the beginning and the end of a &#34;take leg&#34; 86 as shown in FIG. 8, taking one or a few photographs of them. Along the take leg 86 is where the sensor data would preferably be taken, and there can be more than one take leg on a given flight. From the location which the recorded flight information indicates these points to have, compared with the actual known locations, the navigation system error or bias can be found. Then the appropriate location correction factors can be computed and applied to all the recorded readings taken on that take leg 86, resulting in extreme accuracy of coordinate location. To apply this procedure in the simplest manner, the aircraft would fly a straight and level course over points A and B in the area of interest to determine the navigation errors. The check points, such as A and B, will of course be prominent structures, previously located land features, or other well mapped points. 
     By corrected reprint of the list of frame count number information shown in FIG. 1, i.e., corrected as to navigation system bias as just described, the processing time that has heretofore been required by ground interpreters will be greatly reduced. 
     While in order to comply with the statute, the invention has been described in language more or less specific as to structural features, it is to be understood that the invention is not limited to the specific features shown, but that the means and construction herein disclosed comprise the preferred mode of putting the invention into effect, and the invention is therefore claimed in any of its forms or modifications within the legitimate and valid scope of the appended claims. ##SPC1## ##SPC2##