Method and system for determining the location in 3D space of an object within an enclosed opaque container

A method and system for determining the location in 3D space of an object of interest within the interior region of an enclosed, opaque container. The invention allows a user or operator to construct a three-dimensional representation of the interior region of the container to allow viewing of objects, components and substances within the interior region. The users or operators now have the opportunity to isolate a particular object of interest within the interior region that may be a threat, such as an explosive device or other energetic component. A disrupter device is aimed at the three-dimensional location and thereafter, the disrupter device fires a projectile or substance at the object of interest in order to disable or destroy the object of interest.

CROSS REFERENCE TO OTHER PATENT APPLICATIONS

FIELD OF THE INVENTION

The present invention relates to a method and system for determining the location in 3D space of an object within an interior of an enclosed, opaque container.

BACKGROUND

Conventional methods and systems to visualize and aim at the contents of the interior of an opaque package or container typically use a fiducial grid in conjunction with an x-ray radiation source. Visualizing the contents of the interior of the opaque item is necessary in order to identify any objects of interest. Typically, an object of interest is considered a suspicious object that may be an explosive device or some other type of energetic. The object of interest is usually adjacent to or surrounded by other objects that may not be deemed suspicious or threatening. A disrupter device or tool is aimed at the location of the object of interest. The disrupter device or tool then fires or propels a projectile or substance at the object of interest in order to disable or destroy the object of interest. However, such conventional techniques and methods limit the aiming position of the disrupter device exclusively to the position of the x-ray radiation source. Consequently, such a limitation significantly constrains the space in which operators or personnel may operate. Furthermore, if it is not feasible to fire the disrupter device from a particular location and/or angle due to sensitive obstructions, it will be necessary to re-position the x-ray radiation source so as to determine if there is a feasible line-of-fire from another position or angle that will not impact such sensitive obstructions or other objects not deemed to be threatening.

What is needed is a new and improved method and system for accurately visualizing the contents of an opaque item and determining the three-dimensional location of an object of interest located within the interior of an opaque item.

SUMMARY OF THE INVENTION

In some exemplary embodiments, the present invention is directed to a method and system for accurately visualizing the contents of an enclosed opaque container and determining the three-dimensional location of an object of interest located within the interior of an opaque item. Specifically, the method and system of the present invention allow a user or operator to construct a three-dimensional representation of the interior region of the enclosed opaque container so as to allow viewing of objects, components and substances within the interior region. The users or operators now have the opportunity to isolate a particular item of interest within the interior region that may be a threat, such as an explosive device or other energetic component. A disrupter device is aimed at the three-dimensional location and thereafter, the disrupter device fires a projectile or substance at the object of interest (i.e. target) in order to disable or destroy the object of interest. The system includes a digital camera, a display element or display screen, a processing element in electronic data signal communication with the digital camera and display element, and at least one memory medium in electronic data signal communication with the processing element. In some exemplary embodiments, the processing element comprises one or more processors. In an exemplary embodiment, the camera and display element are positioned or located adjacent to the disrupter device. In some embodiments, the camera, display element and processing element are realized by a smart phone. The camera is configured to provide live video feeds. The camera, display element, and processing element cooperate to provide augmented reality (AR) and virtual reality (VR) imagery. The processing element executes one or more algorithms, including advanced computer vision (ACV) and digital image processing algorithms, which provide visualization, based on a virtual reality environment and/or augmented reality environment. The system includes a collapsible frame having two partition members or walls. In some exemplary embodiments, the partition members are movably attached to each other such that the collapsible frame opens and closes like a book. Each partition member has a unique image and a plurality of tungsten fiducial markers. The collapsible frame is placed near the enclosed opaque container so that one partition member faces one side of the enclosed opaque container and the other partition member faces the other side of the enclosed opaque container. The computer vision algorithms enable recognition of the unique image on each of the partition members. A first x-ray film is placed opposite one side of the opaque item such that the enclosed opaque container is positioned between a first partition member and the first x-ray film. A second x-ray film is placed opposite another side of the enclosed opaque container such that the enclosed opaque container is positioned between a second partition member and the second x-ray film. Two x-rays are taken along planes that are orthogonal to the unique images on both partition members. The processing element determines orthogonality of the x-ray radiation with respect to the unique images on the partition members. Once the x-rays are taken, the x-ray images are digitized and imported into at least one memory medium that is in electronic data signal communication with the processing element. The tungsten fiducial markers are visible on the digitized x-ray images. Each digitized x-ray image is then scaled, positioned and oriented. The unique images on each partition member do not appear on the digital x-ray images. The processing element processes the images of the partition members captured by the camera and executes software that generates a virtual reality depiction of each unique image. The tungsten fiducial markers are represented by corresponding icons in the virtual reality depictions of the unique images. In an exemplary embodiment, each fiducial marker shown in a virtual reality depiction is represented by a different color icon. As a result, for each digital x-ray image, there is a corresponding virtual reality depiction of the predefined unique image that is on the real world partition member that was x-rayed. One at a time, each digital x-ray image is selected and imported to the processing element. For the first selected digital x-ray image, the first step is to retrieve the virtual depiction of the predefined unique image that is on the real world partition member that corresponds to that first selected digital x-ray image. Two fiducial markers on the virtual reality depiction are selected, one at a time, and are mapped to corresponding fiducial markers on the digital x-ray image in order to define a first reference point. The same mapping steps are done for the second digital x-ray image so as to produce a second reference point. During the mapping procedure, a user or operator can zoom in and out and/or drag the digital x-ray image for better accuracy. The processing element implements a scaling process that uses the reference points produced in the mapping step to calibrate the digital x-ray images to true size and position. In some embodiments, the scaling process allows the user or operator to input, by the graphical user interface, the real distance from the unique image on the real world partition member to the corresponding x-ray film. The aforementioned method steps generate a three-dimensional representation of the interior region of the enclosed opaque container which allows users or operators to view the objects, components, substances, etc. that are within the interior region. Objects of interest within the interior region are then identified and highlighted using geometrical shapes provided by the software executed by the processing element. The geometrical shapes can moved, scaled, rotated and deleted. The color of the geometrical shapes may be changed to distinguish between the object of interest and other objects that are not of interest. A calibration scheme is then implemented which provides disrupter sight-alignment. In the calibration scheme, two points in three-dimensional space are selected and used to create a vector that is aligned with the longitudinally extending axis of the bore of the disrupter device. The camera is shifted so that the view of the camera is aligned with the vector so as to provide a view as seen from the bore of the disrupter, also referred to as “boresight perspective”. Therefore, the vector constitutes an aiming vector for the disrupter device. The disrupter device fires or propels the projectile along the aiming vector such that the projectile impacts the object of interest within the interior of the enclosed opaque container.

A significant feature of the present invention is that the advanced computer vision (ACV) algorithms executed by the processing element provide the camera, or other augmented reality (AR) device, with spatial awareness and enables generation of a three-dimensional (3D) rendering of the x-rayed contents of the enclosed opaque container in true space and position.

In some embodiments, the present invention is directed to a method and system for accurately visualizing the contents of an enclosed, opaque container and determining the three-dimensional location of an object of interest located within the interior of the enclosed opaque container. The method and system of the present invention allow a user or operator to construct a three-dimensional representation of the interior region of the enclosed, opaque container so as to allow viewing of objects, components and substances within the interior region. As a result, users have the opportunity to isolate a particular item of interest (i.e. target) within the interior region of the enclosed, opaque container that may be a threat, such as an explosive device or other energetic component. The enclosed, opaque container may be any type of nontransparent article having an interior region in which objects may be kept or stored. Typical enclosed opaque containers include luggage, knap sacks, brief cases, shipping crates, barrels or any other nontransparent container, box or package. The method of the present invention further comprises providing a first real world partition member having a surface with a predefined unique image thereon and a second real world partition member having a surface with another predefined unique image thereon. The predefined unique image on the first real world partition member is different than the predefined unique image on the second real world partition. Each real world partition member is fabricated from a relatively low density material and includes a plurality of fiducial markers. The first real world partition member is positioned so as to face one side of the enclosed, opaque container and the second real world partition member is positioned so as to face another side of the enclosed, opaque container. In some embodiments, the first real world partition member and second real world partition member are movably attached together so as to form a collapsible frame that can be opened or closed. The method includes providing a first digital data set that defines the unique image on the first real world partition member and a second digital data set that defines the unique image on the second real world partition member. The method includes providing an image capturing device, a display element, at least one memory medium and at least one processing element that is in electronic data signal communication with the image capturing device, display element and at least one memory medium. At least one processing element is programmed with the first digital data set and the second digital data set and is also programmed to generate a graphical user interface for display on the display element. The image capturing device captures the unique image on each real world partition member. At least one processing element processes the unique images captured by the image capturing device so as to generate a virtual reality environment based on a virtual reality coordinate system. The virtual reality environment includes virtual depictions of the real world partition members and the corresponding unique images thereon. The method includes generating x-ray radiation that penetrates the first real world partition member and the enclosed, opaque container so as to produce a first x-ray image and generating x-ray radiation that penetrates the second real world partition member and the enclosed, opaque container so as to produce a second x-ray image. The first x-ray image and second x-ray image are digitized so as to produce digital x-ray images. The digital x-ray images are imported into the memory medium. The method includes, for each real world partition member and corresponding digital x-ray image, displaying the virtual depiction of the real world partition member and the predefined unique image thereon, wherein the three-dimensional location of the virtual depiction is based on a virtual environment coordinate system. In an exemplary embodiment, the fiducial markers on the virtual depiction of the real world partition member are depicted as different colored icons and the location of each colored icon in the virtual depiction corresponds to the location of a corresponding fiducial marker on the real world partition member. A user uses the graphical user interface to select a first colored icon on the virtual depiction of the real world partition member. The selected first colored icon is then mapped to a corresponding fiducial marker shown in the digital x-ray image corresponding to the real world partition member so as to define a first point. A user then uses the graphical user interface to select a second colored icon on the virtual depiction of the real world partition member. The selected second colored icon is then mapped to a corresponding fiducial marker shown in the digital x-ray image corresponding to the real world partition member so as to define a second point. The method includes scaling, positioning and orienting the digital x-ray image corresponding to the real world partition member based on the mapping of the first colored icon and the second colored icon to the respective corresponding fiducial markers shown in the digital x-ray image. Thereafter, the scaled, positioned and oriented digital x-ray images are reviewed to determine if any object inside the opaque item warrants further examination or is considered an object of interest or a suspicious object. If it is determined that an object inside the opaque item warrants further investigation or is considered a suspicious object, then the method includes, for each scaled, positioned and oriented digital x-ray image, displaying, by the display element, the scaled, positioned and oriented digital x-ray image and the virtual depiction of the predefined unique image of the real world partition member that corresponds to the scaled, positioned and oriented digital x-ray image, selecting a reference geometrical shape from a plurality of reference geometrical shapes and dragging the selected reference geometrical shape over the scaled, positioned and oriented digital x-ray image, and matching the selected reference geometrical shape to an X-Y coordinate of the suspicious object shown in the scaled, positioned and oriented digital x-ray image so as to yield the three-dimensional (3D) location of the suspicious object in relation to the virtual depiction of the predefined unique images on the real world partition members. The color of the geometrical shapes may be changed to distinguish between the object of interest from other objects that are not of interest. A calibration scheme is then implemented which provides disrupter sight-alignment. In the calibration scheme, two points in three-dimensional space are selected and used to create a vector that is aligned with the longitudinally extending axis of the bore of the disrupter device. The camera is shifted so that the view of the camera is aligned with the vector so as to provide a view “as seen” from the bore of the disrupter, also referred to as “boresight perspective”. Therefore, the vector constitutes an aiming vector for disrupter device. The disrupter device fires or propels the projectile along the aiming vector such that the projective impacts the object of interest within the interior of the enclosed, opaque container.

In some embodiments, the present invention is directed to a system for determining the location in 3D space of an object within an interior of an enclosed, opaque container. The system comprises a first real world partition member having a surface with a first predefined unique image thereon. The first real world partition member is positioned so as to face one side of the enclosed opaque container. The system includes a second real world partition member having a surface with a second predefined unique image thereon. The second real world partition member is positioned so as to face another side of the enclosed opaque container. Each real world partition member is fabricated from a relatively low density material and includes a plurality of fiducial markers. The system further includes an apparatus configured to generate a first x-ray radiation that penetrates the first real world partition member and the enclosed, opaque container so as to produce a first x-ray image that corresponds to the first real world partition member and which shows the fiducial markers of the first real world partition member and a view of the interior of the enclosed, opaque container from a first angle. The apparatus generates a second x-ray radiation that penetrates the second real world partition member and the enclosed, opaque container so as to produce a second x-ray image that corresponds to the second real world partition member and which shows the fiducial markers of the second real world partition member and a view of the interior of the enclosed, opaque container from a second angle. The system further includes a device to digitize the first x-ray image and the second x-ray image to produce a first digital x-ray image and a second digital x-ray image. The system further includes an image capturing device configured to capture an image and provide the captured in digital form, a display element, one or more processors in electronic data signal communication with the image capturing device and the display element, and at least one memory in electronic data signal communication with the one or more processors. At least one memory includes data storage resource for storing a first data set that defines the first predefined unique image and a second data set that defines the second predefined unique image. At least one memory further includes computer readable code executable by the one or more processors to:

generate a graphical user interface for display on the display element;

capture, by the image capturing device, the first predefined unique image on the first real world partition member and the second predefined unique image on the second real world partition member;

process, by the one or more processors, the captured first predefined unique image and second predefined unique image to generate a virtual reality environment that includes virtual depictions of the first real world partition member with the first predefined unique image thereon and the second real world partition member with the second predefined unique image thereon; and

import the first digital x-ray image and a second digital x-ray image into the at least one memory medium.

For each real world partition member and corresponding digital x-ray image, at least one memory includes computer readable code executable by the one or more processors to display, by the display element, the virtual depiction of the predefined unique image and fiducial markers of the real world partition member. The fiducial markers are shown as a plurality of icons in the virtual depiction. Each icon is differentiated from the other icons and has a specific location that corresponds to a location of a corresponding fiducial marker on the real world partition member.

At least one memory includes computer readable code executable by the one or more processors to:

prompt a user to select, using the graphical user interface, a first icon on the virtual depiction of the real world partition member;

prompt a user to map, using the graphical user interface, the selected first icon with a corresponding fiducial marker shown in the digital x-ray image corresponding to the real world partition member so as to define a first point;

prompt a user to select, using the graphical user interface, a second icon on the virtual depiction of the real world partition member;

prompt a user to map, using the graphical user interface, the selected second icon with a corresponding fiducial marker shown in the digital x-ray image corresponding to the real world partition member so as to define a second point;

scale, position and orient the digital x-ray image corresponding to the real world partition member based on the mapping of the selected first icon and the selected second icon with the respective corresponding fiducial markers shown in the digital x-ray image that corresponds to the real world partition member; and

prompt the user to review, using the display element, the scaled, positioned and oriented digital x-ray images to determine if there is an object of interest inside the enclosed opaque article.

At least one memory includes computer readable code executable by the one or more processors to, for each scaled, positioned and oriented digital x-ray image:

display, by the display element, the scaled, positioned and oriented digital x-ray image and the virtual depiction of the predefined image of the real world partition member that corresponds to the scaled, positioned and oriented digital x-ray image;

prompt a user to select, by the graphical user interface, a reference geometrical shape from a plurality of reference geometrical shapes and drag the selected reference geometrical shape over the scaled, positioned and oriented digital x-ray image; and

prompt a user to match, using the graphical user interface, the selected reference geometrical shape to an X-Y coordinate of the object of interest shown in the scaled, positioned and oriented digital x-ray image so as to determine a location of the object of interest in 3D space in relation to the virtual depiction of the predefined images on the real world partition members.

Certain features and advantages of the present invention have been generally described in this summary section. However, additional features, advantages and exemplary embodiments are presented herein or will be apparent to one of ordinary skill of the art in view of the drawings, specification and claims hereof. Accordingly, it should be understood that the scope of the invention shall not be limited by the particular exemplary embodiments disclosed in this summary section.

It is to be understood that throughout this description, terms such as “vertical”, “horizontal”, “top”, “bottom”, “upper”, “lower”, “middle”, “above”, “below” and the like are used for convenience in identifying relative locations of various components and surfaces relative to one another in reference to the drawings and that the system of the present invention may be installed and used in substantially any orientation so that these terms are not intended to be limiting in any way.

Approximating language, as used herein throughout the specification and claims, may be applied to modify any quantitative representation that could permissibly vary without resulting in a change in the basic function to which it is related. Accordingly, a value modified by a term such as “about” or “approximately” is not limited to the precise value specified.

As used herein, the term “enclosed, opaque container” shall refer to enclosed, nontransparent containers, including luggage, knap sacks, bags, packages, shipping crates, cardboard boxes, wooden boxes, barrels or any other nontransparent structure or article configured to store, hold or carry items.

As used herein, the term “real world” refers to the real world physical environment and all matter existing therein, as opposed to a “virtual reality world” or an “augmented reality world”.

As used herein, “processing element” or “processor” include, for example, circuits such as an ASIC (Application Specific Integrated Circuit), portions or circuits of individual processor cores, entire processor cores, individual processors, central processing units (CPU), signal processors having analog-to-digital conversion circuitry, programmable hardware devices such as field programmable gate array (FPGA) and or larger portions of systems that include multiple processors.

As used herein, “computer”, “computer system” or “computing device” includes any of various types of computing or processing systems, including a personal computer system (PC), mainframe computer system, workstation, network appliance, Internet appliance, personal digital assistant (PDA), personal communication device, smart phone, notebook computer, desk top computer, tablet, television system, grid computing system, or other device or combinations of devices, or any device (or combination of devices) having at least one processing element that executes instructions from a memory medium. The aforesaid computing or processing systems are configured to open and process image files having formats, such as JPEG, PDF, PNG, etc.

As used herein, the terms “memory medium” or “memory” include non-transitory computer readable storage mediums and any of various types of memory devices or storage devices, such an installation medium, e.g., a CD-ROM, floppy disks, or tape device. “Memory medium” also includes a computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, SDRAM, EDO RAM, Rambus RAM, and non-volatile memory such as a Flash, magnetic media, hard drive, optical storage, registers or other similar types of memory elements. “Memory medium” may include other types of memory as well or combinations thereof. The term “memory medium” may include two or more memory mediums which may reside in different locations, e.g. in different processing elements or computers that are connected over a network. The memory medium may store program instructions (e.g., embodied as computer programs) that may be executed by one or more processing elements.

In accordance with an exemplary embodiment of the invention, the components, process or method steps, and/or data structures may be implemented using various types of operating systems (OS), computing platforms, firmware, computer programs, application computer programs (e.g. “app” or “App”), computer languages, and/or general-purpose machines. Computer programs include any expression, in any language, code or notation, of a set of instructions intended to cause a system having an information processing capability to perform a particular function. The method may be run as a programmed process running on one or more processors or processing elements. Accordingly, the methods disclosed herein may be embedded on a non-transitory computer-readable storage medium, which comprises all the features enabling the implementation of the methods described herein, and which when loaded in a computer system, is able to carry out these methods.

Actual Method: Referring toFIG. 1, the present invention is directed to a method and system to accurately determine the three-dimensional location of a target or object of interest10that is located within the interior region of an enclosed opaque container12. Other objects14and15(shown in phantom) are also located within the interior13of enclosed opaque container12. In this example, as shown inFIG. 1, object15was positioned on top of object14and object10was positioned in front of object14. Objects14and15also may be objects of interest, or they can be items to be avoided, or they can be neutral objects that are neither hazardous nor pose a threat. Specifically, the method and system of the present invention allow a user or operator to construct a three-dimensional representation of the interior region of an enclosed opaque container12so as to allow viewing of objects, components and substances within the interior region. The users or operators now have the opportunity to isolate a particular object of interest within the interior region that may be a threat, such as an explosive device or other energetic component. As will be described in the ensuing description, disrupter device200was aimed at the three-dimensional location and thereafter, disrupter device200fires a projectile or substance at the object of interest in order to disable or destroy target the object of interest. Examples of possible targets or objects that may be in enclosed opaque container12include containers, packages or bottles that may contain improvised-explosive-devices (IED), plastic explosives, explosive fluids therein or any other type of energetic devices.

Referring toFIGS. 1 and 2, the system comprises collapsible frame or template20that is placed near enclosed opaque container12. Frame20comprises first partition member22. First partition member22includes perimetrical support member23. Frame20further includes second partition member24. Second partition member24includes perimetrical support member25. In an exemplary embodiment, partition member22and partition member24are pivotally or hingedly attached to each other via hinge members26A,26B and26C such that frame20can open and close like a book. First partition member22and second partition member24may be fabricated from any type of suitable material, e.g. plastic, Plexiglas, rubber, etc. First partition member22includes first side28and opposite second side30. First partition member22defines a first plane. First partition member22includes a predefined unique pattern or image32formed on the first side28. In an exemplary embodiment, the predefined unique image32comprises a camouflage design. However, it is to be understood that predefined unique image32may be configured to have any other suitable design. Since first partition member22defines a plane, the camouflage image32of first wall22is also referred to herein as a “camouflage plane” or “camouflage image”. Image32is visible only on side28. Image32is formed or fabricated with low-density material. The formation of image32is described in detail in the ensuing description. First partition member22includes handle section34. Similarly, second partition member24includes first side36and opposite second side38. Second partition member24defines a second plane. Second partition member24includes a predefined unique image40on first side36. Predefined unique image40is different than image32on first side28of first partition member22. As a result of such a configuration, image40was distinct from image32. In an exemplary embodiment, image40comprises a camouflage design. Since second partition member24defines a plane, camouflage image40is also referred to herein as a “camouflage plane” or “camouflage image”. Image40is visible only on first side36. Image40was formed or fabricated with the same low-density material used to form or fabricate image32. The formation of image40is described in detail in the ensuing description. The purpose of images32and40is explained in the ensuing description. Second partition member24further includes handle42. Handles34and42allow a user to open frame20and to carry or transport frame20when not in use. When frame20is closed, handles34and42contact each other.

Images32and40are random patterns. In one embodiment, these random patterns were created by suitable image creation software or raster graphics editor software that is configured to graphically design, create and edit images. The user sets or predefines the height and width of the images using the image creation software. Suitable image creation software includes, but is not limited to, Adobe® Photoshop, Adobe® Illustrator, Fatpaint® DrawPlus™, InkScape™ and Ultimate Paint™. Once images32and40are created, the images were engraved onto corresponding low-reflectivity and low-density surfaces using a laser-engraver. In an exemplary embodiment, the low-reflectivity and low-density surfaces are rugged and waterproof. In one embodiment, the low-reflectivity and low-density surfaces are plastic sheets, wherein each image is engraved onto a corresponding plastic sheet. In an exemplary embodiment, images32and40have substantially the same thicknesses. The camouflage patterns of images32and40are just examples and it is to be understood that patterns other than camouflage may be used as well. In some embodiments, first partition member22includes a frame to which a first plastic sheet, having image32thereon, is mounted or attached. In some exemplary embodiments, second partition member24includes a frame to which a second plastic sheet, having image40thereon, is mounted or attached.

The completed images32and40and their dimensions were processed with one or more commercially available algorithms in order to generate a data set for each image32and40. In one exemplary embodiment, a desktop computer was used to process images32and40and the corresponding dimensions to generate the data set for each image32and40. In another exemplary embodiment, images32and40and their corresponding dimensions were fed or inputted into a commercial-off-the-shelf (COTS) online algorithm that generates the data set for each image32and40.

Referring toFIGS. 1 and 2, first partition member22further includes a first plurality of fiducial markers50. Fiducial markers50were arranged in a predetermined pattern. In an exemplary embodiment, fiducial markers50were arranged in an “X” pattern. In an exemplary embodiment, each fiducial marker50is a tungsten, disc shaped fiducial marker. However, the quantity, pattern and shape of the fiducials markers may be varied in other embodiments. For example, in some embodiments, fiducial markers50were arranged in columns and rows. Similarly, second partition member24further includes a second plurality of fiducial markers60. Fiducial markers60were arranged in a predetermined pattern. In an exemplary embodiment, fiducial markers60were arranged in an “X” pattern. In an exemplary embodiment, each fiducial marker60comprises a tungsten disc. However, the quantity, pattern and shape of fiducials markers60may be varied in other embodiments. For example, in some embodiments, fiducial markers60were arranged in columns and rows.

Referring toFIGS. 1 and 2, when frame20was positioned adjacent to the enclosed opaque container12, frame20was opened so that first partition member22was perpendicular to second partition member24and first partition member22faces one side of enclosed opaque container12and second partition member24faces another side of enclosed, opaque container12. In some embodiments, first partition member22and the second partition member24are separate pieces and are not joined together. In such embodiments, first partition member22and the second partition member24are placed at different locations. In other embodiments, only first partition member22or second partition member24is used, but not both.

The system of the present invention further comprises a camera that is in electronic data signal communication with a processing element. In an exemplary embodiment, the processing element comprises one or more processors. The processing element was programmed with the data sets that were generated by the aforementioned algorithms for each predefined unique image32and40. Accordingly, the processing element was programmed with real life measurements of every single line, dot and detail that appears in camouflage images32and40. In some embodiments, the camera and processing elements are separate components, such as a digital camera that is in electronic signal communication with a tablet computer or a desktop, laptop or notebook computer. In an exemplary embodiment, the camera and processing element are combined into a single computing device70, which includes a built-in camera. In an exemplary embodiment, computing device70comprises a smart phone. For purposes of describing the system and method of the present invention, computing device70is described herein as a smart phone70, which includes the internal processing element (not shown), camera72and display element or screen74(seeFIG. 4). The processing element is in electronic data signal communication with camera72and display screen74. Smart phone70includes home button75, which is well known in the field of smart phones. As shown inFIG. 1, smart phone70is mounted on support structure80, which has a plurality of leg members82. In an exemplary embodiment, support structure comprises a tripod. The processing element of smart phone70is also programmed with one or more computer application programs or apps including an Advanced Computer Vision (ACV) image processing algorithm. The ACV image processing algorithm processes the image data of a live video feed provided by the camera72. The ACV algorithm utilizes the data sets of images32and40to determine if an image captured in the live video feed is the same as image32or image40. As a result, smart phone40is capable of recognizing the pre-defined patterns of camouflage images32and40.

The Advanced Computer Vision software processes the live feed from camera72and creates a virtual reality environment (or “virtual environment”) with virtual depictions of the real world environment images32and40. The virtual reality environment was based on a virtual environment coordinate system or world coordinate system. The Advanced Computer Vision software executed by the processing element of smart phone70allows the user to define where everything will appear in the virtual reality environment. Therefore, the user may define the size of the virtual depictions of camouflage images32and40. The user may also define or specify the location of the virtual depictions of camouflage images32and40with respect to the virtual environment coordinate system. In the virtual reality environment, the virtual depictions of camouflage images32and40were positioned in the same way as images32and40are positioned in the real world environment. Specifically, the virtual camouflage images32and40were joined to each other to form the same corner and are angulated about 90° apart just as they are in the real world environment. The fiducial markers50and60are visible in the virtual depictions of camouflage images32and40, respectively, and are depicted as colored icons that are in the shape of buttons or discs. This feature is described in detail in the ensuing description. The virtual depictions of camouflage image32and40may be viewed on display screen74one at a time, during the step of selecting a fiducial marker by touching the corresponding colored icon shown in the virtual reality description. Both virtual depictions of images32and40may be seen simultaneously during the steps of “Calibrating”, “Finding Orthogonality” or “Aiming,” which are all described in detail in the ensuing description.

In other exemplary embodiments, computing device70includes a desktop computer having a display screen, keyboard and mouse and wherein the mouse is used to select the desired icons displayed on the computer display screen.

As described in the foregoing description, enclosed opaque container12has interior region13within which is located suspect object10and other objects14and15. Suspect object10may be a bottle, box, carton, etc. that may hold explosive material or other energetic device. In order to obtain a view of the contents of enclosed opaque container12, x-rays were taken of enclosed opaque container12from two different angles. As shown inFIG. 1, first x-ray machine100was positioned so that it faces first partition member22and first partition member22is between x-ray machine100and enclosed opaque container12. As a result of this configuration, x-ray radiation emitted by x-ray machine100was directed to first side28of first partition member22. First x-ray film102was positioned on the opposite side of enclosed article12such that enclosed article12is positioned between first partition member22and first x-ray film102. In an exemplary embodiment, first x-ray film102was aligned with and substantially parallel to first partition member22. Next, the first-x-ray was taken such that first x-ray radiation104passes through first partition member22, enclosed opaque container12and first x-ray film102. The plane of first x-ray radiation104is orthogonal to camouflage image32on first partition member22. Since camouflage image32was made from a low-density material, first x-ray radiation104passes through camouflage image32. As a result, camouflage image32does not appear on the first x-ray film102. However, fiducial markers50of first partition member22and interior region13of enclosed opaque container12, as seen from that angle, do appear on the first x-ray film102. The image of first x-ray film102was scanned via digital scanner106so as to digitize the image on x-ray film102thereby producing a first digital x-ray image. In an exemplary embodiment, digital scanner106is in electronic data signal communication with computing device70via USB cable107. In other embodiments, digital scanner106is in electronic data signal communication with computing device70via wireless connections or networks. The first digital x-ray image is then loaded into smart phone70where it is then imported into the smart phone App and then stored in the memory (e.g. ROM, RAM, etc.). In an exemplary embodiment, the first digital x-ray image is in the form of a JPEG image when it is imported into the smart phone app.

As shown inFIG. 1, second x-ray machine120was positioned so that it faces the first side36of second partition member24of frame20and second partition member24is between x-ray machine120and enclosed opaque container12. As a result of this configuration, the x-ray radiation122emitted by x-ray machine120was directed to first side36of second partition member24. Second x-ray film124was positioned on the opposite side of enclosed opaque container12such that enclosed opaque container12is between second partition member24and second x-ray film124. In an exemplary embodiment, second x-ray film124was aligned with and substantially parallel to second partition member24. Next, the second x-ray was taken such that second x-ray radiation122passes through second partition member24, enclosed opaque container12and second x-ray film124. The plane of second x-ray radiation122was orthogonal to camouflage image40on second partition member24. Since camouflage image40was made from a low-density material, second x-ray radiation122passes through camouflage image40. As a result, camouflage image40does not appear on second x-ray film124. However, fiducial markers60of second partition member24and interior13of enclosed opaque container12, as seen from that angle, do appear on second x-ray film124. Second x-ray film124is digitally scanned with digital scanner106and loaded into smart phone70wherein it is imported into the smart phone App and stored in the memory (e.g. ROM, RAM, etc.) in smart phone70. In an exemplary embodiment, the image of second x-ray film124is in the form of a JPEG image when it is imported into the smart phone app.

In another exemplary embodiment, a single x-ray machine is used. In such an embodiment, the x-ray machine is first set up so that the first x-ray radiation passes through first partition member22and enclosed opaque container12as described in the ensuing description. After the first x-ray is taken, the x-ray machine is then re-positioned and set up so that the second x-ray radiation passes through the second partition member24and enclosed opaque container12.

In an exemplary embodiment, first partition member22was substantially perpendicular to second partition member24. In such a case, the method of the present invention implements an “Orthogonality Mode” in order to locate the position of x-ray machine100and x-ray machine120with respect to first partition member22and second partition member24, respectively. Orthogonality is found or determined with a combination of virtual reality and augmented reality. The processing element of smart phone70executes software that determines orthogonality. Referring toFIG. 3, in order to establish orthogonality, the Advanced Computer Vision (CV) software executed by the processing element of smart phone70displays, by display screen74, intersecting horizontal plane130and vertical plane132that are always perpendicular to each other and are adjacent to first partition member22or second partition member24. For example, as shown inFIG. 3, horizontal plane130and vertical plane132are adjacent to first side36of second partition member24and intersect at the point where the axis of x-ray radiation122meets first side36. This provides the user with a qualitative and intuitive indicator for orthogonality between the x-ray radiation and the partition member of interest (i.e. first partition member22or second partition member24) as these intersecting planes130and132may be observed from the viewpoint of the x-ray machine100or x-ray machine120. In order to accurately determine orthogonality, the “Calibration Mode” must be done first. The Calibration Mode was described in detail in the ensuing description. The software executed by the processing element of smart phone70also causes display element74to display “standoff distance” and a value referred to as “orthogonality factor” which is related to the “dot product” between a partition member's normal vector and the vector aligned with the axis of the x-ray radiation. This aids in quantifying orthogonality. Full orthogonality is achieved when the value equals “1”.

In some embodiments, the Calibration Mode is implemented prior to finding “orthogonality”. The “Calibration Mode” was described in detail in the ensuing description. Once the Calibration Mode has been implemented, it does not have to be implemented again during processing of the first x-ray image and the second x-ray image. For purposes of brevity and to facilitate understanding of the Orthogonality Mode, the ensuing description is in terms of the Calibration Mode having already been completed.

The beams of x-ray radiation104and122must be as orthogonal as possible to the surface of camouflage images32and40, respectively, in order to achieve a high level of fidelity. During the Orthogonality Mode, live feed videos of first partition member22and second partition member24were routed to the smart phone App. The Advanced Computer Vision (CV) software executed by the smart phone App includes an algorithm that “sees” the intersection of the planes of first partition member22and second partition member24assuming the smart phone camera72is not orthogonal to either first partition member22or second partition member24. Since camouflage images32and40are pre-defined as described in the foregoing description, the ACV software recognizes both camouflage images32and40simultaneously.

In some situations, finding orthogonality may not be necessary. The need to find orthogonality depends upon the user's assessment of the type of enclosed opaque container12or the object of interest10within enclosed opaque container12.

Each x-ray image was selected one at a time for processing. Processing each x-ray image includes scaling, positioning and orienting the x-ray image. The order in which the x-ray images are processed does not matter. For example, the first x-ray image derived from x-ray film102may be processed first and then the second x-ray image derived from x-ray film124, or vice versa. Before any processing of x-ray images occurs, an x-ray image must be selected. Smart phone70implements an “X-Ray Selection Mode” which utilizes a virtual reality environment. Display screen74of smart phone70displays a menu that lists several icons, one of which being “X-Ray Selection Mode”. When the user presses the icon labelled “X-Ray Selection Mode”, two additional icons appear on display screen74. One of the additional icons corresponds to first partition member22and the other icon corresponds to second partition member24. The user then touches one of these icons and, in response, display screen74displays the x-ray image corresponding to the selected partition member of frame20along with an icon labelled as “Continue”. Once the user touches the “Continue” icon, display screen74displays a virtual depiction of the camouflage image corresponding to the selected real world partition member of frame20. As described in the foregoing description, the data sets defining the camouflage images were previously loaded into the smart phone App. As a result, the smart phone App is able to generate the virtual depiction of the camouflage images. In the virtual depiction of the camouflage image, the fiducial markers (e.g. tungsten discs) are depicted as different colored icons. The location of each colored icon in the virtual depiction corresponds to the actual location of a corresponding fiducial marker (e.g. tungsten disc) on the real world camouflage image of the selected real world partition member of frame20. This depiction is illustrated inFIG. 4. As described in the foregoing description, first partition member22has camouflage image32and fiducial markers50. Smart phone70displays, via display74, a virtual depiction140of the camouflage image32and also displays a plurality of different colored circular icons150that represent fiducial markers50in first partition member22. The location of each icon150corresponds to an actual location of a corresponding fiducial marker50in the first real world partition member22. For example, icon150A corresponds to real world fiducial marker50A (seeFIG. 2) in first real world partition member22and icon150B corresponds to real world fiducial marker50B in first real world partition member22. Although the foregoing description is in terms of colored icons150being circular in shape, it is to be understood that icons150may be other shapes, e.g. square, oval, triangular, rectangular, etc.

Referring toFIG. 5A, there is shown x-ray image160resulting from x-rays passing through first partition member22and enclosed opaque container12. Fiducial markers50on first real world partition member22were shown as fiducial marker images170in x-ray image160. Therefore, each fiducial image170corresponds to a fiducial marker50on real world partition member22. X-ray image160shows objects10,14and15from a first angle. Objects10,14and15are located within interior13of enclosed opaque container12. Object15is sitting on top of object14as described in the foregoing description and shown inFIG. 1.FIG. 5Bshows x-ray image162resulting from x-rays passing through second partition member24and enclosed opaque container12. Fiducial markers60on second real world partition member24were shown as fiducial marker images164in x-ray image162. X-ray image162shows objects10,14and15from a second angle.

The next step is to scale and position each x-ray image160and162. For purposes of describing this aspect of the invention, x-ray image160was scaled and positioned first. Therefore, the first step was to map two of the colored icons150shown in virtual depiction140(seeFIG. 4) with two of corresponding fiducial marker images170in corresponding x-ray image160(seeFIG. 5A). In order to accomplish the mapping step, the user first presses, clicks or touches one of the colored icons150in virtual depiction140. Once one of the colored icons is selected, display screen74displays x-ray image160, a crosshair180having the same color as the selected colored icon, and icon190labelled “Point1” as shown inFIG. 6. Crosshair180was positioned in the center of display screen74and remains stationary in the center of display screen74. Crosshair180is not part of x-ray image160. For example, referring toFIG. 4, icon150A is colored blue and corresponds to fiducial marker images170A shown inFIGS. 5A and 6. If the user touches blue colored icon150A inFIG. 4, then display element74displays x-ray image160as shown inFIG. 6wherein crosshair180is colored blue. Then, the user drags x-ray image160so that fiducial marker image170A was aligned with the center of crosshair180as shown inFIG. 7. Since the x-ray image160was dragged in a downward, diagonal direction (i.e. downward and to the left), portions of objects10,14and15are off the display screen74. The user can enlarge or decrease the size of the view of x-ray image160in order to facilitate alignment of fiducial marker image170A with the center of crosshair180. When the user believes the alignment is as accurate as possible, the user touches the “Point1” icon190. Touching the “Point1” icon190correlates or maps colored icon150A to fiducial marker image170A on x-ray image160. After the user touches the “Point1” icon190, display screen74once again displays the virtual depiction140of the camouflage image32with the colored circular icons150(seeFIG. 4). The user must select another one of the colored icons150shown inFIG. 4. However, the user cannot select a colored icon150that has the same color as the previously selected colored icon150A. The user must select a colored icon150that has a different color. In this example, the user selects red colored icon150B shown inFIG. 4which corresponds to fiducial marker image170B on x-ray image160shown inFIG. 5B. Once the user touches the second colored icon150B inFIG. 4, display screen74shows x-ray image160, crosshair194, which is also colored red, and “POINT2” icon196. The user then drags x-ray image160so as to align corresponding fiducial marker170B with the center of crosshair194as shown inFIG. 8. When the user believes the alignment is as accurate as possible, the user touches icon196labelled as “Point2”. Touching the “Point2” icon196correlates or maps colored icon150B inFIG. 4to fiducial marker image170B on x-ray image160. X-ray image160is then automatically scaled, positioned and oriented thereby completing the correlation or mapping process for one of the partition members of frame20. In an exemplary embodiment, the scaling, positioning orientation of x-ray image160takes place in the background while display screen74once again displays the virtual depiction140of the camouflage image of the selected partition member of frame20. However, when virtual depiction140reappears on display screen74, colored circular icons150were not shown because the mapping process for this selected partition member of frame20is now complete. At this time, display screen74also displays the menu icon and two icons that allow the user to select either first partition member22or second partition member24. Next, the user will press the icon corresponding to the partition member of frame20that was not previously selected so that this partition member can undergo the mapping, scaling and positioning processes as described in the foregoing description. Since x-ray image160has now undergone the mapping, scaling and positioning processes, the user will now select second partition member24so that x-ray image162will undergo the mapping, scaling and positioning steps. However, the user may repeat the process for partition member22if the user believes an error was made during the scaling, positioning and orienting processes performed on x-ray image160.

In some embodiments, the user measures the distance between first partition member22and first x-ray film102and the distance between second partition member24and second x-ray film124and then enter those measured distances into the App of smart phone70. The measurements can be taken manually, via a measuring tape, yard stick, etc.

The next steps of the method of the present invention were referred to as the “Build Mode”. In the “Build Mode”, x-ray images160and162are reviewed to find any object in enclosed opaque container12that appears suspicious or which warrants further examination. If such a suspicious object is located within enclosed opaque container12, then the next step is to trace the suspicious object and determine where it is located in 3-D space. For example, if object10(seeFIG. 1) is the suspicious object, then it is necessary to trace object10and determine where it is located in 3-D space. In order to accomplish these steps, the Build Mode utilizes a virtual reality setting. Display screen74displays the Menu icon, a second icon labelled “Shapes”, a third icon labelled “Functionality” and a fourth icon labelled “Colors”. If the “Shape” icon is pressed, then a plurality of icons appear on the screen, wherein each icon indicates a particular geometrical shape, e.g. cuboids, cylinders, spheres, sphero-cylinders, etc. A library of such geometrical shapes or objects is stored in the memory in the smart phone70. In order to insert a geometrical shape into the x-ray image on display screen74, the user touches the icon with the desired shape and then drags the shape to a desired location on the x-ray image. Similarly, if the “Functionality” icon is pressed, then a plurality of function icons appears on display screen74, wherein each icon indicates a particular function. For example, each function icon indicates a particular function such as changing the position, orientation and scale of any of the shapes dragged into the displayed x-ray image. An additional function icon allows the user to delete any of the shapes that have been dragged onto the displayed view of the x-ray image. Since the user can only view one x-ray image at a time, the function icons include a “Change Image” button that allows the user to view the other x-ray image corresponding to the other partition member of frame20. If the “Colors” icon is touched, then a plurality of icons appears on the display screen, wherein each icon indicates a particular color. The color icons allow the user to colorize any of the shapes that are dragged over the x-ray image. There is no limit on the amount of shapes a user may introduce into the x-ray image displayed by display screen74. In order for the user to interact with any one shape that has been introduced onto the x-ray image displayed by display screen74, the user must first select that shape using the appropriate icon. After the user selects that shape, the user can then move, scale, rotate or change the color of the shape. The user can also delete the shape.

Next, the user must trace suspicious object10(hereinafter referred to as “target10”). The user presses the “Shapes” icon and then presses an icon that corresponds to a desired geometrical shape. As a result, the desired geometrical shape appears on display screen74and the user then drags the selected geometrical shape over to target10. The user then attempts to match one of the shapes to an X-Y coordinate of target10. Using the functionality icons, the user scales, positions or rotates the shapes as needed. Once the user has finished these steps for the first x-ray image (e.g. x-ray image160), the user then uses the appropriate functionality icon to change the view on display screen74to that of the second X-ray image (e.g. x-ray image162). The user locates target10in the second x-ray image and then attempts to match the geometrical shapes to an X-Y coordinate of target10. Once these steps were completed for the second x-ray image, target10can now be located in 3D space in relation to the camouflage images in the virtual environment. As described in the foregoing description, each camouflage image is predefined and its location in the virtual environment was based on a predefined coordinate system. Specifically, the completion of these steps provides 3D vision of frame20, interior13of enclosed opaque container12and the location of target10with respect to the camouflage images32and40on real world partition members22and24, respectively. These steps were repeated for other targets of interest that are inside enclosed opaque container12.

Display screen74displays the 3D image of the interior of enclosed opaque container12. The 3D image shows the objects within the interior of enclosed opaque container12, including target10. The user then uses his or her finger to touch the image of target10shown on display screen74. Automatically, target10was colored with a predetermined color (e.g. green) and the non-selected objects within the interior of enclosed article12are colored with different colors. In some embodiments, if any of the non-selected objects are considered a hazard, such objects are colored red, and any object that is neither a target nor a hazard, is colored with a third color, e.g. yellow. The computer program App automatically generates a sphere at the centroid of target10. This virtual depiction is illustrated inFIG. 9. Display screen74displays a virtual reality view through the “virtual camera” showing sphere210at the centroid of target10. In this example, target10is colored green, sphere210is colored with a different color (e.g. blue), object15is colored with red to signify that it is a hazard and object14is colored yellow to signify that it is neither a hazard nor a target. The purpose of sphere210is to provide a visual aid when aiming disrupter device200at target10during the Aim Mode, which is described in the ensuing description.

Disrupter device200was configured to propel or fire a substance or projectile at target10in order to destroy or disable target10while avoiding other objects in enclosed opaque container12such as objects14and15. Disrupter device200may fire or propel such projectiles or substances. Such projectiles and substances include projectiles made from metal or other materials, cartridges that explode upon impact and pressurized fluids such as water. As shown, inFIG. 1, disrupter device200is aimed at target10based on the information and data obtained by the steps described herein. Disrupter device200includes laser boresighter204(shown in phantom) that generates laser beam206. In an exemplary embodiment, laser boresighter204was inserted or positioned within a bore of disrupter device200. In an exemplary embodiment, smart phone70also was mounted to support structure80and is located adjacent to disrupter device200. In actuality, disrupter device200is positioned a safe distance away from enclosed opaque container12. In another exemplary embodiment, the process of determining orthogonality, as described in the foregoing description, may be implemented into order to find orthogonality between the flight path of a projectile that is to be fired by disrupter device200and target10. In such an embodiment, the determination of orthogonality prevents the projectile from ricocheting inside enclosed opaque container12.

The user now refers to the Menu wherein all of the Modes are listed. The two remaining modes are Calibration Mode and Aim Mode. In some embodiments, the Calibration Mode was implemented prior to finding orthogonality. As described in the foregoing description, the process of finding orthogonality is done prior to taking x-rays. In some embodiments, the Calibration Mode was implemented prior to the Aim Mode. For example, if Calibration Mode was previously implemented to calibrate the software (e.g. smart phone App) based on the location of x-ray machine100but x-ray machine100was subsequently replaced with disrupter device200, then Calibration Mode may have to be implemented again because the location from where the x-ray beam was emitted may be different than the location from where the projectile is fired.

As shown inFIG. 10, each partition member22and24of frame20has a center point represented by a small circle and a cross-hair symbol superimposed over the small circle. The intersection of the two lines that form the cross-hair defines the center point of the partition member. The center point of the partition member is also the center of the corresponding camouflage image on the partition member. For example, first partition member22has center point220defined by a circle crosshair symbol. Center point220functions as a known location on first partition member22to which the user can direct laser beam206. Similarly, second partition member24has center point230, which is defined by another circle and crosshair symbol. Center point230has the same function as center point220of first partition member22. The software App executed by smart phone70knows where center points220and230are located because camouflage images32and40are predefined and recognizable by the ACV software executed by the processing element of smart phone70. Thus, center points220and230are represented by corresponding symbols on the virtual depictions of the camouflage images. For example, crosshair symbol220was represented as center point240in the virtual depiction shown inFIG. 4.

When Calibration Mode is selected, a “Live Feed” seen by camera72of smart phone70is displayed on display screen74. The Calibration Mode determines two points in space, Point1and Point2, that are defined by (X, Y, Z) coordinates and which define a virtual line. This virtual line defines an aiming vector used in the Aim Mode, which is described in the ensuing description. In order to define the two points in space, two (X, Y, Z) coordinate frames are used. One (X, Y, Z) coordinate frame is moving (X, Y, Z) coordinate frame270, the origin of which being the center of the lens of camera72. The other (X, Y, Z) coordinate frame is static (X, Y, Z) coordinate frame272, the origin of which is at bottom corner the selected partition member. For example, inFIG. 10A, static (X, Y, Z) coordinate frame272is at bottom corner of274of partition member22. The two (X, Y, Z) coordinates that define the virtual line belong directly to moving (X, Y, Z) coordinate frame270because these two (X, Y, Z) coordinates are fixed with respect to camera72and will go wherever camera72goes. Stated another way, after the Calibration Mode, the virtual line or aiming vector, which is represented by the two (X, Y, Z) coordinates in the moving (X, Y, Z) coordinate frame270, moves with camera72wherever camera72goes. The virtual line or aiming vector was not fixed to center point220on first real world partition member22.

Referring toFIG. 10A, the first step in the Calibration Mode is to select either camouflage image32or camouflage40. The user must use the same camouflage image in the determination of both (X, Y, Z) coordinates. For purposes of describing this aspect of the invention, the ensuing description is in terms of the user selecting camouflage image32on first partition member22. Display screen74displays an icon labelled “Point1”. Point1is the first point on the virtual line that will be determined in the Calibration Mode. Support structure80, with smart phone70and disrupter device200attached thereto, is first positioned at a first location with respect to first partition member22. Point1is located at the center point220of first partition member22. The user aligns disrupter device200so that laser beam206is aligned with center point220on first partition member22. When the alignment is complete, the user touches the “Point1” icon. As a result, Point1is recorded relative to moving (X, Y, Z) coordinate frame270. As stated in the foregoing description, moving (X, Y, Z) coordinate frame270moves with camera72. The 3D camera is now aware of the location of Point1relative to camera72. After Point1is recorded, display screen74displays an icon labelled “Point2”. Point2is the second point on the virtual line that will be determined in the Calibration Mode. Point2is the second point in space, which is farther away from the camera's moving (X, Y, Z) coordinate frame270than Point1. Although frame20cannot be physically moved at this time, Point2can still be determined by moving camera72and disrupter device200backward from first partition member22. This is illustrated by the diagram inFIG. 10B. Support structure80, with smart phone70and disrupter device200attached thereto, is moved back a distance from first partition member22and positioned at a second location with respect to first partition member22. The distance disrupter device200is moved back can be just a few inches or several feet. In this example, disrupter device200is move backward approximately three feet from the corner of first partition member22. The previous positions of smart phone70and disrupter device200are represented in phantom and indicated by reference numbers70A and200A, respectively, shown inFIG. 10B. The previous path of laser beam206, shown inFIG. 10A, is now represented by the dashed line206A inFIG. 10B. In this step, Point2is located at center point220of first partition member22and Point1is now located at the point in space that corresponds to the previous location of the forward end of the barrel of disrupter device200(seeFIG. 10A). Disrupter device200is adjusted again so that laser beam206is aligned with center point220on first partition member22. When the alignment is complete, the user touches the “Point2” icon. As a result, Point2is recorded relative to moving (X, Y, Z) coordinate frame270. As stated in the foregoing description, moving (X, Y, Z) coordinate frame270moves with camera72. The 3D camera is now aware of the location of Point2relative to camera72. After these aforementioned steps are completed, the 3D camera is now aware of the location of both Point1and Point2relative to moving (X, Y, Z) coordinate frame270. The software App executed by the processing element of smart phone70now generates a virtual line between Point1and Point2and records the position and orientation of this virtual line. This virtual line is equal to the longitudinally extending axis of the bore or barrel of disrupter200in position and orientation and is used as the aiming vector in the Aim Mode, which is described in the ensuing description. The Calibration Mode is now complete.

After the Calibration Mode was completed, the Aim Mode automatically begins. Referring toFIGS. 11 and 12, there is shown smart phone70and disrupter device200mounted to support structure80. Laser boresighter204is within the bore of disrupter device200(seeFIG. 1) and emits laser beam206(seeFIGS. 1, 10A and 10B). The view displayed on display screen74is that of a virtual reality (VR) view300(seeFIG. 12) that is based on the virtual items that were created in the Build Mode and based on the real world items shown in x-ray image160and x-ray image162. Virtual reality view300shows target10, hazardous object15that is to be avoided and neutral object14that is neither a target nor a hazardous item. Therefore, display screen74displays a view of a “virtual camera”. The virtual reality (VR) view300, which is to be displayed on display screen74, is shown inFIG. 12. The view inFIG. 12shows objects10,14and15and sphere210which are all shown in the view shown inFIG. 9. InFIG. 12, display screen74now displays crosshairs310that are located in the center of display screen74and are stationary. Crosshairs310provide a reference to the center of display screen74. Specifically, the crosshairs310coincides with the virtual aiming vector that was calibrated in the Calibration Mode. Aligning the crosshairs310with target10actually aligns the center of display screen74with target10and therefore the aiming vector. Therefore, the “virtual camera” was aligned with the aiming vector calibrated in the Calibration Mode so that when target10is within crosshairs310, it actually means that disrupter device200is aimed at target10. Hence, the view through the virtual camera is actually the view seen looking from the “bore” or “barrel” of disrupter device200. As a result, the user need only move camera72and point it at target10, similar to taking a photograph, so that target10was aligned with crosshairs310.

Referring toFIG. 13, in some embodiments of the Aim Mode, display screen74also displays a live-feed augmented reality (AR) view320in the corner of display screen74while simultaneously displaying view300. Augmented reality view320provides the user with a view of what the aiming vector looks like from the camera standpoint. Specifically, augmented reality view320shows target10, hazardous object15and neutral object14, all of which being positioned within enclosed opaque container12. In other embodiments of the Aim Mode, the augmented reality view320was used for calibration monitoring. Such calibration monitoring includes aligning the elements in the augmented reality (AR) view to reference features on frame20in order to ensure the alignment of the augmented reality view is correct. This calibration monitoring procedure includes inspecting the augmented reality view right before disrupter device200fires the projectile.

Next, frame20was removed so that there are no items or objects between disrupter device200and enclosed opaque container12. Laser boresighter204was removed from the bore or barrel of disrupter device200and is replaced by the projectile that is to be fired at target10. The user then fires disrupter device200such that the projectile travels along the aiming vector and impacts target10at the point defined by crosshairs310thereby destroying or disabling target10.

Finally, any numerical parameters set forth in the specification and attached claims are approximations (for example, by using the term “about”) that may vary depending upon the desired properties sought to be obtained by the present invention. At the very least, and not as an attempt to limit the application of the doctrine of equivalents to the scope of the claims, each numerical parameter should at least be construed in light of the number of significant digits and by applying ordinary rounding.