Patent ID: 12230025

DETAILED DESCRIPTION

Before any embodiments of the invention are explained in detail, it is to be understood that the invention is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the following drawings. The invention is capable of other embodiments and of being practiced or of being carried out in various ways.

Packaging System

FIG.1illustrates one embodiment of a pharmacy packaging system100. In the example illustrated, the pharmacy packaging system100includes a storage unit110and two automatic packaging units120. The storage unit110stores a plurality of canisters130containing a variety of pharmaceuticals. The canisters130may be containers or cassettes containing a variety of pharmaceuticals. The packaging units120package pharmaceuticals from those canisters130into pouches for distribution to patients. In some embodiments, the pharmacy packaging system100may include fewer or more packaging units120. Additionally or alternatively, the packaging units120may be positioned on both sides of the storage unit110. For example, the pharmacy packaging system100may include four packaging units120, with two packaging units120positioned on each side of the storage unit110. Such an arrangement allows multiple, independent packaging units120to access the same pharmaceutical array.FIG.1illustrates only one exemplary embodiment of the pharmacy packaging system100. In other embodiments, the pharmacy packaging system100may include more of fewer components and may perform functions that are not explicitly described herein.

FIG.2illustrates one embodiment of the storage unit110. In the example illustrated, the storage unit110includes a frame210and a gantry assembly220. The frame210includes a plurality of shelves or other supports for storing the canisters130in an array of rows and columns. The gantry assembly220is coupled to the frame210for retrieving canisters130from within the frame210. The gantry assembly220is a canister-moving assembly that is operable to move the canisters130within the frame210. The gantry assembly220includes a track230and a robotic head240that is operable to move along the track230to retrieve the canisters130. The track230is movable horizontally within the frame210to align the robotic head240with a specific column of canisters130. The robotic head240, or carriage assembly, is movable vertically along the track230to align with a specific row of canisters130. When the robotic head240is aligned with the desired canister130, the robotic head240grabs the canister130and carries the canister130to one of the automatic packaging units120, as further described below. The robotic head240can also retrieve a canister130from the packaging unit120and return the canister130to the proper column and row within the frame210.FIG.2illustrates only one exemplary embodiment of the storage unit110. In other embodiments, the storage unit110may include more of fewer components and may perform functions that are not explicitly described herein.

FIG.3illustrates one embodiment of the automatic packaging unit120. In the example illustrated, the packaging unit120includes a motor base310, a manifold320, a receptacle330, two feed stock rolls340,350, and a take-up roll360. Canisters130are placed on top of the motor base310. The motor base310is offset from the shelves of the frame210and includes a ledge for supporting the canisters130. When positioned adjacent the storage unit110, the illustrated motor base310is only offset from the shelves a relatively short distance to reduce the range of horizontal movement required by the gantry assembly220to place canisters130on or remove canisters130from the ledge. The motor base310includes one or more motors operable to operate the active canisters130to dispense the pharmaceuticals stored within the canisters130. The motor base310may also include one or more cameras or other sensors to verify whether pills were dispensed from the canisters130and/or to capture images of the pills as they are dispensed from the canisters130. The motor base310thereby provides dispensing areas for the active canisters130.

The manifold320, or chute, includes a plurality of discrete tracks370corresponding to each of the canisters130mounted on the motor base310. The illustrated tracks370are independent channels that together form the manifold320. The tracks370direct pharmaceuticals from the motor base310toward packaging equipment of the packaging unit120. The motor base310is positioned generally above the packaging equipment such that pharmaceuticals slide down the manifold320toward the packaging equipment. The tracks370isolate the pharmaceuticals from each other as the pharmaceuticals slide down the manifold320to the receptacle330.

The receptacle330collects the pharmaceuticals from the manifold320. After the pharmaceuticals pass through to the receptacle330, the pharmaceuticals are sandwiched between two strips of material (e.g., plastic, paper, etc.) from the feed stock rolls340,350. The first feed roll340may be made of a first material that is white in color (e.g., a translucent material). The second feed roll350may be made of a second material that is clear (e.g., a transparent material) to allow a pharmacist to look through the pouch to see the pharmaceuticals inside the pouch. The two strips of material are then heat sealed together to form a pouch for the pharmaceuticals. In some embodiments, the packaging unit120may include a single feed stock roll having a single strip of material that is folded and/or heat sealed to form pouches. Once filled and sealed, the pouches are wrapped around the take-up roll360to create a single spool of pouches. The spool may correspond to pharmaceuticals requested by a particular patient or a particular facility. In other embodiments, the pouches may be cut and separated as they are filled, rather than spooled onto the take-up roll360continuously. The pouches are dispensed through, for example, a dispenser or dispensing port connected to the pharmacy packaging system100.

In some embodiments, each packaging unit120may include a printer380to print a patient's name, the date, the amount and type of pharmaceuticals contained within, a bar code, and/or other indicia on the pouches as the pouches are formed. The printer380may be, for example, a thermal printer. In other embodiments, the printer380may include an ink ribbon or an ink jet. In addition, each packaging unit120may include a bar code scanner or vision system390to monitor and check the pouches as they are spooled onto the take-up roll360or dispensed.

FIG.4illustrates one embodiment of a control system400for the pharmacy packaging system100. The control system400is electrically coupled to the packaging equipment and the gantry assembly220to control operation of the pharmacy packaging system100. In particular, the control system400coordinates movement of the gantry assembly220to move the canisters130between the storage unit110and the packaging unit120, controls operation of the feed stock rolls340,350to release and form a pharmaceutical pouch, and controls when the active canisters130positioned on the motor base310are opened and closed.

In the example illustrated, the control system400includes a processor410, a memory420, a transceiver430, and an input/output interface440. The processor410, the memory420, the transceiver430, and the input/output interface440communicate over one or more control and/or data buses (e.g., a communication bus450).FIG.4illustrates only one exemplary embodiment of a control system400. The control system400may include more or fewer components and may perform functions other than those explicitly described herein.

In some embodiments, the processor410is implemented as a microprocessor with separate memory, such as the memory420. In other embodiments, the processor410may be implemented as a microcontroller (with memory420on the same chip). In other embodiments, the processor410may be implemented using multiple processors. In addition, the processor410may be implemented partially or entirely as, for example, a field-programmable gate array (FPGA), an application specific integrated circuit (ASIC), and the like, and the memory420may not be needed or be modified accordingly. In the example illustrated, the memory420includes non-transitory, computer-readable memory that stores instructions that are received and executed by the processor410to carry out functionality of the control system400described herein. The memory420may include, for example, a program storage area and a data storage area. The program storage area and the data storage area may include combinations of different types of memory, such as read-only memory and random-access memory.

The transceiver430enables wireless communication from the control system400to, for example, a remote electronic device such as a server or a smart telephone or a tablet computer of a remote pharmacist. In other embodiments, rather than the transceiver430, the control system400may include separate transmitting and receiving components, for example, a transmitter and a receiver. In yet other embodiments, the control system400may not include a transceiver430and may communicate with a remote device via a network interface and a wired connection to a communication network such as the Internet.

As noted above, the control system400may include the input/output interface440(or more commonly referred to as a user interface). The input/output interface440may include one or more input mechanisms (e.g., a touch screen, a keypad, a button, a knob, and the like), one or more output mechanisms (e.g., a display, a printer, a speaker, and the like), or a combination thereof. The input/output interface440receives input from the input devices actuated by a user, and provides output to the output devices with which a user interacts. In some embodiments, as an alternative or in addition to managing inputs and outputs through the input/output interface440, the control system400may receive user inputs, provide user outputs, or both by communicating with an external device, such as a console computer, over a wired or wireless connection.

A user can interact with the packaging unit120through the control system400to input patient information, facility information, and/or the pharmaceuticals needed. The control system400can control movement of the gantry assembly220to move the canisters130from the shelves of the storage unit110to one of the motor bases310. In addition, the control system400can control operation of the motor bases310to selectively open and close the canisters130. Furthermore, the control system400may optimize orders by minimizing movement of the gantry assembly220and the canisters130or by filling all the orders for a particular patient or facility consecutively.

FIG.5illustrates a pouch500containing different pharmaceuticals510therein. The illustrated pouch500is an example of a pouch that may be formed using the packaging equipment of the packaging unit120described above. The pouch500may be a plastic bag having three closed edges520and an open edge530. A heat seal540extends across the pouch500adjacent the open edge530to seal the pouch500. In some embodiments, all four edges520,530of the pouch500may be closed via heat seals. Additionally or alternatively, the pouch500may be composed of an opaque and/or non-plastic material. For example, one or both sides of the material may be opaque or colored (e.g., white or amber colored). As discussed above, identification indicia550(e.g., a patient's name, a barcode, types of pharmaceuticals, etc.) are printed on the pouch500using, for example, a thermal printer, an inkjet printer, a thermal transfer ribbon, or the like. In other embodiments, the identification indicia550may be printed on a label that is coupled to the pouch500with adhesives. In further embodiments, the pouch500may include a header area and/or a footer area without medication, but that provides space to print or apply the identification indicia550.

Verification System

FIG.6illustrates another view of the packaging unit120. In the example illustrated, the packaging unit120also includes a verification system610. The verification system610is positioned downstream of the receptacle330and the pouch sealing mechanism, between the feed stock rolls340,350and the take-up roll360(or dispenser). As further described below, the verification system610includes components to compare the actual contents of a filled pharmaceutical pouch to the expected contents of the pouch.

FIG.7is a schematic of part of the packing unit120including the verification system610. The verification system610provides a final check of the filled pouches before the pouches are distributed. In the example illustrated, the verification system610includes an agitator710, a sensor720, and a camera system730comprised of one or more cameras. In other embodiments, the verification system610may include fewer or more cameras. After the pouches are filled and sealed, the strip of pouches is directed by sheaves or pulleys750into the verification system610. At least some of the sheaves may be driven by, for example, motors to move the strip of pouches.

As the pouches enter the verification system610, the agitator710engages (e.g., agitates, vibrates, etc.) the pouches to help the pharmaceuticals within each pouch settle and separate or spread out. In some embodiments, the agitator710may include fingers, brushes, or the like that engage and agitate the pouches. Additionally, the fingers or brushes may be driven by, for example, a vibration motor to help agitate the pouches and move the pharmaceuticals around within each pouch. In other embodiments, the fingers or brushes may be passive elements that help comb the pharmaceuticals apart. In further embodiments, the agitator710may be omitted.

After the pouches are agitated, the pouches pass a sensor720. In some embodiments, the sensor720may be, for example, a barcode scanner, a QR code reader, or an image sensor. In other embodiments, the sensor720may be a label camera that captures images. The sensor720is positioned on one side of the pouches (e.g., a label side of the pouches) and reads a code or captures an image of the label on each pouch. A light source725(e.g., a visible light) is also positioned on the same side of the pouches as the sensor720to selectively illuminate the pouches while the sensor720reads codes or captures images. The sensor720can communicate with the processor410and memory420of the control system400to store the image or data related to the read/scanned code. The processor410can also control operation of the light source725(e.g., when the light source725is turned on and off).

The pouches then pass the camera system730. In the example illustrated, the camera system730includes a single camera with an infrared filter removed to capture both infrared images and the visible light images. The camera system730is positioned on an opposite side of the pouches (e.g., on the side of the pouches facing the transparent material) from the sensor720so the labels on the pouches do not interfere with the camera system730. A first light source760(e.g., an infrared (IR) or near infrared (NIR) spectrum light) is positioned on a side of the pouches opposite the camera system730(e.g., the side of the pouches facing the translucent material and label). The first light source760shines light through the pouches toward the camera system730to illuminate the pouches. Light from the first light source760passes through the labels so that the pharmaceuticals within each pouch cast shadows against, for example, the opaque or translucent white paper of the labels. As described above, in some embodiments, the label information on the pharmaceutical pouch is printed by a thermal printer. In these embodiments, the label information lets infrared light through the printed indicia without causing a reflection on the white paper or plastic that makes up one-half of the pharmaceutical pouch. The camera system730can then capture an image of the shadows cast by the pharmaceuticals within each pouch without any additional shadows or noise created by the label information. Similar to the sensor720, the camera system730can communicate with the processor410and memory420to store the image.

As the pouches are positioned under the camera system730, a second light source770also illuminates the pouches. The second light source770(e.g., a visible spectrum light) is positioned on the same side of the pouches as the camera system730(e.g., on the side of the pouches facing the transparent material) to shine light on the pouch and illuminate the contents of the pouch. When the second light source770illuminates the pouches, the camera system730captures a visible light image of each pouch, showing the color, shape, contour, surface finish, etc. of each pharmaceutical. In the illustrated embodiment, the pouches are stopped under the camera system730to capture the IR image and the visible light image. The camera system730captures both images without moving so the images are lined-up for computer manipulation.

In the illustrated embodiment, the first light source760and the second light source770operate in relatively rapid succession. For example, the first light source760can be turned on to briefly illuminate a pouch with infrared light, and the camera system730can capture an image of the pouch while the pouch is illuminated with the infrared light. Then, the first light source760can be turned off, the second light source770can be turned on to briefly illuminate the pouch with visible light, and the camera system730can capture an image of the pouch while the pouch is illuminated with the visible light (and before the pouch is disturbed/moved). In some embodiments, the order in which the light sources760,770are turned on and the images are captured may be reversed (e.g., the camera system730may first capture an image of the pouch while the pouch is illuminated with visible light, and then capture an image of the pouch while the pouch is illuminated with infrared light). In further embodiments, the camera system730may capture another image while both light sources760,770are on.

Although the camera system730is described as having a single camera with the infrared filter removed, in other embodiments, the camera system730may include two separate cameras. For example, one camera may be an infrared (IR) or near infrared (NIR) camera that operates in conjunction with the first light source760to capture images in the infrared spectrum, and the other camera may be a color camera or visible spectrum camera that operates in conjunction with the second light source770to capture images in the visible spectrum.

In some embodiments, the verification system610may provide real-time monitoring of the pharmaceutical pouches as they pass through the system. For example, after the camera system730captures an infrared image of a pouch, the processor410analyzes the image to determine the number and approximate size of each pharmaceutical within the pouch. Based on the label (which is associated with an expected prescription order), the processor410determines whether the correct number of pharmaceuticals were loaded into the pouch. In addition, after the camera system730captures a visible light image of the pouch, the processor410analyzes the image (in conjunction with the infrared image and the data from the label) to verify that each pouch is properly filled with the correct type of pharmaceuticals. Once the pouches are verified, the pouches are directed to the take-up roll360, a bin, or other location for distribution.

When the verification system610determines that a pouch is improperly filled, the verification system610can provide immediate feedback to discard the pouch and/or fill a new pouch. For example, the verification system610can include a printer or other device to mark an improperly filled pouch with a large X, a colorful dot, a spray, or the like to identify that the pouch has an error. A pharmacist or other technician can then discard this pouch before the pouch is distributed. In addition, the verification system610can communicate with the packaging unit120to notify the packaging unit120to fill another pouch with the same prescription order, without having to wait for further input from the pharmacist or other technician. As such, disruptions to the packaging system are reduced.

Chain of Custody

FIG.8depicts a method800of operating a pharmacy packaging system, such as the pharmacy packaging system100discussed above. The method800leverages the automated nature of the pharmacy packaging system to verify the correct type and/or number of pharmaceuticals are loaded into a pouch. In particular, the method800involves monitoring the pharmaceuticals from when the pharmaceuticals are loaded into a canister130until the pharmaceuticals are loaded and sealed in a pouch to create a chain of custody for the pharmaceuticals. This chain of custody may be stored as an electronic file in the non-transitory memory of the pharmacy packaging system or a remote computer or server.

First, a canister130is placed on a filling station (e.g., a platform, a countertop, a table, etc.) with the ability to read an RFID tag, or other information, of the canister130. The filling station then prompts an operator (e.g., pharmacist, technician, or other authorized personnel) which National Drug Code (NDC) number is needed. The user is instructed to obtain a bulk storage bottle with this NDC number, and scans a barcode on the bulk storage bottle to verify it is correct. If the barcode does not match the expected NDC number, the canister130will not later be accepted by the packaging system100(e.g., a security door on the packaging system100will not open to allow the filled canister to be loaded into the system100).

At step810, the canister130(e.g., one of the cassettes/container) is filled with pharmaceuticals by the operator. The canister130is filled by opening the canister130and inserting the desired pharmaceuticals into the canister130. After filling, the canister130is closed. In some embodiments, the canister130may be locked and tamper-proof such that only authorized personnel know how or are able to reopen the canister130. During this filling step, a camera may be positioned (e.g., located over the top of the canister130) to record that the canister130is filled. The camera can take pictures or record a video as the pharmaceuticals are poured into the canister130. In some embodiments, the canister130may include an RFID tag or other identifier to help track the canister130, as noted above.

At step820, a pharmacist verifies the canister130was properly filled by either filling the canister130him or herself, or by watching a technician or other authorized personnel fill the canister130via a video or still images. The video may be a live stream of the filling or a replay of the filling. The pharmacist then logs that the canister130was properly filled into the electronic chain of custody file. Among other things, the pharmacist may enter his/her name, the date of filling, the type of pharmaceuticals, and/or the amount of pharmaceuticals into the electronic file. By recording the filling process, tagging the canister130, and identifying the operator (e.g., through biometric, a login, etc.), the filled canister130can be certified as correctly filled by the remote pharmacist.

At step830, the filled canister130is loaded into the pharmacy packaging system100. For example, the canister130is inserted by operator into an input port (when the security door is open, if applicable) of the storage unit110. The storage unit110scans a barcode, RFID tag, or other indicia on the canister130to identify the canister130. Once verified as the correct canister130filled from the correct bulk storage bottle, the security door is closed. While the door is closed and no tampering is possible, the canister130is automatically weighed by a scale positioned in the input port to determine the number of pharmaceuticals that were added. This data can be stored by the system100to help accurately track on-hand inventory and avoid interruptions in production. The gantry assembly220of the storage unit110then moves the filled canister130from the input port to a suitable storage location. In some embodiments, the storage location may be randomized (i.e., the canister130is not always stored the same location) based on size, frequency of use, etc. As the gantry assembly220moves the filled canister130, the packaging system automatically logs where the canister130is moved to in the electronic chain of custody file.

In some embodiments, the filled canister130may be loaded into the pharmacy packaging system100before the pharmacist verifies that the canister130was filled properly. In such embodiments, the pharmacist can verify proper filling of the canister130after the canister130is stored in the storage unit110, or can flag that the canister130was improperly filled and needs to be corrected. The pharmacy packaging system100inhibits using the canister130to fill prescription orders until the canister130is verified by the pharmacist. In other embodiments, the canister130may be used to fill pouches, but the final pouches may not be released until the canisters130are verified by the pharmacist.

When a prescription order is to be filled, a technician or other personnel logs into the system100via biometrics, a passcode, or the like. The technician then verifies that the manifold320of the pharmacy packaging system is clear, at step840. If necessary, the technician may remove any pharmaceuticals or other debris from the manifold320. Once the manifold320is clear, the technician notifies the pharmacy packaging system100that the manifold320was inspected and to start filling the prescription order. In some embodiments, the technician may confirm that the manifold320is clear by pushing a button on or entering a code into the pharmacy packaging system100.

After the manifold320is cleared, at step850, the pharmacy packaging system100begins filling the prescription order. In particular, the gantry assembly220of the storage unit110retrieves the desired canister130and moves the canister130to the motor base310of the packaging unit120. As discussed above, the motor base310can operate the canister130to release pharmaceuticals from the canister130. The pharmaceuticals travel down the manifold320to be packaged into a pouch.

At step860, the pharmacy packaging system100monitors the pharmaceuticals as the pharmaceuticals are released from the canister130. For example, the motor base310may include a sensor (e.g., a light sensor, a laser beam or array, etc.) adjacent each outlet to verify whether a pharmaceutical was properly released from the canister130. The sensor can also verify that only the expected number of pharmaceuticals (e.g., one) is being released from the canister130. This information is logged in the electronic chain of custody file to record that the proper number of pharmaceuticals was released.

After the pharmaceuticals drop down the manifold320and are inserted into a pouch, at step870, the pharmacy packaging system100verifies that the correct number of pharmaceuticals reached the pharmacy packaging system and was loaded into the pouch. For example, a sensor (e.g., an image sensor or camera) on the packaging unit120captures an image of the filled pouch. The image acquired by the sensor can be analyzed by the processor410of the pharmacy packaging system100or a remote computer for object recognition. That is, the processor410simply needs to detect that the correct number of objects reached the pouch, since the pharmaceuticals have been monitored and verified from the time of loading into the canister130. When the expected number of objects is detected, the processor410logs this verification in the electronic chain of custody file. The pouch can then be released and delivered to a patient. On the other hand, if the processor410does not detect the expected number of objects, the pharmacy packaging system100can flag the pouch for further review by a pharmacist. The pharmacist then can manually verify whether the pouch was properly filled, discard the pouch, and/or refill the prescription order in a new pouch.

Method of Verifying Pharmaceuticals

The method800depicted inFIG.8provides pharmacists with a high degree of certainty that a pharmaceutical pouch was properly filled because the pharmaceuticals are tracked from when they are loaded into a canister to when they are sealed in a pouch in a closed system. In some scenarios, however, the pharmacist may be required to verify the contents of a pouch after the pouch is filled.FIG.9is a flowchart illustrating one example method900for verifying pharmaceuticals in a pharmaceutical pouch dispensed by the pharmacy packaging system100. The method900uses the verification system610described above with reference toFIG.7. This method900may be used in combination with or as an alternative to the method800.

After the pouches are filled, the pouches are directed into the verification system610. In some embodiments, the agitator710(FIG.7) may engage or agitate the pouches to help spread apart the pharmaceuticals within the pouch. In other embodiments, this step may be omitted.

Once the pouches enter the verification system610, the sensor720(e.g., label camera) can capture an image of the label on each pouch. The processor410sends a signal or command to the sensor720to capture the image. The processor410also directs the sensor720to store the captured image in the memory420or other suitable storage device.FIG.10shows an example of a label image1040that is captured by the sensor720. In other embodiments, the sensor720may read a barcode or other indicia on the label, rather than capturing an image of the label itself. In such embodiments, the processor410may retrieve information regarding the pouch from a database and display the retrieved information in place of the label image1040.

As illustrated inFIG.9, the method900includes activating the first light source760to illuminate the pharmaceutical pouch when the pharmaceutical pouch is under the camera system730(at step910). In the illustrated embodiment, the processor410sends a signal or command to the first light source760to turn the first light source760on. As explained above, the pharmaceutical pouch includes a translucent or opaque white material (e.g., paper, plastic, etc.) that makes up one side of the pouch, and a transparent material (e.g., plastic, etc.) that makes up the other side of the pouch. As can be seen fromFIG.7, when the pouch is under the camera system730, the pouch is above the first light source760. The pouch is positioned in the verification system610such that the white paper side is on the bottom side facing the first light source760, while the transparent side is on the top side facing the camera system730. When the first light source760illuminates the pouch, the pharmaceuticals within the pouch form shadows against the background of the white paper side of the pouch.

At step920, the camera system730captures a first image of the pouch while the pouch is illuminated by the first light source760. The processor410sends a signal or command to the camera system730to capture the image while the first light source760is on. The processor410also directs the camera system730to store the captured image in the memory420or other suitable storage device.FIG.10shows an example infrared image1010that is captured by the camera system730. After the image is captured, the processor410sends a signal or command to the first light source760to turn the first light source760off.

As noted above, the label may be printed on the pouch by a thermal printer. In such embodiments, the infrared light from the first light source760passes through the indicia without the indicia appearing as “noise” on the image captured by the camera system730. In embodiments where the label is applied to the pouch using other techniques, the indicia may need to be removed from the captured image. In such embodiments, the processor410can remove the indicia from the image using, for example, a software module in the control system400. For example, the processor410can analyze the label image captured by the sensor720to determine the location of the indicia “noise.” The processor410can then flip the label image (due to the orientation of the sensor720relative to the pouch versus the orientation of the camera system730relative to the pouch), and subtract the flipped label image from the infrared image captured by the camera system730.

At step930, the processor410activates the second light source770to illuminate the pharmaceutical pouch when the pharmaceutical pouch is under the camera system730. In particular, the processor410sends a signal or command to the second light source770to turn the second light source770on. The second light source770illuminates the pouch such that the pharmaceuticals within the pouch are visible to the camera system730through the transparent side of the pouch.

At step940, the camera system730captures a second image of the pouch. In the illustrated embodiment, the processor410sends a signal or command to the camera system730to capture the image while the second light source770is on. The processor410also directs the camera system730to store the captured image in the memory420or other suitable storage device.FIG.10shows an example visible light image1020that is captured by the camera system730. After the image is captured, the processor410sends a signal or command to the second light source770to turn the second light source770off.

At step950, the processor410generates a third image based on the first (i.e., infrared) image and the second (i.e., visible light) image. Several techniques are contemplated by this invention to generate the third image. One example technique of generating the third image includes super imposing the first image on the second image, and removing all other information from the second image that is not within the borders of the shadows created by the pharmaceuticals in the first image. Removing all other information may include, for example, filling regions in the third image that are not within the borders of the shadows created by the pharmaceuticals in the first image with a solid color (e.g., black). Another example technique includes filling in the colors from the second image corresponding to the shadow areas of the first image, and filling the remained of the image with a solid color (e.g., black). Once the third image is created, the processor410stores the image in the memory420or other suitable storage device.FIG.10shows an example third image1030generated by the processor410.

In some embodiments, the pill images shown in the third image1030can be rearranged by the processor410on the solid background. For example, the processor410can reorganize the pill images in a row or column to align the pill images in the same order from pouch to pouch. Such an arrangement enhances the ability of a pharmacist to observe and inspect the individual pills in each pouch. Additionally, processor410can order the individual pill images in the same order as images of expected pills1050(FIGS.10-12), which represent “gold images” of the pills from a database. Ordering the pill images in the same order as the expected pill images1050also enhances a pharmacist's ability to compare the actual pills in each pouch to the expected pills in each pouch.

The above-described methods800and900provide, among other things, verification that a pouch is filled with the proper type of pharmaceuticals because the pharmaceuticals are monitored from the time they are loaded into the canister130until they are dispensed to the customer. After the pharmaceuticals are loaded into a canister130, the pharmaceuticals are maintained in a closed system that is generally not accessible to outside users. In addition, the methods800and900verify that the pouch is filled with the proper number of pharmaceuticals using relatively low-cost sensors (e.g., light sensors on the motor base310, and an image sensor on the packaging unit120). Further, the methods800and900create an electronic chain of custody file that can be referenced later, if necessary. The method900may also create a database of filled pouch images should it be necessary to review an allegedly faulty pouch at a later time.

Dashboard

FIG.10illustrates a dashboard1000that may be generated by the processor410to aid a pharmacist in verifying that the pouches are correctly filled. The dashboard1000is a compilation of images captured by the verification system610and generated by the processor410. The dashboard1000may simultaneously display images from multiple pouches, or may only display images from one pouch at a time. In some embodiments, the dashboard1000may be displayed on a user interface (e.g., the input/output interface440) of the pharmacy packaging system100. In other embodiments, the dashboard1000may be sent to a computing device (e.g., a smart telephone or a tablet computer) of a remote pharmacist.

In the example illustrated, the dashboard1000includes an infrared first image1010, a visible light second image1020, a third image1030based on the first and second images1010,1020, label information1040(e.g., a label fourth image), and images of expected pills1050. The illustrated dashboard1000also includes an interface for navigating the dashboard1000, including patient information1060, a “Previous Pouches” button1070, an “Approve Pouches” button1080, and pouch information1090. As shown, the first image1010, the second image1020, the third image1030, and the label information1040may be arrayed in a grid on the dashboard1000, with each column of the grid corresponding to the images associated with one of the pouches. The first image1010is the infrared image captured by the camera system730. The second image1020is the visible light image captured by the camera system730. The third image1030is generated by the processor410, as explained above. The label information1040may be the label information printed on the pharmaceutical pouch captured by the sensor720, or may be a computer-generated list of data associated with the barcode of the pouch. The images of expected pills1050are stock images (e.g., “gold images” from a database) of the medication that a pharmacist should be expecting to see in the pouch based on the information input to pack the pouch.

The patient information1060provides information about a patient to whom the pharmaceutical pouch is being dispensed. The patient information1060may include, for example, patient name, administration time, facility, and the like. The “Previous Pouches” button1070allows a user (for example, a pharmacist) to navigate to screens or columns of previous pouches stored in the system. The “Approve Pouches” button1080allows the user to approve the current pouches displayed on the dashboard1000and dispense the pouches to the patient. Once the displayed pouches are approved by actuating the button1080, the dashboard1000can automatically navigate to the next screen or columns of pouches. The pouch information1090displays information about the pouches being shown on the dashboard1000. The pouch information1090may include, for example, a pouch number, an administration time, a prescription number, and the like.

In some embodiments, the processor410may also automatically determine whether the pouches are correctly filled or incorrectly filled. This automatic verification can be used to assist a pharmacist in making his or her determination on whether the pouches were properly filled. In one example technique, the processor410determines whether the number of pharmaceuticals or the type of pharmaceuticals (e.g., a first characteristic of pharmaceuticals) included in the pouch matches the number of pharmaceuticals or the type of pharmaceuticals (e.g., a second characteristic of expected pharmaceuticals) of expected pharmaceuticals, based on the label information1040or information that is in the prescription order input. The processor410may analyze the first image1010, the second image1020, the third image1030, or a combination of the three images to determine the number of pharmaceuticals included in the pouch. For example, the processor410may determine the number of pharmaceuticals included in the pouch based on the distinct shadows created in the first image1010. In other embodiments, other known techniques may be used to determine the number of pharmaceuticals included in the pharmaceutical pouch based on analyzing an image of the pharmaceutical pouch.

The processor410may also analyze the third image1030of the pharmaceutical pouch to determine the type of pharmaceuticals included in the pharmaceutical pouch. The processor410may employ known color analysis techniques on the third image1030in addition to the known information from the expected pills1050to determine whether the correct type of pharmaceuticals are included in the pharmaceutical pouch. Additionally or alternatively, the processor410may use images captured by cameras on the motor base310to help identify the pharmaceuticals and to verify that the correct pharmaceuticals were dropped.

Based on these comparisons, the processor410provides indications on the dashboard1000to identify whether the processor410thinks the pouches were correctly or incorrectly filled. For example, the dashboard1000may display a first indication on one or more of the images1010,1020,1030,1040, identifying that the processor410thinks the pouch was properly filled. In some embodiments, the dashboard1000may display a border1015having a first color (e.g., green) around the images1010,1020,1030,1040to indicate the pharmaceutical pouch is correctly filled. Similarly, the dashboard1000may display a second indication on one or more of the images1010,1020,1030,1040, identifying that the processor410thinks the pouch was not properly filled. In some embodiments, the dashboard1000displays a border1015having a second color (e.g., red) around the images1010,1020,1030,1040to indicate the pharmaceutical pouch might be incorrectly filled. Other indicia (e.g., check marks, X's, etc.) may also be used to show that a pharmaceutical pouch is correctly or incorrectly filled.

FIGS.11and12illustrate another dashboard1100that may be generated by the processor410to aid a pharmacist in verifying that the pouches are correctly filled. The dashboard1100is displayed similar to the dashboard1000and includes similar components as dashboard1000. The dashboard1100includes a fourth image1130that is generated based on the first image1010and the second image1020, similar to the third image1030. However, in illustrated the fourth image1130, the pharmaceuticals identified in the pouch are arranged in, for example, rows or columns. In addition, the images of the expected pills1050are displayed on a side of the dashboard1100such that they are along the same eye level as the fourth image1130.

In some embodiments, the fourth image1130may be generated based on the third image1030. As described above, the first image1010is used to define borders or boundaries of the pharmaceuticals within the pouch. Once the third image1030is generated, individual pharmaceuticals may similarly be identified based on the borders or boundary information from first image1010. The individual pharmaceuticals may then be arranged in rows or columns to generate the fourth image1130. The fourth image1130may be displayed such that the individual pharmaceuticals in the fourth image1130align with individual pharmaceuticals in the images of expected pills1050, as shown.

In addition, the processor410may determine whether the individual pharmaceuticals in the fourth image1130correspond to the individual pharmaceuticals in the images of expected pharmaceuticals1050. When the processor410determines that an individual pharmaceutical in the fourth image1130corresponds with the images of expected pharmaceuticals, the dashboard1100may display an indication, for example, a green border around the individual pill to indicate that the pouch includes the expected pharmaceutical (shown inFIG.12). When the processor410determines that an individual pharmaceutical in the fourth image1130does not correspond with the images of expected pharmaceutical (i.e., a wrong kind of pharmaceutical is included in the pouch), the dashboard1100may display an indication, for example a red border around the individual pill to indicate that the pouch includes a pharmaceutical that should not be included in the pouch (shown inFIG.12). In addition, the dashboard1100may also display an indication, for example a red border around an empty space aligned with an image of an expected pill1050when the expected pill is not found in the fourth image1130(e.g., when the pill is missing from the pouch) (shown inFIG.12). In some embodiments, the system may not allow the “Approve Pouches” button1080to be actuated until each red-bordered image is individually examined and accepted, rejected, quarantined, or otherwise explained by the pharmacist.

FIG.13illustrates an example fourth image1130in further detail. In the example illustrated, the fourth image1130displays a first pharmaceutical1310, a blank space1320(which indicates where a correct pill should have been displayed), and a second pharmaceutical1330. The fourth image1130also displays indications1340,1350, and1360in the form of red and green borders around the first pharmaceutical1310, the blank space1320, and the second pharmaceutical1330respectively. The first pharmaceutical1310corresponds to an image of an expected pill1050and is shown with the green border1340. The first pharmaceutical may be displayed along the same level (e.g., horizontally across or vertically above) of the corresponding image of expected pill1050(as shown inFIGS.11and12). As such, the pharmacist verifying the pouches may easily conclude that the current pouch includes a correctly filled pharmaceutical. The blank space1320is shown with the red border1350. As described above, a blank space is displayed in the fourth image1130to indicate that an expected pill is not found in the pouch being verified. The blank space1320may be displayed along the same level (e.g., horizontally across or vertically above) of a corresponding image of an expected pill1050that is missing from the pouch (as shown inFIGS.11and12). As such, the pharmacist verifying the pouches may easily conclude that the current pouch is missing a pharmaceutical. The illustrated second pharmaceutical1330is also shown with the red border1360. The second pharmaceutical1330may be a pharmaceutical that was incorrectly included in the pouch. The second pharmaceutical1330may be displayed at the end of image (e.g., after or below the correctly filled pharmaceuticals and any blank spaces). As such, the pharmacist verifying the pouches may easily conclude that the current pouch includes an improper pharmaceutical.

In some embodiments, the pharmacist may select or click the “Approve Pouches” button1080after reviewing all the pouches and each individual suspected pouch to verify that the pouch corresponding to the first column of images is correctly filled. When the pharmacist selects or clicks the “Approve Pouches” button1080, the dashboard moves the grid of images to the left by one set of images to display the contents of the next pouch. Scrolling through the pouch images in this manner allows each set of images to be aligned next to the “gold images” of the expected pills1050. As such, the pharmacist may quickly and efficiently verify the contents of the pouch when the fourth image1130is next to the images of expected pills1050.

Various features and advantages of the invention are set forth in the following claims.