Patent ID: 12211718

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

This disclosure generally relates to alignment of robotic systems, and more specifically, to methods and systems of image based robot alignment. Aspects of this disclosure will be described with reference to a semiconductor wafer handling robot and a semiconductor wafer processing device, but these aspects may be applied to alignment of any other robotic system.

FIG.1is a simplified block diagram of an example system100for monitoring alignment of a first component102and a second component104. More specifically, the system monitors alignment of the second component104relative to the first component102during an operation of the second component. In the example embodiment, the first component102is a semiconductor wafer processing device, and the second component104is a semiconductor wafer handling robot. The semiconductor wafer handling robot retrieves a semiconductor wafer (not shown inFIG.1) and positions the semiconductor wafer in the semiconductor wafer processing device to allow the semiconductor wafer processing device to perform a process on the wafer. The process may be, for example, an annealing process, an etching process, a polishing process, or any other semiconductor wafer process. At least a portion of the second component104is moveable relative to the first component102. For example, the second component104may include an arm that holds the semiconductor wafer and moves relative to the first component102to move the wafer into the first component102.

A camera106is positioned in fixed relationship to the first component102and is operable to capture images of the first component102and the second component104during operation of the second component104. The camera106is a visible light camera capturing visible light images of at least a portion of the first component102and the second component104during the operation. In other embodiments, the camera106may be an infrared camera or any other suitable imaging device. In some embodiments, the camera106is a video camera and the images captured by the camera106are the individual frames of the video captured by the video camera. The camera106is positioned relative to the first component to capture an image of at least part of each of the first component102and the second component104when the second component104is in a predetermined position relative to the first component102. The predetermined position is the position of the second component104relative to the first component102at a certain step of the operation. For example, when used in semiconductor wafer processing, the predetermined position may be the position of the semiconductor wafer handling robot (second component104) when stops to deposit a semiconductor wafer in the semiconductor wafer processing device (first component102).

A controller108is communicatively coupled to the camera106to receive images captured by the camera106and to control the camera106, such as to adjust the camera settings, to instruct the camera106when to capture an image, and the like.

FIG.2is a block diagram of an example computing device200that may be used as, or included as part of, the controller108. The computing device200includes a processor201, a memory202, a media output component204, an input device206, and a communications interface208. Other embodiments include different components, additional components, and/or do not include all components shown inFIG.2.

The processor201is configured for executing instructions. In some embodiments, executable instructions are stored in the memory202. The processor201may include one or more processing units (e.g., in a multi-core configuration). The term processor, as used herein, refers to central processing units, microprocessors, microcontrollers, reduced instruction set circuits (RISC), application specific integrated circuits (ASIC), a programmable logic circuit (PLC), and any other circuit or processor capable of executing the functions described herein. The above are examples only, and are thus not intended to limit in any way the definition and/or meaning of the term “processor.”

The memory202stores non-transitory, computer-readable instructions for performance of the techniques described herein. Such instructions, when executed by the processor201, cause the processor201to perform at least a portion of the methods described herein. That is, the instructions stored in the memory202configure the controller108to perform the methods described herein. In some embodiments, the memory202stores computer-readable instructions for providing a user interface to the user via media output component204and, receiving and processing input from input device206. The memory202may include, but is not limited to, random access memory (RAM) such as dynamic RAM (DRAM) or static RAM (SRAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and non-volatile RAM (NVRAM). Although illustrated as separate from the processor201, in some embodiments the memory202is combined with the processor201, such as in a microcontroller or microprocessor, but may still be referred to separately. The above memory types are example only, and are thus not limiting as to the types of memory usable for storage of a computer program.

The media output component204is configured for presenting information to a user (e.g., an operator of the system). The media output component204is any component capable of conveying information to the user. In some embodiments, the media output component204includes an output adapter such as a video adapter and/or an audio adapter. The output adapter is operatively connected to the processor201and operatively connectable to an output device such as a display device (e.g., a liquid crystal display (LCD), light emitting diode (LED) display, organic light emitting diode (OLED) display, cathode ray tube (CRT), “electronic ink” display, one or more light emitting diodes (LEDs)) or an audio output device (e.g., a speaker or headphones).

The computing device200includes, or is connected to, the input device206for receiving input from the user. The input device206is any device that permits the computing device200to receive analog and/or digital commands, instructions, or other inputs from the user, including visual, audio, touch, button presses, stylus taps, etc. The input device206may include, for example, a variable resistor, an input dial, a keyboard/keypad, a pointing device, a mouse, a stylus, a touch sensitive panel (e.g., a touch pad or a touch screen), a gyroscope, an accelerometer, a position detector, an audio input device, or any combination thereof. A single component such as a touch screen may function as both an output device of the media output component204and the input device206.

The communication interface208enables the computing device200to communicate with remote devices and systems, such as the camera106, remote sensors, remote databases, remote computing devices, and the like, and may include more than one communication interface for interacting with more than one remote device or system. The communication interfaces may be wired or wireless communications interfaces that permit the computing device200to communicate with the remote devices and systems directly or via a network. Wireless communication interfaces may include a radio frequency (RF) transceiver, a Bluetooth® adapter, a Wi-Fi transceiver, a ZigBee® transceiver, a near field communication (NFC) transceiver, an infrared (IR) transceiver, and/or any other device and communication protocol for wireless communication. (Bluetooth is a registered trademark of Bluetooth Special Interest Group of Kirkland, Washington; ZigBee is a registered trademark of the ZigBee Alliance of San Ramon, California.) Wired communication interfaces may use any suitable wired communication protocol for direct communication including, without limitation, USB, RS232, I2C, SPI, analog, and proprietary I/O protocols. In some embodiments, the wired communication interfaces include a wired network adapter allowing the computing device200to be coupled to a network, such as the Internet, a local area network (LAN), a wide area network (WAN), a mesh network, and/or any other network to communicate with remote devices and systems via the network.

The computer systems discussed herein may include additional, less, or alternate functionality, including that discussed elsewhere herein. The computer systems discussed herein may include or be implemented via computer-executable instructions stored on non-transitory computer-readable media or medium.

FIG.3is a view of the system100from above the system100, for example as seen from the location of the camera106. The controller108and the camera106are not shown inFIG.3. As seen inFIG.3, the example second component104includes a moveable arm300holding a wafer302. The second component104is placing the wafer302within a chamber304of the first component102. The arm300extends over a basement portion306of the first component and into the chamber304. In the example embodiment, the arm300includes a distinct visible feature that is a notch308through which part of the first component102is visible from the viewpoint of the camera106. The visible feature may be any visible feature that can be identified by the controller108. In other embodiments, the arm300includes a different visible feature (such as a protrusion, a through-hole, an engraving, or the like) or includes an added-on distinct visible feature (such as a distinctly colored or shaped sticker/mark, a QR code, or the like).

The camera106captures images of an image area310. The memory202stores instructions executed by the processor201to configure the controller108to receive a first captured image (of image area310) from the camera106when the second component104is in the predetermined position relative to the first component102. This first image is captured when the second component104is known to be in the predetermined position. For example, the user may position the second component104in the predetermined position or make adjustments to the position of the second component104to place it in the predetermined position. This first image is thus an image of the correct predetermined position, such as the position that properly aligns a wafer for processing by the first component102.

FIG.4is an example image400of the image area310. As can be seen, the basement portion306is different color (or a darker color) than the arm300. The notch308in the arm300, through which part of the basement portion306may be seen, is a visibly distinct feature in the image400.

The controller108receives a selection of a region of interest (ROI)402in the first captured image400. In the example embodiment, the ROI402includes a portion of the first component102and a portion of the second component104. Alternatively, the ROI may include only a portion of the second component104(e.g., when the visible feature is a sticker surrounded completely by the arm300). The ROI402may be selected manually by a user, such as by using the input device106. In other embodiments, the ROI402is selected by the controller108, such as by performing object detection and recognition on the image to identify the first component102and the second component104and/or the visible feature.

FIG.5is a close-up view of the portion404in the image400. The controller108is configured to identify a visible feature (e.g., the notch308) of the second component104within the ROI402of the first captured image. In this embodiment, the controller108specifically identifies a corner500of the notch308within the ROI402. In the example embodiment, the controller108identifies the corner500the first image400based on intensity values of pixels within the first image400. That is, the basement portion306is identified in the image400by its darker color, and the arm300is identified by its lighter color (and differing intensity values) than the darker colored basement portion306. The intersection imaginary lines marking the transitions between the basement portion306in the ROI402of the image400and the arm300in the ROI402of the image400(as determined from the intensity values) is the corner500. In other embodiments, the controller108identifies the visible feature by receiving a selection of the visible feature from a user input.

Once the visible feature (e.g., corner500) is identified, the controller108determine first coordinates of the identified visible feature of the second component within the ROI of the first captured image400. The coordinates are the X-Y coordinates (as identified inFIG.4) of the point at which the visible feature is located within the image400. The X-Y coordinates may be, for example, determined by the location of the central pixel in the visible feature in the image400. The controller108then stores the first coordinates in the memory. The controller108also determines the coordinates of the ROI402and stores the ROI coordinates in the memory202. The ROI coordinates may be two coordinates, such as the coordinates of diagonally opposite corners, a single coordinate of one corner along with the dimensions and direction of the ROI402relative to that corner, or any other suitable coordinate or coordinates that identify the location of the ROI402.

Because the camera106is fixed in relation to the first component102and only captures images of the same image area310, any time the arm300is in the predetermined position, the visible feature should be in the same location (i.e., at the same X-Y coordinates) within a captured image. Thus, the first coordinates may be used to determine if the arm400is in the predetermined position during subsequent operations.

Thus, the controller108is configured to receive captured images from the camera106during a subsequent operation. The controller identifies a second captured image from the received captured images during the subsequent operation when the second component104is expected to be in the predetermined position relative to the first component102. The time when the second component104is expected to be in the predetermined position relative to the first component102may be determined by the instructions for controlling the second component104. For example, if the controller108also controls the second component104, when the controller108knows from the instructions and/or from communication from the second component104that the second component104is believed to be at the predetermined location. That is, when the second component is at the step in the process at which the second component104should be at the predetermined location, the controller108knows that the second component104should be at the predetermined location and the image captured at this time is used as the second captured image.

In other embodiments, the controller108determines that the second component104is expected to be in the predetermined position based on analysis of the captured images. For example,FIG.6is a graph of image intensity of two points in images captured by the camera over time (or more specifically over a sequence of video frames when the camera106is a video camera). The two points are selected so that one point in is a point on the first component102in the image and one point is a point on the second component104in the image, when the second component104is in the predetermined position. For example, the upper left corner and the lower left corner of the ROI402shown inFIGS.4and5may be used as the two points. As noted above, in the example, the basement portion306is darker than the arm300. Thus, when the arm300is not in the predetermined position, both points will have a relatively low intensity because both points in the image will be points of the basement portion. When the arm300is moving into position, the intensity of the points seen in the image of may vary as what is located at the two points varies. When the arm300is in position, it will stop for a period of time to place the wafer in the first component102, and one point will be on the basement (and have a low intensity) and one point will be on the arm (and have a high intensity). Thus, when the intensity of the point associated with the arm300is relatively high and stable for a period of time, the arm300is determined to be in position, and the captured image from this time is used as the second captured image. In still other embodiments, the controller108determines that the second component104is expected to be in the predetermined position by locating and tracking the visible feature through multiple captured images. When the visible feature is within the ROI402and has not moved for a period of time, the controller108determines that the second component104is expected to be in the predetermined position, and uses the captured image from that time as the second captured image.

The controller108identifies the visible feature of the second component104in the second captured image. In the example embodiment in which the visible feature is the corner500of the notch308, a FAST (Features from Accelerated Segment Test) algorithm is used. The FAST algorithm detects any corners in ROI so the important thing is to give a clear difference in pixel intensity around the corner. The number of corners detected is dependent on the threshold in the function of FAST. The format of FAST is FAST(InputArray image, std::vector<KeyPoint>& keypoints, int threshold, bool nomaxSuppression=true). As the threshold goes lower, the number of corners detected increases. The appropriate threshold for every image to return only one corner may be determined by looping the threshold from 200 to 1.

Although the example embodiment uses a corner, any point can be used as reference as long as it has different pixel intensity in the ROI402. For example, a black spot or a white spot can be used as the visible feature. Instead of using the FAST function in OPENCV, the position of the black or white spot may be found by analyzing pixel intensity in ROI402. This is simply calculated with the function of “minMaxLoc”. cv::minMaxLoc(ROI, &min, &max, &min_loc, &max_loc), where min_loc is used if the visible feature is darker (black spot) than the area around it around it, while max_loc is used if the visible feature is brighter (white spot) than the area around it around it.

After the visible feature is identified in the second captured image, the controller108determines if the second component104is in the predetermined position relative to the first component102based on the second captured image and the identified visible feature of the second component104within the ROI402of the first captured image. In some embodiments, the controller108determines second coordinates of the identified visible feature of the second component in the second captured image. The second coordinates are X-Y coordinates of the location of the visible feature in the second captured image. If the second component104is properly in the predetermined position, the second coordinates should match the first coordinates determined from the first captured image. Thus, the controller108is configured to determine if the second component104is in the predetermined position relative to the first component102by comparison of the second coordinates to the first coordinates.

In some embodiments, the second component104is determined to be in the predetermined position relative to the first component102when the second coordinates are the same as the first coordinates. Alternatively, the second component104may be determined to be in the predetermined position relative to the first component102when the second coordinates are within a threshold distance of the first coordinates. The threshold distance may be selected to account for possible slight variations in the images or the detection of the visible feature, or to allow for a certain amount of acceptable variation in the positioning of the second component104.

When the controller108determines that the second component104is expected to be in the predetermined position but it is not, the controller108may generate an alarm. That is when the controller108determines that the second component104is expected to be in the predetermined position, but the visible feature in the second captured image does not have second coordinates equal to (or within a threshold distance of) the first coordinates, the controller108generates an alarm. The alarm may be a human cognizable alarm, such as a flashing light or siren, a computer cognizable alarm, such as an alarm message sent to a system controller, or both. In response to the alarm, a user may adjust or repair the second component104as needed to return the second component104to the proper alignment relative to the first component102.

FIG.7is a flowchart of an example method700for monitoring alignment of a second component relative to a first component. The method may be performed by system100and its components, or may be used with any other suitable system or components.

Any logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. In addition, other steps may be provided, or steps may be eliminated, from the described flows, and other components may be added to, or removed from, the described systems. Accordingly, other embodiments are within the scope of the following claims.

It will be appreciated that the above embodiments that have been described in particular detail are merely example or possible embodiments, and that there are many other combinations, additions, or alternatives that may be included.

Also, the particular naming of the components, capitalization of terms, the attributes, data structures, or any other programming or structural aspect is not mandatory or significant, and the mechanisms that implement the disclosure or its features may have different names, formats, or protocols. Further, the system may be implemented via a combination of hardware and software, as described, or entirely in hardware elements. Also, the particular division of functionality between the various system components described herein is merely one example, and not mandatory; functions performed by a single system component may instead be performed by multiple components, and functions performed by multiple components may instead performed by a single component.

Approximating language, as used herein throughout the specification and claims, may be applied to modify any quantitative representation that could permissibly vary without resulting in a change in the basic function to which it is related. Accordingly, a value modified by a term or terms, such as “about” and “substantially”, are not to be limited to the precise value specified. In at least some instances, the approximating language may correspond to the precision of an instrument for measuring the value. Here and throughout the specification and claims, range limitations may be combined and/or interchanged, such ranges are identified and include all the sub-ranges contained therein unless context or language indicates otherwise.

Various changes, modifications, and alterations in the teachings of the present disclosure may be contemplated by those skilled in the art without departing from the intended spirit and scope thereof. It is intended that the present disclosure encompass such changes and modifications.

This written description uses examples to describe the disclosure, including the best mode, and also to enable any person skilled in the art to practice the disclosure, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the disclosure is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims.