Control apparatus, positioning apparatus, lithography apparatus, and article manufacturing method

A control apparatus for controlling a controlled object includes a measuring device configured to measure a state of the controlled object, and a controller configured to generate a manipulated variable corresponding to an output of the measuring device and a target value. The controller includes a compensator configured to output an index corresponding to the output of the measuring device and the target value, and a converter configured to convert the index into the manipulated variable such that a probability at which a predetermined manipulated variable is generated is a target probability.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to a control apparatus, a positioning apparatus, a lithography apparatus, and an article manufacturing method.

Description of the Related Art

Japanese Patent Laid-Open No. 2019-71405 describes a control system for controlling a stage by using a neural network. This control system includes a first control unit that outputs a first manipulated variable based on control deviation information and a second control unit that outputs a second manipulated variable by the neural network based on the control deviation information, and an adder for adding the first and second manipulated variables. The second control unit includes a restricting unit for restricting the upper and lower limits of the second manipulated variable.

Unfortunately, when the upper and lower limits of the manipulated variable are simply restricted as described in Japanese Patent Laid-Open No. 2019-71405, the neural network generates unnecessary manipulated variables, and this may prevent an improvement of the control characteristics.

SUMMARY OF THE INVENTION

The present invention provides a technique advantageous in improving the control characteristics.

One of aspects of the present invention provides a control apparatus for controlling a controlled object, comprising: a measuring device configured to measure a state of the controlled object; and a controller configured to generate a manipulated variable corresponding to an output of the measuring device and a target value, wherein the controller includes a compensator configured to output an index corresponding to the output of the measuring device and the target value, and a converter configured to convert the index into the manipulated variable such that a probability at which a predetermined manipulated variable is generated is a target probability.

DESCRIPTION OF THE EMBODIMENTS

FIG.1shows the configuration of a system SYS of an embodiment. The system SYS can include, for example, a control apparatus1including a controlled object, a control server2for controlling the control apparatus1, and a learning server3for executing learning of the control apparatus1via the control server2. The control apparatus1includes a compensator including a neural network. The learning server3can send a parameter value of the neural network to the control apparatus1via the control server2, and send a driving command to the control apparatus1via the control server2. The control apparatus1can execute an operation of driving the controlled object in accordance with the driving command, and send the driving result (for example, a control deviation) to the learning server3via the control server2. The learning server3can calculate a reward based on the driving result, and update the parameter value of the neural network based on the reward.

The functions of the control server2can wholly or partly be incorporated into the control apparatus1. Alternatively, the functions of the control server2can wholly or partly be incorporated into the learning server3. Alternatively, the functions of the control server2and the learning server3can partly be incorporated into the control apparatus1. Alternatively, the control apparatus1, the control server2, and the learning server3can be integrated. The control server2can be a host apparatus of the control apparatus1. For example, the control apparatus1can be one constituent element of a lithography apparatus, and the control server2can be a main control apparatus of the lithography apparatus, or a comprehensive control apparatus for controlling a plurality of lithography apparatuses including the lithography apparatus.

FIG.2shows an example in which the control apparatus1in the system SYS shown inFIG.1is applied to a positioning apparatus. The control apparatus1configured as a positioning apparatus can be so configured as to control a stage ST as a controlled object. The control apparatus1shown inFIG.2can include a stage mechanism5, a measuring device6, a driver7, and a control board (control unit)8. The stage mechanism5can include the stage ST, and an actuator (not shown) for driving the stage ST. The stage ST can hold a positioning target such as a substrate. The actuator can include at least one of a linear motor, an electromagnetic actuator, a voice coil motor, and a rack-and-pinion mechanism. The controlled object can be the stage ST, or a member (for example, a substrate) held on the stage ST. More macroscopically, the controlled object can be understood as the stage mechanism5. The measuring device6can measure the state (for example, the position) of the controlled object. The control board8can send a command (manipulated variable) to the driver7, and output an electric current to the actuator of the stage mechanism5. The state (driving result) of the stage mechanism5or the stage ST can be measured or detected by the measuring device6, and provided to the control board8.

FIG.3shows a more detailed configuration example of the control apparatus1shown inFIG.2. The control board8can include a calculator9for calculating a difference (control deviation) between the state (for example, the position) of a controlled object measured by the measuring device6and a driving command (target value) for controlling the controlled object, and a controller10for generating a manipulated variable corresponding to the output from the calculator9. The controller10can also be understood as a constituent element for generating a manipulated variable based on the state of the controlled object and the driving command. The controller10can operate as a compensator defined by the parameter value of a neural network provided by the learning server3. The driver7can convert the manipulated variable provided by the control board8or the controller10into an electric current. In other words, the driver7outputs, to (the actuator of) the stage mechanism5, an electric current having a magnitude corresponding to the manipulated variable provided by the control board8or the controller10.

FIG.4shows a neural network learning sequence S100using reinforcement learning. First, in step S101, the learning server3communicates with the control apparatus1via the control server2, and initializes the parameter value of the neural network of the controller10. Then, in step S102, the learning server3sends a predetermined operation command to the control apparatus1so as to drive the stage mechanism5(the stage ST), via the control server2. In an example, upon receiving the operation command, the control server2can supply a driving command (target value string) to the control apparatus1so as to drive the stage ST of the stage mechanism5along a driving orbit corresponding to the operation command. The control apparatus1can be so configured as to accumulate the driving results of the stage mechanism5, for example, the differences (control deviations) calculated by the calculator9, and provide the driving results to the control server2or the learning server3in accordance with a request from the control server2or the learning server3.

In step S103, the learning server3acquires the driving results accumulated by the operation in step S102from the control apparatus1via the control server2. In step S104, the learning server3calculates a reward based on the driving results acquired in step S103. An equation for calculating the reward can be so determined that, for example, a high reward is obtained when the control deviation is small. In step S105, the learning server3determines whether the reward calculated in step S104satisfies a learning termination condition. If the reward satisfies the learning termination condition, the learning server3advances the process to step S106; if not, the learning server3advances the process to step S107. In step S107, the learning server3changes the parameter value of the neural network of the controller10, and executes steps S102to S105again after that. In step S106, the learning server3saves the latest parameter value of the neural network as a learning result.

FIG.5shows an example of a configuration for determining the manipulated variable for the driver7by inputting the control deviation to the controller10. The controller10can include a compensator510for outputting an index503corresponding to the control deviation (the difference between the output from the measuring device6and the target value), and a converter520for converting the index503into a manipulated variable504in accordance with a conversion rule521. The conversion rule521can be given by, for example, a conversion table for defining a plurality of manipulated variables504respectively corresponding to a plurality of indices503. The compensator510is formed by a neural network, and the neural network can include an input layer500, a hidden layer501, and an output layer502. The output layer502can be formed by a plurality of neurons respectively corresponding to the plurality of indices503that can be taken by the input with respect to the conversion rule521. The operation of the neural network forming the compensator510is defined by a preset parameter value, and the neural network calculates the activity of each neuron in the hidden layer501and the output layer502based on a control deviation input to the input layer500. Then, the neural network forming the compensator510selects the index503corresponding to the most active neuron of the plurality of neurons in the output layer502, and outputs the index503as the calculation result of the neural network. The most active neuron of the plurality of neurons is a neuron whose activity has the largest numerical value. The converter520outputs the manipulated variable504corresponding to the input index503in accordance with the conversion rule521. In other words, the converter520converts the input index503into the manipulated variable504in accordance with the conversion rule521, and outputs the manipulated variable504.

One feature of this embodiment is the conversion rule521. To deeply understand the conversion rule521of this embodiment, a comparative example and a problem of the comparative example will be explained first.

FIG.6shows a conversion rule in the comparative example. Referring toFIG.6, the abscissa indicates the index, and the ordinate indicates the conversion rule as a manipulated variable Fn. In this comparative example, the conversion rule is so defined that when a minimum value of the manipulated variable Fn is F0, a maximum value of the manipulated variable Fn is FN, and the index is n, the index n and the manipulated variable Fn have a linear relationship as indicated by equation (1) below:
Fn=n×(FN−F0)/N+F0(n=0 toN)  (1)

Learning was performed in a state in which the conversion rule as described above was set. As the configuration of the control board8, each of the number of neurons in the output layer502, the number of indices503, and the number of manipulated variables504was set at 9, and F0was determined such that F4=0. Under this condition, learning was performed so that the control deviation decreased when the driving command (target value) for the control apparatus1had a predetermined value.

FIG.7shows the frequency of use of each manipulated variable when a maximum reward was obtained by learning in the comparative example. This result shows that manipulated variables F0, F2, F5, and F7were not used at all. That is, it was determined as a result of the learning that these manipulated variables are unnecessary. The configuration shown inFIG.3has the restriction that the controller10must generate an output (that is, a manipulated variable) at a predetermined period, so the calculation amount of the controller10desirably has no waste. In the comparative example, however, unnecessary manipulated variables exist among the calculated manipulated variables, so it is necessary to delete these manipulated variables or replace them with other manipulated variables.

The conversion rule521of the converter520according to this embodiment will be explained below in comparison with the abovementioned comparative example. In this embodiment, the conversion rule521is set such that the probability distribution of the manipulated variables output from the converter520or the controller10follows a target probability distribution. The manipulated variable probability distribution is the distribution of probability at which each value of the manipulated variable can appear within the range of the minimum and maximum values of the manipulated variable. When learning is started by defining or setting the conversion rule521as described above, waste calculations by the controller10can be omitted by reducing the generation of unnecessary manipulated variables. This is advantageous in improving the control characteristics of the control apparatus1.

The manipulated variable must be so output as to decrease the control deviation of the stage ST. Therefore, when the manipulated variable is plotted on the abscissa and the probability is plotted on the ordinate, the conversion rule521preferably has a shape (probability distribution) projecting upward in the entire area between the minimum and maximum values of the manipulated variable. Also, this shape preferably has a peak in a position where the manipulated variable is 0 when an offset value is 0, and a peak in a position where the manipulated variable is an offset value when the offset value is not 0. Note that the offset value can be used to cancel an individual difference caused by a tolerance or the like. The shape that projects upward can be, for example, a normal distribution.

In an example, as shown inFIG.8, a plurality of manipulated variables F0to F8can be determined based on a target probability distribution900so that the frequencies of use (probabilities) (an area inFIG.8) of the plurality of manipulated variables F0to F8are constant.FIG.9shows the relationship (that is, the conversion rule) between the index and the manipulated variable, which corresponds to the target probability distribution900and the plurality of manipulated variables F0to F8shown inFIG.8. The maximum value F8and the minimum value F0of the manipulated variable can be determined based on, for example, the weight and the maximum driving acceleration of the stage ST as a controlled object, and the magnification of an assumed maximum control deviation. In addition, when the target probability distribution900having the shape projecting upward is a normal distribution, its variance value is related to the resolution of the manipulated variable and hence can be determined in accordance with an assumed maximum control deviation.

FIG.10shows a control deviation in a state in which a parameter value with which a maximum reward is obtained is set in a neural network, in relation to the comparative example (a dotted line) and this embodiment (a solid line). When compared to the comparative example, a manipulated variable close to zero (F4) can be used in this embodiment, so the output resolution to the driver7improves, and as a consequence the control deviation is suppressed more. This means that the control characteristics of this embodiment are superior to those of the comparative example. Also, the time required for learning of this embodiment is shorter than that of the comparative example. In addition, the number of neurons in the neural network forming the compensator510of this embodiment is the same as that of the comparative example, so the calculation amount of the controller10of this embodiment is also the same as that of the comparative example. However, the position accuracy of the stage ST of this embodiment is higher than that of the comparative example. Furthermore, the effects of this embodiment can be obtained even when the compensator510is not formed by a neural network.

FIG.11shows another example of the configuration for determining the manipulated variable to the driver7by inputting a control deviation to the controller10. The controller10can include a compensator610for outputting an index603corresponding to a control deviation (the difference between the output value of the measuring device6and a target vale), and a converter620for converting the index603into a manipulated variable in accordance with a conversion rule621. The compensator610is formed by a neural network, and the neural network can include an input layer600, a hidden layer601, and an output layer602. The output layer602is formed by a single neuron. The operation of the neural network forming the compensator610is defined by a preset parameter value. The neural network calculates the activity of each neuron in the hidden layer601and the output layer602based on a control deviation input to the input layer600. Then, the neural network forming the compensator610outputs the index603in which the numerical values of the neuron activity in the output layer602are normalized to numerical values from0to1, as the calculation result of the neural network. The conversion rule621of the converter620gives a manipulated variable604when an area (integrated value) from F0matches the index603, in a target probability distribution (for example, a normal distribution) having a shape projecting upward. In other words, the conversion rule621is so set that the probability distribution of the manipulated variable604follows the target probability distribution. The converter620calculates the manipulated variable604in accordance with the conversion rule621like this, and outputs the manipulated variable604as an output to the controller10. When the target probability distribution defining the conversion rule621is a normal distribution, the average value and the variance value of this normal distribution are arbitrary values.

When a plurality of driving profiles are prepared to control driving of the stage mechanism5or the stage ST, it is possible to install the controller10for each driving profile, and select the controller10corresponding to the driving profile from the plurality of controllers10. In this case, the conversion rules of the converters520and620can be determined for each controller10.

In the controller10having a relationship (conversion rule) between the index and the manipulated variable determined based on a given target probability distribution, in order to achieve a higher positioning accuracy after a series of learning sequences are performed, the learning sequences can be performed again by changing the target probability distribution. For example, in the controller10having an index-manipulated variable relationship determined based on a given target probability distribution, in order to achieve a higher positioning accuracy after a series of learning sequences are performed, the learning sequences can be performed again by changing the target probability distribution. This change of the target probability distribution can include, for example, a change of at least one of the average value and the variance value.

A case in which a plurality of stage control apparatuses as shown inFIG.3are adjusted will be explained below. In this case, after a parameter value20of a neural network is obtained by performing the learning sequence S100in a first stage control apparatus, the learned parameter value20can be applied to a second stage control apparatus. Consequently, the second stage control apparatus can achieve a positioning accuracy equivalent to that of the first stage control apparatus without performing any learning. It is also possible to perform the learning sequence S100by applying the learning parameter value20as an initial value to the second stage control apparatus. In this case, the learning can also be performed by changing the spread amount and the shift amount in the manipulated variable direction of the target probability distribution that provides the conversion rule of the controller10of the second stage control apparatus.

The second embodiment will be explained below. Items not mentioned in the second embodiment follow those of the first embodiment.FIG.12shows the configuration of a control apparatus1of the second embodiment. The control apparatus1of the second embodiment is configured as a positioning apparatus in this example, but the control apparatus1can also be configured as a control apparatus of another form. A control board8can include a calculator9for calculating a difference (control deviation) between the state (for example, the position) of a controlled object measured by a measuring device6and a driving command (target value) for controlling the controlled object, and first and second controllers30and40for generating first and second manipulated variables corresponding to the output of the calculator9. The controllers30and40can also be understood as constituent elements for generating manipulated variables based on the state of a controlled object and a driving command. The first controller30can be, for example, a PID controller. The second controller40operates as a compensator defined by a parameter value of a neural network provided from a learning server3. Like the controller10of the first embodiment, the second controller40includes a compensator510(610) and a converter520(620). The control board8can also include a calculator (adder)60for generating a manipulated variable for manipulating a controlled object based on the first and second manipulated variables, and a switch50for opening/closing a path connecting the second controller40and the calculator60. A driver7can convert the manipulated variable provided by the control board8or the controller10into an electric current.

FIG.13shows an example of a learning sequence200of the control apparatus1of the second embodiment.FIG.13shows a neural network learning sequence using reinforcement learning corresponding to the configuration shown inFIG.12. First, in step S201, the learning server3turns off the switch50. Consequently, the control board8or the control apparatus1is set in a mode by which the calculator60generates a manipulated variable based on not the second manipulated variable but the first manipulated variable. Then, in step S202, the learning server3sends a predetermined operation command to the control apparatus1so as to operate a stage mechanism5, via a control server2. In this example, upon receiving the operation command, the control server2can supply a driving command (target value string) to the control apparatus1so as to drive a stage ST along a driving orbit corresponding to the operation command. The control apparatus1can be so configured as to accumulate manipulated variables generated by the control board8(the first controller30), and provide the driving result to the control server2or the learning server3in accordance with a request from the control server2or the learning server3. In step S203, the learning server3obtains the manipulated variables accumulated by the operation in step S202from the control apparatus1via the control server2. In step S204, the learning server3determines whether the number of times of execution of steps S202and S203is equal to or larger than a predetermined value. If the number of times of execution is equal to or larger than the predetermined value, the learning server3advances the process to step S205; if not, the learning server3executes steps S202and S203again.

In step S205, the learning server3generates a manipulated variable frequency distribution based on the manipulated variables obtained over a predetermined number of times, and determines a target probability distribution corresponding to the frequency distribution. This target probability distribution corresponding to the frequency distribution can be a probability distribution having a shape identical or similar to that of the frequency distribution, or a probability distribution having the same feature as a feature extracted from the frequency distribution. However, the target probability distribution can also have another correlation with respect to the frequency distribution.

In step S206, the learning server3generates a conversion rule corresponding to the target probability distribution determined in step S205, and sets this conversion rule in the converter of the second controller40. This process is the same as that of the first embodiment. In step S207, the learning server3turns on the switch50. As a consequence, the control board8or the control apparatus1is set in a mode by which the calculator60generates a manipulated variable to be supplied to the driver7, based on the first and second manipulated variables. Then, in step S208, the learning server3performs steps S101to S107shown inFIG.4. Note that the neural network parameter value is set in the neural network of the second controller40.

In the second embodiment, the conversion rule to be set in the converter of the second controller40is a conversion rule corresponding to the probability distribution of the first manipulated variable when a controlled object is controlled in the mode by which the calculator60generates a manipulated variable based on not the second manipulated variable but the first manipulated variable. The learning server3functions as a setting device for setting the conversion rule like this in the converter of the second controller40. The setting device like this can also be incorporated into the control apparatus1.

The abovementioned positioning apparatus can be incorporated into a lithography apparatus for transferring an original pattern onto a substrate, and the stage ST of this positioning apparatus can be so configured as to hold and position the substrate.FIG.14shows an example of the configuration of an exposure apparatus EXP as an example of the lithography apparatus incorporating the abovementioned stage mechanism. The exposure apparatus EXP can include a projection optical system PO for projecting a pattern of an original R onto a substrate W. The exposure apparatus EXP can also include a substrate positioning apparatus PA for positioning the substrate W, an original positioning apparatus RSM for positioning the original R, and an illumination optical system IO for illuminating the original R. The abovementioned positioning apparatus is applicable to the substrate positioning apparatus PA. The substrate positioning mechanism PA can include a stage WS as a movable part for holding the substrate W, and an actuator WSA for driving the stage WS. Alternatively, the abovementioned positioning apparatus is applicable to the original positioning apparatus RSM.

FIG.15shows an example of the configuration of an imprint apparatus IMP as an example of the lithography apparatus incorporating the abovementioned stage mechanism. The imprint apparatus IMP transfers a pattern of an original M onto an imprint material IM on the substrate W. The imprint apparatus IMP can include the substrate positioning apparatus PA for positioning the substrate W, an imprint head IH for driving the original M, and a curing unit CU for curing the imprint material. The imprint head IH can bring a pattern region of the original M into contact with the imprint material IM on the substrate W, and can separate the original M from the cured imprint material IM. The curing unit CU can irradiate the imprint material IM with curing energy (for example, light energy) in a state in which the pattern region of the original M is in contact with the imprint material IM on the substrate W, thereby curing the imprint material IM. The abovementioned positioning apparatus can be applied to the substrate positioning apparatus PA.

The abovementioned lithography apparatus is applicable to an article manufacturing method of manufacturing an article. This article manufacturing method can include a pattern formation step of forming a pattern on a substrate by using the abovementioned lithography apparatus, and a processing step of obtaining an article by processing the substrate on which the pattern is formed. The processing step can include a step of etching the substrate by using the pattern as an etching mask. The processing step can include a step of forming a film on the substrate, a step of sealing the substrate, and the like. The article manufacturing method can also perform a pattern formation method a plurality of times.

OTHER EMBODIMENTS

This application claims the benefit of Japanese Patent Application No. 2021-027746, filed Feb. 24, 2021, which is hereby incorporated by reference herein in its entirety.