Source: http://www.google.com/patents/US20020123812?dq=5,941,947
Timestamp: 2017-02-24 13:29:46
Document Index: 174239458

Matched Legal Cases: ['art 240', 'art 242', 'art 246', 'art 246', 'art 248', 'art 200', 'art 223']

Patent US20020123812 - Virtual assembly design environment (VADE) - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsA virtual assembly design environment that simulates axial and planar constrained motion for multiple parts in any combination and application order. Dynamic simulation methods are used to simulate object behavior in the design environment using physical laws and collision detection algorithms. The physical...http://www.google.com/patents/US20020123812?utm_source=gb-gplus-sharePatent US20020123812 - Virtual assembly design environment (VADE)Advanced Patent SearchTry the new Google Patents, with machine-classified Google Scholar results, and Japanese and South Korean patents.Publication numberUS20020123812 A1Publication typeApplicationApplication numberUS 09/888,055Publication dateSep 5, 2002Filing dateJun 21, 2001Priority dateDec 23, 1998Also published asWO2000038117A1, WO2000038117B1Publication number09888055, 888055, US 2002/0123812 A1, US 2002/123812 A1, US 20020123812 A1, US 20020123812A1, US 2002123812 A1, US 2002123812A1, US-A1-20020123812, US-A1-2002123812, US2002/0123812A1, US2002/123812A1, US20020123812 A1, US20020123812A1, US2002123812 A1, US2002123812A1InventorsSankar Jayaram, Uma Jayaram, Yong Wang, Hrishikesh Tirumali, Hiral Chandrana, Hugh Connacher, Kevin Lyons, Peter HartOriginal AssigneeWashington State University Research Foundation.Export CitationBiBTeX, EndNote, RefManPatent Citations (4), Referenced by (137), Classifications (15), Legal Events (2) External Links: USPTO, USPTO Assignment, EspacenetVirtual assembly design environment (VADE)
[0238] Obviously, the X, Y, Z offsets and X, Y, Z rotation angles. [0239] Once this UDF is defined, it is saved as PARTCS in the UDF library. A flowchart 240 is shown in FIG. 43 for a UDF method that employs automatic assembly and swept volume generation. On the base part, a default coordinate system is first created by using DEFAULTCS, then the coordinate systems from PARTCS UDF is created by referencing this DEFAULTCS. The actual positions and orientations are decided by the values we obtained from the trajectory calculation. Also, a DEFAULTCS is created on the part that is going to be assembled. The part can then be placed by referencing the PARTCS on the base part and DEFAULTCS on the part. Once they are assembled, the merge function is used to merge the part into the base part. All the instances can be processed this way and finally the base part represents the parametric model of the swept volume. The complicated processes of surface intersecting, merging and re-parameterization are taken care of inside the CAD system. [0240] No matter what kind of method is used to compute the swept volumes, analytical or numerical or the method discussed above, the first task is to obtain the trajectory of the part as it moves in the space. [0241] When the user moves the part with his/her hands in the 3D virtual space of the VAE system, the invention determines the trajectory of the part during the motion. The volume of the part occupied in a certain time is called an instance. Actually, it is the combination of the part geometry, part position, and part orientation. The whole swept volume is the union of the all the instances. The user is given a choice whether he/she wants to create the swept volume while the part is moving held in his/her right hand. All the actions below will be effective if the user chooses to create the swept volume. Once the system is in the volume creation mode, the defined volume begins whenever the user grabs the part and stops whenever the user releases the part. [0242] Referring to the scene graph in FIG. 9, in every frame, the invention calculates the transformation matrix of the part in the global DCS using Equations 4.1.1, 4.1.2 and 4.1.3. [partLocationXform]=[part — matrix]×[palm — matrix]×[hand — matrix] (4.1.1) (geometry of part in global)=(geometry of part)×[partLocationXform] (4.1.2) (geometry of base in global)=(geometry of base)×[baseLocationXform] (4.1.3) [0243] where [partLocationXform]-transformation from part DCS to the global DCS [0244] [part_matrix]-transformation from part DCS to palm DCS [0245] [palm_matrix]-transformation from palm DCS to hand DCS [0246] [hand_matrix]-transformation from the hand DCS to global DCS [0247] [baseLocationXform]-transformation from base part DCS to the global DCS [0248] In theory, where the swept volume is created makes no difference since the swept volume will form a whole boundary and can be moved to anywhere. If no environment evaluation is performed, the swept volume is created in the base part DCS. Equation 4.2 is used to calculate the transformation of the part in base part DCS. [partInBaseLocationXform]=[part — matrix]×[palm — matrix]×[hand — matrix]×[BaseLocationXform] −1 (4.2) [0249] A matrix array T is declared that can hold the transformation matrices for every instance. Also, the instance number is stored before the user stops the swept volume. For every instance, the part geometry is copied and transformed using [partInBaseLocationXform]. So if the base part moves, all the instances will move with it. The reason for the need to copy the geometry of the part is because the instances are picked individually and independently. Otherwise, the instances can be displayed by referring to the same geometry, but they can not be picked separately. [0250] The trajectory represented by T is time independent since the trajectory totally depends on the transformation matrices. This is a very useful property is discussed in greater detail below. [0251] Swept Volume generation is usually not a real time process. Sometimes, it may be time consuming. One question is what happens if the user is not satisfied with a swept volume after it is created? ( e.g. maybe the user wants to take a different trajectory path.) The invention provides for real time swept volume editing functionality before the swept volume is created from all the instances. The editing functionality includes removal and modification. [0252] If the user does not care about the information or the shape of the swept volume between two instances, the in-between instances can be removed. The removal of one or more instances may change the swept volume greatly. [0253] In order to let the user pick the instances by his/her virtual fingers, the finger positions are computed relative to the swept volume and the invention is aware when the swept volume is moving with the base. The calculation of the positions or the fingers in the global space is relatively simple when the virtual hand model is fully dexterous. For simplicity, the position and orientation of the finger tip in the virtual hand DCS is represented by [fingerInHandXform]. [0254] Equations 4.3.1 and 4.3.2 are employed to bring the fingers and the swept volume to the global DCS so that they can be compared in the same coordinate system. In one embodiment, the invention employs a built in intersection check mechanism in the graphical environment facility to create some line segments on each finger tip on the user's right hand and call the intersection function to perform the intersection of the line segments with the geometry that was copied for the instances. This symbolizes the finger picking. [fingertipxform]=[fingerInHandXform]×[hand matrix] (4.3.1) [fingertipInBaseXform]=[fingertipxform]×[BaseLocationxform] −1 (4.3.2) [0255] Once the instance is picked, its geometry is removed from the scene graph. Also the matrices array T is updated. A flowchart 242 of the removal mode process is shown in FIG. 44. The symbol “NoI” means the total number of instances. [0256] Besides instance removal, the invention also provides for instance modification functionality. This allows the user to change the shape of the swept volume by changing the position and orientation of the instances. It is kind of a “refining process”. In many cases, the user may not want to move the instance in a large step, so the invention lets the instance translate and rotate in its own coordinate system. The invention makes a transformation matrix called [modifyXform]. All the translation and rotation in its own coordinate system are concatenated to this matrix. Suppose the transform matrix before the modification is [locationxform] ( in global DCS or in base DCS ), then Equation 4.4 is used to get the new matrix. [newLocationXform]=[modifyXform]×[locationXform] (4.4) [0257] The [newLocationXform] is copied into the trajectory array T. In order to assist the modification process, we use the highlighting feature to clearly indicate the picked instances. In addition, three line segments are created to represent a physical coordinate system and will be displayed on the instance when the user picks an instance. The user can easily see where the translation and rotation is going to be performed. [0258] In some cases, translation and rotation may be not convenient if the user wants to move some instances freely. It is easier sometimes to position and orient an instance directly by one's hands. It may not be practical to grab the instance and move it around since all the instances are close to each other and it is difficult to grab a certain instance. [0259] However, the invention can still use a virtual finger tip to pick an instance. After the instance is picked, this instance is attached to the right hand DCS. When the instance is finally placed in a certain position, global transformation matrix for the instance is calculated. The new matrix can be computed by equations 4.5.1 and 4.5.2. In the equations, [instanceInHand] represents the transformation between the instance and the hand at the time when the instance is picked. [instanceMatrix] is the global matrix of the instance before it is picked. [instanceInBase] is the new matrix of the instance in base DCS. Also notice that [hand_matrix] in the two equations are obtained at different time. [instanceInHand]=[instanceMatrix]×[hand — matrix] −1 (4.5.1) [instanceInBase]=[instanceInHand]×[hand — matrix]×[baseLocationXform] −1 (4.5.2) [0260] One interesting problem is how to tell the system to stop the movement of the instance. Within the virtual environment, the primary interaction actor is the user's right hand since the left hand is holding a base part and the invention lets the fingers carry out this task. Because all of the fingers are dexterous and one finger can be used to pick the instance, the invention can use the distance between some other fingers to indicate the command. When the instance is picked, two fingers are moved close to each other, i.e., let the distance between the two fingertips be smaller than some predefined value. The distance between two fingers is calculated while the instance is picked and moved around. If the user opens those two fingers, i.e., the distance is greater than a certain value, the movement of the instance is stopped and the new matrix is calculated using equations 4.5.1 and 4.5.2. [0261] This interaction method provides an additional interaction method in the virtual environment. Usually 3D GUI picking is not very efficient when the user needs to send a quick command. And in some cases, both the user's hands may be busy. Using the fingertip position testing can be used to generate some simple yes/no, start/stop commands and the implementation of the interaction is easy. [0262] A flowchart 246 of the instance modification process is shown in FIG. 45. Using the instance removal, and modification, a swept volume is created. In this way, the evaluation is almost done before the swept volume is created. [0263] After the swept volume is created, the invention can load it back into the VAE system in the position where it is created. For convenience, the transformation matrix for the first instance to represent the transformation of the swept volume is stored. The created swept volume behaves as a new part in the assembly model. The user can now perform the swept volume related evaluation tasks in the virtual environment. [0264] One interesting combination is the creation of a swept volume while the part is undergoing the constrained motion. Another byproduct of the swept volume editing is the real time part design. Since the invention can modify the shape before the volume is created, it can be used as a design tool. Very sophisticated parts can be created out of very simple geometry. [0265] Additionally, the created swept volume can be a reference to the subsequent assembly operations. As discussed above, the invention enables a complex assembly to be studied. For some critical parts, its path is reserved by the representation of the swept volume, which means that when other parts are assembled, they should not interfere with the swept volume. For example, if the assembly of an engine is being studied, it is well known that the spark plugs are the parts that need to be replaced sometime and it is important to make sure that their trajectory path remains clear. In this case, a user could sweep a spark plug in the assembly, edit it till the required path is known, then create the swept volume and load it back. The spark plug swept volume would be left on the assembly when performing the assembling process for other parts. If no parts cut the spark plug's swept volume at their final position, then the user would know for sure that the maintainability of the engine for this item is guaranteed. [0266] Definitely the collision detection plays an important role in the invention. The interference check is done accurately using the geometry data instead of just visually. The collision detection makes it possible that every operation will be valid. Real time collision detection is been included in the Invention. The combination usage of swept volume and collision detection is also a powerful feature. For example, if a part is swept along certain paths and checked for collision between the part and other parts, and if collision occurs, the user can clearly find the positions or locations of the interference. PARAMETRIC DESIGN MODIFICATION [0267] In one embodiment, the invention employs a parametric CAD system (Pro/Engineer™) and a developer's toolkit that provides access to its database: ProDevelop™ (or ProToolkit™). Hereinafter, ProE and ProD, respectively. ProD is a programming interface to ProE that allows developers to directly access to the database of ProE to perform unique and specialized engineering analysis, drive automated manufacturing, and integrate proprietary applications with ProE. [0268] Interactively, when the user wants to modify the design models, he/she just selects the “Modify” button and the dimensions of the selected feature show up. The user needs to pick the dimensions he/she wants to modify, enter new values, and then ask the system to “Regenerate” the part. The model is updated according to the modified values. This becomes difficult if the task needs to be performed non-interactively. The invention enters the database of ProE through ProDevelop™, finds the dimensions of the selected part, changes the part values to the dimensions that the user wants to modify, sends the changed values to the CAD system and lets the system regenerate the part. A flowchart 246 of a process for modifying dimensions of a part is shown in FIG. 46. This figure shows a logical overview for design changes of a part within ProE. [0269] In the virtual assembly environment, the user can pick a part, recognize the part and assemble the part. When the user wants to perform some design modification to a part, the invention starts the ProD application program, tells the ProD application the part we want modify, asks ProD to go into ProE database to find the part, extracts the dimensions, sends the dimensions to virtual environment, does the changes in the virtual environment, sends the changed dimensions back to ProD, asks ProD to update the part and reloads the part into the virtual environment, as shown in a flowchart 248 in FIG. 47. In this figure, the VAE system and ProE operate separately during the design modification process. [0270] The first problem of this method is that the virtual system can hang during the design process since it will take several minutes to just start a ProE session. Also, it will take some time to search the ProE database to find the part and extract the dimensions. Therefore, to accelerate the process, the time to start ProD should be eliminated and the time for searching the database should be reduced. To accomplish these goals, the ProD application process should be running in parallel with the invention. [0271] When two processes are running in parallel, it is natural to use signal handling methods and functions to set up communications between the two processes. In FIG. 48, a schematic overview 250 is shown for parallel operation of ProD and the VAE system. In this figure, the dashed arrows mean signal sending. This parallel method is much better than the non-parallel method since a bi-directional connection is established. However, it also has some problems. First, there is too much signal handling. Please note that ProE is also running in parallel with ProD and there are lots of signal communications between them. Second, although signal handling will not slow down the VAE system, it is not clear where the invention is when a signal comes from ProD. In this situation, it will be difficult to determine when and how to continue the design process. Thirdly, it is not easy to use the same ProD session for several different VAE sessions (starting a ProE session needs several minutes). Therefore, this integration architecture can be improved in several ways. [0272] One improvement is to reduce the signal handling between the VAE system and the ProD. If several sessions of the VAE system use the same session of ProD, the invention can know the ProD process and it is not necessary for ProD to know the different VAE sessions. Secondly, once the “Design Mode” in VADE is selected, the status of the information processing and supply from ProD is checked before anything else is executed. This requires that the VAE system knows the status information directly in ProD all of the time, not just when a signal arrives. However, ProD also needs to know the data processing in the VAE system. A data sharing requirement between the two processes leads to a method of using shared memory provided by the operating systems. [0273] The status flags of the two VAE and ProD systems are placed into shared memory, which is initialized by ProD. The data structure of the shared memory is defined as: struct CommunicationData{ int ProD_PID; /*holding the pid of ProD*/ int requestDimension; /*ask ProD to get dimensions*/ int dimensionReady; /*tell VADE dimension ready*/ int dimensionChanged; /*tell ProD dimension modified*/ int partRegenerated; /*tell VADE part regenerated */ char *partName; /*holding the part name */ }; [0274] In the data structure, ProD_PID holds the process id of ProD. Since the shared memory is initialized by a ProD process, this value can be obtained by a simple system call. One reason to set a PID is that design modification is just one feature of the VAE system and it can communicate with ProD when a user wants to perform design modifications. At that time, the VAE system sends a signal to ProD and starts the communication. Here, one signal from the VAE system to ProD is needed. In FIG. 49, a schematic overview is shown for the VAE system and ProD using shared memory. This figure illustrates a parallel method of design modification in the VAE system through the CAD system using a shared memory that employs one signal between the VAE system and ProD. [0275] The pseudo-code of “checking, setting, processing” on the VAE system side is illustrated in FIG. 50 and the pseudo-code of “checking, setting, processing” on ProD side is shown in FIG. 51. Please note that the “requestDimension” flag is set when the user picks a part and indicates he/she wants to modify the part. [0276] In the pseudo-code, the flags are set up and checked in different processes. “requestDimensions” and “dimensionChanged” are set in the VAE system and checked in ProD, while “dimensionReady” and “partRegenerated” are set in ProD and checked in the VAE system. This procedure makes sure that the flags are harmonized and the VAE system and ProD know the status of each other all of the time. [0277] Usually, even a simple part has many dimensions. In the VAE sytem, in most cases, the user only wants to modify several key dimensions. So a dimension “filtering” mechanism is set up. In the part design session, the dimensions are identified that the users are allowed to change and they are named using a predefined convention. The method used to name the dimensions that are going to be modified start with “vade_”. For instance, “vade_length”, “vade_diameter”, “vade_outerdia”, etc. When looping through the dimensions, the names of the dimensions are checked first and a dimension is extracted if its name follows this convention. [0278] The interaction between the user and the graphics system is through a “3D Gui”, a 3D graphical user interface library. The Gui can display buttons, tiles, and messages, and also can handle the selections of the buttons. In the VAE system, the “3D Gui” displays the dimensions and the user selects the dimensions. However, the user can input modified values from the keyboard because entering floating numbers from a “3D Gui” is not always practical. [0279] Although the mechanism discussed above is fast enough to perform design modifications, there are still cases where the user wants to go back to stages before the design changes if he/she is not satisfied with the modification results. So an “undo change” mechanism is provided. Before a newly modified part is loaded into the VAE system, the old part is backed up. If the user wants to “undo change”, the invention switches back to the old saved part. If the user wants to keep the new part, then the old part is deleted. [0280] The achievement of fast design modifications in virtual environment through the CAD system can greatly enhance the viability and functionality of the applications of virtual reality technology in design and manufacturing. It proves that virtual reality can go beyond the functionality of just being a visualization tool and serve an important role in the integration and extension of CAD systems. FINGER TWIRLING [0281] The model for the virtual hand can be enhanced for more accurate performance and simplified incorporation of haptic feedback. For example, the simulated skin of the virtual hand can be improved as the number of sensors around the fingers of the CYBERGLOVE are increased. Instead of checking for the release status of a gripped part, every frame, the gripped part is checked for gripping conditions again so that an improperly gripped part would be dropped. Also, a part can be twirled if it is gripped by the same two fingers as in the two previous frames. Basic mechanical principles are applied to determine the amount of twirl based on finger movement. [0282] The virtual skin on the fingers of the virtual hand are simulated through sensors which are line segments attached to the fingers. For each frame, the endpoints of these line segments are used for intersection traversal. Twenty four line segments are used for each finger in three circles of nine, equispaced. Five sensors are set up on the palm to enable the gripping of parts between the fingers and the palm. The two point gripping method is used to decide the gripping status of the parts. Since the number of sensors has increased, the skill level for a user is reduced which prevents the parts to be gripped unrealistically. To prevent parts from being grabbed on the rear side of the hand, the sensor pair is checked if it forms a feasible combination for fair gripping before evaluating the two point gripping method. [0283] The gripping status of a grabbed part is checked every frame. In the other model discussed above, a part once gripped would is made a child object of the hand. This resulted in the part following the wrist translational and rotational motions. The part would not be available for intersection traversal as it moved from the global DCS and was made the child of the palm DCS. This prevented the gripping status of the gripped part to be checked every frame. [0284] Twirling involves the manipulations of a part usually using mainly finger movements. This functionality is important to define a hand model as dexterous. Twirling is accomplished in two steps. First, a part is grabbed and in the second step it is twirled by the finger movements. The gripping status of a part is recorded and checked by the InteractionManager discussed above and the functions of the hand class are called when the part is twirled. [0285] In FIG. 52, a flow chart 200 illustrates the twirl process for the hand model. Moving from the start block, the logic advances to a block 202 where sensor data is retrieved from a CYBERGLOVE and a FLOCK OF BIRDS virtual reality device. The logic flows to a decision block 204 where a determination as to whether an intersection with a part is detected. If false, the logic moves to a block 220 where the scene is updated in the VAE system and then the logic steps to an end block and terminates. However, if the determination at the decision block 204 is true, the logic advances to a decision block 206. A determination is made as whether the user is attempting to grip the part. If false, the logic moves to the block 220 and repeats substantially the same actions discussed above. But, if the determination is true at the block 206, the logic moves to a block 208 where the part is grabbed by the virtual fingers of the virtual hand. [0286] At the decision block 210, a determination is made as to whether the CYBERGLOVE sensors are gripping the part. If false, the logic moves to the block 220 and repeats substantially the same logic discussed above. Else, the logic moves to a block 212 where the current transform of the gripped part is determined. The logic advances to a block 214 where the twirl transform is calculated based on the finger movements of the user of the CYBERGLOVE. Next, the logic steps to a block 216 where the twirl matrix is premultiplied by the part matrix. The logic advances to a block 218 where the current sensor data from the CYBERGLOVE is stored as previous sensor data. The logic flows to a block 220 and updates the scene in the VAE system. Lastly, the logic moves to the end block and terminates. [0287] Additionally, in FIG. 53, a scene graph 183 of the dynamic coordinate systems (DCS) for twirling a virtual part with virtual fingers in the VAE system is illustrated, which is similar to the other scene graphs discussed above. However, in this case, the part DCS 178 is under a finger DCS, which is directly under the palm DCS 182. Also, the palm DCS 182 is under the hand DCS 184 is directly under the global DCS 186. [0288] [0288]FIG. 54 illustrates a schematic overview 222 of finger locations on a part for twirling. Initially, a first finger gripping point and a second finger gripping point are disposed at A1 and B1, respectively, on a part 223. After twirling, the new gripping points of the first finger and the second finger are A2 and B2, respectively. The angle between the initial gripping points and the second gripping points is represented by θ. [0289] Consider the two gripping points {right arrow over (A)}1 and {right arrow over (B)}1 in frame “n.” In frame “n+1”, the two gripping points occupy the positions {right arrow over (A)}2 and {right arrow over (B)}2 respectively. The axis of rotation of the part passes through the origin in the direction of {right arrow over (θ)}. This axis {right arrow over (θ)} can be calculated using the following relation {right arrow over (B)} 2 −{right arrow over (B)} 1=({right arrow over (A)} 2 −{right arrow over (A)} 1)+{right arrow over (θ)}×({right arrow over (B)} 1 −{right arrow over (A)} 1) (5.1) [0290] The three components of rotation are given by the following equations θ -> z = ( θ -> x × BA  z ) + R -> y BA  x  ( 5.2 ) θ -> y = ( θ -> x × BA  y ) - R -> z BA  x  ( 5.3 ) [0291] Assuming there is no slip, ({right arrow over (B)} 1 −{right arrow over (A)} 1)×{right arrow over (θ)}=0 (5.4) [0292] We obtain {right arrow over (θ)}x θ -> x = ( R -> z × BA  y ) - ( R -> y × BA  z ) BA  x 2 + BA  y 2 + BA  z 2  ( 5.5 ) [0293] Where, {right arrow over (R)}=({right arrow over (B)} 2 −{right arrow over (B)} 1)−({right arrow over (A)} 2 −{right arrow over (A)} 1) (5.6) {right arrow over (BA)}=({right arrow over (B)} 1 −{right arrow over (A)} 1) (5.7) [0294] The angle of rotation {right arrow over (θ)} of the part is given by the relation |{right arrow over (θ)}|={square root}{square root over (θi 2+θj 2+θz 2)} (5.8) [0295] The translation of the part {right arrow over (T)} is approximated to the average of the difference of the positions of points {right arrow over (a)} and {right arrow over (b)} as the fingers gripping the object move in opposite direction approximately by the same distance T -> = ( B -> 2 - B -> 1 ) + ( A -> 2 - A -> 1 ) 2  ( 5.9 ) [0296] All the above calculations are done in the part's co-ordinate system. The translation and rotation matrices are premultiplied to the current part matrix in the Global space. [0297] [0297]FIG. 55 illustrates a system for a client 10 comprising components of a computer suitable for executing an application program embodying the present invention. In FIG. 55, a processor 12 is coupled bi-directionally to a memory 14 that encompasses read only memory (ROM) and random access memory (RAM). ROM is typically used for storing processor specific machine code necessary to bootup the computer comprising client 10, to enable input and output functions, and to carry out other basic aspects of its operation. Prior to running any application program, the machine language code comprising the program is loaded into RAM within memory 14 and then executed by processor 12. Processor 12 is coupled to a display 16 on which the visualization of the HTML response discussed above is presented to a user. Often, programs and data are retained in a nonvolatile memory media that may be accessed by a compact disk-read only memory (CD-ROM) drive, compact disk-read/write memory (CD-R/W) drive, optical drive, digital versatile disc (DVD) drive, hard drive, tape drive and floppy disk drive, all generally indicated by reference numeral 18 in FIG. 55. A network interface 22 couples the processor 12 to a wide area network such as the Internet. [0298] As noted above, the invention can be distributed for use on the computer system for the client 10 as machine instructions stored on a memory media such as a floppy disk 24 that is read by the floppy disk drive. The program would then typically be stored on the hard drive so that when the user elects to execute the application program to carry out the present invention, the machine instructions can readily be loaded into memory 14. Control of the computer and selection of options and input of data are implemented using input devices 20, which typically comprise a keyboard and a pointing device such as a mouse (neither separately shown). Further details of system for the client 10 and of the computer comprising it are not illustrated, since they are generally well known to those of ordinary skill in the art. [0299] As described in detail above, the invention presents a complete scenario for assembly design. Multiple parts can be manipulated efficiently for assembly evaluations. Constrained motion simulation and dynamic simulation assist the assembly evaluation operation. The overall process is simulated realistically mimicking the physical assembly processes. Dynamic behaviors of objects in the virtual environment are implemented using physical laws and increases realistic feeling. [0300] Interactive editing of assembly path and swept volume directly by the user is achieved in the virtual environment. The editing includes swept instance addition, removal, and modifications of positions and orientations. The editing of the swept volume before the assembly geometry is finalized ensures the validity and significance of the swept volume. The swept volume is also converted to a parametric model and loaded back into the CAD system for further evaluation. Collision detection functionality is also provided in the VAE system. [0301] Bi-directional interaction is achieved between the VAE and CAD systems. For relatively simple parts, the interaction cycle is real-time. For sophisticated parts with many dimensions, the interaction speed may be slower. However, with more powerful computers, real time interaction could be achieved with even the most complex parts. [0302] Test cases have been carried out with models from industry. Results from the invention compare very well with results from the Boothroyd methodology (which is widely used in industry) for predicting assembly time. [0303] A significant deviation from reality occurs in the process of gripping the part. This occurs primarily from the “sluggishness” of VR systems created by tracking frequency, tracking latency, frame rates, and graphics latency. This sluggishness does not seem to affect gross motor movements (moving a part into place and aligning it) except in acute situations with large data bases. However, it significantly affects fine motor movements (e.g. finger and wrist movements). [0304] While the preferred embodiment of the invention has been illustrated and described, it will be appreciated that various changes can be made therein without departing from the spirit and scope of the invention. Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS4988981 *Feb 28, 1989Jan 29, 1991Vpl Research, Inc.Computer data entry and manipulation apparatus and methodUS5513303 *Aug 5, 1994Apr 30, 1996Xerox CorporationMoving an object in a three-dimensional workspaceUS5999185 *Mar 30, 1993Dec 7, 1999Kabushiki Kaisha ToshibaVirtual reality control using image, model and control data to manipulate interactionsUS6091410 *Nov 26, 1997Jul 18, 2000International Business Machines CorporationAvatar pointing mode* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS6629093 *Jan 31, 2001Sep 30, 2003Autodesk, Inc.Method and apparatus for simplified computer aided design (CAD) model search and retrievalUS6647306 *Mar 7, 2001Nov 11, 2003Daimlerchrysler CorporationInterference removal system for automated path planningUS6677943 *Nov 27, 2000Jan 13, 2004Autodesk, Inc.Method and apparatus for simplified thin body creationUS6825856 *Jul 26, 2000Nov 30, 2004Agilent Technologies, Inc.Method and apparatus for extracting measurement information and setting specifications using three dimensional visualizationUS6826500 *Jun 29, 2001Nov 30, 2004General Electric CompanyMethod and system for automated maintenance and training instruction generation and validationUS6870548 *Sep 4, 2001Mar 22, 2005Mtu Aero Engines GmbhMethod for modifying the design of a structural partUS7139685 *Nov 2, 2001Nov 21, 2006Siemens AktiengesellschaftVideo-supported planning of equipment installation and/or room designUS7149677Oct 30, 2001Dec 12, 2006Translation Technologies, Inc.Geometric model comparator and methodUS7155375 *Feb 8, 2002Dec 26, 2006ImpactxoftMethod and system for designing objects using design intent mergeUS7158112 *Aug 22, 2001Jan 2, 2007Immersion CorporationInteractions between simulated objects with force feedbackUS7171344 *Dec 21, 2001Jan 30, 2007Caterpillar IncMethod and system for providing end-user visualizationUS7203634Oct 30, 2001Apr 10, 2007Translation Technologies, Inc.Computational geometry system, interrupt interface, and methodUS7216011 *Mar 18, 2005May 8, 2007Daimlerchrysler CorporationConcurrent modeling technique for a part and its toolingUS7292964 *Dec 22, 2003Nov 6, 2007The Mathworks, Inc.Translating of geometric models into block diagram modelsUS7319941 *Dec 22, 2003Jan 15, 2008The Mathworks, Inc.Translating mates in geometric models into joint blocks in block diagram modelsUS7403833 *Apr 3, 2006Jul 22, 2008Stratasys, Inc.Method for optimizing spatial orientations of computer-aided design modelsUS7508910Apr 25, 2007Mar 24, 2009The Boeing CompanySystem and methods for x-ray backscatter reverse engineering of structuresUS7529343May 3, 2007May 5, 2009The Boeing CompanySystem and method for improved field of view X-ray imaging using a non-stationary anodeUS7599820 *Jun 23, 2005Oct 6, 2009Autodesk, Inc.Graphical user interface for interactive construction of typical cross-section frameworksUS7613594 *Dec 27, 2007Nov 3, 2009Dassault SystemesMethod and computer program product of computer aided design of a product comprising a set of constrained objectsUS7623626Feb 6, 2009Nov 24, 2009The Boeing CompanySystem and methods for x-ray backscatter reverse engineering of structuresUS7649976 *Feb 10, 2006Jan 19, 2010The Boeing CompanySystem and method for determining dimensions of structures/systems for designing modifications to the structures/systemsUS7650260Feb 8, 2002Jan 19, 2010Impact XoftMethod and system for designing objects using functional object representationUS7698016Feb 17, 2004Apr 13, 2010Tti Acquisition CorporationFeature-based translation system and methodUS7756321 *Feb 28, 2007Jul 13, 2010The Boeing CompanyMethod for fitting part assembliesUS7783460Aug 20, 2007Aug 24, 2010The Mathworks, Inc.Translating of geometric models into block diagramsUS7933858 *Mar 23, 2007Apr 26, 2011Autodesk, Inc.General framework for graphical simulationsUS7937359 *Apr 27, 2009May 3, 2011Nvidia CorporationMethod of operation for parallel LCP solverUS7945431Jan 11, 2008May 17, 2011The Mathworks, Inc.Translating mates in geometric models into joint blocks in block diagram modelsUS7979251 *Mar 16, 2007Jul 12, 2011Lego A/SAutomatic generation of building instructions for building element modelsUS8009171 *Mar 21, 2007Aug 30, 2011Sony CorporationImage processing apparatus and method, and programUS8103484 *May 20, 2008Jan 24, 2012Archi.Con.Des Inventions (Uk) LimitedMethod and apparatus for computer-aided design of three-dimensional objects to be fabricatedUS8112255 *May 20, 2008Feb 7, 2012Archi.Con.Des. Inventions (UK) LimitedMethod and apparatus for computer-aided design of three-dimensional objects to be fabricatedUS8112256 *May 20, 2008Feb 7, 2012Archi.Con.Des Inventions (Uk) LimitedMethod and apparatus for computer-aided design of three-dimensional objects to be fabricatedUS8117011 *May 20, 2008Feb 14, 2012Archi.Con.Des Inventions (Uk) LimitedComputer-aided design of three-dimensional objects to be fabricatedUS8175927 *Jun 17, 2008May 8, 2012Thomas T BuzzellE-commerce based method and system for manufacturer hosting of virtual dealer stores and method for providing a systemization of machine partsUS8203556 *Dec 17, 2007Jun 19, 2012Ricoh Company, Ltd.System and method for generating parts catalog, and computer program productUS8217995Jan 17, 2009Jul 10, 2012Lockheed Martin CorporationProviding a collaborative immersive environment using a spherical camera and motion captureUS8280701Aug 9, 2011Oct 2, 2012Archi con des Inventions (UK) LimitedMethod and apparatus for computer-aided design of three-dimensional objects to be fabricatedUS8374829Sep 15, 2009Feb 12, 2013Lego A/SAutomatic generation of building instructions for building element modelsUS8380465Dec 29, 2011Feb 19, 2013Archi.Con.Des Inventions (Uk) LimitedMethod and apparatus for computer-aided design of three-dimensional objects to be fabricatedUS8396869 *Jan 4, 2008Mar 12, 2013International Business Machines CorporationMethod and system for analyzing capabilities of an entityUS8433432 *Feb 4, 2010Apr 30, 2013International Business Machines CorporationApparatus and method for supporting creation of assembly dataUS8532966Oct 27, 2009Sep 10, 2013The Mathworks, Inc.Translating of geometric models into block diagram modelsUS8571840 *Jun 12, 2007Oct 29, 2013Autodesk, Inc.Constraint reduction for dynamic simulationUS8588955 *Jun 26, 2009Nov 19, 2013Robert Bosch GmbhMethod and apparatus for optimizing, monitoring, or analyzing a processUS8615383Jan 17, 2009Dec 24, 2013Lockheed Martin CorporationImmersive collaborative environment using motion capture, head mounted display, and caveUS8624924Jan 17, 2009Jan 7, 2014Lockheed Martin CorporationPortable immersive environment using motion capture and head mounted displayUS8700791Oct 18, 2006Apr 15, 2014Immersion CorporationSynchronization of haptic effect data in a media transport streamUS8738458Apr 9, 2012May 27, 2014Caterpillar Inc.E-commerce based method and system for manufacturer hosting of virtual dealer stores and method for providing a systemization of machine partsUS8762877 *Feb 13, 2009Jun 24, 2014Ice Edge Business Solutions Ltd.Creation and modification of valid functional design layoutsUS9008836 *Jan 7, 2008Apr 14, 2015Abb Inc.Method and system for robotic assembly parameter optimizationUS9030492 *Feb 25, 2006May 12, 2015Kuka Roboter GmbhMethod and device for determining optical overlaps with AR objectsUS9367950 *Jun 26, 2014Jun 14, 2016IrisVR, Inc.Providing virtual reality experiences based on three-dimensional designs produced using three-dimensional design softwareUS9390534 *Apr 13, 2012Jul 12, 2016Hitachi, Ltd.Computer system and assembly animation generation methodUS9424378 *Feb 3, 2014Aug 23, 2016Siemens Product Lifecycle Management Software Inc.Simulation using coupling constraintsUS9514434 *Jan 6, 2009Dec 6, 2016The Boeing CompanyApparatus and method for automatic work instruction generationUS9558676 *Sep 14, 2010Jan 31, 2017Centre National De La Recherche Scientifique (C.N.R.S.)Method for simulating specific movements by haptic feedback, and device implementing the methodUS20020107674 *Nov 2, 2001Aug 8, 2002Benedicte BascleVideo-supported planning of equipment installation and/or room designUS20020120920 *Oct 30, 2001Aug 29, 2002Sankar JayaramComputational geometry system, interrupt interface, and methodUS20030078682 *Oct 11, 2002Apr 24, 2003Nobuhiko TezukaSimulation apparatus and simulation methodUS20030120472 *Dec 21, 2001Jun 26, 2003Caterpillar Inc.Method and system for providing end-user visualizationUS20030135846 *Oct 30, 2001Jul 17, 2003Sankar JayaramGeometric model comparator and methodUS20040004624 *Sep 4, 2001Jan 8, 2004Johannes BarnerMethod for modifying the design of a structural partUS20040199367 *Apr 20, 2004Oct 7, 2004Koichi KondoApparatus and method for obtaining shape data of analytic surface approximate expressionUS20050046624 *Feb 17, 2004Mar 3, 2005Sankar JayaramFeature-based translation system and methodUS20050251494 *Jul 14, 2003Nov 10, 2005Inter-Technology Crystal N.V.System for access to, exchange of information relating to, analysis and design of industrial plants with a substantial complexityUS20060028476 *Aug 3, 2004Feb 9, 2006Irwin SobelMethod and system for providing extensive coverage of an object using virtual camerasUS20060155402 *Nov 20, 2002Jul 13, 2006Dale Read3d virtual manufacturing processUS20060212160 *Mar 18, 2005Sep 21, 2006Philippe BriseboisConcurrent modeling technique for a part and its toolingUS20060212821 *Dec 19, 2005Sep 21, 2006Bernard CharlesProduct edition and simulation database system with user interactive graphical toolUS20070083280 *Oct 6, 2005Apr 12, 2007Timothy StumpfMethod and system for three dimensional work instructions for modification processesUS20070160961 *Jan 11, 2007Jul 12, 2007Cyrus LumTransportation simulatorUS20070182759 *Jul 17, 2006Aug 9, 2007Fujitsu LimitedGraphics processing apparatus, method and program storage medium thereofUS20070189454 *Feb 10, 2006Aug 16, 2007Georgeson Gary ESystem and method for determining dimensions of structures/systems for designing modifications to the structures/systemsUS20070233298 *Apr 3, 2006Oct 4, 2007Stratasys, Inc.Method for optimizing spatial orientations of computer-aided design modelsUS20070269006 *Apr 25, 2007Nov 22, 2007Morteza SafaiSystem and methods for x-ray backscatter reverse engineering of structuresUS20070269014 *May 3, 2007Nov 22, 2007Morteza SafaiSystem and method for improved field of view x-ray imaging using a non-stationary anodeUS20070299642 *Jun 18, 2007Dec 27, 2007Kabushiki Kaisha ToshibaApparatus and method for verifying control program through simulationUS20080012864 *Mar 21, 2007Jan 17, 2008Teruyuki NakahashiImage Processing Apparatus and Method, and ProgramUS20080027684 *Jun 23, 2005Jan 31, 2008Autodesk, Inc.Graphical user interface for interactive construction of typical cross-section frameworksUS20080027691 *Dec 4, 2006Jan 31, 2008Fujitsu LimitedDevice manufacturing support apparatus, simulation method for device manufacturing support apparatus, and device manufacturing apparatusUS20080150965 *Feb 25, 2006Jun 26, 2008Kuka Roboter GmbhMethod and Device For Determining Optical Overlaps With Ar ObjectsUS20080162091 *Dec 27, 2007Jul 3, 2008Emmanuel LechineMethod and Computer Program Product of Computer Aided Design of a Product Comprising a Set of Constrained ObjectsUS20080165189 *Dec 21, 2007Jul 10, 2008Toyota Jidosha Kabushiki KaishaMethod and system for automatically generating process animationsUS20080170070 *Dec 17, 2007Jul 17, 2008Junichi YamagataSystem and method for generating parts catalog, and computer program productUS20080172208 *Dec 27, 2007Jul 17, 2008Dassault SystemsMethod and computer program product of computer aided design of a product comprising a set of constrained objectsUS20080174598 *Jan 14, 2008Jul 24, 2008Max RisenhooverDesign visualization system, apparatus, article and methodUS20080205763 *Feb 28, 2007Aug 28, 2008The Boeing CompanyMethod for fitting part assembliesUS20080223627 *Oct 18, 2006Sep 18, 2008Immersion Corporation, A Delaware CorporationSynchronization of haptic effect data in a media transport streamUS20080228450 *Mar 16, 2007Sep 18, 2008Jakob Sprogoe JakobsenAutomatic generation of building instructions for building element modelsUS20080235171 *Mar 23, 2007Sep 25, 2008Autodesk, Inc.General framework for graphical simulationsUS20080243456 *Jun 12, 2007Oct 2, 2008Autodesk, Inc.Constraint reduction for dynamic simulationUS20080262947 *Jun 17, 2008Oct 23, 2008Caterpillar Inc.E-commerce based method and system for manufacturer hosting of virtual dealer stores and method for providing a systemization of machine partsUS20080291199 *May 20, 2008Nov 27, 2008Archi.Con.Des Inventions (Uk) LimitedComputer-aided design apparatusUS20080294389 *May 20, 2008Nov 27, 2008Archi.Con.Des Inventions (Uk) LimitdComputer-aided design apparatusUS20080294390 *May 20, 2008Nov 27, 2008Archi.Con.Des Inventions (Uk) LimitedComputer-aided design apparatusUS20080294391 *May 20, 2008Nov 27, 2008Archi.Con.Des Inventions (Uk) LimitedComputer-aided design apparatusUS20080294392 *May 20, 2008Nov 27, 2008Archi.Con.Des Inventions (Uk) LimitedComputer-aided design apparatusUS20090168964 *Feb 6, 2009Jul 2, 2009Morteza SafaiSystem and methods for x-ray backscatter reverse engineering of structuresUS20090177665 *Jan 4, 2008Jul 9, 2009International Business Machines CorporationMethod and system for analyzing capabilities of an entityUS20090187389 *Jan 17, 2009Jul 23, 2009Lockheed Martin CorporationImmersive Collaborative Environment Using Motion Capture, Head Mounted Display, and CaveUS20090213114 *Jan 17, 2009Aug 27, 2009Lockheed Martin CorporationPortable Immersive Environment Using Motion Capture and Head Mounted DisplayUS20090278917 *Jan 17, 2009Nov 12, 2009Lockheed Martin CorporationProviding A Collaborative Immersive Environment Using A Spherical Camera and Motion CaptureUS20090326680 *Jun 26, 2009Dec 31, 2009Robert Bosch GmbhMethod and apparatus for optimizing, monitoring, or analyzing a processUS20100175013 *Jan 6, 2009Jul 8, 2010Lance Gerard KrauterApparatus and method for automatic work instruction generationUS20100198385 *Feb 4, 2010Aug 5, 2010International Business Machines CorporationApparatus and method for supporting creation of assembly dataUS20100211204 *Jan 7, 2008Aug 19, 2010Abb Inc.Method and system for robotic assembly parameter optimizationUS20100306681 *Feb 13, 2009Dec 2, 2010Dirtt Environmental Solutions Ltd.Creation and modification of valid functional design layoutsUS20110307090 *Mar 23, 2011Dec 15, 2011Fujitsu LimitedTolerance analyzing apparatus, designing apparatus, viewer apparatus, and assembly order converting methodUS20120100520 *Sep 1, 2011Apr 26, 2012Electronics And Telecommunications Research InstituteAssembly process visualization apparatus and methodUS20120259604 *Sep 14, 2010Oct 11, 2012Centre National De La Recherche Scientifique (C.N.R.S.)Method for simulating specific movements by haptic feedback, and device implementing the methodUS20130332118 *May 29, 2013Dec 12, 2013Dassault SystemesComputer-Implemented Method For Defining Initial Conditions For Dynamic Simulation Of An Assembly Of Objects In A Three-Dimensional Scene Of A System Of Computer-Aided DesignUS20130332119 *May 30, 2013Dec 12, 2013Dassault SystemesMethod And System For Dynamically Manipulating An Assembly Of Objects In A Three-Dimensional Scene Of A System Of Computer-Aided DesignUS20140012546 *Aug 31, 2012Jan 9, 2014Siemens Product Lifecycle Management Software Inc.Ordering optional constraints in a variational systemUS20140032181 *Jul 17, 2013Jan 30, 2014Dassault SystemesDesign Operation In An Immersive Virtual EnvironmentUS20140118358 *Apr 13, 2012May 1, 2014Hitachi, Ltd.Computer system and assembly animation generation methodUS20150220666 *Feb 3, 2014Aug 6, 2015Siemens Product Lifecycle Management Software Inc.Simulation using coupling constraintsUS20150278400 *Mar 28, 2014Oct 1, 2015Siemens Product Lifecycle Management Software Inc.Hybrid variational solving in cad modelsCN101799845A *Mar 1, 2010Aug 11, 2010南京航空航天大学Method for realizing flexible cable assembling model in virtual assembling environmentCN102246139A *Apr 16, 2010Nov 16, 2011西门子公司Alternate sequence descriptions for producing alternate execution paths in an automation systemCN102640199A *Sep 14, 2010Aug 15, 2012国家科研中心Method for simulating specific movements by haptic feedback, and device implementing the methodCN102789514A *Apr 20, 2012Nov 21, 2012青岛理工大学Induction method of three-dimensional (3D) online induction system for mechanical equipment dismountingDE102005016847A1 *Apr 12, 2005Oct 19, 2006UGS Corp., PlanoThree-dimensional computer-aided design object visualization method, involves determining position of user-controlled cursor on display device and displaying view on device based on position of cursor relative to another viewDE202005001702U1 *Feb 2, 2005Jun 14, 2006Sata Farbspritztechnik Gmbh & Co.KgVirtuelles Lackiersystem und FarbspritzpistoleEP1640920A1 *Sep 25, 2003Mar 29, 2006Lattice Technology, Inc.Process animation automatic generation method and systemEP1640920A4 *Sep 25, 2003Jun 10, 2009Lattice Technology IncProcess animation automatic generation method and systemEP2419822A1 *Apr 16, 2010Feb 22, 2012Siemens AktiengesellschaftAlternate sequence descriptions for producing alternate execution paths in an automation systemEP2419822A4 *Apr 16, 2010Jun 4, 2014Siemens AgAlternate sequence descriptions for producing alternate execution paths in an automation systemWO2007047960A3 *Oct 19, 2006Jan 17, 2008Immersion CorpSynchronization of haptic effect data in a media transport streamWO2007130225A3 *Mar 15, 2007Dec 24, 2008Stratasys IncMethod for optimizing spatial orientations of computer-aided design modelsWO2008095574A2 *Jan 10, 2008Aug 14, 2008Sew-Eurodrive Gmbh & Co. KgUse of graphs, method and computer system for creating a construction drawing, method for manufacturing a product, and use of said methodWO2008095574A3 *Jan 10, 2008Apr 9, 2009Gerhard SchellUse of graphs, method and computer system for creating a construction drawing, method for manufacturing a product, and use of said methodWO2008121562A1 *Mar 19, 2008Oct 9, 2008Autodesk, Inc.Constraint reduction for dynamic simulationWO2010121104A1 *Apr 16, 2010Oct 21, 2010Siemens AktiengesellschaftSequence element replacementWO2010121109A1 *Apr 16, 2010Oct 21, 2010Siemens AktiengesellschaftAlternate sequence descriptions for producing alternate execution paths in an automation systemWO2016025811A3 *Aug 14, 2015Aug 11, 2016Daqri, LlcSpatial data visualization* Cited by examinerClassifications U.S. Classification700/98, 359/458, 345/419, 382/154, 700/95, 345/420, 700/97, 706/919, 345/475International ClassificationG06T19/00Cooperative ClassificationG06T2219/2008, G06T2210/21, G06T2219/2004, G06T19/20European ClassificationG06T19/00Legal EventsDateCodeEventDescriptionMar 1, 2002ASAssignmentOwner name: WASHINGTON STATE UNIVERSITY RESEARCH FOUNDATION, WFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:JAYARAM, SANKAR;JAYARAM, UMA;WANG, YONG;AND OTHERS;REEL/FRAME:012634/0450;SIGNING DATES FROM 20020211 TO 20020220Aug 1, 2002ASAssignmentOwner name: GOVERNMENT OF THE UNITED STATES OF AMERICA, AS REPFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LYONS, KEVIN;HART, PETER F.;REEL/FRAME:012943/0436;SIGNING DATES FROM 20020710 TO 20020722RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services