Three dimensional visualization of system components and data

In an approach to three dimensional component visualization, one or more computer processors receive a request associated with one or more reference tags received from one or more requesting devices. The one or more computer processors identify component data associated with one or more requested reference tags. The one or more computer processors analyze the one or more three dimensional renderings of one or more components using component data. The one or more computer processors determine whether to augment the one or more three dimensional renderings. The one or more computer processors create an augmented three dimensional rendering using component data. The one or more computer processors send one or more three dimensional renderings of one or more components using component data to the one or more requesting devices.

BACKGROUND OF THE INVENTION

The present invention relates generally to the field of three dimensional imaging, and more particularly to three dimensional visualization applied to systems analysis and repair.

Three dimensional (3D) visualization is any technique for creating images, diagrams, and/or animation using 3D modeling and representation to communicate a message, such as information. Visualization using 3D imaging techniques is an effective way to communication abstract and concrete ideas. For example, many technical drawings for scientific and engineering purposes are translated into 3D representations for the purposes of research, manufacturing, and general communication of information.

Visualization continues to expand into an ever increasing number of scientific, educational, engineering, interactive media, and medical applications. For example, the application of computer graphics to visualization allows engineers to engage in product visualization thereby quickening the pace of engineering applications such as manufacturing, repair, and simulation.

SUMMARY

Embodiments of the present invention disclose a method, a computer program product, and a system for three dimensional component visualization. The method may include one or more computer processors receiving a request associated with one or more reference tags received from one or more requesting devices. The one or more computer processors identify component data associated with one or more requested reference tags. The one or more computer processors analyze the one or more three dimensional renderings of one or more components using component data. The one or more computer processors determine whether to augment the one or more three dimensional renderings. The one or more computer processors create an augmented three dimensional rendering using component data. The one or more computer processors send one or more three dimensional renderings of one or more components using component data to the one or more requesting devices.

DETAILED DESCRIPTION

Present day component locating systems rely on service documentation and location codes to identify particular components in a device. More advanced component locating systems point to certain components by mechanisms such as turning on light emitting diodes on the component to be identified. Therefore, current technologies rely on active user directed inputs and participation to physically locate the correct components of a device. As such, component location systems can benefit from the integration of three dimensional (3D) visualization tied to vital product data in the field of engineering. Applying 3D visualization in component locating systems allows for significant improvements in the efficiency and accuracy of component location. Further, 3D visualization opens up possibilities in how devices are analyzed by expanding the capabilities of a user to include applications that look deeper into the mechanics of an individual component, assess how performance of a larger system changes when one component is replaced by another, and manipulate 3D representations of a device to better visualize a component in the given environment. For example, allowing a user to locate a device and learn about the product's specifications using a 3D representation rather than manually locating the components, identifying the component identification information, and locating the vital product data associated with the component simplifies the component location and identification process to passively complement a workflow rather than being a time and labor intensive endeavor. Embodiments of the present invention recognize that 3D component visualization tied to vital product data can complement present day engineering platforms by cutting steps and improving information flow in the engineering process. Implementation of embodiments of the invention may take a variety of forms, and exemplary implementation details are discussed subsequently with reference to the Figures.

Distributed data processing environment100includes client computing device104and server computer108, all interconnected over network102. Network102can be, for example, a telecommunications network, a local area network (LAN), a wide area network (WAN), such as the Internet, or a combination of the three, and can include wired, wireless, or fiber optic connections. Network102can include one or more wired and/or wireless networks that are capable of receiving and transmitting data, voice, and/or video signals, including multimedia signals that include voice, data, and video information. In general, network102can be any combination of connections and protocols that will support communications between client computing device104and server computer108, and other computing devices (not shown) within distributed data processing environment100.

Client computing device104can be a smart watch, a smart television, a laptop computer, a tablet computer, a smart phone, or any programmable electronic device capable of capturing images, displaying 3D images, and communicating with various components and devices within distributed data processing environment100, via network102. Client computing device104may receive direct input from the user via user interface106. Client computing device104may represent any programmable electronic device, pre-configured electronic device, or combination of programmable and pre-configured electronic devices capable of executing machine readable program instructions and communicating with computing devices, such as server computer108, and other computing devices (not shown) within distributed data processing environment100via a network, such as network102. In an embodiment, client computing device104may be limited to communicating with other computing devices (not shown) within distributed data processing environment100via a network, such as network102. In the depicted embodiment, client computing device104includes an instance of user interface106. In another embodiment, client computing device104does not include an instance of user interface106.

User interface106, hosted on client computing device104, provides an interface to component visualization program110on server computer108. In one embodiment, user interface106may be a graphical user interface (GUI) or a web user interface (WUI) and can display text, documents, web browser windows, user options, application interfaces, and instructions for operation, and include the information (such as graphic, text, and sound) that a program presents to a user and the control sequences the user employs to control the program. In another embodiment, user interface106may also be mobile application software that provides an interface between a user of client computing device104and server computer108. Mobile application software, or an “app,” is a computer program designed to run on smart phones, tablet computers and other mobile devices. User interface106enables the user of client computing device104to register with and configure component visualization program110to adjust preferences for 3D imaging functions, such as which one or more databases112to access, the type of data stored for each component, and user-specific authorization to use component visualization program110. Further, user interface106may enable a user to view 3D representations and vital product data of one or more components using augmented reality. By overlaying a 3D component image over an image or video captured by client computing device104, component visualization program110can dynamically assist a user through real-time 3D visualization and analysis of component issues. For example, component visualization program110may manipulate the image overlay in augmented reality to show where a particular component failed or show how an alternative component would fit in the available space. However, user interface106is not limited to the aforementioned examples and may be used to control any parameters associated with component visualization program110.

Server computer108can be a standalone computing device, a management server, a web server, a mobile computing device, or any other electronic device or computing system capable of receiving, sending, and processing data. In other embodiments, server computer108can represent a server computing system utilizing multiple computers as a server system, such as in a cloud computing environment. In another embodiment, server computer108can be a laptop computer, a tablet computer, a netbook computer, a personal computer (PC), a desktop computer, a personal digital assistant (PDA), a smart phone, or any other programmable electronic device capable of communicating with client computing device104and other computing devices (not shown) within distributed data processing environment100via network102. For example, server computer108may be a smart phone that is capable of remotely controlling and sending registration and configuration data to client computing device104. In another embodiment, server computer108represents a computing system utilizing clustered computers and components (e.g., database server computers, application server computers, etc.) that act as a single pool of seamless resources when accessed within distributed data processing environment100. Server computer108includes component visualization program110and database112. Server computer108may include internal and external hardware components, as depicted and described in further detail with respect toFIGS. 3 and 4.

Component visualization program110executes a series of steps to link component data, such as 3D representations and vital product data, to one or more reference tags received from client computing device104and communicate the component data to a user via client computing device104. Component visualization program110aims to streamline and automate much of the management of component data required to improve engineering-related functions by cutting information gathering steps and improving information flow to a user throughout the engineering process. For example, component visualization program110may save a user time and effort in assessing a component failure by organizing and linking component data of a broken component, such as 3D representation and vital product data, to a reference tag and sending the data to client computing device104, thereby allowing a user to quickly receive component data and manipulate a 3D model of the component in a larger system. In another example, component visualization program110may save a user time and effort in quality control by sending a user requested component data, such as vital product data explaining known issues with a component and 3D representations of the component that may be visually overlaid and manipulated on one or more images of a malfunctioning device, of a component that is responsible for one or more inefficiencies, so the user can quickly identify alternative components that may improve the finished product.

In yet another embodiment, component visualization program110may communicate component data associated with a faulty component to client computing device104through user interface106. For example, component visualization program110may create a 3D representation and send vital product data of a faulty component to user interface106on client computing device104which subsequently overlays the vital product data and the created 3D representation over one or more images or videos captured by client computing device104(i.e., augmented reality). In addition, component visualization program110may direct a user to a faulty component in the overall system and visually instruct the user on how to fix the issue upon receiving one or more signals via network102indicating a malfunction in one or more devices associated with one or more reference tags via user interface106on client computing device104. The malfunction associated with one or more components which are associated with one or more reference tags may be reported by a user via user interface106on client computing device104or automatically by one or more devices.

To achieve the aforementioned functions, component visualization program110receives component data from one or more databases112. Components may include any hardware present in a system. Component data may include 3D representations and vital product data of a component. Vital product data may include one or more serial numbers, one or more part numbers, historical component information, one or more location codes, and any other information associated with the component. Component data may be received for a specific component or a group of components associated with each other, such as power supplies, hard drives, random access memory, motherboards, graphics cards, and CPUs compatible with a type of computer. Component data may also be received from larger databases linked to component visualization program110that contain a large variety of component data associated with a large variety of components. Component data may be any data associated with one or more components, such as data associated with replacement, repair, enhancement, or any other application of component data. Component visualization program110stores component data in one or more databases. For example, component visualization program110may store 3D representation data in a first database and vital product data in a second database. After storing the component data, component visualization program110links the component data to one or more reference tags. For example, component visualization program110may link the 3D representation data and the vital product data to one or more reference tags associated with a component. Component visualization program110determines whether component visualization program110receives one or more requests associated with one or more reference tags.

For example, a user may manually input or scan one or more reference tags with client computing device104which are subsequently sent to component visualization program110as a request for data associated with the reference tag. If component visualization program110does not receive one or more requests associated with one or more reference tags, then component visualization program110returns to a default state of waiting to receive component data. If component visualization program110receives one or more requests associated with one or more reference tags, then component visualization program110locates the component data associated with the one or more reference tags. For example, component visualization program110may locate the 3D representation data and vital product data associated with the reference tag. Component visualization program110sends the component data to one or more requesting devices. Component visualization program110determines whether component visualization program110received subsequent component data associated with the one or more reference tags. If component visualization program110determines that component visualization program110received subsequent component data associated with the one or more reference tags, then component visualization program110stores the subsequent component data in one or more databases. If component visualization program110determines that component visualization program110did not receive subsequent component data associated with the one or more reference tags, then component visualization program110ends.

Database112is a repository for data used by component visualization program110. In the depicted embodiment, database112resides on server computer108. In another embodiment, database112may reside elsewhere within distributed data processing environment100provided component visualization program110has access to database112. Database112can be implemented with any type of storage device capable of storing data and configuration files that can be accessed and utilized by server computer108, such as a database server, a hard disk drive, or a flash memory. In some embodiments, database112may store any data that component visualization program110uses to store and locate component data. For example, database112may store parameters set by a user that give component visualization program110permission to store and locate component data associated with particular reference tags. In various embodiments, database112may store data received by component visualization program110and registration information including configuration data of component visualization program110and client computing device104. Database112may also store previously created 3D renderings of previous issues and associated vital product data associated with one or more devices. Examples of registration data include, but are not limited to, data identifying user preferences for which reference tags component visualization program110may access. For example, a user may input user preferences that do not allow component visualization program110to access particular databases containing sensitive or confidential information. However, the aforementioned examples serve to illustrate a few scenarios, and the user preferences can be in the form of any restrictions and allowances a user may input.

FIG. 2depicts operational steps for storing component data and requesting component data using reference tags, on a computing device within the computing environment ofFIG. 1, in accordance with an embodiment of the present invention.FIG. 2is a flowchart depicting operations of an instance of component visualization program110on server computer108within distributed data processing environment100. The operational steps ofFIG. 2begin when component visualization program110receives component data.FIG. 2as described herein is based on the operational steps of component visualization program110located outside of client computing device104. However, alternative embodiments and configurations may execute the operational steps of component visualization program110.

Component visualization program110receives component data. (step202). Component data may include vital product data and 3D representation data. Vital product data may include, but is not limited to, one or more serial numbers, one or more part numbers, and one or more location codes. 3D representation data allows a user to visually inspect one or more components in a 3D environment, such as a rendered 3D environment or a real 3D environment (i.e., augmented reality), and manipulate the 3D representation. The user may manipulate the viewing angles, run an animated model of one or more components in a system, and view the one or more components in an exploded view. Component visualization program110may receive vital product data and 3D representation data directly from one or more manufacturers, from accessing available materials on the internet, or from vital product data and 3D representation data inputted by one or more users. In an alternative embodiment, component visualization program may create one or more 3D renderings based on visual references, manufacturer specifications, and/or internet sources. However, a user is not limited to receiving, creating, or using 3D representation data and vital product data in the same way as illustrated aforementioned embodiments and accompanying examples.

Component visualization program110stores component data on one or more databases (step204). In an embodiment, the component data comprising both the vital product data and 3D representation data may be stored on a singular database. In another embodiment, the vital product data and 3D representation data associated with a component may be stored on one or more databases. For example, component visualization program110may store 3D representation data in a database dedicated to imaging applications and store vital product data in an information database. By separating component data across multiple databases, separate restrictions may be placed on one or more databases, such as user access restrictions, which allows for a higher degree of control over the stored data. Further, separating component data across multiple databases may allow updates to particular parts the component data which may increase efficiency by avoiding an update of the entire data file when one or more changes are only made to a part of the overall component data. In an alternative embodiment, component visualization program110may access information from existing sources, such as manufacturer databases and internet databases, in order to conserve software and hardware resources associated with data storage. For example, component visualization program110may access a manufacturer webpage containing the specifications of one or more components for direct reference.

After storing the component data, component visualization program110links the component data to one or more reference tags (step206). Reference tags associated with one or more databases containing data about one or more components allow component data to be aggregated efficiently. For example, if a 3D representation file on a first database and vital product data on a second database associated with the same component exist, then one reference tag associated with both files will allow a user to access both sets of data from separate databases using a common reference tag. In an embodiment, component visualization program110may use an Extensible Markup Language (XML) as a reference tag. In another embodiment, one component may have multiple reference tags if the component is used in multiple devices. For example, a general purpose hard drive may be used in multiple devices. Reference tags specific to each device would allow all the relevant components to a particular device to be pulled simultaneously from one or more databases. By linking components to one or more reference tags, component visualization program110would need access to significantly fewer databases rather than a database pertaining to each separate device. However, component visualization program110may use any method to tag data.

Component visualization program110receives requests from one or more client computing devices, wherein the one or more requests are associated with one or more reference tags (step208). In an embodiment, a user of client computing device104may scan a reference tag using client computing device104to send a request over network102to component visualization program110. Component visualization program110receives the request associated with the one or more reference tags. For example, a user of a smart phone may scan a barcode associated with a component in a car which is subsequently sent to component visualization program110as a request associated with a reference number. In another embodiment, a user of client computing device104may manually input a reference tag using user interface106located on client computing device104to send a request over network102to component visualization program110. As with the prior example, component visualization program110receives the request associated with the one or more reference tags. For example, a user of a smart phone may input a reference tag located on a hard drive in a main frame computer into an application on the smartphone. A request is subsequently sent to component visualization program110associated with the inputted reference tag. In yet another embodiment, a user may input multiple reference tags associated with one or more components which is then received as a request by component visualization program110. For example, a user may input reference tags for a hard drive, a CPU, and a power supply which is received as a request for each of the reference tags by component visualization program110.

Component visualization program110locates the component data associated with the one or more reference tags to create a three dimensional rendering (step210). In an embodiment, component visualization program110accesses a database containing both vital product data and 3D representation data associated with one or more reference tags. For example, component visualization program110may locate a file on a database associated with a reference tag for a hard drive containing both the 3D representation data and vital product data, such as the serial number, part number, and location code associated with the hard drive. In another embodiment, component visualization program110may access a first database containing vital product data and a second database containing 3D representation data associated with one or more reference tags. For example, component visualization program110may locate a first file on the first database containing a serial number, location code, and part number and locate a second file on the second database containing 3D representation data for a reference tag associated with a hard drive. In yet another embodiment, component visualization program110may access one or more databases to locate more than one sets of component data associated with multiple reference tags.

Following the locating of the component data associated with one or more reference tags, component visualization program110creates a three dimensional rendering using the component data. In an embodiment, component visualization program110uses 3D representation data, such as an XML file, and vital product data to render a 3D model of the one or more components of interest. Further, component visualization program110renders one or more 3D models of one or more components associated or interconnected with the one or more components of interest to create a 3D rendering of a larger system of related components. For example, component visualization program110may create a 3D rendering of a problematic piston in an engine block and also render the rest of the components in the engine block to allow a user to see a 3D rendering of a functioning engine block. In a related example, component visualization program110may overlay one or more 3D renderings of one or more components of interest and one or more associated components on an image or a video to mix the 3D rendering in the actual environment. In another embodiment, component visualization program110may create a 3D rendering using data pulled from manufacturer databases, internet resources, and visual references of a component. In yet another embodiment, component visualization program110may show a user historical component information, such as information regarding past configurations of the components and accompanying 3D models. For example, a user may request and receive a 3D model and information about a component used in a previous iteration of the system, such as a mainframe computer. In yet another embodiment, component visualization program110may make x-ray views, subcomponent views, and time lapse views of one or more 3D models in a larger system available to one or more users. For example, component visualization program110may make an augmented 3D rendering of an iteration of a system one year prior and display information regarding the components, such as an augmented three dimensional rendering, created by component visualization program110during the designated time frame. As a result, a user may gain insight into why particular decisions were made regardless of a change in the user working with the system. By rendering a 3D model of one or more components and creating systems model that can be manipulated, component visualization program110allows a user to quickly visualize faulty components, locate inefficient components, and/or manipulate 3D renderings of one or more components to better visualize how existing systems function.

Component visualization program110analyzes the component data and one or more 3D component renderings to create one or more augmented three dimensional renderings (step212). An augmented three dimensional rendering may be any analysis of one or more components in one or more 3D environments. For example, augmented three dimensional renderings may focus on repair instructions, component efficiency reports, and system modification based on component data and 3D component renderings. In essence, augmented three dimensional renderings augment the ability of one or more users to assess the viability of any system and to address any anticipated changes in the efficiency of the system. In one embodiment, component visualization program110creates an augmented three dimensional rendering that assesses how a component and all moving parts within the component and all subcomponents will function in a system to find potential vulnerabilities or inefficiencies in a user proposed system. As a result, component visualization program110compiles a set of repair and modification instructions that illustrate how to repair likely future breakdowns and propose modifications to the system with known components to increase the viability of the system over time. In another embodiment, component visualization program110may analyze how the 3D components using vital product data, such as subcomponent durability and heat tolerances, will function at present and over time to create component efficiency reports. As a result, one or more users may determine a repair or modification schedule based on the life span of the one or more components in a system.

Component visualization program110sends the component data, one or more 3D renderings, and one or more augmented three dimensional renderings to one or more requesting devices (step214). In an embodiment, component visualization program110sends the component data to a mobile computing device, such as a smart phone. A user may view and manipulate 3D visual representations of the component and view information associated with the component on a smart phone. In another embodiment, component visualization program110may send the component data associated with one or more reference tags to one or more devices. For example, component visualization program110may send 3D representation data and vital product data to the laptop computer, smart phone, and desktop computer of each member of a five-person team. In yet another embodiment, component visualization program110send the component data to a camera-enabled requesting device, such as a camera-enabled smart phone, a camera-enabled tablet computer, and a camera-enabled laptop computer.

Component visualization program110determines whether component visualization program110received subsequent component data associated with the one or more reference tags. (decision block216). In an embodiment, component visualization program110may receive subsequent component data associated with the one or more reference tags after any changes are made to the hardware. For example, a team debugging the manufacturing process of a main frame computer may replace a hard drive with a new hard drive, swap a power supply with a more powerful one, and move the motherboard to a new location. When the team makes changes to the system and logs them using smart phones and laptops, component data associated with the changes may be sent to component visualization program110. In another embodiment, the component or device itself may send the changes to component visualization program110. Component visualization program110may then determine the changes by comparing the original component data with the manipulated component data. Users may also directly note any changes made to the component data in one or more client computing devices104and choose to send the subsequent component data to component visualization program110.

Responsive to component visualization program110determining that component visualization program110received subsequent component data associated with the one or more reference tags (“Yes” branch, decision block216), component visualization program110stores the subsequent component data in one or more databases (step204). In an embodiment, component visualization program110may store the subsequent component data to one database containing both 3D representation data and vital product data. In another embodiment, component visualization program110may store the subsequent component data to a first database for storing vital product data and a second database for storing 3D representation data.

Responsive to component visualization program110determining that component visualization program110did not receive subsequent component data associated with the one or more reference tags (“No” branch, decision block216), component visualization program110ends.

FIG. 3depicts component visualization program110running on client computing device104, in accordance with an embodiment of the present invention. In the depicted embodiment, client computing device104captures and displays an image304of device302. Component visualization program110analyzes device302from a video feed received from client computing device104and the existing components of device302via visual cues, manual input from a user through user interface106on client computing device104, and/or any other method of registering the existing components of device302with component visualization program110. Client computing device104overlays 3D renderings of components306created by component visualization program110subsequent to analysis, such as missing components, user-requested components, and any other component of interest. Further client computing device104overlays vital product data308pulled from one or more databases by component visualization program110, such as repair instructions, installation instructions, and replacement instructions.

For example, client computing device104may capture and display images in substantially real-time (i.e., streaming video) of a computer tower. Component visualization program110may receive one or more user-inputted references (i.e., serial numbers, component names, and component types). After analyzing the one or more user-inputted references, component visualization program110creates a 3D rendering of the components306and overlays the 3D renderings of the components306of the components on the streaming video of the computer tower in relation to the current position of the camera on client computing device104relative to the device302and the vital product data308on client computing device104.

In another embodiment, component visualization program110may create animated 3D renderings of the components306of one or more components in a particular environment. For example, component visualization program110may show how a cooling fan moves in relation to other components in the environments and display potential obstructions, such as wires, to cooling fan movement. In yet another embodiment, component visualization program110may create 3D renderings of sub-components within 3D renderings of the components306of one or more components in a particular environment. For example, component visualization program110may render the screws, motors, and internal wiring of one or more cooling fans. As a result, a user of client computing device104can see into a cooling fan by manipulating 3D renderings through user interface106on client computing device104by requesting a semi-transparent view of one or more 3D renderings of the components306one or more components or requesting an exploded view of one or more 3D renderings of the components306one or more components. In yet another embodiment, client computing device104may display a completely 3D rendered image without overlaying 3D renderings of the components306over image304.

Server computer108can include processor(s)404, cache414, memory406, persistent storage408, communications unit410, input/output (I/O) interface(s)412and communications fabric402. Communications fabric402provides communications between cache414, memory406, persistent storage408, communications unit410, and input/output (I/O) interface(s)412. Communications fabric402can be implemented with any architecture designed for passing data and/or control information between processors (such as microprocessors, communications and network processors, etc.), system memory, peripheral devices, and any other hardware components within a system. For example, communications fabric402can be implemented with one or more buses.

Memory406and persistent storage408are computer readable storage media. In this embodiment, memory406includes random access memory (RAM). In general, memory406can include any suitable volatile or non-volatile computer readable storage media. Cache414is a fast memory that enhances the performance of processor(s)404by holding recently accessed data, and data near recently accessed data, from memory406.

Program instructions and data used to practice embodiments of the present invention, e.g., component visualization program110and database112, are stored in persistent storage408for execution and/or access by one or more of the respective processor(s)404of server computer108via cache414. In this embodiment, persistent storage408includes a magnetic hard disk drive. Alternatively, or in addition to a magnetic hard disk drive, persistent storage408can include a solid-state hard drive, a semiconductor storage device, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, or any other computer readable storage media that is capable of storing program instructions or digital information.

Communications unit410, in these examples, provides for communications with other data processing systems or devices, including resources of client computing device104. In these examples, communications unit410includes one or more network interface cards. Communications unit410may provide communications through the use of either or both physical and wireless communications links. component visualization program110, database112, and other programs and data used for implementation of the present invention, may be downloaded to persistent storage408of server computer108through communications unit410.

I/O interface(s)412allows for input and output of data with other devices that may be connected to server computer108. For example, I/O interface(s)412may provide a connection to external device(s)416such as a keyboard, a keypad, a touch screen, a microphone, a digital camera, and/or some other suitable input device. External device(s)416can also include portable computer readable storage media such as, for example, thumb drives, portable optical or magnetic disks, and memory cards. Software and data used to practice embodiments of the present invention, e.g., component visualization program110and database112on server computer108, can be stored on such portable computer readable storage media and can be loaded onto persistent storage408via I/O interface(s)412. I/O interface(s)412also connect to a display418.

Display418provides a mechanism to display data to a user and may be, for example, a computer monitor. Display418can also function as a touchscreen, such as a display of a tablet computer.