Synchronizing multiple devices

An apparatus, and computer program product for synchronously starting programs on multiple devices connected to a server is provided. A synchronous point of a program to be synchronously started for each of the multiple devices is identified. A wait function is dynamically injected into the synchronous point for each of the multiple devices. A start time from the server is received in response to the multiple devices entering a waiting state. The programs are synchronously started in response to the start time arriving for each of the multiple devices.

BACKGROUND

The disclosure relates generally to systems for synchronously starting programs on devices connected to a server. Specifically, the systems dynamically inject a wait function into the program on each of the devices to control a start time for the devices.

2. Description of the Related Art

The Internet of things (IoT) refers to the interconnectivity of physical things, such as devices, automobiles, trains, airplanes, and buildings. The IoT may provide direct integration of the physical things into computer-based systems. The IoT may permit grids, houses, transportation, and cities to collect and exchange data. Each of the physical things may be connected electronically by processors, software, and sensors to enable the collection and exchange of data.

In the IoT, each physical thing may be uniquely identifiable through its embedded computing system and may interoperate within an existing Internet infrastructure. In the area of transportation, vehicles may be connected to collect and exchange data regarding speed, direction, fuel efficiency, travel distance, and a record of travel destinations. Vehicle may include driverless cars. Testing of driverless cars may require the collection and exchange of data from devices located on each driverless car. Each device may have its own internal time.

Execution of time-critical test scenarios for driverless cars may depend on other devices. Execution of time-critical test scenarios may be difficult or impossible if the devices are not synchronized. Multiple devices need to be synchronously operated, not only for testing, but also for actual real-life operation. For example, devices may be temporarily stopped in order to change configuration information, and then simultaneously operated after the configuration change. Therefore, a need exists to start multiple driverless cars simultaneously.

SUMMARY

According to one illustrative embodiment, a computer-implemented method for synchronously starting programs on multiple devices connected to a server is provided. A synchronous point of a program to be synchronously started for each of the multiple devices is identified. A wait function is dynamically injected into the synchronous point for each of the multiple devices. A start time, from the server, is received in response to the multiple devices entering a waiting state. The programs are synchronously started in response to the start time arriving for each of the multiple devices.

DETAILED DESCRIPTION

In an illustrative embodiment, synchronization may be performed with multiple devices in multiple gateways. As used herein, an edge gateway is a virtual router for networks that may be virtual device context networks. Preparation periods from the activation of the devices to the actual transmission of the data may be aligned to allow the data to be synchronously transmitted. Synchronous points of programs on multiple devices may be identified by a server. A wait function may be injected dynamically into the synchronous point for each of the multiple devices. As used herein, “wait function” means a function that provides inter-thread synchronization or process-process synchronization on an operating system, on different operating systems and for multiple devices on an Edge gateway. In an illustrative embodiment, a wait function may assign a start time. Changes in configuration information may be distributed from the server. Changes in configuration information may be changes in a type of data to be acquired, in the time intervals for acquisition of data, and in designations of data format. In order to make the changes in configuration information, devices may be temporarily suspended by the injected wait function. Each device is informed of the information. The devices are initialized. As used herein, initialization is the assignment of an initial value for a data object or variable. A time server and device management function may detect that the initialization of all the devices is finished and send a start time to release the wait function and execute the changes in the multiple devices in the multiple gateways. At the start time, changes to the device are performed. The time server and device management function insures that changes are performed synchronously.

In automotive applications of IoT, data devices may be created to acquire and read test data in vehicles. Data devices in the vehicles may need to be operated simultaneously in multiple edge gateways. Situations may arise where multiple data from separate devices must be recorded at the same moment. Location data, such as longitude and latitude from a car-navigation device or speed data from a vehicle body device, may need to be recorded at the same moment to prevent inconsistencies. Inconsistencies may arise from time lags due to manual activation or the state of the devices. If a time lag occurs, data acquired from vehicles running at a certain interval may not reflect the same intervals. Indeed, in the case of a driverless car, if the data is used to control the driverless car, a collision may occur due to the differences in data.

Even if a device is designed to operate a particular action at a predetermined interval, such as a recording and sending sensor data, it may be difficult or impossible to completely synchronize the device with other devices. An example may be data unit conversion. Configuration of vehicle devices may require conforming data from one device in kilometers per hour with data from other devices in miles per hour. Data units may need to be converted at the same time in all devices.

Synchronization with a time server may be used to insert the wait function. The edge gateway may synchronize with the time server when activated. When there is a wait function for each device, synchronization may be performed at a designated time referred to as a “start time”. In an illustrative embodiment, designated time is designated by relative time elapsed in seconds after an occurrence of a trigger. The time server converts the relative time into absolute time and sends the absolute time to each device. When there is a manager in the device, the manager may serve as a relay for the time server and the wait function.

A start time may be designated by relative time elapsed, in seconds, after an occurrence of a certain state. A time server and device management function may convert the relative time into absolute time and send the absolute time to each device. The wait function may be released at the start time. The start time may be different for each device.

Triggers may be employed for sending the start time. A trigger may be when a particular device enters a waiting state. A trigger may be when wait functions in all devices cause the devices to enter waiting states. A trigger may be when a plugin manager of the edge gateway or a time server is informed that all devices have entered waiting states. A trigger may be when a property file of a time server indicates that multiple devices have entered a waiting state.

The programs may be synchronously started in response to the start time arriving for each of the multiple devices. The start time received by one of the multiple devices may be different from the start time received by another one of the multiple devices. The server may be provided with a function of a time server and each of the multiple devices may be synchronized with the time server. Time may be corrected in the device. A system for synchronously starting programs on multiple devices connected to a server, may comprise a bus system, a storage device connected to the bus system, and a processor connected to the bus system, wherein the storage device stores program instructions and the processor executes the program instructions to identify a synchronous point of a program to be synchronously started for each of the multiple devices and to dynamically inject a wait function into the synchronous point for each of the multiple devices. As used herein a “synchronous point” is a point selected dynamically by a computer system to embed a wait function into the source code of a program in accordance with one or more criteria. In an illustrative embodiment, a synchronous point to embed a wait function into the source code of a program may be selected manually by a programmer at a display. In an illustrative embodiment, the point at which the wait function is embedded may be the end of an initialization part of a start method. In another illustrative embodiment, wait functions may be embedded into the source code when the source code is written. In another illustrative embodiment a programmer may use a graphical user interface to select a point to insert a wait function, and then to dynamically inject the wait function into the program at a selected point. In an illustrative embodiment, the selected point may be the point at which initialization is finished. Dynamic insertion may be accomplished by the computer system automatically determining a time at which initialization is finished, and dynamically injecting the wait function into the part of the program at which initialization is finished so that the synchronous point is established by the computer system. Other criteria may be used in other embodiments. In an illustrative embodiment, when a thread class is defined such as Java and the thread class is extended to implement the class, a start method generally includes an initialization part. The wait function may be dynamically injected at the end of the initialization part of the start method. The storage device may store further program instructions and the processor execute the further program instructions to receive a start time from the server in response to the multiple devices entering a waiting state and synchronously start the programs in response to the start time arriving for each of the multiple devices. Time may be corrected in the device. The start time may not be relative time. The start time may be absolute time. The start time may be received from the server, and the wait function may be released when the start time arrives.

FIG. 1depicts a pictorial representation of a network of data processing systems in which illustrative embodiments may be implemented. Network data processing system100is a network of computers and other devices in which the illustrative embodiments may be implemented. Network data processing system100contains network102, which is the medium used to provide communications links between the computers and the other devices connected together within network data processing system100. Network102may include connections, such as, for example, wire communication links, wireless communication links, and fiber optic cables.

In the depicted example, server104and server106connect to network102, along with storage108. Server104and server106may be, for example, server computers with high-speed connections to network102. In addition, server104or server106may, for example, manage recovery of a customer workload after failure of a primary computing environment executing the customer workload. The failed primary computing environment may be, for example, a server or a set of servers in a data center environment or a cloud environment. Server104or server106also may generate a secondary virtual machine seed image storage at a secondary data processing site for the failure recovery. The configuration of the secondary data processing site is similar to the configuration of the primary data processing site.

Client110, client112, and client114also connect to network102. Clients110,112, and114are clients of server104and/or server106. Server104and server106may provide information, such as boot files, operating system images, virtual machine images, and software applications to clients110,112, and114.

In this example, clients110,112, and114may each represent a different computing environment. A computing environment includes physical and software resources used to execute a set of one or more customer workloads or tasks. A computing environment may comprise, for example, one server, a rack of servers, a cluster of servers, such as a data center, a cloud of computers, such as a private cloud, a public cloud, or a hybrid cloud, or any combination thereof. In addition, each of clients110,112, and114may be a primary data processing site or a secondary data processing site. A primary data processing site initially executes a customer workload using a set of primary virtual machines and images. A secondary data processing site executes the customer workload using a set of secondary virtual machines and seed images when one or more primary virtual machines fail while processing the customer workload at the primary data processing site.

Storage108is a network storage device capable of storing any type of data in a structured format or an unstructured format. The type of data stored in storage108may be, for example, a list of computing environments with corresponding available resources, a list of primary data processing sites, a list of secondary data processing sites, a list of customer workloads, a plurality of virtual machine images, and the like. Further, storage unit108may store other types of data, such as authentication or credential data that may include user names, passwords, and biometric data associated with system administrators, for example.

In addition, it should be noted that network data processing system100may include any number of additional servers, clients, storage devices, and other devices not shown. Program code located in network data processing system100may be stored on a computer readable storage medium and downloaded to a computer or other data processing device for use. For example, program code may be stored on a computer readable storage medium on server104and downloaded to client110over network102for use on client110.

In the depicted example, network data processing system100may be implemented as a number of different types of communication networks, such as, for example, an internet, an intranet, a local area network (LAN), and a wide area network (WAN).FIG. 1is intended as an example only, and not as an architectural limitation for the different illustrative embodiments.

With reference now toFIG. 2, a diagram of a data processing system is depicted in accordance with an illustrative embodiment. Data processing system200is an example of a computer, such as server104inFIG. 1, in which computer readable program code or instructions implementing processes of illustrative embodiments may be located. In this illustrative example, data processing system200includes communications fabric202, which provides communications between processor unit204, memory206, persistent storage208, communications unit210, input/output unit212, display214and Edge gateways215.

Processor unit204serves to execute instructions for software applications and programs that may be loaded into memory206. Processor unit204may be a set of one or more hardware processor devices or may be a multi-processor core, depending on the particular implementation. Further, processor unit204may be implemented using one or more heterogeneous processor systems, in which a main processor is present with secondary processors on a single chip. As another illustrative example, processor unit204may be a symmetric multi-processor system containing multiple processors of the same type.

In this example, persistent storage208may store programs220and data240. Programs220may include time server and device management230and injector232. Injector232may be configured for dynamic injection of a wait routine into any function so that the wait routine may be embedded in the function. Injector232may comprise any technology that may embed a wait routine in any function. For example, a wait routine may be embedded at the beginning of a start function to start the process. Thus, when the process having the start function and the embedded wait routine runs, the process may wait before executing an original start because the wait routine causes the process to wait until an appointed time. Further, when all processes have a start function and a wait routine embedded, all processes can be run synchronously at an appointed time. The name of the start function may be arbitrary. For example, a name of a start function of process A may be “begin” and a name of a start function of a process B may be “run”. Injector232may use API Hook244in data240to hook “begin” function for process A and to hook “run” function for process B. As used herein, “hooking” means one or more techniques to change the behavior of an operating system, an application, or a software component by intercepting one or more function calls, messages, or events. As used herein, “hook” means code that performs the interception of the function calls, events, or messages. API Hook244may be located in load library246. Load library246may be a dynamic load library (DLL). Persistent storage208may also store the following programs or functions: time stamp221, graphical user interface (GUI)222, initialization function224, plug-in manager231, absolute time233, device plug-in234, and start times235.

Data may include target process242, API hook244, synchronized data241, start points245, load library246, wait function or routine248, and breakpoints249. Devices260may be connected to time server and device management230by a wired connection or a wireless connection via communications fabric202and communications unit210. Devices260may be devices depicted in the illustrative embodiments ofFIG. 3,FIG. 4andFIG. 5. Devices260may be activated and synchronized with time server and device management230to create absolute time233and start times235in the devices. Synchronization with time server and device management230may require a time stamp used for sending data to time server and device management230for equalization. Time stamp221may provide a time stamp for synchronization with time server and device management230. Start times from start times235may be acquired by devices260from time server and device management230.

Absolute time233may be set as a start time to the devices by start times235. In an alternate embodiment, start time may be absolute time233plus an amount of time determined to be after the multiple devices finish inquiring about time may be set as a start time. Acquisition of a start time may be used to insert the wait function. When an edge gateway, such as edge gateways215synchronizes with time server and management function230, a start time from start times235may be sent to edge gateways215. When multiple devices finish initialization, a start time may be sent to edge gateways215, according to a setting on time server and management215. When multiple devices enter a waiting state with a wait function, a start time may be sent to edge gateways215according to a setting on time server and management230. When a particular device finishes initialization, a start time may be sent to edge gateways215according to a setting on time server and device management230. Edge gateways215may receive a start time by polling. The start time may indicate absolute time released by the wait function of the device and be represented by year, month, day, hour, minute, and second, instead of time elapsed.

Devices260may have start points245. Devices260may be synchronized with timeserver and device management230, and then initialized by initialization function224. After initialization, devices260may be brought into a waiting state at start points245. Devices260brought into a waiting state may be reflected in waiting state243in data240. A waiting state may be indicated by a property file configured to receive notifications that devices260have entered a waiting state. Devices brought into waiting state may continue to inquire into time server and device management230for a time until a start time or an absolute time may be acquired from time server and device management230using start times235and absolute time233. When a start time from start times235may be acquired, devices260wait until the start time and then wait function or routine248is released.

Embedding a function such as wait( ) from wait function or routine248into a program may set one or more start points in start points245and may require a source code to be modified. Alternatively, a function from wait function or routine248may be embedded dynamically into an appropriate point of the program by setting one or more breakpoints in breakpoints249using graphical user interface (GUI)222. Wait function or routine248may be injected by injector232without modifying existing software. Alternatively, responsive to an automatic search for initialization function224, a wait function from wait function or routine248may automatically be injected by injector230into end points of initialization function224.

Program code252is located in a functional form on computer readable media254that is selectively removable and may be loaded onto or transferred to data processing system200for running by processor unit204. Program code252and computer readable media254form computer program product256. In one example, computer readable media254may be computer readable storage media258or computer readable signal media250. Computer readable storage media258may include, for example, an optical or magnetic disc that is inserted or placed into a drive or other device that is part of persistent storage208for transfer onto a storage device, such as a hard drive, that is part of persistent storage208. Computer readable storage media258also may take the form of a persistent storage, such as a hard drive, a thumb drive, or a flash memory that is connected to data processing system200. In some instances, computer readable storage media258may not be removable from data processing system200.

Alternatively, program code252may be transferred to data processing system200using computer readable signal media250. Computer readable signal media250may be, for example, a propagated data signal containing program code252. For example, computer readable signal media250may be an electro-magnetic signal, an optical signal, and/or any other suitable type of signal. These signals may be transmitted over communication links, such as wireless communication links, an optical fiber cable, a coaxial cable, a wire, and/or any other suitable type of communications link. In other words, the communications link and/or the connection may be physical or wireless in the illustrative examples. The computer readable media also may take the form of non-tangible media, such as communication links or wireless transmissions containing the program code.

In some illustrative embodiments, program code252may be downloaded over a network to persistent storage208from another device or data processing system through computer readable signal media250for use within data processing system200. For instance, program code stored in a computer readable storage media in a data processing system may be downloaded over a network from the data processing system to data processing system200. The data processing system providing program code252may be a server computer, a client computer, or some other device capable of storing and transmitting program code252.

As another example, a computer readable storage device in data processing system200is any hardware apparatus that may store data. Memory206, persistent storage208, and computer readable storage media258are examples of physical storage devices in a tangible form.

Illustrative embodiments are capable of being implemented in conjunction with any type of computing environment now known or later developed. Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources, such as, for example, networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services, which can be rapidly provisioned and released with minimal management effort or interaction with a provider of the service. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

The characteristics may include, for example, on-demand self-service, broad network access, resource pooling, rapid elasticity, and measured service. On-demand self-service allows a cloud consumer to unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with the service's provider. Broad network access provides for capabilities that are available over a network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms, such as, for example, mobile phones, laptops, and personal digital assistants. Resource pooling allows the provider's computing resources to be pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to demand. There is a sense of location independence in that the consumer generally has no control or knowledge over the exact location of the provided resources, but may be able to specify a location at a higher level of abstraction, such as, for example, country, state, or data center. Rapid elasticity provides for capabilities that can be rapidly and elastically provisioned, in some cases automatically, to quickly scale out and rapidly released to quickly scale in. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be purchased in any quantity at any time. Measured service allows cloud systems to automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate to the type of service, such as, for example, storage, processing, bandwidth, and active user accounts. Resource usage can be monitored, controlled, and reported providing transparency for both the provider and consumer of the utilized service.

Service models may include, for example, Software as a Service (SaaS), Platform as a Service (PaaS), and Infrastructure as a Service (IaaS). Software as a Service is the capability provided to the consumer to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface, such as a web browser (e.g., web-based e-mail). The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings. Platform as a Service is the capability provided to the consumer to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including networks, servers, operating systems, or storage, but has control over the deployed applications and possibly application hosting environment configurations. Infrastructure as a Service is the capability provided to the consumer to provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but has control over operating systems, storage, deployed applications, and possibly limited control of select networking components, such as, for example, host firewalls.

Deployment models may include, for example, a private cloud, community cloud, public cloud, and hybrid cloud. A private cloud is a cloud infrastructure operated solely for an organization. The private cloud may be managed by the organization or a third party and may exist on-premises or off-premises. A community cloud is a cloud infrastructure shared by several organizations and supports a specific community that has shared concerns, such as, for example, mission, security requirements, policy, and compliance considerations. The community cloud may be managed by the organizations or a third party and may exist on-premises or off-premises. A public cloud is a cloud infrastructure made available to the general public or a large industry group and is owned by an organization selling cloud services. A hybrid cloud is a cloud infrastructure composed of two or more clouds, such as, for example, private, community, and public clouds, which remain as unique entities, but are bound together by standardized or proprietary technology that enables data and application portability, such as, for example, cloud bursting for load-balancing between clouds.

FIG. 3depicts a schematic illustrating configuration300for synchronizing multiple devices using edge gateways, an application server, and a time server and device management application in accordance with an illustrative embodiment.

Multiple devices comprise car data device323, navigation device325, and device327. Car data device323, navigation device325, and device327may be devices260inFIG. 2. Car data device323may be a device that obtains a vehicle speed, direction, travel distance and other desired vehicle information. Navigation device325may be a device to get longitude, latitude, map information and so on. Device327may be a health device, (for example to get heart rate of a driver), a drive recorder device, or some other device to get desired information. Device plugins322,324, and326may be device plug in234inFIG. 2. Device plugin322changes a physical device to a logical device by providing for communication between plugin manager321and a physical device such as car data device323, navigation device325or device327. For example, device plugin322may change units in kilometers per hour to miles per hour or degrees to millidegrees. Plug-in device322may transform a data format from raw data from a device to XML data. Device plugin322may be configured for car data device323. Device plugin324may be configured for navigation device325. Device plugin326may be configured for device327. In an embodiment, plugin devices322,324, and326may be configured for car data device323, navigation device325, and device327.

Plugin manager321may be a module to manage a logical device (a pair of device plugin and physical device). Plugin manager321may collect formatted and normalized data from some physical devices through the device plugins. Plugin manager may actuate psychical devices through device plugins such as plugin devices322,324, and326. Plugin manager321may filter data and may also interpolate data. For example, plugin manager321may get data from device321every 500 milliseconds and may send averaged data every one second to time server and device management310. Plugin manager321may receive data from device327every one second, and may send approximated data that may be made by an interpolation algorithm every 500 milliseconds. Communication318may be communications unit210inFIG. 2. Communication318may be a module to communicate between edge gateway320and app server350.

Timer312may be a timer module that supports network timer protocol and be synchronized with correct time. Device manager314may be a module to manage a beginning time for each device. Generate trigger316may be a module to set a trigger and notify a trigger. Generate trigger316may be assigned a start time and may send a trigger to plugin manager321. Timer312may receive a correct time through plugin manager321and communication318from an external Network Time Protocol (NTP) server. ETP server may be data processing system200inFIG. 2.

Each device may get a correct time from timer312. Beginning time for each device may be set to generate trigger316. Generate trigger316may set a beginning time for each device to device manager314. Device manager314may get time from timer312. If it is the beginning time of the device, device manager314may notice it to generate trigger316. Generate trigger316may wake up each device. Synchronized data may be sent to app server350from plugin manager321. Edge gateway330and edge gateway340are similar to edge gateway320. App server350may provide communication352between edge gateway320, edge gateway330and edge gateway340. App server350may be data processing system200inFIG. 2.

FIG. 4depicts a schematic illustrating configuration400for synchronizing multiple devices using edge gateways, an application server, and a time server and device management application illustrating a process in accordance with an illustrative embodiment. The illustrative embodiment ofFIG. 4is similar to the illustrative embodiment ofFIG. 3. InFIG. 4, timer server and device management410communicates directly with communication428of edge gateway420, communication438of edge gateway430, and communication448of edge gateway440. App server450communicates directly with communication428of edge gateway420, communication438of edge gateway430, and communication448of edge gateway440. Plugin manager421is similar to plugin manager321inFIG. 2. Device plugins422,424, and426are similar to device plugins322,234, and326inFIG. 3. Car data device423is similar to car data device323inFIG. 3. Navigation device425is similar to navigation device325inFIG. 3. Device427is similar to device327inFIG. 3.

FIG. 5depicts a schematic illustrating configuration500for synchronizing multiple devices in an edge gateway in accordance with an illustrative embodiment. In an illustrative embodiment, synchronization may be performed in edge gateway520. Time and device management510may be placed in edge gateway520to allow device plugins522,524, and526in edge gateway520to be synchronized directly with edge gateway520. In the configuration ofFIG. 5, time and device management function510communicates directly with plugin manger521and is not connected to communication528of edge gateway520or to communication552of app server550. Timer512is similar to timer312inFIG. 3. Device manager514is similar to device manager312inFIG. 3. Generate trigger516is similar to generate trigger316inFIG. 3. Car data device525is similar to car data device323inFIG. 3. Navigation device525is similar to navigation device325inFIG. 3. Device527is similar to device327inFIG. 3. The configuration ofFIG. 5may be used when there is no need for synchronization with multiple devices in multiple edge gateways.

FIG. 6depicts an example of injection of a wait function in accordance with an illustrative embodiment. In this context, “hooking” means that an application's request for executing a function may be automatically redirected to the injected code so that some additional operations can be done before or after running an original function's code without modifying or recompiling a caller's module. Injector602may be a module that performs hooking in order to inject a wait function. Target process610may be a process having one or more target functions that may be hooked by injector602.

API hook612may be a module that consists of a new function for an intercepted original function. When target process610is hooked by injector602, a new function may be called instead of an original function. In an illustrative embodiment, if target process610has CreateFile API. CreateFile API is hooked by injector602, the original function, CreateFile API, may be replaced by MyCreateFile in API hook612. Once the hook and replacement is performed, MyCreateFile will be called instead of CreateFile. Injection may be performed dynamically by injector602. Injector602may call SuspendThread620to suspend all threads of target process610. VirtualAllocEx ( . . . ) (see620) may allocate memory in target process610. Injector602may save a start address of the memory to variable “pMem.”

Injector602may write a full path of API hook612on the allocated memory. Injector602may call CreateRemoteThread622with an address of Load Library with pMem variable606. CreateRemoteThread622) may call LoadLibrary with pMem variable606in target process610internally. LoadLibrary with pMem variable606may load API hook612in target process610and LoadLibrary with pMem variable606may be started internally. API hook612performs hooking in target process610internally. API hook612may replace any function in API hook612. For example, API Hook612may replace CreateFile to MyCreateFile. After injector602calls ResumeThread (see622), all threads of target process610may resume.

FIG. 7depicts another example of injection of a wait function in accordance with an illustrative embodiment. Conceptual diagrams of instruction codes stored in memory, comparing the original state before “hooking” with the modified state after “hooking” are depicted. For transferring from the original state to the hooking state, several bytes of code at the top of the function are replaced with other instructions so that the function call may be redirected to another function. Those original instructions are backed up in the virtual memory area so that the original function can be executed after finishing the redirected codes. API Hook DLL in hooking state704may be a dynamically-injected module which implements additional function codes for caller applications to execute. Original state702may be the state before enabling Wait( ) function, and hooking state704may be the state after enabling it. In hooking state704, any application automatically calls the Wait( ) function in API Hook DLL in hooking state704after entering into start( ) function, without modifying or recompiling application code.

In original state702, when a caller—Target.exe—in the illustrative embodiment—calls function start( ), the central processing unit executes the instruction code at the function's entry point address (77d10d2) and returns to the caller after executing the final “ret” instruction. On the other hand, in hooking state704, when Target.exe calls function start( ) (see arrow line 1), the central processing unit executes the ‘jmp’ instruction indicating a jump to injected code (see arrow line 2) and then executes additional operations, Our Additional Code, and calls the “Wait( )” function. After executing Our Additional Code and Wait( ), the central processing unit calls the original function's code which has been backed up in virtual memory (see arrow line 3). The last instruction in virtual memory may be a ‘jmp’ instruction, which causes a jump to the remaining code in the original function (see arrow line 4) consequently completing the original function code. The final ‘ret’ instruction makes the central processing unit execute the remaining part of injected codes (see arrow line 5), which may be code to do nothing and return to the caller (see arrow line 6).

Code segment 77d10d2 may be an example of a memory address pointing to the start( ) function in target.exe of original state702. Target.exe may be a memory block containing instruction codes of function start( ). The code “ret” may be a final instruction of “start( )” function. Target.exe in original state702may be similar to target.exe in hooking state704, except for the first two instructions which have been replaced with other instructions so that the function call may be redirected to the code in API Hook DLL of hooking state704. Virtual memory in hooking state704stores instruction code at the top of an original function. Two instructions in original state702may be replaced with other instructions in hooking state704. Wait( ) function may cause the central processing unit to wait for all other devices to be synchronized.

FIG. 8depicts a flowchart illustrating a process for synchronizing multiple devices in accordance with an illustrative embodiment. The use of the technique prevents differences in data that arise from lack of synchronization. The technique can be used not only for tests, but also for the actual synchronization of the multiple devices. Synchronization logic may be dynamically injected without software on the devices being modified, which does not require modification cost. As used herein “dynamically injected” means instruction codes in memory are modified while the application may be running so that the additional logic can be executed when the application operates that function. A synchronous point may be the memory address of a function which should be synchronously executed with other devices, and may be the address where wait( ) function may be placed.

Process800is for synchronously starting programs on multiple devices connected to a server start. A synchronous point of a program to be synchronously started is identified for each of a number of devices (Step802). A wait function is dynamically injected into the synchronous point for each of the multiple devices (Step804).

A start time is received from the server in response to the multiple devices entering a waiting state (Step806). A determination may be made whether a start time for a device has arrived (Step808). If a start time for the device has not arrived, process800returns to step806. Responsive to receiving a start time for a device, a program is synchronously started in response to the start time arriving for the device (Step810). A determination is made as to whether all devices have been started (Step812). If all devices have not been started, process800returns to step802. If all devices have been started, process800terminates.

In an embodiment, the start time received by one of the multiple devices may be different from the start time received by another one of the multiple devices. In an embodiment, a server may be provided with a function of a time server and each of the multiple devices may be synchronized with the time server. In an embodiment, time in the device may be corrected. In an embodiment, the start time may not be relative time but absolute time. In an embodiment, the wait function may be operated according to a sequence that waiting may be performed until the start time may be sent, the start time may be received from the server, and the start time may arrive.

The time server can be put not only on multiple edge gateways, but also on a single edge gateway to allow the multiple devices in the edge gateway to be synchronized. Delay activation of the devices can also be intentionally made as well as synchronization.

FIG. 9depicts a flowchart illustrating a process for dynamically inserting a wait function in accordance with an illustrative embodiment. There may be several ways to insert the wait function. Wait functions may be embedded into source codes. A process, such as process900is displayed on a graphical user interface (Step902). A point into which a wait function is inserted is shown on the graphical user interface (Step904). Timing at which initialization is finished is automatically determined (Step906). The wait function is dynamically injected into the part (Step908). Thereafter, the process terminates. For example, when a thread class may be defined like Java and extended to implement the class, a start method generally bears an initialization part to dynamically inject the wait at the time of ending the method.

FIG. 10depicts a flowchart of a process for dynamically releasing a wait function in accordance with an illustrative embodiment. Process1000starts. A start time is sent to the edge gateway according to the setting on the time server (1002). The edge gateway receives a start time. (Step1004) A determination is made whether the time is the same as the start time (Step1006). If not, process1000waits (Step1010). If the time is the same as the start time, the wait function is released (Step1008). At the start time, the wait function may be released. Release of the start time may cause all the devices to be synchronized. Thereafter, the process terminates.

FIG. 11depicts a flowchart of a number of triggers for sending a start time in accordance with an illustrative embodiment. Process1100starts. A wait function is connected with a time server to perform synchronization with time (Step1102). Wait functions in all devices enter the waiting state (Step1104). A determination is made whether the plugin manager of the edge gateway has been informed that all devices have entered the wait state (step1106). If the plugin manager of the edge gateway has been informed that all devices have entered the wait step, the start time may be sent (Step1112). If not, a determination is made whether a time server has been informed that all devices have entered the waiting state (Step1108). If the time server has been informed that all devices have entered the waiting state, process1100goes to step1112. If not, a determination is made whether the time server recognizes that the multiple devices have entered the waiting state, with the property file of the time server (Step1110). If the time server recognizes that the multiple devices have entered the waiting state, with the property file of the time server, the process goes to step1112. Process1100determines whether the time is the same as the start time (Step1114). If the time is the same as the start time, then the wait function may be released (Step1116). If the time is not the same as the start time, process1100goes to step1106. After the wait function is released in step1116, process1100terminates thereafter.