Patent Description:
With rapid development of artificial intelligence technologies, an artificial intelligence device emerges. Intelligent control is a key technology in a working process of the artificial intelligent device (for example, an intelligent vehicle, a robot, and an intelligent ship). Accuracy of time synchronization determines reliability of intelligent control.

For example, a sensor timestamp, sensor data fusion, positioning accuracy compensation, planning decision making, and the like in intelligent control all strongly depend on system time. Therefore, an intelligent control process has a higher requirement for time synchronization.

Currently, a time synchronization process in intelligent control is as follows: A global navigation satellite system (GNSS) module in an intelligent control system receives a satellite timing signal, and obtains accurate time by using the satellite timing signal; and a system time management module uses, as system time, the accurate time obtained by using the satellite timing signal, and completes time synchronization to ensure time accuracy.

When the artificial intelligence device is in a blocking scenario such as a tunnel or a city canyon, the GNSS module cannot obtain the satellite timing signal. In this case, the system time management module uses time of an internal clock as system time for intelligent control. If the satellite timing signal has been lost for a relatively long time, there is a relatively large time difference between the time of the internal clock and time of a satellite due to insufficient accuracy of the internal clock. Therefore, when the GNSS module re-obtains the satellite timing signal, if accurate time obtained by using the satellite timing signal is directly used as the system time, the system time hops. Once the system time hops, stability of intelligent control is severely affected. For example, there is a case in which a sensor loses a frame, fused data cannot be matched, positioning is inaccurate, or an exception occurs in planning control. Consequently, running of the artificial intelligent device is abnormal. Document <CIT> generally discloses a distribution device that distributes time information to at least one sensor device, the distribution device including a storage unit that stores a time adjustment amount to be used for adjusting a local time, a calculation unit that calculates a time difference between a reference time and the local time, an adjustment unit that calculates an adjusted local time by adjusting the local time by an amount equal to or less than the time adjustment amount, when the time difference is greater than the time adjustment amount, and a distribution unit that distributes time information of the adjusted local time to the sensor device.

This application provides a time synchronization method and apparatus, to improve accuracy of time synchronization in the artificial intelligence field and improve stability of intelligent control. In particular, embodiments of the present application are defined by the independent claims. Additional features of embodiments of the invention are presented in the dependent claims. In the following, parts of the description and drawings referring to former embodiments which do not necessarily comprise all features to implement embodiments of the claimed invention are not represented as embodiments of the invention but as examples useful for understanding the embodiments of the invention. Embodiments of the present application (or the present disclosure) provide a time synchronization method, a time synchronization apparatus, and a computer-readable storage medium.

According to a first aspect, a time synchronization method is provided and is applied to a time synchronization apparatus in an artificial intelligence device. The method may include: determining a time difference between reference time and system time of the artificial intelligence device, where the reference time is timed by an internal clock of the artificial intelligence device and is aligned based on a satellite timing signal, or the reference time is timed by an internal clock of the artificial intelligence device; and adjusting the system time based on a preset step value if the time difference is greater than a preset value.

According to the time synchronization method provided in this application, determining of the time difference between the reference time and the system time of the artificial intelligence device is added. When the time difference between the reference time and the system time of the artificial intelligence device is greater than the preset value, the system time is adjusted based on the preset step value, to implement progressive synchronization. In this way, system time hopping caused by direct alignment performed when the time difference is relatively large is avoided, accuracy of time synchronization in an artificial intelligence field is improved, and stability of intelligent control is improved.

The artificial intelligence device may specifically include but is not limited to an intelligent vehicle, an intelligent ship, an intelligent robot, and the like.

The reference time is a reference time value for synchronization of the system time in the time synchronization apparatus. When the artificial intelligence device can normally receive the satellite timing signal transmitted by a satellite, accurate time obtained by the artificial intelligence device based on the timing signal is used as the reference time. When the artificial intelligence device cannot normally receive the satellite timing signal, the time of the internal clock of the artificial intelligence device is used as the reference time.

It should be noted that when the artificial intelligence device can normally receive the satellite timing signal transmitted by the satellite, the time of the internal clock of the artificial intelligence device may alternatively be aligned with the reference time.

The preset value is a preconfigured threshold used to determine whether hopping occurs when the reference time is directly synchronized with the system time. A specific value of the preset value may be configured based on an actual requirement.

The preset step value is a preconfigured time adjustment value for progressive synchronization. When the time difference between the reference time and the system time of the artificial intelligence device is greater than the preset value, the system time is adjusted to the reference time by one preset step value.

With reference to the first aspect, in a possible implementation, after the adjusting the system time based on a preset step value, the time synchronization method provided in this application further includes: redetermining the time difference between the reference time and the system time of the artificial intelligence device after preset duration; and adjusting the system time based on the preset step value if the time difference is greater than the preset value. The time synchronization method provided in this application is performed for a plurality of times by setting the preset duration, to implement progressive synchronization.

With reference to the first aspect and any one of the foregoing possible implementations, in another possible implementation, the preset duration is less than or equal to a sending period of the satellite timing signal.

Optionally, the preset duration may be equal to the sending period of the satellite timing signal. In other words, a time interval of two times of performing the time synchronization method provided in this application is the same as the sending period of the satellite timing signal. Alternatively, the preset duration may be less than the sending period of the satellite timing signal. In other words, a time interval of two times of performing the time synchronization method provided in this application is greater than the sending period of the satellite timing signal, to implement fast synchronization.

With reference to the first aspect and either of the foregoing possible implementations, in another possible implementation, the time synchronization method provided in this application further includes: determining whether the artificial intelligence device enters an intelligent control mode; and if the artificial intelligence device enters the intelligent control mode, determining the time difference between the reference time and the system time of the artificial intelligence device. In this way, a progressive synchronization solution is used only when the artificial intelligence device enters the intelligent control mode, so that processing efficiency is improved.

With reference to the first aspect and any one of the foregoing possible implementations, in another possible implementation, the time synchronization method provided in this application further includes: synchronizing the system time with the reference time if the artificial intelligence device does not enter the intelligent control mode.

Synchronizing the system time with the reference time may be implemented as follows: directly aligning the system time with the reference time.

With reference to the first aspect and any one of the foregoing possible implementations, in another possible implementation, the time synchronization method provided in this application further includes: if the time difference between the reference time and the system time of the artificial intelligence device is less than the preset value, synchronizing the system time with the reference time.

It should be noted that when the time difference between the reference time and the system time of the artificial intelligence device is equal to the preset value, the system time may be adjusted based on the preset step value, or the system time may be synchronized with the reference time.

With reference to the first aspect and any one of the foregoing possible implementations, in another possible implementation, the time synchronization method provided in this application may further include: obtaining an initial value of the system time, where the initial value of the system time is reference time when the artificial intelligence device is started.

According to a second aspect, this application provides a time synchronization apparatus. The apparatus may be an apparatus or a chip system in an artificial intelligence device, or an apparatus that can be used together with an artificial intelligence device. The time synchronization apparatus may implement functions performed in the foregoing aspects or possible designs, and the functions may be implemented by hardware, or may be implemented by hardware by executing corresponding software. The hardware or the software includes one or more modules corresponding to the functions. For example, the time synchronization apparatus may include a determining unit and an adjustment unit.

The determining unit is configured to determine a time difference between reference time and system time of the artificial intelligence device, where the reference time is timed by an internal clock of the artificial intelligence device and is aligned based on a satellite timing signal, or the reference time is timed by an internal clock of the artificial intelligence device. The adjustment unit is configured to adjust the system time based on a preset step value if the time difference determined by the determining unit is greater than a preset value.

It should be noted that the time synchronization apparatus provided in the second aspect is configured to perform the time synchronization method provided in the first aspect. For specific implementation, refer to specific implementation of the first aspect.

According to a first example not encompassed by the claims but useful for understanding the present application, there is provided a time synchronization apparatus. The time synchronization apparatus may include a processor and a memory. The processor is coupled to the memory, and the memory may be configured to store computer program code. The computer program code includes computer instructions. When the computer instructions are executed by the time synchronization apparatus, the time synchronization apparatus is enabled to perform the time synchronization method according to the first aspect or any possible implementation.

According to a second example not encompassed by the claims but useful for understanding the present application, an artificial intelligence device is provided, and includes the time synchronization apparatus according to any one of the foregoing aspects or possible implementations.

According to a third aspect, an embodiment of this application provides a computer-readable storage medium. The computer-readable storage medium may include computer software instructions. When the computer software instructions are run on a computer, the computer is enabled to perform the time synchronization method according to any one of the first aspect or the possible implementations of the first aspect.

According to a third example not encompassed by the claims but useful for understanding the present application, there is provided a computer program product. When the computer program product is run on a computer, the computer is enabled to perform the time synchronization method according to any one of the first aspect of the claims or the possible implementations.

According to a fourth example not encompassed by the claims but useful for understanding the present application, there is provided a chip system. The chip system is applied to an artificial intelligence device, and the chip system includes an interface circuit and a processor. The interface circuit is interconnected with the processor by using a line. The interface circuit is configured to receive a signal from a memory of the artificial intelligence device, and send the signal to the processor, where the signal includes computer instructions stored in the memory. When the processor executes the computer instructions, the chip system performs the time synchronization method according to any one of the first aspect or the possible implementations.

It should be understood that descriptions of technical features, technical solutions, beneficial effects, or similar words in this application do not imply that all features and advantages can be implemented in any individual embodiment. On the contrary, it may be understood that descriptions of the features or the beneficial effects mean that at least one embodiment includes a specific technical feature, technical solution, or beneficial effect. Therefore, the descriptions of the technical features, the technical solutions, or the beneficial effects in this specification may not necessarily belong to one embodiment. Further, the technical features, the technical solutions, and the beneficial effects described in embodiments may be combined in any proper manner. A person skilled in the art understands that an embodiment may be implemented without one or more specific technical features, technical solutions, or beneficial effects in a specific embodiment. In other embodiments, additional technical features and beneficial effects may be further identified in a specific embodiment that does not reflect all embodiments.

In this specification, claims, and accompanying drawings of this application, the terms "first", "second", "third", and the like are intended to distinguish between different objects but do not limit a particular order.

In addition, in embodiments of this application, the word "example" or "for example" is used to represent giving an example, an illustration, or a description. Any embodiment or design scheme described as an "example" or with "for example" in embodiments of this application should not be explained as being more preferred or having more advantages than another embodiment or design scheme. Exactly, use of the word such as "example" or "for example" is intended to present a related concept in a specific manner.

For ease of understanding, terms in this application are explained first.

A satellite timing signal may be a signal that is sent by a satellite to indicate accurate time. For example, the satellite timing signal includes a time value and a pulse signal. A device that receives the satellite timing signal uses a moment of a rising edge of the pulse signal as the time value included in the satellite timing signal, to perform time synchronization. In actual application, the satellite timing signal is periodically sent. The satellite in this specification may be Beidou, a global positioning system (Global Positioning System, GPS), a glonass satellite, or the like.

The reference time may be a reference time value for synchronization of system time in a time synchronization apparatus. When the artificial intelligence device can normally receive the satellite timing signal transmitted by the satellite, accurate time obtained by the artificial intelligence device based on the timing signal is used as the reference time. When the artificial intelligence device cannot normally receive the satellite timing signal, time of an internal clock of the artificial intelligence device is used as the reference time.

The time of the internal clock is time recorded by an apparatus with a timing function deployed in the artificial intelligence device. When the artificial intelligence device can normally receive the satellite timing signal transmitted by the satellite, the time of the internal clock of the artificial intelligence device may alternatively be aligned with the reference time.

The system time is time information used when the artificial intelligence device performs intelligent control, and the system time is synchronized based on the reference time.

Currently, a solution for implementing time synchronization in the artificial intelligence device is as follows: A GNSS module in the artificial intelligence device receives a satellite timing signal sent by a satellite, to obtain accurate time as a reference signal, and directly synchronizes the system time with the reference signal. When the satellite timing signal cannot be obtained, time of an internal clock is used as the reference signal; in other words, the time of the internal clock is used as the system time. However, timing accuracy of the internal clock is insufficient. Consequently, when the satellite signal is not received in a long time, there is a relatively large time difference between the time of the internal clock and time of the satellite. When the satellite timing signal is restored, system time hopping is caused if the system time is directly synchronized with the time of the satellite. This affects stability of intelligent control.

For example, the artificial intelligence device is an intelligent vehicle, and accuracy of an internal clock of the intelligent vehicle is <NUM> part per million (part per million, PPM). When the intelligent vehicle is in a tunnel, an internal GNSS module cannot obtain a satellite timing signal. Time of the internal clock is used as a reference signal, and the time of the internal clock is used as the system time.

If the intelligent vehicle is in the tunnel for one minute, the satellite timing signal is lost for one minute, and a timing error of the internal clock is <NUM>×<NUM>/<NUM>=<NUM> milliseconds (millisecond, ms).

If the intelligent vehicle is in the tunnel for five minutes, the satellite timing signal is lost for five minutes, and a timing error of the internal clock is <NUM>×<NUM>/<NUM>=<NUM>.

If the intelligent vehicle is in the tunnel for ten minutes, the satellite timing signal is lost for ten minutes, and a timing error of the internal clock is <NUM>×<NUM>/<NUM>=<NUM>.

If the intelligent vehicle is in the tunnel for <NUM> minutes, the satellite timing signal is lost for <NUM> minutes, and a timing error of the internal clock is <NUM>×<NUM>/<NUM>=<NUM>.

It can be learned from the foregoing data that, longer time in which the artificial intelligence device cannot obtain the satellite timing signal leads to a larger timing error of the internal clock. When the satellite timing signal is restored, a hopping value caused if the system time is directly synchronized with accurate time indicated by satellite timing time is also relatively large.

Currently, accuracy of system time for intelligent control in the intelligent vehicle greatly influences a network element for intelligent control. For example, a sensor frame rate is as follows: For a camera (camera) sensor, <NUM> is one frame, and for a lidar (lidar) sensor, <NUM> is one frame. If an error between the system time and actual accurate time exceeds specific time, an intelligent control system directly considers that the sensor loses a frame and an alarm is accidentally touched or a function is abnormal. In a fused system of the intelligent vehicle, a camera is fused with a lidar frame within specific time. If the error between the system time and the actual accurate time exceeds the specific time, the intelligent control system cannot select data, and consequently, a fusion function is abnormal. The intelligent control system calculates a positioning point by calculating/transmitting time and a speed, an acceleration, and an angular velocity. Once the system time hops, system positioning is abnormal. The intelligent control system usually performs planning control at an interval of <NUM>. If the system time hops, next delivery of planning control is abnormal.

For example, <FIG> shows a time synchronization process in a traveling process of an artificial intelligence device. As shown in <FIG>, before a moment X<NUM> timed by a satellite, the artificial intelligence device is located outside a tunnel, and may obtain a satellite timing signal, and system time is synchronous with accurate time indicated by the satellite timing signal. The artificial intelligence device enters the tunnel in a second X<NUM> timed by the satellite, and exits the tunnel in a second Xx+<NUM> timed by the satellite. In the tunnel, the artificial intelligence device cannot obtain the satellite timing signal, and the system time is time of an internal clock. It is assumed that traveling time of the artificial intelligence device in the tunnel is n seconds, and an accumulated error of internal timing is ±α×n seconds (α is accuracy of the internal clock). In a second before exiting the tunnel, that is, a second Xx timed by the satellite, the system time is a second Xx±α×n. The artificial intelligence device re-obtains the satellite timing signal in the second Xx+<NUM>, and the system time is directly synchronized to the second Xx+<NUM>. The system time hops from the second Xx±α×n to the second Xx+<NUM>.

Based on this, an embodiment of this application provides a time synchronization method. A time difference between reference time and system time of an artificial intelligence device is first determined, and when the time difference is greater than a preset value, the system time is synchronized with the reference time in a stepwise manner, to avoid hopping of the system time, improve accuracy of time synchronization in an artificial intelligence field, and improve stability of intelligent control.

The following describes implementations of embodiments of this application in detail with reference to the accompanying drawings.

A time synchronization method provided in embodiments of this application may be applied to a time synchronization system shown in <FIG>. As shown in <FIG>, the time synchronization system may include an artificial intelligence device <NUM> and a satellite <NUM>. The satellite <NUM> is used for accurate timing and periodically sending a satellite timing signal, to perform time synchronization with the artificial intelligence device <NUM>. The artificial intelligence device <NUM> synchronizes system time with accurate time indicated by the satellite timing signal, to perform intelligent control.

Actual product forms of the satellite and the artificial intelligence device in the time synchronization system shown in <FIG> are not specifically limited.

Specifically, as shown in <FIG>, the artificial intelligence device <NUM> may include a server <NUM>, an intelligent controller <NUM>, a sensor <NUM>, a timing receiver <NUM>, and an execution controller <NUM>.

The server <NUM> is configured to: connect the artificial intelligence device <NUM> to the cloud by using a connection medium (wired or wireless, shown as wireless in <FIG>), share information with another artificial intelligence device in the cloud, and cache and process related data.

The intelligent controller <NUM> is configured to implement an intelligent control function of the artificial intelligence device <NUM>.

The sensor <NUM> is configured to collect environment data in a running process of the artificial intelligence device <NUM>. For example, the sensor <NUM> may be a millimeter wave radar, a camera, an ultrasonic radar, or a lidar.

The timing receiver <NUM> may include a combined positioning module <NUM> and an antenna <NUM>. The antenna <NUM> is configured to receive the satellite timing signal sent by the satellite <NUM>, and the combined positioning module <NUM> obtains the accurate time based on the satellite timing signal.

The execution controller <NUM> may include a control module <NUM> and an execution module <NUM>. The control module <NUM> receives a control signal sent by the intelligent controller <NUM>, and controls the execution module <NUM> to implement a function of the artificial intelligence device <NUM>.

An entity form of the intelligent controller <NUM> may be shown in <FIG>, and includes a processor <NUM>, a memory <NUM>, and an interface module <NUM>.

The processor <NUM> may include a system on chip (system on chip, SOC) and a microcontroller unit (microcontroller unit, MCU). The memory <NUM> is configured to store program code, a configuration file, or other content for implementing the intelligent controller <NUM>. The interface module <NUM> is configured to provide a connection interface such as a sensor interface, a controller interface, a gateway interface, or another interface for another device. Specifically, the sensor interface is configured to collect data of each sensor. The controller interface is configured to receive and send a related control signal. The gateway interface is configured to implement communication between modules by using a network.

Specifically, the processor <NUM> may perform various functions of the intelligent controller <NUM> by running or executing a software program and/or a module stored in the memory <NUM> and invoking data stored in the memory <NUM>.

As shown in <FIG>, the intelligent controller <NUM> may include a sensor time management module <NUM>, a processing module <NUM>, a planning and control module <NUM>, and a system time management module <NUM>.

The system time management module <NUM> may be deployed inside an MCU or an SOC of the intelligent controller <NUM>. The sensor time management module <NUM> is separately deployed inside the MCU and the SOC of the intelligent controller <NUM> based on an attribute of the sensor. The processing module <NUM> may be deployed inside the SOC of the intelligent controller <NUM>. The planning and control module <NUM> may be deployed inside the MCU or the SOC of the intelligent controller <NUM>.

The system time management module <NUM> is responsible for overall system time management of the artificial intelligence device <NUM>, including time management of the controller and the sensor.

The sensor time management module <NUM> is mainly responsible for data timestamp management of each frame of the sensor.

The processing module <NUM> may be responsible for receiving the data collected by the sensor <NUM>, and analyzing the data to obtain the environment information, to guide planning and control of the artificial intelligence device <NUM>.

The planning and control module <NUM> is mainly configured to: make a behavior decision based on information provided by the processing module <NUM>, generate a control instruction, and transmit the control instruction to the execution controller <NUM>.

An intelligent vehicle is used as an example of the artificial intelligence device <NUM> below to describe in detail functions of the components of the artificial intelligence device <NUM>.

<FIG> is a schematic diagram of a system architecture in an intelligent driving scenario according to an embodiment of this application. As shown in <FIG>, the system architecture may include an intelligent driving vehicle <NUM> and a satellite <NUM>.

The intelligent driving vehicle <NUM> may include a server <NUM>, an intelligent driving domain controller <NUM>, a sensor <NUM>, a timing receiver <NUM>, and an execution controller <NUM>. The server <NUM>, the intelligent driving domain controller <NUM>, the sensor <NUM>, the timing receiver <NUM>, and the execution controller <NUM> are deployed inside an intelligent driving system. The satellite <NUM> may be a GNSS global satellite navigation system, and includes a glonass, a GPS, and Beidou.

Specifically, in an intelligent driving scenario, the server <NUM> in <FIG> may be the server <NUM> in <FIG>. As shown in <FIG>, the server <NUM> may specifically include a telematics box (telematics box, TBOX) <NUM> and a central gateway <NUM>.

The intelligent controller <NUM> in <FIG> may be the intelligent driving domain controller <NUM> in <FIG>. As shown in <FIG>, the intelligent driving domain controller <NUM> may specifically include a sensor time management module <NUM>, a perception and fusion module <NUM>, a positioning software module <NUM>, a road planning and control module <NUM>, and a system time management module <NUM>.

The sensor <NUM> in <FIG> may be the sensor <NUM> in <FIG>. As shown in <FIG>, the sensor <NUM> may specifically include a millimeter wave radar <NUM>, a camera <NUM>, a lidar <NUM>, and an ultrasonic radar <NUM>.

The timing receiver <NUM> in <FIG> may be the timing receiver <NUM> in <FIG>. As shown in <FIG>, the timing receiver <NUM> may specifically include a combined positioning module <NUM> and an antenna <NUM>.

The execution controller <NUM> in <FIG> may be the execution controller <NUM> in <FIG>. As shown in <FIG>, the execution controller <NUM> may specifically include a chassis electronic control unit <NUM>, a steering system <NUM>, and a wheel speedometer <NUM>.

The TBOX <NUM> is mainly responsible for storing and processing cloud data received by the intelligent driving vehicle <NUM>.

The central gateway <NUM> is mainly responsible for communication between the intelligent driving vehicle <NUM> and the cloud.

The sensor time management module <NUM> is mainly responsible for data timestamp management of each frame of the sensor <NUM>.

The perception and fusion module <NUM> is divided into two parts: perception and fusion. Perception is mainly responsible for intelligent analysis of lidar point cloud data and camera image frame data that are input by a preprocessing module, and implements functions such as static/dynamic target object detection and tracking, lane line identification, traffic light identification, and obstacle identification. A fusion unit is configured to: receive an obstacle list, depth information, lane line information, and travelable area information from a perception module, output obstacle state estimation and track prediction in a range of interest through smoothing processing, and output a final travelable area and obstacle information to a planning decision making unit.

The positioning software module <NUM> is mainly responsible for receiving road feature information provided by the perception and fusion module, and outputting spatial positioning and posture information of the vehicle with reference to the GNSS and wheel speedometer fusion positioning information and high-precision map information.

The road planning and control module <NUM> mainly makes a behavior decision based on the travelable area information, the positioning information, and the obstacle information that are provided by the perception and fusion module <NUM> and the positioning software module <NUM> and real-time motion information of the vehicle, including horizontal and vertical motion control and planning; and generates a control instruction (including a brake, a throttle, a steering wheel, a gear, a steering lamp, and the like) based on motion control and planning and a given speed.

The system time management module <NUM> is mainly responsible for overall time management of a driving system, including time management of the controller and the sensor.

The camera <NUM> is mainly responsible for collecting picture information in an environment.

The millimeter wave radar <NUM>, the lidar <NUM>, and the ultrasonic radar <NUM> are mainly responsible for collecting object information in the environment.

The antenna <NUM> is mainly responsible for receiving a timing signal transmitted by the satellite.

The combined positioning module <NUM> is mainly responsible for obtaining accurate time based on the satellite timing signal.

The chassis electronic control unit <NUM> is mainly responsible for receiving a control signal sent by the intelligent driving domain controller <NUM>, and controlling the steering system <NUM>.

The steering system <NUM> is mainly responsible for performing an action based on an instruction of the chassis electronic control unit <NUM>.

The wheel speedometer <NUM> is mainly responsible for detecting and controlling a rotational speed of a wheel.

A time synchronization procedure of the system architecture shown in <FIG> is as follows: The satellite <NUM> sends a satellite timing signal to the intelligent driving vehicle <NUM>. After receiving the satellite timing signal, the antenna <NUM> in the timing receiver <NUM> in the system of the intelligent driving vehicle <NUM> transmits received satellite timing information to the combined positioning module <NUM> in the timing receiver <NUM>. The combined positioning module <NUM> obtains, through parsing, accurate time indicated by the satellite timing information, and sends the accurate time to the system time management module <NUM> in the intelligent driving domain controller <NUM> as system time. The system time is specifically used in each module in the intelligent driving domain controller <NUM> for time management and intelligent control.

Specifically, the sensor time management module <NUM> performs data timestamp management for each frame of each sensor (the millimeter wave radar <NUM>, the camera <NUM>, the lidar <NUM>, and the ultrasonic radar <NUM>) based on the system time. The perception and fusion module <NUM> performs intelligent analysis on input point cloud data and image frame data, completes functions such as static/dynamic target object monitoring and tracking, lane line identification, traffic light identification, and obstacle identification with reference to the system time, and outputs obstacle status estimation, track prediction, and a travelable area in a range of interest. The positioning software module <NUM> outputs spatial positioning and posture information of the vehicle by combining received road feature information provided by the perception and fusion module with the positioning information and the high-precision map information. The road planning and control module <NUM> makes a behavior decision based on the travelable area information, the positioning information, and the obstacle information that are provided by the fusion unit and the positioning unit and the real-time motion information of the vehicle, including horizontal and vertical motion control and planning; and generates a control instruction (including a brake, a throttle, a steering wheel, a gear, a steering lamp, and the like) based on motion control and planning and a given speed. The generated control instruction is sent to the execution controller <NUM>, and the chassis electronic control unit <NUM> in the execution controller <NUM> controls the steering system <NUM> and the like.

The following describes embodiments of this application in detail with reference to the accompanying drawings.

According to one aspect, an embodiment of this application provides a time synchronization apparatus, configured to perform the time synchronization method provided in this application. The time synchronization apparatus may be deployed in the artificial intelligence device in the time synchronization system shown in <FIG>. <FIG> shows a time synchronization apparatus <NUM> related to embodiments of this application. As shown in <FIG>, the time synchronization apparatus <NUM> may include a processor <NUM>, a memory <NUM>, and a transceiver <NUM>.

Components of the time synchronization apparatus <NUM> are specifically described below with reference to <FIG>.

The memory <NUM> may be a volatile memory (volatile memory), such as a random access memory (random access memory, RAM); or a nonvolatile memory (non-volatile memory), such as a read-only memory (read-only memory, ROM), a flash memory (flash memory), a hard disk drive (hard disk drive, HDD), or a solid-state drive (solid-state drive, SSD), or a combination of the foregoing types of memories, configured to store program code, a configuration file, or other content that can implement the method in this application.

The processor <NUM> is a control center of the time synchronization apparatus <NUM>. For example, the processor <NUM> may be a central processing unit (central processing unit, CPU), or may be an application specific integrated circuit (application specific integrated circuit, ASIC), or may be configured as one or more integrated circuits implementing embodiments of this application, for example, one or more microprocessors (digital signal processor, DSP), or one or more field programmable gate arrays (field programmable gate array, FPGA).

The transceiver <NUM> is configured to communicate with another device.

The processor <NUM> performs the following functions by running or executing an application program and a configuration file and/or a module that are stored in the memory <NUM>, and/or by invoking data stored in the memory <NUM>:
determining a time difference between reference time and system time of an artificial intelligence device, where the reference time is timed by an internal clock of the artificial intelligence device and is aligned based on a satellite timing signal, or the reference time is timed by an internal clock of the artificial intelligence device; and adjusting the system time based on a preset step value if the time difference is greater than a preset value.

According to another aspect, an embodiment of this application provides a time synchronization method. The time synchronization method is performed by the foregoing time synchronization apparatus, and the time synchronization apparatus may be deployed in the foregoing artificial intelligence device.

As shown in <FIG>, the time synchronization method provided in this embodiment of this application may include the following steps.

S601: A time synchronization apparatus determines a time difference between reference time and system time of an artificial intelligence device.

The reference time is timed by an internal clock of the artificial intelligence device and is aligned based on a satellite timing signal, or the reference time is timed by an internal clock of the artificial intelligence device.

Specifically, the reference time is a reference time value for synchronization of the system time in the time synchronization apparatus. When the artificial intelligence device can normally receive the satellite timing signal transmitted by a satellite, accurate time obtained by the artificial intelligence device based on the timing signal is used as the reference time. When the artificial intelligence device cannot normally receive the satellite timing signal, the time of the internal clock of the artificial intelligence device is used as the reference time.

For example, it is assumed that the time synchronization apparatus performs S601 at a moment A. The time synchronization apparatus calculates a difference between reference time at the moment A and system time at the moment A to obtain a time difference C between the reference time and the system time of the artificial intelligence device.

S602: If the time difference between the reference time and the system time of the artificial intelligence device is greater than a preset value, the time synchronization apparatus adjusts the system time based on a preset step value.

The preset step value is less than the preset value. The preset value is a preconfigured threshold used to determine whether hopping occurs when the reference time is directly synchronized with the system time. In actual application, a specific value of the preset value may be configured based on an actual requirement.

For example, the preset value may be a fixed value that is set by a user based on experience, or the preset value may be calculated based on a minimum tolerance time variation value of each module in a system.

The preset step value is each preconfigured time adjustment value for progressive synchronization. When the time difference between the reference time and the system time of the artificial intelligence device is greater than the preset value, the system time is adjusted to the reference time by one preset step value.

In a possible implementation, the user may set a fixed preset step value based on experience.

In another possible implementation, the preset step value may be a dynamic value calculated based on the time difference obtained in S601. A specific calculation manner may be: performing calculation based on a preset expression, and the preset expression may be a linear expression or another expression. This is not specifically limited in this embodiment of this application.

For example, it is assumed that the preset value is β, and the preset step value is y. If the time difference C obtained by the time synchronization apparatus in S601 is greater than β, in S602, the time synchronization apparatus may adjust the system time at the moment A to the reference time at the moment A by y. The adjustment described herein may be "increase" or "decrease". When the system time at the moment A is earlier than the reference time at the moment A, in S602, the time synchronization apparatus may increase the system time at the moment A by y. When the system time at the moment A is later than the reference time at the moment A, in S602, the time synchronization apparatus may decrease the system time at the moment A by y.

It should be noted that when the artificial intelligence device is started, the system time needs to be initialized to obtain an initial value of the system time. The initial value of the system time is reference time when the artificial intelligence device is started.

Further, before S602, as shown in <FIG>, the time synchronization method provided in this embodiment of this application may further include S602a.

S602a: The time synchronization apparatus determines a magnitude relationship between the preset value and the time difference between the reference time and the system time of the artificial intelligence device.

The magnitude relationship includes "greater than", "equal to", or "less than".

Optionally, if it is determined in S602a that the time difference between the reference time and the system time of the artificial intelligence device is greater than the preset value, S602 is performed; if it is determined in S602a that the time difference between the reference time and the system time of the artificial intelligence device is less than the preset value, S603 is performed; or if it is determined in S602a that the time difference between the reference time and the system time of the artificial intelligence device is equal to the preset value, S602 or S603 is performed.

S603: The time synchronization apparatus synchronizes the system time with the reference time.

That the time synchronization apparatus synchronizes the system time with the reference time in S603 may be specifically implemented as follows: The time synchronization apparatus uses the reference time as the system time.

For example, it is assumed that the preset value is β. If the time difference C obtained by the time synchronization apparatus in S601 is less than β, in S602, the time synchronization apparatus may use the reference time at the moment A as the system time at the moment A.

Further, before S601, as shown in <FIG>, the time synchronization method provided in this embodiment of this application may further include S601a.

S601a: The time synchronization apparatus determines whether the artificial intelligence device enters an intelligent control mode.

Specifically, in S601a, the time synchronization apparatus may query status information recorded by the artificial intelligence device, to determine whether the artificial intelligence device enters the intelligent control mode.

Optionally, if the time synchronization apparatus determines, in S601a, that the artificial intelligence device enters the intelligent control mode, S601 is performed; or if the time synchronization apparatus determines, in S601a, that the artificial intelligence device does not enter the intelligent control mode, the time synchronization apparatus directly performs S603 to synchronize the system time with the reference time.

It should be noted that the time synchronization apparatus may perform, for a plurality of times, the time synchronization method provided in this embodiment of this application. Operations in all times of execution are the same, but only system time and reference time in all times of execution are currently the latest. Details are not described again.

Specifically, after S602 or S603, the time synchronization apparatus may re-perform S601 or S601a and subsequent steps to perform next synchronization. When S601 is re-performed, the system time and the reference time for calculating the time difference are the system time and the reference time in S601.

For example, after S602, the time synchronization apparatus may re-perform, after preset duration, S601 or S601a and subsequent steps to perform next synchronization.

The preset duration may be less than or equal to a sending period of the satellite timing signal.

Optionally, the preset duration is equal to the sending period of the satellite timing signal. In other words, a time interval of two times of performing the time synchronization method provided in this application is the same as the sending period of the satellite timing signal. For example, the sending period of the satellite timing signal may be once per second, and in this case, the preset duration may be one second.

Optionally, the preset duration is less than the sending period of the satellite timing signal. In other words, a time interval of two times of performing the time synchronization method provided in this application is greater than the sending period of the satellite timing signal, to implement fast synchronization. For example, the sending period of the satellite timing signal may be once per second, and the preset duration may be <NUM> milliseconds.

It should be noted that when the preset duration is less than the sending period of the satellite timing signal, a value of the preset duration may be configured based on an actual requirement. This is not specifically limited in this embodiment of this application.

An intelligent vehicle is used as an example of the artificial intelligence device below to describe, with reference to a specific intelligent driving scenario, the time synchronization method provided in this embodiment of this application.

<FIG> shows a time synchronization process in a traveling process of an intelligent vehicle. As shown in <FIG>, before a moment X<NUM> timed by a satellite, the intelligent vehicle is located outside a tunnel, and may obtain a satellite timing signal, and system time is synchronous with accurate time indicated by the satellite timing signal. The intelligent vehicle enters the tunnel in a second X<NUM> timed by the satellite, and exits the tunnel in a second Xx timed by the satellite. In the tunnel, the artificial intelligence device cannot obtain the satellite timing signal, and system time is time of an internal clock. It is assumed that traveling time of the artificial intelligence device in the tunnel is n seconds, and an accumulated error of internal timing is ±α×n seconds (α is accuracy of the internal clock). In a second Xx, the system time is a second Xx±α×n.

The intelligent vehicle re-obtains the satellite timing signal in a second Xx+<NUM> timed by the satellite. In this case, reference time is the second Xx+<NUM>, and the system time is the second (Xx±α×n). A time synchronization apparatus deployed in the intelligent vehicle performs the time synchronization method provided in this application, determines a time difference between the reference time, namely, the second Xx+<NUM>, and the system time (Xx±α×n), and determines a magnitude relationship between the time difference and a preset value β.

If the time synchronization apparatus determines that the time difference is greater than the preset value β, as shown in <FIG>, the time synchronization apparatus adjusts the system time from (Xx±α×n) to the reference time, namely, the second Xx+<NUM>, by y, and adjusts the system time to (Xx±α×n)±γ. After preset duration t, the time synchronization apparatus provided in this application redetermines a time difference between reference time, namely, a second Xx+t, and system time (Xx±α×n)±γ+t, determines a magnitude relationship between the time difference and the preset value β, and if the time difference is greater than the preset value β, adjusts the system time from (Xx±α×n)±γ+t to the reference time, namely, the second Xx+t, by y. In a second Xx+m, the time synchronization apparatus determines that a time difference between reference time, namely, the second Xx+m, and system time at this moment is less than the preset value β, and the time synchronization apparatus synchronizes the system time to the reference time, namely, the second Xx+m. t may be less than or equal to a sending period of the satellite timing signal.

The foregoing describes, mainly from a perspective of the artificial intelligence device, the solutions provided in embodiments of this application. It may be understood that, to implement the foregoing functions, the artificial intelligence device includes a corresponding hardware structure and/or a corresponding software module that perform various functions. A person of ordinary skill in the art should easily be aware that, in combination with the examples described in embodiments disclosed in this specification, this application may be implemented by hardware or a combination of hardware and computer software. Whether a function is performed by hardware or hardware driven by computer software depends on a particular application and a design constraint of the technical solutions.

In embodiments of this application, the artificial intelligence device may be divided into function modules based on the foregoing method examples. For example, each function module corresponding to each function may be obtained through division, or two or more functions may be integrated into one processing module. The integrated module may be implemented in a form of hardware, or may be implemented in a form of a software functional module. It should be noted that, in embodiments of this application, module division is an example, and is merely a logical function division. In an actual implementation, another division manner may be used.

When each function module is obtained through division by using corresponding functions, <FIG> shows a time synchronization apparatus <NUM> according to an embodiment of this application, and the time synchronization apparatus <NUM> is configured to implement a function of the artificial intelligence device in the foregoing method. The time synchronization apparatus <NUM> may be an apparatus in the artificial intelligence device, or may be an apparatus that can be used together with the artificial intelligence device. The time synchronization apparatus <NUM> may alternatively be a chip system. In this embodiment of this application, the chip system may include a chip, or may include a chip and another discrete component. As shown in <FIG>, the time synchronization apparatus <NUM> may include a determining unit <NUM> and an adjustment unit <NUM>. The determining unit <NUM> is configured to perform S601 in <FIG> or S601 in <FIG>, and the adjustment unit <NUM> is configured to perform S602 in <FIG> or S602 in <FIG>. All related content of the steps in the foregoing method embodiments may be cited in function descriptions of corresponding function modules.

Further, as shown in <FIG>, the time synchronization apparatus <NUM> may further include a judgment unit <NUM>, configured to perform step S601a in <FIG>, and may further include a synchronization unit <NUM>, configured to perform step S603 in <FIG>.

<FIG> shows a time synchronization apparatus <NUM> according to an embodiment of this application, and the time synchronization apparatus <NUM> is configured to implement a time synchronization function of the artificial intelligence device in the foregoing method. The time synchronization apparatus <NUM> may be an apparatus in the artificial intelligence device, or may be an apparatus that can be used together with the artificial intelligence device. The time synchronization apparatus <NUM> may be a chip system. The time synchronization apparatus <NUM> includes at least one processing module <NUM>, configured to implement a time synchronization function of the artificial intelligence device in the methods provided in embodiments of this application. For example, the processing module <NUM> may be configured to perform processes S601, S601a, S602, S602a, and S603 in <FIG> or <FIG>. For details, refer to detailed descriptions in the method examples.

The time synchronization apparatus <NUM> may further include at least one storage module <NUM>, configured to store program instructions and/or data. The storage module <NUM> is coupled to the processing module <NUM>. The coupling in this embodiment of this application may be an indirect coupling or a communication connection between apparatuses, units, or modules in an electrical form, a mechanical form, or another form, and is used for information exchange between the apparatuses, the units, or the modules. The processing module <NUM> may cooperate with the storage module <NUM>. The processing module <NUM> may execute the program instructions stored in the storage module <NUM>. At least one of the at least one storage module may be included in the processing module.

The time synchronization apparatus <NUM> may further include a communication module <NUM>, configured to communicate with another device by using a transmission medium, so that an apparatus used in the time synchronization apparatus <NUM> may communicate with another device.

When the processing module <NUM> is a processor, the storage module <NUM> is a memory, and the communication module <NUM> is a transceiver, the time synchronization apparatus <NUM> in <FIG> in this embodiment of this application may be the time synchronization apparatus shown in <FIG>.

As described above, the time synchronization apparatus <NUM> or the time synchronization apparatus <NUM> provided in embodiments of this application may be configured to implement functions of the electronic device in the methods implemented in embodiments of this application. For ease of description, only a part related to embodiments of this application is shown. For specific technical details that are not disclosed, refer to embodiments of this application.

Some other embodiments of this application further provide a computer-readable storage medium. The computer-readable storage medium may include computer software instructions. When the computer software instructions are run on an artificial intelligence device, the electronic device is enabled to perform the steps performed by the artificial intelligence device in the foregoing embodiment shown in <FIG> or <FIG>.

Some other examples not part of this application further provide a computer program product. When the computer program product is run on a computer, the computer is enabled to perform the steps performed by the artificial intelligence device in the foregoing embodiment shown in <FIG> or <FIG>.

Some other examples not part of this application further provide a chip system, and the chip system may be applied to an artificial intelligence device. The electronic device includes a display and a camera. The chip system includes an interface circuit and a processor. The interface circuit is interconnected with the processor by using a line. The interface circuit is configured to receive a signal from a memory of the artificial intelligence device, and send the signal to the processor, where the signal includes computer instructions stored in the memory. When the processor executes the computer instructions, the chip system performs the steps performed by the electronic device in the foregoing embodiment shown in <FIG> or <FIG>.

Through the foregoing descriptions about implementations, a person skilled in the art can clearly understand that, for the purpose of convenience and conciseness of description, only division of the foregoing function modules is used as an example for illustration. In actual application, the foregoing functions can be allocated to different function modules for implementation based on a requirement, that is, an inner structure of an apparatus is divided into different function modules to implement all or some of the functions described above.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the described apparatus embodiments are merely examples. For example, division of the modules or units is merely logical function division, and may be other division during actual implementation. For example, a plurality of units or components may be combined or may be integrated into another apparatus, or some features may be ignored or not performed. In addition, displayed or discussed mutual couplings or direct couplings or communication connections may be implemented through some interfaces. The indirect couplings or communication connections between the apparatuses or units may be implemented in electrical, mechanical, or other forms.

The units described as discrete components may or may not be physically separate, and components displayed as units may be one or more physical units, that is, may be located in one place, or may be distributed to a plurality of different places.

In addition, function units in embodiments of this application may be integrated into one processing unit, or each of the units may exist alone physically, or two or more units may be integrated into one unit.

When the integrated unit is implemented in the form of a software function unit and sold or used as an independent product, the integrated unit may be stored in a readable storage medium. Based on such an understanding, the technical solutions in embodiments of this application essentially, or the part contributing to the current technology, or all or some of the technical solutions may be implemented in a form of a software product. The software product is stored in a storage medium and includes several instructions for instructing a device (which may be a single-chip microcomputer, a chip, or the like) or a processor (processor) to perform all or some of the steps of the methods described in embodiments of this application. The foregoing storage medium includes: any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a diskette, or an optical disc.

Claim 1:
A time synchronization method, applied to a time synchronization apparatus in an artificial intelligence device, wherein the method comprises:
determining (S601a) whether the artificial intelligence device enters an intelligent control mode;
if the artificial intelligence device enters the intelligent control mode, determining (S601) a time difference between reference time and system time of the artificial intelligence device; and
adjusting (S602) the system time based on a preset step value if the time difference is greater than a preset value; and
if the artificial intelligence device does not enter the intelligent control mode, synchronizing (S603) the system time with the reference time,
wherein the reference time is a reference time value for synchronization of the system time in the time synchronization apparatus.