Patent Description:
Remotely operated equipment, such as, for example, remote controlled machines with camera/video feedback to an operator, are often operated with the operator located far away from the location of the remotely operated equipment. Thus, it may not be feasible or practical for the operator of a remote controlled machine to visually inspect the remote controlled machine while starting up the machine. In the event that there are several remote controlled machines operating within the same remote control system, that is, the remote control system is arranged to control multiple different remote controlled machines, it may be critical to ensure that the remote controlled machine which an operator is under the impression that he is currently controlling is the remote controlled machine actually being controlled.

Although this problem may become moot or trivial once the remote controlled machine is in remote operation, an accident may still occur during start-up in case the operator is trying to control one remote controlled machine while actually controlling another remote controlled machine. In such cases, an operator may, for example, aggressively move his controls to obtain a reaction from a remote controlled vehicle since the operator is perceiving that nothing is happening via his remote display screen. This may cause damage on the remote controlled machine and people if another remote controlled machine is receiving said control commands and is thus controlled accordingly.

Hence, there is a need to improve the safety of remote operations of a vehicle, in particular during a start-up sequence of said remote control.

<CIT> describes an apparatus for use on unmanned vehicles, wherein a plurality of activation codes are generated in order to activate a payload.

<CIT> describes a vehicle lock control method and device using activation codes.

The invention is set forth in the independent claims <NUM>, <NUM>, <NUM>, <NUM> and in the dependent claims <NUM> to <NUM>, <NUM> to <NUM>, <NUM>, <NUM> and <NUM> to <NUM>.

It is an object of embodiments herein to improve the safety of remote operations of a vehicle.

According to a first aspect of embodiments herein, the object is achieved by a method performed by an on-board control unit of a vehicle for enabling a safe operation of the vehicle. The method comprise receiving, via a wireless communication interface, a second activation code from an off-board control station capable of remotely operating the vehicle. The method also comprise activating the vehicle for operation in case the received second activation code correspond to a first activation code comprised in the on-board control unit.

By having an on-board control unit checking the received activation code against an on-board activation code, it may be ensured that the video feedback used to operate the vehicle is indeed transmitted from the vehicle that a remote control interface is sending its control signals to. Hence, the safety of remote operations of the vehicle is improved.

According to some embodiments, the method may comprise receiving, from the off-board control station, information indicating that the vehicle is to be activated for operation. This enables the on-board control to receive an indication from the off-board control station that an operator is requesting the vehicle to be activated. In this case, the method may also comprise transmitting the first activation code to the off-board control station in response to the received information. Here, according to some embodiments, transmitting of the first activation code may comprise embedding a visual representation of the first activation code in camera images obtained from one or more on-board cameras, and transmitting the camera images comprising the visual representation of the first activation code to the off-board control station. This allows the first activation code to be unambiguously linked to the camera images from the vehicle in a non-separable manner; thus, providing a visual assurance to an operator viewing the camera images of the activation code of vehicle from which the camera images are originating.

In some embodiments, the method may also comprise transmitting, to the off-board control station, information indicating that the vehicle has been activated for operation. This provides an assurance to the operator that the vehicle has actually been activated. Also, in some embodiments, in case the vehicle is an autonomous vehicle, the method may comprise activating the autonomous vehicle to begin performing one or more autonomous operations. Alternatively, in case the vehicle is a remote-controlled vehicle, the method may comprise activating the remote-controlled vehicle to be remotely operated from the off-board control station.

According to a second aspect of embodiments herein, the object is achieved by an on-board control unit of a vehicle for enabling a safe operation of the vehicle. The on-board control unit is configured to receive, via a wireless communication interface, a second activation code from an off-board control station capable of remotely operating the vehicle. The on-board control unit is also configured to activate the vehicle for operation in case the received second activation code correspond to a first activation code comprised in the on-board control unit.

According to some embodiments, the on-board control unit may be configured to receive, from the off-board control station, information indicating that the vehicle is to be activated for operation, and transmit the first activation code to the off-board control station in response to the received information. Here, according to some embodiments, the on-board control unit may be configured to embed a visual representation of the first activation code in camera images obtained from one or more on-board cameras, and transmit the camera images comprising the visual representation of the first activation code to the off-board control station.

In some embodiments, the on-board control unit may be configured to transmit, to the off-board control station, information indicating that the vehicle has been activated for operation. Also, in some embodiments, in case the vehicle is an autonomous vehicle, the on-board control unit may be configured to activate the autonomous vehicle to begin performing one or more autonomous operations. Alternatively, in case the vehicle is a remote-controlled vehicle, the on-board control unit may be configured to activate the remote-controlled vehicle to be remotely operated from the off-board remote control station.

According to a third aspect of the embodiments herein, the object is achieved by a method performed by an off-board control station for enabling a safe operation of a vehicle. The method comprise receiving, via a wireless communication interface, a first activation code from an on-board control unit. The method also comprise indicating the received first activation code to an operator of the off-board control station. Further, the method comprise receiving manual input indicating a second activation code from the operator of the off-board control station. Also, the method comprise transmitting, via the wireless communication interface, the received second activation code to the on-board control unit of the vehicle.

According to some embodiments, the method may comprise receiving, from the operator of the off-board control station, manual input indicating that the vehicle is to be activated for operation. In this case, according to some embodiments, the method may comprise transmitting, to the on-board control unit, information indicating that the vehicle is to be activated for operation. Also, in some embodiments, the method may also comprise receiving, via the wireless communication interface, information from the on-board control unit indicating that the vehicle has been activated for operation. In this case, according to some embodiments, the receiving of the information may comprise receiving camera images originating from one or more on-board cameras of the vehicle comprising an embedded visual representation of the first activation code. Further, the indicating of the received first activation code may comprise displaying the camera images comprising an embedded visual representation of the first activation code to the operator of the off-board control station.

According to a fourth aspect of the embodiments herein, the object is achieved by an off-board control station for enabling a safe operation of a vehicle. The off-board control station being configured to receive, via a wireless communication interface, a first activation code from an on-board control unit. The off-board control station is also configured to indicate the received first activation code to an operator of the off-board control station. Further, the off-board control station is configured to receive manual input indicating a second activation code from the operator of the off-board control station. Also, the off-board control station is configured to transmit, via the wireless communication interface, the received second activation code to the on-board control unit of the vehicle.

According to some embodiments, the off-board control station may also be configured to receive, from the operator of the off-board control station, manual input indicating that the vehicle is to be activated for operation. Here, in some embodiments, the off-board control station may also be configured to transmit, to the on-board control unit, information indicating that the vehicle is to be activated for operation. In some embodiments, the off-board control station may be configured to receive, via the wireless communication interface, information from the on-board control unit indicating that the vehicle has been activated for operation. According to some embodiments, the off-board control station may be configured to receive the first activation code by receiving camera images originating from one or more on-board cameras of the vehicle comprising an embedded visual representation of the first activation code. In this case, according to some embodiments, the off-board control station may be configured to indicate the first activation code by displaying the camera images comprising an embedded visual representation of the first activation code to the operator of the off-board control station.

According to a fifth aspect of the embodiments herein, the object is achieved by a computer program comprising instructions which, when executed in a processing circuitry, cause the processing circuitry to carry out the method described above. According to a sixth aspect of the embodiments herein, the object is achieved by a carrier containing the computer program described above, wherein the carrier is one of an electronic signal, optical signal, radio signal, or computer-readable storage medium.

<FIG> shows an example of a system <NUM> for remote operation of a vehicle <NUM>. The vehicle <NUM> may, for, example, be a working machine <NUM>, such as, an excavator, fork lift, loader, hauler, etc., but may be any remote operated vehicle. In this example, the vehicle <NUM> may comprise a working tool <NUM>. The working tool <NUM> may be any of a large variety of different attachable working tools, such as, e.g. loader buckets, buckets of different sizes, fork lifts, etc. The working tool <NUM> may be arranged to be connected or attached to the vehicle <NUM> and operated therefrom. In the example illustrated in <FIG>, the vehicle <NUM> is a loader comprising a tool <NUM> in the form of a bucket to, e.g. move a pile of loose material <NUM>.

As part of the system <NUM>, the vehicle <NUM> also comprise an on-board control unit <NUM>, at least one camera <NUM>, <NUM>, and at least one antenna <NUM>. The on-board controller <NUM> may be connected to and arranged to communicate with the at least one camera <NUM>, <NUM>, e.g. wirelessly or by wire. Via the antenna <NUM>, the on-board control unit <NUM> may receive signals remotely from an off-board control station <NUM>. Thus, the on-board control unit <NUM> is adapted to be arranged on the vehicle <NUM>. The on-board control unit <NUM> is also adapted to receive signals from the off-board control station <NUM> remotely controlling the operation of the vehicle <NUM>. The on-board control unit <NUM> is arranged to obtain camera images from at least one camera <NUM>, <NUM> on the vehicle <NUM>.

Further, also as part of the system <NUM>, the off-board control station <NUM> may be connected via a communication link <NUM> to a communications server <NUM>. The communications server <NUM> may be part of a wireless communications network <NUM> arranged for wireless communication over an air interface with the antenna <NUM> and on-board control unit <NUM>. The air interface may, for example, be provided by a telecommunications network, such as, e.g. a <NUM>/<NUM>/LTE/<NUM>/<NUM> network or other future networks, and/or by a local wireless network, such as, e.g. WiFi, etc. This allows the off-board control station <NUM> to communicate with the on-board control unit <NUM> located on the vehicle <NUM>. The off-board control station <NUM> may also be arranged in, or connected to, a remote control interface <NUM>. The remote control interface <NUM> may comprise a manual work station <NUM> and at least one display <NUM>. The off-board control station <NUM> may be arranged to communicate with the manual work station <NUM> and the at least one display <NUM> over one or more communication links <NUM>.

Thus, even though the embodiments herein for enabling a safe operation of a vehicle <NUM> are described with respect to the example of working machines according to <FIG>, the embodiments herein should not be considered restricted to this particular type of vehicle but may also be used in other types of vehicles, such as, e.g. a conventional passenger cars, or heavy-duty vehicles, such as. , e.g. semi-trailer vehicles or trucks for cargo transport. The on-board control station <NUM> and the off-board control station <NUM> are described below in more detail in reference to <FIG>.

As part of the developing of the embodiments described herein, it has been realized that, in the event that there are several remote controlled vehicles operating within the same remote control system, it is highly important from safety perspective that the camera or video feed presented to an operator of remote control station is originating from the same vehicle that is currently being remotely operated. Although this problem may become moot or trivial once the remote controlled vehicle is in remote operation, an accident may still occur during start-up if the camera or video feed is not corresponding to the vehicle that is being remotely operated, i.e. the camera or video feed is coming from the another vehicle. In such cases, an operator may, for example, aggressively move his controls in order to see a reaction from the remote controlled vehicle in the camera or video feed, since the operator is not witnessing any remote operations being carried out via the camera or video feed in response to the operator's control input. This may cause damages to the remote controlled vehicle (or property and/or people in its vicinity) that is actually receiving the control input, and is thus moving accordingly. This issues are addressed in the embodiments of the on-board control unit <NUM> and off-board control station <NUM> described below with reference to <FIG>.

One particular use case of the embodiments herein is for tele-operated vehicles, such as, e.g. trucks, excavators, wheel loaders and haulers, etc. Such vehicles may be operated in within a system or fleet with a large number of vehicles and several different remote control interfaces that may be located at several different locations. For various reasons, such as, e.g. partial automation, down-time, etc., the number of remote control interfaces does not necessarily correspond to the number of vehicles. Hence, it is a requirement that the vehicle needs to be able to switch remote control station, and likewise, the remote control stations need to be able to switch between different vehicles. Therefore, it is important that the operator is forced to verify that the operator actually controls the vehicle corresponding to their current video feedback, before they may begin operating or motioning the vehicle. Otherwise, there could be a high level of risk for accidents.

Examples of embodiments of a method performed by an on-board control unit <NUM> of a vehicle <NUM> for enabling a safe operation of the vehicle <NUM>, will now be described with reference to the flowchart depicted in <FIG> is an illustrated example of actions, steps or operations which may be taken by the on-board control unit <NUM> on-board the vehicle <NUM> in the system <NUM> shown in <FIG>. The method may comprise the following actions, steps or operations.

Action <NUM>. Optionally, the on-board control unit <NUM> may receive, from the off-board control station <NUM>, information indicating that the vehicle <NUM> is to be activated for operation. This means that the on-board control unit <NUM> may receive a start-up command to initiate remote operations of the vehicle <NUM> from an operator working at a remote control interface <NUM> of an off-board control station <NUM>. This may, for example, comprise beginning to transmit camera images, i.e. a camera or video feed, from one or more on-board cameras <NUM>, <NUM> of the vehicle <NUM> to the off-board control station <NUM>. The off-board control station <NUM> may then display the received camera images to the operator via the remote control interface <NUM> of the off-board control station <NUM>.

Action <NUM>. After the reception in Action <NUM>, the on-board control unit <NUM> may transmit a first activation code <NUM> to the off-board control station <NUM> in response to the received information. This means that the on-board control unit <NUM> may send an individual code or activation key representing the vehicle <NUM>, such as, e.g. a three-digit machine identity number or identity (ID), to the off-board control station <NUM>. Here, in some embodiments, the on-board control unit <NUM> may first embed a visual representation of the first activation code <NUM> in camera images obtained from one or more on-board cameras <NUM>, <NUM>. This means that individual code representing the vehicle <NUM> is transformed from a digital representation into a visual representation, i.e. hardcoded into the camera images from the one or more on-board cameras <NUM>, <NUM> of the vehicle <NUM>. The on-board control unit <NUM> may then transmit the camera images comprising the visual representation of the first activation code <NUM> to the off-board control station <NUM>. This means that the individual code representing the vehicle <NUM> is only transmitted in the form of image data to the off-board control station <NUM>.

Action <NUM>. The on-board control unit <NUM> receives, via a wireless communication interface <NUM>, a second activation code <NUM> from an off-board control station <NUM> capable of remotely operating the vehicle <NUM>. This means that an off-board control station <NUM> attempts to confirm to the on-board control unit <NUM> of the vehicle <NUM> that it is the vehicle <NUM> that is currently desired to be remotely controlled by an operator working at a remote control interface <NUM> of an off-board control station <NUM>. The second activation code <NUM> may, for example, be digital information or a digital representation of manual inputs received from the operator working at the remote control interface <NUM> of the off-board control station <NUM>. For example, the operator may manually input information corresponding to the visual representation of the individual code of the vehicle <NUM> embedded in the camera images that is being presented via the remote control interface <NUM> of the off-board control station <NUM>. The wireless communications interface <NUM> is described in more detail with reference to <FIG>.

Action <NUM>. After receiving the second activation code <NUM> in Action <NUM>, the on-board control unit <NUM> activates the vehicle <NUM> for operation in case the received second activation code correspond to a first activation code <NUM> comprised in the on-board control unit <NUM>. This means that the on-board control unit <NUM> has confirmed that the received second activation code <NUM> is corresponding to, e.g. is identical to, the first activation code <NUM>, e.g. by comparing the digital information representing the received second activation code with the digital information representing the first activation code <NUM> comprised in the on-board control unit <NUM>. Therefore, the on-board control unit may safely start up the remote control of the vehicle <NUM> to enable the operator working at a remote control interface <NUM> of an off-board control station <NUM> to remotely operate the vehicle <NUM>. However, it should also be noted that until the on-board control unit <NUM> has received the correct activation key, i.e. a second activation code <NUM> corresponding to the first activation code <NUM>, the on-board control unit <NUM> will ensure that the vehicle <NUM> does not respond to any other commands sent from the remote control interface <NUM> of the off-board control station <NUM>. This means that the vehicle <NUM> cannot be moved until the operator has confirmed that the camera images, i.e. image or video stream/feed, used for remote motion control of the vehicle <NUM> corresponds to the camera images being sent from the vehicle <NUM>.

Optionally, the on-board control unit <NUM> may initiate different operations depending on which type of vehicle <NUM> it is arranged in. According to some embodiments and in case the vehicle <NUM> is an autonomous vehicle, the on-board control unit <NUM> may activate the autonomous vehicle to begin performing one or more autonomous operations. This means that the on-board control unit <NUM> may be used to remotely start up a vehicle for autonomous operation, by giving the operator of the fleet of different autonomous vehicles a safe way of visually inspecting the surroundings of each vehicle before starting autonomous operation. Alternatively, in some embodiments and in case the vehicle <NUM> is a remote-controlled vehicle, the on-board control unit <NUM> may activate the remote-controlled vehicle to be remotely operated from the off-board control station <NUM>. This means that the on-board control unit <NUM> may be used to remotely start up a vehicle for remote controlled operation, by giving the operator of the fleet of different remote controlled vehicles a safe way of controlling the motion of each vehicle without any risk of remote controlling the wrong vehicle.

Action <NUM>. After the activation of the vehicle <NUM> in Action <NUM>, the on-board control unit <NUM> may transmit, to the off-board control station <NUM>, information indicating that the vehicle <NUM> has been activated for operation. This means that the on-board control unit <NUM> may inform the off-board control station <NUM>, and thus the operator working at a remote control interface <NUM> of an off-board control station <NUM>, that the vehicle <NUM> is now being operated via the off-board control station <NUM>. In some embodiments, this information may also be sent by changing the embedded visual representation of the individual vehicle code, i.e. the first activation code <NUM>, in the camera images from the one or more on-board cameras <NUM>, <NUM> of the vehicle <NUM>. For example, a background field in the visual representation may change from red to green once the on-board control unit <NUM> has activated the vehicle <NUM> for operation.

Examples of embodiments of a method performed by an off-board control station <NUM> for enabling a safe operation of a vehicle <NUM>, will now be described with reference to the flowchart depicted in <FIG> is an illustrated example of actions, steps or operations which may be taken by the off-board control station <NUM> in the system <NUM> shown in <FIG>. The method may comprise the following actions, steps or operations.

Action <NUM>. Optionally, the off-board control station <NUM> may initially receive, from an operator of the off-board control station <NUM>, manual input indicating that the vehicle <NUM> is to be activated for operation. This means, for example, that an operator working at a remote control interface <NUM> of an off-board control station <NUM> may indicate or inform the off-board control station <NUM> that the operator requests that the vehicle <NUM> is to be activated and start operating.

Action <NUM>. After receiving the manual input in Action <NUM>, the off-board control station <NUM> may transmit, to an on-board control unit <NUM>, information indicating that the vehicle <NUM> is to be activated for operation. This means, for example, that the off-board control station <NUM> may, in response to the request from the operator, indicate or inform the on-board control unit <NUM> of the vehicle <NUM> that an operator requests that the vehicle <NUM> is to be activated and start operating.

Action <NUM>. The off-board control station <NUM> receives, via a wireless communication interface <NUM>, a first activation code <NUM> from an on-board control unit <NUM>. This means, for example, that the on-board control unit <NUM> of the vehicle <NUM> may, e.g. in response to the information in Action <NUM>, transmit an individual code or activation key representing the vehicle <NUM> to the off-board control station <NUM>. The individual code or activation key, i.e. the first activation code <NUM>, may, for example, be a three-digit machine identity number or identity (ID) of the vehicle <NUM>. In some embodiments, the reception may be performed by the off-board control station <NUM> by receiving camera images originating from one or more on-board cameras <NUM>, <NUM> of the vehicle <NUM> comprising an embedded visual representation of the first activation code <NUM>. This means that as the off-board control station <NUM> receives camera images from the on-board cameras <NUM>, <NUM> from the on-board control unit <NUM>, the off-board control station <NUM> will then also receive the first activation code <NUM> because the first activation code will be comprised in the camera images in the form of image data embedded within the camera images. This is advantageous since it will ensure that the first activation code and the camera images are non-separable and fully integrated with each other. This will result in that the first activation code will uniquely assert that the camera images is specifically originating from the vehicle <NUM>, i.e. tag the camera images emanating from the on-board cameras <NUM>, <NUM> of the vehicle <NUM>.

The wireless communications interface <NUM> is described in more detail with reference to <FIG>.

Action <NUM>. After receiving the first activation code <NUM> in Action <NUM>, the off-board control station <NUM> indicates the received first activation code <NUM> to an operator of the off-board control station. This means, for example, that the off-board control station <NUM> may present the first activation code <NUM> to an operator working at a remote control interface <NUM> of an off-board control station <NUM>, thus providing a confirmation to the operator that the operator that he is attempting to activate a specific vehicle, i.e. the vehicle <NUM>. In some embodiments, the indication may be performed by the off-board control station <NUM> by displaying the camera images comprising an embedded visual representation of the first activation code <NUM> to the operator of the off-board control station <NUM>. This means, for example, that the operator may be presented with an immediate and non-ambiguous confirmation that the camera images that is currently being viewed by the operator is actually originating from the on-board cameras <NUM>, <NUM> of the vehicle <NUM>. Here, it should be noted that the first activation code <NUM> may also be displayed to the operator via a separate display or indication device, and not necessarily together with the camera images on the same display.

Action <NUM>. After the indication in Action <NUM>, the off-board control station <NUM> receives manual input indicating a second activation code <NUM> from the operator of the off-board control station. This means, for example, that an operator working at a remote control interface <NUM> of an off-board control station <NUM> may provide the off-board control station <NUM> with a second activation code <NUM> corresponding to the first activation code <NUM> indicated in Action <NUM>.

Action <NUM>. After receiving the manual input in Action <NUM>, the off-board control station <NUM> transmits, via the wireless communication interface <NUM>, the received second activation code <NUM> to the on-board control unit <NUM> of the vehicle <NUM>. This means, for example, that the off-board control station <NUM> may transmit the manual input, i.e. second activation code <NUM>, from the operator working at a remote control interface <NUM> of an off-board control station <NUM> that the operator has inputted in response to being presented with the first activation code <NUM>.

Action <NUM>. Optionally, the off-board control station <NUM> may receive, via the wireless communication interface, information from the on-board control unit <NUM> indicating that the vehicle <NUM> has been activated for operation. This means, for example, that if the manual input, i.e. second activation code <NUM>, from the operator working at a remote control interface <NUM> of an off-board control station <NUM> is determined by the on-board control unit <NUM> to correspond to the first activation code <NUM> in the on-board control unit <NUM>, then the on-board control unit <NUM> may provide information back to the off-board control station <NUM> confirming the activation of the vehicle <NUM>.

<FIG> shows a signalling diagram illustrating embodiments of the on-board control unit <NUM> and the off-board control station <NUM> in a system <NUM>.

Action <NUM>. Optionally, an operator working at a remote control interface <NUM> connected to the off-board control station <NUM> may request remote control of the vehicle <NUM>.

Action <NUM>. In response to the request in Action <NUM>, the off-board control station <NUM> may request remote control of the vehicle <NUM> from the on-board control unit <NUM>.

Action <NUM>. The on-board control unit <NUM> may continuously transmit camera images, i.e. stream images or video received from the one or more on-board cameras <NUM>, <NUM>, to the off-board control station <NUM> which may display the camera images to the operator via the display <NUM> of the remote control interface <NUM> connected to the off-board control station <NUM>.

Action <NUM>. Prior to activating the vehicle <NUM>, the on-board control unit <NUM> may embed a visual representation of the first activation code <NUM> of the vehicle <NUM> in the received camera images from the one or more on-board cameras <NUM>, <NUM>.

Action <NUM>. After the embedding in Action <NUM>, the on-board control unit <NUM> may transmit the camera images comprising the embedded visual representation of the first activation code <NUM> to the off-board control station <NUM> which may display the camera images to the operator via the display <NUM> of the remote control interface <NUM> connected to the off-board control station <NUM>.

Action <NUM>. In response to viewing the camera images via the display <NUM>, an operator may via the remote control interface <NUM> connected to the off-board control station <NUM> provide manual inputs corresponding to a second activation code <NUM>.

Action <NUM>. In response to receiving the manual input, the off-board control station <NUM> may transmit information indicating the second activation code <NUM> to the on-board control unit <NUM>.

Action <NUM>. Upon receiving the second activation code <NUM> in Action <NUM>, the on-board control unit <NUM> may compare the received second activation code <NUM> with the first activation code <NUM> of the vehicle <NUM>. If the received second activation code <NUM> match the first activation code <NUM>, the on-board control unit <NUM> may activate the vehicle <NUM> for remote control by the operator working at the remote control interface <NUM> connected to the off-board control station <NUM>. If the received second activation code <NUM> does not match the first activation code <NUM>, the on-board control unit <NUM> will not activate the vehicle <NUM> for remote control.

Action <NUM>. In case the on-board control unit <NUM> activated the vehicle <NUM> for remote control in Action <NUM>, the on-board control unit <NUM> may transmit information indicating the activation of the vehicle <NUM> to the off-board control station <NUM>. The off-board control station <NUM> may thus indicate this to the operator, e.g. via the display <NUM> of the remote control interface <NUM> connected to the off-board control station <NUM>.

<FIG> illustrates an example of a display <NUM> presenting camera images from the on-board cameras <NUM>, <NUM> to an operator working at a remote control interface <NUM> connected to an off-board control station <NUM>.

In a first stage A, the off-board control station <NUM> may control the display <NUM> to present a first graphics <NUM> allowing the operator to enter manual input indicating which vehicle the operator is requesting to control, such as, the vehicle <NUM>. In response to entering the manual input, the off-board control station <NUM> may send information to the on-board control unit <NUM> of the requested vehicle, e.g. the vehicle <NUM>. Optionally, this stage may be skipped and start with stage B below.

In a second stage B, the on-board control unit <NUM> may, e.g. in response to the information from the off-board control station <NUM>, transmit camera images from the on-board cameras <NUM>, <NUM> to the off-board control station <NUM>. In this stage B, the on-board control unit <NUM> may also embed a visual, graphical representation of a first activation code <NUM> of the vehicle <NUM> into the camera images being sent to the off-board control station <NUM>. Here, the visual, graphical representation of the first activation code <NUM> may comprise a colouring indicating that the vehicle <NUM> has not yet been activated, such as, e.g. red. Upon receiving the camera images, the off-board control station <NUM> may display the received camera images comprising the visual, graphical representation of the first activation code <NUM> on the display <NUM> to the operator.

In a third stage C, the off-board control station <NUM> may control the display <NUM> to present a second graphics <NUM> allowing the operator to enter manual input corresponding to the first activation code <NUM>, i.e. a second activation code <NUM>. In response to entering the manual input, i.e. the second activation code <NUM>, the off-board control station <NUM> may then send information to the on-board control unit <NUM> indicating the second activation code <NUM>.

In a fourth stage D, the on-board control unit <NUM> may in response to the second activation code <NUM> from the off-board control station <NUM> activate the vehicle <NUM> in case the received second activation code <NUM> correspond to the first activation code <NUM> in the on-board control unit <NUM>. Here, the on-board control unit <NUM> may also change the visual, graphical representation of the first activation code <NUM> to, for example, comprise a different colouring indicating that the vehicle <NUM> has been activated, such as, e.g. green.

To perform the method actions for enabling a safe operation of the vehicle <NUM>, an on-board control unit <NUM> of the vehicle <NUM> may comprise the following arrangement depicted in <FIG> shows a schematic block diagram of embodiments of the on-board control unit <NUM>. It should also be noted that, although not shown in <FIG>, known conventional features of an on-board control unit <NUM>, such as, for example, a connection to a power source, e.g. a battery or an electric grid, may be assumed to be comprised in the on-board control unit <NUM> but is not shown or described in any further detail in regards to <FIG>.

The on-board control unit <NUM> may comprise a processing circuitry <NUM> and a memory <NUM>. The processing circuitry <NUM> may be arranged to execute instruction sets stored in the memory <NUM>. The processing circuitry <NUM> may also comprise wireless communications interface <NUM>. According to some embodiments, the wireless communications interface <NUM> may comprise RF circuitry and baseband processing circuitry capable of transmitting and receiving a radio signal in a wireless communications network, e.g. a telecommunications network. Optionally, the wireless communications interface <NUM> may also be configured to use WLAN, UWB or Bluetooth wireless communications for the wireless communication. Also, the wireless communications interface <NUM> may also be arranged to be connected to and use an antenna <NUM> for the transmissions. The antenna <NUM> may be part of the on-board control unit <NUM> or located on the vehicle <NUM>. Here, the wireless communications interface <NUM> may further be adapted to receive camera images, i.e. image or video feeds, from one or more on-board cameras <NUM>, <NUM>. This may be performed wirelessly or via one or more wired connections (as shown in <FIG>). It should also be noted that some or all of the functionality described in the embodiments above as being performed by the on-board control unit <NUM> may be provided by the processing circuitry <NUM> executing instructions stored on a computer-readable medium, such as, e.g. the memory <NUM> shown in <FIG>. Alternative embodiments of the on-board control unit <NUM> may comprise additional components, such as, for example, an embedding module <NUM> and an activating module <NUM>, whereby each module may be configured and responsible for providing its dedicated functionality to support the embodiments described herein.

The on-board control unit <NUM> or processing circuitry <NUM> is configured to, or may comprise a wireless communications interface <NUM> configured to, receive, via a wireless communication interface <NUM>, a second activation code <NUM> from an off-board control station <NUM> capable of remotely operating the vehicle <NUM>. The on-board control unit <NUM> or processing circuitry <NUM> is also configured to, or may comprise the activating module <NUM> configured to, activate the vehicle <NUM> for operation in case the received second activation code <NUM> correspond to a first activation code <NUM> comprised in the on-board control unit <NUM>.

In some embodiments, the on-board control unit <NUM> or processing circuitry <NUM> may be configured to, or may comprise a wireless communications interface <NUM> configured to, receive, from the off-board control station <NUM>, information indicating that the vehicle <NUM> is to be activated for operation. In this case, the on-board control unit <NUM> or processing circuitry <NUM> may be configured to, or may comprise a wireless communications interface <NUM> configured to, transmit the first activation code <NUM> to the off-board control station <NUM> in response to the received information. Here, according to some embodiments, the on-board control unit <NUM> or processing circuitry <NUM> may be configured to, or may comprise the embedding module <NUM> configured to, embed a visual representation of the first activation code <NUM> in camera images obtained from one or more on-board cameras <NUM>, <NUM>. In this case, the on-board control unit <NUM> or processing circuitry <NUM> may be configured to, or may comprise a wireless communications interface <NUM> configured to, transmit the camera images comprising the visual representation of the first activation code <NUM> to the off-board control station <NUM>.

In some embodiments, the on-board control unit <NUM> or processing circuitry <NUM> may be configured to, or may comprise a wireless communications interface <NUM> configured to, transmit, to the off-board control station <NUM>, information indicating that the vehicle <NUM> has been activated for operation. Further, in some embodiments, the on-board control unit <NUM> or processing circuitry <NUM> may also configured to, or may comprise the activating module <NUM> configured to, in case the vehicle <NUM> is an autonomous vehicle, activate the autonomous vehicle to begin performing one or more autonomous operations. Optionally, in some embodiments, the on-board control unit <NUM> or processing circuitry <NUM> may also configured to, or may comprise the activating module <NUM> configured to, in case the vehicle <NUM> is a remote-controlled vehicle, activate the remote-controlled vehicle to be remotely operated from the off-board remote control station.

Furthermore, the embodiments for enabling a safe operation of the vehicle <NUM> described above may be at least partly implemented through one or more processors, such as the processing circuitry <NUM> in the on-board control unit <NUM> depicted in <FIG>, together with computer program code for performing the functions and actions of the embodiments herein. The program code mentioned above may also be provided as a computer program product, for instance in the form of a data carrier carrying computer program code or code means for performing the embodiments herein when being loaded into the processing circuitry <NUM> in the on-board control unit <NUM>. The computer program code may e.g. be provided as pure program code in the on-board control unit <NUM> or on a server and downloaded to the on-board control unit <NUM>. Thus, it should be noted that the on-board control unit <NUM> may in some embodiments be implemented as computer programs stored in a memory, e.g. in the memory <NUM> in <FIG>, for execution by processors or processing modules, e.g. the processing circuitry <NUM> in the on-board control unit <NUM> of <FIG>.

Those skilled in the art will also appreciate that the processing circuitry <NUM> and the memory <NUM>, i.e. computer readable storage unit, described above may refer to a combination of analog and digital circuits, and/or one or more processors configured with software and/or firmware, e.g. stored in a memory, that when executed by the one or more processors such as the processing circuitry <NUM> perform as described above. One or more of these processors, as well as the other digital hardware, may be included in a single application-specific integrated circuit (ASIC), or several processors and various digital hardware may be distributed among several separate components, whether individually packaged or assembled into a system-on-a-chip (SoC).

To perform the method actions for enabling a safe operation of a vehicle <NUM>, an off-board control station <NUM> may comprise the following arrangement depicted in <FIG> shows a schematic block diagram of embodiments of the off-board control station <NUM>. It should also be noted that, although not shown in <FIG>, known conventional features of an off-board control station <NUM>, such as, for example, a connection to a power source, e.g. a battery or an electric grid, may be assumed to be comprised in the off-board control station <NUM> but is not shown or described in any further detail in regards to <FIG>.

The off-board control station <NUM> may comprise a processing circuitry <NUM> and a memory <NUM>. The processing circuitry <NUM> may be arranged to execute instruction sets stored in the memory <NUM>. The processing circuitry <NUM> may also comprise wireless communications interface <NUM>. According to some embodiments, the wireless communications interface <NUM> may, for example, a communications interface or module connected via a communication link <NUM> to a communications server <NUM> of a wireless communications network <NUM> (as shown in <FIG>). It should however be noted that other wireless network configurations also may be implemented in the off-board control station <NUM>, wherein the wireless communications interface <NUM> may also be configured to use WLAN, UWB or Bluetooth wireless communications for the wireless communication. Here, the wireless communications interface <NUM> may further be adapted to receive camera images, i.e. image or video feeds, from one or more on-board cameras <NUM>, <NUM> of the vehicle <NUM>. It should also be noted that some or all of the functionality described in the embodiments above as being performed by the off-board control station <NUM> may be provided by the processing circuitry <NUM> executing instructions stored on a computer-readable medium, such as, e.g. the memory <NUM> shown in <FIG>. Alternative embodiments of the off-board control station <NUM> may comprise additional components, such as, for example, an indication module <NUM>, whereby the indication module <NUM> is configured and responsible for providing its dedicated functionality to support the embodiments described herein.

The off-board control station <NUM> or processing circuitry <NUM> is configured to, or may comprise the wireless communications interface <NUM> configured to, receive, via a wireless communication interface <NUM>, a first activation code <NUM> from an on-board control unit <NUM>. The off-board control station <NUM> or processing circuitry <NUM> is also configured to, or may comprise the indication module <NUM> configured to, indicate the received first activation code <NUM> to an operator of the off-board control station <NUM>. Further, the off-board control station <NUM> or processing circuitry <NUM> is configured to, or may comprise the wireless communications interface <NUM> configured to, receive manual input indicating a second activation code <NUM> from the operator of the off-board control station <NUM>. Additionally, the off-board control station <NUM> or processing circuitry <NUM> is configured to, or may comprise the wireless communications interface <NUM> configured to, transmit, via the wireless communication interface <NUM>, the received second activation code <NUM> to the on-board control unit <NUM> of the vehicle <NUM>.

In some embodiments, the off-board control station <NUM> or processing circuitry <NUM> may be configured to, or may comprise the wireless communications interface <NUM> configured to, receive, from the operator of the off-board control station <NUM>, manual input indicating that the vehicle <NUM> is to be activated for operation. In this case, according to some embodiments, the off-board control station <NUM> or processing circuitry <NUM> may be configured to, or may comprise the wireless communications interface <NUM> configured to, transmit, to the on-board control unit <NUM>, information indicating that the vehicle <NUM> is to be activated for operation. Further, in some embodiments, the off-board control station <NUM> or processing circuitry <NUM> may be configured to, or may comprise thewireless communications interface <NUM> configured to, receive, via the wireless communication interface <NUM>, information from the on-board control unit <NUM> indicating that the vehicle <NUM> has been activated for operation.

Also, in some embodiments, the off-board control station <NUM> or processing circuitry <NUM> may be configured to, or may comprise the wireless communications interface <NUM> configured to, receive the first activation code <NUM> by receiving camera images originating from one or more on-board cameras <NUM>, <NUM> of the vehicle <NUM> comprising an embedded visual representation of the first activation code <NUM>. In this case, according to some embodiments, the off-board control station <NUM> or processing circuitry <NUM> may be configured to, or may comprise the indication module <NUM> configured to, indicate the first activation code <NUM> by displaying the camera images comprising an embedded visual representation of the first activation code <NUM> to the operator of the off-board control station <NUM>. In some embodiments, this may be performed using a display <NUM> connected to the off-board control station <NUM>.

Furthermore, the embodiments for enabling a safe operation of a vehicle <NUM> described above may be at least partly implemented through one or more processors, such as the processing circuitry <NUM> in the off-board control station <NUM> depicted in <FIG>, together with computer program code for performing the functions and actions of the embodiments herein. The program code mentioned above may also be provided as a computer program product, for instance in the form of a data carrier carrying computer program code or code means for performing the embodiments herein when being loaded into the processing circuitry <NUM> in the off-board control station <NUM>. The computer program code may e.g. be provided as pure program code in the off-board control station <NUM> or on a server and downloaded to the off-board control station <NUM>. Thus, it should be noted that the off-board control station <NUM> may in some embodiments be implemented as computer programs stored in a memory, e.g. in the memory <NUM> in <FIG>, for execution by processors or processing modules, e.g. the processing circuitry <NUM> in the off-board control station <NUM> of <FIG>.

The description of the example embodiments provided herein have been presented for purposes of illustration. The description is not intended to be exhaustive or to limit example embodiments to the precise form disclosed, and modifications and variations are possible in light of the above teachings or may be acquired from practice of various alternatives to the provided embodiments. The examples discussed herein were chosen and described in order to explain the principles and the nature of various example embodiments and its practical application to enable one skilled in the art to utilize the example embodiments in various manners and with various modifications as are suited to the particular use contemplated. The features of the embodiments described herein may be combined in all possible combinations of methods, apparatus, modules, systems and computer program products. It should be appreciated that the example embodiments presented herein may be practiced in any combination with each other.

It should also be noted that the various example embodiments described herein are described in the general context of method steps or processes, which may be implemented in one aspect by a computer program product, embodied in a computer-readable medium, including computer-executable instructions, such as program code, executed by computers in networked environments. A computer-readable medium may include removable and non-removable storage devices including, but not limited to, Read Only Memory (ROM), Random Access Memory (RAM), compact discs (CDs), digital versatile discs (DVD), etc. Generally, program modules may include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Computer-executable instructions, associated data structures and program modules represent examples of program code for executing steps of the methods disclosed herein.

Claim 1:
A method performed by an on-board control unit (<NUM>) of a vehicle (<NUM>) for enabling a safe operation of the vehicle (<NUM>), wherein the method comprise:
receiving (<NUM>), from an off-board control station (<NUM>), information indicating that the vehicle (<NUM>) is to be activated for operation;
transmitting (<NUM>) a first activation code (<NUM>) to the off-board control station (<NUM>) in response to the received information by embedding a visual representation of the first activation code (<NUM>) in camera images obtained from one or more on-board cameras (<NUM>, <NUM>) and transmitting the camera images comprising the visual representation of the first activation code (<NUM>) to the off-board control station (<NUM>);
receiving (<NUM>), via a wireless communication interface (<NUM>), a second activation code (<NUM>) from an off-board control station (<NUM>) capable of remotely operating the vehicle (<NUM>); and
activating (<NUM>) the vehicle (<NUM>) for operation in case the received second activation code correspond to the first activation code (<NUM>) comprised in the on-board control unit (<NUM>).