In-memory protection for controller security

In one implementation, a method for providing security on controllers includes detecting computer-readable code running on a controller, the computer-readable code including code portions that each include instructions to be performed by the controller; identifying a current code portion of the computer-readable code; accessing an in-memory graph that models an operational flow of the computer-readable code, wherein the in-memory graph includes a plurality of nodes, each of the nodes corresponding to one of the code portions and each of the nodes having a risk value for the associated code portion that is a measure of security risk for the associated code portion; identifying the risk value for the current code portion; selecting, from a plurality of available flow control integrity (IMV) schemes, an IMV scheme based on the identified risk value; and applying, to the code portion as the code portion is running on the controller, the selected IMV scheme.

TECHNICAL FIELD

This specification generally relates to security for computer-based controllers, such as controllers for Internet of Things (IoT) devices.

BACKGROUND

More devices are becoming “smarter” with hardware and software that permit them to communicate via the internet, such as through cellular wireless networks, Wi-Fi, and Bluetooth. These internet-connected devices are often identified as being part of the “Internet of Things” (IoT), which is a term that broadly encompasses internet-connected devices configured to transmit and receive information related to their operation, such as status information. For example, many consumer products are now IoT devices with internet-connected features, such as home automation devices (e.g., wirelessly controllable light switches), appliances (e.g., smart refrigerators able to transmit images of the fridge's contents), and automobiles (e.g., internet-connected components, such as infotainment and navigation devices). For instance, modern vehicles can have over 100 controllers, or Electronic Control Units (ECUs), that are responsible for running most of the car's functions, such as the steering wheel, engine, braking system, airbags, and navigation systems.

Like any other externally connected computers, IoT devices (e.g., ECUs in connected cars) are vulnerable to cyber attack and have become targets for hackers. For example, controllers on several makes and models of cars, such as the JEEP CHEROKEE, TOYOTA PRIUS, TESLA MODEL S, and NISSAN LEAF, have been successfully targeted and exploited by white hat hackers. Those hackers were able to compromise the vehicles and take command of nearly all of the control aspects, ranging from turning on the radio and windshield wipers to killing the engine while the car drove on the freeway. These exploits caused some of these car manufacturers to issue a recall on affected vehicles.

Cyber attacks come in many forms and flavors, but they generally share the same basic concepts: find a preexisting security bug (vulnerability) in the system's software, exploit it, and run malware. A common security bug is neglecting to verify the size of input buffers, which hackers can exploit by passing long buffers that get out of the boundaries allocated for that buffer on the software stack. By getting out of the buffer boundaries, hackers may be able to access and change the pointer structure that controls the functional flow of code, which hackers can use to direct the controller to execute malware code. Although malware code can vary (e.g., keylogger, ransomware, e-mail spam), the exploitation mechanism is often similar—find a security bug, research and learn how to exploit it in order to gain control, and use the control to run the malware code.

SUMMARY

This document generally describes a technological solution that hardens externally connected controllers (e.g., ECUs) within an IoT device (e.g., connected automobile) against hackers. Customized security policies for controllers can be generated and added to controllers with security layers, which can use the security policies and layers to ensure that only valid code and valid behaviors are allowed to run on the controllers. Various features can be used to ensure controllers are limited to operating with valid code and valid behaviors, such as using whitelists that define permitted code, permitted contexts within which the code can run, and permitted relationships between code segments, such as code that is permitted to call and/or return to other code segments.

For example, ECUs on a car can each include a custom security policy that defines conditional in-memory validation (IMV) schemes to be applied when executing software instructions. Generally, in the process of IMV, software instructions are changed to include special control commands that guarantee that each function call jump and returns only to a predefined list of known addresses. Jumping to a different address can be interpreted as an attack and the protection mechanism blocks this from happening. Conditional IMV can permit for differing levels of IMV to be applied to appropriate code segments based on any of a variety of factors, such as risk levels determined for code segments.

In one implementation, a method for providing security on controllers includes identifying a risk level of a code portion of computer-readable code running on a controller; and applying conditional in-memory validation to the computer-readable code based on the identified risk level.

In another implementation, a method for providing security on controllers includes detecting computer-readable code running on a controller, the computer-readable code being stored in a memory in a plurality of code portions, each code portion including one or more instructions to be performed by the controller; responsive to detecting the computer-readable code running on the controller, identifying a current code portion of the computer-readable code that is running; accessing an in-memory graph that models an operational flow of the computer-readable code, wherein the in-memory graph includes a plurality of nodes, each of the nodes corresponding to one of the code portions and each of the nodes having a risk value for the associated code portion that is a measure of security risk for the associated code portion; identifying the risk value for the current code portion; selecting, from a plurality of available flow control integrity (IMV) schemes, an IMV scheme based on the identified risk value; applying, to the code portion as the code portion is running on the controller, the selected IMV scheme; and responsive to a determination that the computer-readable code running on the controller fails the selected IMV scheme, taking a corrective action.

Such a method can optionally include one or more of the following features, which can be combined in each possible sub-combination of features. The method can further include analyzing the computer-readable code to determine the operational flow of the computer-readable code; generating the in-memory graph from the analysis of the computer-readable code; and providing the in-memory graph to the controller for later access. The method can further include analyzing the computer-readable code to determine the operational flow of the computer-readable code. Generating the in-memory graph from the analysis of the computer-readable code can further include generating the risk values from the analysis of the computer-readable code to determine the operational flow of the computer-readable code. The plurality of available IMV schemes can consist of three schemes and the risk values consist of three risk values. The selected IMV scheme can specify that no IMV checking should be done and wherein applying the IMV scheme includes performing no IMV checking to the instructions of the current code portion. The selected IMV scheme can specify that function validation should be done. Applying the IMV scheme can include verifying that memory locations referenced by instructions of the current code portion contain the computer-readable code. Verifying that memory locations referenced by instructions of the current code portion contain the computer-readable code can include determining that a memory location referenced by an instruction of a current code portion is configured to cause control of the computer-readable code to move to a memory address that does not contain a portion of the computer-readable code. The selected IMV scheme can specify that memory addresses should be verified and wherein applying the IMV scheme includes verifying memory address referenced by instructions of the current code portion. Verifying memory address of instructions of the current code portion can include accessing a predefined list of approved destinations for the instructions of the current code portion; and determining that a memory address referenced by an instruction of a current code portion is configured to cause control of the computer-readable code to move to a memory address that is not included in the predefined list. Applying, to the code portion as it is running on the controller, the selected IMV scheme can include analyzing return calls in the instructions of the current code portion. The method further can further include setting watchpoints in the memory on unallocated addresses; detecting control of the running computer-readable code at one of the watchpoints; and responsive to detecting control of the running computer-readable code at one of the watchpoints, applying a corrective action.

In another implementation, a method for providing security on controllers includes initiating static analysis on computer-readable code that is programmed to be run by a controller to control operation on the controller; building, by a computer system, an in-memory graph that includes nodes representing distinct code portions within the computer-readable code and edges representing calls between the distinct code portions; identifying, by the computer system, first code portions from among the distinct code portions based on the first code portions each performing at least one memory modification operation; identifying, by the computer system using the in-memory graph, second code portions from among the distinct code portions based on each of the second code portions calling, either directly or indirectly, one or more of the first code portions; identifying, by the computer system using the in-memory graph, third code portions from the distinct code portions that are not included in either the first or second code portions; determining, by the computer system, risk levels for each of the distinct code portions based on their corresponding designation as either first, second, or third code portions; and recording the risk-level determinations for each of the code portions in a security policy for the controller, the security policy being used to implement conditional IMV by the controller for each of the code portions.

Such a method can optionally include one or more of the following features, which can be combined in each possible sub-combination of features. The distinct code portions can include functions in the computer-readable code. The method can further include tagging, by the computer system, first nodes in the in-memory graph that correspond to the first code portions as being the first code portions. Identifying the second code portions can include performing a reverse walk through the in-memory graph emanating from each of the first nodes and moving backward along directional edges indicating a code portion calling another code portion, wherein each node traversed in the reverse walk is identified as a second node that corresponds to one of the second code portions. The method can further include tagging, by the computer system, the second nodes as being the second code portions. Identifying the third code portions can include identifying third nodes in the in-memory graph that are not yet tagged as either first nodes or second nodes, the third nodes corresponding to the third code portions. The first code portions can be determined to have a high-risk level, the second code portions can be determined to have a medium-risk level, and the third code portions can be determined to have a low risk-level. The security policy, when implemented at runtime by the controller, can cause the controller to perform a full-IMV scheme for the first code portions, the controller to perform a partial-IMV scheme for the second code portions, and the controller to perform a no-IMV scheme for the third code portions. Performing the full-IMV scheme can include verifying memory locations for calls to other code portions and one or more addresses to which the current code portion returns processing flow. Performing the partial-IMV scheme can include verifying that calls to other code portions are function calls. Performing the no-IMV scheme can include not performing any IMV checks for the current code portion. The first code portions can be identified based on each of the first code portions each performing at least one memory modification operation that uses one or more values dynamically-determined at runtime.

In another implementation, a method for providing security on controllers includes detecting computer-readable code running on a controller, the computer-readable code being stored in a memory in a plurality of code portions, each code portion including one or more instructions to be performed by the controller; responsive to detecting the computer-readable code running on the controller, identifying a current code portion of the computer-readable code that is running; determining whether to dynamically set one or more watchpoints on the controller with regard to the current code portion, the determination being made before performing the current code portion; in response to determining to dynamically allocate the one or more watchpoints, identifying one or more locations in memory related to the current code portion at which to set the one or more watchpoints; setting, by the controller, the one or more watchpoints at the one more locations in memory; and running, by the controller, the current code portion with the one or more watchpoints dynamically set at the one or more locations.

Such a method can optionally include one or more of the following features, which can be combined in each possible sub-combination of features. The method can further include detecting, by the controller, that one or more of the watchpoints have been triggered; and applying, by the controller, corrective action with regard to the triggered watchpoint. The corrective action can include transmitting an alert to a remote computer system that the watchpoint has been triggered. The determination of whether to dynamically set the one or more watchpoints can be made based on a risk level associated with the current code portion. The risk level associated with the current code portion can be determined based on, at least, whether the current code portion allocates memory or calls other code portions that allocate memory. The determination of whether to dynamically set the one or more watchpoints can be made based on a current condition on the controller. The risk level associated with the current code portion can be determined based on, at least, whether the current code portion modifies memory using one or more values dynamically-determined at runtime or calls other code portions that modify memory using one or more values dynamically-determined at runtime.

Certain implementations can provide one or more of the following advantages. For example, the operations of IoT devices may be made more secure by applying special techniques to the execution of software on a controller of the IoT device. By using varying levels of security through conditional IMV, for example, controllers can scale resources that are dedicated to security to correlate to the current known security threat to the controller. This can permit controllers to conserve resources (e.g., memory, processor cycles, network bandwidth) that are used to ensure secure controller operation and to allocate additional resources when increased risks are present. Conditional IMV can be used to allocate resources (e.g., apply resources differing levels of IMV-type checks) to areas of code in which there are greater risks of insecure operations. For example, by identifying portions of an in-memory graph that are likely to present a low security risk, such as portions that do not modify (e.g., write to) and/or allocate any memory in stack and which call no functions that modify and/or allocate memory in stack, those portions of the in-memory graph may be pruned from IMV operation, eliminating the need to use IMV-related overhead to safe portions of code. In another example, by identifying portions of an in-memory graph that are likely to present a moderate security risk, a partial IMV operation (as opposed to a full IMV operation) can be performed that uses fewer resources than a full IMV operation, such as validating only that the caller is a function, and not the address for the function. Such partial IMV operations can, for example, strike a balance between risk and resource usage-permitting some types of functions that pose a smaller security risk to have some protection from in-memory attacks without the processing and resource overhead needed to perform full IMV validation (e.g., validating addresses of functions).

In another example, security risks can be dynamically determined based on changing conditions on a controller and can be used to dynamically adjust the level of security provided under a conditional IMV scheme. For example, in the event that the controller is operating in a safe mode of operation, which may occur when the controller or other controllers have blocked a malware or hacking attempt, risk levels associated with one or more types of functions can be dynamically increased, which can cause additional resources may be allocated to implement full IMV features to ensure secure operation of the controller in the face of a current security threat. For instance, by controllers notifying each other of an attack attempt (e.g., ECUs notifying each other of an attack on a vehicle), not only can the security of the controller under attack be increased during the attack, but the overall security of all controllers can be increased during the attack.

Additional and/or alternative advantages are also possible, as described below.

DETAILED DESCRIPTION

This document generally describes conditional in-memory validation (IMV) schemes to provide enhanced security for controllers. In general, IMV schemes can provide some guarantees that function calls and returns are addressed only to safe or expected memory addresses. Conditional IMV schemes include multiple different and varying levels of IMV security that can be applied for different code and operating conditions, such as risk levels posed by code segments (e.g., functions) and/or current controller context (e.g., current mode of operation). For example, a conditional IMV scheme can include multiple different levels of IMV security that are selected and applied to particular code segments based on levels of security risk posed by the code segments. The different levels of IMV security can use different levels of controller resources (e.g., processor cycles, memory, disc reads/writes), with higher levels of IMV security using more controller resources than lower levels of IMV security. By selecting appropriate levels of IMV security from a conditional IMV security scheme to correspond to the likely security risk posed by code segments, an appropriate balance can be struck between IMV-related overhead (e.g., controller resource usage) and security risks posed by code segments to controller security. For example, the IMV security level applied to code segments can be reduced in safe conditions (e.g., low risk posed by the code segments) while additional resources can be used to implement higher levels of IMV security for riskier conditions.

Security risks posed by portions of code (e.g. individual functions, branches in an in-memory graph) can be determined based on a variety of factors, such as based on analysis of the specific code portions and/or the context of the controller, more generally. For example, specific code portions can be analyzed to determine the risk of exposure to in-memory attacks. Indicators of risk can include, for example, the modification of memory, the type of memory that is being modified (e.g., modification using statically-defined value at compile time, modification using value dynamically during runtime through runtime calculation or input data), the allocation of memory, calling of other functions, and/or other risk indicators. For instance, functions that do not modify memory, allocate memory, and/or call other functions can be considered present a low security risk, while functions that modify memory, allocate memory, call other functions, and/or jump to other memory addresses can be classified with one or more levels of heightened riskiness. No validation or lesser levels of validation can be applied to relatively safe (low security risk) portions of code, while greater levels of validation can be applied to relatively riskier portions of code.

Additionally and/or alternatively, the risk of the operating environment on controllers can be monitored and used to dynamically adjust the level of IMV security applied by some or all code segments on the controller, with greater validation generally being applied when the environment is comparatively more risky. For example, controller context (e.g., current mode of operation, current security status) can be monitored and used to determine the level of risk on the controller, which can be used to dynamically adjust the level of IMV security for some or all functions on the controller. For instance, IoT devices may generate reports for attempted or successful intrusions including in-memory attacks. These reports can be aggregated and used to determine if a broad series of attacks are being leveled against the IoT devices. If, from those aggregated reports, it is determined that attacks are being leveled, the controllers of the IoT devices can be notified and operate in a secure mode (applied when the device is under attack) with greater IMV security level protection than the default IMV security level that would otherwise be applied when operating under a normal mode of operation (when the device is not under attack).

FIG. 1Ais a conceptual diagram of an example system100for generating and implementing a custom context-based security policy on an example controller using modes of operation. The example system100includes a policy generation computer system104(e.g., computer server system, cloud computing system, client computing device) that is programmed to generate a custom security policy for a controller, an example IoT device112(e.g., ECU) that includes an example controller114that will use the generated security policy to operate securely and to prevent malware, and a management computer system122(e.g., computer server system, cloud computing system, client computing device) that is programmed to receive real-time controller information, to detect anomalous controller behavior, and to provide an interface for users to view real-time controller/device status information. Although not depicted, the system104, the IoT device112, and the system122can communicate over one or more communication networks, such as the internet, local area networks (LAN), wide area networks (WAN), virtual private networks (VPN), controller area networks (CAN), wired networks, wireless networks, mobile data networks, or any combination thereof.

The policy generation computer system104can receive controller software102, which can include an operating system and/or applications that are to be run on a controller. The policy generation computer system104can use the controller software to automatically generate a custom context-based security policy108(with varying security levels for different modes of operation) for the controller that is to execute the software102, as indicated by step A (106). For example, the computer system104can analyze the software102to determine an in-memory graph for the software that describes the flow of the software as it is being executed. The generation of this in-memory graph may be a static analysis of the code that generally does not change over the time software102is executing. The in-memory graph may be generated on a file-by-file basis, with addresses of instructions being recorded relative to the file's eventual location in memory, for example by using an offset value from the head of the file. Further, the computer system104can identify an inherent risk level for elements of that graph based on any of a variety of factors, such as memory modifications (e.g., write operation), types of memory modifications (e.g., modification using statically-defined values, modification of dynamically-defined values), memory allocation, function calls, jumps, and/or other features within each of the elements.

In this flow control graph, for example, each node may represent a function (or other logical unit) of the software, with edges of the graph representing calls to other functions. The functions of the software may be examined to identify operations that provide some risk of in-memory attack (e.g., memory modifications, memory allocation, decryption and execution of encrypted code). Based on this examination, a risk level may be applied to branches with the node corresponding to the function. For example, if the function is found to have no operations that do not present risks of an in-memory attack, the node may be given the lowest risk level (e.g., “safe,” a 0 rating). If the function is found to have operations that present significant risks of an in-memory attack, the node may be given a greater risk level (e.g., “risky,” a 1 rating). Other risk levels (e.g., moderate risk, scores between 0 and 1) can also be determined, and/or other schemes for the in-memory graph may be used. For example, each node may represent a basic block of code without a memory jump, or may represent a single instruction. The flow control graph can be used to implement conditional IMV security on the controller114and the IoT device112, more broadly.

The computer system104can assemble the custom security policy108, which may include the in-memory graph with the risk ratings, as well as other data. For example, the custom security policy may specify multiple different IMV schemes that can conditionally be applied to portions of the controller software102based on factors, such as, code risk levels and/or a current context for the controller114and/or the IoT device112. These different IMV schemes may involve different way of validating in-memory operations. Often, different IMV schemes will provide different trade-offs in terms of efficiency and security. For example, in some testing, it was found that one IMV scheme providing greater levels of security on a controller created a 20% increase in overhead while a lighter, less robust, and less secure IMV scheme was found to create a 6% increase in overhead. By selectively applying different IMV schemes on the controller114as appropriate (e.g., applying IMV schemes that correspond to risk levels), a controller's security and efficiency may be balanced and improved according to the operating needs of the controller in different circumstances.

Generating the security policy can additionally include generating one or more signatures for components of the controller software102, such as processes/functions that are part of the software102, and that can be used to verify that the code being executed as part of the software102is authentic and has not been modified/altered/replaced by malware. By automatically generating a security policy108from the controller software102—meaning without needing manual design for implementation/generation—the system100can reduce the burden, cost, and time to generate and implement security layers on controllers, which can increase controller security.

The policy generation can be performed by the computer system104in a way that does not necessitate any sort of modification to the controller software102. For example, the custom policy108can be separate from and not rely on modification of the software102in order to operate. By generating and implementing the security policy108without having to modify or alter the controller software102, the system100can additionally reduce the burden on security layer implementation, which can increase security layer implementation and overall controller security. For example, if the controller software102were to be modified in significant ways in order to incorporate the security policy108, the software102would need to be verified and tested again after the security policy108has been integrated into the system, which can slow time to deployment and can delay the incorporation of security layers on controllers.

The computer system104(and/or other computer systems, such as original equipment manufacturers (OEM)) can load the software102and the security policy108for the controller114of the IoT device112, as indicated by step B (110). For example, the controller software102and the security policy108can be flashed onto the controller114.

The controller114can securely operate using the controller software102, which is confined to operating within the confines of the security policy108, as indicated by step C (116). For example, the security policy108can include rules outlining various security features to use during particular operating conditions within the controller114, such as implementing different IMV schemes depending on assessed risk levels. For instance, the controller114can determine risk levels for particular functions based on a variety of factors (e.g., risk level assessed from in-memory graph analysis) and can apply a variable level of IMV checking based on the determined risk levels. If, for example, a particular function is determined to be “safe” and have a low risk level—meaning that there is a low likelihood that the particular function could pose a security vulnerability—the controller114can execute the particular function without IMV checking. If the function is determined to have a high risk level—meaning that the function includes one or more features that could be exploited and pose a security risk to the controller114—the controller114can execute the function with one or more levels of IMV checking.

Such hardening of the controller114—meaning using conditional levels of IMV checking by the controller114as outlined in the security policy108, which can outline and implement other security features on the controller114—can provide memory security that provides a variety of benefits. For example, it can identify and prevent attacks before they are able to install/run malware120on the controller114because the controller is prevented from reading or executing instructions that are outside of the memory ranges in which the control software102is loaded.

The controller114can log information about its operation, including blocked out-of-range attempts as well as information on secure operation of the controller114over time, including contexts for the controller114and the device112while various processes are executed, and the mode of operation that the controller114and/or the device112were operating in. Traces of blocked malware attempts can include a variety of information, such as the malware itself, the origin of the malware (e.g., IP address from which the malware originated), the context of the device112and/or controller114when the malware attempt was blocked, the mode of operation for the device112and/or the controller114, and information identifying the code segment that provided the malware exploit. The controller114can report information on controller operation, as indicated by step E (124). Such reporting can be provided in real-time. For example, the controller114can report malware traces in response to the malware120attempt being blocked. The controller114can balance reporting with controller performance against the timeliness of reporting for less critical information, such as information about secure operation of the controller114during periods of time when no malware attacks were attempted/blocked. For instance, such reports can be delayed until periods of time when the controller114and/or the device112have at least a sufficient amount of processing capacity and/or network bandwidth available.

The management computer system122can receive reports from the controller114as well as from multiple other controllers and devices, and can aggregate the reports into a central database system. The reports can be used to provide real-time controller/device information, as indicated by step E (126). For example, the computer system122can transmit real-time information that is presented on client computing devices (e.g., mobile computing devices, laptops, desktop computers) in user interfaces, such as the example user interface130that includes status information132for example controllers C1-C6and malware information134that identifies particular malware that has been blocked by these controllers, as well as other information (e.g., device/controller context when the malware was blocked). The real-time information can be at any of various levels of granularity, such as a device-level (status information for a specific device) and/or a population-level (status information across multiple devices/systems).

The computer system122can additionally use the information reported by controllers to detect anomalies, as indicated by step E (128). For example, the computer system122can use statistical analysis to identify operation/behaviors that are outside of the normal operation of a controller, such as identifying a particular context for a particular process that is a statistical outlier outside of the normal operation of a controller.

FIG. 1Bis a conceptual diagram of an example system150for generating and implementing custom context-based security policies on example ECUs that are part of an example vehicle152. The example system150is an example implementation of the system100to a specific IoT context, which in this example is the vehicle152. The system100and the system150can be implemented on a variety of other IoT devices and systems.

In this example, the vehicle152includes a control system154that includes multiple ECUs156a-nthat each have their own custom security policy158a-n, which each define in-memory graphs and risk levels for the processes. Although not depicted, the security policies158a-ncan be generated in a similar manner described above with regard toFIG. 1Aand the policy generation computer system104. The security policies158a-ncan harden the ECUs156a-nand can effectively block malware attempts160a-n, which can be attempts by hackers to find a way into the CAN Bus of the vehicle152. While the vehicle152can include over a hundred ECUs connected to the CAN Bus, only a few may be open externally (accessible to external networks outside of the vehicle152, such as the internet). These external ECUs (e.g., ECUs156a-n) can be the gateways into the car and the security policies158a-ncan stop attackers at these gateways, which can significantly reduce, if not eliminate, the risk of attacks penetrating the car's network, which can disrupt the car's operation.

For example, the security policies158a-ncan identify portions of code to which IMV checking is to be applied and levels of IMV checking that are to be used, under various operating conditions, in order to ensure that out-of-range memory is not read or executed while at the same time minimizing the impact on controller performance. By doing so, malicious code may be prevented from running on the ECUs156a-nwhile having minimal performance impact on the ECUs156a-n. By using the security policies158a-nthat are specific to the ECUs156a-n, any processes or functions that are outside of the ECUs permitted/designed operating behavior can be immediately detected and stopped from running on the ECUs156a-n. This can allow for the ECUs156a-nto stop malicious code from ever being executed by and possibly taking control of an ECUs' operation.

For instance, hackers targeting the vehicle152can use a “dropper,” which is a small piece of code or operation, to try to exploit a vulnerability and implant the malware160a-n. The malware160a-nis the code that ultimately tampers with or takes control of the function of the vehicle152, which can cause significant damage and put the safety of the driver and others on the road at risk. By adding an endpoint security layers and policies158a-nto ECUs156a-nso that they use policies outlining conditional IMV checking, the ECUs156a-nare able to detect the unexpected behavior or operation of a dropper and immediately report on the attack attempt in real-time, as indicated by step162. The early warning can give the original equipment manufacturers (OEMs) and system providers of the vehicle152(and its subparts) time to address the threat, as indicated by the computer system164providing real-time status information to a client computing device168with information170on malware that has been blocked across the ECUs156a-n(step166). For example, an alert on the malware160a-ncan include the complete trail of the attack on the ECUs156a-n, including its source, path, and context of the vehicle152and/or ECUs156a-nwhen the attack was blocked, so vulnerabilities can be fixed and blocked to prevent any malware from infiltrating the CAN Bus on the vehicle152.

Dropper and other hacker attempts to introduce the malware160a-non the externally connected ECUs156a-ncan be detected by the endpoint security layers and policies158a-nas foreign code based on attempts to read or execute out-of-range instructions. Additionally, these attempts can be detected using conditional IMV checking so as to have minimal impact on the performance of the ECUs156a-n.

Endpoint security layers (e.g., security policy108, security layer and policies158a-n) can be implemented on newly deployed controllers and can be retrofitted on previously released controllers that may not have previously included security layers. Such retrofitting can improve the security of devices already in use and can be added as part of regular software updates that drivers receive during regular maintenance and updating. Once retrofitted, previously deployed controllers can be protected with endpoint security and will be hardened against the cyber threats targeting them.

FIG. 2is a diagram of an example system200for processing and providing controller security information. The example system200includes a controller202that can be similar to the controller114protected by security policy106and the ECUs156a-nprotected by security policies158a-ndescribed above with regard toFIGS. 1A-B.

The controller202includes an application layer224at which one or more applications operate on the controller202through use of an operating system226for the controller200. The operating system204includes a kernel238and the security middleware layer228, which can intercept commands from the application layer224to the kernel238for inspection, alteration, or prevention.

The kernel238includes processes and functions that provide an interface for the operating system226to perform operations on the controller202using hardware, which includes one or more processors204(e.g., CPUs), memory206(e.g., volatile memory, non-volatile memory, RAM), and input/output (I/O) network components222(e.g., wired and wireless network cards/chip sets, network interface cards (MC)). The kernel238includes functions/process that direct operation of the hardware, such as program loading (e.g., functions and processes to load processes into a software stack208in memory206for execution by the processor(s)204), in-memory services (e.g., functions that modify memory206, functions to allocate information into and out of memory206), networking services (e.g., processes to open network sockets and to transmit/receive network packets), and peripheral device processes (e.g., processes to interface with peripheral devices).

The security middleware layer228includes security agents232that can provide multiple different layers of security and that can implement various portions of the security policy230on the controller202. The security agents232can, for example, apply conditional IMV checking to implement different IMV schemes, as appropriate, to calls from the application later224to the kernel238. For calls that pass the conditional IMV checking, the security middleware layer228can permit the call to pass. As described above, for some functions no IMV checking will be performed and the security middleware layer228will permit the call to pass without any IMV check. However, other calls can have one or more levels of IMV checking applied depending on the determined risk level for the function, and will need to pass the determined IMV check in order for normal processing flow for the controller to continue (e.g., return processing flow to code that called a function under evaluation). For calls that fail the IMV checking, the security middleware layer228can take one or more corrective actions. These corrective actions can include block the call from reaching the kernel238, halting the application layer224code, generate alerts, and other actions.

The security middleware layer238includes a reporting agent234that can collect and report forensic information and alerts on security threats, such as malware dropping attempts, as well as information on normal operation of the controller202. The security middleware layer228can harden the controller202against malwares and other security threats, and can be integrated into the operating system226of the controller202, in kernel and system levels, which can include enforcement as well as reporting and forensics capabilities through the reporting agent234. For example, the security middleware layer228(and/or its individual components) can be registered as one or more drivers with the kernel238to be executed in response to various action being performed at a kernel level, such as particular functions that are part of the kernel processes being called.

The reporting agent234can incorporated into the security middleware layer228by, for example, being invoked/called by the security agents232whenever the security agents232block malware attacks, as indicated by step A (240), and/or at various intervals (e.g., time-based intervals, code/processing based intervals) whenever they determine that the controller is operating normally (no malware attack detected), as indicated by step B (242). The reporting agent234can collect forensic trace information on system workflows within the controller202. This collection can be automatically adjusted and optimized based on controller202performance, memory usage, and/or storage limitations. The reporting agent234can be designed to obtain and report relevant information, but to also do so while minimally impacting performance of the controller202. Periodically and upon attack attempts, the forensic information is reported to a server system (e.g., management computer system122, management computer system164) for reporting and further analysis and/or is used for generating alerts and providing the alerts to one or more other controllers (e.g., other controllers264a-n, in communication with the controller202over the CAN Bus262and the network260).

For example, the reporting agent234can automatically analyze attack attempts (blocked malware240), including identifying the attacker's entry point (exploit in the operating system226) and reporting that information to the vendor to be addressed and fix the vulnerability. The reporting agent234can further include an auditing agent236that is an internal component that collects activity traces, stores them in a queue216(e.g., compressed cyclic buffer) for transmission, and sends them, when needed, to the backend server system (management computer system122, management computer system164), which may reside on either a security provider's data center or at a vendor/manufacturer's data center.

For example, in response to receiving an indication that malware has been blocked (240) and/or that the security agents232have determined the controller202is operating normally (242), the reporting agent234can request the auditing agent236to obtain trace information, as indicated by step C (244). Obtaining trace information can involve the auditing agent236transmitting requests to the kernel238for information that is stored in memory206, including information contained within the software stack208indicating processes that are being performed by the controller202(as well as a sequence of processes that are being performed) and/or information that has been blocked by the security middleware layer228that is stored in one or more buffers210used by the controller202(e.g., blocked malware212, blocked network packets214).

The auditing agent236can additionally call to the kernel238to obtain information on the current state of the controller202, such as current resource usage (e.g., processor204usage, memory206usage, network transmission levels using the networking components222) and/or current network connections established by the network components222(e.g., Wi-Fi, cellular network).

The auditing agent236can call to the kernel238to obtain information on a current context within which the controller202currently exists/resides, which can be a broader external state beyond than the current internal state of the controller202. For example, the current context can include information about a device/system that the controller202is controlling (e.g., infotainment center in a vehicle), information about a broader system of which the controller202is a part (e.g., collection of ECUs that together provide control operations within a vehicle), and/or other appropriate information. Obtaining context information may include accessing contextual information sources250through the kernel238. Contextual information sources250may be local to the controller202or they may be external, such as being provided by one or more other controllers that are part of a system that the controller202is a part of (e.g., collection of ECUs in a vehicle). Such information can include, for instance, a current physical location (e.g., geolocation), a current operational state of the system (e.g., vehicle driving, vehicle idling), and/or other appropriate context information.

Having obtained relevant information, the reporting agent234and the auditing agent236can generate an alert (for blocked malware attempts), as indicated by step D (246). Alerts, for example, can be used to provide information to a backend system (management computer system122, management computer system164) about a current state of the controller202, which can be combined with information from other controllers to provide a global view of the security state of a population of controllers/devices. As another example, alerts can be provided over the network260to one or more other controllers264a-nthat are connected to the controller202by the CAN Bus262. In response to receiving one or more alerts218from the controller202, for example, another controller (e.g., one or more of the other controllers264a-n) may enter a safe mode in which operations are restricted to a subset of normal operations, until such time that any possible security breach can be resolved.

Generated alerts can be transmitted (e.g., to one or more other controllers and/or a backend system), as indicated by step E (248). For example, the alerts218can be loaded into the reporting queue216and the log entries220can be loaded into a reporting queue216. The reporting queue216can be designed to handle alerts218differently than log entries220, and may prioritize the transmission of the alerts218over the log entries220. For example, the reporting queue216can transmit the alerts218immediately upon receiving them and regardless of a current capacity of the network components222. In contrast, the log entries220(detailing normal behavior) can be entered into a buffer that is flushed at appropriate times, such as when the network components222have sufficient capacity to transmit the contents of the buffer. The buffer may have a limited or fixed size, and allow for non-transmitted log entries not yet transmitted to be overwritten with new log entries in the event that the network components222did not have sufficient capacity while the non-transmitted log entries were awaiting transmission in the buffer. Since log entries220are not as critical as the alerts218, losing some log entries220in order to reduce performance impacts on the controller202(e.g., minimize memory usage for log entries220, restrict network transmissions to times when the networking components222have sufficient capacity) can be a fair tradeoff. In contrast, alerts218can be prioritized and may not be dropped in lieu of system performance gains.

In addition to including a variety of information, such as trace information, controller information, and/or context information, the alerts218can include actual copies of the blocked malware212and/or blocked network packets214. Such information can be used by the backend system to better understand the security threat that was blocked and the exploit in the operating system226that permitted the security threat to reach the controller202. As another example, alerts218that may be provided to the other controllers264a-nmay exclude copies of the blocked malware212and/or blocked network packets214, as such information may not be used by the other controllers264a-n.

Although not depicted, the reporting agent234and the auditing agent236can additionally include features to compress the size of alerts and logs that are transmitted to the backend system. For example, the reporting agent234and the auditing agent236can compress the information being transmitted using one or more data compression techniques, such as through using a dictionary to abbreviate particular common bits of information with less information. Other compression techniques can also be used.

FIG. 3is an example diagram of an in-memory graph300of software stored in memory320, which can be used to assess risk levels for implementing conditional IMV on the software. For example, the in-memory graph300may be created as part of the security policy106generation for the controller software102by the computer system104, and/or can be created/monitored during runtime for the controller software102. The in-memory graph300may be used, for example, by the controller202, with the memory320being, for example, a part of the memory206. The in-memory graph300can be used, for example, to assess risk levels of various parts of the controller software102, which can be used to determine and implement an appropriate IMV scheme to be applied to the part of the controller software102during runtime.

The in-memory graph300can be created to reflect the order of operations that are possible with the software stored in the memory320. For example, the software can be stored in memory320and can include distinct portions302-318(e.g., functions). The in-memory graph300can be created to include nodes302-318that correspond to the portions302-318. The edges of the flow control graph300can represent jumps from one portion to another, such as functions calling each other. For example, control can flow from portion302to either portion304or306—meaning that code within portion302can call portions304and/or306. As such, there are edges in the flow control graph300from the node302to the nodes304and306. The flow control graph300can take any technologically appropriate form for use in a computing system. For example, the graph300may take the form of a list of edges, as an adjacency matrix, an adjacency list, and/or other appropriate data structure to represent the graph300.

Risk levels for each of the nodes302-318can be determined, in part, based on a variety of intrinsic factors within each of the code portions (e.g., operations that are performed by each of the corresponding code portions) and/or extrinsic factors for each of the code portions (e.g., relationships/connections with code portions). Risk levels assessed for the nodes302-318can indicate the risk of in-memory attack to the corresponding code portions302-318stored in memory320. In this example, there are three levels of risk, with 0 indicating low risk or no risk, and a 2 indicating the highest level of risk. These risk levels are determined, for example, by the computer system104based on an analysis of the instructions within the software portions302-318. In other examples, more or fewer levels of risk may be used, and they may be recorded using numbers or other indicia such as character strings.

Various heuristics may be used to identify the risk of in-memory attacks for each of the code portions302-318. One such heuristic can include the presence instructions that modify memory (e.g., writes operation), either directly in each code portion or by calling other code portions that modify memory. In such an example, memory modifications operations in general can pose potential security risks that could potentially be exploited, and can be a potential indicator of risk. In some instances, the type of memory modification that is being performed can be evaluated to determine whether the code portion including the memory modification poses is risky. For example, memory modification operations that use values (e.g., value to be written, memory address to be modified) statically defined at compile time may be less risky than other memory modification operations that use one or more dynamically defined values at runtime, such as values that are calculated by the code at runtime (as opposed to at compile time) and/or that use values passed into the code portion, such as parameters from another function and/or other input data. Accordingly, code portions can be assigned a risk level based on whether a memory modification operation is present as well as the type of memory modification that is being performed (e.g., modification using statically-defined values, modification using dynamically-defined values). For example, code portions that include memory modification operations using one or more dynamically-defined values (e.g., value to be written, memory address to be written), can be assigned a high risk level. Code portions that do not include memory modification operations or that include memory modification operations using only statically-defined values, but that directly or indirectly call a high risk level code portion can be assigned a medium risk level. Code portions that do not include memory modification operations or that include memory modification operations using only statically-defined values, and that do not directly or indirectly call a high risk level code portion can be assigned a low risk level. Additional levels of risk and/or ways to assign risk levels can also be used, for example, in some instances code portions that include memory modification operations using statically-defined values can be assigned a medium risk level (or a different risk level between low risk and high risk) regardless of whether they indirectly or directly call a high risk code portion.

Referring toFIG. 3to illustrate an example of assigning risk levels to the code portions302-318, the code portions that do not modify memory or that modify memory using only statically-defined values, and that do not call any code portions that modify memory using dynamically-defined values can be determined to have a risk level of 0 (low risk level), as is done for example nodes312,314, and318. If a code portion302-318does modify memory (e.g., perform a write( ) or other memory modification operation) using dynamically-defined values, then the code portion can be determined to pose a high-level of risk and a corresponding node302-318can be assigned a risk level of 2. For instance, in this example nodes302,304, and310are assigned a risk level of 2. If a code portion302-318does not modify memory or modifies memory using only statically-defined values, but does directly or indirectly call another code portions that does modifies memory (e.g., perform a write( ) or other memory modification operation) using dynamically-defined values, the corresponding node302-318can be determined to have a medium level of risk and can be assigned a risk level of 1. For example, nodes306and308directly call nodes310and304(which have a risk level of 2 since they include memory modification operations), respectively, and are assigned a risk level of 1. Node316indirectly calls node310(via node306) and, accordingly, is also assigned a risk level of 1 since execution of the code portion316can lead to execution of the code portion310, which modifies memory.

As an example, code portion302(e.g., function) may include code that modifies a portion of memory320using a value that is either calculated during runtime or input to the code portion302(e.g., user input, parameter passed from another function, input from another device or system). As such, the code portion302can be determined to pose a high level of risk and the node302can be assigned highest risk value of 2. Code portion316may not modify memory itself, but it may indirectly call another code portion310that modifies memory320using dynamically-determined values. As such, the code portion316can be determined to present a medium level of risk and the corresponding node310can be assigned the medium risk value of 1. Code portion318does not modify memory or performs a memory modification operation using only statically-defined values, and does not call any functions that modify memory using dynamically-determined values. As such, it can be determined to have a low level of risk and can be assigned the lowest risk value of 0.

Additional and/or alternative heuristics models can be taken into account when assessing risk levels for code portions represented by the nodes302-318in the graph. For example, risk levels can additionally and/or alternatively be determined based on heuristics, such as performing memory allocation operations, examination of the instructions that pass control and parameter checking for the code that receives control, and/or whether the code creates a new thread of control. For example, risk levels can alternatively and/or additionally be determined based on whether code portions allocate memory, either directly in each code portion or by calling other code portions that allocate memory. In such an example, memory allocation operations can pose potential security risks that could potentially be exploited and that can be used to augment risk levels determined based on memory modifications. Accordingly, if a code portion302-318does not allocate memory and does not call any code portions that allocate memory, then the corresponding node302-318can be determined to have a low risk level based on memory allocation risk, and the determined risk level for the corresponding code portion can be unchanged by this determination. If a code portion302-318does allocate memory (e.g., perform a malloc( ) or other memory allocation operation), then the code portion can be determined to pose a high-level of risk based on memory allocation risk and the determined risk level can be enhanced (e.g., increment risk level) and/or automatically set to a high risk level (e.g., assign risk level of 2). If a code portion302-318does not allocate memory but does directly or indirectly call another code portions that does allocates memory (e.g., perform a malloc( ) or other memory allocation operation), the risk level for the corresponding node302-318can be enhanced (e.g., increment risk level) and/or automatically set to a medium level of risk (e.g., assign a risk level of 1). As will be described later, different IMV schemes can be applied to different portions302-318based on the risk value of the corresponding nodes302-318.

As another example, code portion302(e.g., function) may include code that allocates unallocated memory322. As such, it can be determined to pose a high level of risk and the node302can be assigned highest risk value of 2. Code portion316may not allocate memory itself, but it may indirectly call another code portion310that allocates unallocated memory324. As such, it is determined to present a medium level of risk and the corresponding node310can be assigned the medium risk value of 1. Code portion318does not allocate memory and does not call any functions that allocate memory. As such, it can be determined to have a low level of risk and can be assigned the lowest risk value of 0.

In the example depicted inFIG. 3, control should never pass to memory322or324, as those portions of memory320does not contain known-safe code that is part of the code portions302-318, which may be, for example, installed by the OEM as previously described. However, one method of attacking the software stored in memory320is to load executable code into the memory322or324and to cause that memory to gain control so that it is executed. This type of attack is sometimes called code injection, in memory, or an out-of-range attack. Due to, for example, flaws in the software, attackers may find a way to cause control to attempt to jump to memory322or324, such as through modifying memory using dynamically input/determined values and/or through performing memory allocation operations. However, IMV schemes can be applied to the execution of the code portions302-318to prevent or minimize such attacks.

While IMV could be applied to each of the code portions302-318to ensure valid in-memory operation passing between the code portions302-318according to the edges between the nodes in the in-memory graph are followed—meaning that jumps and returns between the code portions302-318are contained within the permitted behavior identified in the in-memory graph—such global application of IMV to the code portions can have a significant performance hit on the operation of the controller. Instead, as described throughout this document, different IMV schemes can be applied separately to the individual code portions302-318based on the corresponding risk level. For example, a first IMV scheme implementing a full IMV check—meaning jump and return address verification—can be performed for high risk code portions (e.g., code portions302,304,310). A second IMV scheme implementing a partial IMV check—meaning that the call to jump to another code portion can be verified for one or more details (e.g., verify that it is a function being called, verify the name of the function is known)—can be performed for medium risk code portions (e.g., code portions306,308,316). A third IMV scheme implementing no IMV check meaning that no additional processing is performed to verify jumps or returns—can be implemented for low risk code portions (e.g., code portions312,314,318). The performance hit on the processor can be the most significant with the first IMV scheme, which will have to retrieve permitted jump and return locations for a code portion, identify their current location within the memory320, and then verify the locations in memory320before permitting the processing flow to continue. The second IMV scheme can additionally have a performance hit, but it can be much less significant than the first IMV scheme since the call itself can be verified, for example, without retrieving and determining additional information to perform the verification. While the second IMV scheme may not provide the same level of security as the first IMV scheme, it can ensure that processing is flowing to a code portion (e.g., a function, although the specific function may not be verified) instead of to other parts of memory320, such as memory322or324. The third IMV scheme can have no performance hit on the controller, but can also provide no security protections. By being selectively allocated to code portions based on assessed risk levels, these example IMV schemes can, in aggregate, maximize the security on the controller while minimizing the performance hit on the controller to implement the protections.

In addition to or as an alternative to the other techniques described here, watchpoints can be used to enhance the security of a controller. For example, watchpoints can be break points set in the hardware of a computing system that interrupt the control of executing software when the control passes to a memory address where the watchpoint was set. For example, watchpoints can be used with debuggers such that when processing flow reaches a watchpointed memory address, processing flow can stop and appropriate action can be taken, including continuing operation of the code. Watchpoints can also be set at the beginning and end of buffers to signal buffer overruns, such that if a watchpointed memory address is written to outside of the buffer, a security issue may be identified and a corrective action may be taken. As shown inFIG. 3, watchpoints326-336can be set in the memory320before and after the contiguous group of code portions302-318, and can be used to generate alerts in the event that processing flow extends outside of the code portions302-318.

The watchpoints that are available on a controller can be limited in number, depending on the hardware or other system/device limitations. For example, some hardware only permits four watchpoints. Watchpoints may be dynamically moved as software executes, though. For example, the security agents232can be programmed to dynamically move watchpoints within the memory320as processing flow across the code portions302-318progresses. For example, the security agents232can dynamically set and unset watchpoints based on, for example, risk levels associated with the code portions302-318, such that watchpoints are assigned at or around code portions that are currently being executed and that are determined to have high risk levels. Other techniques and factors for allocating a limited number of watchpoints across the memory320are also possible.

In addition to using hardware watchpoints, which are hardware memory modification monitors, memory modification monitors can also be implemented in software, which can include software checks being be executed at defined time moments (e.g., when some syscalls are performed) to ensure memory overrun didn't happen. Unlike hardware watchpoints, software watchpoint checks may, in some instances, be too late to prevent memory overruns and may be triggered after the memory modification has already caused bad code execution (instead of being triggered by the memory modification request itself, as with hardware watchpoints). However, in the case of limited set of hardware watchpoints such software wachpoint checks may be performed to check structures that are considered as having less risk of attack while hardware watchpoints can be allocated to protect against the riskiest code portions at a given moment. For instance, if a function allocates memory, it is likely that this function will then proceed to access this newly allocated memory region at that point in time. Accordingly, the newly allocated memory region can be considered to currently be the greatest risk attack target and the hardware watchpoints can be allocated to protect this newly allocated memory. In this example, the software watchpoint checks can be applied for “older” previously allocated regions of memory.

FIGS. 4A-Bare flowcharts of example processes400and450for implementing conditional in-memory validation on a software controller. The processes400and450can be run on any of a variety of systems, such as the systems100,150,200, and300, and/or controllers, such as the controller114, the ECUs156a-n, the controller202, the controller using the memory320, and/or other controllers and/or systems. The processes400and450are described below with reference to some of these systems, controllers, and components, but can be applied to any appropriate system, controller, and/or component.

Referring toFIG. 4A, the example process400is depicted for generating a security policy for a controller that can be used to implement conditional IMV at runtime on a controller. The example process400can be performed, for example, as part of the security policy generation described above with regard to the policy generation computer system104. For example, the policy generation computer system104can determine initial/baseline security risks posed by various code portions, can identify conditions on the controller that may elevate and/or decrease those risk levels, and can store those risk levels and conditions as part of the security policy108that can be implemented at runtime by the controller114.

Static analysis of the controller code can be initiated (402), and from the static analysis an in-memory graph of the code portions in the controller software can be built (404). For example, the computer system104can perform static analysis on the controller software102and, from the static analysis, can build an in-memory graph of the different functions (example code portions) that call each other, as depicted inFIG. 3, for example.

First code portions that modify memory can be identified (406). For example, the computer system104can identify functions that make system calls to modify memory, such as write( ) and/or other memory allocation operations, which may vary across different coding languages and on different operating systems. In some instances, the first code portions may be limited to those portions that perform memory modification operations using values that will be dynamically determined during runtime (as opposed to being statically defined at compile time). Such determinations can be made by evaluating how one or more values (e.g., value to be written, memory address) used to modify memory are determined in the compiled code. If any of these values are determined during runtime as opposed to be being statically defined in the compiled code, they can be identified as a first code portion. Referring to the example above inFIG. 3, the example code portions302,304, and310are identified as including memory modification operations. Nodes302,304, and310corresponding to the code portions302,304, and310can be tagged by the computer system104as being first code portions that include memory modification operations.

Second code portions that call the first code portions can be identified (408). For example, the computer system104can use the in-memory graph and the directional edges between the nodes to identify other nodes that call the first code portions. Referring to the example above inFIG. 3, the computer system104can start at each of the nodes tagged as having memory modification operations (nodes302,304, and310), and can walk through the graph backward along each of the directional edges that point to those nodes. Any other node that is included in this reverse walk through the in-memory graph can be identified as a second code portion. For example, starting with the node302, there are no edges in the graph that point to it, so there is no reverse walking to perform for this node. Moving on to the node304, there are two edges that point to this node—the edge from the node302to the node304and the edge from the node308to the node304. Since the node302is already identified as having a memory modification operation, the edge between the nodes302and304can be ignored. The node308, however, can be identified as a second code portion since it does not include a memory modification operation. Edges pointing to the node308can then be analyzed to continue the reverse walk, but in this example the node308does not have any edges pointing to it, so the reverse walk emanating from the node304can stop. The reverse walk can then continue with the node310, which has edges pointing to it from the node304(which can be ignored, as described previously) and from the node306. The node306can be identified as a second code portion based on its edge pointing to the node310, and then edges pointing to the node306can be analyzed as part of the revers walk. In the depicted example, an edge from node302(which can be ignored, as described previously) and an edge from node316point to the node306, which can cause the node316to additionally be identified as a second code portion. Since no other nodes point to the node316, the reverse walk can be concluded.

Other risk-based heuristics for the code portions can be identified (410). For example, as described above, there can be other types of heuristics that may additionally and/or alternatively be taken into account when determining risk levels for code portions, such as the parameters that are passed into a function as part of a function call, the function's parameter checking and verification procedures (e.g., checking size constraints on functions), whether the functions call or use any network operations, function pointers, buffer allocations in-memory, any memory address that is being passed as a parameter, and/or other factors.

Risk levels for the code portions can be determined (412). For example, as described above with regard toFIG. 3, the example first code portions (nodes302,304, and310) can be determined to have a highest risk level. The second code portions (nodes306,308,316) can be determined to have a medium risk level. Third code portions (nodes312,314,318)—those that are not identified as either the first a second code portions—can be determined to have a low risk level. Other risk levels and ways for assessing risk levels can also be used.

Conditions for risk level modifications for the code portions can be identified (414). For example, if a current security threat to the controller is identified (e.g., an attempt to load malware onto a controller is detected and blocked), then each of the determined risk levels for code portions can be elevated in response to that condition (current security threat). Elevated risk level can include, for example, elevating low-risk processes to medium-risk processes, medium-risk processes to high-risk processes, and high-risk processes to processes that additionally implement watchpoint controls (described above with regard toFIG. 3) as part of their implementation. In another example, if the controller is determined to not be able to currently meet performance thresholds and there is not a current security threat, then the determined risk levels for code portions can be reduced. Other conditions and modifications to risk levels for code portions are also possible, such as the ECU context as described above.

Risk levels and conditions for modifying risk levels can be recorded as part of the security policy for the controller (416). For example, the computer system104can store the in-memory graph, the determined risk levels, the conditions for risk level modifications, and/or other details (e.g. other heuristic information) as part of the security policy108that the controller114will use during runtime. The security policy108, in some instances, can be “burned” into patched code that is delivered and installed on controllers, and may not be a separate policy file or data structure apart from the binary itself. For example, the distributed code can have the security policy burned in so that binary includes the runtime IMV code. Each code portion then includes appropriate code to check and verify the code portion to prevent an in-memory attack. The in-memory graph (as a singular data file or structure) may not be provided in such instances, but may instead be used to generate and insert the code to check for in-memory verification at appropriate locations, which can collectively represent the in-memory graph.

Referring toFIG. 4B, the process450is used to implement a custom security policy on a controller using conditional IMV, such as implementing the custom security policy determined using the process400. For example, the process450can be performed by the controller114during runtime using the security policy108.

Computer-readable code running on a controller is detected (452). For example, computer-readable code in the application layer224may be executed, interpreted, or otherwise run by the controller202. The security agents232, as instructed by the security policy230, may detect the running of the computer-readable code. For example, messages between the application layer224and the kernel238may pass through the security agents232, which may use listeners to detect those calls.

The computer-readable code is stored in a memory in a plurality of blocks, each block including one or more instructions to be performed by the controller. For example, the computer-readable code may be stored in the memory320in blocks302-318. The blocks may include instructions that include, but are not limited to, modifying the memory320, reading from the memory320, passing control of the computer-readable code to a different memory address, and communicating with other computing devices.

Responsive to detecting the computer-readable code running on the controller, a current code portion (e.g., function) of the computer readable code that is running is identified (454). For example, the security agents may monitor the address in memory320of the instruction that is being executed by the controller202, and identify the code portions302-318to which that address belongs.

An in-memory graph that models an operational flow of the computer-readable code is accessed (456). For example, the security agents232may access the in-memory graph, as determined in the process400, from its storage location in the memory206by reading one or more memory locations that contain the in-memory graph.

The risk value for the current code portion is identified (458). The security agents232may look up the baseline/starting point risk value of the node in the in-memory graph (determined using the process400) that corresponds to the code portion that is running on the controller202. For example, if the code portion310is currently running on the controller202, the security agents may look up the baseline risk value stored with node310, which is a risk value of “2.”

An IMV scheme is selected (460) from a plurality of IMV schemes based on the identified risk value. For example, the security policy230may use memory206to store a mapping of risk values to IMV schemes to indicate which IMV scheme should be used. This mapping may be a simple one-to-one mapping, with a particular risk value being mapped to one and only one IMV scheme. In some cases, the number of risk values may be the same as the number of IMV scheme (e.g., three risk values and three risk schemes). In some examples, the mapping between risk values and IMV schemes may be more complex than a simple one-to-one mapping. For example, the mapping may be conditional based on one or more variables. A medium risk value, for example, may be conditionally mapped to a low-intensity IMV scheme and a high-intensity IMV scheme. This mapping may incorporate variable that reflect the state of the environment in which controller (conditions on the controller) is operating, with a high-risk environment meaning the high-intensity IMV scheme should be used and a low-risk environment meaning a low-intensity IMV scheme should be used. For example, if risk levels can be elevated and/or reduced depending on the current condition of the controller, as described above with regard to step414.

The selected IMV scheme is applied (462,464,466,468) to the code portion as the code portion is running on the controller.

Responsive to a determination that the computer-readable code running on the controller fails the selected IMV scheme, a corrective action is taken (470). For example, if it is decided (462) by the security agents232to select an IMV scheme that specifies that no IMV checking should be done, the security agents232can allow system calls to pass with no IMV checking for the current block. This type of IMV scheme may be selected, for example, when there are is little to no risk of an out-of-range attack. An example of a function that provides little to no risk of an out-of-range attack is a function that does not modify memory or call, either directly or indirectly, any other functions that modify memory. Because the risk is found to be so small for such a block, the IMV checking may be avoided to prevent degradation of performance of the controller202. In this way, code which does not provide a substantial security risk be run without overhead for security functions that are not needed to improve the security of the controller202.

In another example, if it is decided (462) by the security agents232to select an IMV scheme that specifies that function validation should be done, the security agents232can verify that memory locations referenced by instructions of the current block contain the computer-readable code. For example, the security agents232may intercept system calls to the kernel238that pass control of the controller202(or other calls). These system calls can include function returns, functions to start a new thread, or “GOTO” style function calls that move control to a particular memory address. The security agents232may examine the parameters of these functions to identify any memory address in the parameters and determine if those memory addresses are within a code portion302-318or not. In some cases, this may be restricted to an entry point of the code portion302-318(e.g. the first instruction or another instruction specifically identified as an entry point). In some cases, this may be permitted as long as it is any address within the code portions302-318, thus indicating that it contains computer-readable code of the control software102. If the call is to an address outside of the permitted addresses, the security agents232can take (470) a corrective action. If the call is to an address within the permitted addresses, the security agents232can permit the call to pass.

In another example, if it is decided (462) by the security agents232to select an IMV scheme that specifies that memory addresses should be verified, the security agents232can verify memory address referenced by instructions of the current code portion. For example, the security agents232may intercept system calls to the kernel238that pass control of the controller202(or other calls). These system calls can include function returns, functions to start a new thread, or “GOTO” style function calls that move control to a particular memory address. The security agents232may examine the parameters of these functions to identify any memory address in the parameters and determine if those memory addresses are on a predefined list of permitted addresses.

For example, the security agents232may access a stored, predefined list of permitted addresses in the memory206. Code portions302-318(or, e.g., each instruction or other logical unit) may be shown in this list to be permitted to use a closed list of permitted memory addresses. These lists may be different for each code portion, or may be shared by more than one code portion. When an instruction calls a system call using a memory address as a parameter (e.g., a return function to return to a particular memory address), the security agents232can access this list to determine if the memory address in the function call is a permitted address for the block. If the address is permitted, the security agents232can permit the call to pass. If the address it not a permitted address, the security agents232can take (470) a corrective action.

Corrective actions include actions taken by the security agent232, or another element, to handle or mitigate the security risk of an unapproved action. Example corrective actions include, but are not limited to, halting the running of the computer-readable code, dropping a system call, modifying a system call (e.g., changing a memory address in a parameter), logging the system call (e.g., in the log220), generating an alert (e.g., in the alerts218), and engaging a user interface controlled by the controller202(e.g., illuminating an amber light). Different corrective actions may be used in different situations. For example, the security policy230may identify some security risks that are so grave, the controller202should be shut down when encountered, while other security risks should be logged only.

While a particular number, type, and order of operations have been described, others are possible. For example, as part of selecting (460), from a plurality of available flow control integrity (IMV) schemes, an IMV scheme based on the identified risk value, the security agents232may consider other input in selecting the IMV scheme. For example, the security agents232may access contextual information250and information received from external sources such as the management computer system126to determine the security environment of the controller202. If the controller202is in a high-risk environment, a more-rigorous IMV scheme may be selected than would be selected otherwise. For example, instead of selecting a function validation scheme (see466), the security agents232may select a memory validation scheme (see468). Likewise, if the controller is in a low-risk environment, a less rigorous IMV scheme may be selected than would be selected otherwise. For example, instead of selecting a function validation scheme (see466), the security agents232may select a pass-through scheme (see464).

Examples of high-risk environments include times in which controllers like the controller202have been under attack. For example, when attackers learn of a security flaw in a particular software deployment, rates of attack on controllers using that software deployment increase sharply. In those time, an external system can send a notification to the controller202indicating that the controller202is operating in a high-risk environment. The security agents232can then use this information when selecting an IMV scheme, such as to select a more rigorous scheme.

Likewise, the operating environment of the controller202may be used as an indication of the security environment. For example, if the controller202disables network and removable media access in some conditions (e.g., an automobile controller when traveling), the security agents232can use the contextual information sources (e.g., speedometer) to determine the security environment—lower security when stopped and higher security when traveling at speed. In this way, a balance of security and performance can be found that takes into account both the riskiness of a particular block of computer-readable code and the context in which the controller202operates in.

FIG. 5is a flowchart of an example process500for dynamically setting watchpoints during controller operation. The process500can be performed by any of a variety of systems, such as the systems100,150,200, and300, and/or controllers, such as the controller114, the ECUs156a-n, the controller202, the controller using the memory320, and/or other controllers and/or systems. The process500can be performed, for example, as part of the process450. For example, the process500be implemented as part of and/or in addition to the conditional IMV steps described above, such as being performed in addition to the validate memory step (468) for particular code portions and/or under particular conditions on the controller. Additionally and/or alternatively, the process500can be performed separately from the process450on the controller.

A current code portion that a controller is performing can be identified (502) and a determination can be made as to whether watchpoints should be dynamically set for the current code portion (504). The determination of whether to dynamically set watchpoints, as described above with regard toFIG. 3, can be made based on any of a variety of factors. For example, watchpoints can be set for code portions that are determined to be high-risk. In another example, watchpoints can be set for code portions that are determined to be high-risk when the controller is operating under one or more current elevated conditions (e.g., currently under/recently blocked attack, notification from a central server system that attacks are currently being launched against a population of similar controllers). In another example, watchpoints can be set for all code portions when the controller is operating under such elevated conditions. Other factors can also be taken into account when determining whether to set watchpoints.

If the determination is that watchpoints should be set, then the locations for the watchpoints with regard to the current code portion can be determined (506). Such locations can be selected from a set of candidate locations that are preidentified as part of the security policy, for example, such as a location in memory at the end of the code portion and/or one or more locations at the end of other code portions (other functions) that are called within the code portion. The number of locations for watchpoints that are set can be limited by the number of watchpoints that are available for allocation on the controller. As described above, the number of watchpoints may be limited to a certain number (e.g., 4 watchpoints in total).

The watchpoints can be dynamically set during runtime at the determined locations in memory by the controller (508). For example, before proceeding with operation of the current code portion, the controller can dynamically set the watchpoints for current code portion. Once the watchpoints are set, the current code portion can be permitted to proceed on the controller (510). If one of the dynamically set watchpoints is triggered (512), then corrective action can be applied (514). The corrective action can be similar to the corrective action discussed above with regard to step470. The process500can then repeat for some or all current code portions, such as part of process450.

FIG. 6is a block diagram of example computing devices600,650that may be used to implement the systems and methods described in this document, as either a client or as a server or plurality of servers. Computing device600is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Computing device600is further intended to represent any other typically non-mobile devices, such as televisions or other electronic devices with one or more processers embedded therein or attached thereto. Computing device650is intended to represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, and other computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the inventions described and/or claimed in this document.

The memory604stores information within the computing device600. In one implementation, the memory604is a computer-readable medium. In one implementation, the memory604is a volatile memory unit or units. In another implementation, the memory604is a non-volatile memory unit or units.

The storage device606is capable of providing mass storage for the computing device600. In one implementation, the storage device606is a computer-readable medium. In various different implementations, the storage device606may be a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid state memory device, or an array of devices, including devices in a storage area network or other configurations. In one implementation, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as the memory604, the storage device606, or memory on processor602.

The high-speed controller608manages bandwidth-intensive operations for the computing device600, while the low-speed controller612manages lower bandwidth-intensive operations. Such allocation of duties is an example only. In one implementation, the high-speed controller608is coupled to memory604, display616(e.g., through a graphics processor or accelerator), and to high-speed expansion ports610, which may accept various expansion cards (not shown). In the implementation, low-speed controller612is coupled to storage device606and low-speed bus614. The low-speed bus614(e.g., a low-speed expansion port), which may include various communication ports (e.g., USB, Bluetooth®, Ethernet, wireless Ethernet), may be coupled to one or more input/output devices, such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or router, e.g., through a network adapter.

Computing device650includes a processor652, memory664, an input/output device such as a display654, a communication interface666, and a transceiver668, among other components. The computing device650may also be provided with a storage device, such as a micro-drive or other device, to provide additional storage. Each of the components650,652,664,654,666, and668, are interconnected using various buses, and several of the components may be mounted on a common motherboard or in other manners as appropriate.

The processor652can process instructions for execution within the computing device650, including instructions stored in the memory664. The processor may also include separate analog and digital processors. The processor may provide, for example, for coordination of the other components of the computing device650, such as control of user interfaces, applications run by computing device650, and wireless communication by computing device650.

Processor652may communicate with a user through control interface658and display interface656coupled to a display654. The display654may be, for example, a TFT LCD display or an OLED display, or other appropriate display technology. The display interface656may comprise appropriate circuitry for driving the display654to present graphical and other information to a user. The control interface658may receive commands from a user and convert them for submission to the processor652. In addition, an external interface662may be provided in communication with processor652, so as to enable near area communication of computing device650with other devices. External interface662may provide, for example, for wired communication (e.g., via a docking procedure) or for wireless communication (e.g., via Bluetooth® or other such technologies).

The memory664stores information within the computing device650. In one implementation, the memory664is a computer-readable medium. In one implementation, the memory664is a volatile memory unit or units. In another implementation, the memory664is a non-volatile memory unit or units. Expansion memory674may also be provided and connected to computing device650through expansion interface672, which may include, for example, a subscriber identification module (SIM) card interface. Such expansion memory674may provide extra storage space for computing device650, or may also store applications or other information for computing device650. Specifically, expansion memory674may include instructions to carry out or supplement the processes described above, and may include secure information also. Thus, for example, expansion memory674may be provide as a security module for computing device650, and may be programmed with instructions that permit secure use of computing device650. In addition, secure applications may be provided via the SIM cards, along with additional information, such as placing identifying information on the SIM card in a non-hackable manner.

The memory may include for example, flash memory and/or MRAM memory, as discussed below. In one implementation, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as the memory664, expansion memory674, or memory on processor652.

Computing device650may communicate wirelessly through communication interface666, which may include digital signal processing circuitry where necessary. Communication interface666may provide for communications under various modes or protocols, such as GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others. Such communication may occur, for example, through transceiver668(e.g., a radio-frequency transceiver). In addition, short-range communication may occur, such as using a Bluetooth®, WiFi, or other such transceiver (not shown). In addition, GPS receiver module670may provide additional wireless data to computing device650, which may be used as appropriate by applications running on computing device650.

Computing device650may also communicate audibly using audio codec660, which may receive spoken information from a user and convert it to usable digital information. Audio codec660may likewise generate audible sound for a user, such as through a speaker, e.g., in a handset of computing device650. Such sound may include sound from voice telephone calls, may include recorded sound (e.g., voice messages, music files, etc.) and may also include sound generated by applications operating on computing device650.

These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor, and can be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. Other programming paradigms can be used, e.g., functional programming, logical programming, or other programming. As used herein, the terms “machine-readable medium” “computer-readable medium” refers to any computer program product, apparatus and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal used to provide machine instructions and/or data to a programmable processor.