BOT DETECTION IN A VIRTUAL DIGITAL ENVIRONMENT

Disclosed embodiments provide bot detection in a virtual environment that is based on avatar motion. The motion of an avatar within a virtual environment such as a metaverse is tracked. The tracked motion is separated into multiple movement segments. The movement segments are compared to a collection of movement patterns in a movement pattern database that are indicative of bots. A similarity metric is determined between the tracked motion and the database of bot motions. In response to the similarity metric exceeding a predetermined threshold, a list of at least one mitigation action is obtained, and the at least one mitigation action in the list is executed, thereby mitigating the adverse effects of bots in a virtual digital environment.

FIELD

The present invention relates generally to malware detection, and more particularly, to bot detection in a virtual digital environment.

BACKGROUND

A computer bot, also known simply as a ‘bot,’ is a software program designed to automate repetitive and simple tasks. Bots can be programmed to perform a variety of functions such as web scraping, data mining, online communication, and automation of simple processes. They can run on various platforms, such as websites, instant messaging apps, and social media platforms, and can interact with users in various ways. Bots can be used for both legitimate and malicious purposes.

Adverse effects of computer bots can include spread of misinformation and propaganda. Bots can be programmed to spread false information and manipulate public opinion on a massive scale. Similarly, bots can be used to manipulate prices in financial markets, causing disruptions and harm to investors. Additionally, bots can flood online forums and social media with spam, making it difficult for legitimate users to enjoy the services and benefits of the platform. Beyond that, bots can be used to carry out cyber-attacks such as hacking, phishing, and malware distribution. Additionally, unwanted bots can use a lot of computer and network resources that may impact the experience of other users and cause an increase in infrastructure costs. Thus, it is important to take the problems associated with bots seriously.

SUMMARY

In one embodiment, there is provided a computer-implemented method for bot detection, comprising: tracking motion of an avatar within a virtual environment; separating the tracked motion into a plurality of movement segments; comparing the plurality of movement segments to a collection of movement patterns; computing a similarity metric for the tracked motion to at least one movement pattern from the collection of movement patterns; in response to the similarity metric exceeding a predetermined threshold: obtaining a list of at least one mitigation action; and executing the at least one mitigation action from the list.

In another embodiment, there is provided an electronic computation device comprising: a processor; a memory coupled to the processor, the memory containing instructions, that when executed by the processor, cause the electronic computation device to: track motion of an avatar within a virtual environment; separate the tracked motion into a plurality of movement segments; compare the plurality of movement segments to a collection of movement patterns; compute a similarity metric for the tracked motion to at least one movement pattern from the collection of movement patterns; in response to the similarity metric exceeding a predetermined threshold: obtain a list of at least one mitigation action; and execute the at least one mitigation action from the list.

In yet another embodiment, there is provided a computer program product for an electronic computation device comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the electronic computation device to: track motion of an avatar within a virtual environment; separate the tracked motion into a plurality of movement segments; compare the plurality of movement segments to a collection of movement patterns; compute a similarity metric for the tracked motion to at least one movement pattern from the collection of movement patterns; in response to the similarity metric exceeding a predetermined threshold: obtain a list of at least one mitigation action; and execute the at least one mitigation action from the list.

The drawings are not necessarily to scale. The drawings are merely representations, not necessarily intended to portray specific parameters of the invention. The drawings are intended to depict only example embodiments of the invention, and therefore should not be considered as limiting in scope. In the drawings, like numbering may represent like elements. Furthermore, certain elements in some of the Figures may be omitted, or illustrated not-to-scale, for illustrative clarity.

DETAILED DESCRIPTION

Virtual environments, such as virtual reality environments, can be used for gaming, entertainment, education, and more. Additionally, metaverses, which are virtual environments that can include immersive experiences beyond gaming are becoming more and more popular.

The term ‘metaverse’ generally refers to a virtual world or a collective space created by the convergence of virtual reality, augmented reality, and/or other forms of digital media. Metaverses can enable socialization that includes interacting with other users in virtual spaces, such as virtual clubs, events, and parties. Metaverses can also include rich gaming environments that allow users to play video games and/or to participate in virtual sports, races, and/or other competitive events. Metaverses can also have an ecommerce component, allowing the purchase of virtual goods, clothing, accessories, and other items to personalize an avatar and allow a user to show off his/her unique style. Additionally, there can be applications for education and/or work within a metaverse. As examples, within a metaverse, it is possible to participate in virtual education and training programs, attend virtual classes, attend virtual conferences and workshops, collaborate with colleagues, attend virtual meetings, and/or perform virtual tasks in a virtual workspace, among other things. With support for the aforementioned activities, it is important to detect and mitigate bot activity within a metaverse.

Disclosed embodiments provide bot detection in a virtual environment, such as a metaverse, that is based on avatar motion. The motion of an avatar within a virtual environment such as a metaverse is tracked. The tracked motion is separated into multiple movement segments. The movement segments are compared to a collection of movement patterns in a movement pattern database that are indicative of bots. A similarity metric is determined between the tracked motion and the database of bot movement patterns. In response to the similarity metric exceeding a predetermined threshold, a list of at least one mitigation action is obtained, and the at least one mitigation action in the list is executed, thereby mitigating the adverse effects of bots in a virtual digital environment.

Reference throughout this specification to “one embodiment,”“an embodiment,”“some embodiments”, or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of the phrases “in one embodiment,”“in an embodiment,”“in some embodiments”, and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment.

Moreover, the described features, structures, or characteristics of the invention may be combined in any suitable manner in one or more embodiments. It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit and scope and purpose of the invention. Thus, it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents. Reference will now be made in detail to the preferred embodiments of the invention.

Characteristics are as follows:

Service Models are as follows:

Deployment Models are as follows:

Program/utility40, having a set (at least one) of program modules42, may be stored in memory28by way of example (which can include cache32, RAM30, and storage system34), and not limitation, as well as an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data or some combination thereof, may include an implementation of a networking environment. Program modules42generally carry out the functions and/or methodologies of embodiments of the invention as described herein.

Implementations of the invention may include a computer system/server12ofFIG.1in which one or more of the program modules42are configured to perform (or cause the computer system/server12to perform) one of more functions of the virtual environment bot detection system96ofFIG.3. For example, the one or more of the program modules42may be configured to: track motion of an avatar within a virtual environment; separate the tracked motion into a plurality of movement segments; compare the plurality of movement segments to a collection of movement patterns; compute a similarity metric for the tracked motion to one or more movement patterns from the collection of movement patterns; in response to the similarity metric exceeding a predetermined threshold: obtain a list of one or more mitigation actions; and execute the one or more mitigation actions from the list.

FIG.4is an ecosystem400for embodiments of the present invention. Virtual Environment Bot Detection System (VEBDS)402comprises a processor440, a memory442coupled to the processor440, and storage444. System402is an electronic computation device. The memory442contains program instructions447, that when executed by the processor440, perform processes, techniques, and implementations of disclosed embodiments. Memory442can include dynamic random-access memory (DRAM), static random-access memory (SRAM), magnetic storage, and/or a read only memory such as flash, EEPROM, optical storage, or other suitable memory, and should not be construed as being a transitory signal per se. In some embodiments, storage444may include one or more magnetic storage devices such as hard disk drives (HDDs). Storage444may additionally include one or more solid state drives (SSDs). The VEBDS402is configured to interact with other elements of ecosystem400. VEBDS402is connected to network424, which can include the Internet, a wide area network, a local area network, and/or other suitable network.

Ecosystem400may include one or more client devices, indicated as416. Client device416can include a laptop computer, desktop computer, tablet computer, smartphone, virtual reality headset, augmented reality goggles, wearable computing device, or other suitable computing device. Client device416may be used to participate in a virtual environment, and/or configure VEBDS402.

Ecosystem400may include one or more machine learning systems422. The machine learning systems422can include, but are not limited to, a convolutional neural network (CNN), Recurrent Neural Network (RNN), Long Short Term Memory Network (LSTM), Radial Basis Function Network (RBFN), Multilayer Perceptron (MLP), Gradient Boosted Network, and/or other suitable neural network types. In some embodiments, the VEBDS402may orchestrate training and inputting data and receiving of output data from the machine learning systems422. In some embodiments, one or more of the aforementioned neural networks may be implemented with the VEBDS402. The training data and output data from machine learning systems422can include avatar movement patterns. The avatar movement patterns can include movement patterns from confirmed bot avatars, and/or confirmed human-controlled avatars. The movement patterns can be stored in a movement patterns database438. The movement patterns database can include an SQL database, or other suitable database type. The movement patterns may be encoded as a sequence of alphanumeric characters. The movement patterns can be used by the VEBDS402to flag possible bot avatars within a virtual environment such as a metaverse.

Ecosystem400may further include account database441. The account database can include an SQL database, or other suitable database type. The account database can include information on user accounts, avatars, and/or other relevant data. The data can include an avatar status. In embodiments, the avatar status may be encoded as a word with various bit fields, where bits within the word convey information about the avatar. The information can include an active/inactive status, a verification status, indicating verified human user, verified bot, suspected bot, and so on. In embodiments, the VEBDS402can update the verification status of an avatar based on movement patterns, results of authentication challenges, and/or other information.

Ecosystem400may include one or more application servers412. The application servers412may implement HTML-based user interfaces and provide backend functionality to support virtual environment and/or metaverse features and functions such as, gaming, virtual meetings, education, e-commerce, banking, and/or financial applications, to name a few.

Ecosystem400includes a virtual digital environment460. The environment460can include a virtual reality environment, augmented reality environment, metaverse, and/or other suitable virtual digital environment. The virtual environment460includes multiple avatars, shown as462,463, and464. These avatars are representations of users, and/or synthetic users that are computer controlled. In some embodiments, the avatars have a humanoid form, including legs, arms, torso, and a facial region. In other embodiments, the avatars can be of a simplified form. As an example, the avatars may have only a torso, arms, and a facial region, with the lower limbs being absent.

The synthetic users can be non-playable characters in a game, or bots that operate autonomously within the virtual environment. The virtual environment can include a rendering467. The rendering467can include renderings of physical objects such as buildings, planes, clouds, cars, trees, and the like. Additionally, the rendering467can include objects that do not exist in reality. The rendering467can provide areas for the avatars462-464to move. As an example, the avatars may move around streets, in and out of buildings, rooms, and other areas and regions that are created in the virtual digital environment. A human user472can wear a virtual reality headset474to view, hear, and/or otherwise experience the virtual digital environment. Virtual reality headset474may be a client device, similar to clients416.

In embodiments, a particular avatar may be associated with a user. As an example, avatar462may be associated with human user472. Thus, user472can control the motion and actions of avatar462, and interact with the other avatars,463and464. The human user may assume that the avatars463and464are associated with other human users. However, it is possible that one or more avatars within the virtual digital environment460are in fact bots. With disclosed embodiments, the motion of avatars within the virtual digital environment are analyzed, and avatars that move in a manner indicative of a bot can be identified, such as with a label or symbol477as indicated on avatar464, such that human users are alerted that an avatar within the virtual digital environment460is a potential bot. In response to the alert of a potential bot, a human user may choose to limit interaction and/or information shared with that bot. Thus, disclosed embodiments can help improve computer security by alerting users about avatars that are potentially bots. Embodiments can include rendering a symbol on an avatar that has a computed probability of being a bot, where the computed probability exceeds a predetermined threshold. As an example, the predetermined threshold can be a number in a range from 0 to 1. As an example, a threshold of 0.7 corresponds to a seventy percent chance that an avatar is a bot, based on avatar motion analysis.

FIG.5is a flowchart500showing steps for embodiments of the present invention. At502, avatar motion is tracked. The avatar motion can be tracked in one or more dimensions. At504, the motion is separated into segments. The segments can be linear segments, arcs, and/or other suitable types of segments. At505the tracked motion may be preprocessed. The preprocessing can include rotation, scaling, mirroring, and/or other transformations. Thus, embodiments can further include performing preprocessing on the tracked motion prior to the comparing, which occurs at506. At506, the segments are compared to movement patterns. Movement patterns can be stored as a sequence of segments. The avatar motion segments can be compared in order and length along with other suitable properties. At508a similarity metric is computed, based on how closely the segments from the avatar being analyzed match a particular motion sequence that is indicative of a bot. At512, the computed metric is compared to a threshold. In response to computing a similarity metric that exceeds a predetermined threshold, a mitigation action list is obtained at514.

The mitigation action list can include at least one action for the VEBDS402to perform. The action can include rendering an indicator on the avatar, such as shown at477inFIG.4. This visually alerts other users that the avatar is potentially a bot. At516the mitigation action(s) are executed. The mitigation can include a virtual environment position freeze, which prevents motion of the avatar that is determined to be a potential bot. At518, instructions are provided to the avatar. The instructions can serve as an authentication challenge. In some embodiments, the virtual environment position freeze is followed by an authentication challenge that is provided to the avatar. If the avatar is in fact controlled by a human user, the human user is able to easily pass the authentication challenge, to remove the virtual environment position freeze. The authentication challenge can be a command to move to a location, and/or other specific instructions. Other mitigations can include a virtual environment account freeze. This prevents the avatar from engaging in the virtual environment with purchases, and/or other interactions. In some embodiments, the mitigation can include deleting the avatar and/or deleting the account. In some embodiments, an authentication challenge is presented before deleting an account or deleting an avatar, and only if the authentication challenge fails, is the mitigation action of deleting an account and/or deleting an avatar performed.

In some embodiments, there can be multiple thresholds and each threshold can have a corresponding action list. As an example, a first threshold can have a value of 0.6, and a second threshold can have a value of 0.9. When the similarity metric computed at508exceeds the first threshold, but is less than the second threshold, then the mitigation actions corresponding to the first threshold are executed. In this example, exceeding the first threshold indicates a bot probability of greater than 60 percent, but less than 90 percent, and in that case, the mitigation actions can include issuing an authentication challenge, such as asking the avatar/user to identify an object, move to a location, and/or answer a question. In a case where the similarity metric computed at508exceeds the second threshold, indicating a bot probability of greater than 90 percent, then a stronger mitigation action, such as a virtual environment position freeze may be performed. Thus, embodiments can further include a second predetermined threshold, and: in response to the similarity metric exceeding the second predetermined threshold: obtaining a second list of at least one mitigation action; and executing the at least one mitigation action from the second list.

FIG.6is a flow diagram600showing additional steps for embodiments of the present invention. At602, the bot discovery engine is initialized. The bot discovery engine may be an application, library, or collection of applications and/or libraries that is executed by the VEBDS402in order to track motion of avatars in a virtual environment for the purposes of detecting bots. At604, the movement of avatars is analyzed. This can be accomplished via application programming interface (API) calls that expose the location and/or movement of avatars. In other embodiments, log files may be scraped and/or parsed to determine avatar motion. The tracked motion is compared to movement patterns in movement pattern database608. The movement pattern database608can include movement patterns of known and/or suspected bot avatars, as well as movement patterns of known and/or confirmed human-controlled avatars. At608, a check is made to determine if a similarity is found to a bot movement pattern. If no at606, then the process returns to604to continue analyzing avatar movement.

If yes at606, then the movement of the avatar is monitored for additional movement. A similarity metric is computed based on the similarity of avatar movement compared with known bot avatar movements stored in the movement patterns database608. If the similarity metric exceeds a predetermined threshold, then the process continues to614where at least one mitigation actions are determined. This can include retrieving actions from actions database614. The flow then continues with executing the actions. In embodiments, the actions can include a virtual environment position freeze, a virtual environment account freeze, a deletion of the avatar, and/or other suitable actions. If no at612, the process continues to610where avatar movement is continuously or periodically monitored.

Thus, disclosed embodiments analyze the avatar's navigation pattern based on the virtual environment/metaverse navigation axis. Embodiments can compute a score based on the delta between user navigation and the navigation parameters. The score can increase if repetitive movements are detected (this is indicative of a scripted bot).

FIG.7Ashows an exemplary avatar motion path704for avatar702. The motion path704comprises a series of movements in various directions.FIG.7Bshows an exemplary avatar motion path after segmentation. InFIG.7B, the path704ofFIG.7Ais divided into a polarity of segments and endpoints. Segment731is bounded by endpoint720and721. Segment732is bounded by endpoint721and722. Segment733is bounded by endpoint722and723. Segment734is bounded by endpoint723and724.FIG.7Cshows the path704ofFIG.7Aafter annotation. Each segment is annotated with a direction and distance. The annotations can be stored in a file in text, binary, or other suitable format. As an example, segment734is annotated with ‘4N’ indicating a direction of ‘north’ and a distance of 4 units. Embodiments can use Cartesian coordinates, polar coordinates, spherical coordinates, and/or other suitable coordinate systems for recording avatar motion.

Embodiments can include preprocessing of avatar motion paths such as rotation, scaling, and/or mirroring. In embodiments, the preprocessed avatar motion paths are also checked against movement patterns in the movement patterns database608(FIG.6).

FIG.7Dshows an exemplary avatar motion path740after rotation. The avatar motion path740is the avatar motion path704ofFIG.7A, rotated90degrees Clockwise.FIG.7Eshows an exemplary avatar motion path750after scaling. The avatar motion path750is the avatar motion path704ofFIG.7A, scaled to be a predetermined percentage larger.FIG.7Fshows an exemplary avatar motion path760after mirroring. The avatar motion path760is the avatar motion path704ofFIG.7A, flipped around its vertical axis. Other preprocessing operations are possible in disclosed embodiments.

In embodiments, comparison between avatar motion and stored movement patterns in the movement patterns database608can include computing the number of matching direction changes, distances, and/or other attributes amongst the movement patterns. A score can be computed based on the number of segments that match in direction and/or distance between an avatar motion path (and/or preprocessed motion path), and stored movement patterns in the movement patterns database608.

FIG.8Ashows an example805of a virtual reality view801with a motion-based authentication message. In the example805, the virtual reality view is of a kitchen, that includes a refrigerator814. In embodiments, the user may use a virtual hand, shown as816, to indicate a location to move to. In embodiments, the virtual hand may be implemented via the user wearing a glove, holding a controller, image analysis from a front-facing camera of a virtual reality headset, and/or other suitable techniques. In the event that an avatar is suspected of being a bot, based on motion analysis, then an authentication message812may be displayed, which includes instructions. If the avatar is controlled by a human user, the human user may follow the instructions. Upon successful completion of the instructions, the avatar status can be changed from suspected bot to confirmed human user, and updated accordingly in the account database (441ofFIG.4). In embodiments, the at least one mitigation action includes providing instructions to the avatar. In embodiments, the instructions include text and/or audio. The audio can include speech that asks a user to perform an action. In embodiments, the instructions include an instruction for the avatar to move to a specified location.

FIG.8Bshows an example855of a virtual reality view851with a puzzle-based authentication message. In the example855, an authentication message862provides instructions for a puzzle-based challenge. In the example, three images, indicated as871,872, and873are shown. Image871is of a bicycle, image872is of a flower, and image873is of mountains. Additionally, image874is an image of a word (“Foggy”) with lines through it to help obfuscate it from optical character recognition, while still allowing a human to read it. The instructions in authentication message862instruct the user to touch the image of the flower, and then utter the word “foggy” as shown in874. Thus, in embodiments, the instructions include an instruction for an utterance. In embodiments, the user may use a virtual hand, shown as866, to touch (select) an image by holding his/her virtual hand over the image for a predetermined duration (e.g., three seconds). In embodiments, the virtual hand may be implemented via the user wearing a glove, holding a controller, image analysis from a front-facing camera of a virtual reality headset, and/or other suitable techniques. In the event that an avatar is suspected of being a bot, based on motion analysis, then an authentication message862may be displayed, which includes instructions. If the avatar is controlled by a human user, the human user may follow the instructions. Upon successful completion of the instructions, the avatar status can be changed from suspected bot to confirmed human user, and updated accordingly in the account database (441ofFIG.4).

FIG.9shows a block diagram of a client device900in accordance with disclosed embodiments. In embodiments, this may represent a virtual reality headset such as474ofFIG.4. Device900includes a processor902, which is coupled to a memory904, which can include random-access memory (SRAM), magnetic storage, and/or a read only memory such as flash, EEPROM, optical storage, and/or other suitable memory. In some embodiments, the memory904may not be a transitory signal per se.

Device900may further include storage906. In embodiments, storage906may include at least one magnetic storage devices such as hard disk drives (HDDs). Storage906may additionally include at least one solid state drives (SSDs).

Device900may, in some embodiments, include a user interface908. This may include a display, keyboard, or other suitable interface. In some embodiments, the display may be touch-sensitive.

The device900further includes a communication interface910. The communication interface910may include a wireless communication interface that includes modulators, demodulators, and antennas for a variety of wireless protocols including, but not limited to, Bluetooth™, Wi-Fi, and/or cellular communication protocols for communication over a computer network and/or operation with a gaming system, and/or other computers and/or computer peripherals for supporting virtual environments and/or metaverses. In embodiments, instructions are stored in memory904. The instructions, when executed by the processor902, cause the electronic computing device900to execute operations in accordance with disclosed embodiments.

Device900may further include a microphone912used to receive audio input. The audio input may include speech utterances. The audio input may be digitized by circuitry within the device900. The digitized audio data may be analyzed for phonemes and converted to text for further natural language processing. In some embodiments, the natural language processing may be performed onboard the device900. In other embodiments, all or some of the natural language processing may be performed on a remote computer.

Device900may further include camera916. In embodiments, camera916may be used to acquire still images and/or video images by device900. Device900may further include one or more speakers922. In embodiments, speakers922may include stereo headphone speakers, and/or other speakers arranged to provide an immersive sound experience. Device900may further include geolocation system917. In embodiments, geolocation system917includes a Global Positioning System (GPS), GLONASS, Galileo, or other suitable satellite navigation system.

Device900may further include an accelerometer932and/or gyroscope934. The accelerometer932and/or gyroscope934may be configured and disposed to track movements of a user, such as head and/or hand movements while donning wearable computing devices such as virtual reality headsets and/or handheld remote-control devices in communication with a virtual reality system.

Device900may further include an eye tracker system936. The eye tracker system936may include one or more cameras configured and disposed to track eye movement of a user, and render portions of a virtual environment based on eye movement. Device900may further include a vibrator938which may be used to provide tactile alerts to a user about suspected bot activity. As an example, disclosed embodiments may issue a tactile alert such as a vibration in response to detecting potential/suspected bot avatars within a field of view and/or proximity to a user avatar within a virtual environment. The components shown inFIG.9are exemplary, and other devices may include more, fewer, and/or different components than those depicted inFIG.9.

As can now be appreciated, disclosed embodiments provide techniques for automatic detection and mitigation of bot avatars in a virtual environment such as a metaverse. Thus, disclosed embodiments improve computer security, as well as improve the utilization of computer resources by preventing the wasting of computer resources on nuisance and/or malicious bot avatar activities, thereby making these systems and platforms safer and more enjoyable for legitimate users.