# Wi-Fi Direct

## Summary

Wi-Fi-Direct is a standard developed by Wi-Fi Alliance so that Wi-Fi devices can connect and communicate with each other directly without using an access point (AP). This is also known as Wi-Fi P2P (Wi-Fi Point-to-Point). This is very useful for direct file transfer, internet sharing, or connecting to a printer. Wi-Fi Direct can communicate with one or more devices simultaneously at typical Wi-Fi speeds.

Wi-Fi Direct supports IEEE 802.11 a/b/g/n/ standards.

## Discussion

### Can you explain how Wi-Fi Direct works?

Wi-Fi Direct works in two steps:

    + **Device Discovery**: A device first broadcasts a probe request message asking for MAC ID from all nearby devices. This stage is similar to the scanning phase in regular Wi-Fi. All devices that hear this message respond with a unicast message to the sender. Devices alternate between sending out and listening for probe requests, and subsequently their responses.
    + **Service Discovery**: Now the sender sends a unicast service request message to each device. The receiving devices responds with unicast message with service details.In terms of the states or phases of Wi-Fi Direct, devices go through scan phase (scan the channels), find phase (which includes service discovery), formation phase (group is formed and includes WPS provisioning), and operational phase. Whereas discovery happens on Social channels 1, 6 and 11 in the 2.4GHz band, operation can be in either 2.4 or 5GHz bands.

### How many devices can Wi-Fi Direct connect?

A Wi-Fi Direct-certified network can be one-to-one, or one-to-many. The number of devices in a Wi-Fi Direct-certified group network is expected to be smaller than the number supported by traditional standalone access points intended for consumer use. Connection to multiple other devices is an optional feature that will not be supported in all Wi-Fi Direct-certified devices; some devices will only make 1:1 connections.

### Can you explain architecture of Wi-Fi Direct?

As Wi-Fi Direct does not require any AP, the device itself has the capability to function like AP. Wi-Fi Direct devices, aka P2P Devices, communicate by establishing P2P Groups, which are functionally equivalent to traditional Wi-Fi infrastructure networks. The device implementing AP-like functionality in the P2P Group is referred to as the *P2P Group Owner (P2P GO)*, and devices acting as clients are known as *P2P Clients*. P2P GO is sometimes referred to as *Soft AP*.

Wi-Direct devices can function as 1:1 and 1:n devices. The shown topology in the diagram explains the scenario of 1:1 and 1:n. In 1:1 scenario, a Wi-Fi Direct device is printing files at a printer. In 1:n scenario, a laptop is sharing files to three other Wi-Fi devices.

### Is it possible for Group Owner to connect to Wi-Fi AP?

Only the P2P GO is allowed to cross-connect the devices in its P2P group to an external network. Wi-Fi Direct does not allow transferring the role of P2P GO within the group. If P2P GO leaves the P2P group, then the group is torn down, and has to re-established.

### Can you explain how group formation happens?

Group formation procedure involves two phases:

    + **Determination of P2P Group Owner**: Two P2P devices negotiate for the role P2P Group Owner based on desire/capabilities. P2P GO role is established at formation or at an application level.
    + **Provisioning of P2P Group**: P2P group session is established using appropriate credentials. Wi-Fi simple configuration is used to exchange credentials.There are three ways of group formation: Standard, Autonomous and Persistent. In **Standard**, P2P devices discover each other and negotiate who will act as P2P GO. In **Persistent**, devices recall if they've had a previous connection with persistent flag set. If so, GO Negotiation Phase (3-way handshake) is replaced with Invitation Phase (2-way handshake). WPS Provisioning is simplified since stored network credentials are reused.

### Can you explain how service discovery happens?

Service Discovery process enables Wi-Fi Direct devices to discover each other and the services they support before connecting. For example, a Wi-Fi Direct device could see all compatible devices in the area and then narrow down the list to only devices that allow printing before displaying a list of nearby Wi-Fi Direct-enabled printers. Before establishment of a P2P Group, P2P Devices can exchange queries to discover the set of available services and, based on this, decide whether to continue the group formation or not.

*Generic Advertisement Service (GAS)* as specified by 802.11u is used. GAS is a layer 2 query and response protocol implemented through the use of public action frames, that allows two non-associated 802.11 devices to exchange queries belonging to a higher layer protocol (e.g. a service discovery protocol).

### How is security implemented in Wi-Fi Direct?

Wi-Fi Direct devices are required to implement Wi-Fi Protected Setup (WPS) to support a secure connection with minimal user intervention. WPS is based on WPA-2 security and uses AES-CCMP as ciphering, and a randomly generated Pre-Shared Key (PSK) for mutual authentication.

### Could you explain Wi-Fi Direct's Power Saving Mode?

Wi-Fi Direct defines two new power saving mechanisms:

    + **Opportunistic Power Save Protocol**: The Opportunistic Power Save protocol (OPS) allows a P2P Group Owner to opportunistically save power when all its associated clients are sleeping. This protocol has a low implementation complexity but, given the fact that the P2P Group Owner can only save power when all its clients are sleeping, the power savings that can be achieved by the P2P Group Owner are limited.
    + **Notice of Absence Protocol**: The Notice of Absence (NoA) protocol allows a P2P GO to announce time intervals, referred to as *absence periods*, where P2P Clients are not allowed to access the channel, regardless of whether they are in power save or inactive mode. In this way, a P2P GO can autonomously decide to power down its radio to save energy.

### What's the speed of Wi-Fi Direct?

A Wi-Fi Direct-certified device supports typical Wi-Fi speeds, which can be as high as 250 Mbps. Even at lower speeds, Wi-Fi provides plenty of throughput for transferring multimedia content with ease. The performance of a particular group of Wi-Fi Direct devices depends on whether the devices are 802.11 a, b, g, or n, as well as the particular characteristics of the devices and the physical environment.

### Can you relate Wi-Fi Direct to Miracast?

Miracast is a Wi-Fi display certification program announced by Wi-Fi Alliance for seamlessly transferring video between devices. The intersection of wireless connectivity and streamed audio/video content can be termed as *Miracast*. This solution enables seamless mirroring of entire displays across devices or sharing of any type of content that a source could display using Wi-Fi Direct.

### Why is the design of Wi-Fi Direct based on infrastructure mode and not ad hoc mode?

Most commercial devices are for infrastructure mode and not ad hoc mode. For easier migration and interworking with legacy devices, Wi-Fi Direct is based on infrastructure mode. Wi-Fi Direct works on the principles similar to that of Wi-Fi AP. Hence, the P2P GO is sometimes also called *Soft AP*. .

### Is it sufficient for only one device to be Wi-Fi Direct-certified to form a group?

Yes. Only the P2P GO needs to be Wi-Fi Direct-certified. Other group members may be legacy Wi-Fi stations that operate in infrastructure mode. Wi-Fi Direct essentially embeds a software access point for the group owner. The soft AP provides a version of Wi-Fi Protected Setup with its push-button or PIN-based setup. Thus, group members see only an AP.

### What are the pros and cons of Wi-Fi Direct?

Wi-Fi Direct can connect multiple devices without a Wi-Fi AP. Wi-Fi Direct is portable. No Internet connection is required to transfer files between two devices. Only one device needs to be Wi-Fi Direct-certified and it can assume the role of the group owner.

Wi-Fi Direct has some limitations. Not all vendors support it. Connection has to be re-established with other devices every time to form the group. As soon as GO leaves the group, all connections are broken and service stops. The group is formed as a star topology with the GO at the center. Group members cannot talk to each other directly.

Protected by WPA2, Wi-Fi Direct is secure, but applications could use it wrongly and thereby compromise overall security. Devices connect using Wi-Fi Protected Setup (WPS) but implementations that use WPS PIN method of setup are insecure. An analysis from April 2020 of some popular Android applications that use Wi-Fi Direct found 17 security issues. These applications (SHAREit, Xender, Xiaomi Mi Drop, Files by Google, Zapya and SuperBeam) often compromised security in favour of usability.

## Milestones

Oct
2010

Wi-Fi Alliance launches Wi-Fi Direct certification program. Within the same month, Atheros, Broadcom, Intel, Ralink and Realtek announce their first certified products. A month later, *Popular Science* magazine recognizes Wi-Fi Direct as one of the best tech innovations of the year.

Oct
2011

Google announces Wi-Fi Direct support in Android 4.0.

Mar
2016

Sony, LG, Philips and X.VISION implement Wi-Fi Direct on some of their televisions.

Jun
2017

With the growing adoption of Software Defined Networking (SDN), Poularakis et al. explore how SDN can be applied to wireless mobile networks. Traditional SDNs are centralized in the network whereas mobile networks require a distributed architecture. They propose a hybrid architecture. In one example, they should how Wi-Fi Direct can also be part of the network. A Wi-Fi Direct smartphone also becomes a virtual switch due to *Open vSwitch* and uses *OpenFlow* protocol.

Wi-Fi Direct


# SQL Injection

## Summary

SQL (Structured Query Language) is a language used to create, update and access data in a database. By carefully crafting SQL commands, a hacker can intentionally cause the application to fail, delete data, steal data or gain unauthorized access. This is what we call SQL injection or **SQL Injection Attack (SQLIA)**. SQL itself is a highly flexible language, which creates opportunities for hackers to acquire sensitive information such as credit card details, user passwords or user IDs. .

There are various methods and tools to attack an SQL database. This article gives an overview of SQL injection attacks and what hackers aim to achieve. We also cover the types and techniques of SQL injection attacks. We note some possible preventive measures towards more secure database-driven websites.

## Discussion

### Why do we need to worry about SQL injection?

CWE (Common Weakness Enumeration) is a system where all weaknesses and vulnerabilities are categorised concerning a particular system architecture, code, hardware or software. **CWE-89: SQL Injection** notes that any data-rich software is vulnerable to attack when user information is stored in an SQL-based database. An attack can lead to loss of data confidentiality, access control and data integrity.

Many widely adopted databases are based on SQL including Oracle, MySQL, Microsoft SQL Server, PostgreSQL and SQLite. In April 2022, these were in the first, second, third, fourth and tenth positions respectively. SQLIA techniques may slightly differ across databases but all are vulnerable if not properly secured.

In 2021, the Open Web Security Project (OWASP) tested 94% of applications under study for injection attacks. They found that 19% had at least one of instance of a vulnerability type (called incidence rate). Injection category was the third-most serious risk and includes 33 CWEs. On a ten-point scale, 7+ values for both exploit and impact shows the threat of SQLIA. In the years 2004/2007/2010/2013/2017, injection attacks were in positions 6/2/1/1/1 respectively.

### Could you introduce SQL injection with a few examples?

The figure illustrates a few examples, each affecting the application in a different way. In all cases, inputs submitted by hackers are directly used to form queries without proper validation.

The first example possibly represents a user sign-up form where the user is expected to enter first name and last name. Instead, a hacker enters first name that includes a single quote. Since single quote is normally used by developers to enclose values, the extra quote in input causes execution error. The application will not respond correctly to the user.

In the second example, the hacker submits the string `name' OR 'a'='a` for item name. In the resulting command, `'a'='a'` will match and return all records to the hacker. This is a serious data breach.

In the third example, using semi-colons, we execute multiple commands, called batched queries. Comment syntax `--` is used to ignore characters after the delete command. This attack results in a serious data loss.

### What's the nature of SQL injection attacks?

SQLIA can take many forms: identifying injectable parameters, performing database fingerprinting, determining database schema (table or column names, datatypes), adding/modifying/extracting data, performing denial of service (locking or dropping rows/tables/databases), evading detection, bypassing authentication, executing remote commands, performing privilege escalation, etc. In general, a hacker tries to gather information about the database, uses that information to handcraft attacking commands, and finally execute those commands.

These attacks can be placed into four categories:

    + **SQL Manipulation**: Modify SQL statements possibly by changing the `WHERE` clause.
    + **Code Injection**: Insert extra SQL statements after the exploitable SQL statement. This is possible on databases that support multiple SQL requests. Semi-colons typically separate multiple queries. `UNION` statements can be used provided the hacker knows the exact number columns and their data types returned by the exploited statement.
    + **Function Call Injection**: Insert function calls within the exploited SQL statement. This can not only manipulate data but also make operating system calls.
    + **Buffer Overflows**: A product of using a function call injection. Hacker exploits a weakness often found on unpatched servers.

### What are the different types of SQL injection attacks?

Different types of SQLIA have been documented:

    + **Boolean-based**: Aka tautology attack. The use of `OR 2 = 2` within the `WHERE` clause makes other conditions in the clause redundant.
    + **UNION-based**: The `UNION` statements combines the results of separate queries. `' UNION SELECT username, password FROM users--` is an example injection. Where multiple columns have be retrieved within fewer columns, columns can be concatenated.
    + **Error-based**: Error messages can give information about database type. For example, a query that fails due to a `SUBSTRING` function call suggests that it's probably an Oracle database where the equivalent function is named `SUBSTR`.
    + **Batch Queries**: Aka piggyback attack. For example on SQL-Server 2005, `; INSERT INTO users VALUES (‘Abubakar’, ‘1234’);#` ends the original statement, appends another statement and comments out any trailing characters with `#` character.
    + **LIKE-based**: Similar to tautology attack but based on pattern matching using `LIKE` and wildcard operator `%`. An example is `OR username LIKE 'S%'`.
    + **Stored Procedure**: Hacker executes built-in stored procedures using malicious SQL code injections.

### What's a blind SQL injection?

A hacker typically submits an SQL query and gathers information from the response. Information can come back in the same channel (in-band) or in a separate channel such as email (out-of-band). But in blind SQL injection (aka inferential attack) the hacker obtains useful information even when no data or error message is returned to the hacker. The hacker does this by observing the behaviour. Blind injections can be Boolean-based or time-based.

With **Boolean-based attacks**, the hacker sees that true and false evaluations result in different behaviour. He then exploits this, perhaps over many queries, to eventually learn sensitive information such as the administrator's password.

With **time-based attacks**, the hacker can insert time delays into SQL command execution. These delays can be conditional or unconditional. Due to synchronous processing, these delays affect the HTTP response to the client. Based on the query, the response time gives the hacker clues to what's going on. For example, an observed delay can confirm that an injected condition was true; or adding `SLEEP(1)` to the `WHERE` clause can inform the number of matching records.

### What's a second-order SQL injection?

Second-order SQL injection occurs when the injected inputs don't do immediate damage. They're simply stored in the database as data. When later retrieved, they end up causing malicious executions. Second-order injections are in a way sophisticated because they bypass prevention techniques that're employed when inputs are validated. However, similar stringent validation may be lacking when data is retrieved from the database.

The figure shows an example in which a hacker creates a user account with name that hides within it an update command. The system may at best check for length, valid characters and uniqueness of username. But at a later time, when the hacker logs in with this handcrafted username, it executes the update command and changes the administrator password. This effectively gives the hacker much more power to cause serious damage.

Second-order injections can be used to create tables or functions that can be exploited later when those constructs are triggered. Shared search criteria, website statistics and customer services are some app features that may be exploited to achieve second-order injections.

### What evasion techniques do hackers use for advanced SQLIA?

There are Intrusion Detection Systems (IDSs) and tools to detect and protect against common injections. Hackers have therefore evolved techniques to achieve more sophisticated attacks that basic systems may not detect:

    + **Encoding**: Instead of using plain ASCII characters, injections are URL-encoded. Alternatively, UTF-8 or hexadecimal encoding may be used. For example, `5' OR '5'='5` is encoded as `%35%27%20%4F%52%20%27%35%27%3D%27%35`; or `admin` is encoded as `char(97,100,109,105,110)`.
    + **White-spacing**: Whitespaces are not significant in SQL. Hackers therefore omit or insert extra whitespaces that may include line feeds, carriage returns and tabs. For example, `OR '5'='5'`, `OR '5'='5'` and `OR'5'='5'` are equivalent.
    + **Comment**: Using multiline comments, attackers can subvert detection techniques that attempt to match signatures. For example, a `SELECT` query can be written as `SELE/**/CT` or `/**/ SELECT/**/`.
    + **Capitalization**: Where the database is configured to be case-insensitive, `AND` is same as `anD`.
    + **Variation**: Techniques in this group include concatenation, variables, and conversions. For example, `'UN'||'ION` is same as `UNION`. A statement is split into multiple variables and combined later for execution.

### Which tools help in detection and prevention of SQLIA?

The figure show tools to either detect or prevent SQLIA, and their handling of different attack types. A 2017 study noted that SQLCheck, SQLIPA, DB IDS, AMNESIA, SQLDOM and WebSSARI all achieve better than 90% protection.

SQLrand is a proxy server between the web server and the database server. It receives randomized SQL from application and submits standard SQL to the database. All user inputs are treated as non-keywords. AMNESIA does static analysis of application code, constructs a model of valid queries and validates queries at runtime. WebSARRI is another tool that combines static analysis and runtime inspection.

SQLDOM creates strongly-typed classes based on the database schema. These classes are used to generate SQL statements. It’s an object model as it's used with the help of object-oriented languages. SQLCheck focuses on grammar and policy that are created and stored on the web server. User inputs are augmented with metacharacters and then passed to the checker for validation.

### What should developers do to protect against SQLIA?

While a production system may employ tools to detect and prevent SQLIA, there are some best practices that can developers can adopt to mitigate the possibility of SQLIA in the first place. Give users only limited database privileges. Application code probably never needs to drop or truncate tables. Configure web server properly so that verbose error messages are not shown to users.

Never trust user inputs. Don't rely only on client-side validation since they can be bypassed with curl, postman or similar tools. Do server-side validation. Adopt safe coding practices. Put quotes around input strings. Replace single quote with two single quotes. Where numbers are expected, check that indeed only digits are present.

Even better is to avoid using raw SQL statements formed from raw input strings. Use custom stored procedures and pass user inputs as parameters into these procedures. Many SQL databases support the `PREPARE` statement and pass in user inputs as parameters to such a statement. This avoids the unsafe practice of concatenating raw inputs to form queries.

Read OWASP SQL Injection Prevention Cheat Sheet and OWASP Query Parameterization Cheat Sheet.

## Milestones

Dec
1998

Jeff Forristal (alias rain.forest.puppy) writes about SQL injection in the Phrack magazine. He notes that MS SQL Server 6.5 allows execution of **batch commands** and gives many examples to exploit this. He concludes,

> Don't assume user's input is ok for SQL queries.

2003

As more and more web applications emerge, there's a need to find loopholes that may endanger the user's security. In this context, *WAVES (Web Application Vulnerability and Error Scanner)* is first introduced. WAVES is a project developed by Hunag et al. to create a platform to assess web application security based on vulnerabilities such as SQL injection and cross-site scripting.

2008

A hacker obtains access to 100 million debit and credit cards by hacking into the Heartland Payment System. As a result, Heartland pays around $140 million in fines and penalties.

2011

Multiple attacks targeting Sony through Sony Pictures, Sony Music Japan and Sony Playstation Network occur. Personal information of 100 million users is leaked, causing a loss of $171 million. The attacks on Sony Pictures and Sony Music Japan used SQL injection.

Mar
2011

As many as 47 MySQL databases on Apache web server get hacked. Hackers publicly post usernames, database schemas and passwords that they gathered using **blind SQL injection**. The breach includes hashed passwords of administrative accounts of webmaster, WordPress, root and forums.

2013

**OWASP Top 10** is published stating how injection attacks take top spot among all types of attacks. This top spot is maintained in a subsequent survey in 2017. The report notes that injection attacks are easy while the impact is severe.

2017

In a survey paper, Alwan and Younis mention two recent types of SQLIA called Fast Flux SQLIA and Compounded SQLIA. The latter is a combination of many attack techniques including DDoS attack, DNS hijacking, Cross-Site Scripting (XSS), and more.

Jun
2019

Lewis of NCC Group shares a proof-of-concept of using **Machine Learning to detect SQLIA** vulnerabilities. They use real-world vulnerability data (though limited in size) and limit their study to only MySQL. They use SVM-Multiclass. In 2021, Erdodi et al. propose using Reinforcement Learning instead with Markov decision process as the model for SQLIA. These are just two examples to illustrate the use of machine learning in SQLIA.

2020

Abikoye et al. use the **Knuth-Morris-Pratt (KMP) string matching algorithm** to detect and prevent SQLIA. Based on syntactic analysis of various types of attacks, parse trees are designed. KMP algorithm is then used to compare user inputs against the known attack patterns.

SQL Injection


# 5G UE Measurements and Reporting

## Summary

Measurements are essential to determine the health of any cellular system given the current configuration. Measurements help the UE and the network make decisions so that resources are managed better and ultimately quality of service is achieved. Measurements are done by both UE and the network, although this article focuses only on measurements performed by a 5G UE.

Typically, a UE measures downlink signals while network measures uplink signals. However, it's possible for a UE to measure uplink signals sent by other UEs.

RRC manages measurement configuration. Most measurements are executed by Layer 1, although some may be at Layer 2. RRC does filtering on Layer 1 measurements. If filtered measurements meet reporting criteria, they're reported to the network. Some measurements are reported by Layer 1 directly to the network.

While UE measures many different signals, the main ones are based on **SSB** and **CSI-RS**.

## Discussion

### What are some acronyms pertaining to 5G UE measurements?

For convenience we use these acronyms related to measurements in this article: Channel State Information (CSI), Demodulation Reference Signal (DMRS), Reference Signal (RS), Reference Signal Received Power (RSRP), Reference Signal Received Power per Branch (RSRPB), Reference Signal Received Quality (RSRQ), Received Signal Strength Indicator (RSSI), Signal-to-Noise and Interference Ratio (SINR), Synchronization Signal (SS) and Synchronization Signal Block (SSB).

### With respect to RRC states, what measurements are done by a 5G UE?

An essential UE procedure in RRC\_IDLE is cell selection. In RRC\_IDLE and RRC\_INACTIVE, the UE can also do cell reselection. For both these procedures, UE measures RSRP and RSRQ of a cell.

For cell reselection, if supported and enabled, UE may do **Relaxed Measurements**. This is typically useful when the UE has low mobility or not at the cell edge.

If configured, a UE in RRC\_IDLE or RRC\_INACTIVE may collect measurements and report them later in RRC\_CONNECTED. This is called **Logged Measurements**. It's related to a feature called *Minimization of Drive Test (MDT)*.

In RRC\_CONNECTED, the UE is configured via dedicated signalling to perform intra-frequency or inter-frequency NR measurements, or inter-RAT measurements for E-UTRA or UTRA-FDD frequencies. The network uses these to decide on carrier aggregation, dual connectivity or handover.

### What are some basic facts about 5G UE measurements?

For downlink channel sounding, 5G NR uses two main downlink signals that a UE measures:

    + **SSB**: Transmitted with a low duty cycle on a limited bandwidth compared to LTE's Cell-Specific Reference Signals (CRS) that's sent on the entire channel bandwidth. SSB measurements are used to determine path loss and average channel quality.
    + **CSI-RS**: Used for tracking rapidly changing channel conditions to support mobility and beam management.In general, measurements in FR1 are from UE's antenna connector. In FR2, measurements are based on the combined signal from antenna elements mapped to a given receive branch. If UE is using receiver diversity, it reports the maximum value.

Although a UE can be configured for measurements early on, measurement reports can be sent to the network only after **AS security activation**.

### Which are the main quantities measured by a 5G UE?

We describe the main UE measurements involving SS:

    + **SS-RSRP**: Average power of resource elements carrying secondary synchronization signals. In addition, resource elements of PBCH-DMRS and CSI-RS can be included.
    + **SS-RSRPB**: Similar to SS-RSRP but for each antenna connector (FR1) or for each receive branch (FR2). Measurements can include PBCH-DMRS but not CSI-RS.
    + **SS-RSRQ**: Given N resource blocks within the measurement bandwidth, this is \(N \cdot SS{\text-}RSRP/RSSI\_{NR\,Carrier}\). Both SS-RSRP and NR carrier RSSI are measured over the same resource blocks.
    + **SS-SINR**: This is SS-RSRP over average noise-plus-interference power. The latter is measured based on RRC configuration or over the same resource elements as SS-RSRP measurement.Equivalent measurements pertaining to CSI-RS are CSI-RSRP, CSI-RSRQ and CSI-SINR.

**RSSI** is average power over certain OFDM symbols in a measurement bandwidth corresponding to channel bandwidth. It includes co-channel serving and non-serving cells, adjacent channel interference, thermal noise, etc.

### Which are some feature-specific measurements done by a 5G UE?

Measurements for inter-RAT include:

    + IEEE 802.11 WLAN RSSI: for handovers to Wi-Fi
    + Reference Signal Time Difference (RSTD) for E-UTRA: relative timing difference between an E-UTRA cell and the E-UTRA reference cell
    + E-UTRA RSRP, RSRQ and RS-SINR
    + UTRA FDD CPICH RSCP, UTRA FDD carrier RSSI and UTRA FDD CPICH Ec/NoMeasurements for MR-DC include:

    + SFN and Frame Timing Difference (SFTD): Measured between PCell and PSCell.Measurements for sidelink channels include:

    + Sidelink RSSI
    + Sidelink Channel Occupancy Ratio (SL CR)
    + Sidelink Channel Busy Ratio (SL CBR)
    + PSBCH-RSRP, PSSCH-RSRP and PSCCH-RSRPMeasurements for UE positioning include:

    + Timing between a E-UTRA cell and a GNSS-specific reference time
    + DL PRS-RSRP measured on Positioning Reference Signal (PRS)
    + DL Reference Signal Time Difference (RSTD) measures the relative time difference between a Transmission Point (TP) and the reference TP
    + UE Rx–Tx time difference measured per TP
    + SS Reference Signal Antenna Relative Phase (SS-RSARP)

### What are Cross Link Interference (CLI) measurements?

CLI is a problem in TDD when a base station receiving in the uplink is facing interference from another base station transmitting in the downlink. It can happen across network operators due to out-of-band emissions. CLI can be mitigated by time-synchronization across base stations, that is, they share a common clock, phase reference and frame structure.

Even within the same operator network, CLI can occur since cell neighbours may be using different TDD DL/UL patterns. CLI can be mitigated by gNBs coordinating their configuration over Xn and F1 interfaces.

If capable, a 5G UE measures and reports CLI-RSSI. These reports can also include SRS-RSRP measurements on Sounding Reference Signal (SRS), which are uplink signals coming from other UEs. This quantifies interference on downlink due to nearby uplink transmissions. Both CLI-RSSI and SRS-RSRP are measured within the active DL BWP. These are applicable only for RRC\_CONNECTED intra-frequency in TDD mode.

In EN-DC and NGEN-DC, Secondary Node (SN) configures CLI measurements. In NE-DC, Master Node (MN) does it. In NR-DC, both MN and SN can do this.

### What's the difference between 5G NR L1 and L3 measurements?

RSRP, RSRQ, SINR, and RSSI are quantities measured at L1, not L3. We use the term "L3 measurement" to imply that L3 does filtering on the values and does the final reporting. Filtering is done to remove the effect of fast fading and ignore short-term variations. Though L1 may collect measurements more often, L3 might report them at a larger configured periodicity. Thus, L3 takes a longer-term view of channel conditions.

To avoid ping-pong behaviour and unnecessary reporting, L3 manages event-based reporting. It evaluates reporting criteria to decide if a report needs to be sent. Apart from thresholds, such criteria include hysteresis.

But L1 reports some measurements to quickly react to changing channel conditions. Beam management is an example. These are referred to as L1-RSCP and L1-SINR. L1 reports are part of Channel State Information (CSI).

CSI-RS measurements and L1 reporting can be periodic, semi-periodic (activated/deactivated by MAC signalling) or aperiodic (triggered by DCI signalling). L1 reporting is on PUCCH (periodic, semi-periodic) or PUSCH (aperiodic, semi-periodic).

### What L2 measurements are reported by a 5G UE to the network?

At Layer 2, there's measurement and reporting of **UL PDCP delay** for packets during the reporting period. Delay is reported at a granularity of 0.1ms and per Data Radio Bearer (DRB). At most one measurement identity per cell group has this quantity configured for reporting. The corresponding measurement object is ignored.

The delay is in fact queuing delay. It's the time taken to obtain uplink grant from the time packet enters PDCP from upper SAP. It's up to gNB to convert these per-DRB reports to delays at the level of QoS flows.

For completeness, we note that many more L2 measurements are done on the network side. These are described in TS 38.314 specification.

### What's in a typical 5G NR measurement configuration?

A 5G UE is given the following measurement details:

    + **Measurement Objects**: Specifies what is to be measured. For NR and inter-RAT E-UTRA measurements, this may include cell-specific offsets, blacklisted cells to be ignored and whitelisted cells to consider for measurements.
    + **Reporting Configuration**: Specifies how reporting should be done. This could be periodic or event-triggered.
    + **Measurement ID**: Identifies how to report measurements of a specific object. This is a many-to-many mapping: a measurement object could have multiple reporting configurations, a reporting configuration could apply to multiple objects. A unique ID is used for each object-to-report-config association. When UE sends a MeasurementReport message, a single ID and related measurements are included in the message.
    + **Quantity Configuration**: Specifies parameters for layer 3 filtering of measurements. Only after filtering, reporting criteria are evaluated. The formula used is \(F\_n = (1–a)\*F\_{n-1} + a\*M\_n\), where \(M\) is the latest measurement, \(F\) is the filtered measurement, and \(a\) is based on configured filter coefficient.
    + **Measurement Gaps**: Periods that the UE may use to perform measurements.

### Which are the event-triggered measurements reported by 5G UE RRC?

Events are triggered based on thresholds, hysteresis and sometimes offsets. RRC specification defines the following:

    + Event A1: Serving becomes better than threshold
    + Event A2: Serving becomes worse than threshold
    + Event A3: Neighbour becomes offset better than SpCell
    + Event A4: Neighbour becomes better than threshold
    + Event A5: SpCell becomes worse than threshold1 and neighbour becomes better than threshold2
    + Event A6: Neighbour becomes offset better than SCell
    + Event B1: Inter RAT neighbour becomes better than threshold
    + Event B2: PCell becomes worse than threshold1 and inter RAT neighbour becomes better than threshold2
    + Event I1: Interference becomes higher than threshold
    + Event C1: The NR sidelink channel busy ratio is above a threshold
    + Event C2: The NR sidelink channel busy ratio is below a thresholdOnly the serving cell is relevant for events A1/A2. For other events, consider only whitelisted cells if enabled; else consider any neighbour cell detected based on the measurement object configuration provided it's not in the blacklist.

Events B1/B2 relate to inter-RAT. They're used in EN-DC deployments as well where they're reported via E-UTRA RRC signalling.

### How do I interpret events used in event-triggered measurement reporting?

A1 is typically used to **cancel a handover procedure** since the UE has re-established good coverage on the serving cell. With A2, UE has poor coverage on serving cell. Since neighbour cell measurements are not available with A2, network can initiate a **blind handover**; or provide UE configuration to perform neighbour cell measurements (eg. A3 or A5).

Events A3 and A6 contain offsets specific to a neighbour cell. Both involve relative measurements, that is, comparing one cell with another. A3 may lead to intra- or inter-frequency handover away from the Special Cell (SpCell). A6 is relevant to carrier aggregation, where a Secondary Cell (SCell) is configured. A6 may not result in a handover but instead **reconfiguration of cell groups** (MCG or SCG).

A4 could trigger a handover but the decision is not based on radio conditions on the serving cell. It could be for other reasons such as **load balancing**. B1 is similar for the inter-RAT case.

A5 can be seen as a combination of A2 and A4, leading to intra- or inter-frequency handover. B2 is similar for the **inter-RAT** case.

### Which are the 3GPP specifications relevant to 5G UE measurements?

We note the following specifications:

    + **TS 37.340**: Multi-connectivity overall description: Stage-2. Specifies measurement model for multi-connectivity operation involving E-UTRA and NR.
    + **TS 38.133**: Requirements for support of radio resource management. Specifies measurement requirements (including performance requirements), procedures and UE measurement capabilities.
    + **TS 38.215**: Physical layer measurements. Specifies measurement quantities.
    + **TS 38.300**: NR and NG-RAN overall description: Stage-2. Specifies measurement model.
    + **TS 38.304**: User Equipment (UE) procedures in idle mode and in RRC Inactive state. Specifies cell reselection measurement rules and relaxed measurements.
    + **TS 38.314**: Layer 2 measurements. Specifies mostly network requirements but section 4.3 is for UE.
    + **TS 38.331**: Radio Resource Control (RRC): protocol specification. Specifies UE reporting of measurements.
    + **TS 38.533**: User Equipment (UE) conformance specification: Radio Resource Management (RRM).Specification *TS 28.552: 5G performance measurements* is mostly about network-side measurements including network slicing. We mention it here for completeness.

## Milestones

Dec
2017

3GPP publishes Release 15 "early drop". In 5G RRC specification TS 38.331, version 15.0.0, measurement configurations specified include measurement objects, measurement identifies, reporting, gaps, L3 filtering, and events A1-A6. Measurements supported include SS-RSRP, SS-RSRQ, SS-CINR, CSI-RSRP, CSI-RSRQ, CSI-CINR, RSSI and more.

Dec
2019

In TS 38.133, version 16.2.0, these are introduced: CLI measurements, UMTS inter-RAT measurements, SRVCC-related measurements, and more.

Jul
2020

3GPP publishes Release 16 specifications. In 5G RRC specification TS 38.331, version 16.3.1, has a number of additions: measurement configuration for RRC\_IDLE and RRC\_INACTIVE; CLI-RSSI and SRS-RSRP reporting; sidelink and UTRA-FDD reporting; UL PDCP delay reporting; IEs `UE-MeasurementsAvailable` and `needForGapsInfoNR` as part of RRCReconfigurationComplete and RRCResumeComplete messages; and UE positioning measurements.

5G UE Measurements and Reporting


# Types of Blockchains

## Summary

Historically, Blockchain started as a public permissionless technology when it was used for powering Bitcoin. Since then, other types of blockchains have been created. These can be categorized as a combination of public/private and permissionless/permissioned. Each type fits a specific set of use cases. When choosing a particular type, we have to be aware the tradeoffs.

In general, public/permissionless blockchains are open, decentralized and slow. Private/permissioned blockchains are closed and centralized, either partially or completely. They're also more efficient. It's also important to note that for some use cases, traditional databases may suffice instead of a blockchain.

## Discussion

### Could you describe the different types of blockchains?

Broadly, there are two types of blockchains:

    + **Permissionless**: Anyone can join the network. They can read/write/verify transactions. The system is open. There's no central authority. This system makes sense when no one wants to use a trusted third party (TTP). Trust is therefore established among peers via an agreed consensus mechanism. While transactions can be read by anyone, it's also possible to hide sensitive information if so desired.
    + **Permissioned**: A central authority grants permissions to only some folks to read/write/verify transactions. Since write access is given to a trusted few, consensus is achieved in a simpler and more efficient way. Public read access may be allowed.Some classify blockchains as public, private and permissioned. In a private blockchain, controlling power is with only one organization. In a permissioned blockchain, controlling power is given to a few selected entities. Thus, no single entity can tamper the system on their own. These are also called federated or consortium blockchains. They are a compromise between the openness of public blockchain and the closed control of private blockchains.

### How does one go about selecting a suitable blockchain type?

Blockchain is useful in applications where multiple entities write to a shared database, these entities don't trust one another and don't want to use a trusted third party intermediary. If entities are unknown or wish anonymity, then a permissionless blockchain is desired. Otherwise, go for a permissioned blockchain.

Blockchain is also useful when multiple copies of a ledger are maintained. In this case, blockchain enables real-time reconciliation without have a third-party trusted intermediary.

A public permissioned blockchain is one in which some trusted entities write to the chain but public is allowed to verify. For example, a consumer might want to verify the source of the fish she buys but only those involved in the supply chain have permissions to write to the chain. In some applications, such as Cryptologic, confidential transaction data is hashed before added to the public blockchain.

A private permissioned blockchain can be used when control rests with a single trusted entity. If multiple organizations are involved, then a consortium blockchain is preferred.

### What are the advantages of using a permissioned blockchain?

A permissioned blockchain is similar to a permissionless one except for an additional access control layer. This layer controls who can participate in the consensus mechanism, and who can create transactions or smart contracts.

A permissioned blockchain gives the following advantages:

    + **Performance**: Excessive redundant computation of permissionless blockchains is avoided. Each node will perform only those computations relevant to its application.
    + **Governance**: Enables transparent governance within the consortium. Also, innovation and evolution of the network can be easier and faster than in permissionless blockchains.
    + **Cost**: It's cost effective since there's no need to do spam control such as dealing with infinite loops in smart contracts.
    + **Security**: It has the same level of security as permissionless blockchains: "non-predictive distribution of power over block creation among nodes unlikely to collude." In addition, an access control layer is built into the network by design.

### Can you name some examples of the different types of blockchains?

Bitcoin and Ethereum are well-known examples of public blockchains but Ethereum can also be used to create a private blockchain. OpenChain enables private blockchains. Chain supports permissioned blockchains suited for financial applications. Patientory is a permissioned blockchain for electronic health records. Ripple is a permissioned blockchain.

Bitcoin Cash, Zilliqa and Cypherium are permissionless blockchains. Universa and Oracle Network are permissioned blockchains.

Some platforms can be configured to manage either any type of blockchain. For example, MultiChain and HydraChain can be used for private or permissioned blockchains. Hyperledger can be used for private or public blockchains. Hyperledger Fabric and R3 Corda are for private or permissioned blockchains.

William El Kaim has curated a useful list of blockchains, blockchain platforms and applications.

### What's the point of private blockchains where immutability can be compromised?

It's true that since a private blockchains is controlled by a single entity or organization, it can be easily tampered. It's therefore argued that private blockchains are no better than shared databases. If trust and robustness are already guaranteed, one could simply use a database. Moreover, databases have for long supported code execution (example, via stored procedures) that are similar to what blockchain calls smart contracts.

However, others argue that the use of cryptography and Merkle trees prevent non-valid transactions from getting added to the chain. With shared databases, hack on a single entity will corrupt the database for everyone. This isn't possible with private blockchains when a consensus algorithm such as Juno is used.

## Milestones

Jan
2009

Bitcoin is launched using blockchain technology. It's the first application of public blockchain.

2012

Ripple, a permissioned blockchain is launched.

2013

Ethereum, an open permissionless blockchain, is described in a white paper by Vitalik Buterin, a programmer involved with Bitcoin Magazine.

Jul
2015

Initial release of Ethereum, an open-source, public, blockchain-based distributed computing platform and operating system featuring smart contract (scripting) functionality.

Dec
2015

Linux foundation announces the creation of the Hyperledger Project, a permissioned open-source blockchain.

2016

This is they year when people start showing greater interest in private plus permissioned blockchains than in public blockchains. This increased interest continues through 2018, as shown by Google Trends data collected in March 2018.

Types of Blockchains


# Lemmatization

## Summary

Consider the words 'am', 'are', and 'is'. These come from the same root word 'be'. Likewise, 'dinner' and 'dinners' can be reduced to 'dinner'. Variations of a word are called **wordforms** or **surface forms**. It's often complex to handle all such variations in software. By reducing these wordforms to a common root, we simplify the input. The root form is called **lemma**. An algorithm or program that determines lemmas from wordforms is called a **lemmatizer**.

For example, Oxford English Dictionary of 1989 has about 615K lemmas as an upper bound. Shakespeare's works have about 880K words, 29K wordforms, and 18K lemmas.

Lemmatization involves word morphology, which is the study of word forms. Typically, we identify the morphological tags of a word before selecting the lemma.

## Discussion

### Why do we need to find the lemma of a word?

Many NLP tasks can benefit from lemmatization. For instance, topic modelling looks at word distribution in a document. By normalizing words to a common form, we get better results. In word embeddings, that is, representing words as real-valued vectors, removing inflected wordforms can improve downstream NLP tasks.

For information retrieval (IR), lemmatization helps with query expansion so that suitable matches are returned even if there's not an exact word match. In document clustering, it's useful to reduce the number of tokens. It also helps in machine translation.

Ultimately, the decision to use lemmas is application dependent. We should use lemmas only if they show better performance.

### What are the challenges with lemmatization?

Out-of-vocabulary (OOV) words is a challenge. For example, WordNet that's used by NLTK package for lemmatization, doesn't have the word 'carmaking'. The lemmatizer therefore doesn't relate this to 'carmaker'.

It difficult to construct rules for irregular word inflections. The word 'bring' might look like an inflected form of 'to bre' but it's not. Even more challenging is a word such as 'gehört' in German. It's a participle of 'hören' (to hear) or of 'gehören' (to belong). Both are valid but only the context of usage can help us derive the correct lemma. It's for these reasons that neural network approaches to learning rules are preferred over hand-crafted rules.

When content comes from Internet or social media, it's impractical to use predefined dictionary. This is another reason for a neural network approach with an open vocabulary.

Even with neural networks, some inflected forms might never occur in training, such as, 'forbade', the past tense of 'forbid'. Many training corpora come from newspaper texts where verbs in second person are rare. This can impact lemmatization of such forms.

### How is lemmatization different from stemming?

Given a wordform, stemming is a simpler way to get to its root form. Stemming simply removes prefixes and suffixes. Lemmatization on the other hand does morphological analysis, uses dictionaries and often requires part of speech information. Thus, lemmatization is a more complex process.

Stems need not be dictionary words but lemmas always are. Another way to say this is that "a lemma is the base form of all its inflectional forms, whereas a stem isn't".

Wordforms are either inflectional (change of tense, singular/plural) or derivational (change of part of speech or meaning). Lemmatization usually collapses inflectional forms whereas stemming does this for derivational forms.

Stemming may suffice for many use cases in English. For morphologically complex languages such as Arabic, lemmatization is essential.

There are two types of problems with stemming that lemmatization can solve:

    + Two wordforms with different lemmas may stem to the same result. Eg. 'universal' and 'university' result in same stem 'univers'.
    + Two wordforms of same lemma may end as two different stems. Eg. 'good' and 'better' have the same lemma 'good'.

### Which are the available models for lemmatization?

The classical approach is to use a **Finite State Transducer (FST)**. FST models encode vocabulary and string rewrite rules. Where there are multiple encoding rules, there's ambiguity. We can think of FST as reading the surface form from an input tape and writing the lexical form to an output tape. There could be intermediate tapes for spelling changes, etc.

One well-known tool is Helsinki Finite State Toolkit (HFST) that makes use of other open source tools such as SFST and OpenFST.

Chrupała formalized lemmatization in 2006 by treating it as a string-to-string transduction task. Given a word w, we get its morphological attributes m. To obtain the lemma l, we calculate the probability P(l|w,m). This uses features based on (l,w,m). It then trains a **Maximum-Entropy Markov Model (MEMM)**, one each for POS tags and lemmas. Müller improved on this by using **Conditional Random Fields (CRFs)** for jointly learning tags and lemmas.

One researcher combined the best of stemming and lemmatization.

### Which are the neural network approaches to lemmatization?

A well-known model is the **Sequence-to-Sequence (seq2seq)** neural network. Words and their lemmas are processed character by character. Input can include POS tags. Every input is represented using word embeddings.

To deal with lemma ambiguity, we need to make use of the context. **Bidirectional LSTM** networks, that are based on RNNs, are able to do this. They take in a sequence of words to produce context-sensitive vectors. Then the lemmatizer uses automatically generated rules (pretrained by another neural network) to arrive at the lemma. However, such ambiguity is so rare that seq2seq architecture may be more efficient. **Encoder-decoder** architecture using GRU is another approach to handle unseen or ambiguous words.

Turku NLP based on NN provides one of the state-of-the-art lemmatizers. Other good ones are UDPipe Future and Stanford NLP, although the latter performs poorly for low-resource languages, for which CUNI x-ling excels.

### Could you mention some tools that can do lemmatization?

In Python, NLTK has `WordNetLemmatizer` class to determine lemmas. It includes the option to pass the part of speech to help us obtain the correct lemmas. Other Python-based lemmatizers are in packages spaCy, TextBlob, Pattern and GenSim.

Stanford's LemmaProcessor is another Python-based lemmatizer. It allows us to select a seq2seq model, a dictionary model or a trivial identity model. For Chinese, a dictionary model is adequate. In Vietnamese, lemma is identical to original word. Hence, identity model will suffice.

TreeTagger does POS tagging plus gives lemma information. It supports 20 natural languages. Another multilingual framework is GATE DictLemmatizer, which is based on HFST and word-lemma dictionaries available form Wiktionary. We can use wiktextract to download and process Wiktionary data dumps.

LemmaGen is an open source multilingual platform with implementations or bindings in C++, C# and Python. In .NET, there's LemmaGenerator. There's a Java implementation of Morpha.

## Milestones

1968

It's in the 1960s that morphological analysis is formalized. Chomsky and Halle show that an ordered sequence of rewrite rules convert abstract phonological forms to surface forms through intermediate representations.

1972

Douglas C. Johnson shows that pairs of input/output can be modelled by **finite state transducers**. However, this result is overlooked and rediscovered later in 1981 by Ronald M. Kaplan and Martin Kay.

1983

Thus far, rules have been applied in a cascade. Kimmo Koskenniemi invents **two-level morphology**, where rules can be applied in parallel. Rules are seen as symbol-by-symbol constraints. Lexical lookup and morphological analysis are done in tandem. It's only in 1985 that the first two-level rules compiler is invented.

1997

Karttunen et al. show how we can **compile regular expressions** to create finite state transducers. The use finite state transducers for morphological analysis and generation is well known but it's application in other areas of NLP are not well known. The authors show how to use them for date parsing, date validation and tokenization.

2000

In a problem related to lemmatization, Minnen et al. at the University of Sussex show how to **generate words** in English based on lemma, POS tag and inflection form to be generated. They write high-level descriptions or rules as regular expressions, which *Flex* compiles into finite-state automata.

2006

Grzegorz Chrupała publishes *Simple data-driven context-sensitive lemmatization*. Lemmatization is modelled as a classification problem where the algorithm chooses one of many "edit trees" that can transform a word to its lemma. Such trees are induced from wordform-lemma pairs. This work leads to a PhD dissertation in 2008 and the system is named *Morfette*.

2015

Many NLP systems take a pipeline approach. They do tagging followed by lemmatization, since POS tags can help the lemmatizer disambiguate. But there's a mutual dependency between tagging and lemmatization. Müller et al. present a system called *Lemming* that jointly does POS tagging and lemmatization using Conditional Random Fields (CRFs). It can also analyze OOV words. This work sets a new baseline for lemmatization on six languages.

2016

At the SIGMORPHON Shared Task, it's noted that various neural **sequence-to-sequence** models give best results. In 2018, a seq2seq model is used along with novel context representation. This model, used within TurkuNLP in the CoNLL-18 Shared Task, gives best performance on lemmatization.

2018

Bergmanis and Goldwater use **encoder-decoder** NN architecture in a lemmatizer they name as *Lematus*. Both encoder and decoder are 2-layer Gated Recurrent Unit (GRU). They compare its performance against context-free systems. They use character contexts of each form to be lemmatized. Thus, training resources needed are less. They note that context-free systems may be adequate if a language has many unseen words but few ambiguous words.

2019

Malaviya et al. use a NN model to jointly learn morphological tags and lemmas. They use a encoder-decoder model with hard attention mechanism. In particular, they use a 2-layer LSTM for morphological tagging. For the lemmatizer, they use 2-layer BiLSTM encoder and 1-layer LSTM decoder. They compare their results with other state-of-the-art models: Lematus, UDPipe, Lemming, and Morfette.

Lemmatization


# 5G Deployment Options

## Summary

Though 5G has been standardized, it has a number of options. Two network operators can deploy 5G in very different ways. This choice of option depends on the spectrum licensed to an operator, the geographic area they serve (terrain and user density), capabilities of the equipment they use, and business factors (cashflow and decision making).

3GPP has defined options covering both 4G and 5G technologies with respect to Radio Access Network (RAN) and Core Network (CN). These options can guide operators as they migrate from current 4G deployments to 5G deployments.

It's generally expected that operators would first deploy 5G NR, let 4G RAN and 5G NR coexist, and finally deploy 5G Core. This implies that 4G+5G handsets would come out first and they would connect to both 4G eNB and 5G gNB.

## Discussion

### What are the broad challenges in deploying a 5G network?

Ideally, an operator acquires 5G licenses, invests in 5G equipment for both RAN and CN, and deploys the network for full coverage. The operator then asks subscribers to switch to 5G. After a short transition period, the old 4G network is retired.

In reality, subscribers may be slow to migrate since they have to invest in 5G-capable handsets. The operator's 5G subscription plans may be costlier. The 5G network may have poor coverage in many areas where it's been deployed in only the mmWave band.

An incumbent operator has most likely invested heavily on 4G licenses and equipment. Their current 4G licenses may be in spectrum bands not supported by 5G. Perhaps the equipment they use can't easily be upgraded to 5G.

It's also possible that the government has delayed the auctioning of 5G spectrum. Operators don't want to wait. They may want to offer 5G services on 4G licensed spectrum. Even with 5G licenses, they would want 4G to coexist with 5G and steadily migrate to 5G as more 5G subscribers are added.

### Which are the main 5G deployment options?

In LTE, both RAN and CN had to use LTE standards. 5G gives more flexibility. For example, 4G RAN can be combined with 5G Core or 5G NR can be combined with 4G EPC. This gives rise to two broad deployment scenarios:

    + **Standalone (SA)**: Uses only one radio access technology, either LTE radio or 5G NR. Both control and user planes go through the same RAN element. Deployment and network management is perhaps simpler for operators. Inter-RAT handover is needed for service continuity. Under SA, we have option 1 (EPC + 4G eNB), option 2 (5GC + 5G gNB), and option 5 (5GC + 4G ng-eNB).
    + **Non-Standalone (NSA)**: Multiple radio access technologies are combined. Control plane goes through what's called the master node whereas data plane is split across the master node and a secondary node. There's tight interworking between 4G RAN and 5G NR. Under NSA, we have option 3 (EPC + 4G eNB master + 5G en-gNB secondary), option 4 (5GC + 5G gNB master + 4G ng-eNB secondary), and option 7 (5GC + 4g ng-eNB master + 5g gNB secondary).

### What are the differences across options 3, 3a and 3x?

In all three options, control plane is between EPC and eNB via S1-C interface, and eNB and gNB via X2-C interface. There's no direct signalling traffic between EPC and gNB. The differences are in how user plane traffic is routed. Below we describe downlink but it applies to uplink as well.

In option 3, user plane traffic is from EPC to eNB where **PDCP sublayer splits** the traffic so that some traffic is sent to gNB across the X2-U interface. In option 3a, EPC has a direct S1-U interface to gNB. In this option **EPC splits** the traffic. Option 3x is a hybrid: EPC splits the traffic for eNB and gNB, and/or gNB PDCP sublayer sends some traffic to eNB. For example, eMBB services use 5G NR whereas VoLTE uses LTE radio.

gNB connected to EPC via S1-U is more specifically called **en-gNB**. It's part of E-UTRA-NR Dual Connectivity (EN-DC). The interface between en-gNB and eNB is also called **Xx interface**.

Similar variations of routing user plane traffic involving 5G Core give rise to options 4a, 7a and 7x.

### Could you compare possible migration paths from 4G to 5G?

Since options 1 and 3 use EPC, they can't support many 5G use cases. It's been noted that,

> There's no real 5G without 5G Core.

However, option 3 enables faster time-to-market since core network can be upgraded later. Due to 5G NR, users can experience better throughput. However, with this increased traffic, EPC may become a bottleneck. Traffic flow is split at the EPC.

From NSA option 3, the operator can migrate to NSA option 7 or SA option 5. With both these options, 5GC enables all 5G services. eNB and en-gNB are upgraded to ng-eNB and gNB respectively to connect to 5GC. Option 3 will continue to support UEs that can't talk to 5GC.

It's also possible to add SA option 2 to complement NSA option 3. An alternative path is to deploy SA option 2 from the outset, thus immediately enabling all 5G use cases. However, operator has to acquire and deploy 5G equipment. Ideally, NR coverage is achieved on a wide area. Otherwise, frequent inter-RAT handover to SA option 1 or NSA option 3 may occur.

### What scenarios can benefit from 5G deployment options 4, 5 and 7?

Options 4, 5 and 7 enable operators to continue using legacy 4G equipment while connecting to 5GC. With the higher bandwidths of options 4 and 7, all 5G use cases are possible.

Migration paths 3→5, 3→7, 3→4+2, 3→4, 7→4, 5→4, 1→4, 1→7, and 4→2 have been suggested.

For some, **option 5** is not attractive. Legacy 4G UEs have to be replaced. eNB has to be upgraded substantially. Lots of interoperability testing are needed. If UE moves out of 5G NR coverage (option 2), traditional MBB/voice use cases can be supported simply with option 1 and inter-RAT handover. In the long term, improving option 2 coverage is a better path. **Option 7** depends on option 5 and inherits the same problems.

**Option 4** is an extension of option 2. Using dual connectivity, LTE radio is added to 5G NR anchor. This improves coverage and bandwidth. However, it requires upgrade to eNB, gNB and UE, along with necessary interoperability testing. Instead, it would be better to focus on improving option 2 coverage.

### Could you highlight the differences among eNB, gNB, ng-eNB and en-gNB?

3G's *NodeB (NB)* has become *evolved NodeB (eNB)* in 4G. In 5G, this has evolved to *next generation NodeB (gNB)*. ng-eNB and en-gNB are variations of eNB and gNB respectively, depending on CN.

We compare the different RAN elements:

    + **eNB**: A 4G network element. Connects to a 4G UE and EPC. This relates to options 1 and 3.
    + **gNB**: A newly introduced 5G network element. Connects to a 5G UE and 5G Core. This relates to options 2, 4 and 7.
    + **en-gNB**: Sits in a 4G RAN and connects a 5G UE to EPC. Both 4G and 5G radio resources are active using dual connectivity. eNB is the master node while en-gNB is the secondary node. "en" refers to E-UTRA New Radio. This relates to option 3.
    + **ng-eNB**: Connects to 5G Core but serves a 5G UE over 4G radio. "ng" refers to Next Generation. This relates to options 4, 5 and 7. There's dual connectivity in option 4 (gNB is master, ng-eNB is secondary) and option 7 (ng-eNB is master, gNB is secondary).

### How do 5G deployment options map to spectrum bands?

Current 4G deployments might be in sub-1GHz and 1-3GHz bands. 5G NR is then deployed in mmWave band and possibly in mid-bands 3.5-8GHz. This brings higher throughput/capacity/density and lower latency. At the same time, 4G RAN ensures good wide-area coverage and serves subscribers who haven't migrated to 5G. With Dual Connectivity (DC), high-band NR downlink can be combined with low-band 4G uplink. More throughput can be achieved via inter-band Carrier Aggregation (CA). This is **option 3**.

When the operator activates 5G Core, **option 2** comes into play. Initially, this will be limited to 5G NR in mmWave band and mid-bands 3.5-8GHz for Fixed Wireless Access (FWA) and industrial deployments. At a later time, when 4G spectra are re-farmed, or via spectrum sharing, option 2 can be deployed to wider areas. When a UE moves out of option 2 5G NR coverage, it triggers intersystem handover to EPC, either in option 3 or 1.

Even when 5G is widely deployed, for Massive Machine Type Communications (mMTC), NB-IoT and LTE-M will be used in **option 1**.

### What's a possible migration path from LTE EPC to 5G Core?

As an example, we describe Samsung's offering. They claim that their LTE EPC is already virtualized and provides Control and User Plane Separation (CUPS). For 5G NSA, EPC software is upgraded for dual connectivity. For 5G SA, EPC *Network Elements (NEs)* become 5GC *Network Functions (NFs)*. Specifically, GW-C, GW-U, HSS and PCRP are upgraded to SMF, UPF, UDM and PCF respectively. New NFs AMF, NRF, NSSF, NEF and UDF are introduced. LTE's MME functionality goes into AMF, SMF and AUMF. The final deployment is a **common core** that covers LTE, 5G and Wi-Fi.

5G Core is a Service-Based Architecture (SBA). NFs will be **virtualized** in the cloud and implemented using microservices and containers. LTE's stateful NEs that store UE state will be replaced with **stateless** NFs. Increasingly, cloud native architecture will be used with lightweight containers. There will be centralized orchestration of containers, network slicing, centralized operation, and centralized analytics. Overall, network control, monitoring and optimization will be **automated**. This will be the bigger impact of moving from LTE EPC to 5G Core.

## Milestones

Dec
2017

3GPP publishes Release 15 "early drop". This includes **NSA option 3**. Corrections to this option are made in June 2018.

Apr
2018

GSMA's report shares Korea Telecom's 4G to 5G migration plan. Early deployments are likely to be NSA option 3 with 5G NR only in 28GHz mmWave band. As 5GC gets introduced, multi-RAT interworking would become important. 5G NR coverage would improve with 3.5GHz band. NSA option 7 would be used, with eLTE being the anchor. Finally, EPC would be retired.

Jun
2018

3GPP publishes Release 15 "main drop". This includes **SA option 2** and **SA option 5**.

Sep
2018

A white paper by Nokia recommends either option 3 or option 2 for initial 5G rollout. The report also identifies **option 3X** in which high bandwidth traffic flows are routed to 5G gNB to avoid overloading 4G eNB. LTE user plane (SGW/PGW) would require performance improvements and a distributed architecture.

Mar
2019

3GPP publishes Release 15 "late drop". This includes **SA option 4** and **SA option 7**.

5G Deployment Options


# React Hooks

## Summary

React is based on components. The logic to manage component state and update the user interface is encapsulated with the component. But what if we wish to reuse stateful logic across components, such as for logging or authentication? React allows this via mixins, higher-order components and render props. However, these complicate the component hierarchy. React Hooks provide a cleaner and more elegant way of doing this.

React's function components are stateless and without side effects. React Hooks combines the simpler syntax of function components but allows for states and side effects.

Class components continue to be available in React. Hooks can coexist with class components, although they can't be used inside a class. In fact, React concepts such as props, state, context, refs and lifecycle are still relevant in Hooks.

## Discussion

### What are the advantages of using React Hooks?

React Hooks solves the problems of class components. With Hooks, we can reuse stateful logic without complicating component hierarchy.

Class components may have code repetition. For example, code may be repeated in lifecycle events `componentDidMount` and `componentDidUpdate`. Custom Hooks enable easy code reuse.

Developers coming from other languages have a hard time understanding the `this` reference. Class components tend to be verbose. They don't minify well. Hot reloading is unreliable. Hooks offer developers a way to more effectively use React's features.

Code is harder to organize logically in class components. For example, fetching data and setting up event listeners are doing different things but they're probably mixed together in the same lifecycle method. Fetching data and its clean up is also likely to be split between `componentDidMount` and `componentWillUnmount`. With Hooks, we can organize logical units of code into functions. For this reason, Dan Abramov noted,

> Hooks apply the React philosophy (explicit data flow and composition) *inside* a component, rather than just *between* the components.

### Could you explain React Hooks with an example?

The figure shows an example that uses two types of Hooks: `useState` and `useEffect`. We note that Hooks are used within a function component, not a class component. Instead of the `render()` method found in class components, we simply return the component at the end, whose syntax is in JSX format. We no longer use the `this` variable that was common in class components.

This component maintains a state called `count` that can be set with a function named `setCount()`. Both these are returned by `useState`. `setCount()` is triggered when the button is clicked. This updates the state, which in turn triggers a re-rendering of the component.

After the component is rendered or re-rendered, the `useEffect` Hook is triggered. This simply updates the document title in this example.

Finally, we observe that Hooks are called in response to UI events, DOM rendering, etc. Therefore, they're not independent of React lifecycle methods. React Hooks "hook into" React state and lifecycle features from function components.

### Which are the Hooks that React provides?

Among the basic Hooks are `useState`, `useEffect`, and `useContext`. Additional Hooks include `useReducer`, `useCallback`, `useMemo`, `useRef`, `useImperativeHandle`, `useLayoutEffect`, and `useDebugValue`. We briefly describe these (refer to the Hooks API for more details):

    + `useState`: Initializes and manages component state.
    + `useEffect`: Called after DOM is rendered to the screen. Side effects such as logging or timers can be executed here.
    + `useContext`: Accepts a context object and returns its current context value based on the nearest ancestor's value prop.
    + `useReducer`: Better than `useState` for managing complex state logic.
    + `useCallback`: Given a callback and its dependencies, obtain a memoized callback. Useful to prevent unnecessary rerenders.
    + `useMemo`: Optimize expensive computations by returning a memoized value. Recomputes happen only when dependencies change.
    + `useRef`: Create a container to hold a mutable value. Useful for accessing child elements imperatively.
    + `useImperativeHandle`: Customize instance value that's exposed to parent components. However, prefer to avoid imperative code.
    + `useLayoutEffect`: Like `useEffect` but called synchronously before browser "paints" the screen. Useful to avoid flicker when an effect manipulates DOM.
    + `useDebugValue`: Display a label for custom Hooks in React DevTools. Useful when such Hooks are part of shared libraries.

### What's the React Hooks lifecycle?

Function components with Hooks can be mounted, updated and unmounted. Each of these go through the render phase and commit phase. During the latter the DOM is updated.

Different Hooks are called at different phases of the React Hooks lifecycle. `useMemo` is called during the render phase when the component is mounted. `useState`, `useReducer`, `useContext` and `useCallback` are called during the render phase when component state is updated. Render phase is not the place for executing side effects.

`useLayoutEffect` and `useEffect` are called after React has updated the DOM but there's an important difference. `useLayoutEffect` is called synchronously. It will block the browser before browser has had a chance to "paint" the screen. `useEffect` is called after browser has painted the screen. Generally, `useEffect` is preferred unless the Hook makes visual updates to the DOM.

Both `useLayoutEffect` and `useEffect` are also called when the component is unmounted, provided these Hooks return a callback function.

### Could you describe the State Hook?

The most basic Hook is `useState`. This takes an initial value and returns a stateful value plus a function to update it. When the component first renders, returned value is same as initial value. Next time the component renders, the current state is available, that is, it's not initialized again. This is what React Hooks enables. For expensive computation, a function can be passed into `useState`.

For updating state, a value can be passed to the update function, `fn(val)`. If current value is needed for the update, a function can be passed, `fn((c) => c+1)`. Where state contains sub-values such as `{name: 'john', age: 22}`, `useState` doesn't merge the values. We need to use the spread syntax, `{...prevState, ...updatedValues}`.

### Could you describe the Effect Hook?

Hook `useEffect` allows us to perform side effects, such as fetching data or updating DOM. Having these side effects inside the main body of the function component (used during the render phase) can result in UI bugs and inconsistencies. Hence, use this Hook to implement side effects.

This Hook enables imperative programming, as a feature that complements React's purely functional programming approach. It accepts a function as an argument and optionally a second argument. Without the second argument, the Hook is called after every render. If the second argument is an empty array, the Hook is called only once. If array is non-empty, the Hook is called conditionally only when those dependencies change.

The Hook can also return a function, which is called when the component unmounts.

Having this Hook inside the component allows it to access component state. This state is never stale since with each render we recreate the function wired to the Hook.

### What are the rules of Hooks?

React Hooks have only two essential rules that developers need to follow. Hooks should be called only at the top level within the component. Don't call them within loops, conditions or nested functions. This is because React Hooks expects that Hooks are always called in the same order whenever the component renders.

The second rule is that don't call Hooks from regular JavaScript functions. Call them only from React function components or from custom Hooks. This ensures that all stateful logic is visible within the component's source code.

To ensure that components meet these rules, developers can install the *React Hooks ESLint Plugin*, aka `eslint-plugin-react-hooks` on NPM. The plugin will warn developers when these rules are violated. When we create the default React app, this plugin is installed by default.

### Can class components interoperate with React Hooks?

React Hooks don't replace class components. An application can use class components, function components and Hooks. In fact, these approaches can be used in the same DOM tree. However, at code level, it's not possible to use React Hooks within a class.

Developers can adopt Hooks for new components. Legacy components could be migrated to Hooks incrementally. Particularly for render props and higher-order components, Hooks offer a simpler approach and reduce nesting of DOM elements.

To relate to component lifecycle methods, we note the following:

    + `constructor`: Not required in Hooks. Initialization can be done with `useState`.
    + `getDerivedStateFromProps`: Update component while it's being rendered.
    + `shouldComponentUpdate`: Wrap a function component within `React.memo`, a higher-order component. This is not exactly a Hook. Compares props (not states) in a shallow manner.
    + `render`: Function component body is returned.
    + `componentDidMount`, `componentDidUpdate`, `componentWillUnmount`: `useEffect` Hook can implement all three.
    + `getSnapshotBeforeUpdate`, `componentDidCatch`, `getDerivedStateFromError`: No equivalent in Hooks.

### What are some criticisms of React Hooks?

For someone used to class components, React Hooks is something new to learn. Learning fatigue is common among JavaScript developers. The new knowledge is React-specific and can't be reused anywhere else.

React Hooks can't be used with class components. This means that legacy code needs to be migrated, which presents the risk of introducing bugs. Developers can't also opt-out, despite what the documentation claims. There's a good chance that the ecosystem and dependencies will move to React Hooks. When that happens, our own projects will have no choice but to adopt it.

Some may find that the rules of React Hooks are rather limiting. Reading and understanding component code and its control flow can be hard when a component uses many types of Hooks. Control flow is determined by component lifecycle and not the ordering in the source code.

Where dependencies are passed to a Hook, object comparison works as intended for primitive types but not for arrays or other objects. Comparison is by reference and developers can't customize this.

### Where can I learn more about React Hooks?

The section on React Hooks in the React official documentation is a good place to start. The same documentation also has the Hooks API Reference. There's a section on building custom Hooks.

Those who like to learn from examples, can visit the useHooks website. Some examples include show/hide UI elements, subscribe to Firestore data, handle asynchronous calls, render component based on current state of user authentication, sync state to local storage, and more.

Xebia has published a useful React Hooks cheat sheet. This includes class-style and Hooks-style code for comparison. Emmanuel's cheat sheet is also worth reading.

Gunawardhana has shared a list of ten useful React Hooks libraries. Mostly, these are custom Hooks built on top of the built-in React Hooks.

Aryan's shares one approach to testing React Hooks.

## Milestones

May
2013

Facebook **open sources React** at the JSConf US. An earlier prototype of this can be traced to 2011 and then developed within Facebook during 2012.

Feb
2019

React v16.8.0 is released. This introduces **React Hooks**. In the change log, it's described as "a way to use state and other React features without writing a class." This change affects React DOM, React DOM Server, React Test Renderer and React Shallow Renderer. All of these must be at least version 16.8 to use React Hooks. This release also includes the **React Hooks ESLint Plugin** for linting.

Mar
2019

React Hooks becomes available in **React Native v0.59**.

Aug
2019

React v16.9.0 is released. The React Hooks ESLint Plugin is updated to treat the use of Hooks at module level (outside a function component) as a violation.

Oct
2020

React v17.0.0 is released. As an experimental Hook, `unstable_useOpaqueIdentifier` is added.

React Hooks


# Antivirus Software

## Summary

Malicious software or malware are code that can harm computer systems and steal personal information, causing severe damage such as financial loss. These malware can enter through various ways including as an email attachment, on a USB drive, or user visiting infected websites or clicking malicious links.

Antivirus software is a program that protects computer systems from malicious software. Antivirus software needs privileged access to the system to function properly. Many antivirus software provide real-time threat protection against harmful applications. It's also possible to manually perform security scans.

It's advisable to use antivirus programs on desktops. They can be optional in mobile phones if users don't download any app from unofficial sources or click suspicious links.

## Discussion

### What are the features of an antivirus software?

Below are some common features of an antivirus software:

    + **Real-time scanning**: Antivirus software that do security checks while using the system are real-time scanning software.
    + **Quarantine**: An isolated place where harmful files are kept.
    + **Bank mode**: A feature that provides a clean and safe virtual environment within the real desktop environment.
    + **Ransomware Shield**: Secures personal photos, documents, and other files from being modified, deleted, or encrypted by ransomware attacks.
    + **Remote Access Shield**: Protects the computer from remote and unauthorized access.
    + **Sandbox**: A sandbox is a system in which suspicious files are tested and analysed inside a virtual machine to detect malware.
    + **Hack alerts**: Warns if sensitive data is leaked on the Dark Web and other online sources.
    + **Password protection**: Protects passwords stored in web browsers.
    + **Webcam shield**: Restricts applications and malware from gaining unauthorized access to the webcam.
    + **Masking IP address**: Hide the IP address while browsing internet.
    + **Freemium vs Paid versions**: Basic features with free versions. More features with paid versions.

### What are the types of antivirus programs?

**Standalone antivirus software** are specialized to scan and remove certain viruses. They can easily be carried from one place to another in a USB drive. These type of antivirus programs do not provide real-time protection. Windows defender offline, Metascan client and Microsoft scanner are some examples.

**Security software suites** provide more security features than antivirus software. They not only scan and remove the viruses but also protect from other malicious attacks. They provide complete protection to the user's computer. They include extra security features such as anti-spyware, firewall, site authentication and parental controls. Bitdefender Total Security, Norton 360 Deluxe, and Kaspersky Total Security are some of its examples.

**Cloud-based antivirus software** analyse the files on the cloud instead of on the user systems. They have two parts: (a) client side that's installed on user systems; (b) cloud side that captures the data gathered from client side and runs the scans. The latter saves a lot of memory and resources on user computers.

### What's the architecture of antivirus software?

An antivirus software consists of four main components namely Anti-virus Manager, Anti-virus Engine, Malware Signature Database and Anti-virus Driver.

The Anti-virus Manager handles overall management functions. It's responsible for executing malware scans, reviewing the results of malware restoration, raising alarms, and setting a periodic scan policy for malware. It updates the Malware Signature DB and the Anti-virus Engine.

The Anti-virus Engine is responsible for handling malware in the system. It analyses the file information and marks it safe or harmful based on the scanning.

In the Malware Signature Database, signatures of malware are kept. The antivirus engine evaluates the suspicious file by comparing its characteristics to these signatures.

The Anti-virus Driver is a real-time malware entry monitoring function that revokes access to malware-infected user files. It checks the input and output of the file and determines the exact file path to check for malware.

### Which are major virus detection methods used by antivirus software?

There are mainly two virus detection techniques used by antivirus programs namely Heuristic-based detection and Signature-based detection.

In **signature-based detection**, the antivirus software scans the files and compares it with the predefined malicious code present in its database. The signature could include malicious network attack behaviour, content of email subject lines, file hashes, known byte sequences or malicious domains. If a file matches any malicious code, it flags it as virus and takes necessary actions. The antivirus can't detect any new malware or even any updated variant of malware that is not written in definition file. With rapid growth in variants of malware, this detection method becomes ineffective in long run.

In **heuristic-based detection** the antivirus software uses data mining and machine learning techniques to learn the behaviour of an executable file. For instance, it may detect commands to deliver payloads disguised within a Trojan horse virus or those used to distribute a worm virus. It's able to identify new or altered variants of malware even in the absence of updated malware code in the database. It's commonly used in combination with signature-based detection.

### What are other virus detection techniques used by antivirus software?

**Behavioural-based detection** analyses every single line of code in the file before its execution. If it finds the objective of the file suspicious or harmful (such as taking access to any critical or irrelevant files, processes, or internal services) it flags it as malware. Degree of potential danger is calculated by the antivirus software and required actions are taken afterwards. Some examples of malicious behaviours include any attempt to discover a sandbox environment, disabling security controls, installing rootkits, and registering for autostart.

**Sandbox** is a system for malware detection that runs any suspicious object in a virtual machine (VM). If the object performs malicious actions in VM, the sandbox registers it as malware. VMs are isolated from the real system.

**Cloud-based detection** identifies malware by collecting data from host computers and analysing it on the provider's infrastructure, instead of performing the analysis locally on the system. It's done by capturing relevant details about the file and the context of its execution and providing them to the cloud engine for processing.

### What are the differences between server, desktop and mobile antivirus software?

Most antivirus software are built for **desktops**, specially Windows as it has the largest user base. Many antivirus software vendors offer their product at different price ranges, starting with free version which offer only basic protection. These free versions often don't protect against malicious attachments in emails, fishy website links and other type of cyber attacks. Microsoft offers a basic free antivirus software with Windows known as Windows Defender.

**Mobile antivirus** software differ than the desktop ones as they have less access and control over the host device. Mobile antivirus software can't do OS file access, website filtering, in-memory scans or real-time protection engines. Android users need antivirus software more than the iOS users as iOS devices have better security features.

A good **server antivirus** software detects virus before it infects the system. It also stops data transfer between server and external drives. It defends information on servers from every kind of malicious applications. It provides security for different server subsystems such as email, firewall and proxy.

### What are the challenges faced with an antivirus software?

One of the biggest challenges faced with antivirus software is that it slows down the computer. Antivirus software loads each time when the system boots up which makes booting slower. Without a powerful processor and a good amount of memory, antivirus software can slow down systems to a high extent.

No antivirus software provide 100% security from all kinds of cyber threats. Inexperienced users might fall into a false assumption of being completely secure. If the antivirus uses heuristic detection, it might flag harmless websites and software as malicious.

Antivirus software runs at the kernel level of the operating system. It has privileged access to all the core files of the OS and system. Hence it becomes a potential target for cyber attacks. The US National Security Agency (NSA) and the UK Government Communications Headquarters (GCHQ) intelligence agencies have been exploiting anti-virus software to spy on users. In fact, it's been noted that popular software such as Acrobat Reader, Microsoft Word or Google Chrome are harder to exploit than 90% of the anti-virus products.

### Are there any alternatives of antivirus software?

Below are some alternatives of antivirus software:

    + **Network firewall**: Network firewalls stop unknown programs and processes from accessing the system but unlike antivirus programs, they don't identify or remove any virus present in the system. They provide safety from malware coming from other websites on a network.
    + **Cloud antivirus**: Cloud antivirus offer features like other antivirus programs but rely on cloud to process malware. They're faster than usual antivirus software as it does most of its computing on the cloud. Due to up-to-date database of malware and viruses they can find new versions of malware. Panda Cloud Antivirus and Immunet are examples of cloud antivirus.
    + **Online scanning**: Some antivirus vendors provide websites that offer free scanning of system. These can be used without downloading heavy software in local systems.
    + **Specialized tools**: Virus removal tools are helpful in removing stubborn/sticky viruses. Avast Free Anti-Malware, AVG Free Malware Removal Tools and Avira AntiVir Removal Tool are some examples of such tools.

## Milestones

1971

The first known virus named as Creeper Virus is discovered. It infects DEC PDP-10 mainframe computers running the TENEX operating system. Ray Tomlinson writes a program to delete creeper virus. This program is known as **The Reaper**. Some people consider the Reaper as the first antivirus program but Reaper is itself a virus made to remove the Creeper Virus.

1987

Bernd Fix becomes the innovator of **first antivirus product**. However, there are competing claims for it. John McAfee establishes the McAfee company. Peter Paško, Rudolf Hrubý, and Miroslav Trnka create the first version of NOD antivirus. The first two heuristic antivirus utilities are released: Flushot Plus by Ross Greenberg and Anti4us by Erwin Lanting.

2000

Rainer Link and Howard Fuhs start the first **open-source antivirus engine** called OpenAntivirus Project.

2001

Tomasz Kojm releases the first version of ClamAV, the first ever open-source antivirus engine to be commercialised.

2005

AV-TEST (a German based independent organization) reports that there are 333,425 unique malware samples in their database. In 2007 alone, AV-TEST reports 5,490,960 new unique malware samples.

2008

McAfee Labs adds the industry-first **cloud-based anti-malware** functionality to VirusScan (an antivirus) under the name Artemis.

2010

A faulty update on the AVG anti-virus suite damages 64-bit versions of Windows 7, rendering it unable to boot, due to an endless boot loop created. If not properly managed, this highlights the danger of antivirus programs since they have privileged access.

2011

Microsoft Security Essentials (MSE) removes the Google Chrome web browser, rival to Microsoft's own Internet Explorer. MSE flags Chrome as a Zbot banking trojan. In 2017, Google Play Protect on Moto G4 flags a Bluetooth system app as malware, causing Bluetooth functionality to become disabled for all apps. In 2022, Microsoft Defender flags all Chromium-based web browsers and Electron-based apps (Whatsapp, Discord, Spotify) as a severe threat. There are some examples of **false positives**.

Antivirus Software