# 5G NR PHY

## Summary

5G NR PHY is designed to meet the main 5G use cases, namely eMBB, mMTC and URLLC. While it's an evolution of LTE PHY, there are many aspects that are unique to 5G NR PHY.

PHY layer sits at the bottom of the 5G NR protocol stack, interfacing to MAC sublayer higher up via transport channels. It provides its services to MAC and is configured by RRC. PHY supports downlink (gNB-to-UE), uplink (UE-to-gNB) and sidelink (UE-to-UE) communications.

Some of the main features include a wide spectrum from sub-GHz bands to mmWave bands, an OFDM-based air interface, scalable numerology, deployments from indoor picocells to outdoor macrocells, FDD and TDD support, flexible and self-contained slot structure, modulation up to 256QAM, polar and LDPC codes, Hybrid-ARQ (HARQ), bandwidth parts, CORESETs, beamforming, and massive MIMO.

## Discussion

### Could you share some technical details of 5G NR PHY?

**5G spectrum** spans a wide range: FR1 (410-7125 MHz) and FR2/mmWave (24250-52600 MHz). **UE bandwidth** per component carrier is of range 5-100MHz (FR1) and 50-400MHz (FR2). Higher bandwidth allocations can be achieved with carrier aggregation.

**Waveform** used in 5G is OFDM with Cyclic Prefix (CP). In uplink, an optional transform precoding of DFT spreading is done before sub-carrier mapping.

**Sub-Carrier Spacing (SCS)** is flexible from 15kHz to 120kHz, with higher values applicable in FR2. **Slot duration** is also flexible from 1ms at 15kHz SCS to 125µs at 120kHz SCS. SCS 240kHz is only for control. **Cyclic prefix** at 60kHz SCS can be normal or extended.

For **duplexing**, both FDD and TDD are supported in FR1. Only TDD is applicable in FR2. In TDD, DL/UL split can be dynamically adjusted.

### What modulation schemes and channel coding are supported in 5G NR PHY?

**Modulation schemes** supported are QPSK, 16QAM, 64QAM and 256QAM. For DFT-s-OFDM in uplink, 5G NR introduces π/2-BPSK for better power efficiency at lower data rates, necessary for mMTC services. DFT spreading in uplink helps coverage-limited scenarios.

**Channel coding** is based on Low Density Parity Check (LDPC) code, applied on transport blocks. A large TB is segmented into multiple equal code blocks and LDPC coding is applied on the code blocks. Polar code is used for BCH, DCI and UCI. In addition, block code is used for UCI.

### Which are the main physical radio resources in 5G NR?

Time and frequency are the two main resources. **Time** is organized into OFDM symbols, slots, subframes and frames. **Frequency** is organized into sub-carriers as needed for OFDM with SCS determined by numerology. Unlike LTE, both have flexible configurations due to 5G's scalable numerology.

5G NR defines the following:

    + **Resource Element (RE)**: The smallest unit of resource. It's one sub-carrier for one OFDM symbol duration.
    + **Resource Block (RB)**: 12 consecutive sub-carriers in the frequency domain. It's not defined for the time domain. *Common Resource Blocks* are numbered from zero for each SCS. A UE is configured one or more bandwidth parts. *Bandwidth part* is a contiguous set of common RBs. *Physical Resource Blocks (PRBs)* are numbered from zero within the bandwidth part. Thus, a UE uses PRBs for actual communication.
    + **Resource Grid**: A combination of subcarriers and OFDM symbols. Defined for each numerology, carrier and antenna port. One set of resource grids is defined for downlink, uplink and sidelink each.
    + **Resource Element Group (REG)**: One PRB and one symbol.
    + **Control Resource Set (CORESET)**: Multiple PRBs with 1, 2 or 3 symbols.

### What's the frame and slot structure in 5G NR?

A frame is 10ms. A subframe is 1ms that's divided into slots. Slot duration depends on numerology. At 15kHz, a subframe has a single slot. At 30kHz, a subframe has two slots, each slot being 500µs. Likewise, we have slot durations 250µs@60kHz, 125µs@120kHz and 62.5µs@240kHz.

A slot has 14 OFDM symbols but only 12 symbols at 60kHz when using extended cyclic prefix. At higher SCS, symbols and slots are shorter. 5G also permits mini-slot transmissions of 2, 4 and 7 symbols.

Because the different numerologies are of the form \(2^µ\), they can coexist. Regardless of numerology, symbols and slots are time aligned. Services with different requirements of bandwidth and latency can be multiplexed on the same frequency.

TDD slot structure is **self-contained**. It allows for fast and flexible TDD switching. DL control, DL data, guard period and UL control are in the same slot. Thus, DL data and its acknowledgment can happen in the same slot. This is also possible for UL data. Symbol allocation to DL or UL can be switched every slot.

### What are the different physical channels used in 5G NR PHY?

We note the following physical channels (with transport channel in parenthesis):

    + **Downlink**: PBCH (BCH), PDSCH (DL-SCH, PCH), PDCCH
    + **Uplink**: PRACH (RACH), PUSCH (UL-SCH), PUCCH
    + **Sidelink**: PSBCH (SL-BCH), PSSCH (SL-SCH), PSCCH, PSFCHPDCCH, PUCCH, PSCCH and PSFCH are standalone physical channels, that is, they're not mapped to transport channels. PDCCH has Slot Format Indicator (SFI) and Downlink Control Information (DCI) fields. The latter informs scheduling for PDSCH and PUSCH. PUCCH carries Uplink Control Information (UCI). UCI carries channel reports, HARQ-ACK and scheduling request.

### What are the different signals used in 5G NR PHY?

PHY has a few signals for the following purposes:

    + **Synchronization**: Primary Synchronization Signal (PSS) and Secondary Synchronization Signal (SSS). These are transmitted along with PBCH.
    + **Acquisition and Channel Estimation**: Demodulation Reference Signal (DM-RS) in downlink and uplink. Sounding Reference Signal (SRS) in uplink when PUCCH and PUSCH are not scheduled.
    + **Positioning**: Positioning Reference Signal (PRS) in downlink. SRS in uplink.
    + **Phase Tracking**: Phase Tracking Reference Signal (PT-RS) for PDSCH and PUSCH. Helps combat path delay spread and Doppler spread.
    + **Beam Management**: Channel State Information Reference Signal (CSI-RS) in downlink towards a connected UE.

### What are the main functions of 5G NR PHY?

The main functions include error detection on the transport channel and indication to higher layers; FEC encoding/decoding of the transport channel; HARQ soft-combining; rate matching of the coded transport channel to physical channels; mapping of the coded transport channel onto physical channels; power weighting of physical channels; modulation and demodulation of physical channels; frequency and time synchronisation; radio characteristics measurements and indication to higher layers; MIMO antenna processing; RF processing.

Consider the PHY model for DL-SCH. At gNB, Transport Blocks (TBs) arrive from MAC on transport channels. PHY adds CRC to each TB, perhaps involving code block segmentation. Channel coding and rate matching are performed, including for HARQ retransmissions. Data modulation is next, followed by mapping to physical resources. Finally, there's antenna mapping before transmission. At UE, the reverse process happens with CRC used for error detection and indication to MAC.

In the figure, blue boxes are configurable by higher layers. For other channels, some steps may be hardcoded in the specification. For example, all steps are fixed for BCH; coding and rate matching are fixed for PCH.

### What are the main procedures in 5G NR PHY?

The main procedures include Cell search; Power control; Uplink synchronisation and Uplink timing control; Random access related procedures; HARQ related procedures; Beam management and CSI related procedures; Sidelink related procedures; Channel access procedures.

We describe a few:

    + **Cell Search**: UE acquires time and frequency synchronization for a cell and detects the Cell ID. UE receives PSS, SSS and PBCH.
    + **Uplink Power Control**: For PUSCH, PUCCH, SRS and PRACH transmissions. When Dual Connectivity (DC) is active, UE is configured with maximum power for both MCG and SCG.
    + **Random Access**: Of either Type-1 or Type-2. Involves preamble transmission on PRACH (and PUSCH MsgA in Type-2), random access response (RAR) reception on PDCCH/PDSCH, PUSCH transmission based on RAR UL grant (fallback in Type-2), and PDSCH contention resolution.
    + **PDSCH Reception**: Typically first requires decoding DCI from PDCCH.
    + **PUSCH Transmission**: Scheduled dynamically by UL grant in DCI or semi-statically by Type-1 or Type-2 grants configured by RRC.
    + **CSI Measurements & Reporting**: Periodic, semi-persistent or aperiodic. Reports are sent on PUCCH or PUSCH and triggered by DCI when applicable.

### Which are the main 5G NR PHY specifications?

For a high-level overview of 5G NR PHY, PHY section of **TS 38.300** is worth reading. This specification gives an overall description of NR and NG-RAN.

A general description of 5G NR PHY layer is found in **TS 38.201**. Beginners can start with this document. For more details into PHY, the following are useful:

    + **TS 38.202**: Services and functions provided by PHY, downlink/uplink/sidelink models.
    + **TS 38.211**: Physical channels and modulation, frame structure, PHY resources, modulation mapping, OFDM signal generation, scrambling, modulation, up-conversion, layer mapping, precoding.
    + **TS 38.212**: Multiplexing and channel coding, rate matching, transport channels, control information.
    + **TS 38.213**: Physical layer procedures for control, synchronization procedures, uplink power control, random access procedure, UE procedure for reporting and receiving control information.
    + **TS 38.214**: Physical layer procedures for data, power control, procedures related to physical shared channels.
    + **TS 38.215**: Physical layer measurements, UE and NG-RAN measurement capabilities.To learn about RF requirements including operating bands, channel bandwidth and transmitter/receiver characteristics, documents to read are **TS 38.101** in two parts (for FR1 and FR2) for the UE,  and **TS 38.104** for base station.

## Milestones

Dec
2017

3GPP approves the first specifications for 5G, called "early drop" of Release 15.

Dec
2019

First updates to specifications towards Release 16 are published. Some of the new features are Remote Interference Management (RIM), two-step RACH procedure, NR-based access to unlicensed spectrum, Integrated Access and Backhaul (IAB), V2X, eURLLC, NR positioning, MIMO enhancements, Dynamic Spectrum Sharing (DSS) enhancements, Multi-RAT DC/CA, NR-DC and cross-carrier scheduling with different numerologies.

Jul
2020

3GPP finalizes **Release 16** specifications.

5G NR PHY


# Ethical AI

## Summary

We're living in a world where machines and algorithms are increasingly giving recommendations, tagging content, generating reviews and even taking decisions. A number of ethical questions arise. Can we trust machines to do this right thing? Was my loan application processed in a fair manner? Why was my application rejected?

Then there are questions about AI replacing humans in the workplace. With widespread use of personal data, we're worried about privacy and data theft. What happens to us if AI agents acquire human-like cognitive abilities?

Ethical AI addresses these issues by building systems that are safe, fair, transparent, accountable and auditable. To practice ethical AI, it's been said that,

> We are not at the point where you can simply download a tool to do the job. Data ethics is still new and it requires critical thinking.

## Discussion

### Are there real-world examples that highlight the need for ethical AI?

In March 2016, Microsoft released a chatbot named Tay. Tay could interact and learn from real users on social platforms. Soon Tay was exposed to nasty tweets and ended up becoming mean and racist on her own.

A study by ProPublica found that software used to predict future criminals is biased against blacks. Risk scoring done by the software is seen by judges and directly influences sentences. White defendants were often mislabelled as low risk compared to black defendants. Only 20% of those predicted to commit violent crimes went on to do so.

In March 2018, an Uber self-driving vehicle killed a pedestrian. Who is responsible: the distracted driver, the pedestrian, Uber, developers who wrote the code, or a sensor manufacturer? It's unrealistic to expect AI systems to be perfect but determining liability isn't trivial.

More recently, it was found that Facebook's ad delivery algorithms discriminate based on race and gender even when ads are targeted to a broad audience. Influencing factors include user profile, past behaviour and even the ad content.

### What are the common ethical concerns raised by AI?

Ethical AI has the following concerns:

    + **Bias**: AI systems can be biased because they're designed to look for patterns in data and favour those patterns.
    + **Liability**: AI systems can't be perfect. When mistakes are made, who's responsible?
    + **Security**: As AI systems advance, how do we stop bad actors from weaponizing them? What happens if robots can fight and drones can attack?
    + **Human Interaction**: There's already a decline in person-to-person interactions. Are we sacrificing humanity's social aspect?
    + **Employment**: Repetitive, predictable jobs that can be automated will be automated. Those replaced have to retrain themselves in areas where robots can't come in easily, such as, creative or critical thinking.
    + **Wealth Inequality**: Companies rich enough to invest in AI will get richer by reducing cost and being more efficient.
    + **Power & Control**: Big companies that use AI can control and manipulate how society thinks and acts.
    + **Robot Rights**: If AI systems develop consciousness and emotions, should we give them rights? Can we punish AI and make them suffer?
    + **Singularity**: What happens if AI surpasses human intelligence? Will they turn against us to defend themselves?

### How can AI be biased when it's based on trusted data?

Even if data is trusted, it may not be a fair representation of the population. Dataset may be skewed in a number of ways including race, gender, education, or wealth. When algorithms are exposed to this data, they acquire the bias that's present in them.

Bias in AI comes from human biases since we are the ones building the algorithms and creating/selecting data. Minorities, often in low-income groups, lack access to technology. As a result, AI systems are not trained on such data. One infamous example is Google tagging photo of a black woman as "gorilla".

There are many ways in which human biases creep into AI systems. Selection bias, interaction bias and latent bias are some examples.

AI system that's trained primarily on American or European faces will not work well when applied on Asian or African faces. In one example, a Nikon S630 camera hinted to the user that perhaps she has blinked when in reality Chinese people have eyes that are squinty when they smile. In another example, a happy tribal woman was wrongly classified to have "disgusted" emotion.

### What can we do to make AI more ethical?

We need to design our systems to avoid biases. Check that proxy metrics don't introduce bias. We can minimize bias by having diversity in teams, data, and validation.

For transparency, data must be marked with metadata detailing the source, intended use, usage rights, etc. Data must be managed well leading to algorithms that are better traceable, reproducible and fixable. Context is key to transparency and explainability.

When AI systems fail, have a plan to fix them in production. We need to monitor them. Let the purpose be clearly defined and bounded. AI systems should not be allowed to explore unintended pathways.

What we need is a holistic approach. It's not just about technology and tools, but also about leaderships, standards and rules. Microsoft's Satya Nadella has said ethical AI also means how humans interact with AI. We must have empathy, education, creativity, judgement and accountability.

Accenture has suggested setting up AI advisory bodies. Have discussions. Publish guidelines. Engage with stakeholders. The AI Ethics Lab is looking into integrating ethics and AI right at the R&D phase.

### Are there published guidelines or code of conduct for practising ethical AI?

The Institute for Ethical AI & ML has identified 8 ML principles for responsible AI: human augmentation, bias evaluation, explainability by justification, reproducible operations, displacement strategy, practical accuracy, trust by privacy, and security risks. They also identify a 4-phase strategy: by principle, by process, by standards, by regulation. They've also formed the *Ethical ML Network*.

The Future of Life Institute has created a interactive map addressing validation, security, control, foundations, verification, ethics and governance.

In 2018, GE Healthcare published a set of AI principles. Google published its own guidelines of what AI applications should and shouldn't do. Microsoft released ten guidelines for developing conversational AI. One of these guidelines says that developers are accountable until bots become truly autonomous.

## Milestones

2014

Amazon starts using AI for filtering resumes and hiring top talent. But in 2015, it's discovered that the algorithm is biased towards male candidates since it was trained on resumes submitted to Amazon over a 10-year period. This dataset had mostly male applicants since they dominated the tech industry. The algorithm just reinforced the bias. The algorithm is found to have other problems as well and is discontinued by Amazon.

2016

This is the year when people start talking about ethics and AI. Interest in ethical AI grows through 2017 and 2018.

May
2018

Microsoft appoints Tim O'Brien to the role of **AI Ethicist**, a new full-time position. The role involves AI ethics advocacy and evangelism. In reality, O'Brien clarifies that the role includes IoT, analytics, and AR/VR, since real-world solutions are hybrids of multiple technologies.

Dec
2018

Maria de Kleijn-Lloyd of Elsevier finds from her research that only 0.4% of published work on AI deals with ethics. She comments that there's lot of discourse on ethical AI but not much in terms of research and rigorous inquiry.

Apr
2019

Google disbands Advanced Technology External Advisory Council (ATEAC), which was supposed to advise on ethical AI. This happens due to controversy over ATEAC membership. This example shows that avoiding human bias and discrimination is not easy.

Jan
2020

Fjeld et al. note that many organizations working with AI have published their own guidelines. They study thirty-six such documents and identify eight main themes. Moreover, recent documents tend to cover all the eight themes.

Ethical AI


# Wi-Fi

## Summary

Wi-Fi is a technology for wireless local area networking with devices based on the IEEE 802.11 standards. User Equipment (laptop/mobile) uses a wireless adapter to translate data into a radio signal and transmit that signal using an antenna. At the receiving end, a wireless router converts radio waves back into data and then sends it to the Internet using a physical connection. Wi-Fi networks either operate in infrastructure mode or ad hoc mode.

Wi-Fi networks typically operates in unlicensed 2.4, 5 and 60 GHz radio bands. Data rates up to 20 Gbps are possible in the 60 GHz band. Range of a Wi-Fi network varies anywhere from a few metres (point-to-multipoint) to many kilometres (point-to-point with directional antennas).

## Discussion

### What are the roles of IEEE and Wi-Fi Alliance in Wi-Fi Technology?

IEEE 802.11 is the Working Group of Institute of Electrical and Electronics Engineers (IEEE) that deals with Local Area Networks (LANs), and its main role is to develop technical specifications for WLAN implementation.

The Wi-Fi Alliance was formed to ensure interoperability testing and certification for the rapidly emerging 802.11 world. This gives consumers the confidence a device from one vendor will work with another from another vendor, as long as they are Wi-Fi certified. It developed Wi-Fi Protected Access (WPA) in response to the poorer security of WEP. While, IEEE standards have technology-centric names, Wi-Fi Alliance has come up with more consumer-friendly naming. For example, IEEE 802.11ax is named Wi-Fi 6.

### What's the difference between WiFi and WLAN?

WLAN (Wireless Local Area Network) is a LAN to which a user (Station) can connect through a wireless connection. However, Wi-Fi is a type of WLAN that adheres to IEEE 802.11x specifications.

### What are the different existing 802.11x Standards?

| 802.11 protocol | Frequency (GHz) | Bandwidth (MHz) | Data Rate (Mbit/s) | Description |
| --- | --- | --- | --- | --- |
| 802.11a | 5 | 20 | 54 | Uses data link layer protocol and frame format as the original standard, but an OFDM-based air interface. |
| 802.11b | 2.4 | 22 | 11 | Uses same media access method defined in the original IEEE standard. |
| 802.11g | 2.4 | 20 | 54 | Uses OFDM-based transmission and operates at physical layer. |
| 802.11n | 2.4/5 | 20, 40 | 600 | Provides multiple-input multiple-output antennas. |
| 802.11ac | 5 | 20, 40, 80, 160 | 3467 | Release incrementally as Wave 1 and Wave 2. More spatial streams, higher-order modulation and the addition of multi-user MIMO. |
| 802.11ad | 60 | 2106 | 6757 | An amendment that defines a new physical layer to operate in the 60 GHz millimeter wave spectrum. |
| 802.11ax | 2.4/5 | 20, 40, 80, 80+80 | 9608 | Successor to 802.11ac meant to increase the efficiency of WLAN networks. |
| 802.11aj | 45/60 |  |  | A rebranding of 802.11ad for China. |
| 802.11ay | 60 | 8000 | 20000 | Extension of the existing 11ad, aimed at extending the throughput, range and use-cases. |

### Which are the types of Wi-Fi products available in market?

Wi-Fi products with a number of features are getting released on a regular basis. Here's a short list of Wi-Fi product types:

    + **Wi-Fi Access Point (AP)** - Used to connect other devices in Wi-Fi Infrastructure mode. All User Equipment will get access to Internet via Access Point.
    + **Wi-Fi Analyzer** - To Test and diagnose wireless performance issues such as throughput, connectivity, device conflict and single multipath.
    + **Wi-Fi Autodoc** - Autodoc is foremost software to generate a comprehensive report from firewall configuration files.
    + **Wi-Fi Adapters** - Adapters permit various devices to connect with cable-less media to perform various type of external or internal interconnects as PC cards, USB, PCI etc.
    + **Wi-Fi Bar Code Scanner** - WiFi bar code scanner continues their workflow in retail and intended to read stock keeping unit by providing efficiency and simplicity.

### Could you explain infrastructure and ad hoc modes of operation?

Infrastructure mode is suitable for any permanent network that's intended to cover a wide area. Ad hoc mode is suitable for a temporary network where the devices are close to each other.

In infrastructure mode, Wi-Fi devices on this network communicate through single access point, which is generally called wireless router. For example, two laptops placed next to each other might connect to the same AP. They don't communicate directly. Instead, they’re communicating indirectly through the wireless access point. Infrastructure mode requires a central access point that all devices connect to.

Ad-hoc mode is also known as **peer-to-peer mode**. Ad-hoc networks don’t require a centralized access point. Instead, devices on the wireless network connect directly to each other.

### Is Wi-Fi a viable technology for IoT applications?

For IoT, wireless technologies commonly proposed include RFID, LoRa, Sigfox, NB-IoT, LTE-M, IEEE 802.15.4, BLE and Bluetooth Mesh. Wi-Fi is not suitable for battery-operated devices due to its higher power consumption. Where a power outlet is available, Wi-Fi can be used in smart homes, home appliances, digital signages, and security cameras. Wi-Fi 6 might cater to connected cars and retail IoT.

The high data rates and low latency offered by Wi-Fi 5 and 6 make them suitable for vehicular services and applications heavy on media such as security cameras.

For low-power long-range applications, **IEEE 802.11ah**, aka Wi-Fi HaLow, is the most suitable standard. It operates in sub-1 GHz band with a range of 1 km. It supports short bursty data transmission and scheduled sleep/wakeup. It's ideal for smart building applications (lighting, HVAC) and smart city applications (parking meters or garages).

**IEEE 802.11p** is for vehicular applications. It aligns with FCC's Dedicated Short-Range Communications (DSRC). Applications seek to improve road safety and traffic management. It competes with LTE-V2V.

### Could you share a list of top WLAN solution providers?

In 2019, some well-known WLAN solution providers included Aerohive Networks, Mojo Networks, Aruba Networks, Cisco Meraki, Ruckus Wireless, Datto Networking, Ubiquiti Networks, Mist Systems, Purple, Edgecore Networks, Cloud4Wi, and Eleven. The best of them provide cloud management, including the use of ML/AI.

A report from IDC showed that in Q1-2019, about 47% of the enterprise market is with Cisco. This is followed by Aruba, Ubiquiti and Ruckus.

## Milestones

1990

NCR Corporation creates *WaveLAN* as a wireless alternative to Ethernet and Token Ring computer networking. In 1991, AT&T acquires NCR. The same year WaveLAN becomes the starting point for the standardization of Wi-Fi.

1996

Australian agency CSIRO's WLAN and its method of recovering data in multipath environments is granted a US patent. It's only in 1999 that the patent goes into the standard IEEE 802.11a. The technology is made available to implementers via non-exclusive licenses. In 2005, CSIRO files first worldwide family litigation. In 2012, it files suits against US carriers. Patent expires in 2013.

Jun
1997

IEEE publishes the first version of Wi-Fi standard, called **IEEE 802.11-1997**. It supports 2 Mbps in the 2.4 GHz band.

1999

Some companies come together to form a global non-profit association to promote and facilitate Wi-Fi adoption and interworking, regardless of brand. This association is initially called *Wireless Ethernet Compatibility Alliance*. In 2000, it's renamed to **Wi-Fi Alliance**. The Alliance also announces **Wi-Fi®** as the formal name for the wireless technology. The term Wi-Fi was in commercial use as early as August 1999. It was a name coined by Interbrand who also designed the Wi-Fi logo. The Alliance announces a certification programme and the first certified devices come out in 2000.

Sep
1999

IEEE publishes two amendments, **IEEE 802.11a** (only 5 GHz band, 54 Mbps max) and **IEEE 802.11b** (only 2.4 GHz band, 11 Mbps max). Although 802.11a offers 54 Mbps, 802.11b offers better range, uses the same modulation as the original standard and leads to dropping prices due to wider adoption. However, in terms data rates Wi-Fi remains far slower than its wired counterparts, Fast Ethernet (100 Mbps, 1995) and Gigabit Ethernet (1Gbps, 1998).

Jun
2003

IEEE publishes the **IEEE 802.11g** standard that provides 54 Mbps data rate although it uses the same 2.4 GHz band as 802.11b. Thus, 802.11g devices can work with 802.11b devices. It uses OFDM as the modulation just as 802.11a. Soon dual-band 802.11a/b products become tri-band 802.11a/b/g.

2004

TVs and smartphones get Wi-Fi certified and are launched in the market. **WPA2** is released to provide higher security. For the first time, Wi-Fi is offered to passengers on a commercial flight.

Jan
2008

NASA installs the first Wi-Fi device on the International Space Station. Two Netgear RangeMax 802.11b/g APs are installed, each giving 240 Mbps. In May 2016, the Wi-Fi network is extended to outside the space station. In 2019, Wi-Fi is integrated into a space suit, takes a space walk and streams HD video.

Oct
2009

IEEE publishes the **IEEE 802.11n** standard. Further amendments appear in later years: 802.11ac (December 2013) and 802.11ax (September 2019). 802.11ax can be seen as an evolution of 802.11ac.

Dec
2012

IEEE publishes the **IEEE 802.11ad** standard that allows operation in the 60 GHz band. It's derived from a WiGig specification completed by *Wireless Gigabit Alliance* in 2009. Since 2010, this alliance has been cooperating with Wi-Fi Alliance to promote WiGig. However, it's only in 2016 that Wi-Fi Alliance starts certifying WiGig products. The delay is mainly because vendors are reluctant to adopt a technology that has little infrastructure support.

Oct
2018

In an effort to simplify naming, Wi-Fi Alliance introduces **consumer-friendly generation names**. For example, 802.11ax is also known as Wi-Fi 6; 802.11ac as Wi-Fi 5; and 802.11n as Wi-Fi 4. In addition, UI visuals are defined to indicate which Wi-Fi standard is currently in use. Meanwhile in 2018, Wi-Fi certifications reach 45,000 and **WPA3** is released for higher security.

2019

The 30 billionth Wi-Fi device is shipped. Wi-Fi adoption is accelerating given that the 10 billionth device was shipped in 2014 and the 20 billionth device was shipped only in 2017.

Wi-Fi


# Web Exploitation

## Summary

Websites are significantly more complex today than in the early 1990s when they mostly served static HTML content. Web applications often serve dynamic content, use databases, and rely on third-party web services. The application server itself is being built from many components, which may come from diverse sources. Servers authenticate users before logging them into the system. They also authorize users to access restricted resources or data. Often applications handle sensitive user data that need to be protected.

Given this complexity, it's not easy to deploy and maintain web applications in a secure way. No application is perfect. Hackers are always on the lookout to discover and exploit vulnerabilities. This article discusses web exploitations and offers tips to improve the security of web applications.

## Discussion

### What aspects of an application are vulnerable to web exploitation?

A web application typically involves a web server, an application server, application middleware, internal or third-party web services, a database, and so on. Any of these components could be attacked.

An attack could be as simple as slowing down the server by making lots of HTTP requests. More serious attacks would involve installing a virus on the server or stealing sensitive data. Defacing the site by modifying site content, or deleting code or data are just as serious but more easily visible. Another attack is to run cryptocurrency miners on server infrastructure.

Web clients/browsers and servers communicate via protocols such as HTTP, HTTPS, FTP, etc. Vulnerabilities in how these protocols are used could be exploited. Protocols are located at different layers of the protocol stack. Although web exploits happen at the application layer (layer 7), it can impact other layers via packet flooding (data link layer) or SYN flooding (network layer). However, web exploits at the application layer are becoming more common than network layer attacks on web servers.

### Which are main types of web vulnerabilities and their related exploits?

Web exploits involve one or more of the following:

    + **Injection**: This results from accepting untrusted input without proper validation. Examples include SQL injection, LDAP injection and HTTP header injection.
    + **Misconfiguration**: This happens when processes are manual and settings are not correctly maintained.
    + **Cross-Site Scripting**: Via user input, server accepts untrusted JavaScript code. When server returns this in response, browser will execute it.
    + **Outdated Software**: With the increasing use of open source and third-party software packages, it's important to keep these updated. Outdated software can be exploited, especially when the vulnerabilities are public.
    + **Authentication & Authorization**: URL may expose session ID. Password may be unencrypted. If timeouts are not correctly implemented, session hijacking is possible. Unauthorized resources can be accessed even when UI doesn't expose them.
    + **Direct Object References**: By poor design or coding error, direct references are exposed to clients. For example, a GET request to `download.php?file=secret.txt` may bypass authorization and allow direct download of a protected file. Another example is to directly reset admin password.
    + **Data Exposure**: Sensitive data is stored in an unencrypted form, or exposed in cookies or URLs. Client-server communicate on a non-HTTPS connection.

### What are some exploits pertaining to HTTP headers?

One technique is to send an arbitrary domain name or port number in the **Host header** and see how the server responds. Duplicating the Host header or formatting it differently can reveal vulnerabilities. These can be exploited to reset passwords, bypass authentication or poison web cache.

Suppose your architecture uses a load balancer or reverse proxy, followed by a backend server. Each component might process the HTTP header in subtly different ways. This can lead to **HTTP request smuggling** attacks in which an extra malicious request is smuggled via the main request.

Newlines are used to separate HTTP header from body. With **CRLF injection**, characters `\r\n` are forced into header fields via their URL encoded form `%0D%0A`. This can be used to modify HTTP access logs to cover up earlier attacks, enable CORS or deliver XSS payload.

To improve web security, HTTP has headers related to Cross-Origin Resource Sharing (CORS). Some of these are Access-Control-Allow-Origin, Access-Control-Expose-Headers, and Access-Control-Max-Age. More security-specific headers include Cross-Origin-Resource-Policy, Content-Security-Policy, Strict-Transport-Security, X-Content-Type-Options, X-Frame-Options, and X-XSS-Protection.

### What are some server-side web exploits?

HTTP DoS attack works by making large number of HTTP requests to the web server. It may also involve crawling the site to learn which pages require more server processing. The attack then involves requesting resource-intensive pages. If requests involve database calls, the site slows down visibly affecting user experience. Another way to slow down the server is to set a high Content-Length header value, send fewer bytes and make the server wait for the rest.

Remote code execution is possible if server is using outdated software. Once server is infiltrated, arbitrary code can be executed to steal data or install malware. Insecure deserialization is a vulnerability that can lead to remote code execution.

Improper authentication and authorization can allow attackers access protected server resources or sensitive data. Passwords could be reset via URL query strings.

Database could be corrupted via SQL injection. This can happen when input data is not validated.

In any case, it's important that server software is kept up to date. Review and validate configuration. Collect sufficient logs and constantly monitor so that any breach is detected as soon as possible.

### What is Cross-Site Scripting?

Cross-Site Scripting (XSS) is when a user visits site A and her browser executes malicious script that then contacts site B. Malicious code is contained within `<script>` tag. The user is initially tricked into clicking a link, perhaps via email. The genuine site loads in browser but the attacker has already passed malicious code via the URL. Another variant of the attack is to input code in forms. In either case, the malicious script can access user cookies on the browser. These are sent to the attacker's site to later impersonate the user on genuine sites she uses.

Although JavaScript code for this attack executes on the browser, XSS attacks have been classified into **Server XSS** or **Client XSS** attacks. In either case, the vulnerability starts with untrusted user input. With Server XSS, the server sends the malicious script in the response. With Client XSS, the DOM is updated to make JavaScript calls.

Context-sensitive output encoding and input validation can protect against Server XSS. Against Client XSS, call only trusted JavaScript APIs but this is easier said than done.

### What are the security problems with Flash?

Created in 1996 by Macromedia and acquired by Adobe in 2005, Flash is used to render rich multimedia content on browsers. Increasing use of Flash on the web started to grow in the early 2000s.

The earliest reported security vulnerability with Flash dates from 2005. As of October 2020, Flash has gathered more than 1000 vulnerabilities. Number of vulnerabilities are 8 (2019), 26 (2018) and 65 (2017). Types of vulnerabilities include code execution, overflow, memory corruption, DoS, bypass of access restrictions, access to sensitive information, and more. XSS attacks are possible via what's called **Flash Parameter Injection (FPI)**.

As an example, CVE-2017-11292 allowed arbitrary code execution. A flawed bytecode verification procedure meant that an array index could be derived from an untrusted value. This led to type confusion and malicious code execution.

**Flash cookies** are also problematic. They have been misused to track users across sites they visit. Users can't easily disable them, unlike HTTP cookies.

Flash is not open source. HTML5 is the standardized way to deliver multimedia content. Due to the many security issues, browser vendors and Abode announced in 2017 that Flash will be discontinued from January 2021.

### What are some common misconfigured security settings?

An application may have a debug mode that results in lots of internal information printed out. By mistake, this mode is used in production. Even without the debug mode switched on, it's possible that the application shows stack traces when an error occurs. Another mistake is to allow the server to list directory contents. This exposes internal directory structure and contents to attackers.

Software components come with default values. For example, MySQL server uses a default username and password, unless the admin changes them to non-default values. If application is deployed with default values, it's easy for an attacker to exploit this.

It's a flaw to store passwords in code or read them from files committed to the code repository. A better way is to set passwords as environment variables that the application can access.

A web server offers its service on well-known default ports. At the minimum, it may offer HTTP on port 80 and HTTPS on port 443. Running other services on the server, many of which may be unnecessary or unused, opens up a larger attack surface.

### Could you share real-world instances of web exploits?

Data breaches have affected many big names including Adobe, eBay, Equifax, LinkedIn, Marriott International, Sina Weibo and Yahoo. The biggest breach is perhaps Yahoo's in 2013-14 involving 3 billion user accounts.

eCommerce sites have been hacked. British Airways was attacked in 2018 leading to stolen customer details. The problem was down to a third-party JavaScript that had not been updated since 2012. In 2019, fashion retailer Sweaty Betty become victim to malicious JavaScript injection.

The Panama Papers leak of 2016 happened due to outdated software. In fact, the site had many vulnerabilities including authentication and SQL injection. An example of XSS exploitation is Steam profile hack of 2017. JavaScript was injected into user profile page. This was used to drain Steam Wallet funds or spread other malware. Vulnerable since 2008 but discovered only in 2017, the REST plugin of Apache Struts 2 did deserialization in an insecure manner leading to arbitrary code execution.

Cloud infrastructure is not immune to attacks. Australian Broadcasting Corporation and Accenture are examples of organizations that failed to adequately protected their Amazon S3 storage instance. They were attacked in 2017.

### Could you share some best practices to make web applications more secure?

Make your application code more secure but this is often hard. Instead, adopt a framework with built-in security features and functions.

Penetration testing must be part of your process. When vulnerabilities are discovered, fix them at the earliest. Automate processes. Use a Web Application Firewall (WAF) as a preventive measure. Design applications in a modular way so that it's easier to apply attributes on resources.

When using third-party components, ensure you're using the correct one and not a typosquatted package. Keep them updated to take advantage of latest security fixes.

Always sanitize user inputs. Avoid blacklists since they're brittle. Whitelists are better. If returning HTML tags from user input to browser, encode the tags. For example, return `<script>` as `<script>`.

Don't include user inputs in HTTP headers. In HTTP headers, encode CRLF special characters. Adopt security headers such as Content Security Policy and X-XSS-Protection to protect against XSS and code injection. X-Content-Type-Options counters MIME sniffing. X-Frame-Options counters clickjacking. Strict-Transport-Security enforces communication over HTTPS.

For authentication, issue new cookies when user logs in or changes role. Limit session duration. Set cookies with attributes HttpOnly, Secure and SameSite=Strict.

### What online resources are available to know about current web vulnerabilities?

Software vulnerabilities are categorized and named individually to help developers and security researchers. These are formalized as **Common Vulnerabilities and Exposures (CVE)**, a system initiated by the MITRE Corporation.

Among the useful databases are National Vulnerability Database (NVD) from NIST, Vulnerability Assessment Platform (Vulners), Vulnerability Database (VulDB) and CVE Details. Vulners offers a useful search functionality and it's been called "Google for hackers". VulDB documents vulnerabilities in electronic products. MITRE's own database is called Common Weakness Enumeration. MITRE and NIST work closely.

Exploits of these vulnerabilities are also documented and organized as **exploit databases**. Among the well-known ones are ExploitDB, Rapid7, CXSecurity, Vulnerability Lab, Oday, SecurityFocus, Packet Storm Security, and Google Hacking Database.

**Tools** are also available for security testing of web applications. Some of these include Zed Attack Proxy (ZAP) from OWASP, Wfuzz, Wapiti, W3af, SQLMap, SonarQube, Iron Wasp, Metasploit, Nessus, Burp Suite, Nikto, and OpenVAS. More tools to scan a site for vulnerabilities include SUCURI, Qualys, Quttera, Intruder, UpGuard, Web Cookies Scanner, Detectify, Probely, and Pentest-Tools.

## Milestones

1990

In the early 1990s, the World Wide Web (WWW) starts to grow. Most websites serve only static pages, thus presenting few security vulnerabilities. From mid-1990s, more websites start to adopt JavaScript, thus presenting opportunities for the first web exploits.

Sep
1999

As a public list of known vulnerabilities, the **CVE List** is launched. The original list contains 321 entries. By December 2000, 29 organizations claimed to offer "CVE-compatible" products and services. Later, CVE List becomes the basis of U.S. Government's National Vulnerability Database (NVD). In 2011, ITU-T adopts CVE List. As of October 2020, the list includes more than 140,000 entries.

2000

**Cross-Site Scripting (XSS)** is first discovered at Microsoft. It's believed that XSS is due to server-side code. By 2005, it's clear that XSS is possible at the client side as well. Although client-side XSS was possible even in 1997, it starts to increase from 2003 with the coming of Web 2.0. This exploit continues to grow for many years and starts to fall only from 2013.

2005

From mid-2000s, the use of JavaScript increases in terms of key metrics: more JavaScript statements per external script that's included, inclusion of scripts from more domains, and higher code complexity. These are indications of richer user interactions and push of application logic towards the client.

2017

At the 26th USENIX Security Symposium Stock et al. give the example of Firefox browser. It supports 83 different APIs whereas it supported only 12 APIs back in 2006. This growth has led to many vulnerabilities on the client side.

2019

Edgescan SaaS application reports that web applications pose a greater risk than applications exposed at the network layer. About 35% of vulnerabilities in web apps are of high or critical risk. On average, it takes 85 days to fix a web app vulnerability. A 20-year vulnerability about SNMP community name is discovered. The most common vulnerability is a birthday attack on long-duration encrypted HTTPS sessions.

2019

A study from Positive Technologies shows that 9 out of 10 websites can be hacked, 39% allow unauthorized access, and 68% can suffer from data breaches. Application code account for 82% of the vulnerabilities. Security misconfiguration is the most common one. XSS attacks exploiting cookies that didn't use HttpOnly and Secure flags affect one out of five tested applications.

Web Exploitation


# TypeScript

## Summary

JavaScript is now truly cross-platform. It works within browsers. It's being used to build standalone desktop applications. Enabled by Node.js, it's also being used for server-side execution. It can work on any OS. The problem with JavaScript is it's lack of scalability. It's not ideal for large projects maintained by large teams.

TypeScript makes JavaScript scalable. It does this by supporting static typing so that type mismatches can be caught at compile time rather than at run time. When coupled with tools that understand TypeScript, developers benefit from autocompletion, navigation and refactoring. TypeScript came out of Microsoft and later open sourced.

TypeScript doesn't replace JavaScript. TypeScript transpiles to JavaScript. In fact, a project can contain a mix of both languages. Eventual execution is within a JavaScript runtime.

## Discussion

### Why do we need TypeScript when there's already JavaScript standardized by ECMA?

JavaScript is a dynamically typed language. This can be seen as a useful feature but for projects maintained by large teams, it can be a source of bugs. For example, a function expects an integer but a string is passed wrongly. JavaScript will allow this until an error occurs at runtime. TypeScript allows for **static types**. This means that type mismatch errors are caught during development.

JavaScript before ES6 didn't allow for classes and modules. This made JavaScript coding difficult for programmers coming from C#, Java or other object-oriented languages. While ES6 remedies this, not all browsers support ES6. Babel offered a solution to this problem. Developers can use the latest ECMAScript version but use Babel to transpile their code into a lower version that's well supported by browsers. TypeScript compiler can also **transpile** TypeScript code to any chosen ECMAScript version. This means developers can use latest developments to TypeScript today without worrying about what browsers support.

### Isn't TypeScript doing the same thing as AtScript, Flow, Dart or CoffeeScript?

AtScript was invented at Google with the same goals as TypeScript. In 2015, given the popularity of TypeScript, Google discontinued AtScript and adopted TypeScript for its Angular 2 framework. Later versions of TypeScript started supporting features introduced by AtScript.

Flow was invented at Facebook to provide static typing for JavaScript. Thus, it's an alternative to TypeScript. However, we've seen projects move away from Flow to TypeScript, including Facebook's own Jest project.

Comparing Dart with TypeScript is not useful. Dart compiles to native code but transpiles to JavaScript for web apps. Dart is not JavaScript although it can interoperate with JavaScript. TypeScript on the other hand *is JavaScript*. Existing JavaScript code can be treated as TypeScript.

Although CoffeeScript transpiles to JavaScript, it's syntax is different from JavaScript. CoffeeScript hides JavaScript syntax from the programmer but TypeScript doesn't do this. In fact, CoffeeScript is great for Ruby programmers since the syntax is Ruby-like. In this context, Luke Hoban commented,

> CoffeeScript is to Ruby as TypeScript is to Java/C#/C++

### Which projects have adopted TypeScript?

The official TypeScript site lists a number of projects that have adopted the language. Among the JS frameworks and libraries to use TypeScript are Angular, Ember, NativeScript, Ionic, Babylon.js, AMCharts and ZoomCharts. Others include Asana, Aurelia, BitHound, Ericsson, EBay Classifieds, Hobsons, JetBrains, Kaggle, Palantir, Ubisoft, and many more.

In September 2018, Expo SDK started working towards supporting TypeScript, in addition to Flow. They noted that TypeScript is stable and expressive. Just as importantly, Babel 7 supports TypeScript and therefore Expo can continue using Babel-based builds.

Vue.js announced that its own codebase for version 3.x and beyond will be based on TypeScript. It's API will be designed to make use of TypeScript's type interference although TypeScript will remain optional for apps.

Atlassian's project that enables beautiful drag-and-drop interfaces moved to TypeScript in December 2018. Storybook, used for building UI components, is now in TypeScript. In January 2019, Facebook's Jest project decided to move away from Flow to TypeScript. PayPal's `paypal-scripts` is another example.

### Could you mention some key features of TypeScript?

Static type checking is an important part of TypeScript but this optional. This means that we can add type checking gradually to a project, especially legacy JavaScript project. The language has new useful types: tuple, enum, any, void, never. Variable declarations can specify `let` or `const`. For object members, we can use `readonly`. Type assertions are similar to type-casting in other languages. A variable can take one of many specified types. Generics are allowed.

Classes with properties, methods, and constructors are now possible with TypeScript. Constructors, and functions in general, can be overloaded. Via interfaces, abstractions are made explicit and this makes code easier to read and maintain.

JavaScript code generated by the TypeScript compiler is very readable, which means that debugging is possible on the generated code without using source maps.

Other ES6 features such as module imports and destructuring are supported. TypeScript supports decorators (also available in ES7).

### Could you share some tips or best practices for beginners to TypeScript?

The best way to initialize a new project is using `tsc --init`. File `tsconfig.json` will be created. It's possible to extend this configuration file with another file so that we can target different ECMAScript versions if necessary.

It often a good practice to store generated JS files in a separate folder. It's common to have TS files in `src/` and JS files in `dist/`. Compiler option `--outfile` helps with this; or you could use `outDir` option in the config file.

You can define a type using either `type` or `interface`. The latter is more flexible because it can be extended to create another interface or implemented in a class. Interfaces can be defined multiple times, adding new members with each definition.

When indexing a data type with a key, such as in a tuple, or accessing a member of an interface, use `keyof` to create a type for the keys. This enables type checking on the keys.

Many more tips are part of official documentation, in a Zalando blog post, and in a Medium post.

### What are the different file extensions for TypeScript?

Typescript source code files are usually saved with `*.ts` extension. When transpiled, the equivalent `*.js` JavaScript files are created.

Files with `*.d.ts` extension contain **type definitions**. Your TypeScript code may be using third-party JavaScript libraries. How can we then ensure type safety when your TypeScript code is calling these JavaScript libraries? This is where these type definitions become useful for IDEs and TypeScript compiler.

JSX is an XML-like syntax for designing UI components. It was popularized by React. TypeScript supports this and files that use this syntax have extension `*.tsx`. When these files are transpiled, they generate either JSX files (to be consumed by other transpilers such as Babel) or JS files that are understood by React or React Native.

## Milestones

2010

Microsoft starts working on creating TypeScript. It's not a new language. Rather, it's an augmentation of JavaScript. One motivating factor is that the Bing Maps team has difficulty making JavaScript scale. There are probably other teams within Microsoft facing similar issues with JavaScript.

Oct
2012

Microsoft releases TypeScript specification and open sources its own TypeScript compiler. Visual Studio gets a plugin for TypeScript. The creator of C#, Anders Hejlsberg, notes that JavaScript was designed as a scripting language, for small to medium-sized projects. TypeScript's static typing will address the scalability problem. He adds,

> JavaScript is an entirely dynamic language that has no static typing, and static typing is really the thing that powers today's rich IDEs.

Jul
2014

Microsoft completes its move of TypeScript codebase from CodePlex to GitHub. The idea is to go where the community is active, and attract Linux and Mac developers.

2015

This year sees many releases of TypeScript: 1.4 (Jan) to 1.7 (Nov). Among the features/types introduced are `let`, `const`, template strings, type guards (`typeof`, `instanceof`), ES6 modules, ES6 destructuring, decorators, `namespace` keyword, `tsconfig.json` config file, JSX, abstract classes, async/await, and `this` typing.

Mar
2015

It's announced that **Angular 2** will be built using TypeScript. Over the next few years, this influences more developers to adopt TypeScript.

Jul
2016

**TypeScript 2.0** is released. Type definitions are now available via NPM at `@types`. This deprecates earlier approaches: DefinitelyTyped, TSD, Typings. As on April 2019, more than 6,300 packages have type definitions.

Jul
2018

**TypeScript 3.0** is released.

Dec
2018

**Babel 7** is released with support for TypeScript. This means that TypeScript can be transpiled to JavaScript using Babel. Thus, TypeScript-based projects can continue to benefit from Babel's rich ecosystem and plugins. TypeScript compiler can still be used for type checking.

TypeScript


# Probability for Data Scientists

## Summary

In mathematics, the notation \(\pi\), pronounced as \(pi\), denotes ratio of circumference of any circle to diameter of same circle. \(\pi\) is constant. It will not vary for circles of any size. But many other facts in the world are not constant.

Let's assume alphabet \(X\) denotes height of adults in India. \(X\) can take any positive real value for any random individual. Hence \(X\) is a variable that takes random values in a range of positive real numbers.

Probability measures how likely or unlikely is an outcome, where outcome is a **Random Variable**. For instance, we can ask, "What's the probability of picking an Indian adult male who is above six feet?"

Probability is intimately related to another branch of mathematics called *Statistics*. Both these are of fundamental importance to the field of Data Science.

## Discussion

### How do we mathematically define the probability of an event?

Mathematically, probability is the ratio of the number of desired outcomes and all possible outcomes:

$$P(Outcome)=\frac{n(Desired\ Outcome)}{n(All\ Outcome)}$$

Any desired outcome is a subset of all possible outcomes. The value of probability therefore ranges from zero to one. The limits have the following interpretation:

    + Zero: the outcome will never occur.
    + One: the outcome is guaranteed to occur.The set of all outcomes is called the **Sample Space**. When the number of outcomes is large or grouping outcomes is more suitable for a study, it's common to group one or more outcomes into what we call an **Event**. An outcome can be part of multiple events.

### Could you illustrate probability with some simple examples?

If we toss a coin, there are only two possible outcomes: head or tail. Assuming both outcomes are equally likely, the probability of getting a head is 1/2 = 0.5. Likewise, the probability of getting a tail is also 0.5. Taken together, the probability of getting either head or tail is 0.5 + 0.5 = 1. This makes sense since there are no other outcomes besides head or tail.

Let's roll a dice. The probability of getting an odd prime number (3 or 5) is 2/6 = 0.33. The probability of getting a number greater than 6 is 0/6 = 0. The probability of getting a number less than or equal to 6 is 6/6 = 1.

### Why probability works?

Any random variable, however random, will have its own identifiable characteristics. For example, the variable may be highly probable at one value with dropping probabilities at neighbouring values. This variability around the most probable value helps us to model random variables. In technical terms, we call this the *distribution of the random variable*. When we plot the number of occurrences against the value, we get a *distribution curve*. Random variables are typically modelled with average value, variability (spread), skewness (asymmetry) and kurtosis ("tailedness").

For example, let's consider the response time of a computing system. When the system is under high load, the average response time increases. What's more interesting is that the spread of response time around this average is also more. Thus, under different loading the response time random variable exhibits different characteristics.

Exceptions (popularly called **Outliers**) will affect probability generalisation. They have to be kept out when building realistic models.

### How is probability related to distributions?

Probability looks at the likelihood of a specific outcome or event. Distribution looks at all outcomes or events.

Let's take the example of a coin toss. We know from theory that the probability of a head is 0.5. However, there's also an experimental approach. For example, experimenting with 100 tosses might result in 49 turning out to be heads. Hence, probability of head is 0.49. Such an experiment is termed **Bernoulli Trials**. When we list probabilities for all outcomes (head and tail), we end up with a **Bernoulli Distribution**.

We can perform a variation of the coin-toss experiment. We can toss a coin 32 times and call this a single experiment. We repeat this experiment many times, say 50,000 times. Finally, we calculate the probability of getting 4 heads in each experiment of 32 coin tosses. Such a series of experiments is called **Binomial Trials**. When we list the probabilities for all outcomes, we end up in **Binomial Distribution**.

### What are probability distributions and how are they useful?

Probability, when identified and listed for all possible outcomes, is called **Probability Distribution**. For instance, if we find probabilities of adult males in India with heights in ranges of 0-3.5, 3.5-4, 4-4.5, 4.5-5, 5-5.5, 5.5-6 and 6+ feet, we have a probability distribution. Such a distribution is closely related to the concept of histogram. With histogram, we plot the count of values within each range. With distribution, we convert these counts into probabilities. In both case, a graphical plot helps us to read easily the average, variability, skewness, kurtosis, outliers, etc.

Given the distribution, an event can be simulated at random within the boundaries of the distribution. In other words, to create random variables for simulation purposes, we need the distribution. For example, let's consider the number of people arriving at ATM every 60 minutes. This can be modeled as Poisson distribution. We can simulate queues at ATM, calculate waiting times and decide if need another ATM needs to be installed. Likewise, if we know distributions of outcomes in a game, we can simulate winning odds and take appropriate risks.

### When probability works?

Often we are unable to gather data from the entire sample space or population. We typically collect a sample of data from the population. Probability works when sample size is large.

For instance, we wish to find the probability of an Indian adult male of height six feet and above. A sample size of 100 will not give a reliable number. However, a sample size 10,000 will be more reliable. The more, the better. Stated formally as the **Law of Large Numbers**, the probability of an event from a sample will converge to the actual value of the population when the sample size is large.

### What are axioms of Probability?

There are obvious rules in probability. These rules are called **Axioms of Probability**. These were formulated by Russian mathematician Andrei Kolmogorov.

These axioms can be explained as follows:

    + The probability of any event is a non-negative real number.
    + The probability of the entire sample space is one. This follows from the fact that there are no events outside the sample space.
    + The probability of the union of two mutually exclusive events is the sum of their individual probabilities.

### What are mutually exclusive and non-mutually exclusive events?

Let us say, there are two events denoted with random variables \(A\) and \(B\). If \(A\) and \(B\) don't occur together, they're mutually exclusive. They're also call **disjoint events**. For instance, cooking is event \(A\) and cycling is event \(B\). These two are mutually exclusive: a person doesn't cook and cycle at the same time.

$$P(A\ and\ B) = 0\ or\ neglibible \\ \Rightarrow P(A\ or\ B)=P(A)+P(B)-P(A\ and\ B) \\ \Rightarrow P(A\ or\ B)=P(A)+P(B), since\ P(A\ and\ B)=0$$

On the contrary, if \(A\) and \(B\) do happen together, they are non-mutually exclusive events. For instance, cooking is event \(A\) and listening to music is event \(B\). They can happen at the same time.

$$P(A\ and\ B)\neq0 \\ \Rightarrow P(A\ and\ B) = P(A)+P(B)-P(A\ or\ B) \\ \Rightarrow P(A\ or\ B)=P(A)+P(B)-P(A\ and\ B)$$

### Could you explain joint, conditional and marginal probabilities?

Let's consider two events: buying Bread \(A\), buying Jam \(B\). Marginal probability is the proportion of customers who bought Bread regardless of whether they bought Jam or not. It's called marginal because it occurs at the margins of the probability table (see figure). Joint probability is proportion of customers who bought both Bread and Jam. Conditional probability is proportion of customers who're likely to buy Bread when they've already bought Jam, and vice-versa.

*Marginal Probability*

$$P(A) = \frac{n(Cust\ with\ Bread)}{n(Cust)} = \frac{90}{1000} \\ P(B) = \frac{n(Cust\ with\ Jam)}{n(Cust)} = \frac{50}{1000}$$

*Joint Probability*

$$P(Cust\ with\ Bread+Jam) \\ = P(A\ and\ B) = \frac{n(A\ and\ B)}{n(Cust)} = \frac{40}{1000}$$

*Conditional Probability*

$$P(Cust\ with\ Bread\ given\ Jam) \\ = P(A|B)=\frac{n(A\ and\ B)}{n(B)} = \frac{40}{50} \\ P(Cust\ with\ Jam\ given\ Bread) \\ = P(B|A)=\frac{n(A\ and\ B)}{n(A)}= \frac{40}{90}$$

Conditional Probability reduces the sample space based on condition. Rather than considering all customers (1000), we only consider customers who bought Bread (90) or customers who bought Jam (50), that is, the marginal numbers.

### How is odds different from probability?

Odds is defined as ratio of chances of an event happening and chances of the same event not happening. Consider the ratio of customers buying milk to those not buying milk. If this ratio is more than 1 then the odds are in favour of hypotheses (buying milk), else odds are against hypotheses.

Consider customers buying bread and jam:

$$Odds(Bread+Jam) \\ = \frac{P(Bread+Jam)}{P(Bread-Jam)} \\ =\frac{40}{50}=0.8 \\ Odds(Jam+Bread) \\ = \frac{P(Jam\ with\ Bread)}{P(Jam-Bread)} \\ =\frac{40}{10}=4$$

For every 0.8 customers who buy Bread and Jam, one customer will buy only Bread. For every 4 customers who buy Jam and Bread, one customer will buy only Jam. Thus, the Odds(Buying Jam with Bread) > Odds(Buying Bread with Jam). This implies,

    + Jam drives Bread purchase.
    + Sizable Bread buyers prefer Bread without Jam.

### What is Bayes' Theorem?

Given hypothesis H and evidence E, Bayes' Theorem can be written as \(P(H|E) = P(E|H) \dot P(H) / P(E)\). Bayes' Theorem, also called Bayes' Rule or Bayes' Law, uses **prior** probability \(P(H)\), accounts for new **evidence** \(P(E|H)\) and results in **posterior** probability \(P(H|E)\).

Often, prior probability is sourced from experts due to challenges in evaluating from evidence. Bayesian probability basically revises probability considering every new evidence. This probability will converge to its true value over many revisions on repeated evidence.

Bayesian approach is used in fields such as epistemology, statistics, and inductive logic. It relies on conditional probabilities and empirical learning. The key insight of the theorem is "that a hypothesis is confirmed by any body of data that its truth renders probable".

### Could you give some applications of Bayes Probability?

One application is in spam filtering. The idea is to classify an email as spam. The email may or may not contain the word "Viagra" and not all mails with this word may be spam. We calculate the probabilities based on our prior knowledge of number of spam mails received. \(P(spam)\) is prior knowledge of spam mails in inbox. But the probability of the word appearing in a previous spam mail can give us a better estimate. P(V|spam) is the **likelihood** and P(V) is the **marginal likelihood**.

$$P(spam|V) = P(spam) \* \frac{P(V|spam)}{P(V)} \\ where\ P(V)=P(V|spam)+P(V|not\ spam)$$

\(P(V|spam)/P(V)\) is evidence from data that probability of word Viagra in spam mail. \(P(spam|V)\) is the posterior probability of mail being spam with word Viagra in it. When 100% of mails with Viagra are spam, then \(P(spam|V)=P(spam)\). When less than 100% of mails with Viagra are spam, then \(P(spam|V) < P(spam)\).

### What are Frequentist and Bayesian approaches to Probability?

*Frequentists* lean on the **Law of Large numbers** to back their probability estimate. For instance, a coin toss has equal probability of head or tail. This is derived from a large number of trials. Frequentists believe any deviation from equal probability is due to chance.

*Bayesians* argue that **belief or prior knowledge** should be accounted for while calculating probability. Belief suggests a probability. New evidence may notch up or notch down the probability and form a new belief. Bayesians do not require Law of Large Numbers backing, but leverage them where applicable. Probability may be revised with a new piece of evidence, eventually converging to true probability after repeated revisions. For instance, the probability of new robot failing at a task starts with a belief, say \(p\), and as new evidence arrives, we revise \(p\).

Frequentist and Bayesian approaches can be applied for all estimates including probability. While the two approaches are distinct, Bayesian probability complements Frequentist probability when,

    + System is not yet stable.
    + There's insufficient data to get backing from Law of Large Numbers.

## Milestones

1564

Sixteenth century Italian mathematician Girolamo Cardano is interested in gambling, to which he applies mathematics. Although gambling has been around for centuries, *randomness* is not a recognized concept. People continue to believe in Gods and oracles, until the Renaissance. Cardano's work is the first of its kind, although his work is published only much later in 1663. Today we know that he made some fundamental mistakes.

1654

In a series of letters analyzing the *problem of points*, French mathematicians Blaise Pascal and Pierre de Fermat develop what can be seen as the foundations of a mathematical theory of probability. Their work is popularized by Christian Huygens in a publication of 1657.

1713

Jakob (James) Bernoulli publishes *Ars Conjectandi*, in which he introduces many important concepts: permutations, a priori, a posteriori, Bernoulli trials, random variable. Bernoulli showed that the probability of an event can be approximated by the frequency of occurrence of the event from a large number trials. This later came to be called the **Law of Large Numbers**.

1763

Thomas Bayes' now famous work on probability is posthumously published. The Bayesian approach to probability is adopted, and popularized by Pierre-Simon Laplace, until it's challenged in the early 20th century by mathematicians R. A. Fisher and Jerzy Neyman. To Bayes, probability is a measure of personal belief or reasonable expectation of the event.

1812

With his publication of *Analytical Theory of Probability*, Pierre-Simon Laplace brings together recent developments in the field. This important work shows the application of probability to scientific problems. Thus, probability is no longer just about games of chance. Laplace himself states,

> It is remarkable that probability, which began with the consideration of games of chance, should have become the most important object of human knowledge... [It] is at bottom nothing but common sense reduced to calculus... It teaches us to avoid the illusions which often mislead us.

1933

Russian mathematician A. N. Kolmogorov provides an axiomatic basis for the mathematical theory of probability, thus laying the foundations for a modern treatment of the subject.

Probability for Data Scientists


# Deadlock

## Summary

Deadlock is a problem that can occur when resources are shared among multiple processes. Suppose process P1 is waiting for a resource R1 currently being used by process P2. Meanwhile, P2 is waiting for resource R2 that's being used by P1. Neither process is able to proceed. This is an example of deadlock.

Shared resources could be files, database tables, memory, peripherals (printers, tape drives), network, etc.  While we commonly talk of deadlocks among processes, it could be among threads, users, transactions, computers in a distributed system, etc.

While it's possible to design systems completely free of deadlocks, it's inefficient not to share resources or avoid multiprocessing. Instead, there are techniques to avoid, detect and recover from deadlocks.

## Discussion

### Could you explain deadlocks with an example?

In real life, traffic deadlocks can occur at intersections. In computer science, the classic example used to illustrate deadlock is the **Dining Philosophers Problem**.

Five philosophers are seated at a round dining table. There are five forks, each placed between two philosophers. Philosophers alternate between eating and thinking. To eat, a philosopher has to acquire both forks on either side of her plate but acquire them one at a time. Once a philosopher successfully gets both forks, she eats for a while, then puts down both forks and thinks for a while. Now her neighbouring philosophers can acquire one of the released forks and start eating if they already have another fork.

Deadlock can occur when each philosopher acquires one fork and waits indefinitely for another fork. Thus, no philosopher can start to eat. They'll wait forever, neither eating nor thinking.

In this analogy, philosophers are the processes. Forks are the shared resources. If resource sharing is poorly coordinated, deadlocks can occur. The system stalls. No useful work gets done.

### What are some areas in computer science where deadlocks can occur?

Deadlocks can occur in database applications. For example, two transactions have locks on two different tables but also require access to the other table. This happens because operations within one transaction executes between operations of the other transaction, something that's common in concurrent systems. The problem is worse when databases are distributed.

Deadlocks can occur in multithreaded applications or multiprocessing systems with shared resources. For example, one process is reading a file and then attempts to print the file. However, another process is using the printer and suddenly wants access to the file held by the first process. Both processes are now deadlocked.

Exclusive access to a resource is typically implemented as a lock or mutex. The term *mutex* is common when accessing critical sections of code. The term *lock* is common in databases.

### Which are the necessary conditions for a deadlock to arise?

There are four necessary conditions for a deadlock:

    + **Mutual Exclusion**: A resource can't be assigned to more than one process at a time.
    + **Hold-and-Wait**: A process that's holding a resource is also waiting for another resource.
    + **No Pre-emption**: Process that's holding a resource can't be forced to or doesn't voluntarily give up that resource.
    + **Circular Wait**: Each process is waiting for a resource held by another process. This condition implies the hold-and-wait condition.

### How do I analyze or visualize a deadlock?

**Resource allocation graphs** help us understand or analyze deadlocks. Resources and processes are nodes of the graphs. Directed edges connect the nodes. A directed edge from a resource to a process implies that the resource is assigned to that process. A directed edge from a process to a resource implies that the process is requesting that resource.

Where there are multiple instances of a resource, a request edge terminates at the edge of the resource box. On the other hand, an assignment edge always terminates at an instance. Each instance is represented as a dot within the resource box.

In the figure we note two examples. Both have circular waits but only one of them is a deadlock. In sub-figure (b), P1 and P3 are waiting for R1 and R2 respectively. However, instances of R1 and R2 will become available once released by P2 and P4 respectively. There's no deadlock since P2 and P4 are not part of the circular wait and neither do they fulfil the hold-and-wait condition.

### Which are some techniques to handle a deadlock?

The four common ways to handle deadlocks are:

    + **Prevention**: System is designed with constraints to prevent at least one of the conditions required for a deadlock.
    + **Avoidance**: Algorithms decide at runtime to deny a resource request if subsequent requests might lead to a deadlock. Decisions may be conservative resulting in lower resource utilization.
    + **Detection**: Periodically check for deadlocks. Frequent checks imply a performance overhead. Infrequent checks imply deadlocks are not caught soon enough. Use triggers such as a drop in CPU utilization to start a check. Perhaps check when a resource request can't be granted.
    + **Recovery**: Take action to break one of the conditions for a deadlock. One of the processes could be terminated, called *victim selection*. Or the process is forced to release resources it holds.Deadlock prevention is easier than deadlock detection. For example, in the Dining Philosophers Problem, a simple rule will prevent deadlocks: given that the forks are numbered, each philosopher must first pick the lower-numbered fork before attempting to pick the higher-numbered fork.

### Which are some concepts closely related to deadlock?

While we often talk about processes blocked on resources, there are also **communication deadlocks**. In these cases, processes are waiting for messages from other processes. If two processes are waiting on each other for messages, they're in a communication deadlock. For generalization, we could consider messages as resources. Communication deadlocks are possible in distributed databases where a transaction requests some nonlocal data and is blocked until a response arrives.

Sometimes the system is not in a deadlock but work is not getting or progressing slowly. For instance, a process is unable to obtain a resource for long periods of time while other processes easily obtain the same resource, perhaps because they have a higher priority. This is called **starvation**.

Another problem is when processes are doing something, such as continually responding to each other or changing states, but not getting their tasks done. This is not a deadlock since processes are still running. It's called a **livelock**.

### As a developer, what best practices should I follow to prevent deadlocks?

To prevent deadlocks, request all necessary resources together. In other words, a process is not allowed to obtain one resource and then request another. If for some reason a process holds a resource and its request for another resource is denied, it must give up the first resource. Another best practice is to impose a linear order in which resources must be requested. For example, always request file access before a printer access.

Impose a constraint that a process can hold at most one lock at a time. It's also been said, "never call other people's code while holding a lock", unless that code doesn't acquire a lock.

To share resources, prefer alternatives to mutexes or locks. The resource could be replicated. Use a higher-level pattern to synchronize operations. For instance, the *pipeline pattern* serializes accesses without using mutexes.

To prevent communication deadlocks, at least one process must use non-blocking IO. An alternative is to use separate threads for send and receive operations, so that the process can still send messages while blocked on the receive thread.

## Milestones

1961

E. J. Braude publishes an IBM technical report titled *An algorithm for the detection of system deadlocks*.

1968

In his paper titled *Co-operating sequential processes*, Dijkstra explains mutual exclusion and the use of semaphores or status variables to solve this. He then points out that the problem of deadlock, though he uses the term **deadly embrace** instead. He gives an example of processes holding some memory and requesting more memory, leading to a deadlock. He proposes **Banker's algorithm** as a method to avoid deadlocks.

1968

Some IBM publications describe the problem of deadlocks observed in ASP/OS/360. For example, when spooling disk space is full with input and output records, an input process and an output process can both be waiting for more disk space. Deadlocks could be avoided by rejecting new jobs once spool utilization reaches 80%; or cancelling a blocked job after a defined timeout.

Jun
1971

Coffman et al. introduce the four necessary conditions for a deadlock. In time, these are called **Coffman conditions**. They also present sequence and state diagrams to explain deadlocks.

Jan
1974

Within the ARPA network (precursor to the Internet), deadlocks are being observed as resources are shared over the network. Robert Chen notes that the **deadlock problem in networks** requires a different approach from deadlocks on resources managed by a single operating system. In a message-switched system, a message travels through many store-and-forward buffers. A buffer can't be released until the message is transferred to the next buffer along the route. Deadlocks are more easily *realizable* (reachable from an empty system state) if routes are fixed and less realizable if alternatives routes (free buffers) can be found.

Dec
1987

Edgar Knapp writes about deadlocks in **distributed databases**. He presents different models of deadlocks and various deadlock detection algorithms. He models resource requests using what he calls a **wait-for-graph (WFG)**. In the figure, the circles are nodes. Node t11 has two outstanding requests to t21 and t31. The cycle t11-t31-t33-t43-t41-t11 is in fact a deadlock. Mukesh Singhal extends this work in November 1989.

Deadlock


# 5G UE Data Rate

## Summary

IMT-2020 specifies the maximum data rate that 5G is required to offer. This is **20Gbps (downlink)** and **10Gbps (uplink)**. This is significant increase compared to IMT-Advanced (4G/LTE Release 14) that offers 1Gbps (downlink) and 50Mbps (uplink).

3GPP gives us a formula to calculate the theoretical peak data rates at Layer 1. Knowing that real-world deployments are unlikely to achieve this, IMT-2020 specifies that "user experienced" data rate should be at least **100Mbps (downlink)** and **50Mbps (uplink)**. This value is 5th percentile value of user throughput that's based on bits correctly delivered to Layer 3.

We should also make a distinction between peak rate, average rate and median rate. Speed tests reporting only peak data rates can be misleading. Often real-world tests will quote average or median rates as well.

## Discussion

### How can I calculate the theoretical data rate in 5G NR for uplink and downlink?

3GPP specifications TS 38.306 shares a formula for calculating the maximum data rate that a UE can support in downlink and uplink. The formula takes into account Carrier Aggregation (CA). The calculation is per component carrier (CC) and then added up over \(J\) carriers.

Among the constants, \(10^{-6}\) gives the final value in Mbps. \(R\_{max}\) is 948/1024. This is the maximum LDPC coding rate. The number 12 refers to the number of sub-carriers in a Resource Block (RB). This is multiplied with \(N^{BW}\_{PRB}\), which is the maximum number of RBs that can be allocated to the UE for a given sub-carrier spacing (SCS) and bandwidth. The formula assumes normal cyclic prefix.

Number of layers relates to MIMO layers. Modulation order \(Q\_m\) translates symbols to bits. It takes values 1/2/4/6/8 for BPSK/QPSK/16QAM/64QAM/256QAM.

We divide bits by symbol time \({T\_s}\). There are 14 OFDM symbols per slot, with 1ms slot time at 15kHz SCS. At higher SCS, the slot duration proportionally decreases, translating to higher data rates.

### Could you explain the scaling factor and overheads in the UE data rate formula?

**Scaling factor** \(f\) can take values 1, 0.8, 0.75 and 0.4. It relates to MIMO layers and modulation order that are considered *per band combination* in the formula.

Consider for example a UE whose Baseband Processing Capability (BPC) is 100MHz, 4 layers and 256QAM. Suppose this UE is configured for two component carriers. Due to BPC, the UE can support only two layers for each CC, resulting in a scaling factor is 0.5. Alternatively, the UE may be configured to use four layers for each CC but a lower modulation order 64QAM is used instead, resulting in a scaling factor is 0.75. Scaling factor is configured via RRC signalling.

**Overhead** \(OH\) ranges from 8% (FR1 UL) to 18% (FR2 DL). This is due to PHY signalling, typically including SSB, TRS, PDCCH, DM-RS, PT-RS, CSI-RS, PUCCH and SRS. We note that downlink mmWave transmissions have the highest overhead.

### How do I calculate the data rate for the multi-RAT case?

For a UE using both E-UTRA and 5G NR, called Multi-Radio Dual Connectivity (MR-DC), rates are calculated separately for each Radio Access Technology (RAT) and then added up. E-UTRA rate that can be computed using E-UTRA's Transport Block Size (TBS). Data rate in Mbps is given by,

$$ 10^{-3}\cdot\sum\_{j=1}^J TBS\_j$$

This considers \(J\) E-UTRA component carriers in MR-DC band combination. \(TBS\_j\) is the maximum number of transport block bits in either DL-SCH or UL-SCH channels transmitted over a 1ms Transmission Time Interval (TTI) for the j-th CC. This factors in the number of MIMO layers used for that CC.

### Could you show some sample calculations of 5G UE data rates?

Consider FR1 data rate for a single CC. Parameters to use are 8 layers, 256QAM (value 8), scaling factor 1 and overhead of 0.14. Given 60kHz SCS, the equivalent numerology µ is 2. At 100MHz bandwidth, the number of RBs is 135. The calculated data rate is 4623Mbps. Instead, if we use 30kHz, at 100MHz we get 273 RBs. This gives us a slightly higher rate of **4674Mbps**.

In FR2, we limit to 2 layers but move up to 120kHz SCS at a bandwidth of 400MHz. Numerology µ is 3. We get 264 RBs. Overhead rises to 0.18. The calculated data rate is **4310Mbps**. By trading off MIMO layers against bandwidth, we obtain about the same data rate. FR2 gives lower latency but suffers from lower transmission range.

Uplink calculations are similar except for the overhead that differs from downlink.

These calculations are only for a single CC. 5G NR allows aggregation of up to 16 CCs. However, maximum aggregated spectrum is 1GHz. Moreover, the total rate that a UE can handle would depend on supported UE categories.

### For 5G NR sidelink channels, how can I calculate the theoretical data rate?

For 5G NR sidelink channels, the above formula gives the maximum data rate. The parameters are almost similar to the case of downlink/uplink data rate calculation but there are a few differences.

In Release 16, sidelink channel is limited to 2 layers of MIMO. Overhead is 0.23 (FR1) and 0.25 (FR2).

### What online resources are available to calculate 5G UE data rates?

5G Tools for RF Wireless has a useful tool for 5G NR data rate calculation. Users can select downlink or uplink, FDD or TDD, number of component carriers, layers, modulation order, LDPC coding rate, scaling factor, SCS and bandwidth. There's also a selection of TDD slot format. The tool computes and displays all values derived from the above inputs.

An alternative is Clarke's calculator.

## Milestones

Jun
2017

**LTE-Advanced Pro Release 14** is finalized. LTE-Advanced Pro is specified in Release 13 and 14. This improves on LTE Release 8 and Release 9 with 10x faster downloads and 3x faster uploads. With LTE-Advanced Pro offering 1Gbps downlink data rate, it's also called *Gigabit LTE*. LTE-Advanced Pro is seen as a suitable migration path to full 5G deployments.

Nov
2017

A report published by ITU-R specifies the minimum requirements for IMT-2020. This specifies data rate, spectral efficiency, user plane and control plane latencies, connection density, energy efficiency, reliability and mobility metrics. For data rate and spectral efficiency, peak and 5th percentile numbers are noted.

Mar
2018

In RAN WG1 meeting, **scaling factor** is introduced into the UE downlink/uplink data rate formula. This clarifies that the formula is for *per band combination*.

Mar
2020

Researchers at the University of Toronto publish their findings on how well 5G NR satisfies IMT-2020 requirements. Based on simulations, they find that user experienced data rates meet IMT-2020 requirements.

Dec
2020

Real-world measurements by Opensignal for the period Oct-Dec 2020 show that 5G networks are delivering average speeds better than **150Mbps (downlink)** and **20Mbps (uplink)**. User rating is excellent for video, voice and gaming experience. Philippines and Thailand have benefited the most for video apps compared to their existing 4G networks. It's expected that as networks deploy 5G Core, latency will improve, thus leading to even better experience. However, 5G availability experienced by users is still poor, at 30% in Kuwait, 25% in South Korea and 20% in the USA.

5G UE Data Rate