# Design Patterns for Microservices and Containers

## Summary

For common software design problems, design patterns provide reusable solutions. Rather than design a solution from scratch, it's faster and easier to understand the context and see what design patterns can be reused for your application.

For a cloud application that uses containers and microservices, there are patterns to do the same. Called either *Design Patterns for Microservices* or *Container Design Patterns*, we use these terms interchangeably in this article.

Let's note that this article is not about container data structures or components. In programming, containers are data structures such as sets and queues that contain other data members. Such containers have design patterns too.

## Discussion

### What's the motivation for adopting design patterns for microservices?

Sometimes there's a need to scale your monolithic application, to release updates faster or to manage large teams of diverse skills. One approach is to re-architect your application into a bunch of microservices that get deployed on the cloud as containers. This refactoring can be a long process for a complex product. It can also be daunting for folks new to the microservices architecture.

This is where design patterns can help. These patterns have evolved from best practices that engineers used when managing microservices and containers. When solutions to problems are known, they can be applied to similar problems in the future. Patterns formalize this concept of reusable solutions.

Patterns help you to build your apps faster and make them more reliable. Moreover, you may able to use libraries or tools–Amalgam8, Hystrix, Eureka, Zuul –that have already implemented some of these patterns.

### What are the elements of a good container design pattern?

A design pattern must be clearly described with concrete examples and best practices. Without a clear understanding, others will not be able to apply or even know if it suits their application needs. The pattern must be agnostic of container runtimes or cloud infrastructure.

For any pattern to be practical, it's necessary that the containers themselves are properly designed. Container design principles can be applied in two different contexts:

    + **Build time**: single concern, self-containment, image immutability
    + **Runtime**: high observability, lifecycle conformance, process disposability, runtime confinementFrom the perspective of delivering software-as-a-service, similar principles have been noted in Heroku's The Twelve-Factor App.

### What are the common design patterns for microservices?

There are dozens of design patterns for microservices that can be grouped by categories such as decomposition, observability, testing, UI, communication, database, security, deployment, and so on. Google engineers identified the following in their work with Kubernetes:

    + **Single container management**: graceful termination, probing, config maps
    + **Single node multi container**: sidecar, ambassador, adapter
    + **Multi node**: leader election, work queue, scatter/gatherMicrosoft has published a number of design patterns including ambassador, backends for frontends, CQRS, event sourcing, gatekeeper, gateway aggregation, sidecar, strangler, throttling, etc. Other patterns to note are model, denormalizer, ingestor, service registry, edge controller, API gateway, circuit breaker, fallback, command, bounded context and failure as a use case.

### Since there are so many patterns, which ones are important or essential to know?

It's hard to claim that one pattern is more important than another. It all depends on the context and the needs of a particular application. However, we can highlight the following patterns:

    + **API Gateway**: Clients call your application via command-line tools or GUI. Rather than call each service of your app separately, all calls will go via the gateway.
    + **Service Registry**: Since microservices are created and destroyed dynamically based on load, this pattern helps in connecting clients to available instances of microservices. Microservices register with the service registry.
    + **Circuit Breaker**: When a service is down, this failure could cascade to other services dependent on it. To prevent escalation, if failures exceed a certain threshold, services will not repeatedly call a failed service until recovery happens.
    + **Fallback**: Also called *Chain of Responsibility* pattern, if a service is unable to complete because of a dependency, a fallback method is called. By combining fallback pattern with circuit breaker pattern, more resilient apps can be built.

### Could you explain patterns for running two containers on a single node?

Let's look at three patterns that each involve two closely associated containers. It makes sense to pair containers within the same node (or same Kubernetes pod) if they have the same lifecycle. One of them is the main container while the other can be seen as a helper. We explain them as follows:

    + **Sidecar**: To provide an auxiliary service to the main container, such as for logging, synchronizing services, or monitoring.
    + **Adapter**: To standardize or normalize the output/input/interface between the main container and the external world, such as reformatting logs or invoking libraries that don't have expected language bindings.
    + **Ambassador**: To connect the main container to the outside world, such as for proxying localhost connections to outside connections. For example, main container will always see a localhost database with the ambassador resolving this to the actual database outside.

### Are there patterns to manage data in a microservices architecture?

For easiest migration from a monolithic app, all services could share the same database. For better isolation and modularity, we can adopt private-tables-per-service and schema-per-service, perhaps on the same database server. For high throughput requirements, each service could have its own database server.

One challenge with microservices is to ensure data consistency across all services. An important principle is to have a single authoritative source of truth: all others are read-only and non-authoritative. Services can synchronously query for data or asynchronously update it via events when that data changes. Whether to use synchronous API or asynchronous messaging depends on your application since both have their trade-offs.

An event-driven architecture using publish-subscribe model aids asynchronous updates. Store only what a service needs. If services are constantly exchanging data, you might want to rethink service boundaries.

A transaction that involves multiple services is not easy to implement. Some patterns are available: Saga Pattern, API Composition, Command Query Responsibility Segregation (CQRS), Scheduler Agent Supervisor, and Compensating Transaction.

## Milestones

2003

To allow trace propagation through multiple microservices, the **Correlation ID** pattern is identified by Gregor Hohpe in his book *Enterprise Integration Patterns*.  It's important to note that the pattern is identified in a different context and only later applied to microservices.

2004

Influenced by fig trees, Martin Fowler documents the **Strangler** pattern. This is useful when migrating a monolithic app to a microservices architecture. Rather than have cut-over date to move from old to new, this pattern enables a gradual transition.

2007

In his book *Release It!*, Michael Nygard introduces the **Circuit Breaker** pattern. With this pattern, a service "fails fast" by not calling another service that it knows is already down. Netflix's Hystrix framework implements this pattern.

2011

The term **microservices** is first used at a workshop in Venice. By 2012, ideas and case studies are presented. Adrian Cockcroft of Netflix sees this as "fine-grained SOA".

2013

Although the history of containers can be traced to UNIX of late 1970s, **Docker** as a container is released in 2013. Developers begin to see containers as a natural fit for microservices.

2016

Brendan Burns and David Oppenheimer present a bunch of patterns for container-based apps at the HotCloud 2016 conference. This paper may be considered foundational and credited with putting the spotlight on container patterns.


# Code Reuse

## Summary

When new projects are started in an organization, it makes sense to identify parts specific to the application and other parts that can be reused from earlier projects. Code reuse is a practice that reduces development time and effort since we're reusing existing code rather than writing from scratch.

Code reuse is not limited to within an organization. Languages often ship with built-in functions and standard libraries. Third-party libraries add functionalities to these. When developers write applications, they make use of these libraries.

There are two approaches to code reuse: design and implement code with the expectation of future reuse; identify and refactor when we see opportunities for immediate reuse. The industry is divided over which of these two approaches is the better one. Some experienced developers have shared useful tips about how best to adopt each approach.

## Discussion

### Could you explain code reuse with an example?

Suppose we're building an accounting application. Obviously it has a lot of business logic related to the domain of accounting. But it also does user authentication, interfaces to a database and logs errors: these are features that are not specific to accounting. In fact, other applications (such as shopping cart or content management) may have similar requirements.

It therefore makes sense to build these components once and reuse them across applications. Thus, we could have a shared library that connects to a database. This library is then reused across applications that need such a connection. Likewise, separate libraries can be created for authentication or logging, and reused across applications.

By reusing code this way developers can focus on the core business logic of their apps instead of "reinventing the wheel". If reusable code is well tested, it improves overall product quality.

### What are some types of software reuse?

Reuse can be vertical (within an application domain) or horizontal (across application domains). Reuse is not just about code. We can reuse other software artefacts including requirements, architecture, design models, test reports, etc.

An application can be decomposed into components, each fulfilling a specific purpose. A **component** hides its complexity behind an interface. Reuse happens via such an interface. Such components could be views, models, controllers, data access objects, or plugins.

When a component is deployed independently, it's called a **service**. Other applications or services can call it. Thus, reuse happens at the service level.

When components are shared across multiple applications and systems, they can be packaged and distributed as **APIs** or **libraries**. Platforms and frameworks extend this philosophy by bringing together many useful APIs or libraries.

Another form of reuse is at the level of source code. Code is **forked** or copied and then adapted to new requirements. A coarser form of this is **cut-and-paste programming**. This is also code reuse but it's not favoured because duplicating code this way makes it less maintainable.

### Isn't code reuse a myth?

During the late 1980s, ad hoc or unplanned reuse was not favoured. It was thought that "reuse should be considered at design time, not after the implementation has been completed". Software architects cared about interface design, low coupling and high cohesion.

By the late 1990s, it was realized the code reuse is hard and needs practice. It's more of an art than a science. It combines both technical and non-technical challenges. Reusing software across business units in a large organization needs coordination. It's also not clear which cost centre will fund it. Developers also don't like management forcing them to use components and frameworks developed by another team.

Striving for a high degree of reuse can lead to complex dependencies and high coupling, not to mention the higher development cost. Changes to shared libraries take longer. Complex abstractions reduce code readability. Developers therefore start implementing their own code rather than reuse code they don't understand.  Sometimes, "reusable code" is too application or domain specific to be reused. It's been said that,

> There is no such thing as reusable code, only code that is reused.

### Could you share some tips towards writing reusable code?

Apply the Don't Repeat Yourself (DRY) principle. If you find yourself repeating the same code, it's time to extract that code into a separate function or class method.

Think about unit testing your code. Code that's easier to unit test implies less coupling, higher modularity and better reuse.

Encapsulation reduces external dependencies and makes code more self-contained. When writing code, imagine you're creating tools for others to reuse.

Chances of reuse are better if a class or method does only one thing. You'll end up with more classes or methods but there's less coupling in your codebase. Better still, use interfaces and abstract classes. These can be easily extended in future. Don't include business logic as they lead to higher coupling and less reuse.

If you're struggling to name your class or method, it's an indication that the abstraction is probably wrong. If the abstraction requires many inputs, it's likely to introduce high coupling.

In conclusion, if you write modular, loosely coupled, maintainable and unit testable code, that code is also likely to be reusable.

### What should I avoid when attempting to write or use reusable code?

Don't develop applications from a framework perspective. Application development should not start focusing on building a framework of reusable components. Framework development should be a separate activity. Otherwise, too much of what goes into the framework will become application specific and never reused. Don't try to reuse code from another application or context. It may require far more effort than coding it afresh.

Don't let reuse become your primary goal at the expense solving real problems. You'll end up with unnecessary interfaces, confusing abstractions and heavily partitioned codebases. Such code will not get reused.

Don't try to solve generic problems and create your own reusable libraries. Most likely, there are already frameworks and libraries that solve them for you.

Don't force developers to use reusable frameworks. Frameworks should speak for themselves in terms of clean interfaces, readability, usability, reliability, performance and security. They should act as "reuse magnets" for developers.

Don't reuse something old and outdated. Take benefit from newer languages, technologies and frameworks. For example, where possible, code in Swift instead of trying to reusing old Objective-C code.

### Is code reuse a good thing from a security standpoint?

Within enterprises, code reuse promotes higher security if code is well tested. If not, it becomes a single point of failure, making many applications vulnerable.

When reusing code from external sources, document the software supply chain. Reuse frameworks and libraries only from trusted sources. Install updates and patches at the earliest. This is critical if a vulnerability in an open source codebase becomes public.

Use Software Composition Analysis (SCA) tools. These can identify vulnerabilities in third-party source code and libraries. Examples include Black Duck, Sonatype and OWASP Dependency-Check. Use them together with other security testing tools.

A checklist can also be useful when selecting software for reuse. Such a checklist would assess feature capabilities, limitations and interfaces. Apart from security considerations, such an assessment would consider initializations, memory usage, throughput, accuracy, synchronization, re-entrancy, etc.

### What support do languages offer towards code reuse?

Generics in Ada and classes in object-oriented languages promote code reuse. However, these features alone don't guarantee code reuse. Proper domain analysis, modularity, abstractions and parameterization are important to get reuse right.

In C++, functions, macros, encapsulation, composition, parameterized types, overloading and polymorphism are seen as useful techniques for code reuse.

In OOP, complex class hierarchies and huge classes with lots of methods and arguments can make reuse difficult to achieve. In functional programming, the approach is to reuse tiny bits of code. Lambda functions enable this. We can implement complex behaviour by composing code from many tiny parts.

In JavaScript, use immutables and functional programming for higher reuse. For example, prefer `map/reduce/filter` over `forEach/push/splice`. Pure functions and Higher-Order Functions (HUFs) are useful constructs. They enable reuse across JavaScript frameworks.

## Milestones

1949

While working on the stored-program computer EDSAC, researchers at the University of Cambridge propose the creation and use of a **subroutine library**. This is probably one of the earliest instances of code reuse.

Oct
1968

At the NATO Software Engineering Conference, McIlroy proposes a **components factory** dedicated to producing parameterized families of components arranged by precision, robustness, generality and time-space performance. Rather than reuse entire compilers, it's easier to start with numerical approximation routines, I/O conversions, 2D-3D geometries, storage management and text processing. As time will tell, only the Japanese pay attention to his suggestions.

1990

Hewlett-Packard creates a corporate software reuse program. This brings together many divisional programs that have been active since the early 1980s. About the same time (early 1990s), many other companies have similar reuse programs: AT&T, IBM, GTE, NEC, Toshiba, etc. HP engineers learn that a large software library doesn't necessarily lead to higher reuse. The library may have many useful low-level, generic parts but created without consideration of quality, documentation, testing or interoperability. They also note that effective software reuse will require changes to processes. The returns will also come only in the long term.

Feb
1992

Krueger notes that though software reuse has been discussed since the late 1960s, it's not yet a standard industry practice. He also looks at reuse from four perspectives: (a) **abstraction**: essential for reuse; (b) **selection**: identify and select what to reuse, classify and organize reusable code; (c) **specialization**: reuse a generalized artefact but specialize it during reuse via parameters, transformations or constraints; (d) **integration**: ways to export and import reusable code.

1995

Gamma et al. publish the book titled *Design Patterns: Elements Of Reusable Object-Oriented Software*. This marks a shift from reusing just code to **reusing design**. These patterns describe proven solutions to common problems.

2004

**Reuse in Software Engineering (RiSE)** project is initiated to provide a framework for effective code reuse. RiSE includes a number of best practices, describes the reuse process and addresses both technical and non-technical aspects.

2008

Haefliger et al. investigate code reuse in six open source software projects. They find that developers reuse code to reduce costs, integrate functionality quickly, and overcome resource constraints. Developers select components based on their popularity.

Code Reuse


# MongoDB Query Optimization

## Summary

MongoDB query performance depends on indexes defined on a collection and how those indexes are employed within the query. MongoDB offers different types of indexes. Precisely what indexes to define and how to use them depends very much on the application. This article introduces indexes and shares some guidelines on their usage.

MongoDB also provides an inspection system to obtain statistics on query execution and what happens under the hood. This information helps a developer figure out ways to optimize the query.

## Discussion

### How can I get insights into MongoDB query performance?

On the MongoDB shell, methods `cursor.explain("executionStats")` and `db.collection.explain("executionStats")` give insights into the performance of a query. For example, we append `explain()` to a query thus, `db.inventory.find( { quantity: { $gte: 100, $lte: 200 } } ).explain("executionStats")`. MongoDB Compass also has Explain Plan tab to display the same statistics graphically.

Explainable classes can be obtained by running `db.collection.explain().help()`. The explain method can be called with verbosity modes queryPlanner (default), executionStats, or allPlansExecution. In all modes, for write operations, we get information about the write without executing the write.

The query plan is presented in the explain results as a tree of stages. These stages can be COLLSCAN (collection scan), IXSCAN (index scan), FETCH (retrieving documents), SHARD\_MERGE and SHARDING\_FILTER. Explain can be run on either unsharded or sharded collections.

### How do I interpret the results of the explain methods?

Among the important explain results are:

    + `executionStats.nReturned`: Number of documents returned.
    + `executionStats.executionTimeMillis`: Total time in milliseconds for query plan selection and execution.
    + `executionStats.totalKeysExamined`: Number of index entries scanned.
    + `executionStats.totalDocsExamined`: Number of documents examined, by COLLSCAN and FETCH stages.
    + `executionStats.executionStages`: Details of execution of the winning plan as a tree of stages. Where indexes are used (IXSCAN), result includes index key pattern, direction of traversal, and index bounds.As an example, if `totalKeysExamined` is zero, it implies that indexes are not used. If `totalDocsExamined` is much higher than `nReturned`, this means that too many documents were scanned when only a few were returned. Both these suggest better performance can be obtained with the use of indexes. Making these changes would give an efficient winning plan with root stage FETCH and an input stage IXSCAN.

### How can indexes improve query performance?

Indexes improve performance by limiting the number of documents that need to be scanned. Indexes are usually in RAM or located sequentially on disk. Though indexes take up space, this becomes a worthwhile trade-off if queries use them often. Indeed, designing the right indexes requires a deep understanding of the application's queries and how often they're called.

For example, if an `inventory` collection is queried on field `type`, create an index on that field: `db.inventory.createIndex({ type: 1 })`. This is an example of a covered query. A **covered query** is one that can be satisfied by indexes alone without needing to examine the documents. More specifically, all fields in the query are part of an index, all returned fields are in the same index, and no fields in the query are equal to null.

Two limitations are worth noting. Geospatial indexes cannot cover a query. Multikey indexes cannot cover queries over array fields.

Usage of indexes can be obtained via the `$indexStats` aggregation stage or in the outputs of `serverStatus`, `collStats` and `dbStats`.

### What are the different types of indexes available in MongoDB?

MongoDB supports the following index types:

    + **Single-Field Index**: Based on a single field of a document, such as, `{score: 1}`.
    + **Compound Index**: Based on multiple fields, such as, `{userid: 1, score: -1}`, with 1 and -1 referring to ascending and descending sort order. The starting keys (`userid`) are called *index prefix keys*.
    + **Multikey Index**: Indexes the content stored in arrays. When you index an array field, MongoDB creates a separate index for every array element. Index `{"addr.zip": 1}` is an example of indexing a nested field.
    + **Geospatial Index**: Supports efficient querying of geospatial coordinate data, either 2D or 3D geometry.
    + **Text Index**: For searching text data. MongoDB does stemming and ignores stop words.
    + **Hashed Index**: Indexing hash of a field value leading to more even data distribution across shards. Supports equality matches but not range-based queries.Indexes have useful properties: unique, partial indexing, sparse (don't index documents without the index field), TTL (documents that expire), hidden (not seen by query planner). Unique index on `_id` is created by default.

### How should I use indexes to improve performance?

Sometimes matching on multiple fields may be inefficient. Instead, use **compound indexes**.

When a query doesn't use index prefix keys or uses a different sort order, compound indexes won't work. Instead use index intersection. **Index intersection** is when MongoDB uses two separate indexes or their prefixes to improve query performance. However, index intersection doesn't apply when the sorting uses an index completely separate from the query predicate.

In addition to using indexes for matching, use **indexes for sorting and lookups** as well.

While the query optimizer often selects the optimal index, we can also use the `hint()` method to force the use of a specific index.

Workload performance can suffer during index builds. For this reason, **rolling index builds** are preferred. Starting from secondary members, rolling builds affect one replica set member at a time. Deleting many documents in a short time may also affect reading performance due to index rebuilds. Use rolling builds or mark document for deletion and delete them later with a background scheduler.

### How can I improve the performance of my aggregation pipeline?

MongoDB's database engine can optimize performance of an aggregation pipeline at runtime. It can **reorder the stages** if it sees that this will improve performance. For example, sorting an entire collection and then matching is inefficient. It's more efficient to match first and then sort so that fewer documents need to be sorted. Matching itself is run in parallel on multiple shards.

Sometimes, the engine needs help from the developer to optimize performance. Assume a pipeline contains a stage that matches on a computed field that's not indexed. The engine can't optimize by moving the matching stage before the computation. Computation is done on the entire collection. An easy solution is to rewrite the matching stage to use an already existing and indexed field. Even if computation happens earlier, the engine can move the matching to precede the computation.

### What are query plans in MongoDB?

For any given query, actual query execution can have a few possible ways. Each of these is called a **query plan**. Based on indexes, the MongoDB query optimizer chooses the most efficient query plan. For each candidate plan, the optimizer looks at the number of work units that need to be performed. More indexes imply more plans to inspect and query selection can take more time.

If a plan is selected, it's also added to the **plan cache**, which contains the query shape, works cost and a state. A query shape consists of a combination of query, sort, and projection specifications. Cache state can be Missing, Inactive or Active. Query plans for subsequent queries of the same shape are read from the cache if active and cost meets the selection criterion. The optimizer considers only those indexes specified in the filter of a given query shape.

If `mongod` restarts or shuts down, cache doesn't persist. Index or collection drops clear the cache. Cache can also be cleared manually. If cache is lower than 0.5GB for all collections, complete debug information is stored. Otherwise, additional cache entries are stored with minimal information.

### Could you share some tips towards optimizing MongoDB queries?

Use a highly **selective query**, that is, a query that matches a small percentage of the collection's documents. Less selective queries can't use indexes effectively. For example, `$nin` or `$ne` operators are not very selective. Given an employees collection, the query `{name: "John", sex: "male"}` with indexed name is likely to be more selective than the query `{sex: "male", name: "John"}` with indexed sex.

Use **projection** to obtain only those fields required for further processing.

To avoid application-level joins, **denormalize data**. For example, instead of having two collections Employees and Companies, and then linking them, the latter can be included as part of employee documents. If this is not possible, use `$lookup` instead.

**Third-party services** such as Rockset does real-time indexing to enable fast joins. Idealo's MongoDB profiler tool can also be useful.

By default `db.collection.update()` updates a single document. Use `multi: true` to update all matching documents efficiently.

### Apart from query optimization, how do I tune the performance of MongoDB?

Analyze lock metrics. Long lock times might suggest problems with schema design, query structure, or system architecture. Look at WiredTiger's cache usage and see if it needs to be increased. For read-heavy applications, increase the size of replica sets. For write-heavy applications, do sharding. Monitor replication state and lag.

In the cloud, MongoDB provides a monitoring tool that gives many useful metrics: execution times, disk utilization, memory usage, network I/O, op-counters, query targeting, queues, and CPU usage. Database operations can also be logged for later analysis.

When doing high-speed writes, enable journaling. In case of a system crash during a write, journaling ensures that data is in a consistent state.

Try to keep documents to a few kilobytes. Avoid large array fields. A large and growing array may cause the document to be relocated on disk, leading to index updates. Alternatively, large fields when not used in queries, could be compressed and stored.

Batching improves resource utilization. For example, Kafka events can be consumed in a batch of 100 rather than processed one by one.

## Milestones

Feb
2009

**MongoDB 1.0** is released. By August, this version becomes generally available for production environments.

Mar
2015

MongoDB 3.0 is released. This includes a new **query introspection system** towards query planning and execution. The formatting and fields of explain results are improved. Starting in this version, with the exception of `_id` index, an index can't cover a query on a sharded collection if the index doesn't contain the shard key.

Nov
2017

MongoDB 3.6 is released. An index can cover a query on fields within embedded documents. Multikey indexes can cover queries over non-array keys if the index tracks which fields make it multikey.

Aug
2019

MongoDB 4.2 is released. Plan cache entry is associated with a **state**: Missing, Inactive or Active. Aggregation stage `$planCacheStats` is available to view cache information. To help identify slow queries, query plans also now have associated hash strings `queryHash` and `planCacheKey`. The former is dependent on only the query shape while the latter depends on query shape and indexes. An aggregation pipeline with `$out` stage can't contain `db.collection.explain()` in executionStats or allPlansExecution modes.

Jul
2021

MongoDB 5.0 is released. This puts a size limit of 0.5GB for the plan cache, which is also backported to updates of some earlier versions.

MongoDB Query Optimization


# 5G NR RLC Acknowledged Mode

## Summary

In 5G NR, an RLC AM entity is bidirectional. The transmitting side of an entity communicates with its peer entity's receiving side. The transmitting side sends data PDUs while the receiving side acknowledges the transmission via control PDUs.

An AM Data (AMD) PDU always includes an RLC header, which in turn contains the Sequence Number (SN) of the RLC SDU. SDUs may be segmented. SN and Segment Offset (SO) are essential header fields that help with acknowledgements and retransmissions. STATUS PDU is the only type of control PDU and it too has a header.

Behaviour is controlled by state variables, constants and timers. Compared to the other two RLC transmission modes (TM and UM), AM is more complex.

## Discussion

### How does an AM RLC entity operate?

**The transmitting side** receives RLC SDU from upper layer. It assigns the next available SN to this SDU. It sends an RLC header plus RLC SDU to MAC. If the entire SDU can't be sent in a single transmission, it's segmented. Header plus the largest possible segment are given to MAC.

**The receiving side** acknowledges via ACK\_SN (good SDUs) and NACK\_SN (lost SDUs or segments). If all segments of an SDU are correctly received, it forwards the RLC SDU to upper layer. Order of delivery is not important for RLC.

Transmitting side **retransmits** SDUs or segments previously lost. While retransmitting, it may re-segment lost portions of the SDU. RLC is permitted a maximum number of retransmissions, after which the error is indicated to upper layer.

For **flow control**, both sides obey the window size, which depends on the SN bit length: 2048 (12-bit SN) or 131072 (18-bit SN).

Via **polling**, transmitting side can request its peer to send current status. Receiving side has a timer to regular status transmissions. Likewise, transmitting side has a timer to regulate polling.

### Could you explain the state variables used by an AM RLC entity at the transmitting side?

The following state variables exist at the transmitting side of AM RLC:

    + **TX\_Next**: Send state variable. Initialized to 0. Assigned as SN to the next RLC SDU received from upper layer; and then incremented.
    + **TX\_Next\_Ack**: Determines lower edge of the transmit window. It's the SN of next in-sequence SDU that needs to be positively acknowledged. Initialized to 0. Updated when there's positive acknowledgment for SDU with SN=TX\_Next\_Ack. It's obvious that TX\_Next\_Ack ≤ TX\_Next.
    + **POLL\_SN**: Highest SN among all PDUs submitted to lower layer at the time of polling. Initialized to 0.

### Could you explain the counters used by an AM RLC entity at the transmitting side?

The following counters exist at the transmitting side of AM RLC:

    + **PDU\_WITHOUT\_POLL**: Number of PDUs sent since the most recent poll. Initialized to 0.
    + **BYTE\_WITHOUT\_POLL**: Number of bytes sent since the most recent poll. Initialized to 0.
    + **RETX\_COUNT**: Number of retransmissions of an SDU or its segment. One counter per RLC SDU.Relevant to the counters are three parameters that RRC configures:

    + `maxRetxThreshold`: Limits the number of retransmissions. When RETX\_COUNT reaches this limit, upper layers are indicated of SDU transmission failure. This may lead to an RRC Re-establishment procedure.
    + `pollPDU`: When PDU\_WITHOUT\_POLL reaches or exceeds this value, polling is triggered.
    + `pollByte`: When BYTE\_WITHOUT\_POLL reaches or exceeds this value, polling is triggered.

### Could you explain the state variables used by an AM RLC entity at the receiving side?

The following state variables exist at the receiving side of AM RLC:

    + **RX\_Next**: Determines the lower edge of the receive window. Initialized to 0. Updated when an SDU with SN=RX\_Next is fully received.
    + **RX\_Next\_Highest**: Contains the next SN following the highest SN of all received SDUs, even if the highest-SN SDU is not fully received. Thus, RX\_Next ≤ RX\_Next\_Highest. Initialized to 0.
    + **RX\_Highest\_Status**: Highest possible value to be indicated in ACK\_SN field of STATUS PDU. SDUs with lower SN that haven't been fully received will be indicated with NACK\_SN in the STATUS PDU. Initialized to 0.
    + **RX\_Next\_Status\_Trigger**: SN following the SN of the RLC SDU that triggered t-Reassembly. Thus, if SN=2 is partially received and t-Reassembly is started for it, this variable is set to 3.

### Could you illustrate the update of RLC AM receiving side state variables?

This example is based on RLC specification TS 38.322. It's accompanied by code in the Sample Code section of this article.

We present three scenarios. All start with SN=0 fully received, then SN=1 and SN=2 partially received. When SN=1 is partially received, t-Reassembly is started. When SN=2 is partially received, there's no action on the timer since it's already running for SN=1.

In scenario (a), SN=1 is fully received. Thus, t-Reassembly is stopped and reset. However, since SN=2 was partially received earlier, t-Reassembly is started for SN=2. When SN=2 is later fully received, reassembly is completed for this SDU, and timer is stopped and reset.

In scenario (b), SN=1 is fully received and hence t-Reassembly is restarted for SN=2. When t-Reassembly expires (could not reassemble SN=2 soon enough), status reporting is triggered. STATUS PDU will include NACK for SN=2.

In scenario (c), SN=3 is partially received. When t-Reassembly expires (could not reassemble SN=1 soon enough), status reporting is triggered. STATUS PDU will include NACK for SN=1 and SN=2, and possibly for SN=3.

### Could you explain the timers used by an RLC AM entity?

An RLC AM entity maintains three timers:

    + **t-PollRetransmit**: At the transmitting side. Started or restarted when poll bit is set in an AMD PDU. Stopped when STATUS PDU is received indicating ACK/NACK for POLL\_SN, which is the highest SN transmitted at the time of the last poll. At expiry, initiate data retransmissions and poll retransmission.
    + **t-Reassembly**: At the receiving side. Detects loss of RLC PDUs. Started when a segment is received but more segments are pending for that SDU. Stopped when SDU is completely received. At expiry, triggers status reporting.
    + **t-StatusProhibit**: At the receiving side. Prohibits frequent transmissions of STATUS PDU. Started when a STATUS PDU is sent. Expiry of t-Reassembly triggers a status report but STATUS PDU is sent only when t-StatusProhibit expires. Status reporting is triggered either by polling (transmitting side initiated) or t-Reassembly expiry (receiving side initiated).At most only one of each of the above timers can be running in an AM RLC entity.

## Milestones

Apr
2017

An early draft of 5G NR RLC specification **TS 38.322** is released. This evolves to version 1.0.0 by September.

Dec
2017

3GPP publishes **Release 15** "early drop". RLC specification TS 38.322 is updated to version 15.0.0.

Jun
2018

The setting of POLL\_SN is corrected in the specification.

Jul
2020

3GPP publishes Release 16 specifications. RLC specification is TS 38.322 version 16.1.0.

5G NR RLC Acknowledged Mode


# 5G NR RLC PDU

## Summary

An RLC PDU (Protocol Data Unit) consists of an **RLC header** and **data**. From an upper layer, RLC receives an RLC SDU (Service Data Unit). The data part of an RLC PDU is either a complete RLC SDU or an SDU segment. A single RLC PDU maps to a single MAC SDU.

RLC has three transmission modes: TM, UM and AM. Each mode has its own **data PDU** formats. STATUS PDU, which is an RLC **control PDU** sent as part of AM RLC entity, has its own formats.

Every RLC SDU coming from an upper layer is byte aligned, that is, multiple of 8 bits. First bit of RLC SDU becomes first bit of RLC PDU. UMD and AMD headers are byte aligned but not the STATUS PDU header.

## Discussion

### What are some essential details when constructing an RLC PDU?

RLC's upper layer is either PDCP or BAP. If RLC SDU is from PDCP, the **maximum data field size** in an RLC PDU is the maximum size of a PDCP PDU.

Unlike LTE RLC, 5G NR RLC supports **SDU segmentation but not concatenation**. This means that an RLC SDU can be sent as multiple segments, each segment going in a separate RLC PDU. But a single RLC PDU can't carry multiple SDUs or their segments.

Since there's no concatenation at RLC, RLC can start **preparing the header** for each RLC SDU before transmission grant is received from MAC. This reduces overall latency. If the grant received is smaller than desired, RLC will segment the SDU, update the RLC header and construct the final RLC PDU.

The standards don't specify a value for **segment size**. If the grant is big enough, the entire SDU shall go in a single PDU. Otherwise, the SDU shall be segmented, with RLC maximizing the segment size given the grant.

### What's the format of a TMD PDU?

A TM RLC entity sends and receives data using the TM Data (TMD) PDU format. Since no RLC header is used by TM RLC entity, and no segmentation is done, RLC SDU becomes an RLC PDU. This is the most trivial of all RLC PDU formats.

### Could you describe header fields common to UMD and AMD RLC PDUs?

Segmentation is one of the essential features of UM and AM RLC entities. The following header fields assist the receiving RLC entity to reassemble a complete SDU from its segments:

    + **Segmentation Info (SI)**: Indicates if the PDU contains a complete SDU or first/last/middle segment. Length is 2 bits.
    + **Segment Offset (SO)**: Indicates the position of the current segment in bytes within the original RLC SDU. Numbering starts from zero. Length is 16 bits.
    + **Sequence Number (SN)**: Indicates the sequence number of the corresponding SDU. Note that numbering is for SDUs, not for their segments. Length is configured by RRC: 6 or 12 bits for UMD PDU, and 12 or 18 bits for AMD PDU.

### What's the format of a UMD PDU?

A UM RLC entity sends and receives data using UM Data (UMD) PDU formats. UMD PDU has five formats: containing a complete SDU without SN; containing 6-bit or 12-bit SN with or without SO.

For groupcast and broadcast on sidelink channels, 6-bit SN is used.

SN is included only if an RLC SDU is segmented. UMD PDU carrying the first segment doesn't include the SO field. When an RLC SDU is not segmented, SN is not incremented. This makes sense since an RLC PDU carrying a complete SDU doesn't include SN.

### What's the format of an AMD PDU?

An AM RLC entity sends and receives data using AM Data (AMD) PDU formats. AMD PDU has four formats: containing 12-bit or 18-bit SN with or without SO.

Similar to UMD PDU, AMD PDU header includes SI, SN and SO. Additional fields for AMD PDU are:

    + **Data/Control (D/C)**: 1-bit field. 0 indicates control PDU and 1 indicates data PDU. This is necessary since an AM RLC entity has both data and control PDUs.
    + **Polling (P)**: Set by the transmitting side to 1 if requesting a STATUS report from its peer entity.Unlike UM, AM doesn't have a dedicated format for complete SDU transmission. However, a complete SDU can still be sent in a single RLC PDU by setting `SI=00`. Even so, AMD PDU always includes SN field, which is necessary for ARQ retransmission procedure.

Like UMD PDU, AMD PDU carrying the first segment doesn't include the SO field.

### What's the format of the STATUS PDU?

STATUS PDU is a control PDU applicable only to AM RLC entity. The header consists of 1-bit Data/Control (D/C) field and and 3-bit Control PDU Type (CPT) field. STATUS PDU is indicated with `D/C=0` and `CPT=000`.

Following the header, the STATUS PDU contains ACK\_SN, NACK\_SN, extension bits (E1/E2/E3), SOstart, SOend, and NACK range. ACK\_SN and E1 are mandatory in every STATUS PDU. Other fields are used only when there's a need to indicate incorrectly received RLC SDUs and segments.

ACK\_SN and NACK\_SN are 12 or 18 bits as configured by RRC. SOstart and SOend occupy 16 bits. NACK range field is 8 bits.

### How are extension bits used in the STATUS PDU?

There are a few ways to indicate missing SDUs or their segments, controlled by extension bits:

    + **E1**: `E1=1` if NACK\_SN/E1/E2/E3 fields follow. In this context, all SDUs prior to ACK\_SN except those indicated by NACK\_SN have been correctly received. `E1=0` implies all SDUs prior to ACK\_SN have been correctly received, common if communication is going smoothly without requiring retransmissions.
    + **E2**: `E2=1` if SOstart/SOend fields follow. Whereas E1 and NACK\_SN specify a lost SDU, E2 and SOstart/SOend specify lost segments. `SOend=0xFFFF` is a special case to indicate that all bytes from SOstart are lost.
    + **E3**: `E3=1` if NACK range field follows. This is the number of consecutive RLC SDUs lost from and including NACK\_SN. This is useful when handling burst errors. E3 can be used together with E2, that is, to inform loss from SOstart of NACK\_SN to SOend of NACK\_SN+NACK range-1. Special value `SOend=0xFFFF` is applicable.In the figure, we illustrate a few loss scenarios and their corresponding STATUS PDUs. We don't show retransmissions or re-segmentation. We assume 12-bit SN and 1024-byte SDUs segmented equally into four 256-byte segments.

## Milestones

Apr
2017

An early draft of 5G NR RLC specification **TS 38.322** is released. This evolves to version 1.0.0 by September.

Dec
2017

3GPP publishes **Release 15** "early drop". RLC specification TS 38.322 is updated to version 15.0.0. This version includes header fields SI, SN, SO, D/C and P for data PDUs; header fields D/C and CPT for STATUS PDU; data fields ACK\_SN, NACK\_SN, SOstart, SOend, NACK range, E1, E2 and E3 for STATUS PDU.

Jul
2020

3GPP publishes Release 16 specifications. RLC specification is TS 38.322 version 16.1.0. From Release 15, there are **no changes** to RLC PDU formats.

5G NR RLC PDU


# CSS Units

## Summary

CSS units are used to specify the size, position, and other properties of elements on a web page using Cascading Style Sheets (CSS). There are several types of CSS units, each with its own unique properties and use cases. These units are either absolute units or relative units.

CSS units are part of the overall CSS specification that's standardized by the W3C. There's generally good support of these units from web browsers and design tools.

Designers and web developers should get familiar with these different CSS units. They should learn when to use what and follow best practices. This will lead to a code that's more maintainable, flexible, and accessible.

## Discussion

### Which are the main categories of CSS units?

There are two main categories of CSS units:

    + **Absolute Units**: These units are fixed, regardless of the context. The most common ones are Pixels (`px`), Inches (`in`), Centimeters (`cm`), Millimeters (`mm`), Points (`pt`), and Picas (`pc`).
    + **Relative Units**: These units are relative to some other value, such as the font size or the dimensions of the containing element. The most common ones are Percentages (`%`), EM (`em`), REM (`rem`), Viewport units (`vw`, `vh`, `vmin`, `vmax`), and Ex (`ex`).It's possible to use a combination of absolute and relative CSS units to achieve complex layouts and designs. For example, a combination of percentages and pixels would create a flexible layout that maintains fixed-size margins and padding. More specifically, an element's width is in `px` but the height's in `em` to make it relative to the font size of the element. Similarly, the margin or padding could be set in `px` but the dimensions of the containing element is in `em` or `rem` to make it relative to the size of the container.

### Could you explain the main CSS units?

Pixels (`px`) is the most commonly used unit in CSS. A pixel is a single dot on a computer screen, and its size is dependent on the device's screen resolution.

Percentages (`%`) is used to define a value relative to the parent element. For example, setting the width of a child element to 50% will make it half the width of its parent.

EM (`em`) is based on the size of the font that is currently in use. For example, `1em` is equal to the current font size of the element. REM (`rem`) is similar to EM, but it is based on the font size of the root element (typically the `<html>` tag), rather than the element that is currently in use.

Viewport units are based on the size of the viewport, which is the visible area of the browser window. The two most common viewport units are `vw` (viewport width) and `vh` (viewport height). Units `vmin` and `vmax` are useful to choose the width or height, whichever is the minimum or maximum respectively.

### As a designer, when should I use what CSS units?

Absolute units are useful for creating fixed and precise designs. This is typical when printing to physical media. Pixels are suited for fixed-size elements, such as borders or margins, that need to be consistent across different screen sizes and resolutions.

Relative units are useful for creating responsive designs, that is, flexible designs that can adapt to different screen sizes and resolutions. This is usually the case for web content. Percentages achieve fluid layouts that adjust to the parent container's size. Similarly, viewport units are suited for creating responsive designs that adjust to the viewport size. For example, setting the height of a hero section to `100vh` will make it stretch to the height of the viewport.

EMs are good for font sizing and other proportional values that need to scale with the element's font size. REMs help implement consistent global styles across an entire website. With REMs, just by changing the root element's font size, the font sizes across the entire website can be changed. This implies that EMs and REMs result in more maintainable code. A consistent unit system will result in code that's more maintainable.

### Could you give accessibility guidelines for using CSS units?

Absolute units for font sizing can make it difficult for users with visual impairments. For some, the text may be too small. For others, it may be too large. Instead, use relative units. In the figure, leftmost view is least accessible. The centre view has more readable font size but line spacing is congested. Rightmost view is most accessible. It uses unitless line height value, which plays well with CSS inheritance.

For accessibility across different devices and platforms, create responsive designs using viewport units. Use media queries to adjust units for different devices. Test the code across different devices.

Avoid using small text sizes. The Web Content Accessibility Guidelines (WCAG) recommends a minimum font size of `16px`, or `1em` for body text. Be consistent in the use of CSS units. This leads to a harmonious and stable interface that's easier to use for users with cognitive or visual impairments.

### Which W3C standards describe CSS units?

CSS units are defined in the W3C CSS specification, which is maintained by the World Wide Web Consortium (W3C). The current version (Feb 2023) of the CSS specification is CSS3. Here are a few essential sections to read:

    + 2.1 Lengths: Absolute and relative length units in CSS, such as pixels (px), centimeters (cm), and ems (em).
    + 5.1 Percentage values: How percentage values are used to specify lengths and other values in relation to a parent element.
    + 5.2 Font-relative lengths: the em, ex, and ch units: Font-relative length units, such as ems and exes, which are based on the font size of an element.
    + 5.3 Viewport-percentage lengths: the vw, vh, vmin, and vmax units: Viewport-percentage length units, which are based on the size of the viewport.
    + 7.1.1 The 'font-size' property: How the font-size property can be used to specify the font size of text on a web page, using units such as pixels or ems.
    + 15 Media Queries: How media queries can be used to create responsive designs that adapt to different screen sizes and resolutions, including how different CSS units can be used to create flexible and adaptive layouts.

### What are some limitations of CSS units?

While CSS units offer a lot of flexibility in web design, there are some limitations to keep in mind:

    + **Rendering Inconsistency**: Different web browsers may render CSS units differently, which can lead to inconsistencies in the way the design appears to users.
    + **Old Browser Support**: Some of the newer CSS units may not be supported by older web browsers, which can limit their usefulness in some cases. Fallbacks or alternative approaches would be needed for older browsers.
    + **Limited Flexibility**: CSS units may be inadequate for special design requirements. For example, when dealing with extremely large or small measurements, or when creating complex layouts with multiple elements, other design approaches may be necessary.
    + **Inaccuracy**: While relative units can be useful for creating responsive designs, they can sometimes be inaccurate, especially when cascading multiple nested elements. This can make it difficult to maintain the intended design, especially on very complex layouts. This also means that pixel-perfect designs are harder to achieve with relative units. Slight variations may appear across different devices.

### Can you describe the units in CSS4?

CSS4 doesn't exist as a name. In fact, even CSS3 is really many smaller modules, each of which is standardized, versioned and published in its own document. As on February 2023, the following are some proposed units for the future:

    + fr unit: Represents a fraction of the available space in a grid or flexbox layout.
    + min(), max(), and clamp() functions: Set a minimum, maximum, or range of values for a property, based on the size of the container or viewport.
    + Modular Scale units: Set font sizes and other values based on a modular scale, which is a set of predefined values that follow a specific ratio.
    + Aspect Ratio units: Set the aspect ratio of an element based on the width and height of the element.
    + CSS Units for Scroll Snap: Set scroll snap properties, such as snap positions and distances, based on units such as pixels, percentages, or fractions.

### Can CSS units be used in SVG?

CSS units can be used in SVG (Scalable Vector Graphics) just as they can be used in HTML documents. For example, they can be used to set the size of an SVG shape, or to position it relative to other elements in the document. However, there are some limitations.

Some CSS units may behave differently in an SVG context compared to an HTML context. For example, the `vw` (viewport width) and `vh` (viewport height) units may not work as expected in SVG images that are embedded in a fixed-size HTML document, because the viewport size differs from the SVG image size.

CSS units that rely on the size of the font or the text, such as `em` or `rem`, may not be as useful in SVG images that don't contain text or use custom fonts. In this case, pixels or percentages may be more suited.

SVG-specific units, such as "pt" (points) and "mm" (millimeters), may also be used in SVG documents, but may not be as widely supported by all browsers and devices as the standard CSS units.

### What's the formula to convert from one CSS unit to another?

CSS Units Converter is an online tool to easily convert between different CSS units. For those who wish to know what happens under the hood, the following formulae may help.

| Conversion | Formula |
| --- | --- |
| Pixels to EMs | \(em = px / fontSize \) |
| Pixels to REMs | \(rem = px / rootFontSize \) |
| Pixels to % | \( \% = (px / (containerWidth \, or \, containerHeight)) \* 100% \) |
| EMs to Pixels | \( px = em \* fontSize \) |
| REMs to Pixels | \( px = rem \* rootFontSize \) |
| Viewport Units to Pixels | \( px = vw \* viewportWidth / 100 \\ px = vh \* viewportHeight / 100 \\ px = vmin \* min(viewportWidth, viewportHeight) / 100 \\ px = vmax \* max(viewportWidth, viewportHeight) / 100 \) |

### What's the browser and tool support for CSS units?

There's good browser support for CSS units. Some older browsers such as Internet Explorer 8 don't support REMs and viewport units. The Can I Use website provides information on browser support. Absolute units (in, cm, mm, pt, pc) are widely supported but their accuracy may depend on the screen or printer resolution.

Many design tools support the use of CSS units for designing web pages and interfaces: Adobe Photoshop, Sketch, Figma, Adobe XD, Canva, etc. Sketch supports pixels, points, and picas by default, but you can also use other units by installing plugins. WIth Figma and Adobe XD, the designer can switch units and the tool will automatically generate correct CSS code.

Popular code editors like Visual Studio Code and Sublime Text support the use of CSS units for web design.

Web designers and developers who wish to learn more can consult the Mozilla Developer Network CSS Units Guide. It provides an overview of each CSS unit and how to use them effectively in web development. Another resource is the CSS-Tricks Guide to Length Units in CSS. It provides an in-depth overview of each CSS unit, including examples and code snippets.

## Milestones

1996

These are the early days of the web. Designers and developers are looking for ways to define and control the layout and appearance of HTML documents. CSS units originate from here. Pixels (`px`) represent a fixed size on the screen. Points (`pt`) are adopted from print design. Percentages (`%`) are also part of the early CSS units.

1998

Relative unit EM (`em`) is introduced in CSS2. In time, they become a popular choice for creating flexible, responsive designs, along with percentages.

1999

W3C publishes the first drafts of CSS3. Among the new CSS units are viewport units (`vw`, `vh`, `vmin`, `vmax`), REMs (`rem`) and Chars (`ch`). Chars are a font-relative unit, based on the width of the character "0" in the current font. They're useful for creating layouts that are proportional to the size of the text.

CSS Units


# Natural Language Understanding

## Summary

Given some text, Natural Language Understanding (NLU) is about enabling computers to understand the meaning of the text. Once meaning is understood, along with the context, computers can interact with humans in a natural way.

If a human were to ask a computer a question, NLU attempts to understand the question. Such an understanding leads to a **semantic representation** of the input text. The representation is then fed into other related systems to generate a suitable response.

Language is what makes us human and manifests our intelligence. NLU is a challenging NLP task, often considered an AI-Hard problem. It combines elements of syntactic and semantic parsing, and predicate logic.

## Discussion

### How is NLU different from NLP?

Natural Language Processing (NLP) is an umbrella term that includes both Natural Language Understanding (NLU) and Natural Language Generation (NLG). NLP turns unstructured data into structured data. NLU is more specifically about the meaning or semantics. For example, if the user is asking about today's weather or the traffic conditions on a particular route, NLU helps in understanding the intent of the user's query. NLG is invoked when framing answers in natural language.

Voice-based human-computer interaction such as Apple Siri or Amazon Alexa is a typical example. Speech is converted to text using Automatic Speech Recognition (ASR). NLU then takes the text and outputs a semantic representation of the input. Once relevant facts are gathered, NLG helps in forming the answer. Text-to-Speech (TTS) synthesis finally converts the textual answer to speech.

Apart from sub-fields such as ASR and TTS, NLP consists of basic language processing tasks such as sentence segmentation, tokenization, handling stopwords, lemmatization, POS tagging and syntactic parsing.

There's also **Natural Language Inference (NLI)**. Given a premise, NLI attempts to infer if a hypothesis is true, false or indeterminate.

### What are the typical challenges in NLU?

Consider the sentence "We saw her duck". 'We' could be a Chinese name. 'Her' could refer to another person introduced earlier in the text. 'Duck' could refer to a bird or the action of ducking. Likewise, 'saw' could be a noun or a verb. This variety of interpretations is what makes NLU a challenging task. This is because language is highly **ambiguous**.

Ambiguity could be syntactic, such as "I saw the man with the binoculars". An example of word sense ambiguity is "I need to go to the bank".

**Synonymy** is also a problem for NLU. This is when many different sentences are expressing the same meaning. This is because language allows for variety and complex constructions.

Human often communicate with errors and less than perfect grammar. NLU systems have to account for these as well. In addition, human language has sarcasms. A sentence may have a literal meaning (semantics) but also a different intended meaning (pragmatics).

### Could you explain semantic parsing?

Semantic parsing translates text into a *formal meaning representation*. This representation is something that's easier for machines to process. In some ways, semantic parsing is similar to machine translation except that in the latter, the final representation is human readable.

The form of the representation depends on the purpose. It could use scalars or vectors. It could be continuous or discrete, such as tuples for relation extraction.

Consider the question "Which country had the highest carbon emissions last year?" Assuming the answer is to be searched in a relational database, the representation would take the form of a database query: `SELECT country.name FROM country, co2_emissions WHERE country.id = co2_emissions.country_id AND co2_emissions.year = 2014 ORDER BY co2_emissions.volume DESC LIMIT 1`.

In a robotic application, the representation might be a sequence of steps to guide the robot from one place to another. In smartphones that process voice commands, the representation might be categorized into intents and their arguments.

### Which are the typical NLU tasks?

Since NLU's focus is on meaning, here are some typical NLU tasks:

    + **Sentiment Analysis**: Understand if the text is expressing positive or negative sentiment. Emotion detection is a more granular form of sentiment analysis.
    + **Named Entity Recognition**: Identify and classify named entities (Person, Organization, Location, etc.) in the text.
    + **Relation Extraction**: Identify and classify the relation between named entities.
    + **Semantic Role Labelling**: Identify and label parts a sentence with their semantic roles. This helps in answering questions of the type "who did what to whom".
    + **Word Sense Disambiguation**: Looking at the context of word usage, figure out the correct sense since a word can have multiple senses.
    + **Inductive Reasoning**: Given some facts, use logic to infer relations not stated explicitly in the text.When combined with NLG, other tasks that require NLU are question answering, text summarization, chatbots, and voice assistants. The use of NLU in chatbots and voice assistants has become increasingly more important. NLU helps chatbots to better understand user intent, and to respond correctly and in a more natural way.

### Could you share examples of real-world NLU systems?

Many examples are in relation to chatbots or voice assistants. Microsoft offers Language Understanding Intelligent Service (LUIS) that developers can use to quickly build natural language interfaces into their apps, bots and IoT devices.

A similar offering from IBM is Watson Assistant. IBM also offers Watson Natural Language Understanding to extract entities, keywords, categories, sentiment, emotion, relations, and syntax.

From Google, we have Dialogflow for voice and text-based conversational interfaces. Other examples are Amazon Lex, SAP Conversational AI, , Rasa NLU, and Snips.

### Which are the main approaches to NLU?

One approach is to initialize an NLU system with some knowledge, structure and common sense. The system then learns from experience via **reinforcement learning**. Since some see this as introducing biases, an alternative approach is to require the system to learn everything by itself. Just as humans learn by interacting with their environments, NLU systems can also benefit from such **embodied learning**. The ability to detect human emotions can lead to deeper understanding of language.

These two approaches are mirrored in Western philosophy by **nativism** (core inbuilt knowledge) and **empiricism** (learned by experience).

NLU systems could combine elements of statistical approaches with knowledge resources such as FrameNet or Wikidata. FrameNet is a lexical database of word senses with examples. FrameNet can therefore help NLU in obtaining common sense knowledge. Other common sense datasets include Event2Mind and SWAG.

Su et al. noted the duality between NLU and NLG. Via **dual supervised learning** they trained a model to jointly optimize on both tasks. Their approach gave state-of-the-art F1 score for NLU.

There are four categories of NLU systems: distributional, frame-based, model-theoretical, interactive learning.

### Which are the common benchmarks for evaluating NLU systems?

The General Language Understanding Evaluation (GLUE) benchmark has nine sentence or sentence-pair NLU tasks. It has good diversity of genres, linguistic variations and difficulty. It's also model agnostic. There's also a leaderboard. This was extended in 2019 to **Super GLUE**.

Quora Question Pairs (QQP) has question pairs. The task is to determine if two questions mean the same. Sharma et al. showed that a Continuous Bag-of-Words neural network model gave best performance. Incidentally, QQP is included in GLUE.

**SentEval** is a toolkit for evaluating the quality of universal sentence representations. The tasks include sentiment analysis, semantic similarity, paraphrase detection, entailment, and more.

**CLUTRR** is a diagnostic benchmark suite for inductive reasoning. It was created to evaluate if NLU systems can generalize in a systematic and robust way.

For evaluating chatbots, Snips released three benchmarks for built-in intents and custom intent engines.

Most models are trained to exploit statistical patterns rather than learn the meaning. Hence, it's easy for someone to construct examples to expose how poorly a model performs. Inspired by this, **Adversarial NLI** is another benchmark dataset that was produced by having humans in the training loop.

### Are current NLU systems capable of real understanding?

Back in 2019, it was reported that NLU systems are doing little more than pattern matching. There's no real understanding in terms of agents, objects, settings, relations, goals, beliefs, etc.

OpenAI's GPT-2 was trained on 40GB of data with no prior knowledge. When prompted with a few words, GPT-2 can complete the sentence sensibly. But despite its fluency, GPT-2 doesn't understand what it's talking about. It fails at answering simple questions. In other words, it's good at NLG but not at NLU.

Language Models (LMs) have proven themselves in many NLP tasks. However, their success in reasoning has shown to be poor or context-dependent. LMs capture statistics of the language rather than reasoning. In other words, prediction does not imply or equate to understanding.

The failure to "understand" could be due to lack of **grounding**. For example, dictionaries define words in terms of other words, which too are defined by other words. Real understanding can come only when words are associated with sensory experiences grounded in the real world. Without grounding, NLU systems are simply mapping a set of symbols to another set or representation.

## Milestones

1966

Joseph Weizenbaum at MIT creates *ELIZA*, a program that takes inputs and responds in the manner of a psychotherapist. ELIZA has no access to knowledge databases. It only looks at keywords, does pattern matching and gives sensible responses. Many users get fooled by ELIZA's human-like behaviour, although Weizenbaum insists that ELIZA has no understanding of either language or the situation.

1971

Terry Winograd at MIT describes *SHRDLU* in his PhD thesis. The task is to guide a robotic arm to move children's blocks. SHRDLU can understand block types, colours, sizes, verbs describing movements, and so on. In later years, SHRDLU is considered a successful AI system. However, attempts to apply it to complex real-world environments prove disappointing. A modern variation of SHRDLU is *SHRDLURN* due to Wang et al. (2016).

Jun
1974

Marvin Minsky at MIT publishes *A Framework for Representing Knowledge*. He defines a **frame** as a structure that's represents a stereotyped situation. Given a situation, a suitable frame is selected along with its associated information. Then it's customized to fit the current situation. This is the frame-based approach to NLU.

1980

Pereira and Warren develop *CHAT-80*, a natural language interface to databases. Implemented in Prolog, it uses hand-built lexicon and grammar. It can answer questions about geography, such as "What countries border Denmark?"

1984

Apple releases the Macintosh 128K along with a computer mouse. Although mouse was invented 20 years earlier, it's the Macintosh that makes it popular, and with it the **Graphical User Interface (GUI)**. This causes some companies to change focus from research into natural language interfaces to adoption of GUIs.

1995

Kuhn proposes **Semantic Classification Tree (SCT)** that automatically learns semantic rules from training data. This overcomes the need to hand code and debug large number of rules. The learned rules are seen to be robust to grammatical and lexical errors in input. In general, the 1990s see a growing use of statistical approaches to NLU.

Jul
2018

Gobbi et al. compare many different algorithms used for concept tagging, a sub-task of NLU. Among the algorithms compared are generative (WFST), discriminative (SVM, CRF) and neural networks (RNN, LSTM, GRU, CNN, attention). LSTM-CRF models show best performance. Adding a CRF top layer to a neural network improves performance with only a modest increase in number of parameters.

Sep
2019

Reasoning is one of the tasks of NLU with practical use in applications such as question answering, reading comprehension, and textual entailment. In a graph-based approach, Khashabi et al. show the impossibility of reasoning in a noisy linguistic graph if it requires many hops in the meaning graph. Meaning space is internal conceptualization in the human mind. It's free of noise and uncertainty. Linguistic space is where thought is expressed via language and has plenty of room for imperfections.

Natural Language Understanding


# Introspection in Python

## Summary

**Introspection** is about getting information about an object at runtime. This information could be about the object's type, its data attributes and methods, its class inheritance hierarchy, documentation, method signature, and more. Introspection can also be about knowing the current runtime context such as stack information.

Introspection is an additional language feature in the programmer's toolbox. It can help the programmer code for specific scenarios that are perhaps harder to solve otherwise. Tracing and debugging tools benefit from introspection. In Python, data attributes and methods can be added or removed from instances even after instantiation. Given this dynamic nature of objects, introspection becomes a useful feature.

## Discussion

### What built-in functions facilitate object introspection in Python?

The following built-in functions are useful:

    + `callable()`: Returns True if object is callable. Eg. `callable(mycar)`. Objects can be made callable by defining `__call__` method in their class.
    + `dir()`: Obtain all attributes (data and methods) of an object. Eg. `dir(mycar)`.
    + `hasattr()`: Check if object has an attribute. Eg. `hasattr(mycar, 'num_gears')`. This is useful since attributes can be dynamically added/removed at runtime. Note that `if 'num_gears' in dir(mycar)` is equivalent to `hasattr(mycar, 'num_gears')`.
    + `help()`: Obtain help on an object's interface based on docstrings.
    + `id()`: Obtain the unique ID of the object. Eg. `id(mycar)`.
    + `issubclass()`: Check if a class is a subclass of a specified class. Eg. `issubclass(Honda, Car)`.
    + `isinstance()`: Check if an object is of a specified class. Eg. `isinstance(mycar, Honda)` will be True when `mycar = Honda()`. If Honda is a subclass of Car, `isinstance(mycar, Car)` is also True. Note that `type(mycar) is Car` is False.
    + `sys.getsizeof()`: Part of the `sys` module, this obtains the size of the object in bytes.
    + `type()`: Obtain the object's type. Eg. `type(2)` will give `<class 'int'>`.
    + `vars()`: Obtain all instance variables (name and value) as a dictionary. Eg. `vars(mycar)`. This is equivalent to `mycar.__dict__`.

### What object attributes help with introspection in Python?

There are some attributes that give useful information about objects. For example, given an instance `mycar`, `mycar.__class__` retrieves its class. Attributes of the class give more information: `__name__`, `__qualname__`, `__str__`, `__repr__`, `__doc__`, and `__self__`. We briefly describe these:

    + `__name__`: Original name of the class, function or method.
    + `__qualname__`: Qualified name of the class function or method. This is often useful where class/function/method definitions are nested.
    + `__doc__`: Documentation string, which can also be retrieved by calling built-in function `help()`.
    + `__self__`: Instance to which a method is bound.

### Could you describe Python's inspect module?

Python module `inspect` provides many functions to enable introspection. The module documentation notes that it provides four main kinds of services: "type checking, getting source code, inspecting classes and functions, and examining the interpreter stack."

Inspect documentation has full details. We note some functions:

    + **Checking types**: `ismodule()`, `isclass()`, `ismethod()`, `isfunction()`, `isgenerator()`, `isabstract()`, `isbuiltin()`, `isawaitable()`, and many more.
    + **Inspecting objects**: `getmembers()` is quite useful. For example, `inspect.getmembers(mycar, inspect.ismethod)` will return only methods of the instance `mycar`. Function `getfullargspec()` returns names and default values of a function. These can be nicely formatted using `formatargspec()`. Callables can be inspected using `signature()`. Class definitions relevant here are `Signature`, `Parameter` and `BoundArguments`.
    + **Retrieving source code**: `getdoc()`, `getcomments()`, `getfile()`, `getmodule()`, `getsourcefile()`, `getsourcelines()`, `getsource()`, and `cleandoc()`.
    + **Examining the stack**: `getframeinfo()`, `getouterframes()`, `getinnerframes()`, `currentframe()`, `stack()` and `trace()`.

### How can I obtain the current runtime context in Python?

The Python interpreter maintains a **stack of frames**. The bottom-most frame is called the global frame or module frame. When a function is called, a new frame is created and pushed to the stack. When that function returns, the frame is popped off the stack.

Python's `inspect` module provides many functions to inspect the interpreter stack. Function `currentframe()` returns a frame object representing the current frame. More information about the frame object can be obtained by calling `getframeinfo()`, which returns a named tuple `Traceback(filename, lineno, function, code_context, index)`.

To obtain the entire stack, call `stack()` instead. To inspect the call graph, we can use `getouterframes()`. When an exception is raised and later handled in an outer frame, we can use `trace()` or `getinnerframes()`. These can be useful for logging and debugging.

Apart from inspecting the stack, two built-in functions help us inspect variable scope. These are `globals()` and `locals()`. At the module level, both functions return the same dictionary.

### How can I inspect class hierarchy in Python?

Class hierarchy can easily be inspected by accessing the class attribute `__bases__` and recursively printing the class names up the hierarchy. If an instance is being inspected, we can first get to its class via the `__class__` attribute and then process the `__bases__` attribute. In the figure, we show an example of how this can be done.

The method `inspect.getclasstree()` from `inspect` module is an alternative. Given a list of classes, it returns the class hierarchy. For every class, its base classes are returned as a tuple.

Another useful method is `inspect.getmro()` to which a class name must be passed as an argument. This is equivalent to calling the built-in class method `mro()`. Either way, these don't return the inheritance hierarchy but they do tell us the order in which methods are looked up the hierarchy.

### What are some specialized modules for Python introspection?

Python processes source code in three steps: (i) parsing, where it tokenizes the source code and produces an Abstract Syntax Tree (AST); (ii) compiling, where it transforms the AST into Python bytecode; (iii) interpreting, where the Python Virtual Machine (VM) interprets and executes the bytecode. In this context, there are two Python modules that help with introspection: `ast` to inspect the AST, and `dis` to inspect the bytecode.

While `inspect` module has functions to inspect the call stack, `sys` module also has functions that can help with debugging recursive calls: `_getframe()`, `getrecursionlimit()` and `setrecursionlimit()`.

Support we decorate the function `foo()`. Accessing `foo.__name__` will return the name of the function returned by the decorator. This is probably not what we want. We can solve this with the `wraps()` function from the `functools` module. The figure shows an example how to use this. An alternative is to use `inspect.signature()` function.

### Could you share some tips on using Python's introspection features?

While `type()` and `isinstance()` could be useful, code that rely on them are hard to maintain. We might be checking for specific types but when another type must be handled, we need to update the code. Instead, Python programmers prefer the approach of *duck typing*.

With duck typing, we don't care about an object's type. Instead, we focus on the interfaces the object exposes. In this approach, `hasattr()` can be useful though this method can cause problems in Python 2 when called on a property attribute.

An alternative to `hasattr()` is to simply call a method or access a data attribute without doing any checks. If it doesn't exist, we should handle the exception. Another alternative is to use an Abstract Base Class (ABC) from the `collections.abc` module.

## Milestones

Jan
1994

Python 1.0 is released.

Apr
2001

Python 2.1 is released. This release introduces the `inspect` module.

Sep
2012

Python 3.3 is released. Via PEP 3155, this release adds **qualified name** attribute `__qualname__` to classes and functions. This enhances introspection for nested classes and functions. Via PEP 362, function `inspect.signature()` is introduced for **inspecting callables** including decorated functions, classes and partial function objects. In January 2013, PEP 362 is backported by community developers to older versions of Python, in a package named `funcsigs`.

Sep
2015

Python 3.5 is released. Via PEP 492, Python gets better support for **asynchronous programming**. As part of this enhancement, `inspect` module includes many functions to inspect coroutine functions and coroutine objects. The module's functions `stack()`, `trace()`, `getouterframes()`, and `getinnerframes()` now return named tuples. `Signature` and `Parameter` objects can be pickled and hashed.

Introspection in Python


# Python Data Structures

## Summary

Among the basic data types and structures in Python are the following:

* Logical: `bool`
* Numeric: `int`, `float`, `complex`
* Sequence: `list`, `tuple`, `range`
* Text Sequence: `str`
* Binary Sequence: `bytes`, `bytearray`, `memoryview`
* Map: `dict`
* Set: `set`, `frozenset`

All of the above are classes from which object instances can be created. In addition to the above, more data types/structures are available in modules that come as part of any default Python installation: `collections`, `heapq`, `array`, `enum`, etc. Extra numeric types are available from modules `numbers`, `decimals` and `fractions`. The built-in function `type()` allows us to obtain the type of any object.

## Discussion

### With respect to data types, what are the differences between Python2 and Python3?

The following are important differences:

    + A division such as `5 / 2` returns integer value 2 in Python2 due to truncation. In Python3, this will evaluate to float value 2.5 even when the input values are only integers.
    + In Python2, strings were ASCII. To use Unicode, one had to use the `unicode` type by creating them with a prefix: `name = u'Saṃsāra'`. In Python3, `str` type is Unicode by default.
    + Python2 has `int` and `long` types but both these are integrated in Python3 as `int`. Integers can be as large as system memory allows.

### What data structures in Python are immutable and mutable?

*Mutable* objects are those that can be changed after they are created, such as updating/adding/removing an element in a `list`. It can be said that mutable objects are changed in place.

*Immutable* objects can't be changed in place after they are created. Among the immutable basic data types/structures are `bool`, `int`, `float`, `complex`, `str`, `tuple`, `range`, `frozenset`, and `bytes`.

The mutable counterparts of `frozenset` and `bytes` are `set` and `bytearray` respectively. Among the other mutable data structures are `list` and `dict`.

With immutable objects it may seem like we can modify their values by assignment. What actually happens is that a new immutable object is created and then assigned to the existing variable. This can be verified by checking the ID (using `id()` function) of the variable before and after assignment.

### What data structures in Python are suited to handle binary data?

The core built-in types for manipulating binary data are `bytes` and `bytearray`. They are supported by `memoryview`, which uses the buffer protocol to access the memory of other binary objects without needing to make a copy.

The `array` module supports efficient storage of basic data types like 32-bit integers and IEEE754 double-precision floating values. Characters, integers and floats can be stored `array` types, which gives low-level access to the bytes that store the data.

### What containers and sequences are available in Python?

*Containers* are data structures that contain one or more objects. In Python, a container object can contain objects of different types. For that matter, a container can contain other containers at any depth. Containers may also be called *collections*.

Sequences are containers that have inherent ordering among their items. For example, a string such as `str = "hello world"` is a sequence of Unicode characters h, e, l, etc. Note that there is no character data type in Python, and the expression "h" is actually a 1-character string.

Sequences support two main operations (eg. sequence variable `seq`):

    + **Indexing**: Access a particular element: `seq[0]` (first element), `seq[-1]` (last element).
    + **Slicing**: Access a subset of elements with syntax seq[start:stop:step]: `seq[0::2]` (alternate elements), `seq[0:3]` (first three elements), `seq[-3:]` (last three elements). Note that the stop point is not included in the result.Among the basic sequence types are `list`, `tuple`, `range`, `str`, `bytes` `bytearray` and `memoryview`. Conversely, `dict`, `set` and `frozenset` are simply containers in which elements don't have any particular order. More containers are part of `collections` module.

### How can I construct some common containers?

The following examples are self-explanatory:

    + str: `a = ''` (empty), `a = ""` (empty), `a = 'Hello'`
    + bytes: `a = b''` (empty), `a = b""` (empty), `a = b'Hello'`
    + list: `a = list()` (empty), `a = []` (empty), `a = [1, 2, 3]`
    + tuple: `a = tuple()` (empty), `a = (1,)` (single item), `a = (1, 2, 3)`, `a = 1, 2, 3`
    + set: `a = set()` (empty), `a = {1, 2, 3}`
    + dict: `a = dict()` (empty), `a = {}` (empty), `a = {1:2, 2:4, 3:9}`We can construct `bytearray` from `bytes` and `frozenset` from `set` using their respective built-in functions.

### What are iterables and iterators?

An *iterable* is a container that can be processed element by element. For sequences, elements are processed in the order they are stored. For non-sequences, elements are processed in some arbitrary order.

Formally, any object that implements the *iterator protocol* is an iterable. The iterator protocol is defined by two special methods, `__iter__()` and `__next__()`. Calling `iter()` on an iterable returns what is called an *iterator*. Calling `next()` on an iterator gives us the next element of the iterable. Thus, iterators help us process the iterable element by element.

When we use loops or comprehensions in Python, iterators are used under the hood. Programmers don't need to call `iter()` or `next()` explicitly.

### Can I convert from one data type to another?

Yes, provided they are compatible. Here are some examples:

    + `int('3')` will convert from string to integer
    + `int(3.4)` will truncate float to integer
    + `bool(0)` and `bool([])` will both return `False`
    + `ord('A')` will return the equivalent Unicode code point as an integer value
    + `chr(65)` will return the equivalent Unicode string of one character
    + `bin(100)`, `oct(100)` and `hex(100)` will return string representations in their respective bases
    + `int('45', 16)` and `int('0x45', 16)` will convert from hexadecimal to decimal
    + `tuple([1, 2, 3])` will convert from list to tuple
    + `list('hello')` will split the string into a list of 1-character strings
    + `set([1, 1, 2, 3])` will remove duplicates in the list to give a set
    + `dict([(1,2), (2,4), (3,9)])` will construct a dictionary from the given list of tuples
    + `list({1:2, 2:4, 3:9})` will return a list based on the dictionary keys.

### Should I use a list or a tuple?

Tuples are used to pass arguments and return results from functions. This is because they can contain multiple elements and are immutable. Tuples are also good for storing closely related data. For example, (x, y, z) coordinates or (r, g, b) colour components can be stored as tuples. Use lists instead if values can change during the lifetime of the object.

Although lists can contain heterogeneous items, tuples are more commonly used for this purpose. Tuples group together items that belong together even if their types are different. A database record containing a student's details can be stored in a tuple.

If a sequence is to be sorted, use a list for in-place sorting. A tuple can be used but it should return a new sorted object. A tuple cannot be sorted in-place.

For better code readability, elements of a tuple can be named. For this purpose, use `collections.namedtuple` class. This allows us to access the elements via their names rather than tuple indices.

It's possible to convert between lists and tuples using functions `list()` and `tuple()`.

### When should I use set and dict?

Sets and dictionaries have no order. However, from Python 3.7, the order in which items are inserted into a dict is preserved. From Python 3.8, we can iterate through a dict in reverse order.

Sets store unique items. Duplicates are discarded. Dictionaries can contain duplicate values but keys must be unique. Since dict keys are unique, often dict is used for counting. For example, to count the number of occurrences of each word in a document, words can be keys and counts can be values.

Sets are suited for finding the intersection/union of two groups, such as finding those who live in a neighbourhood (set 1) and/or also own a car (set 2). Other set operations are also possible.

Strings, lists and tuples can take only integers as indices due to their ordered nature but dictionaries can be indexed by strings as well. In general, dictionaries can be indexed by any of the built-in immutable types, which are considered *hashable*. Thus, dictionaries are suited for key-value pairs such as mapping country names (keys) to their capitals (values). But if capitals are the more common input to your algorithm, use them as keys instead.

### How can I implement a linked list in Python?

*Linked list* is a collection of nodes connected by links or pointers. A *node* is single data point in the linked list. It not only holds the data, but also has a pointer to the next node in a single-linked list. Thus, the definition of a node is recursive. For a double-linked list, the node holds two pointers, one to the previous node and one to the next node. A linked list can be designed to be ordered or unordered.

The head of the linked list must be accessible. This allows us to traverse the entire list and perform all possible operations. A double-linked list might also expose the tail for traversal from the end.

While a `Node` class may be enough to implement a linked list, it's common to encapsulate the head pointer and all operations within `LinkedList` class. Operations on the linked lists are methods of the class. One possible implementation is given by Downey. A `DoubleLinkedList` can be a derived class from `LinkedList` with the addition of a tail pointer and associated methods.

## Milestones

2001

Iterators are introduced into the language for looping through containers.

2001

Types `int` and `long` are unified into a single type.

2001

Division operator is changed to have three variants: classic division (truncation), true division, floor division (using // operator).

2009

Python's creator Guido van Rossum relates the early history of Python with respect to numbers. Implementation used machine integers and machine binary floating point. He explains that `int` were treated as signed normally but were unsigned in bitwise operations; but `long` were always signed and this caused problems. The `int` type could also overflow and raise an exception. Today integers are unbounded.

Jun
2018

Python data model is changed such that the insertion order of items into a `dict` is preserved. In October 2019, with the release of Python 3.8, `dict` can be iterated in reverse using built-in function `reversed()`.

Python Data Structures