# Xamarin Forms

## Summary

In traditional app development process, developers had to write code for iOS, Android and Windows platform separately. They had to maintain the codebase of each platform for the maintenance and upgradation process. It was a heavy time and money consuming task. Xamarin solved this problem to a great extent by providing the facility of backend code sharing across the platforms. It allowed developers to create the business logic once and share it across the iOS, Android and Windows platform. Eventually, Xamarin.Forms was developed to allow the developers to share the code of user interface of app across the platforms and make the development process easier.

Xamarin.Forms is an open source cross-platform UI development framework. It allows developers to build Android, iOS and Windows application from a single shared codebase. It supports C# language. Visual Studio is the recommended IDE for Xamarin.Forms.

## Discussion

### How does Xamarin.Forms work?

A Xamarin.Forms application is designed similarly as a traditional cross-platform application. Shared code is usually placed in a .NET Standard library and platform-specific applications consume the shared code.

Xamarin.Forms applications have a single class named App that instantiates the application on each platform. The *AppShell* class defines the visual hierarchy of the application. Shell uses this visual hierarchy to produce the user interface of the app.

At runtime, Xamarin.Forms uses platform renderers to convert the cross-platform UI elements into native controls on Android, iOS and Windows devices. It enables the developer to give the native look, feel and user experience to the application while using the feature of code-sharing across platforms.

### What are the benefits of Xamarin.Forms?

It allows developers to create native UI layout and designs across platforms using a single shared codebase which reduces the time and money deployed in the development process . It allows developers to share code, test and business logic of app across multiple platforms. It reduces the overhead of managing different codebases for app due to separate code for different platforms.

It's been claimed that while Xamarin enables 70% code sharing, Xamarin.Forms allows up to 95% code sharing. It's best for apps where code sharing is more important than custom UI. One team reported that with Xamarin.Forms they had only 135 lines of platform-specific code out of 3744 lines (96% reuse). Earlier with Xamarin the team had achieved 80% code reuse.

PGS Software implemented European airline Volotea's mobile app in Xamarin.Forms. With Forms, consistent branding and styling were achieved across iOS and Android. With Forms, 70% of custom renderers were removed.

### What platforms are supported in Xamarin Forms?

Xamarin.Forms is used for making mobile apps (iOS or Android) and Universal Windows Platform (UWP) apps. UWP enables creation of apps that can run on any device with Windows 10 or 11. Visual Studio is the recommended IDE for it. For iOS, Xamarin.Forms applications can be written for iOS 9 or higher versions. For Android, apps can be written for Android 4.4 (API 19) or higher but Android 5.0 (API 21) is recommended as minimum API for full compatibility with all the Android supported libraries. For Windows platform, Windows 10 Universal Windows Platform, build 10.0.16299.0 or greater for .NET Standard 2.0 support. However, build 10.0.18362.0 or greater is recommended.

Apart from iOS, Android and Windows Xamarin.Forms supports platforms including Samsung Tizen, macOS 10.13 or higher, GTK# and WPF. Windows 8.1 / Windows Phone 8.1 WinRT and Windows Phone 8 Silverlight are deprecated platforms for Xamarin.Forms version 3.0 or newer.

To Develop for iOS platform on Windows computer, there must be a Mac computer accessible on the network because Apple's build tools are required to build native iOS apps, which run on Mac only. So, Visual Studio should be connected to a network-accessible Mac.

### What are some essential terms used in Xamarin.Forms?

Below are some essential terms used in Xamarin.Forms-

    + **Views**: Views are the basic building blocks of the user interface. Labels, buttons, and sliders are different types of views. Different views serve different purposes such as setting values and editing text.
    + **Layouts**: Layouts are the visual structure of the user interface. Layouts are containers for views and other layouts. Layouts and views derive from the *View* class. Layouts are also views containing a collection of child views.
    + **Effects**: Effects are used to customize the controls accordingly without implementing custom Renderer classes. They should be used when a small styling change is required. While, Custom renderers are better option for creating a new control.
    + **Gestures**: Gesture recognizers are used to detect user interaction with views in a Xamarin.Forms application. The Xamarin.Forms GestureRecognizer class supports tap, pinch, pan, swipe, and drag and drop gestures on View instances of the app.
    + **Triggers**: Triggers are used to define the actions in XAML after a certain events or property changes. State triggers are a specialized group of triggers that define when a VisualState should be applied.

### What are the key building blocks of Xamarin.Forms?

The key building blocks of a Xamarin.Forms application are as follows:

    + **Xamarin.Essentials**: Essentials provides a single cross-platform API that can work with any Xamarin.Forms, Android, iOS, or UWP application accessible from shared code regardless of how the user interface is created.
    + **eXtensible Application Markup Language (XAML)**: XAML is used for instantiating and initializing objects and organizing those objects in a hierarchical way. In XAML, developers can use all the Xamarin.Forms views, layouts, pages as well as custom classes to create the UI. It is very brief and readable than the equivalent code.
    + **Behaviors**: Behaviors allows to add functionality to the user interface controls without extending to any other class. The functionality is already implemented in the behavior class so developers only need to attach it to the controls. They are created by deriving from the Behavior or Behavior<T> class.
    + **Data binding**: It is used to keep the objects synchronized so that changes in one object's property are automatically reflected in other related object's property. It is an important part of the Model-View-ViewModel (MVVM) application architecture.

### What are the code-sharing techniques in Xamarin Forms?

The.NET Standard Libraries and Shared Projects are two main methods of code-sharing. Developers can use a large number of .NET APIs to share the code across multiple platforms. .NET Standard 1.0 - 1.6 provides larger sets of APIs, whereas .NET Standard 2.0 provides the best coverage of the .NET Base Class Libraries (including the .NET APIs available in Xamarin apps). . In the shared projects method, a project is shared containing all the code files and assets of the app. Separate application projects for iOS, Android and Windows are shared inside the shared project. The common code for all the platforms also resides in the shared project.

Portable Class Libraries allow developers to choose the combination of platforms for their project. However, Portable Class Libraries are deprecated in the latest version of Visual Studio, and .NET Standard Libraries are recommended instead.

### What are platform-specific features and how to create them?

A platform may have features and functionalities that doesn't exist on other platforms. For instance, Android has a functionality of fast scrolling in a list view but iOS doesn't have the same. Xamarin.Forms platform-specifics allow developers to utilize such platform-specific functionality that is only available on a specific platform without creating custom renderers or effects. There are some built-in platform-specifics in Xamarin.Forms, developers can use them or create the platform-specifics by themselves. Below are the steps for creating a platform-specific -

    + Implement the specific functionality as an Effect.
    + Create a platform-specific class that will use the Effect.
    + Make an attached property in the platform-specific class to allow the platform-specific to be used through XAML.
    + Implement extension methods in the platform-specific class, to allow the platform-specific to be used through a fluent code API.
    + Set the effect implementation to be applied only when the platform-specific has been invoked on the same platform as the Effect.Using an Effect as a platform-specific makes the effect easily consumable through XAML and a fluent code API.

### What are the shortcomings of Xamarin forms?

Though Xamarin.Forms can create a native UI for different platforms but it is not the best choice for creating native UI of complicated apps or apps that use more native functionalities such as camera, bluetooth.  Also, the file size of Xamarin.Forms app is comparatively greater than the native ones. It is not a good choice for building games.

### As a beginner, how do I get started with Xamarin.Forms?

Xamarin.Forms has a complete documentation and code samples. There are many resources to learn Xamarin.Forms provided by the Microsoft, including:

    + Complete official documentation
    + Xamarin.Forms book
    + Xamarin.Forms tutorials
    + Code samples
    + Question/Answer communityA beginner should use Xamarin.Forms Shell for the mobile app development. Xamarin.Forms Shell provides a common navigation user experience, a URI-based navigation scheme, and an integrated search handler. Thus, it makes the app development process easier.

## Milestones

2011

Engineers who had created Mono, launch a new company named **Xamarin**. Mono itself was an open source project started in 2001 to bring .NET applications to Linux.

2014

Xamarin.Forms is created, enabling a single user interface to be used for different platforms. **Xamarin.Forms 1.0.6197** is released as a bug fix release.

Apr
2018

**Xamarin.Forms 2.5.1** is released. Multiple issues such as BindingContext object not being passed through CommandParameter and app crash while navigating to a TabbedPage are solved.

May
2018

**Xamarin.Forms 3.0.0** is released. Flex layout is released. It's optimized for UI to adapt to various screen sizes and dimensions. Support for right to left languages is released. Developers can now create and customize Styles using CSS. VisualStateManager can now be used to handle different states of controls.

May
2020

At the BUILD 2020 conference, Microsoft announces **.NET Multi-platform App UI (MAUI)**. This is meant as an evolution of Xamarin.Forms. Xamarin.Forms 5.0 will be the last major release before .NET MAUI.

Nov
2020

**Xamarin.Forms 4.5.0.725** (4.5.0 Service Release 6) is released. Xamarin.Forms now uses the latest AndroidX libraries from Google instead of Android Support libraries.

Jul
2021

**Xamarin.Forms 5.0.0.2337** (5.0.0 Service Release 9) is released. The release includes several new controls and features including App Themes (Dark Mode), Brushes, CarouselView, RadioButton, Shapes and Paths, and SwipeView. Visual Studio 2017 is no longer supported.

Xamarin Forms


# Three Rs of Security

## Summary

Security is an important concern in all computer systems. This is more so with complex enterprise systems deployed on cloud infrastructure. An application composed of dozens of microservices running hundreds of containers across multiple nodes is hard to secure without good guiding principles.

The methodology of three Rs—**Rotate**, **Repave** and **Repair**—offers a simple approach towards greater security of your cloud deployments. The basic idea is to be *proactive* than be *reactive* as seen in traditional enterprise security. Speed is of essence. The longer a deployment stays in a given configuration, the greater is the opportunity for threats to exploit any vulnerabilities.

## Discussion

### What problem does the three Rs model solve?

With DevOps, it's become possible to deliver software faster. Yet security is one aspect that's been slow to change. This is often seen in firewall rules, long-lived credentials, and hard-to-update databases. The choice is between moving quickly and accept the associated risks; or moving slowly to mitigate risk. Often, enterprises choose to move slow.

In 2016, the U.S. Intelligence identified cyberthreats as a greater threat than terrorism or traditional weapons of mass destruction. Enterprises are spending a lot on securing endpoints and networks. Yet, Gartner's research shows that 70% of vulnerabilities are in the application layer.

The basic premise of the three Rs model is that the more time you give to attacks, the more opportunity they get to cause some serious damage. So it's best to embrace change and move quickly. It's been said that,

> To get safer, you have to go faster, and that is the exact opposite of how organizations work today.

### Could you explain each of the three Rs?

The three Rs of security are the following:

    + **Rotate**: Rotate datacenter credentials every few minutes or hours. Credentials could be passwords, certificates, access tokens, etc.
    + **Repave**: Repave every server and application in the datacenter every few hours from a known good state. Rather than patch a particular software, patch the whole stack. Destroy old VMs and containers and recreate from a known good state. Use rolling deployments to minimize downtime.
    + **Repair**: Repair vulnerable operating systems and application stacks consistently within hours of patch availability.The idea is to give attacks less time to snoop around and learn about your system. It's feasible to adopt the three Rs today due to cloud-native architectures and continuous deployments. In fact, it may no longer be a badge of honour to claim that your server has been running for years without a reboot. From a security perspective, it's better to limit the maximum uptime of a server.

### How are the three Rs of security related to DevOps?

DevOps is a combination of people, processes and tools that enables teams to deliver software faster. Teams don't work in silos. A DevOps workflow that includes security is what we call **DevSecOps**. DevSecOps attempts to include security concerns at all stages of a development workflow.

The three Rs model may be seen as a subset of DevSecOps. Security at the application layer can never be perfect. The solution is to mitigate the risks during operations. DevOps already includes a wealth of tools and automation. These can be used to implement the three Rs model.

## Milestones

Apr
2016

Justin Smith, a security expert at Pivotal, blogs about the three Rs model. In May, he presents the model at the Cloud Foundry Summit in Silicon Valley. At about the same time, a news article on The New Stack names the three Rs as Cloud Foundry's security strategy.

Nov
2017

The three Rs model makes it to the *Technology Radar* of ThoughtWorks. They note that this is feasible because of cloud-native architectures. Apps must be designed to be resilient to failures.

Mar
2018

The use of the three Rs model at the network edges, including embedded or IoT devices, is discussed. Many devices are still being deployed with default passwords. Rotating credentials at the edge is a challenge.

Jun
2018

The three Rs model is presented at Devoxx Poland conference. This indicates a growing awareness of the model within the developer community.

Three Rs of Security


# Speech Synthesis Markup Language

## Summary

Speech synthesis is the process of producing natural-sounding human speech from text so that humans can interact with machines via voice interfaces. Typical applications are reading for the blind, speaking aids for the handicapped, remote access to email, proofreading and so on. Speech synthesis system consists of analysis of input text, followed by synthesis of speech.

There are inherent difficulties in these systems. They don't handle symbols or foreign words suitably. Some systems translate leading whitespaces into extra pauses. Some words may need extra stress or change of pitch. It's for these purposes that **Speech Synthesis Markup Language (SSML)** becomes useful.

SSML adds markup on input text to aid speech synthesizers construct speech waveforms that sound more natural. SSML is a W3C standard, though some implementations have proprietary extensions. Popular voice assistants (Alexa, Assistant, Cortana) are known to use SSML.

## Discussion

### Which are the main features of SSML?

SSML has many useful features to make synthesized speech sound more natural:

    + **Voice**: Different parts of a text could use different voices (male/female/neutral), which is useful for reading out dialogues.
    + **Variations**: Some words could be emphasized. Others could be stretched in time. Some phrases could be said in a high pitch. Swear words could be censored.
    + **Special Cases**: Telephone numbers could be read out as individual digits. Date and time fields should not be read out as individual digits. Abbreviations could be expanded or read out as individual letters.
    + **Pauses**: Pauses could be introduced, for example, to suggest the speaker thinking or expecting a response.
    + **Recording**: A recorded audio file can be played, and if unavailable, an alternate text could be synthesized.
    + **Multilingual**: A default language could be specified at the root level. This can be overridden for specific foreign language phrases.

### Where does SSML fit in the overall speech synthesis process?

Most text-to-speech (TTS) engines process their input in stages: structure analysis, text normalization, text-to-phoneme conversion, prosody analysis and waveform production. All of these can be enhanced by SSML elements. For example, `p` and `s` SSML elements mark paragraphs and sentences; `say-as` is useful for rendering special cases; `sub` for expanding abbreviations; and so on.

Text normalization converts text into tokens suitable for speech. For example, '$200' would be converted to 'two hundred dollars'; '1/2' would become 'half'; and 'AAA' would become 'triple A'. These tokens are then converted into units of sounds called **phonemes**.

To speak all words in the same tone or loudness, creates monotony. **Prosody** is therefore useful to make speech more natural and intelligible. It draws attention to certain words by way of emphasis. Prosody is about volume, pitch, and rate of speech. We can specify duration of a word and its pitch contour.

### What exactly is a phoneme and how does SSML use it?

Once tokens are obtained via text normalization, the synthesizer must replace each token with a sequence of phonemes. A **phoneme** is a unit of sound that distinguishes one word from another. For most cases, a dictionary lookup is adequate but when there's ambiguity or non-standard pronunciation, `phoneme` or `say-as` elements can be used. One example is "read", which has differing pronunciation based on verb tense. Another example is "Caius College", which should be pronounced as "keys college".

Phonemes are language dependent. US English typically has 45 phonemes, Hawaiian has 12-18, and some languages may have even 100.

The `phoneme` element has attribute `alphabet` that must at least support "ipa" as value, which refers to **International Phonetic Association (IPA)**. Other alphabets include Speech API Phone Set (SAPI), Universal Phone Set (UPS), , IBM TTS, and Extended Speech Assessment Methods Phonetic Alphabet (X-SAMPA).

### Which are the tags defined in SSML?

Without being exhaustive, we mention a few important ones here. SSML is basically an application of XML. The root of an SSML document is `speak`. Using attributes, we can also specify the namespace and schema. Attribute `xml:lang` specifies the language.

To load a lexicon from a known URI, we can use `lexicon` element. To translate tokens into phonemes using a specific lexicon, `lookup` element is useful.

The `say-as` element has `interpret-as` as a mandatory attribute. The standard doesn't specify values for this attribute. Typical implementations include address, cardinal, characters, date, digits, fraction, ordinal, telephone, time, expletive, unit, interjection, etc.

The `sub` element replaces the contained text with the `alias` attribute value for pronunciation.

When we need to specify language for a phrase, we can use `lang` with attribute `xml:lang`. Where applicable, prefer to use the attribute with text structural elements `p`, `s`, `w` and `token`.

Element `phoneme` with attribute `ph` specifies phonemic/phonetic pronunciation. Attribute value doesn't go through text normalization or lexicon lookup.

Many elements control prosody: `voice`, `emphasis`, `break`, `prosody`. Attributes of `prosody` include `pitch`, `contour`, `range`, `rate`, `duration` and `volume`.

### Which are some real-world applications that use SSML?

Since 2019, the Guardian has been providing users important news in audio as well. They use of Google's text-to-speech API, to which the input included SSML. They noted that SSML parsing is slow. The API took about 8-10 seconds to generate the audio. Therefore, they opted to serve cached audio rather than just-in-time generation.

In the UK, NHS is using Amazon Polly to stream synthesized speech through telephone lines. This is a low-cost approach that uses widespread telephone networks to deliver healthcare remotely. A typical response latency of 60ms was observed. They use SSML, although many features are not yet used.

One blogger has suggested voice-based document reviews during long commutes. A document is converted into multiple MP3 files, each with different voice and cadence. AWS Lambda is used to convert the document to multiple SSML files. Another Lambda call triggers conversion of SSML files to MP3 files.

### What tips can you give for content writers and developers working with SSML?

With SSML, content creators can miss a closing tag or double quotes for element attributes. In the world of HTML, this problem was solved by Markdown syntax. Likewise, a replacement for SSML is **Speech Markdown**. However, we need converters to SSML until synthesizers can natively support Speech Markdown.

An alternative is to use an SSML editor. Examples are from Verndale, PlayX-team at Swedish Radio and SSML Editor. You could even create your own SSML editor, which is based on Sanity.io and React.js.

Content creators can refer to an Amazon Alexa SSML cheatsheet. YouTube audio library provides more than 5,000 free sounds that we can use in our SSML.

Developers can refer to a JavaScript implementation of an SSML parser. The Node.js package ssml-builder allows us to create SSML programmatically using the builder pattern.

Among the open source speech synthesizers are FreeTTS (Java) and eSpeech (C). There are also commercial text-to-speech engines that support SSML. Cepstral and CereVoice are examples. CereVoice includes Scottish-accented female voice, a vocal gesture library and patented Emotional Synthesis.

### What are some criticisms of SSML?

Although SSML is a W3C standard, not all its features are being supported by vendors. For example, IBM's text-to-speech service doesn't support or provides only partial support for many SSML elements or attributes. Google Assistant doesn't support the `phoneme` element.

Moreover, each vendor is introducing its own proprietary elements. Amazon Alexa's `amazon:effect` is proprietary. Google Assistant makes use of `par`, `seq` and `media`. These can be used to add background music; or create containers to play media in sequence or in parallel.  Elements `par` and `seq` are part of another W3C standard called *Synchronized Multimedia Integration Language (SMIL)*.

Parsing SSML in real time could be slow. For content writers, writing in SSML could be cumbersome and they may prefer Speech Markdown in future.

## Milestones

1995

Amy Isard at the University of Edinburgh completes her thesis on SSML with supervisor Paul Taylor. She describes SSML as an application of SGML. She also presents a prototype implementation that's understood by the CSTR Speech Synthesizer. This implementation includes phrase boundaries, emphasized words, specified pronunciations, and inclusion of other sounds files. The concept of SSML was first introduced by Paul Taylor in 1992.

Oct
1998

W3C organizes a workshop titled "Voice Browsers". The idea is to allow people with telephone connections to access Web content. This leads to the formation of **Voice Browser Working Group (VBWG)** in March 1999. These are the first steps towards the later standardization of SSML and related technologies.

1999

One study compares many speech synthesizers in the market. Each supports different aspects of prosody. There's no mention of SSML in the report. TrueTalk is said to be using escape sequences, which is very system specific. Since each system uses its own proprietary annotations or escape sequences, such annotated input is not portable. SSML is an attempt to introduce a standard to solve this.

Jun
2000

**JSpeech Markup Language (JSML)** is published as a W3C Note. It's inspired by Isard's SSML thesis and is derived from Java Speech API Markup Language that was developed at Sun Microsystems in the late 1990s.

Dec
2003

Version 1.0 of **SSML** is published as a W3C Candidate Recommendation. A draft of this can be traced to August 2000.

Mar
2004

**Voice Extensible Markup Language (VoiceXML)** is published as a W3C Recommendation. VoiceXML is designed for "creating audio dialogs that feature synthesized speech, digitized audio, recognition of spoken and DTMF key input, recording of spoken input, telephony, and mixed initiative conversations". SSML elements can be used within the `prompt` element of VoiceXML. SSML elements such as `audio` and `say-as` can have additional attributes when used within VoiceXML.

Sep
2010

Version 1.1 of **SSML** is published as a W3C Recommendation. Compared to V1.0, this version addresses the needs of many natural languages.

Apr
2017

Amazon Alexa introduces five new SSML tags: `amazon:effect name="whispered"`, `say-as interpret-as="expletive"`, `sub`, `emphasis` and `prosody`.

Mar
2018

Amazon Polly, a text-to-speech service, adds SSML **breath feature**. Instead of inserting pauses between words, breath sounds can result in more natural sounding speech. The feature allows for manual, automated and mixed modes of inserting breath.

Speech Synthesis Markup Language


# Continuous Delivery

## Summary

Continuous Delivery (CD) goes one step further from Continuous Integration (CI). It ensures that every code change is tested and ready for the production environment, after a successful build. CI ensures every code is committed to the main code repository whereas CD ensures the system is in an executable state at all times, after changes to code.

The primary goal of Continuous Delivery is to make software deployments painless, low-risk events that can be performed at any time, on demand. However, the actual process of automatic deployment in the target customer environment comes under Continuous Deployment (CD).

Continuous Delivery can work only if Continuous Integration is in place. It involves running extensive regression, UI, and performance tests to ensure that the code is production-ready. It's applicable to both bug fixes and new feature releases.

## Discussion

### What's the need for Continuous Delivery?

In a typical waterfall lifecycle, a customer release (minor or major) is a huge milestone requiring months of effort. The entire team is geared towards this milestone. Features going into the release are managed as a batch – developed, tested and integrated into the main branch. There's a high price to pay for failed releases, high pressure to meet deadlines. Customers also wait for long periods without getting fixes for their issues. There's a significant downtime when the release patch is being applied.

Contrast this with the short release lifecycle of Agile model. Here the features can be checked into the main repository as often as every day (Continuous Integration). Automated unit/regression/system test suites ensure the newly committed code is immediately sanctified. Impact on related features is verified. If this code can then be immediately released to customers, then the whole pressure around a customer release is removed. Release becomes a frequent, low risk, automated event. Manual errors in the release process are eliminated. Urgent fixes and patches can be released even during holidays, with minimal staff. This is why Continuous Delivery evolved.

### What are the benefits of Continuous Delivery?

CD is impossible without automation of build and test processes. Automation is therefore the first benefit. The entire process gets automated: *code commit → system integration build → unit test → regression test → system test*. This results in several direct and indirect benefits:

    + Smaller maintenance and support teams: Apart from developers, product teams generally maintain several parallel maintenance teams. In the traditional model, each team consisting of build/integration/test members is dedicated to a release version. However, in the automated CD process, these tasks happen without human intervention without compromising product quality.
    + Reducing risk of failure for customer releases: CD aims to make production changes safe and routine. Deploying to production becomes a boring non-event that can be performed as often as needed. Since the magnitude of code change in each release is very small, risk of failure is also minimised.
    + Enforces process discipline in the team: Since code is always meant to be production ready, team members are constantly tuning their code, refactoring, reviewing, automating repetitive build tasks, reworking test scripts and getting feature-related documentation done without any delays.

### Can you explain with an example case study?

Continuous Delivery is used to enable online controlled experiments (A/B Testing) at Amazon.

Amazon (Microsoft, Booking.com, Facebook and Google too) conducts more than 10,000 online controlled experiments annually, engaging millions of users. Developing these experimental features doesn't require much effort. They're deployed to gauge customer reaction and measure their effectiveness.

One such experiment at Amazon revealed that moving credit card offers from its home page to the shopping cart page boosted profits by tens of millions of dollars annually. In fact, Amazon makes changes to production code every 11.6 seconds on average (May 2011).

Ken Exner, Director of AWS, Amazon explains the transformational impact of CD, "One of the main uses of Continuous Delivery is to ensure we are building functionality that really delivers the expected customer value. This approach created hundreds of millions of dollars of value for Amazon, and saved similar amounts through reducing the opportunity cost of building features that were not in fact valuable (even if delivered on time, on budget, and at high quality). This approach depends on the ability to perform extremely frequent releases."

### How does Continuous Delivery work?

By the end of Continuous Integration, production code has been committed to version control such as Git, build has been completed and automated unit testing has been performed.

Taking things further in CD, unit tested code is automatically transferred to a test environment and/or a production environment after the build stage. Test driven development is followed - implementing a short test/code cycle, ideally coding a single requirement to pass a single test case.

Next, we execute relevant regression test cases. Validated executables are uploaded to a repository (like GitHub or container registry), ready for deployment to a live production environment by operations team. CI/CD tools like Jenkins are used to perform these pipeline steps.

In the case of failure at any step, code rollbacks are also automated into this pipeline. This preserves the sanctity of production code.

Approval for deployment is the last step in the task pipeline. To decide whether a feature is ready for deployment, a designated person may approve through a workflow or ticketing system. In the next paradigm of Continuous Deployment, this step is also part of the automated pipeline.

### What are the challenging scenarios for implementing Continuous Delivery?

Continuous Delivery is difficult to implement in particular types of projects and scenarios (though benefits are realized once implemented):

    + Open-ended research projects: Since such projects generally deal with new, experimental technologies that may not have a customer identified yet, production code is not meant for customer deployment. Team might still implement Continuous Integration, but may not have a system testing environment yet. CD can be done later in such cases.
    + Large legacy projects with low test automation: Projects with high level of manual interventions (approval hierarchies, security checklists) built into their system and processes will find it difficult to migrate to the CD paradigm.
    + System architecture limitations: Large monolithic architectures are difficult to automate, scale or debug. In contrast, products with modular and scalable architecture are conducive to CD. Impacted functions are easy to localise and manage every time there's a code update. Microservices, SOA, web applications are good examples of CD success stories.
    + Open-source collaborative development: With several peer developers working in parallel, open-source platforms require an equally robust build configuration and test automation team. Otherwise, it becomes difficult to synchronise their work, making implementation of CD difficult.

### What are some best practices for implementing Continuous Delivery?

Here are some CD best practices:

    + **Version Control**: Your repository be the single source of truth for code, configuration, documentation, and even the database. Avoid local or one-off changes.
    + **Automation**: Automate processes and tasks for consistency across all environments. Avoid manual intervention.
    + **Build Once**: Commit build artefacts and reuse them across environments. This saves time and improves consistency.
    + **Bite-Sized**: Keep commits small since problems are easier to track down and fix.
    + **Speed**: The CD pipeline must be fast so that it provides quick feedback to developers. Where possible, parallelize tasks.
    + **Smoke Test**: This isn't full testing but it can be run frequently and quickly. It's designed to quickly detect basic problems.
    + **Staging**: Deploy and test in an environment that mirrors the production environment.
    + **Metrics**: Measure and monitor in terms of code coverage, test success rate, support tickets, deployment failures, performance, percentage of code changed, etc.

## Milestones

2010

Jez Humble and David Farley, employees of Google and ThoughtWorks respectively, release the book titled *Continuous Delivery*. This popularizes the term for the first time.

2014

Several traditional companies talk about their DevOps journey and experiences with Continuous Delivery at the DevOps Enterprise Summit 2014 and 2015.

2015

Jez Humble and David Farley start their own company named Continuous Delivery Ltd. It's founded to help organisations achieve their software ambitions, to release world-class software that will delight their users.

Continuous Delivery


# Principal Component Analysis

## Summary

Big Data is increasingly becoming the norm and affecting many domains. When there's lots of data involving multiple variables, the work of a data scientist gets difficult. Algorithms will also take longer to complete. Wouldn't it be sensible to identify and consider only those variables that influence the most and discard others?

**Principal Component Analysis (PCA)** extracts the most important information. This in turn leads to compression since the less important information are discarded. With fewer data points to consider, it becomes simpler to describe and analyze the dataset.

PCA can be seen a trade-off between faster computation and less memory consumption versus information loss. It's considered as one of the most useful tools for data analysis.

## Discussion

### Could you explain PCA with a simple example?

We can describe the shape of a fish with two variables: height and width. However, these two variables are not independent of each other. In fact, they have a strong correlation. Given the height, we can probably estimate the width; and vice versa. Thus, we may say that the shape of a fish can be described with a single component.

This doesn't mean that we simply ignore either height or width. Instead, we transform our two original variables into two orthogonal (independent) components that give a complete alternative description. The first component (blue line) will explain most of the variation in the data. The second component (dotted line) will explain the remaining variation. Note that both components are derived from both height and width.

More intuitively, the first component line can be seen as the best-fit line that minimizes information loss. Alternatively, it can also be seen as the line that maximizes the variation; that is, it tries to explain as much of the variation in the dataset as possible.

### Could you mention some real-world use cases of PCA?

PCA has been applied for facial recognition. For 90% capture variance, only a third of the components had to be retained. This may be sufficient for Machine Learning applications. The other two-thirds contain most of the image details.

In another study, the consumption of 17 different food types was studied across 4 countries in the UK. Thus, this problem has 17 features and hence non-trivial to analyze. With PCA, the first component showed that Northern Ireland was unique. People of Northern Ireland consumed fresh potatoes and fresh fruit differently from other populations.

The lower molar teeth of an ancient mammal named Kuehneotherium was studied in nine variables. PCA showed that just two components are enough to explain over 95% of total variation. When plotted, it was easy to see the clusters and relate them back to the original features. One cluster stood for a species of Kuehneotherium while another broader cluster suggested an unidentified animal.

To detect lactose in lactose-free milk using NIR spectroscopy, containing 601 dimensions, PCA identified distinct clusters with just two principal components.

### Isn't PCA similar to Dimensionality Reduction?

In a complex data-intensive problem, there are usually many influencing variables. The term *variable* is equivalent to other commonly used terms: *feature* or *dimension*.

The idea of reducing the number of variables or dimensions is called **Dimensionality Reduction**. This can be done in two ways:

    + **Feature Elimination**: We drop some features that we may consider unimportant. While the approach is simple, we lose useful information present in those dropped features.
    + **Feature Extraction**: We transform the original set of features into another set of features. The idea is to pack the most important information into as few derived features as possible. We can reduce the number of dimensions by dropping some of the derived features. But we don't lose complete information from the original features: derived features are a linear combination of the original features.PCA is in fact a method for doing feature extraction. In PCA, derived features are also called *composite features* or *principal components*. Moreover, these principal components are linearly independent from one another.

### What are advantages of the PCA technique?

PCA minimizes information loss even when fewer principal components are considered for analysis. This is because each principal component is along a direction that maximizes variation, that is, the spread of data. More importantly, the components themselves need not be identified a priori: they are identified by PCA from the dataset. Thus, PCA is an adaptive data analysis technique. In other words, PCA is an unsupervised learning method.

By reducing the number of dimensions, PCA enables easier data visualization. Visualization helps us to identify clusters, patterns and trends more easily. Fewer dimensions means less computation and lower error rate. PCA reduces noise and makes algorithms work better.

Finding the principal components is really an eigenvalue/eigenvector problem, which has been well studied with lots of algorithms available for practical use.

Although Gaussian distribution of data is assumed, as a descriptive tool PCA doesn't need this assumption. It can be used for exploratory analysis on data of any distribution. There are also variations of PCA that cater to different data types and structures.

### What are drawbacks of the PCA technique?

Here are some drawbacks of PCA:

    + PCA works only if the observed variables are linearly correlated. If there's no correlation, PCA will fail to capture adequate variance with fewer components.
    + PCA is lossy. Information is lost when we discard insignificant components.
    + Scaling of variables can yield different results. Hence, scaling that you use should be documented. Scaling should not be adjusted to match prior knowledge of data.
    + Since each principal components is a linear combination of the original features, visualizations are not easy to interpret or relate to original features.

## Milestones

1850

Mid-nineteenth century works by Cauchy and Jacobi in classical analytic geometry show that the equations for the principal axes of quadratic forms and surfaces are known.

1889

Francis Galton in his *Natural Inheritance* connects principal axes for the first time with the correlation ellipsoid.

1901

Karl Pearson invents PCA while working to find the major and minor axes of an ellipse. However, he does not use the term PCA. In his geometric interpretation of the problem, he's trying to find "lines and planes of closest fit to systems of
 points in space".

1930

Harold Hotelling develops PCA independently and names the technique. His approach is what is familiar to us today, using successive orthogonal linear combinations with maximum variance. The 1930s is also the decade when the development of **Factor Analysis** is started. This is closely related to PCA.

1960

Around 1960, Malinowski introduces PCA to chemistry. After 1970, many chemical applications of PCA appear in literature.

1966

How does one determine how many principal components to retain for analysis? In the context of factor analysis, R.B. Cattell proposes a method called *Scree Test*. A **Scree Plot** is used for this purpose. It represents graphically the eigenvalues or the percentages of total variation accounted for by each principal component.

Principal Component Analysis


# Latent Dirichlet Allocation

## Summary

Given a document, topic modelling is a task that aims to uncover the most suitable topics or themes that the document is about. It does this by looking at words that most often occur together. For example, a document with high co-occurrence of words 'cats' and 'dogs' is probably about the topic 'Animals', whereas the words 'horses' and 'equestrian' is partly about 'Animals' but more about 'Sports'. **Latent Dirichlet Allocation (LDA)** is a popular technique to do topic modelling.

LDA is based on probability distributions. For each document, it considers a distribution of topics. For each topic, it considers a distribution of words. This information helps LDA discover the topics in a document.

LDA and its many variants support diverse applications. LDA is well-supported in a few programming languages and software packages.

## Discussion

### What are the shortcomings of earlier topic models that LDA aims to solve?

Before LDA, there were LSA and pLSA models. LSA was simply a dimensionality reduction technique and lacked a strong probabilistic approach. pLSA remedied this by being a probabilistic generative model. It picked a topic with probability *P(z)*. Then it selected the document and the word with probabilities *P(d|z)* and *P(w|z)* respectively.

While LSA could model synonymy well, it failed in polysemy. In other words, a word with multiple meanings needed to appear in multiple topics but didn't. pLSA partially handled polysemy. However, pLSA ignored *P(d)*. Each document was a mixture of topics but there was no model to generate this mixture. This made pLSA grow linearly as the corpus size increased, leading to overfitting. Also, pLSA was unable to assign topic probabilities to new documents.

LDA is inspired by pLSA. Like pLSA, it's also a probabilistic generative model. Unlike pLSA, LDA also considers the generation of documents. This is where the **Dirichlet distribution** becomes useful. It determines the topic distribution for each document.

### Could you describe some example applications where LDA has been applied?

LDA is an algorithm or method for topic modelling, which has been applied in information retrieval, text mining, social media analysis, and more. In general, topic modelling uncovers hidden structures or topics in documents.

LDA has been applied in diverse tasks: automatic essay grading, anti-phishing, automatic labelling, emotion detection, expert identification, role discovery, sentiment summarization, word sense disambiguation, and more.

For analysing political texts, LDA-based model was used to find opinions from different viewpoints. In software engineering, LDA was used to find similar code in software repositories and suggest code refactoring. Another study made use of geographic data and GPS-based documents to discover topics.

LDA has been used on online or social media data. By applying it on public tweets or chat data, we can detect and track how topics change over time. We can identify users who follow similar distribution of topics. On Yelp restaurant reviews, LDA was used to do aspect-based opinion mining. LDA was used on a school blog to uncover main topics and who's taking about them.

### Why is LDA called a generative model?

In a **generative model**, observations are generated by latent variables. Given the words of a document, LDA figures out the latent topics. But as a generative model, we can think of LDA as generating the topics and then the words for that document.

In the figure, we note that \(\alpha\) fixes a particular distribution of topics \(\theta\). There's one such distribution for each document. For a document, when we pick a topic from this distribution, we're faced with word distribution \(\beta\_i\) for that topic. These word distributions are determined by \(\eta\). From our topic's word distribution, we pick a word. We do this as many times as the document's word count. Thus, the model is generative.

### What's the significance Dirichlet priors in LDA?

Topic distribution \(\theta\) and word distribution \(\beta\) are created from \(\alpha\) and \(\eta\) respectively. The latter are called **Dirichlet priors**. A low \(\alpha\) implies a document might have fewer dominant topics. A large \(\alpha\) implies many dominant topics. Similarly, a low (or high) \(\eta\) means a topic has a few (or many) dominant words.

Given the Dirichlet distribution \(Dir(\alpha)\) we sample a topic distribution for a specific document. Likewise, from \(Dir(\eta)\) we sample a word distribution for a specific topic. In other words, the Dirichlet distribution generates another distribution. For this reason, it's called *distribution over distributions*.

Suppose we have k topics and a vocabulary V. \(\alpha\) will be a vector of length k. \(\eta\) will be a vector of length V. If all elements of the vector have the same value, we call this **symmetric Dirichlet distribution**. It simply means we have no prior knowledge and assume all topics or words are equally likely. In this case, the prior may be expressed as a scalar, called **concentration parameter**.

In an alternative terminology, \(\eta\) symbol is not used. \(Dir(\beta)\) generates \(\phi\).

### What are the methods of inference and parameter estimation under LDA?

In LDA, words are observed, topic and word distributions are hidden, and \(\alpha\) and \(\eta\) are the hyperparameters. Thus, we need to infer the distributions and the hyperparameters. In general, this problem is intractable. The two distributions are coupled in the latent topics.

We note three common techniques for inference and estimation:

    + **Gibbs Sampling**: A method for sampling from a joint distribution when only conditional distributions of topics and words can be efficiently computed.
    + **Expectation-Maximization (EM)**: Useful for parameter estimation via maximum likelihood.
    + **Variational Inference**: Coupling between distributions is removed to yield a simplified graphical model with free variational parameters Now we have an optimization problem to find the best variational parameters. *Kullback-Leibler (KL) divergence* between the variational distribution and the true posterior can be used.

### What's the typical pipeline for doing LDA?

The actual working of LDA is **iterative**. It starts by randomly assigning a topic to each word in each document. Then the topic and word distributions are calculated. These distributions are used in the next iteration to reassign topics. This is repeated until the algorithm converges. Once the distribution is worked out during training, the dominant topics of a test document can be identified by its location in the topic space.

Suppose a document has only a few topics. Suppose a topic has only a few highly likely words. These two goals are at odds with each other. By trading off these two goals, LDA uncovers tightly co-occurring words.

### Could you describe some variants of the basic LDA?

While LDA looks at co-occurrences of words, some LDA variants include metadata such as research paper authors or citations. Another approach is to look at word sequences with Markov dependencies. For social network analysis, the **Author-Recipient-Topic** model conditions the distribution of topics on the sender and one recipient.

For applications such as automatic image annotation or text-based image retrieval, **Correspondence LDA** models the joint distribution of images and text, plus the conditional distribution of the annotation given the image.

**Correlated Topic Model** captures correlations among topics.

Word co-occurrence patterns are rarely static. For example, "dynamic systems" more recently co-occur with "graphical models" more than "neural networks". **Topics over Time** models time jointly with word co-occurrences. It uses a continuous distribution over time.

LDA doesn't differentiate between topic words and opinion words. Opinions can also come from different perspectives. **Cross-Perspective Topic** model extends LDA by separating opinion generation from topic generation. Nouns form topics. Adjectives, verbs and adverbs form opinions.

Jelodar et al (2018) note many more variants.

### What are some resources for working with LDA?

In Python, *nltk* is useful for general text processing while *gensim* enables LDA. In R, *quanteda* is for quantitative text analysis while *topicmodels* is more specifically for topic modelling. In Java, there's *Mallet*, *TMT* and *Mr.LDA*.

Gensim has a useful feature to automatically calculate the optimal asymmetric prior for \(\alpha\) by accounting for how often words co-occur.

LDA is built into Spark MLlib. This can be used via Scala, Java, Python or R. For example, in Python, LDA is available in module `pyspark.ml.clustering`.

There are plenty of datasets for research into topic modelling. Those labelled with categories or topics may be more useful. Some examples are Reuters-21578, Wiki10+, DBPL Dataset, NIPS Conference Papers 1987-2015, and 20Newgroups.

## Milestones

1990

Deerwester et al. apply Singular Value Decomposition (SVD) to the problem of automatic indexing and information retrieval. SVD brings together terms and documents that are closely related in the "semantic space". Their idea of semantics is nothing more than a topic or concept. They call their method **Latent Semantic Indexing (LSI)** or **Latent Semantic Analysis (LSA)**.

Aug
1999

Hofmann presents a statistical analysis of LSA. He coins the term **Probabilistic Latent Semantic Analysis (pLSA)**. It's based on *aspect model*, which is a latent variable model. It associates unobserved class variables (topics) with each observation (words). Unlike LSA, this is a proper generative model.

Jan
2003

First presented at NIPS 2001 conference, Blei et al. describe in detail a probabilistic generative model that they name **Latent Dirichlet Allocation (LDA)**. They note that pLSA lacks a probabilistic model at the document level. LDA overcomes this. Their work uses the bag-of-words model but they note that LDA can be applied for larger units such as n-grams or paragraphs.

Jul
2003

Blei and Jordan consider the problems of automated image captioning and text-based image retrieval. They study three hierarchical probabilistic mixture models. They arrive at **Correspondence LDA (CorrLDA)** that gives best performance.

Dec
2009

Typically, symmetric Dirichlet priors are used in LDA. Wallach et al. study the effect of structured priors for topic modelling. They find that **asymmetric Dirichlet priors** over document-topic distributions is much better than symmetric priors. To use asymmetric priors over topic-word distributions has little benefit. The resulting model is less sensitive to number of topics. With hyperparameter optimization, computation can be made practical. Related research with similar results are reported in 2018 by Syed and Spruit.

2016

Word2vec came out in 2013. It's a word embedding that's constructed by predicting neighbouring words given a word. LDA on the other hand looks at words at the document level. Moody proposes **lda2vec** as an approach to capture both local and global information. This combines the power of word2vec and the interpretability of LDA. Word vectors are dense but document vectors are sparse.

Latent Dirichlet Allocation


# Internet Engineering Task Force

## Summary

The Internet Engineering Task Force (IETF) can be described "as a large open international community of network designers, operators, vendors, and researchers concerned with the evolution of the Internet architecture and the smooth operation of the Internet." Equivalently, "IETF is a loosely self-organized group of people who contribute to the engineering and evolution of Internet technologies. It is the principal body engaged in the development of new Internet standard specifications. The IETF is unusual in that it exists as a collection of happenings, but is not a corporation and has no board of directors, no members, and no dues."

IETF is driven by individuals, not companies or governments. Although IETF produces standards (and non-standards documents), it's not called a standards body. IETF has been successful due to the openness in its processes.

## Discussion

### Why do we need IETF?

The Internet is complex: diversity of applications, variety of protocols, multiple vendors, different implementations, massive scale of deployment, worldwide reach across geographies, and so on. If two endpoints on the Internet have to communicate and be understood, they need to talk the same language. IETF fulfils the role of standardizing architecture and protocols for the Internet. This enables things to work together despite the diversity of vendors, implementations and protocols.

The goal of IETF is to make the Internet work better. It does this by making relevant protocol standards, best current practices and informational documents.

### What are cardinal principles that guide IETF?

To fulfil its mission, IETF follows these principles:

    + Open process: All activities of IETF are open for public participation. All published documents are openly accessible.
    + Technical competence: Quality is ensured by staying within domains where members have technical competence.
    + Volunteer core: Work is done on voluntary basis by those who believe in IETF's mission.
    + Protocol ownership: When IETF takes ownership of a protocol it also becomes responsible for it.

### What does openness mean for IETF?

Openness within IETF takes three forms:

    + Open standards: Anyone can join mailing lists, Working Groups and attend meetings. Anyone can create and upload a new document. Anyone can suggest modifications to proposals. The process of standardization is open to all without fees or formal membership.
    + Open documents: All documents are easily and freely accessible on the Internet.
    + Open source: Working implementations and proven interoperability are important before a proposal becomes a standard. Such implementations are open sourced.While other organizations or standards bodies may be open in terms of publication and ownership, IETF adopts the most open approach that includes development and participation.

### Why has IETF been successful so far?

When Open Systems Interconnection (OSI) standards were all in rage in the mid-1980s, the Internet and its protocols were part of a government-funded research project. The Internet of the 1980s did not carry much commercial or for-profit traffic. Yet by early 1990s, the Internet model, not the OSI model, became the de facto architecture for the Internet. This, and the Internet's continued success, can be attributed to the open working model and organization of IETF.

Besides openness, IETF takes a bottom-up approach. In other words, people get together to create Working Groups. Neither IAB or IESG create WGs on their own. IETF does not require unanimity. Typically a consensus is achieved if 80-90% of people agree to a proposal. There's no voting but a show of hands may be used. Working implementations that interoperate prove that the proposal works in practice. This forces WGs to simplify protocols and even take out stuff that doesn't work.

Dr. David C. Clark of MIT summed it up nicely,

> We reject kings, presidents, and voting. We believe in rough consensus and running code.

### Do authors have copyright over the documents they produce?

All IETF are publicly available on the Internet. They can be downloaded free of cost by anyone. IETF gets a limited copyright to publish and derive from the documents. Once published on IETF, authors cannot withdraw the documents at a later point. All other rights are with authors.

### What's the organizational structure within IETF?

Closely associated with IETF are the following:

    + **Internet Society (ISOC)** : IETF by itself is not a legal entity. It can be seen as an organized activity of the Internet Society (ISOC). ISOC provides the legal umbrella for IETF and funds its activities.
    + **Internet Research Task Force (IRTF)** : While IETF focuses on near term objectives and making standards, its sister organization IRTF looks at longer term research. It's activities are managed by the Internet Research Steering Group (IRSG).
    + **Internet Architecture Board (IAB)** : IETF and IRTF are overseen by the IAB. IAB focuses on architectural issues and procedural issues. It also manages all external relations. IAB was previously called the Internet Activities Board. It is chartered by ISOC.
    + **Independent Submissions Editorial Board (ISEB)** : Led by the Independent Submissions Editor (ISE), this board reviews and publishes documents relevant to the Internet but outside the scope of IETF/IRTF/IAB.
    + **Internet Engineering Steering Group (IESG)** : IESG manages IETF activities and processes. It approves the standards produced by IETF. IAB advises IESG.
    + **Nominating Committee (NomCom)** : NomCom nominates suitable candidates to serve on IESG and IAB.

### Which external organizations are associated with IETF?

IETF works with other standards bodies because its work often builds upon or interfaces with other standards. For example, IETF cooperates with IANA (Internet numbers), IEEE (Ethernet), ETSI (Cellular Radio), ITU-T (PHY layer standards), ISO/IEC (UTF-16, mime types), W3C (HTML), and more.

### How to become a member of IETF?

IETF has no formal membership. Anyone who joins a mailing list or attends an IETF meeting can be called a member.

### What are types of documents produced by IETF?

Broadly, documents can be either Internet-Drafts (I-Ds) or RFCs. Internet-Drafts are temporary documents that are valid for six months. Because of their temporary nature, they are not to be cited. Anyone can start such a draft even without a Working Group. They have no formal status until they become adopted by a WG or becomes an RFC.

RFC originally stood for Request for Comments but today (March 2017) a published RFC cannot be modified. An RFC could be supplemented by errata. In fact, Internet-Drafts are more in the spirit of documents requesting comments.

When Internet-Drafts become RFCs, they fall into one of these categories:

    + Non-standards documents: Informational, Historic, Experimental.
    + Standards documents: Proposed Standard or Internet Standard. An intermediate Draft Standard has been discontinued since 2011.
    + Best Current Practices

### What's the IETF standardization process?

IETF work happens within Working Groups (WGs) that are part of a topical Area. As of March 2017, there were 7 Areas and 100+ WGs. Each Area has a couple of Area Directors (ADs). Each WG will have its set of deliverables clearly defined plus any liaison with other WGs. Except for the annual meetings, WGs do all their work via mailing lists. Documents produced by WGs are reviewed by IESG. RFC Editor publishes the final document.

Internet-Drafts can become Proposed Standard RFCs, where they will remain for at least six months. Interoperatiblity of multiple implementations must be proven at this stage. A last call for comments is put out. Finally, it becomes an Internet Standard RFC. RFCs superseded by newer updates become Historic RFCs. Experimental RFCs are used for exploring new technology including feasibility studies. The complete process is documented as RFC 2026, BCP 9.

### How is IETF funded?

From the time of its inception till 1997, IETF was funded by the US government, notably ARPA, NSF, NASA and DOE. From 1998, ISOC is the main funding body for IETF. However, ISOC did fund IETF prior to this. For example, US$225K was given in 1993. Since March 1991, IETF also charges meeting fees for attendees. These are used to cover meeting expenses plus secretariat expenses.

As of 2016, IETF operated on an annual budget of US$6million, of which about US$2million comes from ISOC In July 2016, some companies and organizations donated US$3million towards a fund called IETF Endowment. This fund's purpose is to "provide long-term stability and increased diversity for funding IETF activities and operations."

Volunteers themselves may be self-funded or sponsored by either their employers or other sponsors.

## Milestones

Jan
1986

First meeting of IETF is held with attendees from agencies funded by the US government. ARPANET's Network Working Group may be regarded as its precursor.

Oct
1986

Fourth meeting of IETF is held for which the public is invited to attend. Since then, all meetings are open to the public.

Feb
1987

Concept of Working Groups (WGs) is introduced.

1991

From 1991, meetings are held thrice a year. Previously, meetings were quarterly events.

1992

The relationship and responsibilities between IAB and IETF come under scrutiny and clarification.

Jan
1992

ISOC is formed. ISOC is an international, membership-based non-profit organization. It also provides the legal umbrella for IETF, which IETF accepted in 1996.

1993

IETF moves from being a US government-funded activity to one funded by Internet Society (ISOC).

1997

US government stops direct funding to IETF. From 1998, ISOC becomes the main funding channel for IETF.

2005

IETF Trust is established to manage copyrights.

Internet Engineering Task Force


# Cloud Native 5G Core

## Summary

5G Core has been standardized as a composition of many Network Functions (NFs) that expose their services via RESTful APIs. This is called Service-Based Architecture (SBA). To implement/deploy such a core, cloud-native is an architectural approach that's becoming increasingly popular. It brings benefits to both vendors and operators. Virtualization, containerization, microservices architecture and Kubernetes are some technologies that facilitate a cloud-native 5G Core.

SBA addresses primarily the control plane. A cloud-native architecture should also consider the data plane and its performance. A cloud-native 5G Core should interwork with legacy architectures. Some operators may want a dual-mode core in which 4G EPC functions are also implemented in the cloud-native manner.

Beyond the standardized NFs, a good cloud-native architecture should include provisioning, configuration, CI/CD, observability, scalability, security, and service orchestration.

## Discussion

### What's the cloud-native approach towards implementing 5GC?

The Cloud Native Computing Foundation (CNCF) has defined what cloud native means. Apps have to use the microservices architecture, that is, small independent components with cleanly defined interfaces. Microservices should be stateless whereby data is separated from the business logic. In 5G Core, unstructured data is stored in UDSF while structured data is stored in UDR.

Via dynamic orchestration, containers can be created or destroyed at will. This leads to resilience and scalability. Immutable infrastructure is another attribute: it's easier to provision/deploy new resources than modify or upgrade existing ones. The system should be observable. To deliver features quickly, DevOps practices and CI/CD pipelines should be followed.

Cloud-Native Functions (CNFs) are NFs realized using cloud-native principles and components. These are deployed using Kubernetes or any Containers-as-a-Service (CaaS) platform based on Kubernetes. Many CNFs can execute within the same Kubernetes cluster.

### How does VNF differ from CNF?

Traditionally, costly proprietary hardware were used to build telecom networks. These are called Physical Network Functions (PNFs). With virtualization, network functionality could be deployed as Virtual Network Functions (VNFs) on cheaper commercial-off-the-shelf (COTS) hardware. Different VNFs could run on the same hardware.

Whereas VNFs run on Virtual Machines (VMs), Cloud-Native Network Functions (CNFs) run inside containers. Containers are more lightweight than VMs. A container includes the application along with dependencies. All containers on a server share the same OS and resources. CNFs give operators more flexibility. Applications can be more easily scaled on demand or ported to different environments (development, staging, production). CNFs bring higher operational efficiency and better use of resources (compute, storage, networking).

Just moving a legacy application into a container won't help. To realize the benefits of cloud native, applications have to be rearchitected into microservices. The standardization of SBA enables this.

### How will VNFs coexist with CNFs?

Adoption of CNFs isn't going to be immediate. 5G Core networks are expected to include both VNFs and CNFs for some time to come. The NFV Orchestrator (NVFO) will interact with both Virtual Infrastructure Manager (VIM) and Kubernetes.

CaaS can be deployed on IaaS (KVM, VMWare, etc). An alternative is to deploy CaaS directly on bare metal. The latter is less secure since it's easier to access host kernel space from host user space. However, the latter approach avoids the virtualization overhead. For better security via isolation, kata containers can be used while keeping the virtualization overhead minimal.

Multiple CaaS (aka independent Kubernetes clusters) can be deployed over the same IaaS. In this case, CNFs in one cluster are isolated from those in another. This could be an alternative to kata containers on bare metal.

CaaS themselves have evolved to supported legacy VM-based workloads. Using technologies such as KubeVirt or RancherVM, we can run VMs within a CaaS environment.

### What are the benefits of going cloud native for the 5GC?

There are benefits for both vendors and operators. With cloud native, features can be developed and deployed faster. This can start small and then quickly scaled up across the network. Efficiency and performance are improved. Resources can be scaled up quickly and automatically to meet peak demands, and vice versa.  Resources can be deployed easily for a 5G feature called network slicing.

Cloud-native is agnostic of platform or cloud provider. Apps become portable and there's less vendor lock-in. From a business perspective, operators benefit from cost optimization and faster innovation.

It's insightful to see what characteristics bring these benefits. A cloud-native application is agnostic of the underlying infrastructure, which could be bare metal, IaaS, PaaS, or Kubernetes-based CaaS. Application is decomposed into microservices, each with its own software lifecycle. This bring agility, flexibility and efficiency. Deployment, observability and recovery are automated. Software as a whole is more resilient because non-responsive containers can be quickly replaced. This is because the microservices they contain are stateless. Application logic and state are cleanly separated.

### What's the role of a service mesh in 5GC?

Since microservices have a dynamic lifecycle, this presents a new set of problems: service discovery, traffic management, security and policy management, and observability. Each application can address these concerns. A better solution is to abstract away such infrastructure logic into a separate component. This is what service mesh offers.

Service mesh is a dedicated infrastructure layer for service-to-service communication. Each application is accompanied by a sidecar proxy container. Via these proxies, we can collect metrics or implement retries, timeouts, circuit breaking, load balancing, secure tunnelling, and service discovery. Service mesh includes both data plane and control plane. Istio, Linkerd and Consul are examples of service mesh. A service mesh can be part of the Kubernetes platform or embedded within each CNF.

3GPP hasn't standardized the use of a service mesh. Instead, NF consumers use NRF to discover NF producers. When there millions of active service flows, NRF could get flooded with discovery requests. This can be mitigated by using SCP. NRF and SCP fulfil the role of a service mesh.

### What are the challenges in making 5G Core cloud native?

Each vendor offers more than just the application logic of NFs. While an operator may aim for a multi-vendor solution, in-house integration is a challenge. At least for some NFs, the operator may adopt pre-integrated solutions.

Cloud-native technologies were developed for the web/mobile services that expose REST APIs. They're not yet mature enough for telco-grade requirements of bandwidth, latency, availability and security. Telecom networks typical demand 99.999% availability. The twelve factors that characterize microservices-based web apps can't directly be applied to telecom.

Adopting the cloud native approach requires change of mindset. Applications are no longer vertically integrated. Common functionality will be horizontally integrated at the platform layer. Neither vendors nor operators should feel they're losing control. Instead they should see the benefits of embracing cloud native.

For the data plane, performance (throughput, latency, jitter) is a challenge. DPDK and VPP are solutions that bypass the kernel space and process data directly in the user space. Another approach called SR-IOV bypasses the hypervisor and gives multiple VNFs direct access to the same physical NIC.

### Who has implemented cloud-native 5G Core?

Huawei's Telco Converged Cloud (TCC) is a cloud-native architecture. It's based on OpenStack and Kubernetes. It allows VMs and containers (within VMs or on bare metal) to coexist. For better resource utilization, it offers the Unified Resource Management layer.

Ericsson offers a layer called Generic Services that includes load balancing, databases, observability, and FCAPS. For business logic, it offers Packet Core Controller and Packet Core Gateway.  It offers end-to-end management from deep edge sites to central sites.

Samsung's 5G Core aims to be FAST: fast, agile, scale and tunable. With closed-loop automation they claim to achieve high operational efficiency. Orchestration, operations and analytics are centralized. Samsung identifies three types of microservices: NF-specific, common (eg. database, event), and OAM (eg. logging, tracing). It's competitive advantage comes from multi-interface, pod-based load balancer, geo-redundancy, inline upgrade, UPF packet acceleration (SR-IOV, OVS-DPDK) and open tracing.

Among the open source offerings, there's some work happening to port Open5GS on Kubernetes and Free5GC on Kubernetes. As a cloud-native service platform, Kube5G offers CI/CD and operator workflows. Its alternative is 5G-Kube.

## Milestones

Oct
2012

At the SDN and OpenFlow World Conference, an introductory white paper titled *Network Functions Virtualization* is published. This triggers interest towards virtualization of telecom infrastructure. NFV enables Software Defined Networking (SDN). SDN advocates the separation of control and data planes. By now, some vendors are already offering virtualized versions of their proprietary software. However challenges remain in terms of interoperability, orchestration, performance, and stability.

2013

**Docker** is released to build and manage containerized applications. It uses Linux kernel features to isolate processes. But for an app composed of dozens of containers, it quickly becomes clear that some sort of orchestration tool is needed.

Dec
2014

ETSI releases v1.1.1 of **NFV Management and Orchestration (NFV-MANO)**.

2015

**Kubernetes 1.0** is released as a tool for container orchestration. At the same time, the **Cloud Native Computing Foundation (CNCF)** is formed with Kubernetes as its first project.

Nov
2017

5G PPP's 5G-MoNArch project publishes the document *5G Mobile Network Architecture*. This talks about a "cloud-enabled protocol stack" that's based on virtualization and orchestration. The term **telco cloud** is used although there's no mention of the term "cloud native".

Jul
2018

5G PPP publishes a white paper titled *From Webscale to Telco, the Cloud Native Journey*. It notes that future 5G networks can't ignore the cloud-native paradigm, which it summarizes as "the shift from the cloud ready to cloud native, from VM to container, from MANO to Kubernetes". They identify six features needed for telecom: multi-network interfaces, service function chaining, data plane acceleration, enhanced platform awareness, environment aware scheduling and multi-site support.

May
2019

The CNCF creates the **Telecom User Group (TUG)**. Under this group, telecom vendors and operators can collaborate for faster adoption of cloud-native. Some operators are deploying containers within VMs, which isn't the right way to containerize apps. There are also networking challenges that projects such as MULTUS, DANM and NSM are addressing.

Dec
2019

Early adoption of VNFs are disappointing because applications are not stateless or platform agnostic. Where CNFs are adopted, they run inside VMs. However, stakeholders realize Kubernetes on bare metal is the future. One trial shows that NF deploy time drops from 220 seconds under OpenStack to less than 30 seconds under Kubernetes.

2020

Samsung offers a cloud-native 5G Core platform where both VNFs and CNFs can coexist. This enables seamless migration from VNFs to CNFs. It's working towards a multi-cloud solution. For example, core can be split between on-prime private cloud and remote public cloud. With a purely public cloud approach, user plane runs on AWS Outpost for performance reasons while the control plane runs centrally. MEC applications can be more easily deployed on public clouds.

2022

While adoption of Kubernetes and CNFs is growing, it's expected that **VNFs and CNFs will coexist** for some time. Platforms are likely to support both. Huawei's Telco Converged Cloud (TCC) is an example, available since 2020. It supports both OpenStack-based VMs and Kubernetes-based containers. It also supports containers that can run within VMs or directly on bare metal.

Nov
2023

The Linux Foundation announces its intention to merge telecom-specific activities of the CNCF to those of the Linux Foundation's Networking group. Certification and test suites related to CNFs will be handled by the Linux Foundation.

Cloud Native 5G Core


# Types of Mobile Apps

## Summary

From a user perspective, mobile apps can be categorised as educational, informational, productivity, gaming, entertainment, communication, eCommerce, and so on. This article is more from a developer perspective, the different technologies that power mobile apps.

The earliest apps interfaced directly with the native capabilities of the platform on which they were deployed. This was the case with apps running on Symbian OS. Later when iPhone (2007) and Android (2008) came out, the app ecosystem became fragmented. App developers had to develop separately for iOS and Android.

To solve this problem and reuse code, hybrid apps were invented but they had performance limitations. In the 2010s, cross-platform development because possible by interfacing to native capabilities without sacrificing performance.

## Discussion

### What are the different types of mobile apps?

There are three broad types of mobile apps:

    + **Native**: App is developed for a specific platform using the platform's native APIs. It's therefore performant and has full access to device capabilities. The platform's ecosystem, such as an app store, helps in distributing the app. Code cannot be reused for other platforms. Examples are iOS (Swift or Objective-C), Android (Kotlin or Java), Windows Phone (C# .NET).
    + **Hybrid**: App is developed using web technologies but is wrapped inside a native app. Code can be reused across platforms. App can also access native APIs. Examples are Ionic, Sencha Touch.
    + **Mobile Web**: App is developed with web technologies and delivered via a web browser. Typically, these are web apps with responsive design so that they display well on smaller devices as well. Business logic is on the server and only a single codebase needs to be maintained. There's no need to install anything from app stores. *Progressive Web Apps (PWA)* deliver mobile app-like experience via a web browser.

### Could you explain how hybrid mobile apps work?

The term "hybrid" implies that the app uses a mix of web technologies and native APIs. HTML5, CSS and JS are used so that same code can be reused across platforms. Native APIs are used to get access to camera, geolocation, etc. In the case of Ionic, which is based on Cordova, *WebView* is used. This is a browser instance that interprets and render the UI. It's wrapped within the app. Cordova takes care of the wrapping and interfacing to the native APIs.

Hybrid apps have to live with the limitations of WebView and therefore performance could suffer. Latest native capabilities may not be available unless platforms such as Ionic keep pace with the changes. To overcome this, platforms such as Appcelerator Titanium (JavaScript), React Native (JavaScript), NativeScript (JavaScript/TypeScript) and Xamarin (C#) take a different approach. App can be written in another language but then compiled or interpreted to call native APIs. Thus, code reuse is possible without sacrificing performance. Such apps may be called "hybrid native apps" or even simply "native apps".

### What are the metrics by which we should evaluate mobile app platforms?

Some metrics include development cost, performance, distribution, monetization, device features and their accessibility, user interface, code portability, and maintenance. For example, native platforms give full access to device features and better performance. Codebase of hybrid apps is easier to maintain at lower cost and can be ported to another platform more easily. Native apps are typically distributed through app stores. Hybrid apps can be distributed directly but they lose the benefits of app store discovery.

You may also want to look at availability, compilation, development environment, programming language/platform and community support. For example, React Native is free but Xamarin is free, subject to terms and conditions of Visual Studio Community Edition. Just-in-time compilation is not possible for iOS on both React Native and Xamarin. React Native and TypeScript are based on JavaScript and therefore easier for web developers. Developers coming from C# and .NET background will find Xamarin easier to adopt. A larger community means that you can find third-party packages and UI components.

### How should I choose the right platform for my next mobile app?

Developers skilled in native platforms are difficult to find and costly. For each platform, you would need a different skill set. If you want your app to reach a broad audience across platforms, then either a mobile web app or hybrid app must be preferred. These allow code reuse across platforms.

If your app involves heavy graphics such as in gaming, requires full access to device capabilities, or full control of the UI, then native apps will offer these. In fact, if your app is targeted at only a specific platform such as iOS, then going native is a sensible approach.

To obtain native performance while also reusing as much of the code as possible across platforms, React Native, NativeScript, Xamarin, etc. can be adopted. What you adopt should depend on the maturity and support of the technology, plus the skill sets available with you. For example, C# and .NET developers can adopt Xamarin while JavaScript developers can adopt React Native or NativeScript.

## Milestones

1998

Psion Software is renamed as Symbian Ltd, and its EPOC32 becomes **Symbian OS**. Symbian OS forms the basis of many UI variants including Series 80, Series 60, UIQ and MOAP. This UI fragmentation makes it difficult to deliver apps across the variants. Symbian programming is via *Symbian C++*. It's low level and difficult. In 2011, this is replaced with Qt-based development but it comes too late against competition from iOS and Android.

Jun
2007

Apple launches the very first iPhone in June 2007. iPhone offers capacitive touchscreen, which is vastly superior to resistive touchscreens of Symbian-based phones. The OS is named *iPhone OS*. The initial idea is that developers would build web apps that would be rendered on the Safari web browser; but in February 2008, Apple releases the **iPhone SDK** to enable third-party native apps. In June 2010, iPhone OS is renamed to *iOS*.

Nov
2007

Google releases beta version 1.0 of Android for developers. Developers can make native apps for the Android platform. The first Android phone named T-Mobile G1 (or HTC Dream) is announced in September 2008.

2009

A startup named Nitobi creates **PhoneGap**, for developing cross-platform hybrid apps. Using only HTML5, CSS and JS, it provides APIs to access native capabilities. Apps created with PhoneGap use *WebView*. In 2011, Adobe purchases PhoneGap. It also open sources the code via Apache Software Foundation (ASF). The open source variant is called **Apache Cordova**.

2010

To facilitate mobile apps using web technologies such as HTML5, CSS and JS, **Sencha Touch** is released. Developers can build UIs that looks like native UIs. It also support Cordova APIs to allow access to accelerometer, camera, geolocation, etc.

Apr
2012

Samsung releases **Tizen**, an OS that can be used on a wide range of embedded devices including smartphones. The first smartphone to run on Tizen is planned for a 2014 launch in Russia but cancelled due to lack of apps. The first smartphone comes out in India in January 2015, the Samsung Z1. Tizen is based on Linux kernel. Developers can use it to build native apps (C++), web apps (HTML5) or even hybrid apps.

Feb
2013

For mobile app development in .NET and C#, **Xamarin 2.0** is released. This includes Xamarin Studio, an IDE that integrates well with iOS and Android SDKs for cross-platform mobile app development. The release also allows developers to write native iOS apps in Visual Studio using C#. Xamarin as a company dates from May 2011.

2013

A startup named Drifty, founded in 2012, releases **Ionic**, an open source HTML5-based platform for building hybrid apps. Ionic itself is based on Cordova.

2015

Facebook open sources **React Native**, which is a framework to build cross-platform native mobile apps using JavaScript and React.

Feb
2018

At the Mobile World Congress 2018, Google releases the first beta of **Flutter**, a mobile UI framework to build native interfaces for both iOS and Android. Flutter is written in *Dart* programming language.

Types of Mobile Apps


# Levenshtein Distance

## Summary

Given two words, we can ask how similar are the two words. We can also ask this question of two sentences or string sequences. To quantify the similarity, we need a measure. Levenshtein Distance is such a measure.

By means of simple operations (such as insertion, deletion and substitution), we can determine how to transform one word or sequence into the other word or sequence. There could be many ways to achieve this. Levenshtein Distance is defined as the minimum number of operations required to make the two inputs equal. Lower the number, the more similar are the two inputs that are being compared.

There are a few algorithms to solve this distance problem. Levenshtein Distance is useful in many application domains including signal processing, natural language processing and computational biology.

Levenshtein Distance is also called **edit distance**.

## Discussion

### Could you explain Levenshtein Distance with an example?

Consider the two words "rain" and "shine". To transform "rain" into "shine", we can replace 'r' with 's', replace 'a' with 'h' and insert 'e'. Thus, the edit distance between these two words is 3. This is assuming that all operations have the same cost of 1. If we assign a higher cost to substitutions, say 2, then the edit distance becomes 2\*2 + 1 = 5. To transform "shine" to "rain", the operations are reversed (insertions become deletions) but the edit distance is the same when costs are symmetric.

Consider the words "train" and "shine". If each letter is replaced, we need 5 operations but this not the minimum. We can do better by aligning "in", doing one insertion, two substitutions and one deletion. Edit distance is therefore 4.

A lower distance implies greater similarity between the two words. In NLP, we generally wish to minimise the distance. In computational biology, we wish to maximize similarity. In error correcting codes, we wish to maximize the distance so that one codeword is not easily confused with another.

### What are "weights" in the context of edit distance?

What we call "weights" in NLP are called "scores" in computational biology. In the simplest case, we can give the same weights to insertions, deletions and substitutions, regardless of the symbol.

In reality, some errors are more likely that others. Consider how keys are laid out on a standard computer keyboard. It's easy to mistype an adjacent key. Some spelling errors are more likely than others. Likewise, in optical character recognition, it's possible to easily misinterpret between 'o' and 'e', or 'm' and 'n'. Likewise, in a DNA sequence some deletions or insertions are more likely than others. Weights help us account for these varying probabilities.

Mathematically, given two words of length N and M, D(N,M) is the distance. Words are 1-indexed but D(i,j) is 0-indexed. Accounting for the weights, edit distance can be computed this way:

*Initialization*

$$D(0,0) = 0\\D(i,0) = D(i-1,0) + del[x(i)]; 1 < i ≤ N\\D(0,j) = D(0,j-1) + ins[y(j)]; 1 < j ≤ M$$

*Recurrence Relation*

$$D(i,j) = min\begin{cases}{D(i-1,j) + del[x(i)]\\D(i,j-1) + ins[y(j)]\\D(i-1,j-1) + sub[x(i),y(j)]}\end{cases}$$

### Which are the algorithms that can compute the Levenshtein Distance?

There are many algorithms to compute the edit distance. Many of them are classified as **dynamic programming** algorithms. One of them is the Wagner-Fischer algorithm that we describe here.

The idea is to make a matrix of edit distances between all prefixes of one string and all prefixes of the other string. Levenshtein Distance is calculated by *flood filling*, that is, a path connecting cells of least edit distances. The approach is to start from upper left corner and move to the lower right corner. Moving horizontally implies insertion, vertically implies deletion, and diagonally implies substitution. Each cell minimizes the cost locally.

Consider the example of transforming "levenshtein" to "meilenstein" with equal weights of 1. There are actually two solutions, both having edit distance of 4. In one solution, we insert 'i' and replace 'v' with 'l': horizontal, then diagonal. In the other solution, we replace 'v' with 'i' and the insert 'l': diagonal, then horizonal.

If we wish to know the sequence of operations, we need to keep track of path, thus requiring more memory.

### What do you mean by Normalized Levenshtein Distance?

Consider two strings of same length 3 with edit distance of 2. Consider another two strings of same length 9 with edit distance of 3. We may say that the latter pair is more similar. To quantify the similarity, we *normalize* the edit distance.

One approach is to calculate edit distance as usual and then divide it by the number of operations, usually called the length of the edit path. This is called **edit distance with post-normalization**. The problem with this approach is that it may not produce the minimum normalized edit distance using Wagner-Fischer algorithm. Suppose weights are unequal for insertions/deletions and substitutions. It's possible to have a longer edit path that results in a lower normalized distance. **Normalized Edit Distance (NED)** algorithm finds this minimum.

For example, assume additions/deletions have a weight of 2, substitutions have a weight of 3. Consider the pair of words (abbb, aaab). When we normalize after Wagner-Fischer algorithm, we get 1.5 but with NED we get 1.33. It's also been observed that NED in most cases obeys the triangle inequality.

### How is Levenshtein Distance related to the triangle inequality?

In mathematics, edit distance can be seen as a *metric* in a metric space. In other words, the problem can be interpreted geometrically. The similarity between two words can be seen as the geometric distance between two points in the metric space. Such a metric obeys the **triangle inequality**. Given distance d, \(d(x,y) + d(y,z) \geq d(x,z)\). A composition of two edits can't be smaller than a direct edit.

Let's consider the pair (rcik, rick) whose edit distance is 2. Another pair (rick, irkc) has edit distance 3. Now consider a direct edit involving (rcik, irkc), which is 4. This direct edit is less than the sum of the other two edits. Levenshtein Distance is indeed a metric and it obeys the triangle inequality.

Wagner and Fischer formally proved this. In fact, their algorithm to compute the edit distance relies on this. The triangle inequality is satisfied due to the constraint that no position in a string is transformed more than once. This implies that all operations can be applied in parallel.

### Are there any variants of Levenshtein Distance?

Based on the work of Fred Damerau (1964) and V.I. Levenshtein (1965) **Damerau–Levenshtein Distance** is sometimes used instead of the classical edit distance. With Damerau–Levenshtein Distance, transpositions are also allowed where two adjacent symbols can be swapped. Consider the pair (rcik, irkc). This has an edit distance of 4, due to 4 substitutions. But if transpositions are allowed then the Damerau–Levenshtein distance is 3: `rcik -> rick -> irck -> irkc`.

For automatic speech recognition, Levenshtein Distance is calculated on words rather than characters. It's also been extended to three or more dimensions.

Levenshtein Distance has been adapted to Chinese language to extract person-affiliation relations. Words "爱(love)" and "喜欢(like)" are synonyms and should be seen as similar. Likewise, "the apples" and "the sweet apples" are not too different. We can use weights to account for these.

### What are some alternatives to Levenshtein Distance?

The choice of a suitable similarity metric depends on the application. If we're comparing two sets where order is irrelevant, then Jaccard Distance can be used. A similar measure to Jaccard Distance is Sørensen–Dice coefficient.

Longest Common Subsequence (LCS) allows only insertion and deletion. Jaro Distance allows only transposition. Hamming Distance allows only substitution and thus can be used to compare only strings of same length. Episode distance is a measure that allows only insertions.

If strings are treated as vectors, Cosine Similarity can be used. This is based on the angle between the two vectors.

### What are some applications of Levenshtein Distance?

Computing the Levenshtein Distance has also been called the *string-to-string correction problem*. It's relevant to string matching problems occurring in various domains such as information retrieval, pattern recognition, error correction, and molecular genetics.

In NLP, Levenshtein Distance is useful for suggesting spelling corrections, detecting plagiarism, and aiding translators in translation memory systems. For example, if a word is misspelt as "ligting", Levenshtein Distance can suggest that "lighting" is most similar, which helps the writer correct the spelling. It's also been used to cluster words that share a root word.

Levenshtein Distance in fact started in signal processing, in particular, to see how errors in communications systems can be corrected. Another application is speech recognition. A perfect match of an audio signal is impossible and edit distance can find the most suitable match.

DNA is a sequence of letters such as A, C, G, T. Searching for specific sequences is often difficult due to measurement errors, mutations or evolutionary alterations. Thus, similarity of two sequences using Levenshtein Distance is more useful than exact matches.

### What software tools are available for calculating Levenshtein Distance?

In Python's NLTK package, we can compute Levenshtein Distance between two strings using `nltk.edit_distance()`. We can optionally set a higher cost to substitutions. Another optional argument if set to true permits transpositions and thus helps us calculate the Damerau–Levenshtein Distance.

In TensorFlow, `tf.edit_distance()` gives us Levenshtein Distance and optionally normalize it. In R language, package *stringdist* can calculate many string edit distances.

In some applications, when comparing sentences or paragraphs, we may want to exclude some words, or invoke some pre-processing tasks such as lemmatization or stemming. In such cases, we could first tokenize the input using `nltk.word_tokenize(s1)` (in Python) before calculating the edit distance.

If we wish to compute Levenshtein Distance by implementing known algorithms, Rosetta Code has a page sharing implementations in many languages. A similar list of implementations is at Wikibooks. Some implementations are more memory efficient than others. It's said that most applications will use heap memory sparingly. In general, a naive recursive implementation will be inefficient compared to a dynamic programming approach.

## Milestones

1964

Fred Damerau at IBM looks the problem of matching an erroneous word to words in a dictionary. The error could be a missing character, an extra character, or a wrong character. By accounting for these error types, he was able to find a match in 95% of all error cases.

1965

In communication systems, information can get corrupted. Typically, 0s may becomes 1s, and vice versa. V.I. Levenshtein also considers the problem of extra bits getting introduced into the information stream or bits that go missing. He states that "the function r(x, y) defined on pairs of binary words as equal to the smallest number of deletions, insertions, and reversals that will transform x into y is a metric, and that a code K can correct s deletions, insertions, and reversals if and only if r(x, y) > 2s for any two different words x and y in K."

1974

The **Wagner-Fischer Algorithm** is invented to compute the edit distance between two strings. Algorithm has a time complexity of \(O(m\,n)\) when comparing two strings of length m and n. The term *edit distance* is also coined by Wagner and Fischer. P.H. Sellers coins *evolutionary distance* as an alternative term.

1975

Bahl and Jelinek provide a **stochastic** interpretation of edit distance. Called **similarity learning**, the algorithm will learn pairwise similarity of words by analysing the corpus. Each edit operation has a cost and the idea is to learn these costs, such as by maximizing the likelihood of data.

1980

Masek and Paterson design an algorithm that brings down the worst case time complexity to \(O(m\,n/log\_{\sigma}^{2}n)\) but requires \(O(n)\) more space. In practice, it can't beat the classical algorithm for text below 40GB.

1986

Oommen considers **constrained edit distance** where constraints are placed on edit operations. Example constraints could be "no more than k insertions" or "exactly k substitutions". They also give an algorithm that has \(O(m\,n\,min(m,n))\) time and space complexity.

1993

Marzal and Vidal define **Normalized Edit Distance (NED)** and an algorithm for computing it. In 2000, two researchers improve the time complexity of finding NED from \(O(m\,n^{2})\) to \(O(m\,n\,log\,n)\).

2007

Cormode and Muthukrishnan consider substring moves and approximation of edit distance. With these modifications, they show that time complexity can be sub-quadratic.

2015

Indyk and Bačkurs prove that the problem of finding the edit distance can't be solved in less than quadratic-time complexity. Thus, we can't do better than the Wagner-Fischer algorithm in terms of time complexity.

Levenshtein Distance