# Domain Name System

## Summary

Internet is a network of networks. Any computer or other network device participating on the Internet for providing or receiving services has an address called *IP address*. This address is used to communicate with other computers. But since it's difficult for human beings to remember IP addresses, the concept of Domain Name, or **Fully Qualified Domain Name (FQDN)** was introduced. This name is a sequence of human-readable characters that end with .com, .in, .org, etc.

The use of domain names is a problem for machines since they communicate using IP addresses. This is because the Internet is based on TCP/IP protocol, which uses IP addresses for identifying devices. Hence, domain names given by humans must be resolved to specific IP addresses. For this purpose, there are name servers. The entire system is called **Domain Name System (DNS)**.

## Discussion

### Could you explain DNS and why is it needed?

Because names are easier to remember than numbers, DNS is useful to refer to online resources by name. DNS is like a "phonebook for the Internet". Another benefit of DNS is that IP addresses of servers can be changed while retaining the same names. Moreover, a name can point to multiple IP addresses for redundancy and performance.

Even before DNS was invented, names were used instead of IP addresses but those names were not organized in a hierarchy. The mapping of names to IP addresses was also kept in a file called *HOSTS.TXT* that was almost manually updated and synchronized across network nodes. Thus, the system was centralized and did not scale well when Internet started to grow exponentially through the 1980s. DNS came in as a scalable solution. The system was designed to be distributed. Domain names were organized into hierarchical namespaces.

Two important aspects of DNS are zones and caching. **Zones** enable more flexible and granular management of domains. **Caching** improves response times by locally storing responses for future queries.

### Could you explain the DNS architecture and protocol flow?

DNS is a hierarchical system. It starts with a root name server below which are the top-level domain (TLD) servers. Examples of TLD are the traditional ones (.com, .net, .uk, .in, etc.) plus newer TLDs (.phone, .ieee, etc.) introduced since 2013. Below these are the second-level domain (SLD) servers, such as wikipedia.org, .co.uk, etc. Finally, domain name is mapped to IP address by the authoritative nameserver. For example, there will be an authoritative nameserver for bbc.co.uk.

Client initiates a DNS query, let's say when trying to access bbc.co.uk. Between the client and the DNS servers is the recursive resolver. It's job is to resolve the domain name to its requested IP address. For bbc.co.uk, the resolver contacts the root name server, which points to the TLD server for .uk TLD; resolver contacts the TLD server, which points to the SLD server that manages .co.uk; resolver contacts the SLD server to obtain the IP address of bbc.co.uk. It then caches it for future queries and forwards it to the client. Client uses this IP address to communicate with BBC's server.

### Could you explain the different types of DNS servers?

There are four types of DNS servers:

    + **Recursive Resolver**: Also called DNS Recursor, it's the first server to receive the DNS request. It mediates between the client and the nameservers. If it has cached data, it will use the cache. If not, it contacts a root server, then a TLD server, and finally the authoritative nameserver.
    + **Root Server**: Based on the TLD extension, it points the recursive server to the TLD server to contact. There are 13 root servers but due to redundancy there were actually 600+ servers back in October 2016.
    + **TLD Server**: Contains records for all domains belonging to that TLD. A TLD server will point the recursive server to contact the correct authoritative nameserver for the specific domain. TLD servers belong to one of two groups: *Generic TLD (gTLD)* (.com, .org, etc.) or *Country Code TLD (ccTLD)* (.uk, .in, etc.).
    + **Authoritative Nameserver**: This has the IP address of the requested domain name. If the domain has a alias record (CNAME), it will respond with the alias domain, which will make the recursive server to start a new DNS lookup.

### Which are the three Rs of DNS?

Three entities are involved in DNS:

    + **Registrant**: Anyone wishing to own a domain name for a period of time must make an application to a registrar. Suppose you wish to start an online business with address *myspecialcakes.co.uk*, you are a registrant.
    + **Registrar**: A registrar processes applications from registrants to register a domain name on their behalf. A registrar can be an ICANN accredited registrar for some TLDs and a reseller for other TLDs. If accredited, the registrar will directly interface with registries. If reseller, the registrar will interface with registrars accredited for those specific TLDs.
    + **Registry**: A registry manages the registration of all domains in a specific TLD. Each registry manages a single TLD, though a single company may run multiple registries. A registry consists of a database of all registered domain names, rules for domains, and sets registration prices.

### What's a zone in the context of DNS?

A zone is really an administrative space so that organizations and administrators can manage domains in a granular way. Each zone can be managed independently of others. A DNS zone can contain multiple subdomains and multiple zones can be on the same server. Zones are not about physical or geographic separation. Rather, they are used for delegating control.

Domains within a zone have to be contiguous. For example, a zone cannot contain only domains "admin.coatbank.com" and "finance.coatbank.com" without also including "coatbank.com". In another example, "blog.cloudflare.com" can be in a separate zone while all other subdomains of Cloudflare can be in another zone.

A **DNS Zone File** is a plain text file saved on a DNS server. This file contains all records for every domain within the zone. A **DNS Record** is a single entry that maps a name to its IP address. A zone file stores different types of records but always starts with SOA (Start of Authority) record. A zone file can contain only one SOA record.

### What are primary and secondary DNS servers?

DNS is a critical aspect of the Internet. If it fails, many Internet services will be affected for any given company: its website, support services, email servers, sales portals, online resource libraries, databases, multiplayer games, etc.

In one test that queried 100,000 authoritative servers for .ca domains, 93% failed to respond at least once. This implies that for reliable Internet services we need redundancy. For this reason, we have secondary servers to take over in case primary servers are down. Secondary servers also help with load balancing to prevent denial-of-service situations.

Each zone can have only one primary DNS server but any number of secondary servers. The controlling zone file is on the primary server. Secondary servers store read-only copies of the controlling zone file via a communication procedure called **Zone Transfer**. A primary server of one zone can also be a secondary server for another zone.

### What are the different types of DNS Records?

Among the different types are Start of Authority (SOA), Name Server (NS), Mail Exchange (MX), Address (A), AAAA, Canonical Name (CNAME), Alias (ALIAS), Text (TXT), Service Locator (SRV), Pointer (PTR), and more.

When recursive name servers need to contact authoritative name servers NS records become useful. The IP address of a domain comes from A. Likewise, for IPv6 address, AAAA records are used. Sometimes a domain name may be redirected to another name, such as wikipedia.com redirecting to wikipedia.org. CNAME and ALIAS records help in implementing these aliases.

### What are some practical challenges with DNS?

Managing DNS is not trivial. DNS is a combination of system administration and network management. Expert IT staff are needed. While DNS was designed for addressing, it's becoming overloaded. It's getting tied to applications and application protocols. The original design assumed DNS would be deeply hierarchical with high ratio of physical hosts to second-level domains. However, it's turned out to be much flatter with SOA count possibly exceeding number of physical hosts.

DNS has security issues. DNS Spoofing can direct users to malicious websites by tampering the cache entries. This can be fixed by deploying DNSSEC.

When changes happen in a DNS zone, such as a website moving to another data centre, other servers may not get the update for as much as 24 hours.

Although DNS is a distributed system, the approval of TLD names is done by the ICANN. Until 2016, this entity was influenced by the U.S. government. Even today, it's said that DNS servers are controlled by governments and large corporations. For this reason, .bit domain names have been created. Dot-Bit domains bypass DNS. They are powered by Namecoin, which is based on Bitcoin.

### What are the RFCs and specifications relating to DNS?

DNS is defined by Request for Comments (RFC) documents published by the Internet Engineering Task Force (IETF). A list of DNS RFCs is available online. We mention a few important ones:

    + RFC 920: Domain Requirements
    + RFC 1034: Domain Names — Concepts and Facilities
    + RFC 1035: Domain Names — Implementation and Specification
    + RFC 1123: Requirements for Internet Hosts — Application and Support

## Milestones

Nov
1983

Paul Mockapetris invents DNS since he finds that other proposals do not meet the requirements of the Internet. He also starts writing *Jeeves*, the first DNS server implementation. IETF publishes two documents on DNS: RFC 882 for concepts and facilities; and RFC 883 for implementation and specification. These early documents are replaced with RFC 1034 and RFC 1035 in November 1987.

Oct
1984

With the publication of *RFC 920: Domain Requirements*, this is the "official" beginning of DNS. This document mentions top-level domain names GOV, EDU, COM, MIL, and ORG. It gives example names: CCN.OAC.LA.UC.EDU, DASH.MIT.EDU, HP-LABS.CSNET.

1985

Around the mid-1980s, **Berkeley Internet Name Domain (BIND)** is created at the University of California at Berkeley. It's an implementation of a DNS server for the UNIX platform. In subsequent years, BIND goes on to become the most widely used DNS software. In September 2000, BIND version 9 is released. This is a major rewrite and leads to deprecation of versions 4 and 8.

Mar
1985

Computer manufacturer Symbolics registers **symbolic.com**, making it the world's first registered domain within DNS. The name is sold almost 25 years later in 2009. While many older hosts continue to use HOSTS.TXT even in the late 1980s, by 1985, some hosts use DNS as the sole means of accessing names.

1995

Network Solutions Inc. (NSI) that manages domain names starts charging for domain name registration. Earlier, domain name registration was free and managed by the U.S. government.

Nov
1998

**The Internet Corporation for Assigned Names and Numbers (ICANN)** is formed to coordinate DNS addressing structure including managing the top-level domain (TLD) name space. ICANN takes over from the Internet Assigned Numbers Authority (IANA). While ICANN is a private entity, its operations are overseen by the US Department of Commerce (DoC).

2012

Mike Mann registers 14,962 domain names in 24 hours, spending about $100,000. It's speculative but buying and selling domain names is big business.

Apr
2013

The original creator of DNS, Paul Mockapetris, says in an interview that it's time for DNS 2.0 that includes security,

> We need to get to the next level of naming, which combines authentication, but more importantly, a reputation system.

Oct
2013

The first four new generic Top-Level Domains (gTLDs) are announced. This is followed by another 69 names in 2013, 406 names in 2014, 391 names in 2015, and 340 names in 2016. Generic top-level domains (gTLDs) are known to users as the text coming at the end of a URL such as COM, ORG, or EDU. Some examples of new gTLDs are PHONE, IEEE, SONG, HYUNDAI, GUIDE.

2014

A study of a regulatory filing reveals **Cars.com** as the most expensive domain name to be sold, valued at $872 million. Other record sales publicly reported till October 2018 include CarInsurance.com ($49.7m), Insurance.com ($35.6m), and VacationRentals.com ($35m).

Sep
2016

ICANN's contract with the U.S Department of Commerce expires. This completes the privatization of ICANN that was envisioned in 1998.

Domain Name System


# R (Language)

## Summary

R is a free software environment for statistical computing and graphics. This definition implies that R is open source, is designed for statistical computing and has strong graphing capabilities. R is a language but also an environment in the sense that it's for users who wish to do interactive statistical analysis and modelling, but who are not necessarily programmers.

R is based on S, a language from the 1970s. While R has evolved a lot, it still retains many of the design constructs of its early days. In recent times, adoption of R is growing due to interest in Big Data and Data Science. However, the use of R should be seen as complementary to other languages such as Python.

## Discussion

### How would you describe R?

R is a language that's influenced by two different paradigms: object-oriented programming and functional programming. Everything in R is an object. Computations in R are function calls. In fact, function definitions and function calls are also objects. It's also an interpreted language (no compilation is required).

Object references in R is a combination of name and context called *environment*. Changing an object by local references will affect the object only within the local environment. A function call translates to the creation of a new environment. Hence, phrases "call by value" or "call by reference" common in other languages, are not applicable in R.

There are no scalar types in R. Vector-based types are used. Lazy arguments are used for functions. In other words, the arguments are evaluated inside the functions only when required.

### What are the advantages of R?

Unlike S-PLUS, SAS or SPSS, R is open source and can be used without any licensing fee. R is cross-platform: it can be used on Linux, Windows or MAC OS.

R is supported by a large community. Along with frequent releases by a core team, the wider community has released thousands of packages that are available at the Comprehensive R Archive Network (CRAN). Beyond this community of volunteers, R is backed by big companies. The R Foundation and the R Consortium are committed to the evolution of R.

R and many third-party packages are suited for data analysis and modelling. R simplifies the work of a data scientist by offering tools for easy data manipulation, visualization and modeling. At its core, R operations are vectorized. This means that vectors and matrices can be easily handled. Because it's interpreted, R is highly interactive. R users can start using R interactively, and as they gather more expertise, they can do more complex things by writing R scripts.

### What are the typical applications where R is used?

R is popular among data scientists because of the many statistical packages that it offers. Quantitative analysts use R. It's been used in diverse fields: biotech, finance, research, high technology industries, and more. Anywhere there's a need to produce production-quality plotting, R is a suitable choice. Likewise, in the area of Machine Learning R has an advantage because of its strong ties with academia.

In econometrics, one researcher explained that R enabled him to do complex regressions. He was able to do this via custom matrix algebra and validate the results against those offered by R packages. When there's a need to look at complex statistical properties, R has an advantage.

According to data scientist Matt Adams,

> I wouldn't even say R is for programmers. It's best suited for people that have data-oriented problems they're trying to solve, regardless of their programming aptitude.

### What are some criticisms of R?

R is said to be poor in memory management. Because of the requirement of environments, all objects must be stored internally rather than as files on disk. In recent times, this limitation has been somewhat remedied. Newer developments from Microsoft are expected to address this as well: ParallelR for distributed computing, Rhadoop for running on Hadoop nodes, Reproducible R Toolkit and RevoPemaR for writing parallel external memory algorithms. Also, there are packages and tools to handle Big Data in R.

R is worse in terms of performance and lacks proper unit testing frameworks.

Security was not built into R at the design stage. This means that we can't use it via web server or embed it in a web browser. However, the use of isolated virtual containers can provide the necessary security to make this possible in a world of cloud computing. Likewise, Shiny is an R package that can be used to create interactive web applications. DeployR is another package for bringing R analytics to web dashboards.

### For Data Science, should I choose R or Python?

Both R and Python are open source. Both are maintained by active communities. Both are interpreted languages. Both can be used interactively. Both have good tools to work with.

While Python has 138,000+ packages, R has 12,000+ packages (May 2018). This is expected since Python is versatile while R is more focused on statistical computing. R is said to have more statistical packages that suit the work of a data scientist. However, Python's capability is getting better due to NumPy, Pandas, scikit-learn, Keras, and TensorFlow. Python's graphical capabilities are also improving with matplotlib, Bokeh, Seaborn, and Plotly.

Because Python is more object-oriented, it may be a better choice for large projects. If the main work is data analysis, R must be preferred. If work involves gathering and cleaning data as well, Python may be a better choice. Having said that, statisticians tend to prefer R while computer scientists go with Python.

A mix of the two, each suited to its strengths, is a good choice. Python could be used for preprocessing data while R could do the statistical analysis.

### As a beginner, how can I get started with R?

Download and install R. This is sufficient to execute R commands and scripts. For better developer experience, download and install RStudio. This is an IDE that integrates a text editor with syntax checking, R console to execute commands, easy access to documentation, a panel to view variables for debugging, a plot viewer, and more.

Read about most useful third-party packages. Install these from CRAN as required by your projects. One approach is to install the tidyverse collection of packages. This will install a number of packages useful for data science work.

Learn one or more of R Markdown, Shiny, Knitr and Bookdown. These will help you to document your code, its results and share the same with others in various formats.

Learn the language syntax. Get familiar with R data types and structures. In particular, learn about vectors, lists and data frames. Next, learn about R's graphing capabilities. Common plotting systems to learn are *Base* and *ggplot2*.

## Milestones

1976

A new language by the name of *S* is initiated in Bell Labs for statistical computations. It's written in Fortran. In 1988, it's rewritten in C. *S-PLUS* is a commercial implementation of *S*.

Aug
1993

*R* language is announced to the public by Ross Ihaka and Robert Gentleman in the Department of Statistics at the University of Auckland. Their work on R started in 1991. For portability, R is coded in ANSI standard C. In syntax, R resembles S but the underlying semantics is based on Scheme. Ihaka mentions,

> We implemented the language by first writing an interpreter for a Scheme subset and then progressively mutating it to resemble S.

Jun
1995

R source code is released to all under the Free Software Foundation's GNU GPL license.

1997

*R Core Group* is formed to handle change requests. Group members are volunteers and contribute whenever possible.

Feb
2000

R version 1.0.0 is released to the public.

2015

*R Consortium* is founded to "support the worldwide community of users, maintainers and developers of R software". The Consortium collaborates with *R Foundation* (founded in 2003), the governing body of the R Project.

2015

Microsoft acquires Revolution Analytics, which offers both community and enterprise distributions of R. In January 2016, Microsoft announces that Revolution products are renamed/integrated into Microsoft's products.

Apr
2018

R version 3.5.0 is released. In May 2018, *Microsoft R Open (MRO)* version 3.5.0 becomes available.

May
2018

There are about 12,500+ packages on the *The Comprehensive R Archive Network (CRAN)*. A similar number is listed on the *Microsoft R Application Network (MRAN)*.

R (Language)


# Node.js

## Summary

Since its birth in 1995, JavaScript has established itself as the de facto programming language on the client side of a web application. It executes within a web browser and thereby delivers more dynamic content and makes user experience more interactive. However, to execute code on the server side, we had PHP, ASP, Perl, Python, etc. Until Node.js, there was no popular JavaScript support on the server side.

Node.js is a JavaScript runtime that can be executed on servers, that is, outside a web browser environment. By default, it's based on Google's V8 JavaScript engine. It's open source and cross platform. It's steered by the Node.js Foundation.

Node.js adopts an event-driven architecture and asynchronous I/O. This enables it to scale better, particularly for real-time web apps or apps that do a lot of I/O. The Node.js ecosystem includes thousands of packages and frameworks.

## Discussion

### What's the context for the creation of Node.js?

The web was invented based on the client-server model. Clients (such as web browsers) request the server for a resource. The server serves the content as a response. In this model, the server cannot initiate a response. Even if data has changed on the server, it has to wait for the client to give a request. This was alright when users were passive consumers of information.

When social sites such as Facebook and Twitter arrived, everyone became a potential producer of information. There was a rise of chat apps, multi-user gaming and apps for real-time collaboration. We needed a method for servers to push data to clients. While Flash and Java Applets enabled this, they were isolated environments that used non-standard ports.

Secondly, typical web servers launch a new thread to handle every client request. This means that scalability is limited by memory and the penalty of switching across threads.

Node.js solves these problems by using single-threaded event-driven processing. It offloads I/O tasks to worker threads and processes them asynchronously via callbacks. Ultimately, this makes it easier for us to develop scalable real-time apps.

### What sort of applications can benefit from Node.js?

Node.js is suited for apps that have lots of database accesses or waits on network connections. When there's a need to scale your app to thousands of parallel requests, Node.js does this well without hitting performance limits. On the flip side, Node.js is not suited for CPU-intensive tasks, which will block the main thread.

With Node.js, stream processing of data is possible. Worker threads do this processing. It's also good for queueing inputs for later batch processing.

Node.js with Socket.IO makes it easy to build real-time apps such as chat apps or games. It can be used as a proxy to handle many simultaneous connections. Apps with rich real-time dashboards can be created. Microservices architecture can be implemented with Node.js.

If your codebase spans both client and server concerns, Node.js helps you to write in a single language, JavaScript. This makes it easier to later refactor the code if necessary. JavaScript is a natural fit for transferring JSON data or working with object databases such as MongoDB without the extra processing of encoding or decoding.

### How does Node.js work under the hood?

When a Node app starts, callbacks are registered for events. A **callback** is nothing more than executable code that must be called when the event occurs. After this initialization phase, Node enter the **Event Loop**. In this loop, Node waits for events to happen and trigger the appropriate callbacks. It's for this reason Node.js is said to use **event-driven programming** paradigm.

For example, a client requests a web page. Event loop will invoke the associated callback. This is **synchronous**: event loop will wait for the callback to complete. However, if the callback requires some data from a database, the event loop being a single thread will get **blocked** since I/O is slower compared to CPU. Other events will start queueing up.

To prevent this, Node.js adopts **non-blocking asynchronous I/O**: slow tasks will be handed over to the **Worker Pool**. This frees the event loop to move on the next event. When a worker completes its task, it will signal an event, at which point, the event loop will process its callback synchronously.

### Could you share more details on the event loop?

Event loop is also called main loop, main thread, or event thread. It's where code executes synchronously, that is, the event loop will wait until execution completes. If there are asynchronous calls to be made, event loop will register their callbacks and handover the tasks to worker pool. The idea is to perform only lightweight tasks on the event loop so that it can handle as many requests as possible. Throughput (requests per second) is an important concern for the event loop.

It's important to note that the application and the event loop run within a single thread. All events are processed in the same thread. Pending events are not stored in a single queue. Rather, the event loop is a set of phases, with each phase having events queued for processing.

### Could you share more details on workers?

Worker Pool manages a number of threads that can be used for both I/O-intensive or CPU-intensive tasks. The worker pool maintains a queue of tasks to be processed. When a worker becomes available, it's assigned a task.

Both event loop and worker threads make use of `libuv`, a multi-platform support library for asynchronous I/O. It's implemented in C. Since operating systems often provide asynchronous interfaces, libuv will use these in preference to its own thread pool.

Since Node v10.5.0, on an experimental basis, it's been possible to create pools, specify the number of workers or have control of one worker communicating with another. This is part of the `worker-threads-pool` package.

### For my next web app, should I use Node.js or Express.js?

Express is a web application framework built on top of Node. You can build a web app with either of these but development will be faster when built with Express. Even better is to adopt the **MEAN** stack. This is a technology stack that combines MongoDB, Express, Angular and Node.

Having said this, developers have a choice. Express is not the only Node.js framework. Others include hapi, koa or Sails. For fullstack, Meteor, Feathers or Keystone are alternatives to MEAN.

What if you're building a REST API? In this case, performance is important. Express may be too heavy and slow for building REST API endpoints or an API gateway. You could use Node directly but there are Node frameworks that are minimalistic and optimized for REST APIs: fastify, restana, restify, koa, polka.

### What are some success stories with Node.js adoption?

Paypal built a Node.js app in half the time with 33% fewer lines of code compared to its Java codebase. LinkedIn's mobile app using Node.js backend is 20x faster and uses only 3 servers compared to 30 that Ruby on Rails needed. At Netflix, Node.js has reduced app startup time by 70%. Groupon web pages are 50% faster with Node.js compared to Rails. They can also serve much higher traffic. At GoDaddy, they now require 10x fewer servers and time-to-first-byte is down to 12ms from 60ms.

Uber is able to process 2 million remote procedure calls per second, thanks to Node.js. Medium is now able to deploy faster. At NASA, by moving to a single database and Node.js they reduced access times by 300%.

### What are some useful resources for a Node.js developer?

Guides and official documentation are available from the main Node.js site. For latest news, you can follow the Node.js Foundation website. The Foundation's site includes resources and case studies.

Among the package managers for Node.js are npm and yarn. As of October 2018, the npm site hosts about 800,000 JavaScript packages. Node Frameworks gives a list of frameworks to consider. To install and manage multiple versions of Node, use Node Version Manager (NVM).

A video series by Net Ninja titled Node JS Tutorial for Beginners is a good place to start for beginners.

A curated list of Node.js resources is available from Angular Minds.

## Milestones

Dec
1995

Netscape Communications and Sun Microsystems announce JavaScript to the world. This includes **Netscape LiveWire** that allows server-side execution of JavaScript. Hence, Node.js is certainly not the first to enable server-side JS execution.

2009

Ryan Dahl and others at Joyent develop Node.js with initial support for only Linux. The name *Node* is coined in March. Being open source, version 0.1.0 is released on GitHub in May. In November, Ryan Dahl presents Node.js at JSConf in Berlin.

2010

As a web development framework based on Node.js, **Express.js** is released. For real-time event-based communication, **Socket.IO** is released.

2011

To manage packages for Node.js, **Node Package Manager (NPM)** is released. An early preview of NPM can be traced to 2009, by Isaac Z. Schlueter.

Dec
2014

Node.js community undergoes a split, with the fork named **io.js**. This is because some feel that Node.js updates are slow, and even after five years the library is still in version 0.x. Some also note that Node.js is controlled by Joyent and it's less open than desired. In January 2015, v1.0.0 of io.js is released.

Feb
2015

Joyent, IBM, Microsoft, PayPal, Fidelity, SAP and The Linux Foundation come together to establish **Node.js Foundation**. The Foundation will see that Node.js is managed in a more open manner and have regular releases. In June, Linux Foundation announces that Node.js and io.js will merge their codebases.

Sep
2015

Version 4.0 is released. This is a combined release of both Node.js and io.js. It includes many ES6 features. Thus, the much awaited version 1.x of Node.js does not happen. Version 4.0 is so numbered probably because the last version of io.js is v3.3.1.

Oct
2015

Node.js v4.2.0 "Argon" is released as the first Long Term Support (LTS) release. This means that it will be supported for 30 months.

Jan
2016

Microsoft releases and open sources **ChakraCore**, a JavaScript engine for Node.js. This is therefore an alternative to V8 engine that's been the main runtime since the birth of Node.js.

Oct
2016

Developers at Facebook release **Yarn** as an alternative to npm.

2017

Node.js is seen to have reached mainstream adoption. Online instances reach 8.8 million with 3 billion npm packages downloaded per week. Node.js is downloaded 25 million times this year, including a million in a single day.

Apr
2018

With Node.js version 9, HTTP/2 is now supported. This was available since July 2017 on an experimental basis.

Node.js


# Design Thinking for Requirements Engineering

## Summary

**Design Thinking (DT)** comes from the design discipline. It explores both the problem space and the solution space. Its attributes include user needs, prototyping, feedback, iterations and multidisciplinary perspectives. **Requirements Engineering (RE)** comes from the engineering discipline. It too considers user needs but it's more technical and formal. Requirements are documented as formal specifications.

Identifying requirements is a complex task and DT is good at tackling complex problems. DT and RE complement each other. In one model, DT is applied to elicit requirements that feed into RE. However, teams can switch between DT and RE as the situation demands.

Research in DT for RE is still nascent and perhaps more so in practice. For example, can DT be applied to other software engineering (SE) tasks? How do DT principles map to SE principles? These are open research questions.

## Discussion

### Why do we need design thinking for requirements engineering?

In RE, the perspective is mainly **technical**. Even when engineers apply human-computer-interaction to create user interfaces/interactions, they may disappoint users. This is because they fail to see the user perspective. Users may not articulate their needs precisely. DT can bring out underlying needs. It's said that asking questions isn't enough to identify user needs. They must be tracked down.

Engineers are guided by **analytical thinking**, both deductive and inductive logic. Design thinking is more open, collaborative and multidisciplinary. DT is better at creativity and innovation. DT helps RE with a better understanding of user and domain. The learning curve associated with DT gives engineers an appreciation of how and why each requirement has emerged. This aids them in implementation.

RE emphasizes formal and elaborate **documentation**. This hinders quick feedback and short iterations. There have been cases of extensive documentation of poorly understood user needs, resulting in unusable code.

### Hasn't Agile already solved the RE problem?

In the mid-80s, it was recognized that the traditional waterfall model isn't adequate. It considered user perspectives only at the start and the end of projects. Agile practices such as Extreme Programming, Feature-Driven Development, Kanban and Scrum were introduced. Agile emphasized working software over formal documentation, customer collaboration over contract negotiation.

But RE in Agile has its limitations: minimal documentation of requirements, unclear requirements, neglect of non-functional requirements, tacit requirements, wrong prioritization.

While Agile does prototyping, it uses the same materials as the final product. The prototype can then be iterated towards the final product. The focus is not on learning, which DT fosters.

In Agile, user scenarios are inputs to modelling. In DT, scenarios are a tool towards creativity. Scenarios are related to personas to give narrative power. Modelling is less important in DT.

Agile practices fall short in terms exploring the problem/solution space. Agile emphasizes incremental refinements on a chosen path. It's been said,

> Agile is always looking to remove options from the table. Design thinking is always trying to keep options on the table as long as possible.

### Could you give an example of DT for RE?

Consider the figure above. The telephone layout makes sense but the calculator layout is in reverse. Calculators evolved partly from cash registers such as the Comptometer (1885). One probable explanation is that levers and rotating drums beneath the buttons implemented the *method of complements*. Technologically, it was easier to implement this in the reverse layout. Today's familiar 3x3 calculator layout first appeared in 1914, a David Sundstrand invention.

The telephone's layout came about in the 1960s. Under the discipline of *human factors engineering*, engineers studied as many as 15 layouts and sought user feedback. The study revealed that users preferred left-to-right, top-to-bottom layout. Layouts 2x5 and 3x3+1 were equally efficient. AT&T adopted the latter, which became the standard for future pushbutton handsets.

This example shows that telephone engineers considered user needs and used prototyping to test ideas. Interestingly, modern calculators continue to use the old layout even when the original technical constraints are gone. We may say that this too is user-centric since users are used to this layout.

### What are the pros and cons of applying DT to RE?

It's been seen that DT promotes better perception of business requirements, better collaboration with colleagues and users, learning for all stakeholders, and delivering value. In the problem space, user needs are better understood. DT fosters creativity and problem reframing. DT achieves better understanding of usability and user experience. RE takes this forward towards design and architecture.

Bringing a multidisciplinary team together is a challenge. Organizational culture may come in the way of open discussions. Not enough time is given. Calls or emails may interrupt people high in the hierarchy. There's a tendency to adopt familiar solutions without adequately understanding the problem. People may not engage simply because they don't see the value. While DT is strong on usability, it fails to capture other non-functional requirements such as security and performance. Since DT is unstructured (uses post-its and low-fidelity prototypes), tracing requirements is not easy. Requirements prioritization and effort estimation are hard to do.

### What's a suitable model for DT in RE?

One proposed model structures the process in three layers: context layer (why the system is needed), requirements layer (what are the user-level requirements and features), and system layer (how the system will be realized). Each layer produces artefacts that feed into the next layer. At the context layer, we apply the DT process. At the requirements layer, detailed technical specifications are produced.

However, the process is not linear. We adopt DT whenever we need a better understanding of user needs or the business context. When we need a more technical view or feasibility analysis, we adopt RE. Both DT and RE are complementary tools that help us customize the process as the situation demands.

### How can an organization adopt DT for RE?

DT can be combined with RE in three ways:

    + **Upfront**: DT is applied before RE. DT is a guiding process. It's useful when there's high uncertainty about the problem or solution.
    + **Infused**: DT is applied to specific RE tasks to clarify those tasks. DT is a toolbox in RE.
    + **Integrated**: DT is a guiding mindset that's continuously applied to the entire RE process. Organizations may start with the infused approach and then move towards the integrated approach.Organizations prefer streamlined processes and quality gates. They have to meet deadlines and budgets. The exploratory and creative nature of DT is perceived as a risk. For these reasons, DT is better applied as a frontend technique rather than fully integrated into RE. DT can be selectively applied to fuzzy requirements. However, as a front-end technique, moving from concept to implementation is a challenge. For this reason, a fully integrated approach is better in the long term.

### What techniques or tools are suited for DT in RE?

The figure shows a selection of tools as applied by DT for RE. User personas help bring out specific needs based on user preferences or behaviours. Prototypes help gather user feedback and refine those needs. User journeys help determine the steps the user will take to fulfil those needs. Service blueprints show interfaces and interactions through which users can realize their journeys.

In fact, design thinking has many more tools that can be used in the phases of immersion, ideation and implementation. Any of these could be used for RE. Some of these are stakeholder mapping, desk research, framing/reframing, interviewing, observation, active listening, clustering, storytelling, brainstorming, brainwriting, role playing, sketching, and feedback capture grid.

Beyond the tools, individuals must be **T-shaped**: able to look at problems horizontally (multidisciplinary and broad perspectives) and vertically (specialized and deep expertise). To support collaboration and open discussions, a DT team without a designated leader is preferred. The office environment should also support collaborative behaviours.

## Milestones

1997

Holzmann et al. publish a paper titled *Design Tools for Requirements Engineering*. They note that many requirements can be captured in **Message Sequence Charts (MSCs)**. MSCs can be formally analyzed for causal structures and race conditions. POGA and TEMPLE are tools to analyze MSCs. However, the use of design tools doesn't equate to design thinking. Requirements engineering in this paper is still rooted in technical perspectives. Sutcliffe presents an alternate approach that uses prototypes, design rationales, and user scenarios: these are closer to design thinking.

2000

This decade sees more awareness among engineers that **user perspective** is important. New roles are introduced: user-interface designer, interaction designer, and user-experience designer. User-centered design is about translating user needs into software design. User experience design has broader scope. However, these roles and practices don't promote shared understanding or collaboration within the team. Design thinking achieves these via a multidisciplinary approach.

2011

Lindberg et al. at the Hasso Plattner Institut publish a paper titled *Design Thinking: A Fruitful Concept for IT Development?*. The paper considers all phases of software development and not just requirements engineering. They note that engineers apply analytical thinking and their perspective is mainly technical. This is where design thinking can help. It's relevant to note that the Hasso Plattner Institute of Design at Stanford University (d.school) was founded in 2003.

Mar
2013

Vetterli et al. explain that RE was a good approach to developing large enterprise software and backend systems. Increasingly app-driven products are coming out. These need faster iterations. Agile development came as a solution but addressed only functional requirements of users. To capture, non-functional requirements DT can help. DT offers methods to elicit user needs. DT is about iterating, learning and ultimately capturing user needs. They note that, "Unlike requirements engineering, design thinking aims to fail early in order to succeed sooner".

May
2013

Soledade et al. study the use of design thinking for a Learning Management System (LMS). They use mockups, interviews and feedback forms. Users, mediators and observers take part. They find that design thinking brought better alignment of proposals with user expectations. Design thinking led to refinement of functional and non-functional requirements. Another finding is that developers might invest lot of time and effort on some features that aren't necessarily noticed by users.

2018

Hehn et al. publish a technical paper that combines design thinking and requirements engineering. Hehn's research leads to further papers and publication of the book *Design Thinking for Software Engineering* in 2022, with Hehn as one of its editors.

May
2020

In the context of Scrum, Alhazmi and Huang propose that any item entering the product backlog for the next sprint shall be derived from user stories. User stories are generated via design thinking. Change requests and bug reports shall go through the design thinking process before getting scheduled for the next sprint.

2022

From a literature review and industry survey, it's found that as many as 86 creativity and DT techniques exist for eliciting requirements. Individuals are more familiar with DT techniques than creativity techniques. Most used DT techniques are interview, brainstorming, uses cases, activity analysis, user story, and rapid prototyping.


# Reinforcement Learning

## Summary

Reinforcement Learning (RL) is a subset of Machine Learning (ML). Whereas supervised ML learns from labelled data and unsupervised ML finds hidden patterns in data, RL learns by **interacting with a dynamic environment**.

Humans learn from experience. A parent may reward her child for getting good grades, or punish for bad grades. By interacting with peers, parents and teachers, the child learns what habits lead to good grades and what lead to bad grades. It learns to follow good habits to obtain good grades and higher rewards. In RL, this sort of feedback is called **reward** or **reinforcement**. The essence of RL is to learn how to act or behave in order to maximize rewards.

A suitable definition is that,

> Reinforcement learning is the problem faced by an agent that must learn behaviour through trial-and-error interactions with a dynamic environment.

## Discussion

### Could you explain reinforcement learning with an example?

Thomas Edison, the inventor of the light bulb, reportedly performed thousands of experiments before arriving at a carbon filament taken from a shred of bamboo. Edison said, "I have not failed 10,000 times. I have succeeded in proving that those 10,000 ways will not work." This anecdote is relevant to reinforcement learning. RL welcomes mistakes. RL is about learning what works and what doesn't through many trial-and-error experiments.

In the game of Pong, a ball bounces back and forth between two paddles. Our RL-trained agent software controls one paddle. The rewards in this simple game are clear: +1 if the ball beats the opponent, -1 if we missed the ball, 0 otherwise. Our agent can see current state of the game in terms of pixels. Based on this input and what the agent has learned so far, it decides if it has move its paddle up or down. The agent may lose the game many times but via negative rewards it will learn to avoid those paddle actions. Likewise, it will learn what paddle actions lead to high rewards.

### Why do we need reinforcement learning?

Reinforcement learning differs from other ML approaches. Unlike supervised learning, there's no supervision in RL, only a reward signal. Feedback is delayed, not instantaneous. In a chess game, long-term reward is apparent only after a series of moves.

In supervised or unsupervised learning, a static dataset is usually the input. Reinforcement learning happens within a dynamic environment. Data is not independent and identically distributed. The agent can take many different exploratory paths through the environment. Immediate action affects the subsequent data the agent receives.

In many complex problems, the only way to learn is by interacting with the environment. In complex board games, it's hard for humans to provide evaluations of a large number of positions. It's easier to learn the evaluation function via rewards.

### Which are some practical applications of reinforcement learning?

RL has wide application in many fields: robotics and industrial automation, data science and machine learning, personalized education and training, healthcare, natural language processing, media and advertising, trading and finance, autonomous driving, gaming and many more.

RL can **optimize operations** in many industries. Google used it to reduce energy consumption in its data centers. Process planning, demand forecasting, warehouse operations (such as picking and placing), fleet logistics, inventory management, delivery management, fault detection, camera tuning, and computer networking are some applications that RL can optimize.

We note a few specific areas:

    + **Machine Learning**: For a given problem, identify suitable neural network architectures. Google's AutoML based on RL does exactly this. RL models could assist software developers to write computer programs.
    + **NLP**: Generate summarizes from long text. Assist chatbots to learn from user interactions and respond to queries in a more natural manner. Useful for question answering and machine translation.
    + **Media & Advertising**: Give personalized content recommendations. Use in cross-channel marketing optimization and real-time bidding systems. Optimize video streaming quality. Deliver more meaningful notifications to users.
    + **Healthcare**: Dynamic treatment regimes, automated medical diagnosis, process control, drug discovery, health management.

### How is reinforcement learning related to other fields?

RL has interesting parallels with other fields:

    + **Neuroscience**: The brain learns via reinforcement. Synaptic associations are strengthened or weakened based on conditioned and unconditioned stimuli. In humans, dopamine serves as the reward.
    + **Psychology**: New behaviours are learned via associations. Reward and punishment are used to modify behaviour. Operant conditioning closely relates to RL.
    + **Economics**: Behaviour economics is concerned with decision making. While rational economic agents maximize utility, humans are not fully rational. We often optimize for satisfaction. The similarity with RL is that agents often have incomplete information of the environment.
    + **Mathematics**: Operations Research (OR) is a field that's similar to RL. OR employs analytical methods to aid decision making. System simulations leading to reward maximization or cost minimization are OR concerns.
    + **Engineering**: Control problems involve a cost function of state and variables. Differential equations model the path of control variables. Equivalent RL concepts are policy (controller), actions (actuator commands), observations (state feedback) and reward/observations (reference signal).

### Which are some essential terms in RL?

It's easier to understand RL, once we get familiar with these essential terms:

    + **Agent**: An entity that's being trained to perform a specific task by interacting with the environment. It's sometimes called *controller*.
    + **Environment**: This is everything except the agent. This includes system dynamics.
    + **State**: Parameter values that define the current configuration of the environment.
    + **Action**: An agent outputs an action to the environment and thereby changes the environment's state. Action itself is determined by observed states and policy.
    + **Reward**: The numerical result of taking an action in a given state. *Return* or *utility* is the sum of current and future rewards when following a sequence of states. Utility is also called *long-term reward*.
    + **Policy**: A function that takes state observations as inputs and outputs actions. It has a logical structure and tunable parameters that the agent learns. The agent's goal is to learn the optimal policy to maximize reward or utility. Equivalently, it's a probabilistic mapping from states to actions.

### How does learning happen in RL?

The agent starts by observing the environment's state. Based on the current policy, the agent acts. This action changes the environment's state. In return, the agent gets a reward. From the reward, the agent determines if the action was beneficial. A positive reward reinforces the action or agent's behaviour. A negative reward informs the agent to avoid such an action, at least in a particular state. This feedback cycle of observe-act-reward-update is repeated many times. The agent learns with each iteration.

A perfect policy is learned when the agent knows exactly what action to take in every possible state. In practice, this is rare. The environment is not static. The agent might encounter new states never seen before during the learning phase. In some cases, it's not possible to accurately observe the environment's state. The technical term for this is **Partially Observable Markov Decision Process (POMDP)**.

Therefore, in practice, we need an **RL algorithm**. It's role is to update the policy based on states, actions and rewards. In this way, it responds to changing environments.

### What are the main approaches to reinforcement learning?

If RL has knowledge of the environment and models it, then we called it **model-based RL**. A model guides the agent to learn faster by ignoring low-reward states. If learning is done only via interactions with the environment without any knowledge of how the environment works, then we call it **model-free RL**.

In **passive RL**, the policy is fixed and agent learns the utilities of states, possibly involving learning a model. In **active RL**, the agent must learn what actions to take. An active agent explores the state space to obtain a better model and higher rewards.

During state space exploration, the current policy may not be followed. We call this **off-policy learning**. Otherwise, the algorithm is **on-policy learning**.  A stochastic policy allows exploration.

Instead of interacting with the environment, it's possible to learn from large datasets of past interactions. This data-driven approach is called **offline or batch RL**.

Where deep neural networks are employed, the term **Deep RL** is common.

### What are some specialized approaches in reinforcement learning?

Typically a single agent learns. Multiple agents coordinating and learning to maximize a common utility function is called **distributed RL**. Multiple agents who typically don't coordinate their actions and having separate utility functions are part of **multiagent RL**. In fact, one agent may work against another agent (such as in Markov games) thus making the environment non-stationary.

**Hierarchical RL** considers hierarchies of policies. Top-level policies might focus on high-level goals while others offer finer control.

Where it's not easy to define the reward function, the agent can learn from examples without explicit rewards. This is called **apprenticeship or imitation learning**. Learning the reward function from examples is called **inverse RL**.

Learning typically happens with atomic states. Instead, if we use structured representations, the approach is called **relational RL**.

### In RL, should an agent learn via simulations or within a real environment?

A real environment is better if it's difficult to model or it's constantly changing. Learning requires lots of samples and to do this in a real environment is time consuming. Simulations are faster than real time. Multiple simulations can be executed in parallel. A simulated environment is also safe. Simulations are useful to test rare events such as car crashes. But the simulated environment may not accurately model many aspects of the real environment.

Balancing an inverted pendulum is a task that can be learned in a real environment because it's safe. Training a walking robot that has had no prior training may not be safe or effective in a real environment. It's better to train it in a simulated environment and then use a real environment for scenarios that simulations didn't cover.

In practice, it's common for an agent to gather real-world observations. These are used to update the current model. Agent then learns within a simulated environment based on the updated model.

### What are some challenges with or shortcomings of RL?

RL has progressed in areas such as gaming and robotics where lots of simulated data is available. Translating these advances to practical applications is not trivial. RL demands much more **training data** than supervised ML.

Learning from scratch with no priori knowledge, called *pure RL*, has been criticized because learning is too slow. RL shares with AI some **common problems**: algorithms are not predictable or explainable, can be trained for only a narrowly-defined task, don't generalize well unless trained on massive amounts of data.

Basic **assumptions** may not hold good in real environments. The environment may not be fully observable. Even observed states can be inaccurate. Sometimes it not obvious or easy to figure out a suitable reward function, especially when there are multiple objectives. While the agent learns by mistakes, sometimes there's limited freedom to explore. For complex problems, it's not clear how to trade-off simulation complexity, training time and real-time performance constraints.

Many approaches use discrete actions and states. In the real world, agents have to interact in a **continuous space**. Policy optimization becomes a lot harder. RL algorithms can get stuck in a local optima.

## Milestones

1927

In an English translation of Pavlov's work on conditioned reflexes, the term *reinforcement* is used for the first time in the context of animal learning. Pavlovian conditioning comes from the work of Ivan Pavlov in the 1890s. Pavlov discovered how dogs would salivate upon seeing food but also when given associated stimuli even without food.

1948

Alan Turing describes in a report the design of a *pleasure-pain system*. He notes that the computer makes and records a random choice when faced with incomplete data. Subsequently, "When a pain stimulus occurs all tentative entries are cancelled, and when a pleasure stimulus occurs they are all made permanent."

1950

This decade sees the development of **optimal control** whose aim is to design a controller that minimizes some measure of a dynamic system's behaviour over time. Richard Bellman develops the relevant theory including the Bellman Equation, dynamic programming and Markov Decision Process (MDP). In 1960, Ronald Howard devises the **policy iteration** method for MDPs.

1959

Arthur Samuels, as part of his program to play checkers, implements a learning method based on **temporal-differences**. This relies on differences between successive estimates of the same quantity. It's inspired by animal learning psychology and the idea of secondary reinforcers. In 1972, Klopf brings together trial-and-error learning with temporal-difference learning.

1961

Minsky publishes a paper titled *Steps Toward Artificial Intelligence* that raises many issues relevant to reinforcement learning. He writes about the **credit assignment problem**, that is, how to credit success when a sequence of decisions have led to the final result. Through the 1960s, the terms "reinforcement" and "reinforcement learning" are increasingly used in literature. At the same time, some researchers working on pattern recognition and perceptual learning (these really belong to supervised ML) confuse these with RL.

1968

Improving on their work in early 1960s, Michie and Chambers train an RL algorithm to play tic-tac-toe and another to balance a pole on a movable cart. The **pole-balancing task** was learned with incomplete knowledge of the environment. This work influences later research in the field. In 1974, Michie notes that trial-and-error learning is an essential aspect of AI.

1989

Chris Watkins integrates the separate threads of dynamic programming and online learning. He formalizes reinforcement learning with MDP, subsequently adopted by other researchers. He proposes **Q-Learning**, a model-free method. The work of Watkins was preceded by Paul Werbos in 1977, who saw how dynamic programming could be related to learning methods.

1990

Sutton proposes **Dyna**, a class of architectures that integrate reinforcement learning and execution-time planning. The system can alternate between real world and a learned model of the world. Using simulated experiences (planning steps) in the world model, the optimal path is discovered faster. He applies Dyna to both policy iteration and Q-Learning.

1992

Gerry Tesauro develops **TD-Gammon** that can compete with human experts in the game of backgammon. TD-Gammon learned from self-play alone without human intervention. It's only rewards came at the end of each game. The evaluation function is a fully connected neural network with one hidden layer of 40 nodes. Previously, Tesauro attempted to train a neural network in a supervised manner with experts assigning relative values to moves. This approach was tedious and the program failed against human players.

Mar
2016

Developed by DeepMind Technologies, **AlphaGo** beats Lee Sedol, a human champion in the game of Go. AlphaGo was trained using RL from games involving human and computer play. The architecture is model-based learning using Monte Carlo Tree Search (MCTS) and model-free learning using neural networks. In 2017, AlphaZero Go is released and it beats AlphaGo by 100-0. Also in 2017, AlphaZero is released as a generalization of AlphaZero Go. AlphaZero can play chess, shogi and Go.

Reinforcement Learning


# Deep Packet Inspection

## Summary

Traditionally, control and regulation of Internet traffic has been managed by a firewall in the router device. However, routers can only scan the header of an IP packet which contains source, destination addresses and some next-hop routing information.

Deep Packet Inspection is a technology that allows a service provider to analyse network traffic in real time using the payload (IP packet content), not merely the IP header. Packets are inspected based on rules assigned by an enterprise, government or internet service provider. Only packets which clear the inspection can enter the network. Even encrypted data can be analysed.

DPI can effectively monitor, speed up, slow down, block, filter, make decisions about the traffic. Mobile and broadband service providers widely employ DPI analysers in their networks. However, unless used judiciously, DPI can also result in invasion of data privacy and other internet governance issues.

## Discussion

### Why do we need Deep Packet Inspection?

DPI was initially intended to manage and safeguard Local Area Network users (such as universities, corporates) from malicious software or viruses. The idea was to intercept the malicious packets in real time, at a checkpoint before they reached end users. This was usually performed as a firewall feature.

DPI (also called Packet analysis) can be used both in Intrusion Detection Systems (IDS) and Intrusion Prevention Systems (IPS). In organizations which have remote users who connect using their laptops for work, DPI is vital in preventing worms, spyware, and viruses from getting into the corporate network.

DPI protects networks from spam, viruses, DDoS (Distributed Denial Of Service) attacks and harmful/illegal content. It also supports regulatory requirements for lawful intercept; and for parental or enterprise content control systems.

DPI allows governments and organizations to define their own rules and policies, so that the network can detect if there are prohibited uses of applications.

Another intended use is for network management such as ensuring a basic quality of service (QoS) for end-users and preventing network congestion due to trivial/spam content.

### How does DPI work and what are its techniques?

Traditional packet analysis tools only scanned packets at the IP and TCP layers, whereas DPI functions at the Application layer of the OSI reference model.

There are two different approaches to packet analysis - (1) Continual, full-scale traffic packet capture that requires high speed processing and large storage arrays, which is expensive. (2) On-demand packet capture only when system compatibility issues occur (missing / damaged packets).

DPI is actually a combination of several techniques:

    + **Flow Tracking**: Determines which packets are part of a flow between the source and destination computers. It's based on a 5-tuple identifier `(SRC-IP, DEST-IP, SRC-PORT, DEST-PORT, PROTOCOL)`.
    + **Pattern Matching**: String patterns (several, not just one) coded as regular expressions are matched with incoming packets. For example, regardless of port, L7-filter classifier for Linux's Netfilter can classify packets as HTTP, Jabber, Citrix, Bittorrent, FTP, etc.
    + **Statistical Analysis**: Indicators (mean, variance) on absolute / relative packet sizes, flow rate per application.For network traffic monitoring, analytics from Gartner estimate that flow analysis should be done 80% of the time, and packet capture with probes should be done 20% of the time.

### What's the architecture of DPI deployment in a network?

DPI engines are usually deployed inline with firewalls in routers, SDN, and packet gateways. Offline packet analysis can also be performed for non-critical analysis.

DPI is a standard option in 4G LTE and 5G packet gateways (P-GWs). For instance, the backbone network of an ISP could be a 40-Gb/s system with four 10-Gb/s DPI modules.

In the dynamic service environment implied by cloud/SDN, DPI could potentially be co-located with network devices (as software running in virtual switches) or in the control layer (in the controller between applications and switches) due to its high CPU resource requirement. Real-time analytics using DPI is fed into big data analytics packages. This helps service providers understand what end users are doing and shape service offerings accordingly.

Network performances are proved by practical tests executed using real traffic in an ISP backbone network. DPI devices are designed to handle thousands of transactions each second. These devices analyse less than 1500 bytes per packet, which is not a very heavy load. But if this happens in real time for every packet, there is network delay overhead.

### What are the various other applications where DPI is used?

    + **Content Optimization**: DPI can act as proxy and modify contents in order to reduce still/video image quality or reformat web pages for mobile devices as per available bandwidth/device constraints so that users can enjoy content with reasonable performance.
    + **Billing Applications**: ISP use DPI to measure traffic volume and calibrate free and paid usages of network subscribers.
    + **Load Balancing**: Redistribution of packet content to alternate servers in a load-balanced network to maintain uniform load across all deployed systems.
    + **User Behavior Analysis**: Web/mobile applications can gauge subscriber behavior, assess what features are popular or which operations take too long, etc.
    + **Targeted Advertising**: ISPs can inject advertisements into websites that match the assumed interests of the users according to their browsing habits.
    + **Copyright Enforcement**: Automatically detect and block unauthorized sharing of music or video files on peer-to-peer platforms.
    + **Content Regulation**: Recognize and block access to illegal/harmful content such as child-abuse websites. Censor content considered a threat to government or public stability.

### If DPI analyses content in real time, will it not slow down the network data transmission?

Many current DPI methods are resource intensive and costly, especially for high bandwidth applications. Since it's done in real time, it doesn't work on normal processors or switches.

DPI has become possible only in the last few years through advances in computer engineering and pattern matching algorithms. Specialized routers are now able to perform DPI. Routers armed with a dictionary of programs help identify the purposes behind the LAN and internet traffic they are routing. Vulnerability from repeat attacks from known viruses is removed. However, new viruses still pose a threat.

Interaction with legacy tools could also be a problem. Some firewalls simply aren't designed to support DPI, prompting worry about sudden performance drops or total failure of protective network systems.

In spite of these overheads, DPI is widely prevalent in network deployments because of the protection it offers from malicious or wasteful bandwidth usage by spams, viruses and malware. By successfully blocking such spurious input requests from untrustworthy clients, the DPI servers are able to save the network from unnecessary congestion and possible DDoS attacks.

### What are the negative consequences of DPI and how can misuse be prevented?

DPI is basically a packet sniffing technology on the network traffic, enabling operators to monitor what is happening in real time. It was meant to be used for benevolent causes such as managing bandwidth, lawful surveillance, copyright enforcement, and network security.

However, DPI may be controversial from a customer privacy and net-neutrality standpoint. ISPs often use DPI servers to inspect Internet traffic to identify what traffic they want to slow down or restrict. At its root, DPI helps operators regain control over a network that primarily carries third-party applications and services by accurately identifying those applications in real time.

Privacy infringement, corporate snooping, governmental suppression of facts and news, and advertisement implantation are some of the negative consequences of DPI. If an ISP sells streaming music, then OTT music applications are its competitors. ISPs can intentionally ignore/cause congestion or degrade performance of the competing service.

One way to bypass DPI is to use **traffic obfuscators** as standalone software. These can change traffic signatures to look like traffic that isn't normally blocked by DPI.

### What are the common tools used for packet analysis in DPI?

Governments of certain countries use proprietary and sophisticated DPI tools for online information censoring. *The Great FireWall of China* is one such example. They are implemented using a combination of flow analysers, filtering of certain IP ranges, flow redirection, URL filtering and Man-in-the-Middle techniques.

ntop, Netify Agent and libtins are open source utilities or toolkits in C/C++.

Corporates and Internet Service Providers may choose among the commonly available DPI tools:

    + **Wireshark**: A popular free and open-source packet analyser which can be configured to be used for Intrusion Detection (ID). The `tshark` utility allows you to filter the contents of a `pcap` file from the command line to study network activity.
    + **Netfilter in Linux**: Classifies packets as HTTP, Jabber, Citrix, Bittorrent, FTP, etc., regardless of port.
    + **Netflow from Cisco**: Introduced on their routers to collect IP network traffic information as traffic enters/exits an interface and build Access Control Lists. It consists of a flow collector and analyser.
    + **SolarWinds Netflow**: Network bandwidth monitoring (collection and analysis) tool. Free and paid versions available.
    + **Scrutinizer from Plixer**: Can handle network flow analysis of Cisco and other vendors' network devices.

## Milestones

1998

*Wireshark*, earlier part of the *Ethereal* project, is released as a free, open source packet sniffing tool. It initially supports **shallow packet inspection**, only at the IP header level.

2002

Traffic inspection solutions *NetScreen* (acquired by Juniper networks) are designed to be installed into firewall systems. Since the operation is expensive, it is triggered only on need basis.

2005

MIMESweeper, ClamAV, NetCache are some of the early open source internet proxy caching servers introduced for scanning content to an ICAP server running an anti-virus software.

2006

*The Great Firewall of China* is deployed successfully. This internet censorship project commenced in 1998 for online traffic regulation in China.

2012

DPI becomes a powerful **network security tool** with deployment on SDN/cloud servers.

2012

DPI analysis tools feed network traffic data into **Big Data Analytics** for ISPs to derive critical insights on user behavior.

Deep Packet Inspection


# Augmented Reality

## Summary

With the coming of the social web, mobile web and Big Data, information has become plenty and easily accessible. As late as the mid-2010s, much of this information remained within the digital world of servers, computers and smartphones. Augmented Reality (AR) attempts to remedy this by bringing digital information into the real world. In essence, AR re-imagines human-computer interaction.

Augmented Reality is a term that requires our presence and interaction in the real world. At the same time, our experience of the real world is augmented by adding virtual elements created by computers. These virtual elements are typically visuals and sounds. The idea is to give us extra information to do things more efficiently and enjoyably.

## Discussion

### What's the definition of Augmented Reality?

One possible definition of AR has three characteristics: it combines real and virtual; it's interactive in real time; and it's registered in 3D. This definition is regardless of the display hardware. It applies equally well to visuals that are optical see-throughs or rendered on monitors. A movie such as Jurassic Park is not AR since it's not interactive for viewers.

Since this definition requires 3D integration into a real environment, 2D overlays on live video are also excluded from AR. Some writers consider this as AR as well, calling it *TV AR*. Augmenting live video with digital information for a sports game or a weather report would be considered TV AR.

Others define AR in terms of the interfaces/hardware used. But conceptually, the definition is still includes merging the real and the virtual while retaining presence and interaction with the real world.

### How is AR different from VR and MR?

The acronyms AR, VR and MR refer to Augmented Reality, Virtual Reality and Mixed Reality.

While AR experiences are rooted in the real world, VR experiences are rooted in the virtual world. In VR, the user is immersed in a synthetic world created by computer graphics. Such a virtual world may be inspired by the real world but it's also free to break the laws of physics that govern real-world interactions.

AR and VR can also be differentiated by user presence. A person sitting in New York, visualizing the Eiffel Tower in Paris surrounds himself in VR, a world that represents the Tower and its surroundings. VR enables him to experience the Tower without going to Paris. A tourist who is already near the Tower, could use an AR app to get directions. When he is at the Tower, the AR app could display or read out relevant information while the user points his camera at the Tower.

MR is an environment where users can interact with both real and virtual objects.

### What are some possible applications of AR?

Early ideas for AR were mostly in the industrial space rather than the consumer space: medical, engineering, military, robotics, etc. Due to smartphone adoption, AR in the consumer segment is gaining interest with education, gaming and advertising showing good potential.

In education, live 3D models can help medical students visualize veins in patients. Pokémon Go and Real Strike are example AR games. GPS-based AR apps can help walkers or drivers navigate. AR can add value in the treatment of Post-Traumatic Stress Disorder. In real estate, AR can help you visualize your living room for its interior decor.

Advertisers can give more information about a product via an interactive AR app, such as the BMW mini ad of 2008. A shopper can try out a dress or a makeup in AR before the actual purchase. AR can be used to create public awareness such as the WWF-Coca-Cola Arctic Home Campaign of 2013. Tourist spots or operators can offer self-guided AR-enabled tours. Actors can prepare for their roles in front of AR mirrors.

### What are the different types of AR?

AR can be categorized based on the technique used:

    + **Marker-based AR**: This involves a camera and image/pattern recognition. Markers in the real world are recognized. They are then overlaid with virtual objects or information. This is also called **Recognition-based AR**.
    + **Location-based AR**: The use of GPS plus sensor data (in smartphones) enables apps that are location-centric such as finding relevant shops nearby or showing directions. Information is overlaid based on present location. This is also called **Markerless AR**.
    + **Projection-based AR**: This works by projecting light onto real-world surfaces. Your empty desk can have a virtual keyboard. Your palm can be lit up with a phone dialler. User interactions are detected based on changes in the projected light. This type of AR includes 3D interactive holograms.
    + **Superimposition-based AR**: Using object recognition, real-world objects are replaced or augmented with digital equivalents. Virtual objects are thus superimposed on the real world.
    + **Outlining AR**: Using object recognition, relevant features are outlined virtually. This may be seen as a specialization of superimposition-based AR.It's possible to combine many of these within a single app.

### I've heard of SLAM. What is it?

**Simultaneous Localization And Mapping (SLAM)** is a well-known problem in mobile robotics. A robot in an unknown environment has the twin tasks of mapping its surroundings while also determining its own location within the map.

SLAM is being increasingly used in AR and is seen as a successor to marker-based AR. The good thing is SLAM doesn't need a marker. Multiple cameras with depth sensing technology are used in SLAM to create a map of the surroundings. Unlike marker-based AR, SLAM lacks context. GPS can be used to give context. For indoor navigation apps, SLAM maps (databases) can be used to create context. The idea is to obtain context with nothing more than visual input.

Solutions to the SLAM problem can be combined with other techniques in AR to effectively combine real and virtual worlds. Apple's ARKit uses SLAM and so does Google's Project Tango. The latter's effort is also towards building context.

### Isn't a QR Code the same thing as AR?

A Quick Response (QR) code is basically a 2D barcode that's machine readable. Once scanned, say with a smartphone, it can give more information about the item to which it's attached. A marker-based AR can do this and more. QR codes have to conform to a specific layout while in AR the marker can be any object or pattern.

QR codes were invented in the 1990s when image recognition had to be simple and fast. In comparison, AR requires more processing but it's also more flexible because any object can be a marker. Moreover, contextual information can be shown at different parts of the same object. There's no need to reduce the object to a code like in QR code. Finally, using AR apps, tracking and analytics can be applied. This is rarely done with QR codes.

With QR codes, information is typically displayed by redirecting the user to a video, URL, registration form, etc. With AR, information is overlaid on top of real-world objects. Immersion of 3D objects in the real world is possible in AR but not so with QR codes. This is a fundamental difference.

### What are the components of an AR system?

An AR system can be said to contain the following components:

    + **Tracking**: Via sensors and camera, the system tracks the user's viewpoint.
    + **Registration**: Virtual objects must be spatially registered or anchored in the real world.
    + **Visualization**: Based on current location and viewpoint, the visualization of virtual objects has to be adjusted.
    + **Spatial Model**: This consists of both the real world and the virtual world. Both must be registered in the same coordinate system.

### Could you mention the technical foundations of AR?

For AR to work, virtual objects must be placed accurately in the real world. We can identify the following essentials:

    + **Visual Keypoint Matching**: Also referred to as *Marker Detection*, this requires image processing, feature extraction and marker detection. The marker's surface is determined so that virtual objects can be placed on the surface.
    + **Spatial Mapping**: The idea is to map the real world to a virtual model. Depth sensing is involved. The virtual model can be used to detect surfaces (walls, floors, tabletops). When virtual objects are placed, occlusion becomes important.
    + **Sensing**: Viewer becomes the anchor of the virtual space and content. Viewpoints are adjusted based on inputs coming from sensors: GPS, accelerometer, gyroscope, etc. Since sensing accuracy may be limited, this can be combined with visual tracking.

### What sort of hardware is needed to realize AR?

AR typically requires some sort of a display; camera and other sensors to enable detection and interaction; computer processing to blend the real and the virtual. Smartphones have all of these, thus lowering the cost of adoption for end users. Today, AR can be delivered as smartphone apps and easily reach a worldwide audience. Wearables including head-mounted display (HMD) or eyeglasses such as Google Glass are AR-specific hardware. HMDs can be see-through or screen based.

To bridge real and virtual worlds, cameras and sensors are used. This information is processed to create a virtual model of the real world. For projection-based AR, miniature projectors are needed and these may be part of a headset wearable.

A typical AR wearable would need sufficient processing power and memory, wireless connectivity and GPS. Sensors may include accelerometer, gyroscope and magnetometer to detect movements and thereby adjust the views of virtual objects. Some devices use mirrors to assist in aligning images to the viewer's eye.

Explicit user control of AR could be via a touchpad or voice commands.

## Milestones

1965

Ivan Sutherland, often called "Father of Computer Graphics", publishes an essay titled "The Ultimate Display". Here he describes ideas that are today part of AR/VR systems.

1968

Ivan Sutherland creates an optical see-through head-mounted display (HMD). The system uses computer-generated graphics. Sutherland calls it the *The Sword of Damocles*.

1990

The term **Augmented Reality** is coined by Boeing researcher Tom Caudell. The idea was display schematics of wire bundle assembly on a see-through HMD to assist workers in an airplane factory. By 1994, the term is increasingly used in literature.

1992

Louis Rosenberg develops an AR system called *Virtual Fixtures* at the U.S. Air Force Research Laboratory (AFRL). Because 3D graphics in the early 1990s were too slow for realistic experiences, Virtual Fixtures employed physical robots controlled by an exoskeleton worn by the user.

1994

Researchers Milgram and Kishino attempt to create a taxonomy for **Mixed Reality (MR)**. They see AR as a subset of MR. AR consists of real environment augmented with virtual objects. They also define **Augmented Virtuality (AV)** as a virtual environment to which real objects are added.

1999

Hirokazu Kato creates *ARToolkit*. It's open source. It overlays computer graphics on real video. In 2009, this is ported to Adobe Flash, thus bringing AR to the web.

2003

Using ARToolkit, researchers at the Vienna University of Technology create an AR system with a Personal Digital Assistant (PDA). This system includes on-device real-time tracking that can at times be offloaded to the network via a wireless connection.

2013

Volkswagen *MARTA (Mobile Augmented Reality Technical Assistance)* app provides technicians assistance on a repair process.

May
2014

Google releases **Google Glass** to the public after about a year of beta testing. It's a head-mounted display designed as eyeglasses. It features Bluetooth connectivity to the Internet via user's smartphone, voice control, touchpad, camera and display.

2015

Microsoft announces **HoloLens** in January. A live demo of the same is done in April.

Jul
2016

Niantic releases a location-based smartphone AR app named *Pokémon Go*. It quickly becomes one of the most popular games, breaking multiple records (downloads and revenue). The game involves players visiting real-world locations to capture virtual creatures.

Sep
2017

Apple shows off capabilities of its **ARKit** along with its AR-capable iPhone 8 and iPhone X. Just two weeks earlier, Google launched its **ARCore**.

Augmented Reality


# Software Architecture

## Summary

All good software has some structure to it. It's composed of components, each with well-defined functionality. It has interfaces, both internal (among components) and external (to its environment). Software architecture is about defining these aspects. It focuses on abstractions and hides implementation details.  More importantly, architecture is about capturing decisions, not just describing structures.

Software architects take system requirements as inputs and produce various artefacts that describe the architecture. Both functional and non-functional requirements are considered. Architects make choices about architectural patterns, platforms, programming languages, databases, third-party libraries, tools, and more.

## Discussion

### How does software architecture differ from software design?

Architecture is about high-level abstractions whereas design is about low-level implementation details. Architecture helps us understand the system and how it's supposed to interact with its environment. It involves multiple stakeholders, presents an overall vision that meets quality attributes, and makes a clear separation of concerns. Software architects make choices and give reasons why those choices were made. Typically these choices are made early on in the project and are costly to change thereafter.

In contrast, software design concerns itself with class methods/properties, data structures, algorithms, actual code, business logic, etc. Certain design methods play well with certain architectural styles.

It's not easy to define what's high-level or important. They're very much contextual. Whatever these are, a good architect identifies them early on. Architecture is about reducing irreversibility: an architect defines what parts are fixed, what parts can be changed and attempts to minimize the former. Architecture is really a shared understanding of the system design.

It's been said that "architecture is design but not all design is architectural". Dashofy defined,

> A software system's architecture is the set of principal design decisions made about the system.

### Could you give examples of software architecture?

For a web application, an architect may define the main components: frontend, backend, storage, notifications, etc. She may decide to use Java, MySQL and Hibernate ORM. She may adopt the MVC pattern, or server-side rendering over client-side rendering. She may identify high-level components such as authentication, load balancer and CDN. She may even prescribe Eclipse IDE along with plugins Checkstyle, SonarLint and TestNG.

For an application server, we may use Java-EE. Functionality is spread across access layer, application processes layer and domain objects layer. Java-EE thus uses layered architecture. An alternative is the service-oriented architecture (SOA). This consists of a service bus to which producers and consumers connect via adapters. While Java-EE focuses on scalability and reliability, SOA focuses on integrability and modifiability.

While there are API architects, in some projects the solution architect mandates an API gateway and developers detail the APIs. In a team of inexperienced developers, the architect may take on more design-specific tasks.

### What are the key benefits of software architecture?

Creating a good architecture leads to right trade-offs, meeting requirements, reducing risks, and timely deliverables. Skipping architecture may save time towards short-term success. In the long term, rework and delays are quite likely. Poor architecture leads to technical debt, more defects and software that's harder to evolve.

We often pay more for higher quality products. However, architecture doesn't exercise this quality-cost trade-off. Investing in architecture pays off within a few weeks. Without a proper architecture, we end up paying more later. Software that works as expected has external quality. Architecture is really **internal quality** that project teams should care about.

Architecture helps maintain conceptual integrity of the system as it evolves. An architectural model is a living record of original decisions. As new features are added, such a model mitigates architectural drift. Architecture aids knowledge reuse and avoids "reinventing the wheel".

Architecture can help modernize legacy systems such as migrating to the cloud or integrating AI/ML capabilities. Existing technical debt can be addressed or managed.

### What are some patterns and frameworks in software architecture?

For common problems, there's no need to solve them from scratch. Architectural patterns can be adopted: client-server, peer-to-peer, event-driven, layered (n-tiered), space-based, service-oriented, microservices, serverless, cloud-native, microkernel, and many more.

Ford and Richards have analysed and compared some of these patterns with respect to quality attributes. They published these as Architecture Styles Worksheet. For example, a microservices architecture rate higher on agility and fault-tolerance compared to layered architecture. But layered architecture is simpler and cheaper.

Frameworks bridge concepts to coding blocks. As part of ISO/IEC/IEEE 42010:2011 work, ISO has been collating various architecture frameworks. There are frameworks for IoT, Big Data, security, management systems, communication systems, for the automotive industry, smart grids, and more. Indeed, **domain-specific reference architectures** are used in many industries. These lead to reusing architectures across products of a product line or family.

### What artefacts does a software architect produce?

**Architecture model** is an essential artefact. It captures some or all of the architectural decisions. It's a tangible record of the architecture. Modelling could be informal (slides, diagrams), semi-formal (UML diagrams) or formal. Architecture Description Languages (ADLs) can be used for formal modelling. An ADL is a set of notations, languages, standards, and conventions. It can lead to formal analysis, evaluation, reuse, and automated code generation.

Software systems can be viewed from different perspectives. Such views are complementary. An architect need to produce many of these views or diagrams: data flows, control flows, state transitions, entity relations, object-oriented hierarchies, data models, message sequences, and more. Another way to classify these views is logical, process, physical and development.

### What processes are applicable for software architecture?

Software architecture is part of the overall discipline of software engineering. Architects need to interact with multiple stakeholders. They need to be clear about product requirements. Non-functional requirements include reliability, usability, efficiency, scalability, maintainability, and more. They should factor in additional constraints such as legal, regulations, costs, time-to-market and talent availability.

Adopting a programming framework or developer tools such as Springboot or Cocoa implicitly selects an architecture. While developers can focus on design decisions, they need to be aware of the architectures they've adopted. Moreover, an architect needs to address runtime requirements including CI/CD, deployment, scalability and observability.

Architects deal with architectural models, description languages, analysis techniques, development environments, canonical solutions, and design processes. Architecture establishes the constraints within which application components need to be developed.

Static analysis of source code can be used to recover or reverse engineer the architecture of legacy systems. Such analysis can also identify deviations between architecture and implementation.

It's foreseen that AI-enabled tools will automate architectural tasks. Decisions will become more data driven. Automated code generation and testing may help to quickly validate and refine architectures.

### What are some pitfalls in software architecture?

Working without properly defining quality attributes leads to poor architecture. Asking vendors to create architectures or adopting architectures from another organization are just as bad: architecture needs context and tradeoffs depend on the context. Waiting for the perfect architecture before starting implementation is another pitfall.

Even the best architecture can be ineffective if not properly communicated. Automated documentation, graphical views and change simulation are some tools that can help. Architects should communicate to upper management in a language they understand. Indeed, organizational structure should evolve to keep up with technological evolution. DevOps and CI/CD are useless if management takes weeks to approve a change.

Architects shouldn't work in isolation. They should collaborate with everyone, looking out for problem areas before they become serious, giving their inputs, explaining the architecture, and mentoring individuals. These interactions create effective feedback loops towards architecture that's fit for purpose.

Modelling tools are often inadequate when mapping the code to the model. Architects and their tools ignore software architecture evolution.  These limitations can result in drift between architecture and implementation.  Architecting should be continuous and an iterative process.

### How can a developer move into the role of an architect?

Developers should move from their narrow focus of specific components or sub-systems. As architects, they should learn to look at the big picture and obtain a holistic view of the entire system. They should resist the urge to write code. A developer may focus on only coding but an architect has to look at requirements, coding, testing, deployments, evolution, and more.

An architect should be able to take important decisions given the constraints and quality attributes. Because architects interface with many stakeholders, they need leadership qualities.

Architects straddle both technical, business and interpersonal concerns. A solid technical background is essential for many tasks: spotting trends, planning, assessing risks, and critical thinking. An architect should have empathy and see things from a developer's perspective. This helps when reviewing and mentoring. Overall, an architect who's been a developer for many years and interacted with customers will do well.

It's been said that software architect is not an independent role. Architecting should be regarded as a skill that developers and agile teams should practice. Architecting should be connected to developing.

### What resources can help learn software architecture?

Two books published by SEI are *Software Architecture in Practice* and *Designing Software Architectures*. To learn about architectural patterns, see *Pattern-Oriented Software Architecture* by Buschmann et al. (1996). Richards has curated a list of books on software architecture.

The SEI offers online courses on software architecture. There are also courses on Udemy and Coursera.

New architects can look at some templates of software architecture documents: Course Registration System and Co-op Evaluation System.

We may need to evaluate software architectures given a set of quality attributes. Some methods exist: Active Design Reviews, Software Architecture Analysis Method (SAAM), and Architecture Tradeoff Analysis Method (ATAM).

## Milestones

1980

From the mid-1980s, practitioners of Software Engineering address architectural concerns to manage the complexity of large projects. They use box-and-line diagram as a tool. The decade ends with a DARPA workshop on the topic of Domain-Specific Software Architectures (DSSAs).

1990

During this decade, architectural patterns, architecture description languages (ADLs), documentation, architecture recovery, tools, and formal methods are studied.  Design patterns and architectural styles aid teams with knowledge reuse.

1992

Perry and Wolf publish a paper titled *Foundations for the study of software architecture*. This is perhaps the earliest use of the term "software architecture" in literature.

1994

Garlan and Shaw of Carnegie Mellon University publish a paper titled *An Introduction to Software Architecture*. This includes a few architectural styles: pipes and filters, data abstraction and object-oriented organization, event-based, layered systems. In one case study, they propose and compare different styles. In 1996, they publish the book *Software Architecture: Perspectives on an Emerging Discipline*. This book brings together most ideas of the time.

1995

The IEEE Software Engineering Standards Committee charters the **IEEE Architecture Planning Group (APG)**. It's objective is "to set the direction for the next generation of architecture-related standards and practices for the IEEE." Their work eventually results in the publication of IEEE 1471 Standard in 2000. Also in 1995, Taylor et al. publish *Software development using domain-specific software architectures*.

Nov
1995

Kruchten proposes the **"4+1" View Model**. The idea is that multiple views are necessary to address the concerns of different stakeholders. The four views are logical (object-oriented), process (synchronization, concurrency), physical (software-hardware mapping, distributed), and development (software organization)

Sep
2000

**IEEE 1471** titled *Recommended Practice for Architecture Description of Software-Intensive Systems* is approved as a standard. Traditionally, architects considered only hardware issues. This standard addresses this short-sightedness. In August 2001, ANSI makes this an American National Standard. In 2007, this is adopted by ISO/IEC as **ISO/IEC 42010:2007**.

Nov
2011

**ISO/IEC/IEEE 42010:2011** titled *Systems and software engineering — Architecture description* is published as an international standard. It's an evolution of IEEE 1471. This harmonizes ISO/IEC 42010, ISO/IEC 12207 and ISO/IEC 15288.

Software Architecture