Draft: draft-hall-censorship-tech-latest

Commentary: Changes implement fixes to Issues , , , and opened based on comments from   . 
 From Nillson : " for 3.2.1 : Host is not required if you are using HTTP/1.0 . Many censorship solutions are not that good at normalization . Consider requests list GET /wrong/ .. /correct or GET /%63orrect . It does n't even have to be correct HTTP , as long as the service understands the request while the censors do n't . " /correct\0junk " Even simple things like HTTP proxies can circumvent some censoring , since it makes the host and IP not match the first stage of pattern matching . " 
 From Nillson : " HTTP SNI should probably be listed as an application layer method of blocking content . Certificates are sent unencrypted in the other direction , so that could also be used to identify the endpoint . " 
 From Nillson : " Another point of control is the service itself . All companies would extend some sort of assistance to a government to avoid being blocked . What that level is and how open they are about it differs . " 
 From Nillson : " It is probably worthwhile to separate black lists from heuristics . Most of the commercial solutions operate on black lists as well , while a few do real time heuristic assessment of the content . In the context of " figuring out what censors would like to block " the heuristics would map to content categories ( again e.g. Porn or Gambling ) , but is distinctly different from category based black lists . "
Aggregation is the process of figuring out what censors would like to
block Glanville-2008.  Generally, censors aggregate "to block"
information in three possible sorts of blacklists: Keyword, Domain
Name, or IP.  Keyword and Domain Name blocking take place at the
application level (e.g.  HTTP), whereas IP blocking tends to take
place in the TCP/IP header.  The mechanisms for building up these
blacklists are varied.  Many times private industries that sell
"content control" software, such as SmartFilter, provide their
services to nations which can then pick from broad categories, such
as gambling or pornography, that they would like to block . In these
cases, the private services embark on an attempt to label every semi-
questionable website as to allow for this metatag blocking.
Countries that are more interested in retaining specific political
control, a desire which requires swift and decisive action, often
have ministries or organizations, such as the Ministry of Industry
and Information Technology in China or the Ministry of Culture and
Islamic Guidance in Iran, which maintain their own blacklists.
Aggregation is the process of figuring out what censors would like to
block Glanville-2008.  Generally, censors aggregate "to block"
information in blacklists or using real-time heuristic assessment of
content Ding-1999.  There are typically three types of blacklists:
Keyword, Domain Name, or IP.  Keyword and Domain Name blocking take
place at the application level (e.g.  HTTP), whereas IP blocking
tends to take place in the TCP/IP header.  The mechanisms for
building up these blacklists are varied.  Many times private
industries that sell "content control" software, such as SmartFilter,
which allows filtering from broad categories, such as gambling or
pornography, that they would like to block.  In these cases, the
private services embark on an attempt to label every semi-
questionable website as to allow for metatag blocking (similarly,
they tune real-time content heuristic systems to map their
assessments onto categories of objectionable content).  Countries
that are more interested in retaining specific political control, a
desire which requires swift and decisive action, often have
ministries or organizations, such as the Ministry of Industry and
Information Technology in China or the Ministry of Culture and
Islamic Guidance in Iran, which maintain their own blacklists.
Services: Application service providers can be pressured, coerced,
or legally required to censor specific content or flows of data.
Service providers naturally face incentives to maximize their
potential customer base and potential service shutdowns or legal
liability due to censorship efforts may seem much less attractive
than potentially excluding content, users, or uses of their
service.
An HTTP header contains a lot of useful information for traffic
identification; although host is the only required field in an HTTP
request header, an HTTP method field is necessary to do anything
useful.  As such, the method and host fields are the two fields used
most often for ubiquitous censorship.  A censor can sniff traffic and
identify a specific domain name (host) and usually a page name (GET
/page) as well.  This identification technique is usually paired with
TCP/IP header identification (see sec_tcpid) for a more robust
method.
An HTTP header contains a lot of useful information for traffic
identification; although host is the only required field in an HTTP
request header (HTTP/1.1 and later), an HTTP method field is
necessary to do anything useful.  As such, the method and host fields
are the two fields used most often for ubiquitous censorship.  A
censor can sniff traffic and identify a specific domain name (host)
and usually a page name (GET /page) as well.  This identification
technique is usually paired with TCP/IP header identification (see
sec_tcpid) for a more robust method.
Tradeoffs: Request Identification is a technically straight-forward
identification method that can be easily implemented at the Backbone
or ISP level.  The hardware needed for this sort of identification is
cheap and easy-to-acquire, making it desirable when budget and scope
are a concern.  HTTPS will encrypt the relevant request and response
fields, so pairing with TCP/IP identification (see sec_tcpid) is
necessary for filtering of HTTPS.
Empirical Examples: Studies exploring censorship mechanisms have
found evidence of HTTP header/ URL filtering in many countries,
including Bangladesh, Bahrain, China, India, Iran, Malaysia,
Pakistan, Russia, Saudi Arabia, South Korea, Thailand, and Turkey
Aryan-2012.  Commercial technologies such as the McAfee SmartFilter
and NetSweeper are often purchased by censors Dalek-2013.  These
commercial technologies use a combination of HTTP Request
Identification and TCP/IP Header Identification to filter specific
URLs.  Dalek et al. and Jones et al. identified the use of these
products in the wild Jones-2014.
Tradeoffs: Request Identification is a technically straight-forward
identification method that can be easily implemented at the Backbone
or ISP level.  The hardware needed for this sort of identification is
cheap and easy-to-acquire, making it desirable when budget and scope
are a concern.  HTTPS will encrypt the relevant request and response
fields, so pairing with TCP/IP identification (see sec_tcpid) is
necessary for filtering of HTTPS.  However, some countermeasures such
as URL obfuscation RSF-2005 can trivially defeat simple forms of HTTP
Request Header Identification.
Empirical Examples: Studies exploring censorship mechanisms have
found evidence of HTTP header/ URL filtering in many countries,
including Bangladesh, Bahrain, China, India, Iran, Malaysia,
Pakistan, Russia, Saudi Arabia, South Korea, Thailand, and Turkey
Verkamp-2012 Nabi-2013 Aryan-2012.  Commercial technologies such as
the McAfee SmartFilter and NetSweeper are often purchased by censors
Dalek-2013.  These commercial technologies use a combination of HTTP
Request Identification and TCP/IP Header Identification to filter
specific URLs.  Dalek et al. and Jones et al. identified the use of
these products in the wild Dalek-2013 Jones-2014.
Empirical Examples: Researchers have discovered keyword
identification by content providers on platforms ranging from instant
messaging applications Senft-2013 to search engines Condliffe-2013.
To demonstrate the prevalence of this type of keyword identification,
we look to search engine censorship.
Empirical Examples: Researchers have discovered keyword
identification by content providers on platforms ranging from instant
messaging applications Senft-2013 to search engines Rushe-2015
Cheng-2010 Whittaker-2013 BBC-2013 Condliffe-2013.  To demonstrate
the prevalence of this type of keyword identification, we look to
search engine censorship.
Search engine censorship demonstrates keyword identification by
content providers and can be regional or worldwide.  Implementation
is occasionally voluntary, but normally is based on laws and
regulations of the country a search engine is operating in.  The
keyword blacklists are most likely maintained by the search engine
provider.  China requires search engine providers to "voluntarily"
maintain search term blacklists to acquire/keep an Internet content
provider (ICP) license Cheng-2010.  It is clear these blacklists are
maintained by each search engine provider based on the slight
variations in the intercepted searches Whittaker-2013.  The United
Kingdom has been pushing search engines to self censor with the
threat of litigation if they don't do it themselves: Google and
Microsoft have agreed to block more than 100,00 queries in U.K. to
help combat abuse Condliffe-2013.
Search engine censorship demonstrates keyword identification by
content providers and can be regional or worldwide.  Implementation
is occasionally voluntary, but normally is based on laws and
regulations of the country a search engine is operating in.  The
keyword blacklists are most likely maintained by the search engine
provider.  China requires search engine providers to "voluntarily"
maintain search term blacklists to acquire/keep an Internet content
provider (ICP) license Cheng-2010.  It is clear these blacklists are
maintained by each search engine provider based on the slight
variations in the intercepted searches Zhu-2011 Whittaker-2013.  The
United Kingdom has been pushing search engines to self censor with
the threat of litigation if they don't do it themselves: Google and
Microsoft have agreed to block more than 100,00 queries in U.K. to
help combat abuse BBC-2013 Condliffe-2013.
Despite these problems, DPI is the most powerful identification
method and is widely used in practice.  The Great Firewall of China
(GFW), the largest censorship system in the world, uses DPI to
identify restricted content over HTTP and DNS and inject TCP RSTs and
bad DNS responses, respectively, into connections Anonymous-2014.
Empirical Evidence: Several studies have found evidence of DPI being
used to censor content and tools.  Clayton et al.  Crandal et al.,
Anonymous, and Khattak et al., all explored the GFW and Khattak et
al. even probed the firewall to discover implementation details like
how much state it stores Khattak-2013.  The Tor project claims that
China, Iran, Ethiopia, and others must being using DPI to block the
obsf2 protocol Wilde-2012.  Malaysia has been accused of using
targeted DPI, paired with DDoS, to identify and subsequently knockout
pro-opposition material Wagstaff-2013.  It also seems likely that
organizations not so worried about blocking content in real-time
could use DPI to sort and categorically search gathered traffic using
technologies such as NarusInsight Hepting-2011.
Despite these problems, DPI is the most powerful identification
method and is widely used in practice.  The Great Firewall of China
(GFW), the largest censorship system in the world, uses DPI to
identify restricted content over HTTP and DNS and inject TCP RSTs and
bad DNS responses, respectively, into connections Crandall-2010
Clayton-2006 Anonymous-2014.
Empirical Evidence: Several studies have found evidence of DPI being
used to censor content and tools.  Clayton et al.  Crandal et al.,
Anonymous, and Khattak et al., all explored the GFW and Khattak et
al. even probed the firewall to discover implementation details like
how much state it stores Crandall-2010 Clayton-2006 Anonymous-2014
Khattak-2013.  The Tor project claims that China, Iran, Ethiopia, and
others must being using DPI to block the obsf2 protocol Wilde-2012.
Malaysia has been accused of using targeted DPI, paired with DDoS, to
identify and subsequently knockout pro-opposition material Wagstaff-
2013.  It also seems likely that organizations not so worried about
blocking content in real-time could use DPI to sort and categorically
search gathered traffic using technologies such as NarusInsight
Hepting-2011.
In encrypted connections using Transport Layer Security, there may be
servers that host multiple "virtual servers" at a give network
address, and the client will need to specify in the (unencrypted)
Client Hello message which domain name it seeks to connect to (so
that the server can respond with the appropriate TLS certificate)
using the Server Name Indication (SNI) TLS extension RFC6066.  Since
SNI is sent in the clear, censors and filtering software can use it
as a basis for blocking, filtering, or impairment by dropping
connections to domains that match prohibited content (e.g.,
bad.foo.com may be censored while good.foo.com is not) Shbair-2015.
Tradeoffs: Some clients do not send the SNI extension (e.g., clients
that only support versions of SSL and not TLS) or will fall back to
SSL if a TLS connection fails, rendering this method ineffective.  In
addition, this technique requires deep packet inspection techniques
that can be computationally and infrastructurally expensive and
improper configuration of an SNI-based block can result in
significant overblocking, e.g., when a second-level domain like
google.com is inadvertently blocked.
Empirical Evidence: While there are many examples of security firms
that offer SNI-based filtering Trustwave-2015 Sophos-2015 Shbair-
2015, the authors currently know of no specific examples or reports
of SNI-based filtering observed in the field used for censorship
purposes.
Censors sometimes identify entire protocols to be blocked using a
variety of traffic characteristics.  For example, Iran degrades the
performance of HTTPS traffic, a protocol that prevents further
analysis, to encourage users to switch to HTTP, a protocol that they
can analyze Aryan-2012.  A simple protocol identification would be to
recognize all TCP traffic over port 443 as HTTPS, but more
sophisticated analysis of the statistical properties of payload data
and flow behavior, would be more effective, even when port 443 is not
used Sandvine-2014.
Censors sometimes identify entire protocols to be blocked using a
variety of traffic characteristics.  For example, Iran degrades the
performance of HTTPS traffic, a protocol that prevents further
analysis, to encourage users to switch to HTTP, a protocol that they
can analyze Aryan-2012.  A simple protocol identification would be to
recognize all TCP traffic over port 443 as HTTPS, but more
sophisticated analysis of the statistical properties of payload data
and flow behavior, would be more effective, even when port 443 is not
used Hjelmvik-2010 Sandvine-2014.
There are a variety of mechanisms that censors can use to block or
filter access to content by altering responses from the DNS ICANN-
SSAC-2012, including blocking the response, replying with an error
message, or responding with an incorrect address (potentially to a
server that can communicate to the end-user a reason for blocking
access to that resource, for example using HTTP Status Code 451
RFC7725).
There are a variety of mechanisms that censors can use to block or
filter access to content by altering responses from the DNS
AFNIC-2013 ICANN-SSAC-2012, including blocking the response, replying
with an error message, or responding with an incorrect address
(potentially to a server that can communicate to the end-user a
reason for blocking access to that resource, for example using HTTP
Status Code 451 RFC7725).
Trade-offs: DNS interference requires the censor to force a user to
traverse a controlled DNS hierarchy (or intervening network on which
the censor serves as a Active Pervastive Attacker RFC7624 to rewrite
DNS responses) for the mechanism to be effective.  It can be
circumvented by a technical savvy user that opts to use alternative
DNS resolvers (such as the public DNS resolvers provided by Google or
OpenDNS) or Virtual Private Network technology.  DNS poisoning also
implies returning an incorrect IP to those attempting to resolve a
domain name, but in some cases the destination may be technically
acessible; over HTTP, for example, the user may have another method
of obtaining the IP address of the desired site and may be able to
access it if the site is configured to be the defualt server
listening at this IP address.  Blocking overflow has also been a
problem, as occasionally users outside of the censors region will be
directed through a DNS servers or DNS-rewriting network equipment
controlled by a censor, causing the request to fail.  The ease of
circumvention paired with the large risk of overblocking and blocking
overflow make DNS interference a partial, difficult, and less than
ideal censorship mechanism.
Trade-offs: DNS interference requires the censor to force a user to
traverse a controlled DNS hierarchy (or intervening network on which
the censor serves as a Active Pervasive Attacker RFC7624 to rewrite
DNS responses) for the mechanism to be effective.  It can be
circumvented by a technical savvy user that opts to use alternative
DNS resolvers (such as the public DNS resolvers provided by Google or
OpenDNS) or Virtual Private Network technology.  DNS poisoning also
implies returning an incorrect IP to those attempting to resolve a
domain name, but in some cases the destination may be technically
accessible; over HTTP, for example, the user may have another method
of obtaining the IP address of the desired site and may be able to
access it if the site is configured to be the default server
listening at this IP address.  Blocking overflow has also been a
problem, as occasionally users outside of the censors region will be
directed through a DNS servers or DNS-rewriting network equipment
controlled by a censor, causing the request to fail.  The ease of
circumvention paired with the large risk of overblocking and blocking
overflow make DNS interference a partial, difficult, and less than
ideal censorship mechanism.
outage in history Anon-SIGCOMM12.  Countries such as China, Iran,
Turkey, and the United States have discussed blocking entire TLDs as
well, but only Iran has acted by blocking all Israeli (.il) domains
Albert-2011.
outage in history AFP-2014 Anon-SIGCOMM12.  Countries such as China,
Iran, Turkey, and the United States have discussed blocking entire
TLDs as well, but only Iran has acted by blocking all Israeli (.il)
domains Albert-2011.
Empirical Examples: In 2012 the U.K.'s GCHQ used DDoS to temporarily
shutdown IRC chat rooms frequented by members of Anonymous using the
Syn Flood DDoS method; Syn Flood exploits the handshake used by TCP
to overload the victim server with so many requests that legitimate
traffic becomes slow or impossible CERT-2000.  Dissenting opinion
websites are frequently victims of DDoS around politically sensitive
events in Burma Villeneuve-2011.  Controlling parties in Russia
Kravtsova-2012, Zimbabwe Orion-2013, and Malaysia Muncaster-2013 have
been accused of using DDoS to interrupt opposition support and access
during elections.
Empirical Examples: In 2012 the U.K.'s GCHQ used DDoS to temporarily
shutdown IRC chat rooms frequented by members of Anonymous using the
Syn Flood DDoS method; Syn Flood exploits the handshake used by TCP
to overload the victim server with so many requests that legitimate
traffic becomes slow or impossible Schone-2014 CERT-2000.  Dissenting
opinion websites are frequently victims of DDoS around politically
sensitive events in Burma Villeneuve-2011.  Controlling parties in
Russia Kravtsova-2012, Zimbabwe Orion-2013, and Malaysia
Muncaster-2013 have been accused of using DDoS to interrupt
opposition support and access during elections.
Empirical Examples: Network Disconnections tend to only happen in
times of substantial unrest, largely due to the huge social,
political, and economic impact such a move has.  One of the first,
highly covered occurrences was with the Junta in Myanmar employing
Network Disconnection to help Junta forces quash a rebellion in 2007
Dobie-2007.  China disconnected the network in the Xinjiang region
during unrest in 2009 in an effort to prevent the protests from
spreading to other regions Heacock-2009.  The Arab Spring saw the the
most frequent usage of Network Disconnection, with events in Egypt
and Libya in 2011 Cowie-2011b, and Syria in 2012 Thomson-2012.
Empirical Examples: Network Disconnections tend to only happen in
times of substantial unrest, largely due to the huge social,
political, and economic impact such a move has.  One of the first,
highly covered occurrences was with the Junta in Myanmar employing
Network Disconnection to help Junta forces quash a rebellion in 2007
Dobie-2007.  China disconnected the network in the Xinjiang region
during unrest in 2009 in an effort to prevent the protests from
spreading to other regions Heacock-2009.  The Arab Spring saw the the
most frequent usage of Network Disconnection, with events in Egypt
and Libya in 2011 Cowie-2011 Cowie-2011b, and Syria in 2012 Thomson-
2012.
about dissenting views, this technique is incredibly common Johnson-
2010.  A good example of swaying public thought is China's "50-Cent
Party", composed of somewhere between 20,000 Bristow-2013 and 300,000
Fareed-2008 contributors who are paid to "guide public thought" on
local and regional issues as directed by the Ministry of Culture.
Creating apathy can be a side-effect of successfully controlling
information over time and is ideal for a censorship regime Gao-2014.
about dissenting views, this technique is incredibly common
Calamur-2013 AP-2012 Hopkins-2011 Guardian-2014 Johnson-2010.  A good
example of swaying public thought is China's "50-Cent Party",
composed of somewhere between 20,000 Bristow-2013 and 300,000
Fareed-2008 contributors who are paid to "guide public thought" on
local and regional issues as directed by the Ministry of Culture.
Creating apathy can be a side-effect of successfully controlling
information over time and is ideal for a censorship regime Gao-2014.