Internet Initiative Japan Broadband measurement in Japan Broadband Performance and QoE: Measurement Technologies Improving the Marketplace Kenjiro Cho, IIJ
• Research Director at IIJ • a board member, mawi-wg chair of WIDE Project • involved in measurement research for 20 years -  data collection and analysis at ISP/academic networks, especially on traffic -  large-scale content distribution: traffic management –  data-center networking and cloud services 2
estimated total broadband traffic in Japan http://www.hongo.wide.ad.jp/InternetTraffic/data/ 3
• steady increase driven by different major factors –  web (late 90’s), broadband deployment/P2P-file sharing (early 2000’s), short-videos (late 2000’s), video subscriptions (mid 2010’s) • recent trends • video streaming by subscriptions • software updates: become larger and more frequent • cloud uploads: SNS, multiple-devices, auto-backups • faster access: NGN, 1G/10G access, LTE+ • closer content: CDN, cache, peering • advertisement and analytics 4
• many potential bottlenecks: hard to distinguish! WiFi interference buffer bloat interference DNS congestion accounting congestion server overload ad & analytics 5
• ISP-side • client-side • content-provider side 6
• interface counters • flow exports • DNS resolvers 7
• transit services: metered billing (e.g., by 95th-percentile over 5-min volumes) • peering: bilateral settlement-free customer-traffic exchange
2011.11 2017.11 8.0 T customer-bb (in_ave: 599429 Mbps, out_ave: 4377574 Mbps) 9
• daily traffic usage distribution –  mean of a week as weekdays and weekends differ 10
• broadband downloads over 12 years –  mean: x5.9, mode: x39 • heavy user traffic hasn't increased much • mode of mobile traffic: about broadband 10 years ago broadband
mobile
l l
l 2
• heavy users: statistically distributed, not outliers -  broadband: top 10% users account for 60% of OUT, 85% of IN -  mobile: top 10% users account for 48% of OUT, 62% of IN CCDF of daily traffic usage per user: broadband (left) and mobile (right) l 3
• port no: min(sport, dport) – increasing TCP 443, UDP 443 broadband mobile
TCP port usage over a week: broadband (left) and mobile (right)

• example: Japan earthquake in 2011
a failed link: one of the US-JP submarine cables damaged a survived link: immediately, traffic rerouted to available links an alternate link: one-day after, a new link brought up l 6
https://stats.labs.apnic.net/ipv6/JP 17
• by mapping TCP 80 and 443 src addresses to ASes
0 Peak Period Applications - North America, Fixed peak traffic in North America Sandvine Global Internet Phenomena Report 2016
• more accurate views from users • probe deployment and sampling issues • crowd-sourcing: promising for scalability - e.g., RIPE Altas, CAIDA ark 19
• started in 2010, 10k+ probes worldwide, 300+ in Japan • periodic ping and traceroute + custom measurements • abundant but noisy data, requiring statistical filtering
• mean differential-RTT of Japanese access links over 5 weeks -  between home routers and ISP’s first hop –  aggregated by the first hop address • diurnal congestion observed in evening hours –  no such congestion observed in non-FLETS access 118.23.22.236 OCN (Tokyo) 6-8ms 203.180.8.11 IIJ (Tokyo) 5-32ms 221.110.198.225 Softbank (Tokyo) 2.5-15ms 21
• http video-streaming –  video as series of segment files – video replay under client’s control • adaptive bit-rate: selected by client –  pros: cost and scalability as web technologies are used (but traffic behavior is different) –  cons: extra-delay, diverse clients' behaviors
22

● eo Hikari (K-Opticom)
● Tokai Communications source: https://ispspeedindex.netflix.com/country/japan/ 23
● Denotes ISPs that are no longer tracked, but have historical data. source: https://ispspeedindex.netflix.com/country/us/ 24
• challenges for live-streaming –  client: to shorten delays (fewer buffering) still avoiding rebuffering -  server: large-scale bursty requests 25
• steady increase in traffic –  driven by different technical and non-technical factors -  dominated by videos (short and long) • technical trends –  rise of https, QUIC, IPv6 • QoE -  better access networks, closer content, better algorithms
26