The	Communications	Sector	– Cybersecurity	Through	the	Risk	

Management	Measurement	Prism

Panelists

Moderator:		Robert	Mayer,	USTelecom	Vice	President	Industry	and	State	Affairs

Chris	Boyer,	AT&T,	AVP	Global	Cybersecurity	Policy

Kathryn	Condello,	CenturyLink,	Director	of	NS/EP

John	Marinho,	CTIA,	Vice	President Technology	&	Cybersecurity

Matt	Tooley,	NCTA,	Vice	President,	Broadband	Technology

Jesse	Ward,	NTCA	Director,	Industry	&	Policy	Analysis	
AGENDA

Ø CSRIC IV WG4 Sector Adaptation of NIST Framework

Ø The Challenges of Measuring Cybersecurity PLUS Risk 

Management

Ø Panel Discussion

Ø Questions and Answers
CSCC WG4 Measurement Working Group
Risk	Management

The	Challenges	of	Measuring	

Cybersecurity	PLUS	Risk	

Management

CSCC Measurement Working Group
Qualitative	vs	Quantitative
Real	World	

Example

DRAFT	- CONFIDENTIAL
Risk	Matrix	Example

DRAFT	- CONFIDENTIAL

Likelihood

Frequent Malware M H

Probable L M M

Infrequent L L Breach

Low Medium High

Impact

• Commodity	malware	

infections

• Happen	every	day

• Relatively	inexpensive	to	

remediate

• Data	breach

• Happens	infrequently

• Much	higher	impact	if	it	

does

Which	one	is	

more	

important?
Do	Numbers	Make	It	Better?

DRAFT	- CONFIDENTIAL

Frequent

67-100% L M H

Probable

34-66% L M M

Unlikely

0-33% L L L

Low Medium High

0-$100k 100k-$1m >	$1m
Examples	with	Numbers

DRAFT	- CONFIDENTIAL

Frequent

67-100% Malware M H

Probable

34-66% L M

Small

Breach

Unlikely

0-33% L L Large

Breach

Low Medium High

0-$100k 100k-$1m >	$1m

According	to	the	

matrix,	commodity	

malware	and	a	data	

breech	are	the	same	

level	of	risk.

Risk	=	Impact	*	Likelihood
Quantitative

DRAFT	- CONFIDENTIAL
DRAFT	- CONFIDENTIAL
DRAFT	- CONFIDENTIAL




Conclusions

Measuring	cybersecurity	is	a	complex	topic	as	we	learned	in	FCC	CSRIC	WG#4	and	doesn’t	

lend	itself	to	a	prescriptive	or	one-size	fits	all	approach. The	NIST	cybersecurity	framework	

is	inherently	flexible. One	of	the	primary	reasons	for	the	success	of	the	framework	is	that	

it	avoids	being	a	checklist	and	is	more	of	a	risk	management	program	that	can	be	

implemented	in	a	variety	of	ways	to	meet	differing	business	needs. NIST	should	maintain	

that	structure	as	it	considers	metrics.

Any	metrics	that	NIST	proposes	should	be	process	oriented,	to	help	companies	determine	

if	their	risk	management	processes	are	adequate	given	the	risks. There	is	a	wide	variety	of	

research	into	how	to	evaluate	the	effectiveness	of	processes. NIST	can	use	the	framework	

process	to	help	companies	best	determine	if	their	risk	management	process	is	sufficient	to	

mitigate	their	risk.

In	order	to	help	companies	measure	their	risk	management	programs	NIST	could	propose	

examples	of	how	companies	measure	these	programs	today	but	shouldn’t	try	to	

standardize	or	identify	best	practices. Similar	to	the	point	above,	companies	are	best	

positioned	to	understand	their	cyber	risk	and	internal	risk	management	programs	and	

NIST	can	help	them	evaluate	the	effectiveness	of	those	programs	by	sharing	how	others	in	

industry	measure	the	effectiveness	of	their	risk	management	programs	today	but	should	

avoid	a	one-sized	fits	all	approach	to	measurement.
Conclusions

NIST’s	proposals	like	mean	time	to	detection	and	other	measures	have	been	looked	at	in	

the	past	and	could	potentially	be	futile It	is	difficult	to	calculate	those	types	of	measures	

because	in	many	cases	companies,	for	example,	don’t	even	know	if	they	have	a	cyber	issue	

until	they	find	out. In	that	context	the	actual	method	of	calculating	the	metric	may	be	

confusing	and	not	relevant	to	addressing	the	attack.

Any	metrics	that	companies	use	have	to	be	outcome	based	and	related	to	an	

actual	issue	that	is	within	the	scope	and	control	of	the	entity	that	is	being	

measured. This	was	a	key	finding	in	Working	Group	#4. Metrics	need	to	be	

actionable	for	that	particular	business. For	example,	communications	

companies	routinely	measure	things	like	service	outages	and	from	a	service	

outage	the	mean	time	to	restore	service	and	other	related	measures. In	that	

case	the	communications	service	provider	is	responsible	for	and	has	control	over	

when	their	service	is	restored. Thus	the	companies	are	measuring	an	outcome	

that	is	within	their	span	of	control. Example	of	this	in	evaluating	a	risk	

management	process	may	be	metrics	like	#	of	employees	trained	on	risks	etc.