Comparison of the Methods Used toAssess the Security of Smartcards
and Information Systems
Pierre Girard, Jean-Luc Giraud{Pierre.Girard, Jean-Luc.Giraud}@gemplus.com

Smart Card Security ConferenceTokyo, March 29-30, 2001

101/03/01

Bull & Innovatron Patents

Introduction■ Device/software performance is easy to quantify :◆ SpecInt ’xx, transaction/s, bandwitdh ...

■ Device/software security level is hard to measure◆ A system seems secure until it is broken■ Nevertheless :◆ Customers want to assess what they buy◆ Manufacturers want to convince them with facts

01/03/01
2
Bull & Innovatron Patents

Comparison of methods ... - P. Girard, J.L. Giraud

Outline■ Classical flaws■ Evaluation of security assessment techniques :◆ Properties of good assessment techniques◆ Available assessment techniques■ Conclusion

01/03/01
3
Bull & Innovatron Patents

Comparison of methods ... - P. Girard, J.L. Giraud

Classical flaws■ Security through obscurity or snake oil◆ Netscape 4.5 mail password « ciphering » [1]■ Faulty design◆ Key length issue of the French « Carte Bancaire »■ Implementation shortcomings◆ Buffer overflows : 8 of the 17 1999-CERT advisories■ Back doors / deliberately introduced weaknesses◆ Exportable products are often suspicious [2]01/03/01
4
Bull & Innovatron Patents

Comparison of methods ... - P. Girard, J.L. Giraud

Security evaluation needs …
■ Expertise■ Time■ Methodology■ Information■ Independence of the evaluator

But should also be cost-effective
01/03/01
5
Bull & Innovatron Patents

Comparison of methods ... - P. Girard, J.L. Giraud

Compared Evaluation Methods■ Comparative test of the press■ Evaluation by the customer■ Audit of the manufacturer and/or partnership with it■ Evaluation by a specialised third party■ Public challenge■ Public review■ ITSEC/Common Criteria■ Public formal proof

01/03/01
6
Bull & Innovatron Patents

Comparison of methods ... - P. Girard, J.L. Giraud

Comparative tests of the Press■ Evaluation based onfunctionalities■ Hardly detects any security
problem

ExpertiseTime■ No cost

Methodology
Information
Independence
Cost

None
Detects :Low

Snake oil Design Implementation Back door¨

01/03/01
7
Bull & Innovatron Patents¨¨¨Comparison of methods ... - P. Girard, J.L. Giraud

MediumHighEvaluation by the customer■ Need in-house experts■ Opposed to the current
externalisation trendExpertise
TimeMethodology
Information
Independence
Cost

None

Detects :Low

Snake oil Design Implementation Back door

þ

01/03/01
8Bull & Innovatron Patentsþ¨¨Comparison of methods ... - P. Girard, J.L. Giraud

MediumHigh

Manufacturer audit and/orpartnership■ Conceivable only for major
customers
ExpertiseTime
Methodology
Information
Independence
Cost

None
Detects :Low

Snake oil Design Implementation Back door

þ

01/03/01
9Bull & Innovatron Patentsþþþ¨Comparison of methods ... - P. Girard, J.L. Giraud

MediumHighEvaluation by a specialised thirdparty■ 2 philosophies:◆ Consultants◆ Laboratories

Expertise■ Real specialists are spare

TimeMethodology
Information
Independence
CostLow

None

Detects :Snake oil Design Implementation Back door

þ

01/03/01
10
Bull & Innovatron Patentsþþþ¨Comparison of methods ... - P. Girard, J.L. Giraud

MediumHigh

Public challenge■ Need to be attractive■ Black box evaluation
Expertise■ Double edged sword for
manufacturers

TimeMethodology■ Examples◆ RSA challenges [3]◆ R. Moreno’s challenge[4]

Information
Independence
CostNoneDetects :Low

Snake oil Design Implementation Back door

þ

01/03/01
11
Bull & Innovatron Patentsþþ¨¨Comparison of methods ... - P. Girard, J.L. Giraud

MediumHighPublic review■ Need to be attractive■ White box evaluation :leakage of manufacturer
technologies■ Double edged sword for
manufacturers■ Definitive solution forparanoids

Expertise
TimeMethodology
Information
Independence■ Examples : AES [5], Linux,Java 2 platform, PGP, SDMI
Detects :CostLow

None

Snake oil Design Implementation Back door

þ

01/03/01
12
Bull & Innovatron PatentsþþþComparison of methods ... - P. Girard, J.L. Giraud

MediumHigh

ITSEC/Common Criteria■ Great variability according
to the evaluation level■ Beware of the content of the
Security Target / Protection
ProfileExpertise
TimeMethodology■ May use formal methods forhigh evaluation level

Information
Independence
CostLow

None
Detects :Snake oil Design Implementation Back door

þ

01/03/01
13
Bull & Innovatron Patentsþþ¨þ¨Comparison of methods ... - P. Girard, J.L. Giraud

MediumHighPublic formal proof■ Released by :◆ The manufacturer◆ A third party■ Beware of the proof level :◆ Specification◆ Design◆ Code generation■ Examples◆ Academic work on Java◆ Gemplus work on
Javacard firewall [6]Expertise
TimeMethodology
Information
Independence
CostLow

None
Detects :Snake oil Design Implementation Back door

þ

01/03/01
14
Bull & Innovatron Patentsþþ¨þ¨Comparison of methods ... - P. Girard, J.L. Giraud

MediumHigh

Conclusion for customers■ Security requires periodic re-evaluation■ Buying without in-house security experts is risky■ Always take into account the results but ALSO theevaluation method

01/03/01
15
Bull & Innovatron Patents

Comparison of methods ... - P. Girard, J.L. Giraud

Conclusion for manufacturers■ State clearly what has been evaluated and how■ Using well established technology can reduce
evaluation costs■ Choose your evaluation method according to yourmarket segment

01/03/01
16
Bull & Innovatron Patents

Comparison of methods ... - P. Girard, J.L. GiraudReferences[1]« Netscape 4.5 vulnerability », Alexey Pavlov, Bugtrack mailing list, Message-ID: <370CE37B.2A066C20@uic.
uic.nnov.
nnov.ru>,ru>, Apr 08 1999

[2]

« Back Doors, Export, and the NSA », Bruce Schneier,Schneier, CRYPTO-GRAM, February 15, 1999

[3]

« Cryptographic Challenges », RSA Labs, http://www.rsasecurity.com/rsalabsrsalabs/challenges//challenges/
http://www.rsasecurity.com/[4]« Carte à puce: Roland Moreno offre un million de francs », Dépêche AFP, 13/03/2000[5]« Advanced Encryption Standard », NIST, http://csrcaes//
http://csrc..nist.
nist.gov/encryption/gov/encryption/aes[6]« Formal Model and Implementation of the Java Card Dynamic Security Policy », Stéphanie Motré,Motré, AFADL2000,
http://www-lsr
motre..doc
http://www-lsr..imag.
imag.fr/afadl2000/Programme/Articles/
fr/afadl2000/Programme/Articles/motre

01/03/01
17
Bull & Innovatron Patents

Comparison of methods ... - P. Girard, J.L. Giraud