Management Summary
and
Reporting Memo
on Verification Observations on Landis+Gyr Testing Report

Author:

Client / contact:

Reliability Core Team

Bert van Creij

Holland Innovative

Liander

Sept-Oct 2017

Sept-Oct 2017

DISCLAIMER:
Aan deze uitspraken in dit document kunnen geen rechten worden ontleend. Deze uitspraken
zijn gebaseerd op de informatie zoals verstrekt door externe partijen. Holland Innovative B.V.
heeft
deze data naar beste kunnen verwerkt, maar kan geen verantwoordelijkheid of
aansprakelijkheid aanvaarden voor de verstrekte informatie, noch voor het gebruik daarvan of
voor enig verlies dat daaruit zou kunnen voortvloeien.
Dit document, ongeacht de verschijningsvorm, mag niet worden gepubliceerd, gereproduceerd,
getransporteerd, opgeslagen of gedistribueerd.
Holland Innovative B.V. behoudt zich het recht voor deze disclaimer te allen tijde met
onmiddellijke ingang en zonder kennisgeving vooraf te wijzigen.

REPORTING MEMO NUMBER: RM171102-17213

Contents
MANAGEMENT SUMMARY ............................................................................................ 3
1.
1.1
1.2
1.3
2.
2.1
2.2

OPDRACHTOMSCHRIJVING ................................................................................... 3
OPDRACHTOMSCHRIJVING LIANDER................................................................................................................................ 3
REFERENTIES.............................................................................................................................................................. 3
REPORTING MEMO ..................................................................................................................................................... 3
MANAGEMENT SUMMARY ...................................................................................... 4
LANDIS+GYR – ONDERZOEK EN TESTEN VAN BOSSES ......................................................................................................... 4
AANBEVELINGEN USE CASE EN UITGANGSPUNTEN ............................................................................................................ 4

REPORTING MEMO – VERIFICATION OBSERVATIONS ON LANDIS+GYR TESTING
REPORTS .................................................................................................................... 5
3.
3.1
3.2

REPORTING MEMO ................................................................................................ 5
REPORTING MEMO OVERVIEW...................................................................................................................................... 5
RELATED DOCUMENTS ................................................................................................................................................. 5

4. REPORTING MEMO – VERIFICATION OBSERVATIONS ON LANDIS+GYR TESTING
REPORTS .................................................................................................................... 6
4.1
4.2
4.3
4.4
4.5
4.6
4.7
4.8

GENERAL VERIFICATION OBSERVATIONS ........................................................................................................................... 6
VERIFICATION OBSERVATIONS ON “HALT – VIBRATION PROFILING” ..................................................................................... 7
VERIFICATION OBSERVATIONS ON “HALT – TCVIB” ......................................................................................................... 7
VERIFICATION OBSERVATIONS ON “DOE – DEFERRED THREADS”.......................................................................................... 9
VERIFICATION OBSERVATIONS ON “DOE – METER FAILURE” ............................................................................................... 9
VERIFICATION OBSERVATIONS ON “DOE – REDUCED TORQUE”.......................................................................................... 10
VERIFICATION OBSERVATIONS ON “DOE – MANIFOLD DEGRADATION”................................................................................ 10
VERIFICATION OBSERVATIONS ON “DSMR4 AND SMR5 EURO METERS - DEFERRED THREAD RELIABILITY TEST - SUMMARY REPORT OVERALL RESULTS – NEED TO EXPAND” ................................................................................................................................... 11

Page 2

REPORTING MEMO NUMBER: RM171102-17213

Management Summary
1.

Opdrachtomschrijving

1.1 Opdrachtomschrijving Liander
De opdracht zoals gesteld door Liander aan Holland Innovative B.V. betrof het verifiëren middels
observerend reviewen van een test rapport van leverancier Landis+Gyr, referentie [1], gedateerd 21-Sept2017. Andere activiteiten zoals validatie, opstellen van testplannen, uitvoeren van testen, (statistische of
fysieke) analyses, of verdere onderzoeken of conclusies door Holland Innovative vormden nadrukkelijk en
expliciet GEEN deel van deze opdracht.

1.2 Referenties
Ref [1]

Report name and number: Not explicitly available. Filename: Infield test report 21 9
17.docx. (First) Report header: DSMR4 AND SMR5 EURO METERS - DEFERRED THREAD
TESTS - OVERALL SUMMARY REPORT. Date according to report: 19-Sept-2017, Date
according to file: 21-Sept-2017. Authors/contributors: Landis+Gyr – David Ballantine,
Richard Brighton, Tony Waters. Confidential.

De referenties (documenten) gebruikt tijdens het observerend reviewen van het Landis+Gyr test rapport
ten behoeve van verificatie worden benoemd in het ‘reporting memo’ welke daarbij is opgesteld.

1.3 Reporting Memo
Het gevraagde verifiëren middels observerend reviewen is gerapporteerd aan Liander middels een
‘reporting memo’, welke de ‘Verification observations’ heeft benoemd per test zoals uitgevoerd door
Landis+Gyr, evenals de ‘Verification observations’ op de ‘results and conclusions’ zoals gesteld door
Landis+Gyr. De ‘recommendations’ van Landis+Gyr (NB deze zijn op een later moment door Landis+Gyr
aangevuld) vielen hierbij buiten de scope van de opdracht.
Dit ‘reporting memo’ is aangevuld met een Management Summary.

Page 3

REPORTING MEMO NUMBER: RM171102-17213

2.

Management Summary

2.1 Landis+Gyr – Onderzoek en Testen van bosses
Gebruikmakend van de use case en uitgangspunten en beschikbare informatie van de Netbeheerders heeft
Landis+Gyr een onderzoek uitgevoerd naar mogelijke lekkage op de boss-nut-verbinding van de DSMR4
en SMR5 gasmeters met KZ manifolds in het veld gedurende de levensduur.
De gegeven use case bevat o.a.:


Beperkte informatie over de gebruiksomstandigheden en bijbehorende invloedfactoren.



Beperkte informatie over opgetreden issues in het veld, historische kwantitatieve data ontbreekt.



Geen kwantitatieve eis welke aangetoond diende te worden.

Binnen dit kader heeft Landis+Gyr geprobeerd een mogelijk verschil in leak performance aan te tonen
tussen gasmeters met zgn. ‘bosses with deferred threads’ en gasmeters met ‘bosses with standard threads’
zoals verbonden met de manifolds. De dimensies van ‘bosses with deferred threads’ betreffen de maximaal
gevonden afwijking. Deze dimensie afwijking is gebaseerd op de beperkt beschikbare productie data van
de Landis+Gyr bosses. De aannames van Landis+Gyr hierbij betreffen het faalgedrag (faalmechanisme)
van boss-nut-verbinding en de invloedfactoren. Andere risico’s zoals andere faalmechanismes, andere
invloedfactoren, of andere lekkage locaties zijn niet in de scope meegenomen door Landis+Gyr.
In verband met de beperkte hoeveelheid beschikbare tijd en test samples is gekozen de testen middels
een Highly Accelerated Life Testing principe uit te voeren. Daardoor kan er geen gebruik gemaakt worden
van een volledige use case. Tevens is een valide statistische onderbouwing door de beperkte
steekproefgrootte niet mogelijk.
Concluderend kan op basis van de resultaten van dit onderzoek gesteld worden dat met de beschikbare
data en informatie het niet mogelijk is om de uitspraak te doen dat de meter populatie met ‘bosses with
standard threads’ en de meter populatie met ‘bosses with deferred threads’ significant verschillend zijn ten
aanzien van de geteste lekkage criteria. Hierbij moet worden opgemerkt dat de beschikbare use case
informatie en de gestelde randcondities/uitgangspunten grote invloed kunnen hebben op zowel het
onderzoek als de uitgevoerde testen. Variaties door productie, installatie en gebruik (use case) kunnen
grote invloed hebben op dit issue en op het onderzoek.

2.2 Aanbevelingen Use Case en Uitgangspunten
De Netbeheerders hebben ettelijke uitgangspunten en criteria beschikbaar gesteld voor de use case voor
het onderzoek. Teneinde deze use case volledig te maken, opgesteld conform industry-best-practice,
worden de volgende aanvullingen aanbevolen voor een use case definitie:

Page 4



Definitie van de reliability requirements voor dit issue.



Definitie van de Critical To Quality van dit issue – lekkage en locatie.



Data en informatie over issues in het veld, historische data, omvang van het risico.



Informatie over gebruikscondities en koppeling met de use case beschrijving van het product.



Installatie proces, metingen ter plekke, met bijbehorende specificaties en testen.



Productie proces, specificaties en kennis van procesvariatie(s).



Assessment van risico’s behorende bij de reliability requirements.

REPORTING MEMO NUMBER: RM171102-17213

Reporting memo – Verification
Landis+Gyr Testing reports
3.

Observations

on

Reporting Memo

3.1 Reporting Memo Overview
This reporting memo re-iterates the verification observations based upon Landis+Gyr Testing Report, with
respect to Gas meters with Deferred threads combined with KZ manifolds.
The verification observations are on the specific tests as described in the Landis+Gyr report. The verification
observations are summarized per test, with additional verification observations on the Landis+Gyr
conclusions and the Landis+Gyr overall summaries.

3.2 Related documents
The Landis+Gyr Testing Report and associated documents written and used, upon which this reporting
memo is based:


Report name: Not explicitly available. Filename: Infield test report 21 9 17.docx. Report header:
DSMR4 AND SMR5 EURO METERS - DEFERRED THREAD TESTS - OVERALL SUMMARY REPORT.



Document name: Not explicitly available. Filename: (A) DSMR SMR5 - TCVIB (Reduced Thread and
Good Meters) - Master Sheet - ....xlsx.



Document name: AN ENGINEERING REPORT for LANDIS AND GYR LIMITED, HALT OF G350-3455
AND G350-5455 GAS METERS, REPORT NO. TRA-010109-21-CR-01A, SEPTEMBER 2012. Filename:
DSMR4 HALT REPORT TRaC TRA-010109-21-CR-01A.PDF



Document name: FAILURE ANALYSIS, Rapport n° / Report number : FA-12-0346-FIN-001, Sept
2012, by Presto Engineering. Filename: FA-12-0346-FIN-001.pdf



Document name: HALT Testing For Landis+Gyr Ltd On G4 and G6 Gas Meter, Report No. TRA028984-21-CR-01A, NOVEMBER 2015, by Element Materials Technology. Filename: TRA-02898421-CR-01A.PDF



Document name: HALT Test Plan For Landis+Gyr Ltd. On G4 and G6 Gas Meter, Report No. TRA028984-21-TP-01A, 15 OCT 2015, by Element Materials Technology. Filename: TRA-028984-21TP-01A.PDF

Page 5

REPORTING MEMO NUMBER: RM171102-17213

4.

Reporting memo – Verification Observations on
Landis+Gyr Testing reports

The verification observations are summarized per test, with additional verification observations on the
conclusions and overall summaries. Specifically:


General verification observations, on conclusions and summaries



Verification observations on “HALT – Vibration profiling”



Verification observations on “HALT – TCVIB”



Verification observations on “DoE – Deferred Threads”



Verification observations on “DoE – Meter Failure”



Verification observations on “DoE – Reduced Torque”



Verification observations on “DoE – Manifold Degradation”



Verification observations on “DSMR4 and SMR5 Euro meters - deferred thread reliability test summary report - overall results – need to expand”

4.1 General verification observations
The approach seems to be based upon ‘learning by doing’ and no clear structure or process or (statistical)
substantiation has been defined and agreed upon on beforehand.
Most statements are given as conclusions from a test on multiple occasions, while they are not and cannot
solely based on that test.
Moreover, often the statement itself is not proven but also “it cannot be proven that the opposite of that
statement is true”. This is also seen in the usage of the wording ‘proves’ instead of ‘shows’/’indicates’.
Details (‘proof’) of the gas leakage having occurred in the field - including how, where, number of
occurrences, leak location on meter, when, population, failure modes/mechanism, conditions, etc. - have
not been provided.
The relationship with design and process risk assessments has not been clarified nor made available.
A revision history plus clear traceability of the Landis+Gyr reporting is not available, and some references
and background reports are missing.
Pass/Fail criteria of the leak test do not detail the leak location on the meter/manifold combination and how
is proven (tested) that no other leaks (and/or other locations than the relevant location) have occurred.
The term DoE is used while the tests executed are set up as experiments without a DoE (Design-ofExperiments) structure or approach (DfSS – Design-for-Six-Sigma, DfR – Design-for-Reliability).

Page 6

REPORTING MEMO NUMBER: RM171102-17213

4.2 Verification observations on “HALT – Vibration profiling”
What the test will prove with respect to leaks or leak tests has not been defined a priori (before the test).
No failures (leaks) have been observed during the test while the initial set-up was a destructive test so
failures had to be induced. So, when no failures are observed, the test is not harsh enough (different
chamber needed to exhibit higher stress levels?) or does not encompass all relevant stressors.
With respect to the failures focused upon: leak tightness – which location ? Connection of the meter –
which connection(s)? The health check (equal to finding functional failures?) has only been done after the
test has been finished – in case of a fail, no additional relevant info could have been gathered.
Pressure levels were measured at 0 min, 1 min, and 5 min, to find leaks. Criterion: when pressure level
dropped more than 1 mBar, then leakage. In the provided datasheets of the tests, for both deferred and
good meters: the pressure levels sometimes went up during the 5-min measurement – without explanation.
The measurement did not include indication of accuracy of measuring pressures, apparently measured in
units of 0.1 mBar – plus no MSA (GR&R) provided. Some pressure drops of 0.9 mBar observed which might
be 1 mBar thus leak when indication of measurement accuracy and R&R available.
Comparison of HALT Grms levels to natural phenomena such as earth quakes does not add to the
significance of the test or its outcome.

4.3 Verification observations on “HALT – TCVIB”
The statement “The test showed there was no difference between standard and deferred meters.” has not
been (statistically) proven. On beforehand, no confidence or test set-up basis has been provided (why 2x4
for both bosses). It can be said that “It has not been proven that there is a (positive or negative) difference
between deferred and standard threads with respect to leaks at XYZ location” – this should however NOT
be translated as “So the standard and deferred bosses are proven the same”.
The statement “The test did show that the nut to boss joint was the weakest in respect to leaking” is not
substantiated, for example by finding other leak locations (internal or external) failing sooner in the same
test.
The statement on “functionally failing before leakage”: see later in memo at relevant DoE.
Pass/Fail criteria of the leak test do not detail the leak location on the meter/manifold combination and how
is proven (tested) that no other leaks (and/or other locations than the relevant boss/nut location) have
occurred.

Page 7

REPORTING MEMO NUMBER: RM171102-17213

No relationship between degrees loosening / tightening and its progression throughout the test and leakage
is observed.
The statement that the tightening torque increased as the test progresses cannot be statistically proven,
see graphs (next page) with the measured torques (shown in Landis+Gyr table) – no trends upwards or
downwards can be observed. The measurement system has not been analyzed by Landis+Gyr with respect
to systematic and random error, and the measured torques are depicted as integer (no decimals). In
addition, Two-sample T-tests show no significant difference ‘over time’ between the measured torques
throughout the tests progressing. Also, regression lines did not show significant correlation between test
progression and measured torques (R-sq < 20%).

Interval Plot of Pre-Inlet; Post-Inlet
95% CI for the Mean
11
10
9
8

Data

7
6
5
4
3
2
Test number

TEST 1

TEST 2

TEST 3

TEST 4

Pre-Inlet
Individual standard deviations were used to calculate the intervals.

Page 8

TEST 1

TEST 2

TEST 3

Post-Inlet

TEST 4

REPORTING MEMO NUMBER: RM171102-17213

Interval Plot of Pre-Outlet; Post-Outlet
95% CI for the Mean
12
10

Data

8
6
4
2
0
Test number

TEST 1

TEST 2

TEST 3

TEST 4

Pre-Outlet

TEST 1

TEST 2

TEST 3

TEST 4

Post-Outlet

Individual standard deviations were used to calculate the intervals.

4.4 Verification observations on “DoE – Deferred Threads”
The statement “Increased deferred threads perform the same as standard bosses.” has not been
(statistically) proven – the test did not show proof (could not provide proof) of a difference between
standard threads and deferred threads that have been tightened.
The statement “This shows that once the nut is engaged and correctly tightened it has the same strength
and leak integrity to that of standard threads.” should refer to installation procedures with respect to
tightening torque, validation, measurement, assurance, etc. since it states ‘correctly tightened’.

4.5 Verification observations on “DoE – Meter Failure”
The statement “From this we can conclude that the meters will fail operationally before there is a leak from
the boss and nut” has not been proven since the accumulated damage built up by the electronics (functional
components) due to the elevated stresses might very well be different (so also: much larger) than the
damage accumulated by the mechanical joints due to the same stresses. (Translated: when this would be
set up as an ALT, then the Acceleration Factor would be different for the electronics than for the mechanical
nut-boss joint (different failure mechanism, different life-stress model, different Acceleration Factor)).

Page 9

REPORTING MEMO NUMBER: RM171102-17213

4.6 Verification observations on “DoE – Reduced Torque”
The meters were tested using possibly degraded manifolds – latter as stated later on. This implies that the
test cannot distinguish between the loosening of the nuts being due to the tightening or due to the
manifolds. Also, loosening of the nut (other than complete detachment) has not been quantitatively
correlated to leakage, so the effect of tightening torque on leakage (risk, probability, frequency) cannot
quantified nor proven. Tightening torque seems a factor, but the criticality of this factor nor the sensitivity
of leakage thereto has not been analyzed nor proven.

4.7 Verification observations on “DoE – Manifold degradation”
The statement “The following tables clearly demonstrates the increased amount of movement and leakages
as the test progressed, when compared to the earlier tests” is based upon 4 meters – 2 meters had leakage,
same amount as earlier test – nut movement of these 4 meters do not seem to follow a trend or clear
pattern (loosen, tighten, loosen again, all between cycles), compared to earlier tests. Too small sample
size to prove degradation, which also makes ‘clearly demonstrates’ less clear.

Page 10

REPORTING MEMO NUMBER: RM171102-17213

4.8 Verification observations on “DSMR4 and SMR5 Euro meters deferred thread reliability test - summary report - overall
results – need to expand”
The results as stated have been discussed in the previous verification observations. In overview re-iterated:


Statement “No difference in leakage performance between standard and deferred meters” has not
been proven – nor has the opposite statement (negative of original) been proven, see paragraph 2.3



Statement “Meters will functionally fail before leaks occur” has not been proven – nor has the opposite
statement (negative of original) been proven, see paragraph 2.5



Statement “Meters with excessive deferred threads cannot be fitted” should refer to dimensions
(thread, etc.) and to installation procedures, see also paragraph 2.4.



Statement “It required considerable force to induce movement and leakage of the nut to boss” seems
to be based upon practical experiences throughout the tests.



Statement “The tightening torque is a critical factor to the strength of the nut / boss joint” has not
been proven quantitatively with respect to criticality and sensitivity, see paragraph 2.6



Statement “The manifolds and manifold nuts wear through testing” – evidence is qualitatively shown
using a small sample size.



Statement “Boss & nut is the weakest element of the meter & manifold system with respect to
leakages” has not been proven – nor has the opposite statement (negative of original) been proven,
see paragraph 2.3

Page 11