##### United States Response

Contact: Wayne Stiefel (<stiefel@nist.gov>)

**Comments for 1^st^ Working Draft "General Requirements for Software
Controlled Measuring Instruments"**

+------+------+-----------------------------+-------------------------+
| **   | **   | **Comments**                | **Proposed Change**     |
| Docu | Page |                             |                         |
| ment | Numb |                             |                         |
| Clau | er** |                             |                         |
| se** |      |                             |                         |
+------+------+-----------------------------+-------------------------+
| 2    | 5    | The D-SW document scope     | Define categories and   |
|      |      | provides no guidance on the | appropriate             |
|      |      | matter of "universal" (user | restrictions in much    |
|      |      | programmable) vs. "built    | more detail, similar to |
|      |      | for purpose" (upgradeable,  | WELMEC Guide.           |
|      |      | but not user programmable)  |                         |
|      |      | equipment. The WELMEC 7.2   |                         |
|      |      | Software Guide document     |                         |
|      |      | immediately breaks out      |                         |
|      |      | these two categories,       |                         |
|      |      | noting the differences      |                         |
|      |      | between the two. In section |                         |
|      |      | 5.2.2, D-SW notes some      |                         |
|      |      | restrictions on the use of  |                         |
|      |      | an "off the shelf general   |                         |
|      |      | purpose computer". Why does |                         |
|      |      | the D-SW document not       |                         |
|      |      | address this matter more    |                         |
|      |      | fully?                      |                         |
+------+------+-----------------------------+-------------------------+
| Sect |      | This restates our previous  | Replace "legally        |
| ions |      | comment on the Pre-draft    | relevant" with "under   |
| 3, 5 |      | that the Secretariat noted  | metrological control".  |
| and  |      | for future discussion. The  |                         |
| 6    |      | term "**legally relevant**" |                         |
|      |      | is found in many places in  |                         |
|      |      | the document and refers to  |                         |
|      |      | both hardware and software. |                         |
|      |      | Example from 5.1.1:         |                         |
|      |      |                             |                         |
|      |      | "**Legally relevant**       |                         |
|      |      | software shall be clearly   |                         |
|      |      | identified." To improve     |                         |
|      |      | clarity we prefer "**under  |                         |
|      |      | metrological control**"     |                         |
+------+------+-----------------------------+-------------------------+
| 3    | 13   | Editorial - Add the words   | To set a special        |
| .3.3 |      | "an" and "the".             | protection to serve as  |
|      |      |                             | **an** indicator for    |
|      |      |                             | **the** case of ...     |
+------+------+-----------------------------+-------------------------+
| 5    | 15   | We concur with the          | Topics for discussion   |
| .1.1 |      | desirability of a single    | and may require further |
|      |      | software identification     | clarification.          |
|      |      | number for a piece of       |                         |
|      |      | equipment. We can, however, |                         |
|      |      | envision a machine          |                         |
|      |      | containing multiple         |                         |
|      |      | measurement channels, each  |                         |
|      |      | containing metrological     |                         |
|      |      | software, each having a     |                         |
|      |      | different purpose and       |                         |
|      |      | identification. For         |                         |
|      |      | example, a PLC equipment    |                         |
|      |      | rack might have a number of |                         |
|      |      | empty slots into which a    |                         |
|      |      | measurement channel card    |                         |
|      |      | could be inserted, perhaps  |                         |
|      |      | for measuring flow, mass,   |                         |
|      |      | temperature, etc. Each card |                         |
|      |      | contains metrological       |                         |
|      |      | software. The equipment     |                         |
|      |      | rack also has its own       |                         |
|      |      | software. The customer can  |                         |
|      |      | configure the equipment     |                         |
|      |      | rack contents. Can/should a |                         |
|      |      | single identification cover |                         |
|      |      | all possible                |                         |
|      |      | configurations? What about  |                         |
|      |      | the case of "universal"     |                         |
|      |      | measurement option cards    |                         |
|      |      | that may be used in         |                         |
|      |      | multiple types of equipment |                         |
|      |      | chassis? Perhaps special    |                         |
|      |      | cases such as this are      |                         |
|      |      | treated separately?         |                         |
+------+------+-----------------------------+-------------------------+
| 5    | 15   | Editorial - Add suffix      | Purpose: Each measuring |
| .1.1 |      | "ant".                      | instrument in use has   |
|      |      |                             | to conform to the       |
|      |      |                             | approved type. The      |
|      |      |                             | software identification |
|      |      |                             | enables surveillance    |
|      |      |                             | personnel and persons   |
|      |      |                             | affected by the         |
|      |      |                             | measurement to          |
|      |      |                             | determine whether the   |
|      |      |                             | instrument under        |
|      |      |                             | consideration is        |
|      |      |                             | conform**ant**.         |
+------+------+-----------------------------+-------------------------+
| 5.2  | 18   | Editorial - Remove repeated | Requirement (b): It     |
| .1.1 |      | word "it".                  | **~~it~~** shall be     |
|      |      |                             | shown...                |
+------+------+-----------------------------+-------------------------+
| 5    | 23   | Clause needs to be added    | 5.2.6 Requirement --    |
| .2.6 |      | regarding the update of     | add the following:      |
|      |      | universal operating systems |                         |
|      |      | (i.e., Linux, Windows,      | *...Only the software   |
|      |      | etc.) that the software may | used for measurement    |
|      |      | reside on. Some updates are | needs to be verified    |
|      |      | small patches while other   | after updating. If the  |
|      |      | updates fix potential       | software resides on a   |
|      |      | security issues.            | universal operating     |
|      |      |                             | system and the          |
|      |      |                             | universal operating     |
|      |      |                             | system is updated,      |
|      |      |                             | verification of the     |
|      |      |                             | software is not         |
|      |      |                             | required. Verification  |
|      |      |                             | of the software is only |
|      |      |                             | required when the       |
|      |      |                             | software is updated.*   |
+------+------+-----------------------------+-------------------------+
| 5.2  | 23   | This restates our comment   | A suggestion might be   |
| .6.1 |      | on the Pre-daft. Traced     | to indicate that a      |
|      |      | electronic updates appear   | manual (i.e. paper)     |
|      |      | to require an electronic    | re-approval or paper    |
|      |      | audit trail entry for the   | audit trail is          |
|      |      | instance of the update;     | necessary for these     |
|      |      | however, non-traced updates | updates.                |
|      |      | in the Figure appear not to |                         |
|      |      | require an audit trail.     |                         |
|      |      | Potentially, non-traced     |                         |
|      |      | updates might require an    |                         |
|      |      | audit trail that is         |                         |
|      |      | manifested as an update to  |                         |
|      |      | the approval certificate or |                         |
|      |      | in essence sort of a paper  |                         |
|      |      | audit trail. The            |                         |
|      |      | Secretariat added a         |                         |
|      |      | sentence "A person          |                         |
|      |      | responsible for             |                         |
|      |      | verification must be at     |                         |
|      |      | place." We do not           |                         |
|      |      | understand what this means  |                         |
|      |      | and it does not address our |                         |
|      |      | concern to document the     |                         |
|      |      | update.                     |                         |
+------+------+-----------------------------+-------------------------+
| 6.1  | 26   | In some cases...            | Delete requirement to   |
|      |      |                             | provide source code.    |
|      |      | What cases and what will be |                         |
|      |      | done with the source code?  |                         |
|      |      | Source code is proprietary  |                         |
|      |      | property of the             |                         |
|      |      | manufacturer. In lieu of    |                         |
|      |      | source code, a high level   |                         |
|      |      | flow diagram provides       |                         |
|      |      | needed information in       |                         |
|      |      | conjunction with "black     |                         |
|      |      | box" testing. The           |                         |
|      |      | Secretariat in responding   |                         |
|      |      | to the comment noted that:  |                         |
|      |      |                             |                         |
|      |      | -   Added an explanation.   |                         |
|      |      |     Where is this           |                         |
|      |      |     explanation added?      |                         |
|      |      |                             |                         |
|      |      | -   We see your point as    |                         |
|      |      |     you also refuse/deny    |                         |
|      |      |     the validation methods  |                         |
|      |      |     that are based on       |                         |
|      |      |     source code analysis.   |                         |
|      |      |     We assume that if high  |                         |
|      |      |     level conformity (5.2.5 |                         |
|      |      |     (d)) or high level      |                         |
|      |      |     fraud protection is     |                         |
|      |      |     required, the high      |                         |
|      |      |     level validation        |                         |
|      |      |     methods are suitable.   |                         |
|      |      |     We disagree. Fraud      |                         |
|      |      |     protection is not       |                         |
|      |      |     achieved through code   |                         |
|      |      |     inspection but through  |                         |
|      |      |     various sealing methods |                         |
|      |      |     and interface           |                         |
|      |      |     protections in          |                         |
|      |      |     conjunction with field  |                         |
|      |      |     enforcement.            |                         |
|      |      |                             |                         |
|      |      | -   The responsible TCs     |                         |
|      |      |     should decide about the |                         |
|      |      |     level of validation, we |                         |
|      |      |     only want to provide    |                         |
|      |      |     the tools for realising |                         |
|      |      |     this level. We would    |                         |
|      |      |     like to keep the        |                         |
|      |      |     paragraph and all the   |                         |
|      |      |     corresponding           |                         |
|      |      |     requirements in section |                         |
|      |      |     6.4. We agree with      |                         |
|      |      |     providing options for   |                         |
|      |      |     TCs however, the        |                         |
|      |      |     application of          |                         |
|      |      |     procedure B should be   |                         |
|      |      |     restricted to the rare  |                         |
|      |      |     instances involving     |                         |
|      |      |     risks to life.          |                         |
|      |      |                             |                         |
|      |      | -   Example (relevant today |                         |
|      |      |     in some countries):     |                         |
|      |      |     Evidential measurements |                         |
|      |      |     require high trust in   |                         |
|      |      |     the software of the     |                         |
|      |      |     device which can only   |                         |
|      |      |     be attested before the  |                         |
|      |      |     court on the basis of   |                         |
|      |      |     source code analysis.   |                         |
|      |      |     We disagree that        |                         |
|      |      |     evidential measurements |                         |
|      |      |     can only be attested on |                         |
|      |      |     the basis of source     |                         |
|      |      |     code analysis. This is  |                         |
|      |      |     insufficient what is    |                         |
|      |      |     required is black box   |                         |
|      |      |     testing of the          |                         |
|      |      |     instrument and software |                         |
|      |      |     to prove it meets       |                         |
|      |      |     requirements in the     |                         |
|      |      |     laboratory. and further |                         |
|      |      |     that the field          |                         |
|      |      |     processes including     |                         |
|      |      |     calibration and         |                         |
|      |      |     operational             |                         |
|      |      |     considerations have     |                         |
|      |      |     been achieved.          |                         |
+------+------+-----------------------------+-------------------------+
| 6.1  |      | Restating our comment to    | Delete requirements to  |
| and  |      | the pre-draft:              | provide source code and |
| th   |      |                             | Procedure B throughout  |
| roug |      | We have difficulty          | the document.           |
| hout |      | understanding the           |                         |
| the  |      | circumstances in which type |                         |
| docu |      | approval would require      |                         |
| ment |      | Validation procedure B.     |                         |
|      |      | Data flow analysis          |                         |
|      |      | (6.4.3.4), Code inspection  |                         |
|      |      | and Walk through (6.4.3.5)  |                         |
|      |      | and Software module testing |                         |
|      |      | (6.4.3.6) are all beyond    |                         |
|      |      | what should be required for |                         |
|      |      | type approval, and almost   |                         |
|      |      | certainly beyond the        |                         |
|      |      | capability of most type     |                         |
|      |      | approval laboratories. We   |                         |
|      |      | believe that **only** the   |                         |
|      |      | software developer has the  |                         |
|      |      | full capability to perform  |                         |
|      |      | these types of evaluations. |                         |
|      |      | In lieu of source code, a   |                         |
|      |      | high level flow diagram     |                         |
|      |      | provides needed information |                         |
|      |      | in conjunction with "black  |                         |
|      |      | box" testing.               |                         |
|      |      |                             |                         |
|      |      | The Secretariat asked that  |                         |
|      |      | we refer to the U.S. FDA's  |                         |
|      |      | "General Principles of      |                         |
|      |      | Software Validation; Final  |                         |
|      |      | Guidance for Industry and   |                         |
|      |      | FDA Staff" which considers  |                         |
|      |      | software validation.        |                         |
|      |      |                             |                         |
|      |      | We are familiar with the    |                         |
|      |      | FDA's Guidance and its      |                         |
|      |      | intended audience of        |                         |
|      |      | medical instrument          |                         |
|      |      | manufacturers. The FDA's    |                         |
|      |      | *General Principals of      |                         |
|      |      | Software Validation: Final  |                         |
|      |      | Guidance for Industry and   |                         |
|      |      | FDA Staf*" provides         |                         |
|      |      | guidance on what FDA        |                         |
|      |      | expects medical device      |                         |
|      |      | manufacturers to do to      |                         |
|      |      | ensure compliance with the  |                         |
|      |      | Quality System regulation   |                         |
|      |      | for software validation. It |                         |
|      |      | is the **device             |                         |
|      |      | manufacturer** who must     |                         |
|      |      | apply the guidance in       |                         |
|      |      | implementing their Quality  |                         |
|      |      | System not the FDA          |                         |
|      |      | compliance official The FDA |                         |
|      |      | regulation also states:     |                         |
|      |      | "When source code and       |                         |
|      |      | design specifications       |                         |
|      |      | cannot be obtained, 'black  |                         |
|      |      | box testing' must be        |                         |
|      |      | performed to confirm that   |                         |
|      |      | the software meets the      |                         |
|      |      | user's needs and its        |                         |
|      |      | intended uses." Again it is |                         |
|      |      | the device manufacturer who |                         |
|      |      | performs the 'black box     |                         |
|      |      | testing'.                   |                         |
+------+------+-----------------------------+-------------------------+
| 6.1  | 26   | We have received comments   | Emphasize methods AD,   |
| to   | to   | from both users and         | VFTM and VFTSw.         |
| 6.4  | 32   | manufacturers of weighing   |                         |
|      |      | and measuring instruments   | Eliminate or            |
|      |      | that provide additional     | de-emphasize methods    |
|      |      | rationale for not providing | DFA, CIWT and SMT       |
|      |      | source code. A summary      |                         |
|      |      | follows:                    |                         |
|      |      |                             |                         |
|      |      | -   Data Flow Analysis,     |                         |
|      |      |     Code Inspection,        |                         |
|      |      |     Walk-Through and        |                         |
|      |      |     Software Module         |                         |
|      |      |     Testing, are of limited |                         |
|      |      |     value because they do   |                         |
|      |      |     not test for the        |                         |
|      |      |     occurrence of           |                         |
|      |      |     unexpected interactions |                         |
|      |      |     between seemingly       |                         |
|      |      |     unrelated modules       |                         |
|      |      |     operating together in   |                         |
|      |      |     real time.              |                         |
|      |      |                             |                         |
|      |      | -   This type of analysis   |                         |
|      |      |     requires the highest    |                         |
|      |      |     level or specialist     |                         |
|      |      |     technical expertise,    |                         |
|      |      |     and requires access to  |                         |
|      |      |     all the necessary       |                         |
|      |      |     debugging tools and     |                         |
|      |      |     support tools that the  |                         |
|      |      |     examiner is unlikely to |                         |
|      |      |     possess.                |                         |
|      |      |                             |                         |
|      |      | -   Requiring that source   |                         |
|      |      |     code 'shall' be made    |                         |
|      |      |     available to the        |                         |
|      |      |     examiner is impractical |                         |
|      |      |     in some cases. Modern   |                         |
|      |      |     system designs utilize  |                         |
|      |      |     dozens if not hundreds  |                         |
|      |      |     of files and libraries. |                         |
|      |      |     Quality management      |                         |
|      |      |     tools in use today are  |                         |
|      |      |     also a barrier to       |                         |
|      |      |     distributing copies of  |                         |
|      |      |     source code. These QC   |                         |
|      |      |     tools are designed to   |                         |
|      |      |     restrict access to, and |                         |
|      |      |     impose version control  |                         |
|      |      |     of software modules.    |                         |
|      |      |     Software development    |                         |
|      |      |     tools such as build     |                         |
|      |      |     scripts, pre-parsers,   |                         |
|      |      |     compilers, editors,     |                         |
|      |      |     assemblers and custom   |                         |
|      |      |     code for final file     |                         |
|      |      |     generation compilers    |                         |
|      |      |     are usually licensed to |                         |
|      |      |     run on specific         |                         |
|      |      |     machines or network     |                         |
|      |      |     servers and limit the   |                         |
|      |      |     number of licensed      |                         |
|      |      |     users. Many of these    |                         |
|      |      |     tools are licensed per  |                         |
|      |      |     seat and are locked to  |                         |
|      |      |     development system via  |                         |
|      |      |     some licensing          |                         |
|      |      |     mechanism.              |                         |
|      |      |                             |                         |
|      |      | -   Potential requirement   |                         |
|      |      |     of source code by type  |                         |
|      |      |     approval examiner may   |                         |
|      |      |     pose problem for        |                         |
|      |      |     non-open-source         |                         |
|      |      |     software. Might be      |                         |
|      |      |     difficult to understand |                         |
|      |      |     the software as a       |                         |
|      |      |     whole; usually,         |                         |
|      |      |     software is written in  |                         |
|      |      |     parts and the parts     |                         |
|      |      |     interact and thus it is |                         |
|      |      |     difficult for one       |                         |
|      |      |     person to understand    |                         |
|      |      |     source code for the     |                         |
|      |      |     entire software         |                         |
|      |      |     package.                |                         |
|      |      |                             |                         |
|      |      | -   For software            |                         |
|      |      |     examination,            |                         |
|      |      |     performance-based       |                         |
|      |      |     testing of software     |                         |
|      |      |     should be used.         |                         |
|      |      |     Performance-based       |                         |
|      |      |     testing of software     |                         |
|      |      |     enables simulation of   |                         |
|      |      |     operating scenarios.    |                         |
|      |      |     Code inspection,        |                         |
|      |      |     walk-through, and data  |                         |
|      |      |     flow analysis do not    |                         |
|      |      |     enable the examiner to  |                         |
|      |      |     test potential          |                         |
|      |      |     operating conditions.   |                         |
|      |      |                             |                         |
|      |      | -   Distributing electronic |                         |
|      |      |     copies of proprietary   |                         |
|      |      |     source code, measuring  |                         |
|      |      |     system algorithms,      |                         |
|      |      |     cryptographically       |                         |
|      |      |     functions, or critical  |                         |
|      |      |     timing constraints,     |                         |
|      |      |     puts at risk the        |                         |
|      |      |     intellectual property   |                         |
|      |      |     rights of the           |                         |
|      |      |     manufacturer and        |                         |
|      |      |     introduces the          |                         |
|      |      |     potential for legal     |                         |
|      |      |     liability to the        |                         |
|      |      |     testing authority.      |                         |
+------+------+-----------------------------+-------------------------+