Repository: prometheus/procfs (go)

Issue:
mountstats error: invalid NFS per-operations stats
<!--
	Please note: GitHub issues should only be used for feature requests and
	bug reports. For general usage/help/discussions, please refer to one of:

	- #prometheus on freenode
	- the Prometheus Users list: https://groups.google.com/forum/#!forum/prometheus-users

	Before filing a bug report, note that running node_exporter in Docker is
	not recommended, for the reasons detailed in the README:

	https://github.com/prometheus/node_exporter#using-docker

	Finally, also note that node_exporter is focused on *NIX kernels, and the
	WMI exporter should be used instead on Windows.

	For bug reports, please fill out the below fields and provide as much detail
	as possible about your issue.  For feature requests, you may omit the
	following template.
-->
### Host operating system: output of `uname -a`
```
Linux n001 5.3.0-18-generic #19-Ubuntu SMP Tue Oct 8 20:14:06 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
```

### node_exporter version: output of `node_exporter --version`
<!-- If building from source, run `make` first. -->
v0.18.1

### node_exporter command line flags
<!-- Please list all of the command line flags -->
```
node_exporter --path.procfs=/host/proc --path.sysfs=/host/sys --web.listen-address=0.0.0.0:9100 --collector.filesystem.ignored-mount-points=^/(dev|proc|sys|var/lib/docker/.+|var/lib/rkt/pods/.+)($|/) --collector.filesystem.ignored-fs-types=^(autofs|binfmt_misc|cgroup|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|mqueue|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|sysfs|tracefs)$ --collector.mountstats
```

### Are you running node_exporter in Docker?
<!-- Please note the warning above. -->
yes

### What did you do that produced an error?
Just trying to get stats on my nfs volume mounts (k8s).

### What did you expect to see?
No errors, and stats in grafana.

### What did you see instead?
```
ERROR: mountstats collector failed after 0.003237s: failed to parse mountstats: invalid NFS per-operations stats: [NULL: 1 1 0 44 24 0 0 0 0]
```

I'm using the latest [prometheus-operator chart](https://github.com/helm/charts/tree/master/stable/prometheus-operator) here and noticed recently that my nfs volume stats are missing. I changed my base OS from container-linux to ubuntu 19.10. I'm also running a single instance of ubuntu 18.04 and that is also not reporting stats.

Example output from `/proc/self/mountstats`:

```
device fs.example.com:/volume4/apps/home-automation/node-red-data mounted on /var/lib/kubelet/pods/1c2215a7-0d92-4df5-83ce-a807bcc2f8c8/volumes/kubernetes.io~nfs/home-automation--node-red-data--pv0001 with fstype nfs4 statvers=1.1
        opts:   rw,vers=4.1,rsize=131072,wsize=131072,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.1.191,local_lock=none
        age:    83520
        impl_id:        name='',domain='',date='0,0'
        caps:   caps=0x3fff7,wtmult=512,dtsize=32768,bsize=0,namlen=255
        nfsv4:  bm0=0xfdffafff,bm1=0xf9be3e,bm2=0x800,acl=0x0,sessions,pnfs=not configured,lease_time=90,lease_expired=0
        sec:    flavor=1,pseudoflavor=1
        events: 52472 472680 16671 57552 2104 9565 749555 9568641 168 24103 1 267134 3350 20097 116581 18214 43757 111141 0 28 9563845 34 0 0 0 0 0 
        bytes:  2021340783 39056395530 0 0 1788561151 39087991255 442605 9557343 
        RPC iostats version: 1.1  p/v: 100003/4 (nfs)
        xprt:   tcp 940 0 2 0 1 938505 938504 0 12756069 0 32 254729 10823602
        per-op statistics
                NULL: 1 1 0 44 24 0 0 0 0
                READ: 34096 34096 0 7103096 1792122744 2272 464840 467945 0
               WRITE: 322308 322308 0 39161277084 56725504 401718334 10139998 411864389 0
              COMMIT: 12541 12541 0 2709896 1304264 342 7179 7819 0
                OPEN: 12637 12637 0 3923256 4659940 871 57185 58251 394
        OPEN_CONFIRM: 0 0 0 0 0 0 0 0 0
         OPEN_NOATTR: 98741 98741 0 25656212 31630800 3366 77710 82693 0
        OPEN_DOWNGRADE: 0 0 0 0 0 0 0 0 0
               CLOSE: 87075 87075 0 18778608 15308496 2026 49131 52399 116
             SETATTR: 24576 24576 0 5825876 6522260 643 34384 35650 0
              FSINFO: 1 1 0 168 152 0 0 0 0
               RENEW: 0 0 0 0 0 0 0 0 0
         SETCLIENTID: 0 0 0 0 0 0 0 0 0
        SETCLIENTID_CONFIRM: 0 0 0 0 0 0 0 0 0
                LOCK: 22512 22512 0 5417628 2521312 1088 17407 18794 2
               LOCKT: 0 0 0 0 0 0 0 0 0
               LOCKU: 21247 21247 0 4589352 2379664 315 8409 9003 0
              ACCESS: 1466 1466 0 298160 246288 22 1394 1492 0
             GETATTR: 52480 52480 0 10015464 12694076 2930 30069 34502 0
              LOOKUP: 11727 11727 0 2518200 2886376 272 16935 17662 3546
         LOOKUP_ROOT: 0 0 0 0 0 0 0 0 0
              REMOVE: 833 833 0 172236 95268 15 4566 4617 68
              RENAME: 11431 11431 0 3150708 1737512 211 52649 53091 0
                LINK: 1 1 0 288 292 0 0 0 0
             SYMLINK: 0 0 0 0 0 0 0 0 0
              CREATE: 77 77 0 18292 23496 0 363 371 11
            PATHCONF: 1 1 0 164 116 0 0 0 0
              STATFS: 7420 7420 0 1394960 1187200 144 4672 4975 0
            READLINK: 4 4 0 704 488 0 1 1 0
             READDIR: 1353 1353 0 304024 2902928 11 4326 4411 0
         SERVER_CAPS: 9 9 0 1548 1476 0 3 3 0
         DELEGRETURN: 232 232 0 48896 37120 811 300 1115 0
              GETACL: 0 0 0 0 0 0 0 0 0
              SETACL: 0 0 0 0 0 0 0 0 0
        FS_LOCATIONS: 0 0 0 0 0 0 0 0 0
        RELEASE_LOCKOWNER: 0 0 0 0 0 0 0 0 0
             SECINFO: 0 0 0 0 0 0 0 0 0
        FSID_PRESENT: 0 0 0 0 0 0 0 0 0
         EXCHANGE_ID: 2 2 0 464 200 0 0 0 0
        CREATE_SESSION: 1 1 0 192 124 0 0 0 0
        DESTROY_SESSION: 0 0 0 0 0 0 0 0 0
            SEQUENCE: 0 0 0 0 0 0 0 0 0
        GET_LEASE_TIME: 0 0 0 0 0 0 0 0 0
        RECLAIM_COMPLETE: 1 1 0 124 88 0 81 81 0
           LAYOUTGET: 0 0 0 0 0 0 0 0 0
        GETDEVICEINFO: 0 0 0 0 0 0 0 0 0
        LAYOUTCOMMIT: 0 0 0 0 0 0 0 0 0
        LAYOUTRETURN: 0 0 0 0 0 0 0 0 0
        SECINFO_NO_NAME: 0 0 0 0 0 0 0 0 0
        TEST_STATEID: 0 0 0 0 0 0 0 0 0
        FREE_STATEID: 10413 10413 0 1416168 916344 147 3518 3871 10413
        GETDEVICELIST: 0 0 0 0 0 0 0 0 0
        BIND_CONN_TO_SESSION: 0 0 0 0 0 0 0 0 0
        DESTROY_CLIENTID: 0 0 0 0 0 0 0 0 0
                SEEK: 0 0 0 0 0 0 0 0 0
            ALLOCATE: 0 0 0 0 0 0 0 0 0
          DEALLOCATE: 0 0 0 0 0 0 0 0 0
         LAYOUTSTATS: 0 0 0 0 0 0 0 0 0
               CLONE: 0 0 0 0 0 0 0 0 0
                COPY: 0 0 0 0 0 0 0 0 0
        OFFLOAD_CANCEL: 0 0 0 0 0 0 0 0 0
             LOOKUPP: 0 0 0 0 0 0 0 0 0
         LAYOUTERROR: 0 0 0 0 0 0 0 0 0
```

Crosspost from https://github.com/prometheus/node_exporter/issues/1642

Pull request:
mountstats: handle extended per-op stats
Linux 5.3 added a field to the per-operator NFS mount stats.
This change reads the extra field when it exists, and ignores
any additional fields which may be added in the future.

Fixes #275 

Signed-off-by: Paul Gier <pgier@redhat.com>

Patch:
diff --git a/.circleci/config.yml b/.circleci/config.yml
index 212f50a..ffd28ca 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -34,7 +34,7 @@ jobs:
     steps:
     - checkout
     - run: sudo pip install codespell
-    - run: codespell --skip=".git,./vendor,ttar,go.mod,go.sum" -L uint,packages\'
+    - run: codespell --skip=".git,./vendor,ttar,go.mod,go.sum" -L uint,packages\',ded,alo,als
 
 workflows:
   version: 2
diff --git a/fixtures.ttar b/fixtures.ttar
index 45a7321..b2ca10c 100644
--- a/fixtures.ttar
+++ b/fixtures.ttar
@@ -1609,6 +1609,37 @@ Mode: 664
 Directory: fixtures/proc/fs
 Mode: 755
 # ttar - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+Directory: fixtures/proc/fs/fscache
+Mode: 755
+# ttar - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+Path: fixtures/proc/fs/fscache/stats
+Lines: 24
+FS-Cache statistics
+Cookies: idx=3 dat=67877 spc=0
+Objects: alc=67473 nal=0 avl=67473 ded=388
+ChkAux : non=12 ok=33 upd=44 obs=55
+Pages  : mrk=547164 unc=364577
+Acquire: n=67880 nul=98 noc=25 ok=67780 nbf=39 oom=26
+Lookups: n=67473 neg=67470 pos=58 crt=67473 tmo=85
+Invals : n=14 run=13
+Updates: n=7 nul=3 run=8
+Relinqs: n=394 nul=1 wcr=2 rtr=3
+AttrChg: n=6 ok=5 nbf=4 oom=3 run=2
+Allocs : n=20 ok=19 wt=18 nbf=17 int=16
+Allocs : ops=15 owt=14 abt=13
+Retrvls: n=151959 ok=82823 wt=23467 nod=69136 nbf=15 int=69 oom=43
+Retrvls: ops=151959 owt=42747 abt=44
+Stores : n=225565 ok=225565 agn=12 nbf=13 oom=14
+Stores : ops=69156 run=294721 pgs=225565 rxd=225565 olm=43
+VmScan : nos=364512 gon=2 bsy=43 can=12 wt=66
+Ops    : pend=42753 run=221129 enq=628798 can=11 rej=88
+Ops    : ini=377538 dfr=27 rel=377538 gc=37
+CacheOp: alo=1 luo=2 luc=3 gro=4
+CacheOp: inv=5 upo=6 dro=7 pto=8 atc=9 syn=10
+CacheOp: rap=11 ras=12 alp=13 als=14 wrp=15 ucp=16 dsp=17
+CacheEv: nsp=18 stl=19 rtr=20 cul=21EOF
+Mode: 644
+# ttar - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 Directory: fixtures/proc/fs/xfs
 Mode: 755
 # ttar - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
diff --git a/fscache.go b/fscache.go
new file mode 100644
index 0000000..8783cf3
--- /dev/null
+++ b/fscache.go
@@ -0,0 +1,422 @@
+// Copyright 2019 The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package procfs
+
+import (
+	"bufio"
+	"bytes"
+	"fmt"
+	"io"
+	"strconv"
+	"strings"
+
+	"github.com/prometheus/procfs/internal/util"
+)
+
+// Fscacheinfo represents fscache statistics.
+type Fscacheinfo struct {
+	// Number of index cookies allocated
+	IndexCookiesAllocated uint64
+	// data storage cookies allocated
+	DataStorageCookiesAllocated uint64
+	// Number of special cookies allocated
+	SpecialCookiesAllocated uint64
+	// Number of objects allocated
+	ObjectsAllocated uint64
+	// Number of object allocation failures
+	ObjectAllocationsFailure uint64
+	// Number of objects that reached the available state
+	ObjectsAvailable uint64
+	// Number of objects that reached the dead state
+	ObjectsDead uint64
+	// Number of objects that didn't have a coherency check
+	ObjectsWithoutCoherencyCheck uint64
+	// Number of objects that passed a coherency check
+	ObjectsWithCoherencyCheck uint64
+	// Number of objects that needed a coherency data update
+	ObjectsNeedCoherencyCheckUpdate uint64
+	// Number of objects that were declared obsolete
+	ObjectsDeclaredObsolete uint64
+	// Number of pages marked as being cached
+	PagesMarkedAsBeingCached uint64
+	// Number of uncache page requests seen
+	UncachePagesRequestSeen uint64
+	// Number of acquire cookie requests seen
+	AcquireCookiesRequestSeen uint64
+	// Number of acq reqs given a NULL parent
+	AcquireRequestsWithNullParent uint64
+	// Number of acq reqs rejected due to no cache available
+	AcquireRequestsRejectedNoCacheAvailable uint64
+	// Number of acq reqs succeeded
+	AcquireRequestsSucceeded uint64
+	// Number of acq reqs rejected due to error
+	AcquireRequestsRejectedDueToError uint64
+	// Number of acq reqs failed on ENOMEM
+	AcquireRequestsFailedDueToEnomem uint64
+	// Number of lookup calls made on cache backends
+	LookupsNumber uint64
+	// Number of negative lookups made
+	LookupsNegative uint64
+	// Number of positive lookups made
+	LookupsPositive uint64
+	// Number of objects created by lookup
+	ObjectsCreatedByLookup uint64
+	// Number of lookups timed out and requeued
+	LookupsTimedOutAndRequed uint64
+	InvalidationsNumber      uint64
+	InvalidationsRunning     uint64
+	// Number of update cookie requests seen
+	UpdateCookieRequestSeen uint64
+	// Number of upd reqs given a NULL parent
+	UpdateRequestsWithNullParent uint64
+	// Number of upd reqs granted CPU time
+	UpdateRequestsRunning uint64
+	// Number of relinquish cookie requests seen
+	RelinquishCookiesRequestSeen uint64
+	// Number of rlq reqs given a NULL parent
+	RelinquishCookiesWithNullParent uint64
+	// Number of rlq reqs waited on completion of creation
+	RelinquishRequestsWaitingCompleteCreation uint64
+	// Relinqs rtr
+	RelinquishRetries uint64
+	// Number of attribute changed requests seen
+	AttributeChangedRequestsSeen uint64
+	// Number of attr changed requests queued
+	AttributeChangedRequestsQueued uint64
+	// Number of attr changed rejected -ENOBUFS
+	AttributeChangedRejectDueToEnobufs uint64
+	// Number of attr changed failed -ENOMEM
+	AttributeChangedFailedDueToEnomem uint64
+	// Number of attr changed ops given CPU time
+	AttributeChangedOps uint64
+	// Number of allocation requests seen
+	AllocationRequestsSeen uint64
+	// Number of successful alloc reqs
+	AllocationOkRequests uint64
+	// Number of alloc reqs that waited on lookup completion
+	AllocationWaitingOnLookup uint64
+	// Number of alloc reqs rejected -ENOBUFS
+	AllocationsRejectedDueToEnobufs uint64
+	// Number of alloc reqs aborted -ERESTARTSYS
+	AllocationsAbortedDueToErestartsys uint64
+	// Number of alloc reqs submitted
+	AllocationOperationsSubmitted uint64
+	// Number of alloc reqs waited for CPU time
+	AllocationsWaitedForCPU uint64
+	// Number of alloc reqs aborted due to object death
+	AllocationsAbortedDueToObjectDeath uint64
+	// Number of retrieval (read) requests seen
+	RetrievalsReadRequests uint64
+	// Number of successful retr reqs
+	RetrievalsOk uint64
+	// Number of retr reqs that waited on lookup completion
+	RetrievalsWaitingLookupCompletion uint64
+	// Number of retr reqs returned -ENODATA
+	RetrievalsReturnedEnodata uint64
+	// Number of retr reqs rejected -ENOBUFS
+	RetrievalsRejectedDueToEnobufs uint64
+	// Number of retr reqs aborted -ERESTARTSYS
+	RetrievalsAbortedDueToErestartsys uint64
+	// Number of retr reqs failed -ENOMEM
+	RetrievalsFailedDueToEnomem uint64
+	// Number of retr reqs submitted
+	RetrievalsRequests uint64
+	// Number of retr reqs waited for CPU time
+	RetrievalsWaitingCPU uint64
+	// Number of retr reqs aborted due to object death
+	RetrievalsAbortedDueToObjectDeath uint64
+	// Number of storage (write) requests seen
+	StoreWriteRequests uint64
+	// Number of successful store reqs
+	StoreSuccessfulRequests uint64
+	// Number of store reqs on a page already pending storage
+	StoreRequestsOnPendingStorage uint64
+	// Number of store reqs rejected -ENOBUFS
+	StoreRequestsRejectedDueToEnobufs uint64
+	// Number of store reqs failed -ENOMEM
+	StoreRequestsFailedDueToEnomem uint64
+	// Number of store reqs submitted
+	StoreRequestsSubmitted uint64
+	// Number of store reqs granted CPU time
+	StoreRequestsRunning uint64
+	// Number of pages given store req processing time
+	StorePagesWithRequestsProcessing uint64
+	// Number of store reqs deleted from tracking tree
+	StoreRequestsDeleted uint64
+	// Number of store reqs over store limit
+	StoreRequestsOverStoreLimit uint64
+	// Number of release reqs against pages with no pending store
+	ReleaseRequestsAgainstPagesWithNoPendingStorage uint64
+	// Number of release reqs against pages stored by time lock granted
+	ReleaseRequestsAgainstPagesStoredByTimeLockGranted uint64
+	// Number of release reqs ignored due to in-progress store
+	ReleaseRequestsIgnoredDueToInProgressStore uint64
+	// Number of page stores cancelled due to release req
+	PageStoresCancelledByReleaseRequests uint64
+	VmscanWaiting                        uint64
+	// Number of times async ops added to pending queues
+	OpsPending uint64
+	// Number of times async ops given CPU time
+	OpsRunning uint64
+	// Number of times async ops queued for processing
+	OpsEnqueued uint64
+	// Number of async ops cancelled
+	OpsCancelled uint64
+	// Number of async ops rejected due to object lookup/create failure
+	OpsRejected uint64
+	// Number of async ops initialised
+	OpsInitialised uint64
+	// Number of async ops queued for deferred release
+	OpsDeferred uint64
+	// Number of async ops released (should equal ini=N when idle)
+	OpsReleased uint64
+	// Number of deferred-release async ops garbage collected
+	OpsGarbageCollected uint64
+	// Number of in-progress alloc_object() cache ops
+	CacheopAllocationsinProgress uint64
+	// Number of in-progress lookup_object() cache ops
+	CacheopLookupObjectInProgress uint64
+	// Number of in-progress lookup_complete() cache ops
+	CacheopLookupCompleteInPorgress uint64
+	// Number of in-progress grab_object() cache ops
+	CacheopGrabObjectInProgress uint64
+	CacheopInvalidations        uint64
+	// Number of in-progress update_object() cache ops
+	CacheopUpdateObjectInProgress uint64
+	// Number of in-progress drop_object() cache ops
+	CacheopDropObjectInProgress uint64
+	// Number of in-progress put_object() cache ops
+	CacheopPutObjectInProgress uint64
+	// Number of in-progress attr_changed() cache ops
+	CacheopAttributeChangeInProgress uint64
+	// Number of in-progress sync_cache() cache ops
+	CacheopSyncCacheInProgress uint64
+	// Number of in-progress read_or_alloc_page() cache ops
+	CacheopReadOrAllocPageInProgress uint64
+	// Number of in-progress read_or_alloc_pages() cache ops
+	CacheopReadOrAllocPagesInProgress uint64
+	// Number of in-progress allocate_page() cache ops
+	CacheopAllocatePageInProgress uint64
+	// Number of in-progress allocate_pages() cache ops
+	CacheopAllocatePagesInProgress uint64
+	// Number of in-progress write_page() cache ops
+	CacheopWritePagesInProgress uint64
+	// Number of in-progress uncache_page() cache ops
+	CacheopUncachePagesInProgress uint64
+	// Number of in-progress dissociate_pages() cache ops
+	CacheopDissociatePagesInProgress uint64
+	// Number of object lookups/creations rejected due to lack of space
+	CacheevLookupsAndCreationsRejectedLackSpace uint64
+	// Number of stale objects deleted
+	CacheevStaleObjectsDeleted uint64
+	// Number of objects retired when relinquished
+	CacheevRetiredWhenReliquished uint64
+	// Number of objects culled
+	CacheevObjectsCulled uint64
+}
+
+// Fscacheinfo returns information about current fscache statistics.
+// See https://www.kernel.org/doc/Documentation/filesystems/caching/fscache.txt
+func (fs FS) Fscacheinfo() (Fscacheinfo, error) {
+	b, err := util.ReadFileNoStat(fs.proc.Path("fs/fscache/stats"))
+	if err != nil {
+		return Fscacheinfo{}, err
+	}
+
+	m, err := parseFscacheinfo(bytes.NewReader(b))
+	if err != nil {
+		return Fscacheinfo{}, fmt.Errorf("failed to parse Fscacheinfo: %v", err)
+	}
+
+	return *m, nil
+}
+
+func setFSCacheFields(fields []string, setFields ...*uint64) error {
+	var err error
+	if len(fields) < len(setFields) {
+		return fmt.Errorf("Insufficient number of fields, expected %v, got %v", len(setFields), len(fields))
+	}
+
+	for i := range setFields {
+		*setFields[i], err = strconv.ParseUint(strings.Split(fields[i], "=")[1], 0, 64)
+		if err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func parseFscacheinfo(r io.Reader) (*Fscacheinfo, error) {
+	var m Fscacheinfo
+	s := bufio.NewScanner(r)
+	for s.Scan() {
+		fields := strings.Fields(s.Text())
+		if len(fields) < 2 {
+			return nil, fmt.Errorf("malformed Fscacheinfo line: %q", s.Text())
+		}
+
+		switch fields[0] {
+		case "Cookies:":
+			err := setFSCacheFields(fields[1:], &m.IndexCookiesAllocated, &m.DataStorageCookiesAllocated,
+				&m.SpecialCookiesAllocated)
+			if err != nil {
+				return &m, err
+			}
+		case "Objects:":
+			err := setFSCacheFields(fields[1:], &m.ObjectsAllocated, &m.ObjectAllocationsFailure,
+				&m.ObjectsAvailable, &m.ObjectsDead)
+			if err != nil {
+				return &m, err
+			}
+		case "ChkAux":
+			err := setFSCacheFields(fields[2:], &m.ObjectsWithoutCoherencyCheck, &m.ObjectsWithCoherencyCheck,
+				&m.ObjectsNeedCoherencyCheckUpdate, &m.ObjectsDeclaredObsolete)
+			if err != nil {
+				return &m, err
+			}
+		case "Pages":
+			err := setFSCacheFields(fields[2:], &m.PagesMarkedAsBeingCached, &m.UncachePagesRequestSeen)
+			if err != nil {
+				return &m, err
+			}
+		case "Acquire:":
+			err := setFSCacheFields(fields[1:], &m.AcquireCookiesRequestSeen, &m.AcquireRequestsWithNullParent,
+				&m.AcquireRequestsRejectedNoCacheAvailable, &m.AcquireRequestsSucceeded, &m.AcquireRequestsRejectedDueToError,
+				&m.AcquireRequestsFailedDueToEnomem)
+			if err != nil {
+				return &m, err
+			}
+		case "Lookups:":
+			err := setFSCacheFields(fields[1:], &m.LookupsNumber, &m.LookupsNegative, &m.LookupsPositive,
+				&m.ObjectsCreatedByLookup, &m.LookupsTimedOutAndRequed)
+			if err != nil {
+				return &m, err
+			}
+		case "Invals":
+			err := setFSCacheFields(fields[2:], &m.InvalidationsNumber, &m.InvalidationsRunning)
+			if err != nil {
+				return &m, err
+			}
+		case "Updates:":
+			err := setFSCacheFields(fields[1:], &m.UpdateCookieRequestSeen, &m.UpdateRequestsWithNullParent,
+				&m.UpdateRequestsRunning)
+			if err != nil {
+				return &m, err
+			}
+		case "Relinqs:":
+			err := setFSCacheFields(fields[1:], &m.RelinquishCookiesRequestSeen, &m.RelinquishCookiesWithNullParent,
+				&m.RelinquishRequestsWaitingCompleteCreation, &m.RelinquishRetries)
+			if err != nil {
+				return &m, err
+			}
+		case "AttrChg:":
+			err := setFSCacheFields(fields[1:], &m.AttributeChangedRequestsSeen, &m.AttributeChangedRequestsQueued,
+				&m.AttributeChangedRejectDueToEnobufs, &m.AttributeChangedFailedDueToEnomem, &m.AttributeChangedOps)
+			if err != nil {
+				return &m, err
+			}
+		case "Allocs":
+			if strings.Split(fields[2], "=")[0] == "n" {
+				err := setFSCacheFields(fields[2:], &m.AllocationRequestsSeen, &m.AllocationOkRequests,
+					&m.AllocationWaitingOnLookup, &m.AllocationsRejectedDueToEnobufs, &m.AllocationsAbortedDueToErestartsys)
+				if err != nil {
+					return &m, err
+				}
+			} else {
+				err := setFSCacheFields(fields[2:], &m.AllocationOperationsSubmitted, &m.AllocationsWaitedForCPU,
+					&m.AllocationsAbortedDueToObjectDeath)
+				if err != nil {
+					return &m, err
+				}
+			}
+		case "Retrvls:":
+			if strings.Split(fields[1], "=")[0] == "n" {
+				err := setFSCacheFields(fields[1:], &m.RetrievalsReadRequests, &m.RetrievalsOk, &m.RetrievalsWaitingLookupCompletion,
+					&m.RetrievalsReturnedEnodata, &m.RetrievalsRejectedDueToEnobufs, &m.RetrievalsAbortedDueToErestartsys,
+					&m.RetrievalsFailedDueToEnomem)
+				if err != nil {
+					return &m, err
+				}
+			} else {
+				err := setFSCacheFields(fields[1:], &m.RetrievalsRequests, &m.RetrievalsWaitingCPU, &m.RetrievalsAbortedDueToObjectDeath)
+				if err != nil {
+					return &m, err
+				}
+			}
+		case "Stores":
+			if strings.Split(fields[2], "=")[0] == "n" {
+				err := setFSCacheFields(fields[2:], &m.StoreWriteRequests, &m.StoreSuccessfulRequests,
+					&m.StoreRequestsOnPendingStorage, &m.StoreRequestsRejectedDueToEnobufs, &m.StoreRequestsFailedDueToEnomem)
+				if err != nil {
+					return &m, err
+				}
+			} else {
+				err := setFSCacheFields(fields[2:], &m.StoreRequestsSubmitted, &m.StoreRequestsRunning,
+					&m.StorePagesWithRequestsProcessing, &m.StoreRequestsDeleted, &m.StoreRequestsOverStoreLimit)
+				if err != nil {
+					return &m, err
+				}
+			}
+		case "VmScan":
+			err := setFSCacheFields(fields[2:], &m.ReleaseRequestsAgainstPagesWithNoPendingStorage,
+				&m.ReleaseRequestsAgainstPagesStoredByTimeLockGranted, &m.ReleaseRequestsIgnoredDueToInProgressStore,
+				&m.PageStoresCancelledByReleaseRequests, &m.VmscanWaiting)
+			if err != nil {
+				return &m, err
+			}
+		case "Ops":
+			if strings.Split(fields[2], "=")[0] == "pend" {
+				err := setFSCacheFields(fields[2:], &m.OpsPending, &m.OpsRunning, &m.OpsEnqueued, &m.OpsCancelled, &m.OpsRejected)
+				if err != nil {
+					return &m, err
+				}
+			} else {
+				err := setFSCacheFields(fields[2:], &m.OpsInitialised, &m.OpsDeferred, &m.OpsReleased, &m.OpsGarbageCollected)
+				if err != nil {
+					return &m, err
+				}
+			}
+		case "CacheOp:":
+			if strings.Split(fields[1], "=")[0] == "alo" {
+				err := setFSCacheFields(fields[1:], &m.CacheopAllocationsinProgress, &m.CacheopLookupObjectInProgress,
+					&m.CacheopLookupCompleteInPorgress, &m.CacheopGrabObjectInProgress)
+				if err != nil {
+					return &m, err
+				}
+			} else if strings.Split(fields[1], "=")[0] == "inv" {
+				err := setFSCacheFields(fields[1:], &m.CacheopInvalidations, &m.CacheopUpdateObjectInProgress,
+					&m.CacheopDropObjectInProgress, &m.CacheopPutObjectInProgress, &m.CacheopAttributeChangeInProgress,
+					&m.CacheopSyncCacheInProgress)
+				if err != nil {
+					return &m, err
+				}
+			} else {
+				err := setFSCacheFields(fields[1:], &m.CacheopReadOrAllocPageInProgress, &m.CacheopReadOrAllocPagesInProgress,
+					&m.CacheopAllocatePageInProgress, &m.CacheopAllocatePagesInProgress, &m.CacheopWritePagesInProgress,
+					&m.CacheopUncachePagesInProgress, &m.CacheopDissociatePagesInProgress)
+				if err != nil {
+					return &m, err
+				}
+			}
+		case "CacheEv:":
+			err := setFSCacheFields(fields[1:], &m.CacheevLookupsAndCreationsRejectedLackSpace, &m.CacheevStaleObjectsDeleted,
+				&m.CacheevRetiredWhenReliquished, &m.CacheevObjectsCulled)
+			if err != nil {
+				return &m, err
+			}
+		}
+	}
+
+	return &m, nil
+}
diff --git a/mountinfo.go b/mountinfo.go
index 9471136..c350cbe 100644
--- a/mountinfo.go
+++ b/mountinfo.go
@@ -144,7 +144,7 @@ func mountOptionsParseOptionalFields(o []string) (map[string]string, error) {
 	return optionalFields, nil
 }
 
-// Parses the mount options, superblock options.
+// mountOptionsParser parses the mount options, superblock options.
 func mountOptionsParser(mountOptions string) map[string]string {
 	opts := make(map[string]string)
 	options := strings.Split(mountOptions, ",")
@@ -161,7 +161,7 @@ func mountOptionsParser(mountOptions string) map[string]string {
 	return opts
 }
 
-// Retrieves mountinfo information from `/proc/self/mountinfo`.
+// GetMounts retrieves mountinfo information from `/proc/self/mountinfo`.
 func GetMounts() ([]*MountInfo, error) {
 	data, err := util.ReadFileNoStat("/proc/self/mountinfo")
 	if err != nil {
@@ -170,7 +170,7 @@ func GetMounts() ([]*MountInfo, error) {
 	return parseMountInfo(data)
 }
 
-// Retrieves mountinfo information from a processes' `/proc/<pid>/mountinfo`.
+// GetProcMounts retrieves mountinfo information from a processes' `/proc/<pid>/mountinfo`.
 func GetProcMounts(pid int) ([]*MountInfo, error) {
 	data, err := util.ReadFileNoStat(fmt.Sprintf("/proc/%d/mountinfo", pid))
 	if err != nil {
diff --git a/mountstats.go b/mountstats.go
index 35b2ef3..861ced9 100644
--- a/mountstats.go
+++ b/mountstats.go
@@ -186,6 +186,8 @@ type NFSOperationStats struct {
 	CumulativeTotalResponseMilliseconds uint64
 	// Duration from when a request was enqueued to when it was completely handled.
 	CumulativeTotalRequestMilliseconds uint64
+	// The count of operations that complete with tk_status < 0.  These statuses usually indicate error conditions.
+	Errors uint64
 }
 
 // A NFSTransportStats contains statistics for the NFS mount RPC requests and
@@ -494,8 +496,8 @@ func parseNFSEventsStats(ss []string) (*NFSEventsStats, error) {
 // line is reached.
 func parseNFSOperationStats(s *bufio.Scanner) ([]NFSOperationStats, error) {
 	const (
-		// Number of expected fields in each per-operation statistics set
-		numFields = 9
+		// Minimum number of expected fields in each per-operation statistics set
+		minFields = 9
 	)
 
 	var ops []NFSOperationStats
@@ -508,12 +510,12 @@ func parseNFSOperationStats(s *bufio.Scanner) ([]NFSOperationStats, error) {
 			break
 		}
 
-		if len(ss) != numFields {
+		if len(ss) < minFields {
 			return nil, fmt.Errorf("invalid NFS per-operations stats: %v", ss)
 		}
 
 		// Skip string operation name for integers
-		ns := make([]uint64, 0, numFields-1)
+		ns := make([]uint64, 0, minFields-1)
 		for _, st := range ss[1:] {
 			n, err := strconv.ParseUint(st, 10, 64)
 			if err != nil {
@@ -523,7 +525,7 @@ func parseNFSOperationStats(s *bufio.Scanner) ([]NFSOperationStats, error) {
 			ns = append(ns, n)
 		}
 
-		ops = append(ops, NFSOperationStats{
+		opStats := NFSOperationStats{
 			Operation:                           strings.TrimSuffix(ss[0], ":"),
 			Requests:                            ns[0],
 			Transmissions:                       ns[1],
@@ -533,7 +535,13 @@ func parseNFSOperationStats(s *bufio.Scanner) ([]NFSOperationStats, error) {
 			CumulativeQueueMilliseconds:         ns[5],
 			CumulativeTotalResponseMilliseconds: ns[6],
 			CumulativeTotalRequestMilliseconds:  ns[7],
-		})
+		}
+
+		if len(ns) > 8 {
+			opStats.Errors = ns[8]
+		}
+
+		ops = append(ops, opStats)
 	}
 
 	return ops, s.Err()
diff --git a/net_conntrackstat.go b/net_conntrackstat.go
index 1e27c83..b637be9 100644
--- a/net_conntrackstat.go
+++ b/net_conntrackstat.go
@@ -38,7 +38,7 @@ type ConntrackStatEntry struct {
 	SearchRestart uint64
 }
 
-// Retrieves netfilter's conntrack statistics, split by CPU cores
+// ConntrackStat retrieves netfilter's conntrack statistics, split by CPU cores
 func (fs FS) ConntrackStat() ([]ConntrackStatEntry, error) {
 	return readConntrackStat(fs.proc.Path("net", "stat", "nf_conntrack"))
 }
diff --git a/proc_maps.go b/proc_maps.go
index 28d5c6e..6e107f0 100644
--- a/proc_maps.go
+++ b/proc_maps.go
@@ -25,6 +25,7 @@ import (
 	"golang.org/x/sys/unix"
 )
 
+// ProcMapPermissions contains permission settings read from /proc/[pid]/maps
 type ProcMapPermissions struct {
 	// mapping has the [R]ead flag set
 	Read bool


Test patch:
diff --git a/fscache_test.go b/fscache_test.go
new file mode 100644
index 0000000..249dd43
--- /dev/null
+++ b/fscache_test.go
@@ -0,0 +1,136 @@
+// Copyright 2019 The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package procfs
+
+import (
+	"reflect"
+	"testing"
+)
+
+func TestFscacheinfo(t *testing.T) {
+	expected := Fscacheinfo{
+		IndexCookiesAllocated:                              3,
+		DataStorageCookiesAllocated:                        67877,
+		SpecialCookiesAllocated:                            0,
+		ObjectsAllocated:                                   67473,
+		ObjectAllocationsFailure:                           0,
+		ObjectsAvailable:                                   67473,
+		ObjectsDead:                                        388,
+		ObjectsWithoutCoherencyCheck:                       12,
+		ObjectsWithCoherencyCheck:                          33,
+		ObjectsNeedCoherencyCheckUpdate:                    44,
+		ObjectsDeclaredObsolete:                            55,
+		PagesMarkedAsBeingCached:                           547164,
+		UncachePagesRequestSeen:                            364577,
+		AcquireCookiesRequestSeen:                          67880,
+		AcquireRequestsWithNullParent:                      98,
+		AcquireRequestsRejectedNoCacheAvailable:            25,
+		AcquireRequestsSucceeded:                           67780,
+		AcquireRequestsRejectedDueToError:                  39,
+		AcquireRequestsFailedDueToEnomem:                   26,
+		LookupsNumber:                                      67473,
+		LookupsNegative:                                    67470,
+		LookupsPositive:                                    58,
+		ObjectsCreatedByLookup:                             67473,
+		LookupsTimedOutAndRequed:                           85,
+		InvalidationsNumber:                                14,
+		InvalidationsRunning:                               13,
+		UpdateCookieRequestSeen:                            7,
+		UpdateRequestsWithNullParent:                       3,
+		UpdateRequestsRunning:                              8,
+		RelinquishCookiesRequestSeen:                       394,
+		RelinquishCookiesWithNullParent:                    1,
+		RelinquishRequestsWaitingCompleteCreation:          2,
+		RelinquishRetries:                                  3,
+		AttributeChangedRequestsSeen:                       6,
+		AttributeChangedRequestsQueued:                     5,
+		AttributeChangedRejectDueToEnobufs:                 4,
+		AttributeChangedFailedDueToEnomem:                  3,
+		AttributeChangedOps:                                2,
+		AllocationRequestsSeen:                             20,
+		AllocationOkRequests:                               19,
+		AllocationWaitingOnLookup:                          18,
+		AllocationsRejectedDueToEnobufs:                    17,
+		AllocationsAbortedDueToErestartsys:                 16,
+		AllocationOperationsSubmitted:                      15,
+		AllocationsWaitedForCPU:                            14,
+		AllocationsAbortedDueToObjectDeath:                 13,
+		RetrievalsReadRequests:                             151959,
+		RetrievalsOk:                                       82823,
+		RetrievalsWaitingLookupCompletion:                  23467,
+		RetrievalsReturnedEnodata:                          69136,
+		RetrievalsRejectedDueToEnobufs:                     15,
+		RetrievalsAbortedDueToErestartsys:                  69,
+		RetrievalsFailedDueToEnomem:                        43,
+		RetrievalsRequests:                                 151959,
+		RetrievalsWaitingCPU:                               42747,
+		RetrievalsAbortedDueToObjectDeath:                  44,
+		StoreWriteRequests:                                 225565,
+		StoreSuccessfulRequests:                            225565,
+		StoreRequestsOnPendingStorage:                      12,
+		StoreRequestsRejectedDueToEnobufs:                  13,
+		StoreRequestsFailedDueToEnomem:                     14,
+		StoreRequestsSubmitted:                             69156,
+		StoreRequestsRunning:                               294721,
+		StorePagesWithRequestsProcessing:                   225565,
+		StoreRequestsDeleted:                               225565,
+		StoreRequestsOverStoreLimit:                        43,
+		ReleaseRequestsAgainstPagesWithNoPendingStorage:    364512,
+		ReleaseRequestsAgainstPagesStoredByTimeLockGranted: 2,
+		ReleaseRequestsIgnoredDueToInProgressStore:         43,
+		PageStoresCancelledByReleaseRequests:               12,
+		VmscanWaiting:                                      66,
+		OpsPending:                                         42753,
+		OpsRunning:                                         221129,
+		OpsEnqueued:                                        628798,
+		OpsCancelled:                                       11,
+		OpsRejected:                                        88,
+		OpsInitialised:                                     377538,
+		OpsDeferred:                                        27,
+		OpsReleased:                                        377538,
+		OpsGarbageCollected:                                37,
+		CacheopAllocationsinProgress:                       1,
+		CacheopLookupObjectInProgress:                      2,
+		CacheopLookupCompleteInPorgress:                    3,
+		CacheopGrabObjectInProgress:                        4,
+		CacheopInvalidations:                               5,
+		CacheopUpdateObjectInProgress:                      6,
+		CacheopDropObjectInProgress:                        7,
+		CacheopPutObjectInProgress:                         8,
+		CacheopAttributeChangeInProgress:                   9,
+		CacheopSyncCacheInProgress:                         10,
+		CacheopReadOrAllocPageInProgress:                   11,
+		CacheopReadOrAllocPagesInProgress:                  12,
+		CacheopAllocatePageInProgress:                      13,
+		CacheopAllocatePagesInProgress:                     14,
+		CacheopWritePagesInProgress:                        15,
+		CacheopUncachePagesInProgress:                      16,
+		CacheopDissociatePagesInProgress:                   17,
+		CacheevLookupsAndCreationsRejectedLackSpace:        18,
+		CacheevStaleObjectsDeleted:                         19,
+		CacheevRetiredWhenReliquished:                      20,
+		CacheevObjectsCulled:                               21,
+	}
+
+	have, err := getProcFixtures(t).Fscacheinfo()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if !reflect.DeepEqual(have, expected) {
+		t.Logf("have: %+v", have)
+		t.Logf("expected: %+v", expected)
+		t.Errorf("structs are not equal")
+	}
+}
diff --git a/mountstats_test.go b/mountstats_test.go
index c6e95b8..146df96 100644
--- a/mountstats_test.go
+++ b/mountstats_test.go
@@ -409,3 +409,91 @@ func mountsStr(mounts []*Mount) string {
 
 	return out
 }
+
+func TestMountStatsExtendedOperationStats(t *testing.T) {
+	r := strings.NewReader(extendedOpsExampleMountstats)
+	_, err := parseMountStats(r)
+	if err != nil {
+		t.Errorf("failed to parse mount stats with extended per-op statistics: %v", err)
+	}
+}
+
+const (
+	extendedOpsExampleMountstats = `
+device fs.example.com:/volume4/apps/home-automation/node-red-data mounted on /var/lib/kubelet/pods/1c2215a7-0d92-4df5-83ce-a807bcc2f8c8/volumes/kubernetes.io~nfs/home-automation--node-red-data--pv0001 with fstype nfs4 statvers=1.1
+	opts:   rw,vers=4.1,rsize=131072,wsize=131072,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.1.191,local_lock=none
+	age:    83520
+	impl_id:        name='',domain='',date='0,0'
+	caps:   caps=0x3fff7,wtmult=512,dtsize=32768,bsize=0,namlen=255
+	nfsv4:  bm0=0xfdffafff,bm1=0xf9be3e,bm2=0x800,acl=0x0,sessions,pnfs=not configured,lease_time=90,lease_expired=0
+	sec:    flavor=1,pseudoflavor=1
+	events: 52472 472680 16671 57552 2104 9565 749555 9568641 168 24103 1 267134 3350 20097 116581 18214 43757 111141 0 28 9563845 34 0 0 0 0 0 
+	bytes:  2021340783 39056395530 0 0 1788561151 39087991255 442605 9557343 
+	RPC iostats version: 1.1  p/v: 100003/4 (nfs)
+	xprt:   tcp 940 0 2 0 1 938505 938504 0 12756069 0 32 254729 10823602
+	per-op statistics
+			NULL: 1 1 0 44 24 0 0 0 0
+			READ: 34096 34096 0 7103096 1792122744 2272 464840 467945 0
+			WRITE: 322308 322308 0 39161277084 56725504 401718334 10139998 411864389 0
+			COMMIT: 12541 12541 0 2709896 1304264 342 7179 7819 0
+			OPEN: 12637 12637 0 3923256 4659940 871 57185 58251 394
+	OPEN_CONFIRM: 0 0 0 0 0 0 0 0 0
+		OPEN_NOATTR: 98741 98741 0 25656212 31630800 3366 77710 82693 0
+	OPEN_DOWNGRADE: 0 0 0 0 0 0 0 0 0
+			CLOSE: 87075 87075 0 18778608 15308496 2026 49131 52399 116
+			SETATTR: 24576 24576 0 5825876 6522260 643 34384 35650 0
+			FSINFO: 1 1 0 168 152 0 0 0 0
+			RENEW: 0 0 0 0 0 0 0 0 0
+		SETCLIENTID: 0 0 0 0 0 0 0 0 0
+	SETCLIENTID_CONFIRM: 0 0 0 0 0 0 0 0 0
+			LOCK: 22512 22512 0 5417628 2521312 1088 17407 18794 2
+			LOCKT: 0 0 0 0 0 0 0 0 0
+			LOCKU: 21247 21247 0 4589352 2379664 315 8409 9003 0
+			ACCESS: 1466 1466 0 298160 246288 22 1394 1492 0
+			GETATTR: 52480 52480 0 10015464 12694076 2930 30069 34502 0
+			LOOKUP: 11727 11727 0 2518200 2886376 272 16935 17662 3546
+		LOOKUP_ROOT: 0 0 0 0 0 0 0 0 0
+			REMOVE: 833 833 0 172236 95268 15 4566 4617 68
+			RENAME: 11431 11431 0 3150708 1737512 211 52649 53091 0
+			LINK: 1 1 0 288 292 0 0 0 0
+			SYMLINK: 0 0 0 0 0 0 0 0 0
+			CREATE: 77 77 0 18292 23496 0 363 371 11
+		PATHCONF: 1 1 0 164 116 0 0 0 0
+			STATFS: 7420 7420 0 1394960 1187200 144 4672 4975 0
+		READLINK: 4 4 0 704 488 0 1 1 0
+			READDIR: 1353 1353 0 304024 2902928 11 4326 4411 0
+		SERVER_CAPS: 9 9 0 1548 1476 0 3 3 0
+		DELEGRETURN: 232 232 0 48896 37120 811 300 1115 0
+			GETACL: 0 0 0 0 0 0 0 0 0
+			SETACL: 0 0 0 0 0 0 0 0 0
+	FS_LOCATIONS: 0 0 0 0 0 0 0 0 0
+	RELEASE_LOCKOWNER: 0 0 0 0 0 0 0 0 0
+			SECINFO: 0 0 0 0 0 0 0 0 0
+	FSID_PRESENT: 0 0 0 0 0 0 0 0 0
+		EXCHANGE_ID: 2 2 0 464 200 0 0 0 0
+	CREATE_SESSION: 1 1 0 192 124 0 0 0 0
+	DESTROY_SESSION: 0 0 0 0 0 0 0 0 0
+		SEQUENCE: 0 0 0 0 0 0 0 0 0
+	GET_LEASE_TIME: 0 0 0 0 0 0 0 0 0
+	RECLAIM_COMPLETE: 1 1 0 124 88 0 81 81 0
+		LAYOUTGET: 0 0 0 0 0 0 0 0 0
+	GETDEVICEINFO: 0 0 0 0 0 0 0 0 0
+	LAYOUTCOMMIT: 0 0 0 0 0 0 0 0 0
+	LAYOUTRETURN: 0 0 0 0 0 0 0 0 0
+	SECINFO_NO_NAME: 0 0 0 0 0 0 0 0 0
+	TEST_STATEID: 0 0 0 0 0 0 0 0 0
+	FREE_STATEID: 10413 10413 0 1416168 916344 147 3518 3871 10413
+	GETDEVICELIST: 0 0 0 0 0 0 0 0 0
+	BIND_CONN_TO_SESSION: 0 0 0 0 0 0 0 0 0
+	DESTROY_CLIENTID: 0 0 0 0 0 0 0 0 0
+			SEEK: 0 0 0 0 0 0 0 0 0
+		ALLOCATE: 0 0 0 0 0 0 0 0 0
+		DEALLOCATE: 0 0 0 0 0 0 0 0 0
+		LAYOUTSTATS: 0 0 0 0 0 0 0 0 0
+			CLONE: 0 0 0 0 0 0 0 0 0
+			COPY: 0 0 0 0 0 0 0 0 0
+	OFFLOAD_CANCEL: 0 0 0 0 0 0 0 0 0
+			LOOKUPP: 0 0 0 0 0 0 0 0 0
+		LAYOUTERROR: 0 0 0 0 0 0 0 0 0
+`
+)