Repository: google/mtail

Issue #106: regression handling float type between rc2..master
Created: 2017-10-12T19:21:21Z
Hi,
following mtail program fail.
gauge metric

/(\d+\.\d+)/ {
  metric += $1
}

by parsing simple log:
1.1

Traceback (on the master branch): ...ufff, it was raising an error a minute ago.
But now I simple do not count the metric without the traceback...
I will try to generate the traceback tomorrow.

Duplicate issue #108: wrong type handling
Created: 2017-10-16T14:14:57Z
Following program raise a traceback:
counter debug

/ time=(\d+\.\d+)/ {
  debug += $1/1000
}

with following runtime error:
lukas@lukas-szn:~/.go/src/github.com/google/mtail$ ~/.go/bin/mtail -logs ~/x.log -progs ~/projects/adminserver/mtail/progs -v=3 -logtostderr -one_shot
I1016 16:14:03.434618   31437 main.go:102] mtail version v3.0.0-rc4 git revision d9f2afc03ae86e203892f25d66cce8c76df8c649 go version go1.8
I1016 16:14:03.434831   31437 main.go:103] Commandline: ["/home/lukas/.go/bin/mtail" "-logs" "/home/lukas/x.log" "-progs" "/home/lukas/projects/adminserver/mtail/progs" "-v=3" "-logtostderr" "-one_shot"]
I1016 16:14:03.436014   31437 checker.go:79] found sym &{debug 0 typeVar4 errors.mtail:1:9-13 <nil> 0}
I1016 16:14:03.436176   31437 types.go:284] Unifying typeVar5→typeVar5→typeVar5 and Float→Int→Float
I1016 16:14:03.436225   31437 types.go:284] Unifying typeVar5 and Float
I1016 16:14:03.436262   31437 types.go:299] Making "typeVar5" type "Float"
I1016 16:14:03.436288   31437 types.go:284] Unifying Float and Int
I1016 16:14:03.436311   31437 types.go:284] Unifying Float and Float
I1016 16:14:03.436334   31437 types.go:284] Unifying typeVar4 and Float
I1016 16:14:03.436366   31437 types.go:299] Making "typeVar4" type "Float"
I1016 16:14:03.436508   31437 loader.go:206] Loaded program errors.mtail
I1016 16:14:03.436538   31437 loader.go:225] Program errors.mtail has goroutine marker 0x6572726f
I1016 16:14:03.436631   31437 vm.go:755] Starting program errors.mtail
I1016 16:14:03.436699   31437 loader.go:229] Started errors.mtail
I1016 16:14:03.436743   31437 loader.go:102] Skipping /home/lukas/projects/adminserver/mtail/progs/errors.mtail.bak_all due to file extension.
I1016 16:14:03.436937   31437 mtail.go:55] Tail pattern "/home/lukas/x.log"
I1016 16:14:03.437030   31437 tail.go:130] glob matches: [/home/lukas/x.log]
I1016 16:14:03.437256   31437 tail.go:347] EOF on first read
I1016 16:14:03.437296   31437 mtail.go:251] Shutdown requested.
I1016 16:14:03.437740   31437 vm.go:197] errors.mtail: Runtime error: unexpected int type float64 %!q(float64=0.0053433000000000005)
I1016 16:14:03.437944   31437 vm.go:198] VM stack:
goroutine 26 [running]:
runtime/debug.Stack(0xc420051800, 0x20, 0xc420051e68)
	/usr/local/go/src/runtime/debug/stack.go:24 +0x79
github.com/google/mtail/vm.(*VM).errorf(0xc4200c3d60, 0x85608b, 0x2, 0xc420051e68, 0x1, 0x1)
	/home/lukas/.go/src/github.com/google/mtail/vm/vm.go:198 +0xe1
github.com/google/mtail/vm.(*VM).execute(0xc4200c3d60, 0xc42001a280, 0x5, 0x7c5220, 0xc4201622d8)
	/home/lukas/.go/src/github.com/google/mtail/vm/vm.go:456 +0x3ff2
github.com/google/mtail/vm.(*VM).processLine(0xc4200c3d60, 0xc4201036e0)
	/home/lukas/.go/src/github.com/google/mtail/vm/vm.go:742 +0x16e
github.com/google/mtail/vm.(*VM).Run(0xc4200c3d60, 0x6572726f, 0xc420071140, 0xc4200711a0, 0xc420071200)
	/home/lukas/.go/src/github.com/google/mtail/vm/vm.go:760 +0x12a
created by github.com/google/mtail/vm.(*Loader).CompileAndRun
	/home/lukas/.go/src/github.com/google/mtail/vm/loader.go:227 +0x668
I1016 16:14:03.438094   31437 vm.go:199] Dumping vm state
I1016 16:14:03.438126   31437 vm.go:200] Name: errors.mtail
I1016 16:14:03.438144   31437 vm.go:201] Input: &tailer.LogLine{Filename:"/home/lukas/x.log", Line:"time=5.3433ms"}
I1016 16:14:03.438196   31437 vm.go:202] Thread:
I1016 16:14:03.438242   31437 vm.go:203]  PC 9
I1016 16:14:03.438261   31437 vm.go:204]  Match true
I1016 16:14:03.438281   31437 vm.go:205]  Matched false
I1016 16:14:03.438294   31437 vm.go:206]  Matches map[0:[time=5.3433 5.3433]]
I1016 16:14:03.438335   31437 vm.go:207]  Timestamp 0001-01-01 00:00:00 +0000 UTC
I1016 16:14:03.438405   31437 vm.go:208]  Stack [0@0]
I1016 16:14:03.438899   31437 vm.go:209] Prog: errors.mtail
Metrics
        0 Metric: name=debug program=errors.mtail kind=Counter type=Float hidden=false keys=[] labelvalues=[LabelValue: [] 0@0] source=errors.mtail:1:9-13
Regexps
        0 /time=(\d+\.\d+)/
Strings
 disasm  l         op  opnd
         0      match     0
         1        jnm    11
         2 setmatched false
         3      mload     0
         4      dload     0
         5       push     0
         6     capref     1
         7       push  1000
         8       fdiv <nil>
         9        inc     0
        10 setmatched  true
I1016 16:14:03.439016   31437 vm.go:197] errors.mtail: Runtime error: panic in thread &vm.thread{pc:10, match:true, matched:false, matches:map[int][]string{0:[]string{"time=5.3433", "5.3433"}}, time:time.Time{sec:0, nsec:0, loc:(*time.Location)(nil)}, stack:[]interface {}{}} at instr "{inc 0}": datum 0@0 is not an Int
I1016 16:14:03.439385   31437 vm.go:198] VM stack:
goroutine 26 [running]:
runtime/debug.Stack(0xc42001a3c0, 0x41, 0xc4201a1748)
	/usr/local/go/src/runtime/debug/stack.go:24 +0x79
github.com/google/mtail/vm.(*VM).errorf(0xc4200c3d60, 0x8687f4, 0x23, 0xc4201a1748, 0x3, 0x3)
	/home/lukas/.go/src/github.com/google/mtail/vm/vm.go:198 +0xe1
github.com/google/mtail/vm.(*VM).execute.func1(0xc4200c3d60, 0xc42001a280, 0x5, 0x7c5220, 0xc4201622d8)
	/home/lukas/.go/src/github.com/google/mtail/vm/vm.go:407 +0x14f
panic(0x7c5c60, 0xc420014420)
	/usr/local/go/src/runtime/panic.go:489 +0x2cf
github.com/google/mtail/metrics/datum.IncIntBy(0xa21880, 0xc420162250, 0x0, 0x0, 0xc400000000, 0x0)
	/home/lukas/.go/src/github.com/google/mtail/metrics/datum/datum.go:118 +0x145
github.com/google/mtail/vm.(*VM).execute(0xc4200c3d60, 0xc42001a280, 0x5, 0x7c5220, 0xc4201622d8)
	/home/lukas/.go/src/github.com/google/mtail/vm/vm.go:462 +0x3f3f
github.com/google/mtail/vm.(*VM).processLine(0xc4200c3d60, 0xc4201036e0)
	/home/lukas/.go/src/github.com/google/mtail/vm/vm.go:742 +0x16e
github.com/google/mtail/vm.(*VM).Run(0xc4200c3d60, 0x6572726f, 0xc420071140, 0xc4200711a0, 0xc420071200)
	/home/lukas/.go/src/github.com/google/mtail/vm/vm.go:760 +0x12a
created by github.com/google/mtail/vm.(*Loader).CompileAndRun
	/home/lukas/.go/src/github.com/google/mtail/vm/loader.go:227 +0x668
I1016 16:14:03.439510   31437 vm.go:199] Dumping vm state
I1016 16:14:03.439527   31437 vm.go:200] Name: errors.mtail
I1016 16:14:03.439545   31437 vm.go:201] Input: &tailer.LogLine{Filename:"/home/lukas/x.log", Line:"time=5.3433ms"}
I1016 16:14:03.439575   31437 vm.go:202] Thread:
I1016 16:14:03.439591   31437 vm.go:203]  PC 9
I1016 16:14:03.439608   31437 vm.go:204]  Match true
I1016 16:14:03.439625   31437 vm.go:205]  Matched false
I1016 16:14:03.439642   31437 vm.go:206]  Matches map[0:[time=5.3433 5.3433]]
I1016 16:14:03.439669   31437 vm.go:207]  Timestamp 0001-01-01 00:00:00 +0000 UTC
I1016 16:14:03.439699   31437 vm.go:208]  Stack []
I1016 16:14:03.439854   31437 vm.go:209] Prog: errors.mtail
Metrics
        0 Metric: name=debug program=errors.mtail kind=Counter type=Float hidden=false keys=[] labelvalues=[LabelValue: [] 0@0] source=errors.mtail:1:9-13
Regexps
        0 /time=(\d+\.\d+)/
Strings
 disasm  l         op  opnd
         0      match     0
         1        jnm    11
         2 setmatched false
         3      mload     0
         4      dload     0
         5       push     0
         6     capref     1
         7       push  1000
         8       fdiv <nil>
         9        inc     0
        10 setmatched  true
I1016 16:14:03.440981   31437 log_watcher.go:74] Shutting down log watcher.
I1016 16:14:03.441108   31437 tail.go:404] Shutting down tailer.
I1016 16:14:03.441181   31437 loader.go:364] Shutting down loader.
I1016 16:14:03.441399   31437 log_watcher.go:74] Shutting down log watcher.
I1016 16:14:03.441464   31437 vm.go:762] Stopping program errors.mtail
I1016 16:14:03.441513   31437 mtail.go:264] All done.
Metrics store:{
  "debug": [
    {
      "Name": "debug",
      "Program": "errors.mtail",
      "Kind": 1,
      "Type": 1,
      "LabelValues": [
        {
          "Value": {
            "Value": 0,
            "Time": 0
          }
        }
      ]
    }
  ]
}

Closing comment (2017-10-30T10:07:40Z): Ah this is the same as #106 which just got fixed.