Repository: google/mtail

Issue #86: Programs are discarded silently on parse errors
Created: 2017-07-31T09:23:44Z
While using the current HEAD, I noticed that the slightest mistake will make mtail ignore the program altogether without any error message.
For example, this works:
$ cat tmp/test.mtail 
counter bar
/foo/ {
 bar++
}

$ mtail --one_shot --progs ./tmp/ --logs ./mail.info -one_shot_metrics  -logtostderr 
I0731 09:19:20.865390   15354 main.go:97] mtail version 3.0.0~rc1+git20170727.6db685b git revision 3.0.0~rc1+git20170727.6db685b-2 go version go1.8.3
I0731 09:19:20.866008   15354 loader.go:192] Loaded program test.mtail
I0731 09:19:20.866056   15354 loader.go:211] Program test.mtail has goroutine marker 0x74657374
I0731 09:19:20.866096   15354 loader.go:214] Started test.mtail
Oneshot results:
I0731 09:19:20.866210   15354 mtail.go:48] Oneshot "./mail.info"
./mail.info: 4 MAXPROCS, 4 CPUs, I0731 09:19:20.866325   15354 vm.go:697] Starting program test.mtail
19726 lines, 98.074324ms total time,  4.972 µs/line
Metrics store:{
  "bar": [
    {
      "Name": "bar",
      "Program": "test.mtail",
      "Kind": 1,
      "Type": 0,
      "LabelValues": [
        {
          "Value": {
            "Value": 0,
            "Time": 0
          }
        }
      ]
    }
  ]
}I0731 09:19:20.964661   15354 mtail.go:322] Shutdown requested.
I0731 09:19:20.964679   15354 mtail.go:329] No tailer, closing lines channel.
I0731 09:19:20.964717   15354 loader.go:343] Shutting down loader.
I0731 09:19:20.964799   15354 log_watcher.go:57] Shutting down log watcher.
I0731 09:19:20.964833   15354 vm.go:702] Stopping program test.mtail
I0731 09:19:20.964850   15354 mtail.go:335] All done.

But if I introduce an error, nothing happens:
$ cat tmp/test.mtail 
counter bar
/foo/ {
 bar++
}
uux

$ mtail --one_shot --progs ./tmp/ --logs ./mail.info -one_shot_metrics  -logtostderr 
I0731 09:22:48.626251   15553 main.go:97] mtail version 3.0.0~rc1+git20170727.6db685b git revision 3.0.0~rc1+git20170727.6db685b-2 go version go1.8.3
Oneshot results:
I0731 09:22:48.626810   15553 mtail.go:48] Oneshot "./mail.info"
./mail.info: 4 MAXPROCS, 4 CPUs, 19726 lines, 37.785204ms total time,  1.916 µs/line
Metrics store:{}I0731 09:22:48.664847   15553 mtail.go:322] Shutdown requested.
I0731 09:22:48.664866   15553 mtail.go:329] No tailer, closing lines channel.
I0731 09:22:48.664905   15553 loader.go:343] Shutting down loader.
I0731 09:22:48.664991   15553 log_watcher.go:57] Shutting down log watcher.
I0731 09:22:48.665018   15553 mtail.go:335] All done.

Duplicate issue #89: Capture groups can't be used in decorated blocks
Created: 2017-08-01T12:44:34Z
Related to #88, I noticed that if I use the decorator syntax, I can't reuse the captured groups created inside the decorator. This would be specially useful for things like the syslog decorator.
For example, this works:
/(...).*/ {
    counter test
    $1 == "Dec" {
        test++
    }
}

But this does not (and also it is silently discarded, as I reported in #86):
def decorator {
    /(...).*/ {
        next
    }
}

@decorator {
    counter test
    $1 == "Dec" {
        test++
    }
}

Closing comment (2017-08-03T11:26:39Z): Yeah, the caprefs in the decorator are only in the scope of the decorator.
Leaking them out into the decorated block was never intended -- decorators
were designed only for the purpose of sharing timestamp parsing code!

As the caprefs are not visible at the point of use of a decorator, I am
hesitant to let them show up in the decorated block in case strange
variable name conflicts occur.  Especially with the numbered references,
which are easy to overload with nested matches.

But that makes me think that explicitly declaring which names are allowed
to appear in scope as part of the decorator syntax would mitigate that, for
example:

@syslog using $description {
   foo[$description]++
}

(not sure about the word 'using' yet though.)

Thanks for the report, good idea, I'll think about this some more!
…
On 1 August 2017 at 22:44, Martín Ferrari ***@***.***> wrote:
 Related to #88 <#88>, I noticed
 that if I use the decorator syntax, I can't reuse the captured groups
 created inside the decorator. This would be specially useful for things
 like the syslog decorator.

 For example, this works:

 /(...).*/ {
     counter test
     $1 == "Dec" {
         test++
     }
 }

 But this does not (and also it is silently discarded, as I reported in #86
 <#86>):

 def decorator {
     /(...).*/ {
         next
     }
 }

 @decorator {
     counter test
     $1 == "Dec" {
         test++
     }
 }

 —
 You are receiving this because you are subscribed to this thread.
 Reply to this email directly, view it on GitHub
 <#89>, or mute the thread
 <https://github.com/notifications/unsubscribe-auth/AC5b-xlYvGokMuEEpUyTlreBFwucO-hKks5sTx2ygaJpZM4OprEx>
 .