opened
ivan-mogilko
username_0
This is done as a preliminary step for #640.

**As a preface**, engine logging system in a nutshell works like this:
* each message has a "verbosity level" ranging from "debug" to "fatal error".
* there may be a number of message groups, which allow to separate messages related to a component or engine's function (main, script, spritecache, managed pool);
* there may be a number of outputs (stdout, log file, in-game console); each output has its own filter settings, that define which groups & level of messages to let through.

When a message is sent to logging system it tries each registered output implementation, and if its filter allows, message is being printed.

---


**What this PR does**: it actually allows to configure these filters by user (player).

A new section called "[log]" is introduced to the user config. It may have a number of entries - one for each log output - which define a list of filters for that output. These entries have a common syntax:

`output_id = group1:level, group2:level, etc`

where `output_id` is output's name, and entry value is a list of "group name" : "message level" pairs.

Possible outputs at this moment are:
* `stdout` - standard output (system console, etc)
* `file` - regular log file
* `console` - in-game console
* `warnfile` - this is a special warning.log for games built in debug mode (more like a backward compatible relic, maybe we can replace this with normal log written to game dir in the future).

Possible log groups:
* `main` - default group for most of messages
* `script` - script API messages
* `sprcache` - verbose sprite cache messages (currently not available unless engine is built with DEBUG_SPRITECACHE)
* `manobj` - managed objects / pool log, very verbose! (currently not available unless engine is built with DEBUG_MANAGED_OBJECTS)

Possible message levels:
* `fatal` - only fatal errors
* `error` - any errors
* `warn` - warnings
* `info` - general notifications
* `debug` - verbose debugging messages
* `all` - all messages (which equals to `debug` level actually)

Finally, there's a special form of requesting default settings:

`output_id = default`

---

**Examples of config:**

<pre>
[log]
stdout = main:info, script:error
file = main:all, script:all
console = default
</pre>
Print only important notifications and script API errors to stdout (system console), print everything from main and script groups into file, and use default settings for in-game console.
3.5.1: Configurable log options
issue
created
ericoporto
username_1
This will be really useful to help debugging AGS games! I have a minor nitpick for future proof, in case there's ever an idea to pass a parameter like `--log-stdout=script:error`,  the colon `:` character has special meaning in both `bash` and `cmd.exe`, so I would use something like `_` .
comment
created
ivan-mogilko
username_0
Underscore may be a part of name. I think it's possible to replace with another '=', because only first one separates key and value in ini format.
comment
created
ericoporto
username_1
Sorry, I was wrong. I just wrote a [mini program](https://github.com/username_1/quicktest/blob/master/main.cpp) and the `:` is fine. Let's keep it, it looks nicer too!
comment
created
ericoporto
username_1
Hey, in the context of #640 , user generated messages would be a new group, a new level (between info and warn) or neither?
comment
created
ivan-mogilko
username_0
I was thinking about renaming current "script" group to "game" (because it's related to game logic), and giving "script" group to messages from user script.
comment
created
rofl0r
username_2
is there something like an ultra-verbose debug level, i.e. "give me all log messages" ?
comment
created
ivan-mogilko
username_0
Do you mean all message levels from all groups? I forgot to add an quick option for that, but I will do.

@username_1 actually, since you mentioned that, I realized that command-line arguments would also be needed.
comment
created
ericoporto
username_1
I think (not sure) there's already a command line argument there, so you do add one with these capabilities, then that one may be removed
comment
created
rofl0r
username_2
sounds like a good idea


btw i'd prefer if this was a commandline switch rather than a config file option
comment
created
morganwillcock
username_3
I had the same thought.
comment
created
ivan-mogilko
username_0
Do you mean, having a command line option to pair the config, or have it instead of config (and not have a config setting at all)?
comment
created
ericoporto
username_1
I imagine the log can config is more comfortable on Windows when you are running your game by double clicking the executable and the command line is more comfortable when invoking the game in Linux system where you run the engine from the command line (double click execution of binaries from file navigator is not a thing in Linux since this belongs to the launcher).
comment
created
rofl0r
username_2
yeah, imo command-line only is preferable. the user interested in log output can run `./game.exe --help` and read the usage output, if it is a config file option he's got to study (and find) the online manual for config files.
comment
created
ericoporto
username_1
@username_2 do you have suggestions for how to write the log help concisely? I was thinking on it and could only figure out text that was more appropriate to a man entry.
comment
created
rofl0r
username_2
@username_1 something like:
```
--log-level=value
  1: fatal errors only
  2: medium
  3: debug
--log-subsys=[mscp]
  specify one or more log subsystems to be logged (default: all)
  m: main
  s: script
  ...
--log-output=value
  specify where log output is displayed (default: stdout)
  stdout: log to stdout
  stderr: log to stderr
  console: log to game console
  file:name.log: log to name.log
```
comment
created
ericoporto
username_1
Engine currently log to `~/.local/share/ags.log` , I would not change this on this PR, if the log goes to stdout the person can redirect it to a file. Also current default I think is info on main and error on script.
comment
created
ivan-mogilko
username_0
IMO there is at least one reason to keep config options as alternative: there are ports that are not normally run from command line (mobile).
comment
created
ericoporto
username_1
Yes that too. On Android the stdout log should show up in logcat.
comment
created
ivan-mogilko
username_0
Looking at username_2's cmdline proposal above, it defines same subsystem/level setup for any output, as opposed to each output having its own message filter, which exists now. Initially I was basing this on some of the existing logging systems that allow to pass different message groups into different targets.

Idk if it's generally necessary for AGS, but there are special cases, like in-game console that normally was used to print all kinds of verbose stuff, like room object state changes, and a "warnings.log" file that is used to output any kind of script warnings when game is run from the editor (bringing user attention to potential mistakes in game).
comment
created
ivan-mogilko
username_0
Okay, I will try to make something up soon (there has to be bit more changes to the code), but one thing that concerns me about above suggestion is that "log-level" default is "nothing", which means that if a user simply runs a game from terminal without parameters, they won't get any messages whatsoever.
comment
created
rofl0r
username_2
well, that's probably not a wise default, at least for non-console-like platforms like android, i suppose it should be 1 for platforms that have a terminal.
comment
created
ericoporto
username_1
`--log-stdout=main:info script:error` or `--log-stdout=m:i s:e` should work as alternative command line parameters to specify a level per loggroup per log sink (stdout, file, console).
comment
created
ivan-mogilko
username_0
In the above, how do you know where the command arguments end?
comment
created
ericoporto
username_1
Yes, you can enclose in double quotes. You can also accept multiple ([item 9](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html)) `--log-stdout=main:info` `--log-stdout=script:error` passed at same time. This [so details both](https://stackoverflow.com/questions/43195553/how-to-pass-multiple-parameters-for-the-same-commandline-argument).  Maybe is best to not accept multiple arguments in the parameter unless it's enclosed by double quotes. If it's desired for some reason, than an option is to require comma as separator.
comment
created
ivan-mogilko
username_0
The problem is that engine still does not have a proper generic parsing of command line; for that reason I'd rather stick to one simple implementation for this moment.
comment
created
ericoporto
username_1
I had a pleasant experience with `cxxopts` here https://github.com/username_1/ags/blob/ags4-fernewelten-agscc/Compiler/agscc.cpp
comment
created
rofl0r
username_2
the simple and generic implementation is getopt(3p) or [getopt_long](https://linux.die.net/man/3/getopt_long) . the latter is a bit less portable but used by almost all command line tools in the unix world.

as far as parsing is concerned, in my proposal you'd parse one or more message groups using a single character representing a group, no need for separator characters, in which case a simple `switch(optarg[x]) { case 'm': message_group_m = true; break; ...` would be sufficient (iterating over each char in the optarg) . the only option that requires a separate parsing step is the suggested --log-output option if used with "file", in which case one would check whether the option argument starts with "file:" and then start parsing the actual filename from position 5.
comment
created
ivan-mogilko
username_0
AFAIK getopt is not supported on Windows.
comment
created
ivan-mogilko
username_0
So, here's an experimental variant done in a kind of smart-arse way. It has *only one* command line option which is parsed similarily to the config, except the option's name is formed slightly differently:

`--log-OUTPUT=group:verbosity,group,verbosity,etc`

Groups may be defined by both full names or letters or groups of letters. Nuance is that to make parsing easier letter groups must be preceded by special letter '+' (I can change it to something else).
Verbosity may now also be defined by either name or number.

Examples of command line arguments:
`--log-file=main:error` - prints into file
`--log-stdout=all:3` - prints error level to stdout
`--log-stdout=all` or `--log-stdout=all:all` - prints everything possible to stdout.

Additionally, log file path may be now specified both in config and command line:
`--log-file-path=./1.txt`
or
<pre>
[log]
file-path = ./1.txt
</pre>
comment
created
ivan-mogilko
username_0
Added help, if it's fine I can merge this.
comment
created
ericoporto
username_1
I verified on Linux and for me it appears fine.
comment
created
ivan-mogilko
username_0
Ammended with examples.
comment