I intended for the delimiter to be tab in this example.

Using sed or awk is an option, yes, but I am so used to the standard utilities that I would rather keep using them.

Also I'm not sure if your sed script does what I intended for it to do.

1. Take ever line that starts with 2017-05-29.

2. Out of the lines we have, remove any that contain INFO or WARN.

3. Take the five last of all of those lines.

4. Take the first and third fields of all of those lines.

Let's create a sample file

cat > /tmp/somefile <<EOF
2017-05-28T08:30+0200	nobody	ERR: Foobar failed to xyzzy.
2017-05-29T13:01+0200	nobody	INFO: Garply initiated by grault scheduler.
2017-05-29T13:37+0200	nobody	DEBUG: Garply exited with 0.
2017-05-29T14:12+0200	nobody	WARN: Plugh quux corge.
2017-05-29T14:55+0200	nobody	ERR: PLUGH QUUX CORGE!
2017-05-30T00:17+0200	nobody	ERR: Failed to retrieve baz needed for thud.
EOF

(Note that due to how HN formatting works you won't be able to copy-paste this because HN requires leading space to format as code but then it includes the spaces in the output also and we don't want leading space in the file.)

grep ^2017-05-29 /tmp/somefile | egrep -v 'INFO|WARN' | tail -n5 | cut -f1 -f3

Note: I had a typo in my original example where I said "grep -v" where I meant "egrep -v".

Result:

2017-05-29T13:37+0200	DEBUG: Garply exited with 0.
2017-05-29T14:55+0200	ERR: PLUGH QUUX CORGE!

Running it with your pipeline

sed '
/^2017-05-29/!d;
/INFO/d;
/WARN/d;
s/ /INFO/;
s/ /INFO/;
s/ .*//;
s/INFO.*INFO/ /;
' /tmp/somefile \
|tail -n5

results instead in:

2017-05-29T13:37+0200	nobody	DEBUG: exited
2017-05-29T14:55+0200	nobody	ERR: QUUX

But that's just because you assumed that space was the delimiter when it was not.

Your use of the word "INFO" as a placeholder for the first two occurrences of the space character threw me off quite a bit when reading your script, since the word "INFO" occurs in the file we are working with itself, but it makes sense to use a word that we know is no longer possible to be present since we've already removed all lines that contain it. However, while a neat trick, those kinds of strange hacks are the kinds of things that has brought me to believe that having objects (like they have in Microsoft PowerShell) instead of pure text would be beneficial in Unix also.

---

As for your comments on compiling into a single program and run that, I don't think you understood what I meant, or I don't think that busybox and those chrunched binaries you mentioned perform the optimization I am talking about, do they.

---

Using the k language you mention in that fashion seems more like a hack and will require a lot of work each time. I would rather rewrite all core commands of my system so that they produced true objects, or actually, rather than objects just structured binary data. I don't need the output to have methods you can call.

One of the main things I want from structured binary data is to be able to select the columns of data by name instead of by index and without having a mess of some commands using tab for delimiter and others space and so on.

So instead of

zfs list -H | cut -f1,3,4

I would like to

zfs list | cut name used avail

for example, or something like that.

Also all commands that output tabular data must have a "header" command that will show the column headers. So to see the headers that the 'list' subcommand of the zfs command will output, I would say

zfs header list

And it would tell me

NAME	USED	AVAIL	REFER	MOUNTPOINT

and furthermore since I am authoring the shell, when you tab to complete a command it will call the binary with the header subcommand and output the headers so that you can have them easily accessible while you're working on a pipeline. And furthermore this shall work with pipes already present.

So if I type

zfs list | cut

and then tab to complete, it will call

zfs header list

and pipe that to

cut header

which will return the input it saw

Naturally, "header" will be a reserved word.

All commands will understand how to work with tabular data.

When you use grep you will either specify which column to use, or you can tell it to look across all columns with *

The shell will only expand * to file names when the * is positionally last in the argument list of a command since all commands will take the list of files as the last ones in their list of arguments.

---

All of this being said, I appreciate all replies, including yours.