Package: linesieve
This is my text munging tool. There are many like it, but this one is mine.linesieveis an unholy blend of grep, sed, awk, and Python,
born out of spite.Featureslinesieveallows you to:split text input into sectionsapply filters to specific sectionssearch and highlight success/failure markersmatch/sub/split with the full power of Python’sreshorten paths, links and module nameschain filters into pipelinescolor output!InstallingInstall and update usingpip:$pipinstall--upgradelinesieveA simple example$ls-1/*|linesieve-s'.*:'showbinmatch^dhead-n2.....
/bin:
dash
date
......
/sbin:
disklabel
dmesg
...This prints the first two files starting withdfrom each directory whose name containsbin(skipped directories are marked with a dot on stderr).LinksPyPI Releases:https://pypi.org/project/linesieve/Documentation:https://linesieve.readthedocs.io/Issue Tracker:https://github.com/lemon24/linesieve/issuesSource Code:https://github.com/lemon24/linesieveExamplesJava tracebacksAssume you’re writing some Java tests with JUnit,
on a project that looks like this:.
├── src
│   └── com
│       └── example
│           └── someproject
│               └── somepackage
│                   └── ThingDoer.java
└── tst
    └── com
        └── example
            └── someproject
                └── somepackage
                    └── ThingDoerTest.javaThis command:linesieve\span-v-X\--start'^ (\s+) at \s ( org\.junit\. | \S+ \. reflect\.\S+\.invoke )'\--end'^ (?! \s+ at \s )'\--repl'\1...'\match-v'^\s+at \S+\.(rethrowAs|translateTo)IOException'\sub-paths--include'{src,tst}/**/*.java'--modules-skip1\sub-X'^( \s+ at \s+ (?! .+ \.\. | com\.example\. ) .*? ) \( .*''\1'\sub-X'^( \s+ at \s+ com\.example\. .*? ) \ ~\[ .*''\1'\sub-X'
    (?P<pre> \s+ at \s .*)
    (?P<cls> \w+ )
    (?P<mid> .* \( )
    (?P=cls) \.java
    (?P<suf> : .* )
    '\'\g<pre>\g<cls>\g<mid>\g<suf>'… shortens this 76 line traceback:12:34:56.789 [main] ERROR com.example.someproject.somepackage.ThingDoer - exception while notifying done listener
java.lang.RuntimeException: listener failed
    at com.example.someproject.somepackage.ThingDoerTest$DummyListener.onThingDone(ThingDoerTest.java:420) ~[tests/:?]
    at com.example.someproject.somepackage.ThingDoer.doThing(ThingDoer.java:69) ~[library/:?]
    at com.example.otherproject.Framework.doAllTheThings(Framework.java:1066) ~[example-otherproject-2.0.jar:2.0]
    at com.example.someproject.somepackage.ThingDoerTest.listenerException(ThingDoerTest.java:666) ~[tests/:?]
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
    ...
    ... 60+ more lines of JUnit stuff we don't really care about ...
    ...
12:34:56.999 [main] INFO done… to just:12:34:56.789 [main] ERROR ..ThingDoer - exception while notifying done listener
java.lang.RuntimeException: listener failed
    at ..ThingDoerTest$DummyListener.onThingDone(:420) ~[tests/:?]
    at ..ThingDoer.doThing(:69) ~[library/:?]
    at com.example.otherproject.Framework.doAllTheThings(:1066)
    at ..ThingDoerTest.listenerException(:666) ~[tests/:?]
    ...
12:34:56.999 [main] INFO doneLet’s break thatlinesievecommand down a bit:Thespangets rid of all the traceback lines coming from JUnit.Thematch-vskips some usually useless lines from stack traces.Thesub-pathsshortens and highlights the names of classes in the current project;com.example.someproject.somepackage.ThingDoerbecomes..ThingDoer(presumably that’s enough info to open the file).The firstsubgets rid of line numbers and JAR names for everything
that is not either in the current project or in anothercom.example.package.The secondsubgets rid of JAR names for things in othercom.example.packages.The thirdsubgets rid of the source file name;..ThingDoer.doThing(ThingDoer.java:69)becomes..ThingDoer.doThing(:69)(the file name matches the class name).Apache Ant outputLet’s look at whylinesievewas born in the first place
– cleaning up Apache Ant output.We’ll use Ant’s own test output as an example,
since itbuilds itself.Running a single test with:antjunit-single-test-Djunit.testcase=org.apache.tools.ant.ProjectTest… produces 77 lines of output:Buildfile: /Users/lemon/code/ant/build.xml

check-optional-packages:

prepare:

compile:

compile-jdk9+:

build:
[delete] Deleting directory /Users/lemon/code/ant/build/classes/org/apache/tools/ant/taskdefs/optional/junitlauncher/confined
        ... more lines

... more targets, until we get to the one that we care about

junit-single-test-only:
    [junit] WARNING: multiple versions of ant detected in path for junit
    [junit]          file:/Users/lemon/code/ant/build/classes/org/apache/tools/ant/Project.class
    [junit]      and jar:file:/usr/local/Cellar/ant/1.10.12/libexec/lib/ant.jar!/org/apache/tools/ant/Project.class
    [junit] Testsuite: org.apache.tools.ant.ProjectTest
    [junit] Tests run: 12, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 5.635 sec
        ... more lines

junit-single-test:

BUILD SUCCESSFUL
Total time: 12 secondsIf this doesn’t look all that bad,
try imagining what it looks like
for a Serious Enterprise Project™.Lots of output is indeed very helpful
– if you’re waiting minutes for the entire test suite to run,
you want all the details in there,
so you can debug failures without having to run it again.However, it’s not very helpful during development,
when you only care about the thing you’re working onright now.
And it’s doubly not helpful if you want to re-run the tests
on each file update with something likeentr.This is where alinesievewrapper script can help:#!/bin/shlinesieve\--section'^(\S+):$'\--success'BUILD SUCCESSFUL'\--failure'BUILD FAILED'\showjunit-batch\showjunit-single-test-only\sub-cwd\sub-paths--include'src/main/**/*.java'--modules-skip2\sub-paths--include'src/tests/junit/**/*.java'--modules-skip3\sub-scompile'^\s+\[javac?] '''\pushcompile\match-v'^Compiling \d source file'\match-v'^Ignoring source, target'\pop\pushjunit\sub'^\s+\[junit] ?'''\span-v\--start'^WARNING: multiple versions of ant'\--end'^Testsuite:'\match-v'^\s+at java\.\S+\.reflect\.'\match-v'^\s+at org.junit.Assert'\span-v\--start'^\s+at org.junit.(runners|rules|internal)'\--end'^(?!\s+at )'\pop\sub-X'^( \s+ at \s+ (?! .+ \.\. ) .*? ) \( .*''\1'\sub-X'
    (?P<pre> \s+ at \s .*)
    (?P<cls> \w+ )
    (?P<mid> .* \( )
    (?P=cls) \.java
    (?P<suf> : .* )
    '\'\g<pre>\g<cls>\g<mid>\g<suf>'\sub--color-X'^( \w+ (\.\w+)+ (?= :\s ))''\1'\sub--color-X'(FAILED)''\1'\read-cmdant"$@"You can then call this instead ofant:ant-wrapper.shjunit-single-test....Successful output looks like this (28 lines):............
junit-single-test-only
Testsuite: ..ProjectTest
Tests run: 12, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 5.635 sec
------------- Standard Output ---------------
bar
------------- ---------------- ---------------
------------- Standard Error -----------------
bar
------------- ---------------- ---------------

Testcase: testResolveFileWithDriveLetter took 0.034 sec
    SKIPPED: Not DOS or Netware
Testcase: testResolveFileWithDriveLetter took 0.036 sec
Testcase: testInputHandler took 0.007 sec
Testcase: testAddTaskDefinition took 0.179 sec
Testcase: testTaskDefinitionContainsKey took 0.002 sec
Testcase: testDuplicateTargets took 0.05 sec
Testcase: testResolveRelativeFile took 0.002 sec
Testcase: testOutputDuringMessageLoggedIsSwallowed took 0.002 sec
Testcase: testDataTypes took 0.154 sec
Testcase: testDuplicateTargetsImport took 0.086 sec
Testcase: testNullThrowableMessageLog took 0.002 sec
Testcase: testTaskDefinitionContainsValue took 0.002 sec
Testcase: testResolveFile took 0.001 sec

.
BUILD SUCCESSFUL… “failure” output looks like this (34 lines):............
junit-single-test-only
Testsuite: ..ProjectTest
Tests run: 12, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 5.638 sec
------------- Standard Output ---------------
bar
------------- ---------------- ---------------
------------- Standard Error -----------------
bar
------------- ---------------- ---------------

Testcase: testResolveFileWithDriveLetter took 0.033 sec
    SKIPPED: Not DOS or Netware
Testcase: testResolveFileWithDriveLetter took 0.035 sec
Testcase: testInputHandler took 0.005 sec
    FAILED
expected null, but was:<..DefaultInputHandler@61dc03ce>
junit.framework.AssertionFailedError: expected null, but was:<..DefaultInputHandler@61dc03ce>
    at ..ProjectTest.testInputHandler(:254)

Testcase: testAddTaskDefinition took 0.182 sec
Testcase: testTaskDefinitionContainsKey took 0.003 sec
Testcase: testDuplicateTargets took 0.043 sec
Testcase: testResolveRelativeFile took 0.001 sec
Testcase: testOutputDuringMessageLoggedIsSwallowed took 0.003 sec
Testcase: testDataTypes took 0.161 sec
Testcase: testDuplicateTargetsImport took 0.088 sec
Testcase: testNullThrowableMessageLog took 0.001 sec
Testcase: testTaskDefinitionContainsValue took 0.001 sec
Testcase: testResolveFile took 0.001 sec
Test ..ProjectTest FAILED

.
BUILD SUCCESSFUL… and true failure due to a compile error looks like this (12 lines):...
compile
.../Project.java:65: error: cannot find symbol
public class Project implements xResourceFactory {
                                ^
symbol: class xResourceFactory
.../Project.java:2483: error: method does not override or implement a method from a supertype
    @Override
    ^
2 errors

BUILD FAILEDBreaking down thelinesievecommand
(skipping the parts from the traceback example):--section'^(S+):$'tellslinesievesections start with a word followed by a colon.Theshows hide all sections except specific ones.--successand--failuretelllinesieveto exit when encountering one of these patterns.
Note that the failing section is shown
regardless ofshow.sub-cwdmakes absolute paths in the working directory relative.The-scompileoption passed tosubapplies it
only to sections matchingcompile.push compileapplies all the following filters, untilpop,
only to sections matchingcompile.The last twosub--color... '1'color
dotted words followed by a colon at the beginning of the line
(e.g.junit.framework.AssertionFailedError:),
andFAILEDanywhere in the input.Finally,read-cmdexecutes a command and uses its output as input.