Package: mdiff
mdiffmdiff is a package, tool and application for comparing and generating diff info for input data.
It has an ability to detect sequence elements displacements (i.e. line in text have been moved up or down).The features are:NewSequenceMatcherclass generating opcodes with sequence elements displacement detection (new opcodes tags:moveandmoved).API for comparing and generating diff for text inputs.CLI for using package as a tool for comparing files.Simple Standalone GUI application for comparing text contents.LinksPyPIGitHubRequirementsPython 3.8+InstallationFor plain python package (no additional dependencies):pip install mdiffFor additional CLI tool and GUI functionalities (uses external packages such ascolorama,
orTyper):pip install mdiff[cli]ExamplesGenerating opcodes for input sequences:frommdiffimportHeckelSequenceMatchera=['line1','line2','line3','line4','line5']b=['line1','line3','line2','line4','line6']sm=HeckelSequenceMatcher(a,b)opcodes=sm.get_opcodes()foropcodeinopcodes:print(opcode)OpCode('equal', 0, 1, 0, 1)OpCode('move', 1, 2, 2, 2)OpCode('equal', 2, 3, 1, 2)OpCode('moved', 1, 1, 2, 3)OpCode('equal', 3, 4, 3, 4)OpCode('replace', 4, 5, 4, 5)Extracting changes from input sequences using opcodes:...fortag,i1,i2,j1,j2inopcodes:print('{:7}a[{}:{}] --> b[{}:{}]{!r:>8}-->{!r}'.format(tag,i1,i2,j1,j2,a[i1:i2],b[j1:j2]))equal     a[0:1] --> b[0:1]  ['line1'] --> ['line1']move      a[1:2] --> b[2:2]  ['line2'] --> []equal     a[2:3] --> b[1:2]  ['line3'] --> ['line3']moved     a[1:1] --> b[2:3]         [] --> ['line2']equal     a[3:4] --> b[3:4]  ['line4'] --> ['line4']replace   a[4:5] --> b[4:5]  ['line5'] --> ['line6']Generating and printing diff for input strings:frommdiffimportdiff_lines_with_similarities,CompositeOpCodea='line1\nline2\nline3\nline4\nline5'b='line1\nline3\nline2\nline4\nline6'a_lines,b_lines,opcodes=diff_lines_with_similarities(a,b,cutoff=0.75)foropcodeinopcodes:tag,i1,i2,j1,j2=opcodeprint('{:7}a_lines[{}:{}] --> b_lines[{}:{}]{!r:>10}-->{!r}'.format(tag,i1,i2,j1,j2,a_lines[i1:i2],b_lines[j1:j2]))ifisinstance(opcode,CompositeOpCode)andopcode.children_opcodes:forltag,li1,li2,lj1,lj2inopcode.children_opcodes:print('\t{:7}a_lines[{}][{}:{}] --> b_lines[{}][{}:{}]{!r:>10}-->{!r}'.format(ltag,i1,li1,li2,j1,lj1,lj2,a_lines[i1][li1:li2],b_lines[j1][lj1:lj2]))equal     a_lines[0:1] --> b_lines[0:1]  ['line1'] --> ['line1']move      a_lines[1:2] --> b_lines[2:2]  ['line2'] --> []equal     a_lines[2:3] --> b_lines[1:2]  ['line3'] --> ['line3']moved     a_lines[1:1] --> b_lines[2:3]         [] --> ['line2']equal     a_lines[3:4] --> b_lines[3:4]  ['line4'] --> ['line4']replace   a_lines[4:5] --> b_lines[4:5]  ['line5'] --> ['line6']equal     a_lines[4][0:4] --> b_lines[4][0:4]     'line' --> 'line'replace   a_lines[4][4:5] --> b_lines[4][4:5]        '5' --> '6'Indented tags shows in-line differences, in this caseline5andline6strings have the only difference at last character.APIGeneral idea is to provide newSequenceMatcherobject that's compatible with built-indifflib.SequenceMatcherclass,
so they can be used interchangeably to generate opcodes.HeckelSequenceMatcherHeckelSequenceMatcheris a class for comparing pairs of sequences of any type, as long as sequences
are comparable and hashable. Unlike builtindifflib.SequenceMatcher, it detects and marks elements
displacement between sequences. This class providesget_opcodes()method which returns Sequence of opcodes
with differences between sequences, similar asdifflib.SequenceMatcher.get_opcodes()does, but
with additionalmoveandmovedtags for displaced elements.HeckelSequenceMatcherimplements Paul Heckel's algorithm described in"A Technique for Isolating Differences Between Files"paper, which can be foundhere.HeckelSequenceMatcher(a: Sequence[Any] = '', b: Sequence[Any] = '', replace_mode=True)Initialize sequence matcher object, parameters:a- source(old) sequence.b- target(new) sequence.replace_mode- if True: it merges consecutive pairs ofinsertanddeleteblocks intoreplaceoperation. Remainsinsertanddeleteopcodes otherwise.get_opcodes() -> List[OpCode]Returns list ofOpCodeobjects describing how to turn sequenceaintob.OpCodeconsists of attributes:tag,i1,i2,j1,j2.OpCodecan be unpacked as tuple.Usually the first tuple hasi1 == j1 == 0, and remaining tuples havei1equal to thei2from the preceding tuple, and, likewise,j1equal to the previousj2. However, this rule is broken whenmoveandmovedtags appears in OpCodes list, due to sequence elements displacement detection.The tags are strings, with these meanings:replace-a[i1:i2]should be replaced byb[j1:j2]delete-a[i1:i2]should be deleted. Note thatj1==j2in this case.insert-b[j1:j2]should be inserted ata[i1:i1]. Note thati1==i2in this case.equal-a[i1:i2] == b[j1:j2]move-a[i1:i2]should be moved tob[j1:j2]position. Note thatj1==j2in this case.moved- is opposite tag formove. It's not an operation necessary for turning sequenceaintob. It indicates thatb[j1:j2]is moved fromi1position (orb[j1:j2]should be moved back toa[i1:i2]). Note thati1==j2in this case. It can be used for sequence elements movement visualisation.DisplacementSequenceMatcherDisplacementSequenceMatcheris a variation ofHeckelSequenceMatcherclass.
The algorithm keeps tracking of every sequence element occurrence, which might give better result when both sequences have common unique elements.
It tries to detect all sequences elements displacements, whereHeckelSequenceMatchermight sometimes treat displaced elements as delete/insert.
ThisSequenceMatcherclass might be useful when finding all sequences displacements is crucial
, however for ordinary text sequences it might produce human-unreadable diff.This object has the same methods asHeckelSequenceMatcherGenerating text diffdiff_lines_with_similarities(...)This function takes two input text sequences, turns them into lists of lines, generates opcodes for those lines
and tries to find single characters differences in similar lines.Parameters:a: str- source text.b: str- target text.cutoff: float = 0.75- value in range [0:1], where 0.0 means that lines are completely different and 1.0 means that lines are exactly the same. Line similarity cutoff is used to determine if sub opcodes for similar lines should be generated. Ifcutoff == 1, then in-line diff won't be generated.line_sm: SequenceMatcherBase = None- SequenceMatcher object used to find differences between input texts lines.HeckelSequenceMatcher()will be used if not specified.inline_sm: SequenceMatcherBase = None- SequenceMatcher object used to find differences between similar lines (i.e. usingdifflib.SequenceMatcherwhen in-line diff displacement detection is not desirable).difflib.SequenceMatcher()will be used if not specified.keepends = False- Whether to keep newline characters when splitting input sequences.case_sensitive = True- Whether to perform string case-sensitive comparison when generating diff.Returns a tuple(a_lines: List[str], b_lines: List[str], opcodes: List[CompositeOpCode])where:a_lines- is list of lines fromainput text sequence.b_lines- is list of lines frombinput text sequence.opcodes- is list ofCompositeOpCodewhich behave the same way asOpCode(hastag i1 i2 j1 j2fields and can be unpacked), but has additionalchildren_opcodeswhich stores list of nested opcodes with SequenceMatcher result for similar lines. List is empty if lines were not similar enough. (note that similar lines opcodes are generated only forreplacetags, so children_opcodes list will be empty for every other tag).CLI Toolmdiff also provides CLI tool (available only if installed usingpip install mdiff[cli]). For more information
typemdiff --helpUsage: mdiff [OPTIONS] SOURCE_FILE TARGET_FILEReads 2 files from provided paths, compares their content and prints diff.If compared lines in text files are similar enough (exceed cutoff) thenextracts in-line diff.There are few possible strategies to choose to use independently in line-level and in-line-level diff:standard: uses built in python SequenceMatcher object to generate diff,elements movement detection not supported.heckel: detects elements movement in a human-readable form, might notcatch all of moves and differences.displacement: detects all differences and movements, might not be veryuseful when both input files contains     many common lines (for examplemany empty newlines).Arguments:SOURCE_FILE  Source file path to compare.  [required]TARGET_FILE  Target file path to compare.  [required]Options:--line-sm [standard|heckel|displacement]Choose sequence matching method to detectdifferences between lines.  [default:heckel]--inline-sm [standard|heckel|displacement]Choose sequence matching method to detectin-line differences between similar lines.[default: heckel]--cutoff FLOAT RANGE            Line similarity ratio cutoff. If valueexceeded then finds in-line differences insimilar lines.  [default: 0.75; 0.0<=x<=1.0]--gui / --no-gui                Open GUI window with diff result.  [default:no-gui]--case-sensitive / --no-case-sensitiveWhether diff is going to be case sensitive.[default: case-sensitive]--char-mode [utf8|ascii]        Terminal character set used when printingdiff result.  [default: utf8]--color-mode [fore|back]        Terminal color mode used when printing diffresult.  [default: fore]--install-completion [bash|zsh|fish|powershell|pwsh]Install completion for the specified shell.--show-completion [bash|zsh|fish|powershell|pwsh]Show completion for the specified shell, tocopy it or customize the installation.--help                          Show this message and exit.ExampleSample output formdiff a.txt b.txtcommand:Showing result in GUImdiff a.txt b.txt --gui:Standalone GUI applicationStandalone app provides simple text editor, which allows generating diff based on user input.Usemdiff-guicommand to launch GUI application in interpreter mode, or use precompiled version fromhere.Building Standalone GUI applicationGUI application can be built directly from source by typing below commands:> pip install -r requirements.txt> ./pyinstaler_start.sh