If you really want to improve readability, i would extract a class here named `DiffExtractor` instead of passing prefix and suffix between all the methods:

    DiffExtractor extractor = new DiffExtractor(expected, actual);
    String compactedPrefix = extractor.compactPrefix();
    String compactedSuffix = extractor.compactSuffix();
    return Assert.format(message,
            compactedPrefix + extractor.expectedDiff() + compactedSuffix,
            compactedPrefix + extractor.actualDiff() + compactedSuffix);