Clirr is a tool that checks Java libraries for binary and source compatibility with older releases. Basically you give it two sets of jar files and Clirr dumps out a list of changes in the public API. Clirr provides an Ant task that can be configured to break the build if it detects incompatible API changes.