tardiff project page

• tardiff.tar.gz v2.1.4 (41,334 bytes)
  "tardiff" is a Perl script used to quickly make a tarball of changes between versions of an archive, or between pre- and post-build of an application. There are many, many other possible uses.

  More complete documentation is now available here.

  Some documentation for applying patches of various sorts is now available here.

  Usage:
  

  • Install tardiff by running the included install.sh program.
  • Just run tardiff without arguments to get this help:

    tardiff [opts...] (outputTAR) (origTAR) [(toTAR)]
     Longer help:  tardiff --help
     TAR options: [z|y|j|Z][c|r|u][i|b][d[R][m]]
     Checksum options: [C[M][F][C|B][o]]
    
     See the manual page for full details and the long option invocation format.
    

  • Or run tardiff with the --help argument to get this help:

    tardiff [opts...] (outputTAR) (origTAR) [(toTAR)]
     TAR options: [z|y|j][c|r|u][i|b][d[R][m]]
      Use "z" to create a tar.gz file or "y" (or "j") to create a tar.bz2 file.
       Or use "Z" to create a .zip file.
      Use "i" to not save a list of removed, moved, renamed, and duplicated files
       as a shell script, or "b" to create a Windows .BATch file in addition to
       the UNIX shell script.
      Use "c" to create a new archive, "r" to append to an existing archive, or
       "u" to "update" an existing archive (only available without (toTAR)).
      Use "d" to scan for duplicate or moved files [requires (toTAR)].
       Use "m" to not scan for moved files while scanning for duplicates.
       Use "R" to add a prompt to the scripts just before removals begin.
      (origTAR) is the base TAR file (what we're comparing from).
      (toTAR) is the newer TAR file (what we're comparing to); if left out, we
       assume the original archive was extracted to the current directory and then
       (possibly) messed with, and check for matching files and directories there.
       ANY new files and directories within any directory MENTIONED in the base
       TAR file will become part of the output TAR file!
    
     Checksum options: [C[M][F][C|B][o]]
      Checksums are only available when (toTAR) is included.
      Use "M" to use "md5sum" instead of "sum".
      Use "F" to speed up checksum generation by not checking if each checksum
       program started properly.  Be warned that using the "B" option with "F" will
       cause files to be considered different if a checksum isn't generated because
       the checksum generator didn't start!
      Use an extra "C" to consider the checksums to be "canonical" -- i.e. don't
       try comparing files with the same checksum, and assume they're identical.
       Use "B" to indicate that (origTAR) is actually a generated CRC listing, and
       not a TAR archive at all; in this case, the CRCs must be "canonical".
      Use "o" if you want to save the checksums generated.  If "md5sum" is being
       used, ".md5" will be added to the TAR file's name; or ".sum" if "sum" is
       being used.
    
      There are additional options available; see the manual page.
    

  • You can also read the manual page, now included in the archive.
  • There's also the tardiff help page, linked above. It's not too much different than the manual (well, so far as tardiff's functionality goes...), but (if your browser supports Cascading Style Sheets, or CSS) it can be a handy reference; feel free to save them for your own local reference!
  • If tardiff beeps while running, it means that it's encountered a problem: either it doesn't know what to do about it, or it knows it's messed up. The following problems, for which tardiff might or might not beep, are known:
    • If an object of a given name changes it's type (i.e. what was a directory becomes a file...), extracting the resulting TAR archive might not do what you want, depending on the extraction program...
    • If you give a checksum file instead of a TAR archive without telling tardiff about it, tardiff will beep for a period of time and (in the end) believe that that file contains nothing. This also happens if you give a ZIP archive, except there's no way to tell tardiff about ZIP archives!
    • It appears that an unknown TAR file creator doesn't include the "ustar" signature in its file headers in the created TAR archive. This causes tardiff to beep constantly, discarding all the (otherwise valid) headers, and finally concludes that the archive in question is empty. I'm not sure what to do about this, since the signature's presence could be considered an important check to ensure that tardiff is not getting out of sync with the archive it's reading...
      Here's a discussion about the problem and the trouble involved in fixing it.
    • It appears that old versions of tar generate spaces (hex 20) instead of NULL bytes (hex 00) in some parts of the file headers generated. As soon as I figure out what I'm doing with the "ustar" signature problem, I'll fix this at the same time.

    If you encounter a new problem, please let me know.

  Short example #1:

  • cd /usr/src
  • tar zxvf ~/download/FlightGear-0.9.5-pre3.tar.gz
  • ./configure
  • ./make
  • tardiff z FlightGear-0.9.5-pre3-build.tar.gz ~/download/FlightGear-0.9.5-pre3.tar.gz

  This compares FROM pre3.tar.gz TO the current directory. Differences are written to FlightGear-0.9.5-pre3-build.tar.gz
  Now the build files can be archived separate from the source code.
  If this directory is needed again in the future, just untar the source tarball first, then the build tarball.

  2nd example, making the patch file requested on mailing list (substitute your locations for tarballs):

  • tardiff zcb fgfs-base-0.9.5-pre3-patch.tar.gz ~/download/fgfs-base-0.9.5-pre2.tar.gz ~/download/fgfs-base-0.9.5-pre3.tar.gz

  This makes a tarball of only the changes FROM pre2 TO pre3

  3rd example, making a patch file and generating md5 checksums at the same time (requires md5sum):
  

  • tardiff zcbCMo fgfs-base-0.9.5-pre2to3-patch.tar.gz ~/download/fgfs-base-0.9.5-pre2.tar.gz ~/download/fgfs-base-0.9.5-pre3.tar.gz

  This creates the tarball of differences (fgfs-base-0.9.5-pre2to3-patch.tar.gz) and writes the MD5 checksums to ~/download/fgfs-base-0.9.5-pre2.tar.gz.md5 and ~/download/fgfs-base-0.9.5-pre3.tar.gz.md5.

  4th example, making a patch file from 0.9.5-pre3 to 0.9.5-final after you've erased your 0.9.5-pre3 TAR file: (requires checksum generated for 0.9.5-pre3 in previous example)
  

  • tardiff zcbCMB fgfs-base-0.9.5-pre3toFinal-patch.tar.gz ~/download/fgfs-base-0.9.5-pre3.tar.gz.md5 ~/download/fgfs-base-0.9.5.tar.gz

  To generate a checksum file for 0.9.5-final, just add an "o" after the "B" in the options.

  You could have renamed and compressed the checksum file as well; tardiff recognizes compressed files by their extention (.tgz, .gz, .bz2); the compression-specifying letter at the beginning (z, y, or j) only controls the compression that will be used in changing the output tarball.

  It has to be told that the input is a checksum file, however, and the type of checksum to use; it won't realize that information on its own.

  5th example, making a checksum file:

  • tardiff iCMo temp fgfs-base-0.9.5-pre2.tar.gz /dev/null

    The tardiff command will try to compare an existing archive and -- from it's point of view -- a blank archive. The file "temp" will never be created, and the "i" option prevents tardiff from writing out its shell script for removing all the files in the archive of interest.

  6th example, making a patch file from 0.9.4 (final) to 0.9.5 (final):

  • tardiff zcb fgfs-base-0.9.4-0.9.5-patch.tar.gz ~/download/fgfs-base-0.9.4.tar.gz ~/download/fgfs-base-0.9.5.tar.gz

  The result isn't as small as it could be (yet), but tardiff can now recognize that 0.9.4 has an extra directory at the top of its archive, and it will discard that directory when it realizes that doing so means at least one file can be found in both archives.

  To make it go faster, "CMC" (without the quotes) can be added to the options -- this tells tardiff to calculate CRCs for both archives using md5sum and to not do any further comparisons, probably eliminating several passes through the tarballs for comparison purposes.

  The action is "speculative" -- it's not really sure if doing it will help at first, but it realizes that something has to be done if the output archive is going to be anything but a "delete everything that's listed in the base archive" mixed with a "store everything that's listed in the "intended result" (toTAR) archive.

  When speculation is in use, it also adds a little extra work onto the receiver; in this case, the patch archive has to be extracted into the directory containing the "data" subdirectory (i.e. just under the original "FlightGear-0.9.4" directory). Had this been a patch from 0.9.5 to 0.9.4, however, the "data" directory extracted for 0.9.5 would have to be placed in a "FlightGear-0.9.4" directory, and the patch would need to be extracted from the directory containing that "FlightGear-0.9.4" directory. In other words, the "result" archive's directory structure is the controlling one.

  This will only catch a single directory level on one side; any worse and the program effectively figures that it's out of its league and makes that massive archive anyway.

  All of these examples used the old, short argument form. A longer form is available in the new version, featuring plenty of dashes before the arguments, for those who prefer their arguments that way.

  A final note about extracting: Unless you tell it not to (via the "i" option), or no files or directories were deleted from the base tarball to the current directory or tarball (whichever you are comparing against), tardiff will generate a finish.sh shell script and stuff it into the archive. A file with this name in the current directory will be erased if it exists. If the "b" option is used, tardiff will similarly generate a finish.bat batch file. When the archive is extracted, one of these (the shell script on a Unix, the batch file on a Windows system) should be executed to delete files that should no longer be present and rearrange the other files (moving and copying), and then these files can be erased. This was done because TAR can't store deletions, duplicates (well, other than as complete files!), and movements as a part of the archive.

  At present, tardiff is shelved; however, the author will fix any reported bugs as quickly as possible. Future suggested features can be seen on the help page (referenced above).

Thanks should go to Boris Koenig for suggesting the addition of checksums, comparison of two TAR archives, and duplicate/removal/moving recognition.