aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorMartin Matuska <mm@FreeBSD.org>2013-07-31 22:54:02 +0000
committerMartin Matuska <mm@FreeBSD.org>2013-07-31 22:54:02 +0000
commiteff47538ce7958ec954550a4d589dd675b342131 (patch)
treea8a0ae3a97cc5e011ea082c1dd1992b3f77dedb1
parent11adf5d1a55793f14889ef0ff675695c4fb97f41 (diff)
downloadsrc-eff47538ce7958ec954550a4d589dd675b342131.tar.gz
src-eff47538ce7958ec954550a4d589dd675b342131.zip
Update vendor/xz from v5.0 branch to post-5.0.5
Git commit: b69900ed0b2f914fc6c0a180dcb522dbe5b80ea7
Notes
Notes: svn path=/vendor/xz/dist/; revision=253848
-rw-r--r--ChangeLog345
-rw-r--r--README4
-rw-r--r--THANKS4
-rw-r--r--src/liblzma/api/lzma/version.h2
-rw-r--r--src/liblzma/check/sha256.c1
-rw-r--r--src/liblzma/common/alone_decoder.c22
-rw-r--r--src/liblzma/common/alone_decoder.h5
-rw-r--r--src/liblzma/common/auto_decoder.c2
-rw-r--r--src/lzmainfo/lzmainfo.14
-rw-r--r--src/xz/coder.c37
-rw-r--r--src/xz/coder.h2
-rw-r--r--src/xz/file_io.c38
-rw-r--r--src/xz/list.c28
-rw-r--r--src/xz/util.c14
-rw-r--r--src/xz/xz.159
-rw-r--r--src/xzdec/xzdec.110
-rw-r--r--src/xzdec/xzdec.c10
17 files changed, 498 insertions, 89 deletions
diff --git a/ChangeLog b/ChangeLog
index 140106eb3d66..5ee316a61591 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,348 @@
+commit b69900ed0b2f914fc6c0a180dcb522dbe5b80ea7
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Sun Jun 30 18:02:27 2013 +0300
+
+ Man pages: Use similar syntax for synopsis as in xz.
+
+ The man pages of lzmainfo, xzmore, and xzdec had similar
+ constructs as the man page of xz had before the commit
+ eb6ca9854b8eb9fbf72497c1cf608d6b19d2d494. Eric S. Raymond
+ didn't mention these man pages in his bug report, but
+ it's nice to be consistent.
+
+commit cf4a1e1879d89be314ef3c064bd2656ea452f87e
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Sun Jun 30 15:55:09 2013 +0300
+
+ Update NEWS for 5.0.5.
+
+commit cb94bb6d1f34e1e93c2d634ea9c3b7dfb3981d05
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Sun Jun 30 15:54:38 2013 +0300
+
+ Bump version and soname for 5.0.5.
+
+commit b7dee202d5b041ccae028d0c5433b83cecbe9e5d
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Fri Jun 28 23:56:17 2013 +0300
+
+ xz: Fix return value type in io_write_buf().
+
+ It didn't affect the behavior of the code since -1
+ becomes true anyway.
+
+commit 265e7b44d804b47373f10b7da28350db7611cea6
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Fri Jun 28 18:46:13 2013 +0300
+
+ xz: Remove an outdated NetBSD-specific comment.
+
+ Nowadays errno == EFTYPE is documented in open(2).
+
+commit 78c2f8db902195468b8249c432252a6b281db836
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Fri Jun 28 18:09:47 2013 +0300
+
+ xz: Fix error detection of fcntl(fd, F_SETFL, flags) calls.
+
+ POSIX says that fcntl(fd, F_SETFL, flags) returns -1 on
+ error and "other than -1" on success. This is how it is
+ documented e.g. on OpenBSD too. On Linux, success with
+ F_SETFL is always 0 (at least accorinding to fcntl(2)
+ from man-pages 3.51).
+
+commit 91750dff8f2c654ff636f12a2acdffe5492374b3
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Fri Jun 28 17:36:47 2013 +0300
+
+ xz: Fix use of wrong variable in a fcntl() call.
+
+ Due to a wrong variable name, when writing a sparse file
+ to standard output, *all* file status flags were cleared
+ (to the extent the operating system allowed it) instead of
+ only clearing the O_APPEND flag. In practice this worked
+ fine in the common situations on GNU/Linux, but I didn't
+ check how it behaved elsewhere.
+
+ The original flags were still restored correctly. I still
+ changed the code to use a separate boolean variable to
+ indicate when the flags should be restored instead of
+ relying on a special value in stdout_flags.
+
+commit e11888a79a4a77a69afde60445880d44f63d01aa
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Wed Jun 26 13:30:57 2013 +0300
+
+ xz: Check the value of lzma_stream_flags.version in --list.
+
+ It is a no-op for now, but if an old xz version is used
+ together with a newer liblzma that supports something new,
+ then this check becomes important and will stop the old xz
+ from trying to parse files that it won't understand.
+
+commit f39ddd88f3222219ada88998cf30abfdd3e0e96c
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Wed Jun 26 12:17:00 2013 +0300
+
+ Build: Require Automake 1.12 and use serial-tests option.
+
+ It should actually still work with Automake 1.10 if
+ the serial-tests option is removed. Automake 1.13 started
+ using parallel tests by default and the option to get
+ the old behavior isn't supported before 1.12.
+
+ At least for now, parallel tests don't improve anything
+ in XZ Utils but they hide the progress output from
+ test_compress.sh.
+
+commit cb84e278027a90e9827a6f4d3bb0b4d4744a2fbb
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Sun Jun 23 17:36:47 2013 +0300
+
+ xz: Validate Uncompressed Size from Block Header in list.c.
+
+ This affects only "xz -lvv". Normal decompression with xz
+ already detected if Block Header and Index had mismatched
+ Uncompressed Size fields. So this just makes "xz -lvv"
+ show such files as corrupt instead of showing the
+ Uncompressed Size from Index.
+
+commit f01780fce454c7489f7dcbf806299b50da5f51b7
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Wed Jun 26 10:58:58 2013 +0300
+
+ Update THANKS.
+
+commit d98ede7d700b892e32d9c2f46563b6ebc566786d
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Fri Jun 21 22:04:45 2013 +0300
+
+ xz: Make the man page more friendly to doclifter.
+
+ Thanks to Eric S. Raymond.
+
+commit 19b447b64b3f520cd5b11429000b092f7c76709b
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Fri Jun 21 21:54:59 2013 +0300
+
+ xz: A couple of man page fixes.
+
+ Now the interaction of presets and custom filter chains
+ is described correctly. Earlier it contradicted itself.
+
+ Thanks to DevHC who reported these issues on IRC to me
+ on 2012-12-14.
+
+commit 45edf2966fc9a4d2eae8f84b2fa027fb4fa1df8b
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Fri Jun 21 21:50:26 2013 +0300
+
+ xz: Fix interaction between preset and custom filter chains.
+
+ There was somewhat illogical behavior when --extreme was
+ specified and mixed with custom filter chains.
+
+ Before this commit, "xz -9 --lzma2 -e" was equivalent
+ to "xz --lzma2". After it is equivalent to "xz -6e"
+ (all earlier preset options get forgotten when a custom
+ filter chain is specified and the default preset is 6
+ to which -e is applied). I find this less illogical.
+
+ This also affects the meaning of "xz -9e --lzma2 -7".
+ Earlier it was equivalent to "xz -7e" (the -e specified
+ before a custom filter chain wasn't forgotten). Now it
+ is "xz -7". Note that "xz -7e" still is the same as "xz -e7".
+
+ Hopefully very few cared about this in the first place,
+ so pretty much no one should even notice this change.
+
+ Thanks to Conley Moorhous.
+
+commit b065984e5a9272eb50bc0c6d3731e6199c0ae8a8
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Fri Apr 8 17:53:05 2011 +0300
+
+ xz: Change size_t to uint32_t in a few places.
+
+commit 32be621f52f2e1686db88baa7b01dc1ae338f426
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Sat Apr 27 22:07:46 2013 +0300
+
+ Build: Use -Wvla with GCC if supported.
+
+ Variable-length arrays are mandatory in C99 but optional in C11.
+ The code doesn't currently use any VLAs and it shouldn't in the
+ future either to stay compatible with C11 without requiring any
+ optional C11 features.
+
+commit efb07cfba65e9e05984c02cd796c1b0338ce04dc
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Mon Apr 15 19:29:09 2013 +0300
+
+ xzdec: Improve the --help message.
+
+ The options are now ordered in the same order as in xz's help
+ message.
+
+ Descriptions were added to the options that are ignored.
+ I left them in parenthesis even if it looks a bit weird
+ because I find it easier to spot the ignored vs. non-ignored
+ options from the list that way.
+
+commit e3c8be13699e2813f5e2879d8187444b46d82d89
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Fri Apr 5 19:25:40 2013 +0300
+
+ Update THANKS.
+
+commit ad8282efe483612f6b5544f9a0d2e4914fb2532a
+Author: Jeff Bastian <jbastian@redhat.com>
+Date: Wed Apr 3 13:59:17 2013 +0200
+
+ xzgrep: make the '-h' option to be --no-filename equivalent
+
+ * src/scripts/xzgrep.in: Accept the '-h' option in argument parsing.
+
+commit 9271a3eb0e022b23e8712154be851d0afe4c02e4
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Fri Apr 5 19:34:09 2013 +0300
+
+ liblzma: Be less picky in lzma_alone_decoder().
+
+ To avoid false positives when detecting .lzma files,
+ rare values in dictionary size and uncompressed size fields
+ were rejected. They will still be rejected if .lzma files
+ are decoded with lzma_auto_decoder(), but when using
+ lzma_alone_decoder() directly, such files will now be accepted.
+ Hopefully this is an OK compromise.
+
+ This doesn't affect xz because xz still has its own file
+ format detection code. This does affect lzmadec though.
+ So after this commit lzmadec will accept files that xz or
+ xz-emulating-lzma doesn't.
+
+ NOTE: lzma_alone_decoder() still won't decode all .lzma files
+ because liblzma's LZMA decoder doesn't support lc + lp > 4.
+
+ Reported here:
+ http://sourceforge.net/projects/lzmautils/forums/forum/708858/topic/7068827
+
+ Conflicts:
+ src/liblzma/common/alone_decoder.c
+ src/liblzma/common/alone_decoder.h
+
+commit 211b931cee58626c1d2e021810cb108cb5cbc10f
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Tue Mar 5 19:14:50 2013 +0200
+
+ Avoid unneeded use of awk in xzless.
+
+ Use "read" instead of "awk" in xzless to get the version
+ number of "less". The need for awk was introduced in
+ the commit db5c1817fabf7cbb9e4087b1576eb26f0747338e.
+
+ Thanks to Ariel P for the patch.
+
+commit 9f62fd9605eade23b62b07a235d1f02156f7a5c6
+Author: Jonathan Nieder <jrnieder@gmail.com>
+Date: Mon Nov 19 00:10:10 2012 -0800
+
+ xzless: Make "less -V" parsing more robust
+
+ In v4.999.9beta~30 (xzless: Support compressed standard input,
+ 2009-08-09), xzless learned to parse ‘less -V’ output to figure out
+ whether less is new enough to handle $LESSOPEN settings starting
+ with “|-”. That worked well for a while, but the version string from
+ ‘less’ versions 448 (June, 2012) is misparsed, producing a warning:
+
+ $ xzless /tmp/test.xz; echo $?
+ /usr/bin/xzless: line 49: test: 456 (GNU regular expressions): \
+ integer expression expected
+ 0
+
+ More precisely, modern ‘less’ lists the regexp implementation along
+ with its version number, and xzless passes the entire version number
+ with attached parenthetical phrase as a number to "test $a -gt $b",
+ producing the above confusing message.
+
+ $ less-444 -V | head -1
+ less 444
+ $ less -V | head -1
+ less 456 (no regular expressions)
+
+ So relax the pattern matched --- instead of expecting "less <number>",
+ look for a line of the form "less <number>[ (extra parenthetical)]".
+ While at it, improve the behavior when no matching line is found ---
+ instead of producing a cryptic message, we can fall back on a LESSPIPE
+ setting that is supported by all versions of ‘less’.
+
+ The implementation uses "awk" for simplicity. Hopefully that’s
+ portable enough.
+
+ Reported-by: Jörg-Volker Peetz <jvpeetz@web.de>
+ Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
+
+commit 1d05980f5b5c2c94d833001daccacce4a466876e
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Wed Oct 3 15:54:24 2012 +0300
+
+ xz: Fix the note about --rsyncable on the man page.
+
+commit fb68497333598688d309a92838d91fd560f7e9f0
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Fri Sep 28 20:11:09 2012 +0300
+
+ xz: Improve handling of failed realloc in xrealloc.
+
+ Thanks to Jim Meyering.
+
+commit 75013db6d4d63c195bd8b8d45729b4be0665a812
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Sat Dec 15 20:01:02 2012 +0200
+
+ A few typo fixes to comments and the xz man page.
+
+ Thanks to Jim Meyering.
+
+commit e44b21839b1dcbac5097be39b87dd2ddb6e114fd
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Thu Aug 2 17:13:30 2012 +0300
+
+ Build: Bump gettext version requirement to 0.18.
+
+ Otherwise too old version of m4/lib-link.m4 gets included
+ when autoreconf -fi is run.
+
+commit fd3dbb23ca7e75a7a888d7e897c381dc06308307
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Thu Jul 5 07:36:28 2012 +0300
+
+ Tests: Remove tests/test_block.c that had gotten committed accidentally.
+
+commit 05a735d279d74af437c31f25f69aded4713c1a3d
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Thu Jul 5 07:33:35 2012 +0300
+
+ Build: Include macosx/build.sh in the distribution.
+
+ It has been in the Git repository since 2010 but probably
+ few people have seen it since it hasn't been included in
+ the release tarballs. :-(
+
+commit 4e6d62793b5e7b87edcc93c7ded072c1ecd94173
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Thu Jul 5 07:24:45 2012 +0300
+
+ Docs: Fix the name LZMA Utils -> XZ Utils in debug/README.
+
+commit dd95b5e7614baf1f07a1316b5106bd616a9efa79
+Author: Lasse Collin <lasse.collin@tukaani.org>
+Date: Thu Jul 5 07:23:17 2012 +0300
+
+ Include debug/translation.bash in the distribution.
+
+ Also fix the script name mentioned in README.
+
commit 20778053a07eb90c159c1377ca8dc05a90fd530b
Author: Lasse Collin <lasse.collin@tukaani.org>
Date: Fri Jun 22 14:36:16 2012 +0300
diff --git a/README b/README
index 1b02f05e3de8..ab8aadfb5338 100644
--- a/README
+++ b/README
@@ -210,8 +210,8 @@ XZ Utils
# <Edit the .po file in the po directory.>
make -C po update-po
make install
- bash debug/translations.bash | less
- bash debug/translations.bash | less -S # For --list outputs
+ bash debug/translation.bash | less
+ bash debug/translation.bash | less -S # For --list outputs
Repeat the above as needed (no need to re-run configure though).
diff --git a/THANKS b/THANKS
index b13503e2d360..36b2e74c9fec 100644
--- a/THANKS
+++ b/THANKS
@@ -6,6 +6,7 @@ Some people have helped more, some less, but nevertheless everyone's help
has been important. :-) In alphabetical order:
- Mark Adler
- H. Peter Anvin
+ - Jeff Bastian
- Nelson H. F. Beebe
- Karl Berry
- Anders F. Björklund
@@ -47,6 +48,7 @@ has been important. :-) In alphabetical order:
- Bela Lubkin
- Gregory Margo
- Jim Meyering
+ - Conley Moorhous
- Rafał Mużyło
- Adrien Nader
- Hongbo Ni
@@ -58,8 +60,10 @@ has been important. :-) In alphabetical order:
- Diego Elio Pettenò
- Elbert Pol
- Mikko Pouru
+ - Pavel Raiskup
- Robert Readman
- Bernhard Reutner-Fischer
+ - Eric S. Raymond
- Cristian Rodríguez
- Christian von Roques
- Jukka Salmi
diff --git a/src/liblzma/api/lzma/version.h b/src/liblzma/api/lzma/version.h
index 629e8e1a5be8..66e93965dcab 100644
--- a/src/liblzma/api/lzma/version.h
+++ b/src/liblzma/api/lzma/version.h
@@ -22,7 +22,7 @@
*/
#define LZMA_VERSION_MAJOR 5
#define LZMA_VERSION_MINOR 0
-#define LZMA_VERSION_PATCH 4
+#define LZMA_VERSION_PATCH 5
#define LZMA_VERSION_STABILITY LZMA_VERSION_STABILITY_STABLE
#ifndef LZMA_VERSION_COMMIT
diff --git a/src/liblzma/check/sha256.c b/src/liblzma/check/sha256.c
index 04231dbaef99..23bda9222dda 100644
--- a/src/liblzma/check/sha256.c
+++ b/src/liblzma/check/sha256.c
@@ -6,7 +6,6 @@
/// \todo Crypto++ has x86 ASM optimizations. They use SSE so if they
/// are imported to liblzma, SSE instructions need to be used
/// conditionally to keep the code working on older boxes.
-/// We could also support using some external libary for SHA-256.
//
// This code is based on the code found from 7-Zip, which has a modified
// version of the SHA-256 found from Crypto++ <http://www.cryptopp.com/>.
diff --git a/src/liblzma/common/alone_decoder.c b/src/liblzma/common/alone_decoder.c
index 678c79e65433..c25112e6875f 100644
--- a/src/liblzma/common/alone_decoder.c
+++ b/src/liblzma/common/alone_decoder.c
@@ -26,6 +26,11 @@ struct lzma_coder_s {
SEQ_CODE,
} sequence;
+ /// If true, reject files that are unlikely to be .lzma files.
+ /// If false, more non-.lzma files get accepted and will give
+ /// LZMA_DATA_ERROR either immediately or after a few output bytes.
+ bool picky;
+
/// Position in the header fields
size_t pos;
@@ -68,13 +73,13 @@ alone_decode(lzma_coder *coder,
|= (size_t)(in[*in_pos]) << (coder->pos * 8);
if (++coder->pos == 4) {
- if (coder->options.dict_size != UINT32_MAX) {
+ if (coder->picky && coder->options.dict_size
+ != UINT32_MAX) {
// A hack to ditch tons of false positives:
// We allow only dictionary sizes that are
// 2^n or 2^n + 2^(n-1). LZMA_Alone created
// only files with 2^n, but accepts any
- // dictionary size. If someone complains, this
- // will be reconsidered.
+ // dictionary size.
uint32_t d = coder->options.dict_size - 1;
d |= d >> 2;
d |= d >> 3;
@@ -103,9 +108,9 @@ alone_decode(lzma_coder *coder,
// Another hack to ditch false positives: Assume that
// if the uncompressed size is known, it must be less
- // than 256 GiB. Again, if someone complains, this
- // will be reconsidered.
- if (coder->uncompressed_size != LZMA_VLI_UNKNOWN
+ // than 256 GiB.
+ if (coder->picky
+ && coder->uncompressed_size != LZMA_VLI_UNKNOWN
&& coder->uncompressed_size
>= (LZMA_VLI_C(1) << 38))
return LZMA_FORMAT_ERROR;
@@ -189,7 +194,7 @@ alone_decoder_memconfig(lzma_coder *coder, uint64_t *memusage,
extern lzma_ret
lzma_alone_decoder_init(lzma_next_coder *next, lzma_allocator *allocator,
- uint64_t memlimit)
+ uint64_t memlimit, bool picky)
{
lzma_next_coder_init(&lzma_alone_decoder_init, next, allocator);
@@ -208,6 +213,7 @@ lzma_alone_decoder_init(lzma_next_coder *next, lzma_allocator *allocator,
}
next->coder->sequence = SEQ_PROPERTIES;
+ next->coder->picky = picky;
next->coder->pos = 0;
next->coder->options.dict_size = 0;
next->coder->options.preset_dict = NULL;
@@ -223,7 +229,7 @@ lzma_alone_decoder_init(lzma_next_coder *next, lzma_allocator *allocator,
extern LZMA_API(lzma_ret)
lzma_alone_decoder(lzma_stream *strm, uint64_t memlimit)
{
- lzma_next_strm_init(lzma_alone_decoder_init, strm, memlimit);
+ lzma_next_strm_init(lzma_alone_decoder_init, strm, memlimit, false);
strm->internal->supported_actions[LZMA_RUN] = true;
strm->internal->supported_actions[LZMA_FINISH] = true;
diff --git a/src/liblzma/common/alone_decoder.h b/src/liblzma/common/alone_decoder.h
index 70d0d2a0ffa4..f666fc3823e1 100644
--- a/src/liblzma/common/alone_decoder.h
+++ b/src/liblzma/common/alone_decoder.h
@@ -16,7 +16,8 @@
#include "common.h"
-extern lzma_ret lzma_alone_decoder_init(lzma_next_coder *next,
- lzma_allocator *allocator, uint64_t memlimit);
+extern lzma_ret lzma_alone_decoder_init(
+ lzma_next_coder *next, lzma_allocator *allocator,
+ uint64_t memlimit, bool picky);
#endif
diff --git a/src/liblzma/common/auto_decoder.c b/src/liblzma/common/auto_decoder.c
index ae6c3e78642c..35c895fd14c2 100644
--- a/src/liblzma/common/auto_decoder.c
+++ b/src/liblzma/common/auto_decoder.c
@@ -54,7 +54,7 @@ auto_decode(lzma_coder *coder, lzma_allocator *allocator,
coder->memlimit, coder->flags));
} else {
return_if_error(lzma_alone_decoder_init(&coder->next,
- allocator, coder->memlimit));
+ allocator, coder->memlimit, true));
// If the application wants to know about missing
// integrity check or about the check in general, we
diff --git a/src/lzmainfo/lzmainfo.1 b/src/lzmainfo/lzmainfo.1
index f2b93b4c643f..ce38eee50324 100644
--- a/src/lzmainfo/lzmainfo.1
+++ b/src/lzmainfo/lzmainfo.1
@@ -4,14 +4,14 @@
.\" This file has been put into the public domain.
.\" You can do whatever you want with this file.
.\"
-.TH LZMAINFO 1 "2010-09-27" "Tukaani" "XZ Utils"
+.TH LZMAINFO 1 "2013-06-30" "Tukaani" "XZ Utils"
.SH NAME
lzmainfo \- show information stored in the .lzma file header
.SH SYNOPSIS
.B lzmainfo
.RB [ \-\-help ]
.RB [ \-\-version ]
-.RI [ file ]...
+.RI [ file... ]
.SH DESCRIPTION
.B lzmainfo
shows information stored in the
diff --git a/src/xz/coder.c b/src/xz/coder.c
index 69b1225d946d..e0867e2aa295 100644
--- a/src/xz/coder.c
+++ b/src/xz/coder.c
@@ -37,15 +37,10 @@ static io_buf in_buf;
static io_buf out_buf;
/// Number of filters. Zero indicates that we are using a preset.
-static size_t filters_count = 0;
+static uint32_t filters_count = 0;
/// Number of the preset (0-9)
-static size_t preset_number = 6;
-
-/// If a preset is used (no custom filter chain) and preset_extreme is true,
-/// a significantly slower compression is used to achieve slightly better
-/// compression ratio.
-static bool preset_extreme = false;
+static uint32_t preset_number = LZMA_PRESET_DEFAULT;
/// Integrity check type
static lzma_check check;
@@ -63,11 +58,9 @@ coder_set_check(lzma_check new_check)
}
-extern void
-coder_set_preset(size_t new_preset)
+static void
+forget_filter_chain(void)
{
- preset_number = new_preset;
-
// Setting a preset makes us forget a possibly defined custom
// filter chain.
while (filters_count > 0) {
@@ -81,9 +74,20 @@ coder_set_preset(size_t new_preset)
extern void
+coder_set_preset(uint32_t new_preset)
+{
+ preset_number &= ~LZMA_PRESET_LEVEL_MASK;
+ preset_number |= new_preset;
+ forget_filter_chain();
+ return;
+}
+
+
+extern void
coder_set_extreme(void)
{
- preset_extreme = true;
+ preset_number |= LZMA_PRESET_EXTREME;
+ forget_filter_chain();
return;
}
@@ -98,6 +102,12 @@ coder_add_filter(lzma_vli id, void *options)
filters[filters_count].options = options;
++filters_count;
+ // Setting a custom filter chain makes us forget the preset options.
+ // This makes a difference if one specifies e.g. "xz -9 --lzma2 -e"
+ // where the custom filter chain resets the preset level back to
+ // the default 6, making the example equivalent to "xz -6e".
+ preset_number = LZMA_PRESET_DEFAULT;
+
return;
}
@@ -134,9 +144,6 @@ coder_set_compression_settings(void)
}
// Get the preset for LZMA1 or LZMA2.
- if (preset_extreme)
- preset_number |= LZMA_PRESET_EXTREME;
-
if (lzma_lzma_preset(&opt_lzma, preset_number))
message_bug();
diff --git a/src/xz/coder.h b/src/xz/coder.h
index 4626466f39ce..2d3add972745 100644
--- a/src/xz/coder.h
+++ b/src/xz/coder.h
@@ -46,7 +46,7 @@ extern bool opt_auto_adjust;
extern void coder_set_check(lzma_check check);
/// Set preset number
-extern void coder_set_preset(size_t new_preset);
+extern void coder_set_preset(uint32_t new_preset);
/// Enable extreme mode
extern void coder_set_extreme(void);
diff --git a/src/xz/file_io.c b/src/xz/file_io.c
index f9b7f305e09c..871a099b6193 100644
--- a/src/xz/file_io.c
+++ b/src/xz/file_io.c
@@ -41,9 +41,10 @@ static bool warn_fchown;
static bool try_sparse = true;
#ifndef TUKLIB_DOSLIKE
-/// File status flags of standard output. This is used by io_open_dest()
-/// and io_close_dest().
-static int stdout_flags = 0;
+/// Original file status flags of standard output. This is used by
+/// io_open_dest() and io_close_dest() to save and restore the flags.
+static int stdout_flags;
+static bool restore_stdout_flags = false;
#endif
@@ -397,10 +398,6 @@ io_open_src_real(file_pair *pair)
was_symlink = true;
# elif defined(__NetBSD__)
- // As of 2010-09-05, NetBSD doesn't document what errno is
- // used with O_NOFOLLOW. It is EFTYPE though, and I
- // understood that is very unlikely to change even though
- // it is undocumented.
if (errno == EFTYPE)
was_symlink = true;
@@ -441,7 +438,7 @@ io_open_src_real(file_pair *pair)
flags &= ~O_NONBLOCK;
- if (fcntl(pair->src_fd, F_SETFL, flags))
+ if (fcntl(pair->src_fd, F_SETFL, flags) == -1)
goto error_msg;
}
#endif
@@ -634,11 +631,11 @@ io_open_dest_real(file_pair *pair)
if (!S_ISREG(pair->dest_st.st_mode))
return false;
- const int flags = fcntl(STDOUT_FILENO, F_GETFL);
- if (flags == -1)
+ stdout_flags = fcntl(STDOUT_FILENO, F_GETFL);
+ if (stdout_flags == -1)
return false;
- if (flags & O_APPEND) {
+ if (stdout_flags & O_APPEND) {
// Creating a sparse file is not possible
// when O_APPEND is active (it's used by
// shell's >> redirection). As I understand
@@ -657,12 +654,14 @@ io_open_dest_real(file_pair *pair)
return false;
if (fcntl(STDOUT_FILENO, F_SETFL,
- stdout_flags & ~O_APPEND))
+ stdout_flags & ~O_APPEND)
+ == -1)
return false;
- // Remember the flags so that io_close_dest()
- // can restore them.
- stdout_flags = flags;
+ // Disabling O_APPEND succeeded. Mark
+ // that the flags should be restored
+ // in io_close_dest().
+ restore_stdout_flags = true;
} else if (lseek(STDOUT_FILENO, 0, SEEK_CUR)
!= pair->dest_st.st_size) {
@@ -703,13 +702,12 @@ io_close_dest(file_pair *pair, bool success)
{
#ifndef TUKLIB_DOSLIKE
// If io_open_dest() has disabled O_APPEND, restore it here.
- if (stdout_flags != 0) {
+ if (restore_stdout_flags) {
assert(pair->dest_fd == STDOUT_FILENO);
- const int fail = fcntl(STDOUT_FILENO, F_SETFL, stdout_flags);
- stdout_flags = 0;
+ restore_stdout_flags = false;
- if (fail) {
+ if (fcntl(STDOUT_FILENO, F_SETFL, stdout_flags) == -1) {
message_error(_("Error restoring the O_APPEND flag "
"to standard output: %s"),
strerror(errno));
@@ -882,7 +880,7 @@ io_write_buf(file_pair *pair, const uint8_t *buf, size_t size)
if (amount == -1) {
if (errno == EINTR) {
if (user_abort)
- return -1;
+ return true;
continue;
}
diff --git a/src/xz/list.c b/src/xz/list.c
index 98307eb247ef..0e73d519ea4a 100644
--- a/src/xz/list.c
+++ b/src/xz/list.c
@@ -203,6 +203,20 @@ parse_indexes(xz_file_info *xfi, file_pair *pair)
goto error;
}
+ // Check that the Stream Footer doesn't specify something
+ // that we don't support. This can only happen if the xz
+ // version is older than liblzma and liblzma supports
+ // something new.
+ //
+ // It is enough to check Stream Footer. Stream Header must
+ // match when it is compared against Stream Footer with
+ // lzma_stream_flags_compare().
+ if (footer_flags.version != 0) {
+ message_error("%s: %s", pair->src_name,
+ message_strm(LZMA_OPTIONS_ERROR));
+ goto error;
+ }
+
// Check that the size of the Index field looks sane.
lzma_vli index_size = footer_flags.backward_size;
if ((lzma_vli)(pos) < index_size + LZMA_STREAM_HEADER_SIZE) {
@@ -429,7 +443,19 @@ parse_block_header(file_pair *pair, const lzma_index_iter *iter,
switch (lzma_block_compressed_size(&block,
iter->block.unpadded_size)) {
case LZMA_OK:
- break;
+ // Validate also block.uncompressed_size if it is present.
+ // If it isn't present, there's no need to set it since
+ // we aren't going to actually decompress the Block; if
+ // we were decompressing, then we should set it so that
+ // the Block decoder could validate the Uncompressed Size
+ // that was stored in the Index.
+ if (block.uncompressed_size == LZMA_VLI_UNKNOWN
+ || block.uncompressed_size
+ == iter->block.uncompressed_size)
+ break;
+
+ // If the above fails, the file is corrupt so
+ // LZMA_DATA_ERROR is a good error code.
case LZMA_DATA_ERROR:
// Free the memory allocated by lzma_block_header_decode().
diff --git a/src/xz/util.c b/src/xz/util.c
index 987b44302539..35850f4c9046 100644
--- a/src/xz/util.c
+++ b/src/xz/util.c
@@ -26,9 +26,19 @@ xrealloc(void *ptr, size_t size)
{
assert(size > 0);
+ // Save ptr so that we can free it if realloc fails.
+ // The point is that message_fatal ends up calling stdio functions
+ // which in some libc implementations might allocate memory from
+ // the heap. Freeing ptr improves the chances that there's free
+ // memory for stdio functions if they need it.
+ void *p = ptr;
ptr = realloc(ptr, size);
- if (ptr == NULL)
- message_fatal("%s", strerror(errno));
+
+ if (ptr == NULL) {
+ const int saved_errno = errno;
+ free(p);
+ message_fatal("%s", strerror(saved_errno));
+ }
return ptr;
}
diff --git a/src/xz/xz.1 b/src/xz/xz.1
index 8edc456c241f..363b90cb4e55 100644
--- a/src/xz/xz.1
+++ b/src/xz/xz.1
@@ -5,16 +5,17 @@
.\" This file has been put into the public domain.
.\" You can do whatever you want with this file.
.\"
-.TH XZ 1 "2012-05-27" "Tukaani" "XZ Utils"
+.TH XZ 1 "2013-06-21" "Tukaani" "XZ Utils"
.
.SH NAME
xz, unxz, xzcat, lzma, unlzma, lzcat \- Compress or decompress .xz and .lzma files
.
.SH SYNOPSIS
.B xz
-.RI [ option ]...
-.RI [ file ]...
-.PP
+.RI [ option... ]
+.RI [ file... ]
+.
+.SH COMMAND ALIASES
.B unxz
is equivalent to
.BR "xz \-\-decompress" .
@@ -708,7 +709,7 @@ be dramatically higher than that of the single-threaded mode.
DecMem contains the decompressor memory requirements.
That is, the compression settings determine
the memory requirements of the decompressor.
-The exact decompressor memory usage is slighly more than
+The exact decompressor memory usage is slightly more than
the LZMA2 dictionary size, but the values in the table
have been rounded up to the next full MiB.
.RE
@@ -897,11 +898,14 @@ if threading will be enabled by default.
.SS "Custom compressor filter chains"
A custom filter chain allows specifying
the compression settings in detail instead of relying on
-the settings associated to the preset levels.
+the settings associated to the presets.
When a custom filter chain is specified,
-the compression preset level options
-(\fB\-0\fR ... \fB\-9\fR and \fB\-\-extreme\fR) are
-silently ignored.
+preset options (\fB\-0\fR ... \fB\-9\fR and \fB\-\-extreme\fR)
+earlier on the command line are forgotten.
+If a preset option is specified
+after one or more custom filter chain options,
+the new preset takes effect and
+the custom filter chain options specified earlier are forgotten.
.PP
A filter chain is comparable to piping on the command line.
When compressing, the uncompressed input goes to the first filter,
@@ -934,6 +938,15 @@ Extra commas in
are ignored.
Every option has a default value, so you need to
specify only those you want to change.
+.PP
+To see the whole filter chain and
+.IR options ,
+use
+.B "xz \-vv"
+(that is, use
+.B \-\-verbose
+twice).
+This works also for viewing the filter chain options used by presets.
.TP
\fB\-\-lzma1\fR[\fB=\fIoptions\fR]
.PD 0
@@ -976,13 +989,12 @@ The only supported modifier is currently
.BR e ,
which matches
.BR \-\-extreme .
-The default
-.I preset
-is
-.BR 6 ,
-from which the default values for the rest of the LZMA1 or LZMA2
+If no
+.B preset
+is specified, the default values of LZMA1 or LZMA2
.I options
-are taken.
+are taken from the preset
+.BR 6 .
.TP
.BI dict= size
Dictionary (history buffer)
@@ -1578,7 +1590,7 @@ is supported only together with
.BR \-\-info\-memory ,
and
.BR \-\-list .
-It will be supported for normal compression and
+It will be supported for compression and
decompression in the future.
.
.SS Version
@@ -2179,14 +2191,15 @@ The output can vary even between different
builds of the same XZ Utils version,
if different build options are used.
.PP
-The above means that implementing
+The above means that once
.B \-\-rsyncable
-to create rsyncable
-.B .xz
-files is not going to happen without
-freezing a part of the encoder
-implementation, which can then be used with
-.BR \-\-rsyncable .
+has been implemented,
+the resulting files won't necessarily be rsyncable
+unless both old and new files have been compressed
+with the same xz version.
+This problem can be fixed if a part of the encoder
+implementation is frozen to keep rsyncable output
+stable across xz versions.
.
.SS "Embedded .xz decompressors"
Embedded
diff --git a/src/xzdec/xzdec.1 b/src/xzdec/xzdec.1
index 7cc9be5de086..1e5ced945aae 100644
--- a/src/xzdec/xzdec.1
+++ b/src/xzdec/xzdec.1
@@ -4,17 +4,17 @@
.\" This file has been put into the public domain.
.\" You can do whatever you want with this file.
.\"
-.TH XZDEC 1 "2010-09-27" "Tukaani" "XZ Utils"
+.TH XZDEC 1 "2013-06-30" "Tukaani" "XZ Utils"
.SH NAME
xzdec, lzmadec \- Small .xz and .lzma decompressors
.SH SYNOPSIS
.B xzdec
-.RI [ option ]...
-.RI [ file ]...
+.RI [ option... ]
+.RI [ file... ]
.br
.B lzmadec
-.RI [ option ]...
-.RI [ file ]...
+.RI [ option... ]
+.RI [ file... ]
.SH DESCRIPTION
.B xzdec
is a liblzma-based decompression-only tool for
diff --git a/src/xzdec/xzdec.c b/src/xzdec/xzdec.c
index b7830db4b523..5cb7530afce4 100644
--- a/src/xzdec/xzdec.c
+++ b/src/xzdec/xzdec.c
@@ -62,13 +62,13 @@ help(void)
{
printf(
"Usage: %s [OPTION]... [FILE]...\n"
-"Uncompress files in the ." TOOL_FORMAT " format to the standard output.\n"
+"Decompress files in the ." TOOL_FORMAT " format to standard output.\n"
"\n"
-" -c, --stdout (ignored)\n"
-" -d, --decompress (ignored)\n"
-" -k, --keep (ignored)\n"
+" -d, --decompress (ignored, only decompression is supported)\n"
+" -k, --keep (ignored, files are never deleted)\n"
+" -c, --stdout (ignored, output is always written to standard output)\n"
" -q, --quiet specify *twice* to suppress errors\n"
-" -Q, --no-warn (ignored)\n"
+" -Q, --no-warn (ignored, the exit status 2 is never used)\n"
" -h, --help display this help and exit\n"
" -V, --version display the version number and exit\n"
"\n"