Repository: libarchive/libarchive
Pull request #2539: FILE* seeking support
Author: login: ljdarj
Created: 2025-03-14T19:18:16Z | Merged: 2025-03-15T18:17:28Z

Adding a seeker function to archive_read_open_FILE().

Fixes #437.

Linked issue #437: Cannot read 7zip, etc, with archive_read_open_FILE
Original [issue 328](https://code.google.com/p/libarchive/issues/detail?id=328) created by Google Code user `moonstar0825` on 2013-07-23T05:21:12.000Z:

```
<b>What steps will reproduce the problem?</b>

1. unzip more.zip (more.com, more.7z, more_7zsfx.exe, more_zipsfx.exe )
2. gcc 1.c -larchive
3. ./a.out more.7z

<b>What is the expected output? What do you see instead?</b>

    more.7z and more_7zsfx.exe
        use archive_read_open_filename, the extract file is correct
        use archive_read_open_FILE, the extract file size is 0

    more_zipsfx.exe
        use archive_read_open_filename, the extract file is correct
        use archive_read_open_FILE, output &quot;not supported&quot;

<b>What version are you using?</b>

    libarchive-3.1.2

<b>On what operating system?</b>

    ubuntu 13.04 x64

<b>How did you build?  (cmake, configure, or pre-packaged binary)</b>

    cmake

<b>What compiler or development environment (please include version)?</b>

    gcc version 4.7.3

<b>Please provide any additional information below.</b>

    #include &lt;stdio.h&gt;
    #include &lt;stdlib.h&gt;

    #include &lt;archive.h&gt;
    #include &lt;archive_entry.h&gt;

    int main(int argc, char* argv[])
    {
        struct archive *a;
        a = archive_read_new();

        archive_read_support_filter_all(a);
        archive_read_support_format_all(a);

        int r;
        //r = archive_read_open_filename(a, argv[1], 10240);

        FILE *in = fopen(argv[1], &quot;rb&quot;);
        r = archive_read_open_FILE(a, in);

        if (r != ARCHIVE_OK)
        {
            printf(&quot;not supported\n&quot;);
            return 1;
        }

        int size;
        char buf[1024];

        FILE *out = fopen(&quot;tmpfile&quot;, &quot;wb+&quot;);

        struct archive_entry *entry;

        while(archive_read_next_header(a, &amp;entry) == ARCHIVE_OK) 
        {
            while((size = archive_read_data(a, buf, 1024)) &gt; 0)
                fwrite(buf, sizeof(char), size, out);

            if(archive_errno(a) != ARCHIVE_OK)
                printf(&quot;%s\n&quot;, archive_error_string(a));

            archive_read_data_skip(a);
        }

        fclose(out);
        fclose(in);

        archive_read_free(a);
        return 0;
    }

```

See attachment: [more.zip](http://libarchive.github.io/google-code/issue-328/comment-0/more.zip)

Linked issue comments: user: login: kwrobot
created_at: 2015-04-11T17:10:04Z
body: Comment [#1](https://code.google.com/p/libarchive/issues/detail?id=328#c1) originally posted by **kientzle** on 2013-07-23T05:38:37.000Z:

```
Currently, archive_read_open_FILE does not support seeking.  archive_read_open_filename does.

Self-extracting Zip archives, 7z, and some other formats cannot be read without seeking.

Patches to add seek support for archive_read_open_FILE would be appreciated.

```

author_association: NONE
user: login: kwrobot
created_at: 2015-04-11T17:10:06Z
body: Comment [#2](https://code.google.com/p/libarchive/issues/detail?id=328#c2) originally posted by **kientzle** on 2013-11-24T17:16:50.000Z:

```
Updating bug title to more accurately reflect the actual issue.


```

author_association: NONE

Diff:
diff --git a/libarchive/archive_read_open_file.c b/libarchive/archive_read_open_file.c
index cf49ebd8fc..ecd56dce61 100644
--- a/libarchive/archive_read_open_file.c
+++ b/libarchive/archive_read_open_file.c
@@ -56,9 +56,10 @@ struct read_FILE_data {
 	char	 can_skip;
 };
 
-static int	file_close(struct archive *, void *);
-static ssize_t	file_read(struct archive *, void *, const void **buff);
-static int64_t	file_skip(struct archive *, void *, int64_t request);
+static int	FILE_close(struct archive *, void *);
+static ssize_t	FILE_read(struct archive *, void *, const void **buff);
+static int64_t	FILE_seek(struct archive *, void *, int64_t, int);
+static int64_t	FILE_skip(struct archive *, void *, int64_t);
 
 int
 archive_read_open_FILE(struct archive *a, FILE *f)
@@ -69,7 +70,7 @@ archive_read_open_FILE(struct archive *a, FILE *f)
 	void *b;
 
 	archive_clear_error(a);
-	mine = malloc(sizeof(*mine));
+	mine = calloc(1, sizeof(*mine));
 	b = malloc(block_size);
 	if (mine == NULL || b == NULL) {
 		archive_set_error(a, ENOMEM, "No memory");
@@ -90,22 +91,22 @@ archive_read_open_FILE(struct archive *a, FILE *f)
 		archive_read_extract_set_skip_file(a, st.st_dev, st.st_ino);
 		/* Enable the seek optimization only for regular files. */
 		mine->can_skip = 1;
-	} else
-		mine->can_skip = 0;
+	}
 
 #if defined(__CYGWIN__) || defined(_WIN32)
 	setmode(fileno(mine->f), O_BINARY);
 #endif
 
-	archive_read_set_read_callback(a, file_read);
-	archive_read_set_skip_callback(a, file_skip);
-	archive_read_set_close_callback(a, file_close);
+	archive_read_set_read_callback(a, FILE_read);
+	archive_read_set_skip_callback(a, FILE_skip);
+	archive_read_set_seek_callback(a, FILE_seek);
+	archive_read_set_close_callback(a, FILE_close);
 	archive_read_set_callback_data(a, mine);
 	return (archive_read_open1(a));
 }
 
 static ssize_t
-file_read(struct archive *a, void *client_data, const void **buff)
+FILE_read(struct archive *a, void *client_data, const void **buff)
 {
 	struct read_FILE_data *mine = (struct read_FILE_data *)client_data;
 	size_t bytes_read;
@@ -119,13 +120,13 @@ file_read(struct archive *a, void *client_data, const void **buff)
 }
 
 static int64_t
-file_skip(struct archive *a, void *client_data, int64_t request)
+FILE_skip(struct archive *a, void *client_data, int64_t request)
 {
 	struct read_FILE_data *mine = (struct read_FILE_data *)client_data;
-#if HAVE_FSEEKO
-	off_t skip = (off_t)request;
-#elif HAVE__FSEEKI64
+#if HAVE__FSEEKI64
 	int64_t skip = request;
+#elif HAVE_FSEEKO
+	off_t skip = (off_t)request;
 #else
 	long skip = (long)request;
 #endif
@@ -167,8 +168,57 @@ file_skip(struct archive *a, void *client_data, int64_t request)
 	return (request);
 }
 
+/*
+ * TODO: Store the offset and use it in the read callback.
+ */
+static int64_t
+FILE_seek(struct archive *a, void *client_data, int64_t request, int whence)
+{
+	struct read_FILE_data *mine = (struct read_FILE_data *)client_data;
+#if HAVE__FSEEKI64
+	int64_t skip = request;
+#elif HAVE_FSEEKO
+	off_t skip = (off_t)request;
+#else
+	long skip = (long)request;
+#endif
+	int skip_bits = sizeof(skip) * 8 - 1;
+	(void)a; /* UNUSED */
+
+	/* If request is too big for a long or an off_t, reduce it. */
+	if (sizeof(request) > sizeof(skip)) {
+		int64_t max_skip =
+		    (((int64_t)1 << (skip_bits - 1)) - 1) * 2 + 1;
+		if (request > max_skip)
+			skip = max_skip;
+	}
+
+#ifdef __ANDROID__
+	/* Newer Android versions have fseeko...to meditate. */
+	int64_t ret = lseek(fileno(mine->f), skip, whence);
+	if (ret >= 0) {
+		return ret;
+	}
+#elif HAVE__FSEEKI64
+	if (_fseeki64(mine->f, skip, whence) == 0) {
+		return _ftelli64(mine->f);
+	}
+#elif HAVE_FSEEKO
+	if (fseeko(mine->f, skip, whence) == 0) {
+		return ftello(mine->f);
+	}
+#else
+	if (fseek(mine->f, skip, whence) == 0) {
+		return ftell(mine->f);
+	}
+#endif
+	/* If we arrive here, the input is corrupted or truncated so fail. */
+	archive_set_error(a, errno, "Error seeking in FILE* pointer");
+	return (ARCHIVE_FATAL);
+}
+
 static int
-file_close(struct archive *a, void *client_data)
+FILE_close(struct archive *a, void *client_data)
 {
 	struct read_FILE_data *mine = (struct read_FILE_data *)client_data;
 
@@ -176,4 +226,4 @@ file_close(struct archive *a, void *client_data)
 	free(mine->buffer);
 	free(mine);
 	return (ARCHIVE_OK);
-}
+}
\ No newline at end of file
diff --git a/libarchive/test/test_open_file.c b/libarchive/test/test_open_file.c
index f4ca82bb01..cc6b04d098 100644
--- a/libarchive/test/test_open_file.c
+++ b/libarchive/test/test_open_file.c
@@ -31,14 +31,14 @@ DEFINE_TEST(test_open_file)
 	struct archive *a;
 	FILE *f;
 
-	f = fopen("test.tar", "wb");
+	f = fopen("test.7z", "wb");
 	assert(f != NULL);
 	if (f == NULL)
 		return;
 
 	/* Write an archive through this FILE *. */
 	assert((a = archive_write_new()) != NULL);
-	assertEqualIntA(a, ARCHIVE_OK, archive_write_set_format_ustar(a));
+	assertEqualIntA(a, ARCHIVE_OK, archive_write_set_format_7zip(a));
 	assertEqualIntA(a, ARCHIVE_OK, archive_write_add_filter_none(a));
 	assertEqualIntA(a, ARCHIVE_OK, archive_write_open_FILE(a, f));
 
@@ -70,9 +70,10 @@ DEFINE_TEST(test_open_file)
 	fclose(f);
 
 	/*
-	 * Now, read the data back.
+	 * Now, read the data back. 7z requiring seeking, that also
+	 * tests that the seeking support works.
 	 */
-	f = fopen("test.tar", "rb");
+	f = fopen("test.7z", "rb");
 	assert(f != NULL);
 	if (f == NULL)
 		return;


Reviews:
user: login: github-advanced-security[bot]
state: COMMENTED
author_association: NONE
submitted_at: 2025-03-14T19:25:34Z
user: login: kientzle
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T23:42:44Z
user: login: kientzle
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T23:43:48Z
user: login: kientzle
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T23:47:18Z
user: login: kientzle
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T23:47:55Z
user: login: ljdarj
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-15T00:48:24Z
user: login: ljdarj
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-15T01:03:15Z
user: login: kientzle
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-15T18:16:42Z

Review comments:
diff_hunk: @@ -31,14 +31,14 @@
 	struct archive *a;
 	FILE *f;
 
-	f = fopen("test.tar", "wb");
+	f = fopen("test.7z", "wb");
path: libarchive/test/test_open_file.c
user: login: github-advanced-security[bot]
body: ## File created without restricting permissions

A file may be created here with mode 0666, which would make it world-writable.

[Show more details](https://github.com/libarchive/libarchive/security/code-scanning/366)
created_at: 2025-03-14T19:25:34Z
author_association: NONE
diff_hunk: @@ -69,7 +70,7 @@ archive_read_open_FILE(struct archive *a, FILE *f)
 	void *b;
 
 	archive_clear_error(a);
-	mine = malloc(sizeof(*mine));
+	mine = calloc(1, sizeof(*mine));
path: libarchive/archive_read_open_file.c
user: login: kientzle
body: Thank you for this change.
created_at: 2025-03-14T23:42:44Z
author_association: CONTRIBUTOR
diff_hunk: @@ -167,8 +168,60 @@ file_skip(struct archive *a, void *client_data, int64_t request)
 	return (request);
 }
 
+/*
+ * TODO: Store the offset and use it in the read callback.
+ */
+static int64_t
+FILE_seek(struct archive *a, void *client_data, int64_t request, int whence)
+{
+	struct read_FILE_data *mine = (struct read_FILE_data *)client_data;
+#if HAVE__FSEEKI64
+	int64_t skip = request;
+#elif HAVE_FSEEKO
+	off_t skip = (off_t)request;
+#else
+	long skip = (long)request;
+#endif
+	int skip_bits = sizeof(skip) * 8 - 1;
+	(void)a; /* UNUSED */
+
+	/* If request is too big for a long or an off_t, reduce it. */
+	if (sizeof(request) > sizeof(skip)) {
+		int64_t max_skip =
+		    (((int64_t)1 << (skip_bits - 1)) - 1) * 2 + 1;
+		if (request > max_skip)
+			skip = max_skip;
+	}
+
+#ifdef __ANDROID__
+	/* Newer Android versions have fseeko...to meditate. */
+	int64_t ret = lseek(fileno(mine->f), skip, whence);
+	if (r >= 0) {
path: libarchive/archive_read_open_file.c
user: login: kientzle
body: Did you mean to use `ret >= 0` here?
created_at: 2025-03-14T23:43:47Z
author_association: CONTRIBUTOR
diff_hunk: @@ -167,8 +168,60 @@ file_skip(struct archive *a, void *client_data, int64_t request)
 	return (request);
 }
 
+/*
+ * TODO: Store the offset and use it in the read callback.
+ */
+static int64_t
+FILE_seek(struct archive *a, void *client_data, int64_t request, int whence)
+{
+	struct read_FILE_data *mine = (struct read_FILE_data *)client_data;
+#if HAVE__FSEEKI64
+	int64_t skip = request;
+#elif HAVE_FSEEKO
+	off_t skip = (off_t)request;
+#else
+	long skip = (long)request;
+#endif
+	int skip_bits = sizeof(skip) * 8 - 1;
+	(void)a; /* UNUSED */
+
+	/* If request is too big for a long or an off_t, reduce it. */
+	if (sizeof(request) > sizeof(skip)) {
+		int64_t max_skip =
+		    (((int64_t)1 << (skip_bits - 1)) - 1) * 2 + 1;
+		if (request > max_skip)
+			skip = max_skip;
+	}
+
+#ifdef __ANDROID__
+	/* Newer Android versions have fseeko...to meditate. */
+	int64_t ret = lseek(fileno(mine->f), skip, whence);
+	if (r >= 0) {
+#elif HAVE__FSEEKI64
+	if (_fseeki64(mine->f, skip, whence) == 0) {
+#elif HAVE_FSEEKO
+	if (fseeko(mine->f, skip, whence) == 0) {
+#else
+	if (fseek(mine->f, skip, whence) == 0) {
+#endif
+#ifdef __ANDROID__
path: libarchive/archive_read_open_file.c
user: login: kientzle
body: This is oddly structured.  I would
```
#ifdef __ANDROID__
  int64_t ret = lseek(...);
  if (ret >= 0) {
    return ret;
  }
#elif HAVE__FSEEKI64
  if (_fseeki64(...) == 0) {
      return _ftelli64(...);
  }
#elif HAVE_FSEEKO
  ... etc ...
#else
   ... etc ...
#endif
  /* If we didn't return above, then input must be corrupted or truncated */
  archive_set_error(...);
  return (ARCHIVE_FATAL);
```
created_at: 2025-03-14T23:47:18Z
author_association: CONTRIBUTOR
diff_hunk: @@ -31,14 +31,14 @@ DEFINE_TEST(test_open_file)
 	struct archive *a;
 	FILE *f;
 
-	f = fopen("test.tar", "wb");
+	f = fopen("test.7z", "wb");
path: libarchive/test/test_open_file.c
user: login: kientzle
body: Clever!  ;-) 
created_at: 2025-03-14T23:47:55Z
author_association: CONTRIBUTOR
diff_hunk: @@ -167,8 +168,60 @@ file_skip(struct archive *a, void *client_data, int64_t request)
 	return (request);
 }
 
+/*
+ * TODO: Store the offset and use it in the read callback.
+ */
+static int64_t
+FILE_seek(struct archive *a, void *client_data, int64_t request, int whence)
+{
+	struct read_FILE_data *mine = (struct read_FILE_data *)client_data;
+#if HAVE__FSEEKI64
+	int64_t skip = request;
+#elif HAVE_FSEEKO
+	off_t skip = (off_t)request;
+#else
+	long skip = (long)request;
+#endif
+	int skip_bits = sizeof(skip) * 8 - 1;
+	(void)a; /* UNUSED */
+
+	/* If request is too big for a long or an off_t, reduce it. */
+	if (sizeof(request) > sizeof(skip)) {
+		int64_t max_skip =
+		    (((int64_t)1 << (skip_bits - 1)) - 1) * 2 + 1;
+		if (request > max_skip)
+			skip = max_skip;
+	}
+
+#ifdef __ANDROID__
+	/* Newer Android versions have fseeko...to meditate. */
+	int64_t ret = lseek(fileno(mine->f), skip, whence);
+	if (r >= 0) {
path: libarchive/archive_read_open_file.c
user: login: ljdarj
body: Indeed I did, thanks for the catch.
created_at: 2025-03-15T00:48:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -167,8 +168,60 @@ file_skip(struct archive *a, void *client_data, int64_t request)
 	return (request);
 }
 
+/*
+ * TODO: Store the offset and use it in the read callback.
+ */
+static int64_t
+FILE_seek(struct archive *a, void *client_data, int64_t request, int whence)
+{
+	struct read_FILE_data *mine = (struct read_FILE_data *)client_data;
+#if HAVE__FSEEKI64
+	int64_t skip = request;
+#elif HAVE_FSEEKO
+	off_t skip = (off_t)request;
+#else
+	long skip = (long)request;
+#endif
+	int skip_bits = sizeof(skip) * 8 - 1;
+	(void)a; /* UNUSED */
+
+	/* If request is too big for a long or an off_t, reduce it. */
+	if (sizeof(request) > sizeof(skip)) {
+		int64_t max_skip =
+		    (((int64_t)1 << (skip_bits - 1)) - 1) * 2 + 1;
+		if (request > max_skip)
+			skip = max_skip;
+	}
+
+#ifdef __ANDROID__
+	/* Newer Android versions have fseeko...to meditate. */
+	int64_t ret = lseek(fileno(mine->f), skip, whence);
+	if (r >= 0) {
+#elif HAVE__FSEEKI64
+	if (_fseeki64(mine->f, skip, whence) == 0) {
+#elif HAVE_FSEEKO
+	if (fseeko(mine->f, skip, whence) == 0) {
+#else
+	if (fseek(mine->f, skip, whence) == 0) {
+#endif
+#ifdef __ANDROID__
path: libarchive/archive_read_open_file.c
user: login: ljdarj
body: I changed the structure, it looks indeed better.
created_at: 2025-03-15T01:03:15Z
author_association: CONTRIBUTOR

PR comments:
user: login: ljdarj
created_at: 2025-03-15T01:32:39Z
body: I have a question though regarding a note I saw in `archive_read_open_fd.c`: when in the TODO note it's talked of storing the offset to use in the read callback, would that be the idea I saw in `archive_read_open_filename.c` of deferring the seeks until the next read call to do them all at once?

Also, why are we using `lseek()` on Android? Aren't they ILP32 platforms, in which case we gain nothing on the less sane NDK versions because we use a `long` offset either way and don't get to use `fseeko()` on the saner NDKs?
author_association: CONTRIBUTOR
user: login: kientzle
created_at: 2025-03-15T18:17:22Z
body: I haven't worked with Android in a long time, so I don't know what the current state of affairs is over there.  I agree the special case here is a little odd.

Deferring seeks probably wouldn't help much:  seek is handled in the kernel just by adjusting the offset used for the next read, no actual work gets done.  Deferring seeks could help reduce system calls in cases where libarchive actually issues multiple seeks before the next read, but that doesn't happen very often.

I'll merge this as-is.  We can discuss additional improvements in separate PRs.
author_association: CONTRIBUTOR