what's up with all the seeking?

[Jeff Breidenbach]

Is TIFF fundamentally incompatible with streaming?

===

#include <tiffio.h>
#include <stdio.h>

main() {
TIFF *tiff = TIFFFdOpen(fileno(stdin), "stdin", "rb");
TIFFClose(tiff);
}

===

$ gcc -o mypgram mypgram.c -ltiff
$ pbmmake 1000 1000 | pnmtotiff -g4 > foo tif
$ myprogram < foo.tif

TIFFFetchDirectory: stdin: Seek error accessing TIFF directory.
TIFFReadDirectory: Failed to read directory at offset 40.
Segmentation fault (core dumped)

[Scott Ribe]

Post by Jeff Breidenbach
Is TIFF fundamentally incompatible with streaming?

It's a random access format.
--
Scott Ribe
***@elevated-dev.com
http://www.elevated-dev.com/
[phone] voice




_______________________________________________
Tiff mailing list: ***@lists.maptools.org
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

[Steve Underwood]

Hi Jeff,

Yes, TIFF is fundamentally incompatible with streaming. There are some
TIFF related specs, such as the earliest RFCs on TIFF for internet FAX,
which try to impose rules which would allow TIFF to be effectively
streamed. In practice those specs failed.

The main issue is you can only interpret the contents of the file by
following the directories, and the directories can be anywhere. In most
TIFF files you will find the directory for a page right after the image
data for that page. This makes sense from a sequential file generation
point of view, as typically you don't fully know what the directory
should contain until you have generated the entire image. It does,
however, mean that on reading the TIFF file you need to seek beyond the
image to get to the directory which tells you the extent of the image
data, and how to interpret its contents.

Regards,
Steve

Post by Jeff Breidenbach
Is TIFF fundamentally incompatible with streaming?
===
#include <tiffio.h>
#include <stdio.h>
main() {
TIFF *tiff = TIFFFdOpen(fileno(stdin), "stdin", "rb");
TIFFClose(tiff);
}
===
$ gcc -o mypgram mypgram.c -ltiff
$ pbmmake 1000 1000 | pnmtotiff -g4 > foo tif
$ myprogram < foo.tif
TIFFFetchDirectory: stdin: Seek error accessing TIFF directory.
TIFFReadDirectory: Failed to read directory at offset 40.
Segmentation fault (core dumped)
_______________________________________________
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

_______________________________________________
Tiff mailing list: ***@lists.maptools.org
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

[Bob Friesenhahn]

Post by Jeff Breidenbach
Is TIFF fundamentally incompatible with streaming?

In order for TIFF to be "streamed" you would need to implement an I/O
layer for libtiff to use and the I/O layer would need to add any
necessary buffering so that it buffers all data up to any requested
seek or read offset. It might be necessary to buffer the entire image
data.

A simple workaround is to read the input into a temporary file.

Bob

[Even Rouault]

Post by Bob Friesenhahn

Post by Jeff Breidenbach
Is TIFF fundamentally incompatible with streaming?

In order for TIFF to be "streamed" you would need to implement an I/O
layer for libtiff to use and the I/O layer would need to add any
necessary buffering so that it buffers all data up to any requested
seek or read offset. It might be necessary to buffer the entire image
data.
A simple workaround is to read the input into a temporary file.

I agree with others that, generally, the TIFF format isn't not compatible with
streaming.

That said, if you have full control on how libtiff is used (and if you master
its behaviour) to produce the TIFF and how it will be read, it *might* be
streamable.

For example I've used GDAL to produced a tiled uncompressed TIFF (tile
dimension:256x256), 3 bands, contiguous planar config : out.tif
If I use GDAL translate to read it and produce a TIFF with exact configuration,
the read-pattern used by the GDAL I/O will be compatible with streaming :

$ cat out.tif | gdal_translate /vsistdin/ out2.tif -co tiled=yes
Input file size is 21600, 21600
0...10...20...30...40...50...60...70...80...90...100 - done.

But that's really a particular case.

I didn't manage to produce for example a JPEG-compressed TIFF that has that
priority. For some reason (probably due to the writing of the JPEGTABLES), the
TIFF directory gets rewritten at the end of the file instead of being at the
beginning. And you loose streaming capability.

Even

[Toby Thain]

Post by Bob Friesenhahn

Post by Jeff Breidenbach
Is TIFF fundamentally incompatible with streaming?

In order for TIFF to be "streamed" you would need to implement an I/O
layer for libtiff to use and the I/O layer would need to add any
necessary buffering so that it buffers all data up to any requested
seek or read offset. It might be necessary to buffer the entire image
data.
A simple workaround is to read the input into a temporary file.

Or mmap()?

--Toby

Post by Bob Friesenhahn
Bob

_______________________________________________
Tiff mailing list: ***@lists.maptools.org
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

[Bob Friesenhahn]

Post by Toby Thain

Post by Bob Friesenhahn

Post by Jeff Breidenbach
Is TIFF fundamentally incompatible with streaming?

In order for TIFF to be "streamed" you would need to implement an I/O
layer for libtiff to use and the I/O layer would need to add any
necessary buffering so that it buffers all data up to any requested
seek or read offset. It might be necessary to buffer the entire image
data.
A simple workaround is to read the input into a temporary file.

Or mmap()?

Mmap() does not help for input via a pipe or socket. It can be used
to map the temporary file (which libtiff does by default).

Bob

[Prophet of the Way]

Jeff Breidenbach wrote:

Is TIFF fundamentally incompatible with streaming?

$ pbmmake 1000 1000 | pnmtotiff -g4 > foo.tif
$ myprogram < foo.tif
TIFFFetchDirectory: stdin: Seek error accessing TIFF directory.
TIFFReadDirectory: Failed to read directory at offset 40.
Segmentation fault (core dumped)

Thank you for bringing up this subject.

I'm working on pamtotiff and tifftopnm. Currently, these programs
use TIFF library functions directly for i/o. Newer versions of
pamtotiff (pnmtotiff) and tifftopnm will be able to read from and
write to pipes.

The code for pamtotiff has been tested and will be available
in the "advanced" version of Neptbm, v. 10.67 scheduled for
release in June 2014.

Note that Debian's Netpbm is a fork that is not following upstream
developments. The above-mentioned functionality will not be available
in Debian and Ubuntu packages unless someone steps up and changes this
situation. The Fedora Netpbm package maintainer favors recent upstream
source releases.


_______________________________________________
Tiff mailing list: ***@lists.maptools.org
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/