Re: [vox-tech] reading a .gz .Z after offset

[Jeff Newmiller <jdMAPSnewmil@dcn.davis.ca.us>]

On Wed, 6 Mar 2002, Eric Engelhard wrote:

> Does anyone on the list have experience using perl read
> (FILEHANDLE,SCALAR,LENGTH,OFFSET) to drop into compressed files (.Z and
> .gz) and extract readable results? I am ignorant of both compressed file
> anatomy and/or perl mods which can handle this issue. 
> I currently process large reference files (up to 1.8GB compressed)
> through a decompression/recompression pipe, but would like to divide and
> conquer on parallel cluster nodes without having to read through the
> entire file at each node.

I don't think you can do seeks in a compressed file... you have to read it
sequentially.

If you have a plan for dividing up the uncompressed data, perhaps you
should do that first and store the split data as separate files
(recompressed or not) for purposes of computation.

---------------------------------------------------------------------------
Jeff Newmiller                        The     .....       .....  Go Live...
DCN:<jdnewmil@dcn.davis.ca.us>        Basics: ##.#.       ##.#.  Live Go...
                                      Live:   OO#.. Dead: OO#..  Playing
Research Engineer (Solar/Batteries            O.O#.       #.O#.  with
/Software/Embedded Controllers)               .OO#.       .OO#.  rocks...2k
---------------------------------------------------------------------------

_______________________________________________
vox-tech mailing list
vox-tech@lists.lugod.org
http://lists.lugod.org/mailman/listinfo/vox-tech