Re: [vox-tech] Decompressing large files

[Richard Cook <rcooMAPSk@llnl.gov>]

Decompress creates intermediate data representations as part of its
decompression, which may be much greater in size than the corresponding
decompressed data.  Perhaps this intermediate data is tacked on to the
end of files it decompresses, thus potentially violating a file size
limit, but if you use -c it creates a separate temp file, which alone is
not enough to exceed the limit?   Just a guess. 

Micah Cowan wrote:
> 
> On Mon, Oct 30, 2000 at 02:18:20PM -0800, Eric Engelhard wrote:
> > I am decompressing large (>1GB compressed) genome sequence files. I have
> > plenty of free space on a fast scsi and 256MB of RAM. When I tried to to
> > use "uncompress filename.Z" or "gzip -d filename.Z" I got a write error
> > each time. The solution (well, MY solution) was to use the "-c" option
> > with uncompress and pipe the output to a specified file. Does anyone
> > know enough about uncompress to tell me why this solves the problem?
> > --
> > Eric
> 
> I don't, but offhand I'd say that uncompress doesn't handle writes as
> stablely (if that's a word :) ) as bash does.  Which makes sense to
> me, since uncompress is known to not verify compressed input files,
> according to the gzip (i think) manpage.  After all, isn't the only
> difference between uncompress filename.Z and uncompress -c, etc. the
> fact that uncompress is/isn't handling the actual disk io?
> 
> Micah

-- 
opinions expressed here are not those of my employer!  
email: rcook@llnl.gov          Rich Cook, Mail Stop L-551 Bldg-3577
Rm-1000
http://www.llnl.gov/graphics   Lawrence Livermore National Laboratory
phone (925) 423-9605 (work)    7000 East Avenue
fax (925) 423-9338             Livermore, CA 94550, USA