[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: file size limit exceeded



Blake and all,

I have reproduced the file size exceeded error reported by Blake on hddm 
and other output files produced by MC and simulation tools.  It seems 
that we have neglected to add the obligatory switch 
-D_FILE_OFFSET_BITS=64 to all of our build procedures.  That switch 
works alike for gcc, g++, and g77/gfortran compilers to instruct them to 
call 64-bit versions of the clib routines for file i/o, which bypass the 
traditional (I would say obsolete) limitation on file sizes to 2GB.  
Thanks to Blake for pointing this out.  I have added the fix to the 
Makefile.orig files in the gxtwist, hdgeant, and genr8 directories in 
subversion.  Dave, could you propagate this change to the BMS build 
system, if it is not already there?  It might be a good default for all 
of our packages, including JANA, if it is not already.

-Richard Jones


Blake Leverington wrote:
> Hi Eugene,
>
> It may just be an ext3 limit on the file size but my files are limited 
> to 2GB, even the hddm files. My work around so far is to just make 
> multiple runs like you suggested where I have a script that changes the 
> random number seed, but I know ROOT has a feature which starts a new 
> file when you exceed 2GB. I was curious if this was/could be implemented 
> here.
>
> -Blake
>
> Eugene Chudakov wrote:
>   
>> Blake,
>>
>> a comment concerning bggen:
>>
>> - it can optionally write out an hddm file (I suppose the size limit 
>> is >=2 GB)
>>   and an NTUPLE file (the practical limit is about 200MB).
>>
>> - one can split the simulation into many runs changing the line
>>   RNDSEQ  N
>>   in the input file, where N can be 0,1,2 etc. This number defines
>>   the random number sequence.
>>
>> Eugene
>>
>>
>> On Tue, 2 Jun 2009, Blake Leverington wrote:
>>
>>     
>>> Hi David,
>>>
>>> How do I get around file size limits for HDGeant, bggen, etc? I can 
>>> only guess at the number of events to put in a data set and split up 
>>> the jobs, but I don't always know how large the file size is coming 
>>> out. Is there a way of making it start a second output file?
>>>
>>> -Blake
>>>
>>>       

S/MIME Cryptographic Signature