Windows Kernel-- deferred cleanup?

we've been having series of issues @ customer sites appear due windows kernel deferring cleanup of open files causing subsequent file contention problems.  in if not versions of windows, xp thru ws2008r2.

the scenario (example in batch file)

:processloop

program.exe >out.txt 2>out2.txt

goto :processloop

normally works fine, when system heavily loaded, intermittent permissions errors on output files.   , since we're seeing in data warehouse load situation we're parallelizing these processes, system under heavy load.

one solution i've deployed fix serialize filenames of temp output files, there no collision, , when done, may see few files left around afterwards because our cleanup deletes failed, otherwise job gets done.

one system occured on customer site following os:

windows server 2008 standard 64 bit sp 2  on hardware 8 cores, running 8 threads in our processing.   performance option set "background processing."   

the problem happens @ customer sites, we've lowered threading count interim fix.   don't know if changing performance setting "programs" instead help, far haven't had chance try it, , don't have luxury of fooling around on customer systems-- plus our program nightly etl load warehouse, runs several hours, , may encounter contention problem few times in period.  @ moment, it's not trivial reproduce , toy around with. 

i see if possible verify assessment of going on correct, when program exits, windows may defer closing of files open (sometimes we've seen files opened internally program well, rather via redirect on console command line), such next time temp file access attempted, it's still busy.   i've concluded since reason file still busy close has not yet occured, job of the kernel, since occurs when we're not using redirection file open within executable.  don't know if doing explicit close of file being done by such executables, since of them are third party-- such bcp.exe from sql server, or similar programs other database vendors.   second, i'd know if there's specific can done it, such way configure windows insure these things cleaned before allows opening process continue on may encounter such contention when tries re-open file in next pass.

adding delays seems poor solution, we've tried these , help, reduce frequency, plus we're trying these programs run fast possible in order overall job done, inserting arbitrary delays not popular solution reason well...

i don't quite know better ask question, on of these issues appreciated.

thanks,

--

keith doyle

---

kdd21

might ask them on here.

http://social.msdn.microsoft.com/forums/en-us/category/windowsdesktopdev

 

 

 

---

regards, dave patrick ....
microsoft certified professional
microsoft mvp [windows]

disclaimer: posting provided "as is" no warranties or guarantees , , confers no rights.

Windows Server  >  Windows Server General Forum