iraf.net

Hi you guys, i'm in STScI trying to do some Hubble archive processing and i'm having this trouble with the xzap task in xdimsum package. I'm trying to get an image with cosmic rays only and a zero background since i'm taking cosmic ray statistics research. I'm using xzap task that generates a cosmic-ray-removed image, that later i substract to the original xzap input image to generate an image with cosmic rays (minus background counts) and a zero background for latter processing in Matlab. I made an script since i'll be processing a bunch of images, like 4000 or 6000 images at once, and processing in lists causes IRAF to PANIC the image-e process. So i made a while loop that i paste below. It seems to work although i had to modify both imarith parameters pixtype="" and calctype="" to ....="real" and the input images i changes too, with chpixtype to "real". The script runs ok, but it generates a lot of blank images with a random nature, not always the same even if the script is re-run with the same images and parameters. BUT, when i run the script with the xzap line outside the while loop (i.e: xzap in a list of images) only the first image gets empty or blank. Could this be a memory allocation problem? hard drive latency for writing a file? or some sort of thing? The thing is that i tried executing xzap on a list of near 5000 images and it hangs up, zero CPU or HDD activity (i gave the PC all night) and when i pressed Ctrl+C it showed a PANIC, so that is why i wanted to do a loop on xzap, that seems to work, but it gives me blank images... Anyway if someone wants to take a look at this it would help me a lot, I'm staying until friday in STSCI and wanted to do some processing before i leave. I attach a couple of images (can duplicate them) and my script with the commented and uncommented options in line 169 and 178.

The complete script makes a cleanup and selection using some bash scripts, but i upload it for you to see the beginning and that.

This is a screenshot of a normal image and a blank one obtained: http://ubuntuone.com/7RiRllPXQKwBS2RrI7CMvf
This is the script i'm doing (i'm kinda noob): http://ubuntuone.com/7LnRrPf5eVeV4NL3ltMcVg
A couple of images: http://ubuntuone.com/4unwn3mo2sQvOC4tNxNDyU
The bad pixel mask: http://ubuntuone.com/4m99JScB8qqExv65Kl9nRr

Please help

---

Lic. Santiago Roland - Los Molinos Observatory - IAU Code 844 - Licensed Radio Amateur CX1DR - Montevideo, Uruguay

If you're running IRAF v2.16 then the problem with processing the lists may be due to issues with the new template code. This should be fixed in the core system but perhaps not all external packages, the workaround is to do

cl\$this->_split2($m[0]) reset use_new_imt = no

to go back to using the old template code. You can make the change permanent by editing your login.cl/loginuser.cl or the system hlib$zzsetenv.def.

To avoid problems with memory corruption in the loop-processing version, I'd suggest adding the command "flpr 0" (zero, not 'oh') following the XZAP command in your script. This will flush the process cache and effectively force each image to be processed with a new invocation of any binaries rather than let uninitialized values accumulate.

I can't say either of these will deal with the blank images you are seeing, unfortunately I'm not familiar with the package and can't offer advice on parameter settings. Hope this helps.

Hi Fitz, thanks for the quick reply. I run that command and tried the looped version with the flpr 0 and the normal list version of xzap. The looped version still throw me bananas (blank images, every single image went bad) and the list processing went fine in the sense that there were no blank images, but the first one has cosmic rays with counts 1x10e8 and higher, could noticed that because of the ds9 scaling. But i'll try with a bunch of images in a list. If this turns out to not work, should i try with an older version of iraf? or this reset trick should be enough? Is this reset prevents for the huge image list PANIC?

Thank you so much,

---

Lic. Santiago Roland - Los Molinos Observatory - IAU Code 844 - Licensed Radio Amateur CX1DR - Montevideo, Uruguay

Assuming the PANIC message is caused by an overflow of the list buffer, then the reset trick should be all you need to work around it even in v2.16 systems. OTOH, if the error is coming from some other task called by the script you'll need to figure out which one, e.g. type "d_trace" to have the scripts (and everything they call) traced during execution. This will produce a LOT of output, but just before the PANIC you should be able to determine which task is being called.

For the blank images, you might try to find a reproducible case and then work with just that one image until you figure out what's going on, e.g. play with the parameters until to you get a working output. You might also have a look at the CRUTIL package in IMRED for alternate ways to deal with the cosmic rays.

Ok, i'll try to debug or find some info in the terminal output at least, i think i read something PANIC on the images-e. something like that process, but i think it's one of the imariths operations that xzap does internally, i'll keep an eye on this and will keep you posted.

Thanks a lot Fitz.

---

Lic. Santiago Roland - Los Molinos Observatory - IAU Code 844 - Licensed Radio Amateur CX1DR - Montevideo, Uruguay