iraf.net

Hi, i'm trying to take advantage of my home system, an AMD FX-8120 8 core system with 16GB of RAM (upgradeable to 32gb) and 2 SATA 3 HDDs. I wonder how can i run IRAF in paralell, so it uses all 8 processors, RAM and/or hard drives IO capabilities, or maybe run multiple IRAF sessions, say, with the same login.cl, and over the same input images. I would like to run a script over the [1] extension of certain images and the [4] extension of the same images at the same time but i don't want to cause any memory problem or system fault.

any ideas?

Best regards.

---

Lic. Santiago Roland - Los Molinos Observatory - IAU Code 844 - Licensed Radio Amateur CX1DR - Montevideo, Uruguay

There are plans for parallelizing IRAF in the Roadmap (posted on the homepage), along with in-memory image operators, that will automate what you describe, but it is still possible to do now with a few caveats:

The login.cl file defines the location of the 'uparm' directory for saving parameters between sessions, and some cases tasks will update their parameters with results (e.g. the IMGETS task, RVCORRECT, etc). Parallel sessions that might update parameters need to either use separate uparm directories, or if you're writing a script use the 'cache' command (e.g. "cache imgets") to create a memory-only version of the parameters for use in the script.

You also need to be sure that the images you're using are being read only, there's no way to protect against two IRAF sessions from writing/updating an image at the same time. Also keep in mind that IRAF is inherently multi-process, i.e. the CL, the IMAGES task and a graphics kernel may all be running at the same time for a particular task and you don't have any control over how many CPUs are used to do this. For an 8-core system you won't necessarily get a factor of 8 increase in speed.

Other than that, you can launch 8 separate background jobs (actually up to 32 in v2.16.1, but only 4 for earlier versions) that process lists or have separate CL sessions if you prefer.

I think i can do that, do mkiraf in several directrories and launch several iraf instances since the images are read and any modification is saved into another file, that draw a lot of disk IO and space but it is the safest way to prevent miss-processing of images and lose the original. I launched 4 iraf sessions with scripts and everything seems to be running fine, i could launched more, but again, the bottleneck are the hard drives. i'm hitting over 220 gb of images, 5 years of darks taken with the hubble wfpc3, not bad.

I appreciate any further comments if you see something that could cause trouble.

thanks,

---

Lic. Santiago Roland - Los Molinos Observatory - IAU Code 844 - Licensed Radio Amateur CX1DR - Montevideo, Uruguay

That should work fine, 220GB is less than one DECam night for us so I'm not too worried it can be done 8-)

You can do this from a single IRAF session using background jobs as e.g.



The virtual memory system will be pretty good at caching those files used frequently but it won't help much when processing many images you touch only once. It would be fair to use more like 6-8 jobs if you want to make sure the machine is fully occupied, just don't expect an 8X speedup.