Memory problems in mscred.combine

[dougmink]

We have just updated our pipeline computers from 2.12 to 2.12.2a and
are running out of memory when combining images which are scaled
or combined with anything but a sum or run in any reject mode. Is there
a buffer size parameter the default value of which might have been changed between IRAF versions? Does anyone have any other ideas? imcombine
seems to work OK, but we need to combine multi-extension images and use the ccdtype option which can only be done in combine.

[valdes]

Hi,

My memory is now hazy but this error has been reported a couple of times. I don't think I was ever able to identify the source of the problem though it may be related to the amount of memory used by masks. Also make sure that you don't have min_lenuserarea set too a large number. This is needed at times for headers with a very large number of keywords but since combine may be openning many images at the same time this can suck up all the memory.

Combine/imcombine are supposed to catch and make adjustments to out of memory errors. However, there are cases where this may not work. So as a workaround, since I have not been able to identify all the sources of failure, I added an environment variable "imcombine_maxmemory" to limit the amount of memory initially asked by the task. This is in bytes and is described in the mscred.readme from the distribution I created today (ftp://iraf.noao.edu/iraf/extern/mscred/). Note that this will not solve all problems, such as the one with min_lenuserarea, but only cases where the algorithm to start with as large an allocation as possible and then reduce this after an error fails because the memory error was not properly caught and handled..

To use the variable set it in your login.cl, loginuser.cl, or interactively (followed by flpr to reset any running process' environment) using:

set imcombine_maxmemory = 100000000

As just noted, this change was not in the last distribution from 2004 so you will need to update mscred at your site.

I hope this provides a solution.

Yours,
Frank Valdes