Some new reduction capabilities were added in IRAF V2.12
and MSCRED V4.7. In this document we illustrate the capabilities with
mosaic exposures the features may also be applied to single CCD images.
The new features revolve around the creation and use of object masks for
combining, pupil pattern removal, and fringe removal. The application to
mosaic exposures is made more convenient by use of a new multiextension
FITS file option for IRAF masks.
The next section presents an introduction to
the new multiextension FITS mask format and how it is used with IRAF tasks.
The following section introduces a key new mask
application, NPROTO.OBJMASKS, that produces object masks and sky maps for
various uses. One of these uses is for stacking images as discussed in
the section on combining . The final two sections
discuss the mosaic data reduction operations of pupil
pattern and fringe removal. The former is
mostly relevant to KPNO mosaic data. Whether pupil and fringe removal
are necessary depends on the filter. The new functionality is that these
two operations can now be reliably automated, whereas previously they
required interactive decisions by the user.
IRAF masks are files that identify a subset of image pixels with
various integer values. These mask files may be used in the same way
as images but are stored in a special format that is more compact than
a regular raster image. Though may IRAF tasks don't currently make use
of masks the number is growing.
Before IRAF V2.12 there was just one mask format.
This is the pixel list format which is a special binary file
with the extension ".pl". In V2.12 support was added for IRAF pixel masks in
FITS binary table extensions based on one of the proposed
FITS compressed image formats . The advantages of this for users
are that these are valid FITS files and that these masks can be stored
as extensions with other FITS data or multiple masks can be stored in
a single file. This last capability is what is most useful for mosaic
data reductions. This section provides a quick primer on creating and using
this new mask capability in IRAF.
IRAF access to the FITS mask format is through the IRAF FITS kernel.
This means that all the syntactical features of this kernel apply;
in particular, how extension positions or extension names are
specified, and how multiextensions files are built up using the
append flag. These features are discussed in greater detail in the IRAF FITS Kernel
The new feature to the syntax is that if the qualifier "type=mask"
is included in the kernel section when a mask is created, the FITS
extension will be in the compact mask format. Note that since masks are
stored as a binary tables and binary tables are always FITS extensions then
mask files in this format are always multiextension files, even if there
is only a single mask. The consequence of this is that the extension name
(or extension number) will always be required.
When accessing a mask file the file extension (e.g. ".fits") is not
needed unless there is another image file with the same root name but
a different extension. Also the "type=mask" qualifier is only needed
when creating a mask extension and is not needed with accessing an
existing mask. Note that when the qualifier is used on the command line,
a parameter query or in epar, the equal sign has to be escaped as shown
in the examples of figure 1. The extension "pl" is just an example and
any extension name may be used.
Figure 1: Command syntax for multiextension FITS masks.
An object mask identifies pixels containing light from the astronomical
objects in an image or mosaic. The complement is, naturally, the pixels
which form the background. There are many data reduction steps which can
benefit from separating the objects or the background. These include
making sky flats and fringe templates and matching pupil and fringe
Object masks are created by applying an object detection algorithm.
The most common algorithm consists of specifying or determining the
image background and background noise and applying a threshold above the
background to identify object pixels. For the purpose of creating a mask
it is not really important to group pixels into objects, though detection
algorithms do this. The IRAF Astronomical Cataloging Environment (ACE) package, which is
under development, includes such an object detection algorithm.
While the complete ACE package is developed, the sky
estimation, object detection, and mask creation task has been
included in IRAF V2.12 in the NOAO prototype package, NPROTO.
TO . The task is called OBJMASKS.
page describes the parameters and algorithms and gives some examples.
Figure 2 shows an example mosaic image in which we will create an object
mask and background sky map. A map is a reduced resolution image
that is interpolated to the full size of the image to which it applies.
The figure shows the full mosaic displayed at reduced resolution and a
small region displayed at full resolution. The displays illustrate various
features of this mosaic data; astronomical objects, a pupil pattern,
fringing, variable sky background, and residual flat field patterns.
Figure 2: Example mosaic field.
The commands in figure 3 create an object mask and sky map for all
the extensions. Also shown is the command to display the object mask
as an overlay on the mosaic image data (see figure 4 for the result).
Note that the bad pixels are overlayed in red while the object pixels are
in green. The last display command, which produces figure 5, illustrates
how to display the sky maps as a mosaic.
Figure 3: Commands to create and display object masks and sky maps.
Figure 4: Example mosaic field with object mask overlays.
Figure 5: Sky map produced by OBJMASKS.
In addition to producing the object mask, OBJMASKS adds or modifies
the image header keyword OBJMASK to record the object mask associated
with the image. The header is only modified if the user
has write permission on the input data. The OBJMASK keyword can then be
referenced in tasks such as DISPLAY, MSCDISPLAY, IMCOMBINE, and COMBINE.
Note that ability to reference keywords other than BPM in these tasks
is new in IRAF V2.12 and MSCRED V4.7. Figure 6 shows the keyword and
some display commands that use the object mask keyword.
A common image stacking operation is to combine only the sky
or background pixels while excluding or rejecting the object pixels.
This type of operation is used to create sky flat fields, pupil pattern
templates, and fringe pattern templates. One way to do this is to apply
a pixel rejection algorithm where the set of input pixels contributing
to a combined output pixel are examined to find and reject statistically
significant outliers. Sometimes additional pixels within some distance
of the rejected pixels are also rejected.
The ability to create object masks provides another approach. The
pixels identified in the object mask for each image contributing to the
output combined pixels may be excluded without requiring a statistical
test. The ability to use masks to exclude pixels has been in earlier
versions of the primary combining tasks COMBINE and IMCOMBINE. What
is new is that now it is fairly easy to generate the object masks
In a sense the two methods are similar since the detection of the
objects for the object mask is also a statistical outlier method. Some
of the techniques of one could be incorporated into the other but the
functionality would them be rather complex. The multiple image rejection
methods primarily consider the statistics at a single pixel across multiple
images while the object detection method considers the statistics of
neighboring pixels in the same image.
In earlier versions of the IRAF COMBINE and IMCOMBINE tasks, the
masks could only be assigned using the keyword BPM. A minor
new feature of the combining tasks is that the "masktype" parameter may
be specified as "!<keyword>", where <keyword> is any keyword
whose value is the mask to be used. In this special case the good values
in the mask are specified by the "maskval" parameter.
Figures 7 and 8 show the results of combining just two mosaic
exposures of the same field with a small dither shift. In normal use
there would be more exposures and they might be from a variety of fields.
Object masks are first created for these two exposures using OBJMASKS
as described previously. The combining is carried out as shown in
figure 7. Because there are only two exposures in this example there are
places where objects overlap. In this case the "blank" value is output.
Figure 8 shows a region of each mosaic exposure with the object masks
overlayed. The third display is of the combined exposure. The signal
to noise ratio has been improved making the fringing and pupil pattern
more visible. The black spots (which are hard to see in these figures)
are the places where objects overlapped in the two exposures.
Figure 7: Combining two mosaic exposures with object masks.
A pupil pattern may be seen in KPNO Mosaic exposures at the Mayall
telescope in certain filters. The data shown in the figures of this
document have this pattern. The pattern is the ring feature in the center
of the mosaic field. This pattern is seen in both the dome flat field
and sky exposures. The pattern must be determined and then scaled and
removed from this data. The dome flat fields are corrected by removing
the pattern through division in order to preserve the underlying detector
response pattern. The sky exposures are then corrected by the dome
flat field. This leaves the pupil pattern in the sky exposures which is
then removed by subtraction. This aspect of mosaic data reductions for
KPNO Mosaic data has been described in various documents.
The problem has been that to obtain the best correction for either
the dome flat field or sky exposures, the pupil pattern must be scaled
interactively. In this section we present a new task which automates
the scaling and removal of the pupil pattern.
The new task is RMPUPIL. Note that there was previously an interactive
task by this name which is now called IRMFRINGE, where the "I" stands for
help page describes the parameters and algorithm.
The key to producing a reliable automatic scaling factor and correction
is the use of object masks to isolate the pattern in the background from
the contaminating objects. The masks are produced as shown in figure 3.
When the pattern is relatively localized a sky map is not needed and would
be compromised by the pupil pattern anyway. A second mask is also required
which defines the regions in the pupil pattern template which are relevant.
The pupil pattern template is created by first making a sky flat
from a set of exposures where the objects are dithered. Note that using
object masks may improve the creation of the sky flat as described earlier.
The same object masks may be used for the sky flat creation and the pupil
scale fitting and removal. The pupil pattern is extracted from the sky
flat using the task MSCPUPIL. These steps are described elsewhere.
However, one new feature of MSCPUPIL is that it also generate a mask
locating the pupil. This is shown in figures 9 and 10. Note that the
pupil mask need be created only once and saved as a standard calibration
file for all reductions.
Figure 10: Object and pupil template with associated masks
Figure 11 shows the commands and terminal output for running RMPUPIL
and displaying the results as shown in figure 12. Since the pupil pattern
only appears in the extensions im2, im3, im6, and im7 the extfit
parameter is set to the extensions "im". In the log output the
independent scale estimates for each extension are shown but the final
scale is computed using all the fitting extensions. The single scale
factor for the mosaic is then applied to create the pupil subtracted
The section describes how to remove fringing from mosaic data, though
these same steps apply equally well to single images. Figure 14 shows a
small region of one CCD in a mosaic exposure and the corresponding piece
of the fringe pattern template. The basic steps for removing fringing are
creating a template of the fringe pattern, creating a mask that isolates
the background in each exposure, creating a background map when there are
background gradients across the exposure, determining the scale factor
that best matches the fringe template to the exposure, and subtracting
the scaled fringe template from the exposure.
Figure 14: Example of fringing
A fringe template is constructed from all the sky exposures which
exhibit the same fringe pattern. The best result is obtained using dark
sky exposures where the fields have been dithered so that every pixel has
several exposures that are uncontaminated by sources. The exposures are
put together to make a fringe template using the COMBINE task. The images
are scaled to a common level and objects are excluded by rejection and
masking techniques. Descriptions of how this is done using rejection
algorithms are given elsewhere. However, use of OBJMASKS to create object
masks to be excluded during combining is a new capability that can be used.
Note that making object masks (and background maps) for each exposure is
needed for automatic computation and removal of the fringing so making the
masks for creating the fringe template does not add unnecessary processing.
One step in the creation of the fringe template recommended in other
descriptions is the removal of the background on scales greater than the
fringe pattern. This is done with surface fitting or median/average
block filtering using large box sizes. The way the fringe template
is fit to the observations computes and corrects for any mean value.
So there is no need to make the mean of the fringe template be zero though
background structure in the fringe template, such as gradients of large
scale undulations, do need to be removed.
One might wonder whether object masks created for pupil pattern removal
might not be used for the fringe removal. The objects detected will be the
same in the data which is not contaminated by the pupil pattern. In the
KPNO mosaic this is the case for four of the eight CCDs. And where the
pattern is present the objects will not change very much. So it is likely
that these would work fine. The main concern is that when the exposures
exhibit background gradients the sky maps produced when the objects are
detected will be affected by the pupil pattern. Even so this may not
significantly affect the quality of the fringe scale determination.
It is also possible to use only those extensions not affected by the
pupil pattern for estimating the the fringe scaling. So it is quite
possible that creating the object masks again is not necessary. But in
this section we describe creating the object masks and sky maps and applying
these in the fringe scale determination using all the extensions in the
pupil pattern corrected data.
The object masks and sky background maps used for the fringe removal
(and possibly for the fringe frame creation) are created using the
task OBJMASKS. Figure 15 shows the commands to create these files for
a single mosaic exposure and to display the sky background as shown
in figure 16. The mosaic exposure is the same one used to illustrate
the pupil pattern removal in the previous section. In this case the
pupil pattern corrected data is used. So the sky map shown in figure 16
is similar to figure 5 except the pupil pattern is no longer evident.
This sky shows a gradient as well as some flat field patterns to be
removed in a final sky flat field correction. Note that the sky
map is only needed if there is a significant sky gradient. The mean
sky level is automatically accounted for during the fringe removal step.
Figure 15: Creating an object masks and sky map for fringe removal.
The object masks and sky maps produced by OBJMASKS are used in the
task RMFRINGE to automatically determine the fringe scaling and subtract
the pattern. There in MSCRED versions before V4.7 the task RMFRINGE was
a different program that required interactively displaying and entering
scaling factors until the pattern was removed to the satisfaction of the
user. This previous task is now called IRMFRINGE and the new version
of RMFRINGE is an automatic pattern fitting algorithm.
Figure 17 shows the parameters, execution, and output from RMFRINGE.
The commands to produce the displays in figure 18 are also shown. It
is difficult to see the details in that figure but the fringing is
well removed. In this example all the extensions are used to determine
the global fringe scale factor for the mosaic exposure. As mentioned
earlier, the parameter "extfit" could be used to select a subset of
extension and the object mask and sky maps would only be needed for
those extensions. However, the fringe pattern template must be created
for all the extensions and the output contains all the extensions.
Figure 18: Before and after comparison of fringe removal.
One feature that that can be seen in figure 18 is a brightening of
background on the left edge. The section shown is on the edge of the
second CCD in the mosaic. The brightening is caused a drop in the fringe
pattern at the edge which can be seen in figure 14. This artifact was
produced during the background removal when creating the fringe pattern
The correction applied by RMPUPIL is recorded in the correct output
image header by the keyword RMPUPIL. Figure 19 shows these keywords in
a mosaic exposure.