Welcome to iraf.net Saturday, April 20 2024 @ 12:02 AM GMT
IRAF Help page for the mscguide
task or topic
MSCGUIDE (Sep98) mscred MSCGUIDE (Sep98) GUIDE TO THE NOAO MOSAIC DATA HANDLING SOFTWARE Francisco Valdes September 1998 MSCRED Version 2.0 SECTIONS 1. Introduction 2. Multiextension FITS Files 3. Examining Mosaic Data 3.1 Displaying the Data 2.1.1 On-the-Fly (OTF) Calibration 2.1.2 Real-Time Display with the DCA 3.2 Examining the Data 3.3 Examining the Headers 3.4 Determining Best Focus 4. Data Reductions 4.1 Some Preliminaries 4.2 Basic CCD Calibration 4.2.1 Calibration Data to Obtain At the Telescope 4.2.2 Preparing Calibration Data 4.2.3 Pupil Image Removal from Flat Fields 4.2.3.1 Broadband Data 4.2.3.2 Narrowband Data 4.2.4 Object Exposure Reductions 4.2.5 Pupil Image Removal from Object Data 4.2.5.1 Broadband Data 4.2.5.2 Narrowband Data 4.2.6 Dark Sky or Twilight Sky Flat Fields 4.2.7 The Variable Pixel Scale and Zero Point Uniformity 4.3 Coordinate Calibration 4.3.1 Setting Coordinate Zero Points and Measuring Coordinates 4.3.2 Matching Coordinate Systems 4.4 Putting the Pieces Together 4.4.1 Removing Sky Gradients 4.4.2 Constructing Single Images 4.4.3 Matching Intensity Scales 4.4.4 Making the Final Stack Image 1. INTRODUCTION This document discusses handling and reducing CCD mosaic data, particularly data from the NOAO CCD Mosaic Imager (referred to here as the NOAO Mosaic), using IRAF and the MSCRED package. It is not a beginner's guide and assumes some previous experience with IRAF and CCD reductions. The first section discusses the mosaic data format and how to use it with IRAF. This format is more complex than single CCD images because of the multiple CCDs and possibly multiple amplifiers per CCD. To keep the data from each exposure self-contained the multiple CCD images are stored in a single file. This multiple image per file has many advantages but it does mean that some commands for dealing with images behave differently. The second section describes the tools used to examine the mosaic data. These tools are used during observing as well as during data reductions. The last section describes the reduction of mosaic data. This includes basic CCD instrumental calibration and combining mosaic exposures into single images. 2. MULTIEXTENSION FITS FILES The data format used by the NOAO Mosaic Data Handling Software (MDHS) is a multiextension FITS (MEF) file. This format is produced by the the Data Capture Agent (DCA) when observing with the NOAO Mosaic. The MEF file for the NOAO Mosaic currently consists of nine FITS header and data units (HDU). The first HDU, called the primary or global header unit, contains only header information which is common to all the CCD images. The remaining eight HDUs, called extensions, contain the images from the eight CCDs. The fact that the image data is stored as a FITS file is not significant. Starting with IRAF V2.11, FITS files consisting of just a single primary image may be used in the same way as any other IRAF image format. The significant feature of the mosaic format is its multi-image structure. With multiextension FITS files you must either use tasks which are specifically designed to operate on these files as a unit or explicitly specify the image within the file that is to be operated upon by general IRAF image processing tasks. The tasks in the MSCRED package are designed to operate on the mosaic MEF files and so you only need to specify the filename. For image tasks outside the MSCRED package you must specify the image in the MEF file using the syntax filename[extension] where "filename" is the name of the MEF file. The ".fits" filename extension is optional provided there is no confusion with other files with the same basename. The image "extension" is specified either using an extension name or the position of the extension in the file (where the first extension is 1). The extension names in the NOAO Mosaic data are "im1" through "im8" for the eight CCDs. For a detail discussion of the IRAF FITS Image Kernel and the syntax it supports for multiextension FITS files see ftp://iraf.noao.edu/iraf/docs/fits_userguide.ps.Z. If you forget to specify an extension to a task that expects only single images you will get the following error which is your reminder to include an extension. ms> imhead obj012 1 ERROR: FXF: must specify which FITS extension (obj012) Two of the most common tasks that require specifying an image extension are DISPLAY to display a single CCD image (the task MSCDISPLAY is used to display all the images at once) and IMHEADER to list the header of a particular CCD. So, for example, the following commands might be used. ms> display obj012[im2] 1 ms> imhead obj012[3] l+ Other tasks you may use this way are IMEXAM and IMPLOT. A common question is how to specify a list of extensions. Modification of the syntax to allow wildcard templates in the extension specification is under study. Currently you must specify each extension explicitly, though the filename itself may be a wildcard; for example the first image in a set of files can be collectively specified with obj*[im1] There are two methods for specifying some or all extensions in tasks that operate upon lists of images. One is to make @files. This can be done explicitly with an editor. However the PROTO task IMEXTENSIONS can expand MEF files into an @file as in the following example. ms> imexten obj012,obj13 > list ms> imhead @list Read the help page for further information, additional parameters, and examples. Another method is to use the special MSCRED task MSCCMD. This task can be used on the command line or as a simple interactive command interpreter. The idea is that you use the special designations "$input" and "$output" for task parameters which allow lists of images. Then lists of MEF filenames are specified for the input and output which are expanded and substituted into the task parameters when it is executed. For example, ms> msccmd "imhead $input l+" input=obj012,obj013 For additional information and examples consult the help page for that task. Note that the tasks IMSTAT and IMARITH are so useful and common that there are specific MSCRED tasks MSCSTAT and MSCARITH that operate on all or a subset of image extensions. So these tasks need not be used with MSCCMD or with @files. We conclude with a discussion of the special operations of copying, renaming, deleting, and reading and writing FITS tapes as they apply to the mosaic MEF files. To copy a mosaic file as a unit use COPY, making sure to explicitly specify the "fits" extension. If you use IMCOPY it will expect you to specify a particular extension and will copy only that extension. While IMCOPY is not the way to copy an complete MEF file the tasks IMRENAME and IMDELETE are the commands for renaming and deleting these files; though RENAME and DELETE will also work provided you are explicit with the extension. Finally the mosaic data should be kept as a MEF file and so the special mosaic tasks MSCWFITS and MSCRFITS should be used. The current WFITS and RFITS are not intended for this type of data. 3. EXAMINING MOSAIC DATA During observing a small set of IRAF commands are commonly used to examine the data. This section describes these commands. While the discussion is oriented towards examining the data at the telescope during the course of observing, the tools described here are also used when reducing data at a later time. 3.1 DISPLAYING THE DATA The two commands DISPLAY and MSCDISPLAY are used to display the data in a display server window. The display server is a separate process which must be running before displaying the images. The observing environment at the telescope will generally have the XIMTOOL display server already running with a window on a separate monitor. If it is not running for some reason it can be started with a menu selection. Away from the telescope you would start XIMTOOL or SAOIMAGE as you do normally. The display server must be told what size "frame buffer" to allocate for holding the display pixels. This determines how many pixels may be loaded at one time. Note that the display window may be smaller than this size and the display server allows you to move the portion viewed and zoom/unzoom any region. If the image size is larger than the frame buffer you can display a portion of the image at full resolution or the full image at a lower resolution. The frame buffer size is queried and set with the commands: ms> show stdimage imt4096 ms> set stdimage=imt2048 There are trade-offs in the frame buffer selection. A large frame buffer allows you to have higher resolution for the large mosaic images but it uses more memory and takes longer to load. The DISPLAY task is used to display individual images in the display server. This task is a standard IRAF task about which you are assumed to have some basic knowledge. There are many display options which are discussed in the help page. The only special factor in using this task with mosaic data is that you must specify which CCD image to display using the image extension syntax discussed previously. As an example, to display the central portion of extension im3 in the first frame and the whole image in the second frame: ms> display obj123[im3] 1 fill- ms> display obj123[im3] 2 fill+ The MSCDISPLAY task is based on DISPLAY with a number of specialized enhancements for displaying mosaic data. It displays the entire mosaic observation in a single frame by "filling" each image in a tiled region of the frame buffer. The default filling (defined by the order parameter) subsamples the image by uniform integer steps to fit the tile and then replicates pixels to scale to the full tile size. The resolution is set by the frame buffer size. As mentioned before, trying to increase the resolution with a larger buffer size has the penalty of longer display times. An example display command is: ms> mscdisplay obj123 1 The default parameters for MSCDISPLAY are shown below. Many of the parameters are the same as DISPLAY but there are also a few that are specific to the task of displaying a mosaic of CCD images as indicated with an asterisk. I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = mscdisplay image = root name for image to be displayed frame = 1 frame to be written into * (mimpars= ) mosaic image parameters * (check = no) check if image is loaded * (onepass= no) load all extensions in one pass? (bpmask = BPM) bad pixel mask (bpdispl= none) bad pixel display (none|overlay|interpolate) (bpcolor= red) bad pixel colors (overlay= ) overlay mask (ocolors= green) overlay colors (erase = yes) erase frame (border_= no) erase unfilled area of window (select_= yes) display frame being loaded (repeat = no) repeat previous display parameters (fill = no) scale image to fit display window (zscale = yes) display range of greylevels near median (contras= 0.25) contrast adjustment for zscale algorithm (zrange = yes) display full image intensity range (zmask = ) sample mask * (zcombin= auto) Algorithm for combining z1 and z2 values... (nsample= 1000) maximum number of sample pixels to use (order = 0) spatial interpolator order (0=replicate,... (z1 = 0.) minimum greylevel to be displayed (z2 = 1000.) maximum greylevel to be displayed (ztrans = linear) greylevel transformation (linear|log|none|user) (lutfile= ) file containing user defined look up table The mapping of the pixel values to grey levels includes the same automatic or range scaling algorithms as in DISPLAY. This is done for each image in the mosaic separately. The new parameter "zcombine" then selects whether to display each image with it's own independent display range ("none") or to combine the display ranges into a single display range based on the minimum and maximum values ("minmax"), the average of the minimum and maximum values ("average"), or the median ("median") of the minimum and maximum values. The independent scaling is most appropriate for raw data while the "minmax" scaling is recommend for processed data which are gain calibrated. The special value "auto" (the default) checks if the display data has been flat fielded, either by separate processing or with on-the-fly calibration, and if so it uses "minmax" scaling and if not it used independent scaling. The "mimpars" (mosaic image parameters) parameter is actually a reference to another set of parameters. The default with no value is to use the parameters from the parameter task MIMPARS. These parameters can be examined and set with EPAR either by typing ":e" when over this parameter in MSCDISPLAY or by running EPAR directly on this task; i.e. epar mimpars. The parameters for NOAO Mosaic data are shown below. I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = mimpars (extname= ) extension name pattern (exttmpl= _![1-9]![1-9]![1-9].*) extension template (xgap = 72) minimum X gap between images (ygap = 36) minimum Y gap between images (process= no) do calibration processing? (oversca= yes) do line-by-line overscan subtraction? (flatfie= yes) do flat field correction? (caldir = mscdb$noao/kpno/4meter/caldir/) calibration directory (filter = !filter) filter The "extname" parameter is used to select as subset of the image extensions to display. It is a pattern to match extension image names. For extensions such as im1, im2, etc. the pattern typically uses the character selection template such as "[1256]" to select anything with a 1, 2, 5, or 6 in the name. The pattern matching syntax can be found in the help for the task MATCH. The "exttmplt" parameter is for use with non-MEF data. The gap parameters define the gap size in the display. The remaining parameters are for the on-the-fly calibration discussed below. 3.1.1 ON-THE-FLY (OTF) CALIBRATION Raw mosaic data can exhibit significant instrumental artifacts which may interfere with inspecting the data prior to reductions. The most significant artifact is gain variations both within each CCD image and between the CCDs. In the simplest case of constant gain variations between the CCDs the independent display scaling, "zcombine" of none or auto, may be sufficient. But when there are significant flat field patterns it may be desirable to apply a quick, approximate flat field calibration as the data are being displayed. MSCDISPLAY can apply an on-the-fly (OTF) calibration to raw mosaic exposures. This does not change the actual data files and the calibration is intended to be quick and approximate. The calibration steps performed are a line-by-line bias subtraction using the overscan region of the data and a division by a flat field. If the data have been overscan corrected or flat field corrected by CCDPROC then the task will automatically skip those steps. The title of the display will indicate if the data have been calibrated by adding "[bias]" for bias subtraction and "[bias,flat=XXX]" for bias subtraction and flat fielding using an OTF flat field called XXX. The bias subtraction is performed by averaging the overscan pixels in a line and subtracting this average from all the pixels in the line. This removes the amplifier bias and line-by-line patterns. The flat field or response calibration is performed by reading special compact flat field calibration data which provides an approximate relative response for each pixel in each amplifier readout. Depending on how the calibration file is derived this will approximately correct for pixel sensitivity variations, gain variations between the amplifiers, sky illumination variations, and any pupil ghost pattern (as occurs with NOAO Mosaic data from the Mayall (KPNO 4meter) telescope). The "process" parameter in the MIMPARS parameter set shown earlier selects whether to turn on or off the OTF processing. If it is no then regardless of the "overscan" or "flatfield" parameter settings no calibration is applied. If it is yes then one or both calibration operations can be selected. Because the MIMPARS parameters can be set on the command line, it is common to leave the "process" parameter set one way, say to "no", and then override the value when displaying. For example, ms> mscdisplay obj023 1 proc+ ms> mscdisplay flat022 2 proc+ flatfield- The flat field calibration requires special calibration files. The "caldir" parameter defines a directory containing the calibration files. This can be a standard directory or a user directory. Note that if a directory is specified it must end with $ or /. Within the calibration directory the calibration file to apply is selected by the "filter" parameter. For automatic selection of calibrations, the calibrations can be selected by the filter string in the header (or by giving the same filter string in the "filter" parameter). To use the filter string in the header the value of the filter parameter is set to "!<keyword>" where <keyword> is the keyword for the filter string. Creating the a calibration directory and calibration files is done with the task MSCOTFFLAT. For the NOAO Mosaic a calibration directory is provided. However you can create your own as described in the help for MSCOTFFLAT. The "filter" parameter can be set to one of these names. 3.1.2 REAL-TIME DISPLAY WITH THE DCA During data acquisition the MSCDISPLAY task can be used to display mosaic data as it is being written to disk by the DCA. It begins execution shortly after the readout begins and displays the portion of the recorded image which has been written to disk. It then continually displays new data which has been written by the DCA until the exposure is completely written to the display. The DCA control panel allows you to select whether to display the data during readout and how it is to be displayed. This includes selecting the OTF calibration. One toggle is equivalent to the "process" parameter. If the processing is turned on the DCA automatically selects only overscan bias subtraction for non-object exposures and selects both bias subtraction and flat field division for object exposures. The "filter" parameter is set by passing through the filter string from the data acquisition system or by overriding this and using the filter menu to select one of the available calibrations. 3.2 EXAMINING THE DATA The task MSCEXAMINE allows interactive examination of mosaic images. It is essentially the same as the standard IMEXAMINE task except that it translates the cursor position in a tiled mosaic display into the image coordinates of the appropriate extension image. Line and column plots also piece together the extensions at the particular line or column of the mosaic display. To enter the task after displaying an image the command is: ms> mscexam As with IMEXAMINE, one may specify the mosaic MEF filename to be examined and if it is not currently displayed it will be displayed using the current parameters of MSCDISPLAY. It is important to realize that this task shares the MIMPARS parameters with MSCDISPLAY. To get data values back that match what is displayed the parameters must agree with those used to display the data. In particular, if the data are display with OTF processing then MSCEXAM must be told this either by explicitly setting the process flat in MIMPARS or setting it on the command line, ms> mscexam proc+ 3.3 EXAMINING THE HEADERS There was discussion earlier concerning the use of generic image tasks with the NOAO Mosaic data. The tasks IMHEADER and HSELECT fall into this category. The two important points to keep in mind are that you must specify either an extension name or the extension position and that the headers of an extension are the combination of the global header and the extension headers. Often one does not need to list all the headers for all the extensions. The image title and many keywords of interest are common to all the extensions. Thus one of the following commands will be sufficient to get header information about an exposure or set of exposures: ms> imhead obj*[1] l- # Title listing ms> imhead obj123[1] l+ | page # Paged long listing ms> hselect obj*[1] $I,filter,exptime,obstime yes If you need to list header information from all the extensions then you need to take the additional step of creating an @file or using MSCCMD. For example to get the default read noise and gain values for each CCD: ms> imextensions obj123 > list123 ms> hselect @list123 $I,rdnoise,gain yes or ms> msccmd "hselect $input $I,rdnoise,gain yes" input=obj123 The CCDLIST task in the MSCRED package is specialized for the mosaic data. It provides a compact description of the name, title, pixel type, filter, amplifier, and processing flags. The "extname" parameter may be used to select a particular extension, a set of extensions, or all extensions. Because all extensions should generally be at the same state of reduction it may be desirable to list only the first extension. Like most of the CCD reduction tasks you can also select only a certain type of exposure for listing. Examples of the two modes are: # Summary for all exposures ms> ccdlist *.fits extname=im1 # Summary for all object exposures ms> ccdlist *.fits extname=im1 ccdtype=object # List of all extensions. ms> ccdlist obj123 extname="" 3.4 DETERMINING BEST FOCUS Focus sequence frames can be evaluated for the best focus using MSCEXAM and the 'r' or 'a' keys. However, there is a special task for measuring the sequence of focus images called MSCFOCUS. This displays a focus exposure with MSCDISPLAY (if needed) and then lets you select one or more bright stars to measure. This task is customized so that all you need do is mark the top image in any CCD. For NOAO Mosaic data, header information tells the task how many exposures, the spacings between the exposures, and the focus values. After the measurements are made they are displayed and analyzed graphically and written to the terminal and logfile. This task is the mosaic analog of the KPNOFOCUS and STARFOCUS tasks for single CCD data. 4. DATA REDUCTIONS The reduction of CCD mosaic data can be divided into two stages. The first is the basic calibration of the individual CCDs. This stage is similar to reducing data from single CCD exposures except that the calibration operations are repeated for all the CCDs in the mosaic. The only significant difference is that any scaling of an exposure, such as in normalizing the flat field calibration, must be done uniformly over all the CCDs. The details of repeating the calibrations for all CCDs and the scaling of the calibration data are taken care of by the software and the data format so that these operations appear the same as with single CCD data. There are some steps which are not typical for CCD data with smaller fields of view or specific to the NOAO Mosaic at the Mayall telescope. At the Mayall telescope there are reflections off the corrector that produce a visible image of the pupil. Coating of the corrector minimizes this image but it may be desirable to remove this instrumental signature which would otherwise cause a small variation of the photometric zero point as well as an unwanted visible feature. There are two sections discussing removal of this feature from the flat field data and from the object exposures. If your data is from the KPNO 0.9 meter telescope or the image is faint enough that it is not of concern then you can skip the extended discussion. A caveat about the pupil removal steps described here is that this document was written prior to the latest removal of the corrector for better anti-reflection coating. So the NOAO staff have little experience with these corrections though earlier work has shown that these steps will do a good job. Another step of the basic CCD calibration stage which has generally been ignored or forgotten with smaller single CCD formats is the variable pixel scale. The large field of view provided by a mosaic and the optics required to provide it can lead to a significant variation in the pixel scale. This effect is important with the Mayall telescope and is also present in the NOAO 0.9 meter data to a smaller degree. It is likely to be present in other telescopes as well. When the pixel scale varies significantly the standard flat field calibration operation will cause the photometric zero point to vary. A simple calibration step can be performed to remove this effect. However, if you intend to produce single images from the mosaic of CCDs this step is not necessary since the resampling operation naturally accounts for this effect. The second stage of data reductions is unique to mosaic data. This stage is the combining of the multiple CCD images and multiple exposures into a single image. Since creating a single image from a single mosaic exposure is of marginal value, the thrust of this stage of the reductions is the combining of multiple exposures which have been spatially offset or "dithered" to cover both the gaps between the individual CCDs and any defects. The steps required to produce a single deep integration from dithered exposures consist of accurately registering the images, mosaicing the exposures into single images with the same spatial sampling, measuring changes in the intensity scale due to variations in transparency and sky brightness, and combining the individual images into a single deep image with the gaps and bad pixels removed. 4.1 SOME PRELIMINARIES The command SETINSTRUMENT is used to set default parameters for the tasks in the MSCRED package appropriate to a particular instrument. For users of the NOAO Mosaic it is recommended you run this command the first time you reduce data. Subsequently you should not do this since it will reset parameters you later changed. To set the parameters for reducing the NOAO Mosaic data type the command ms> setinstrument kpno 4meter CCDMosaThin1 review- Substitute "36inch" for "4meter" if the data is from the Kitt Peak 0.9 meter telescope. For some of the operations it is useful to specify lists of exposures corresponding to a dither set. The examples in this guide show using @files for dither sets. An @file is simply a list of filenames. These can be created in several ways including using a text editor. One way is with the FILES command to expand a file template. For example, ms> files obj021,obj022,obj023,obj024,obj025 > field1 ms> dir @field1 obj021 obj002 obj003 obj004 obj005 4.2 BASIC CCD CALIBRATION Basic CCD instrumental calibrations consist of correcting each CCD for electronic bias levels, zero exposure patterns, dark counts, and pixel sensitivities. A cosmetic replacement of bad pixels may also be included. For the Mayall telescope the pupil image due to reflections off the corrector must be removed from the flat field and object exposures. An additional calibration is required to correct for the variable pixel scale across the field of view if you intend to do photometry on the individual CCD images. 4.2.1 CALIBRATION DATA TO OBTAIN AT THE TELESCOPE Good data reductions begin with obtaining good calibration data at the telescope. This section discusses the NOAO Mosaic but the general principles will apply to other detectors, though the relative importance of different calibrations will depend on the quality of the CCDs and the stability of the camera. The standard calibration data are sequences of zero exposures and sequences of dome flat field exposures. While dark count exposures, matched to the typical object exposure times, were important for the first generation (engineering grade) NOAO Mosaic, dark counts are expected to be low in the science grade detectors. Thus dark count exposures are probably not necessary. Dome flat fields (dome flats) provide a fair basic flattening of the data to 2% or so, but sky flat fields (sky flats) are required to produce dithered data that can be combined without introducing obvious artifacts. Good sky flats can flatten the data to 0.1%. In our experience twilight exposures do not work well. Instead dark sky flat fields are derived from unregisted object exposures taken during the night or run. If your observing program consists of only large extended objects or single pointings then you should also take some dithered exposures of "blank" sky. At the Mayall telescope there is a pupil image caused by reflections off the corrector. For broadband photometry the effects of the pupil image are small but they can be reduced even further by reduction steps to remove the image. One useful calibration for this removal is a narrowband dome flat field. The idea is that the narrowband flat field has a more prominent pupil image that can be used as a template for the much fainter broadband pupil image. Lastly, good astrometry is required to register and stack the Mosaic images. The NOAO Mosaic data contains previously determined astrometry recorded in the headers of the raw exposures. This is sufficient for most purposes. However, for cameras without astrometry or to generate your own astrometry solutions, fields with a reasonable density of stars with cataloged accurate coordinates must be taken. Note that with the new generation of large astrometric catalogs and the large field of view of a mosaic, it may be that the object exposures already contain sufficient information for deriving new astrometric calibrations or corrections. Note that this guide does not yet discuss how to create the astrometric coordinate system solutions. 4.2.2 PREPARING CALIBRATION DATA This section describes how to prepare the basic calibration data. The steps are virtually the same as with the CCDRED package and, in fact, the command names and parameters are the same. The basic calibration data of zero level, dark count, and dome flat fields are generally taken as a sequence of identical exposures which are combined to minimize the noise. A later section discusses preparing a sky flat field calibration using the object exposures. The calibration exposures are individually reduced by CCDPROC and then combined. Thus, it is necessary to first set the CCDPROC parameters. Because this task knows which operations are appropriate for particular types of calibration exposures you can set all the parameters for object exposures. Below is a typical set of parameters. The main optional setting is whether or not to replace bad pixels by interpolation, which is purely a cosmetic correction. However, it is recommended that this be done to avoid possible arithmetic problems in the processing. I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = ccdproc images = List of Mosaic CCD images to process (output = ) List of output processed images (ccdtype= object) CCD image type to process (noproc = no) List processing steps only? (oversca= yes) Apply overscan strip correction? (trim = yes) Trim the image? (fixpix = yes) Apply bad pixel mask correction? (zerocor= yes) Apply zero level correction? (darkcor= no) Apply dark count correction? (flatcor= yes) Apply flat field correction? (sflatco= no) Apply sky flat field correction? (biassec= !biassec) Overscan strip image section (trimsec= !trimsec) Trim data section (fixfile= BPM) List of bad pixel masks (zero = Zero) List of zero level calibration images (dark = Dark) List of dark count calibration images (flat = Flat*) List of flat field images (sflat = Sflat*) List of secondary flat field images (minrepl= 1.) Minimum flat field value (interac= no) Fit overscan interactively? (functio= legendre) Fitting function (order = 1) Number of polynomial terms or spline pieces (sample = *) Sample points to fit (naverag= 1) Number of sample points to combine (niterat= 1) Number of rejection iterations (low_rej= 3.) Low sigma rejection factor (high_re= 3.) High sigma rejection factor (grow = 0.) Rejection growing radius The overscan correction has two methods as selected by the fitting function. A value of "legendre" (or "chebyshev" or "spline3") take all the overscan data and fit a smooth function along the column direction. The "order" value of 1 shown above fits a single constant value. This leaves to the zero level calibration to subtract any details of line-by-line structure. A value of "mean", "median", or "minmax" take the mean, median, or mean excluding the minimum and maximum values, of the overscan at each line and subtract that value from that line. The other fitting parameters are ignored. The advantage of this is that systematic line-by-line patterns are subtracted. The disadvantage is, since the sample of overscan pixels is small at each line, that this can also introduce a statistical line-by-line pattern. There is currently no recommendation for the NOAO Mosaic. The first step is generally to process and combine sequences of zero, dark, and dome flat exposures. This is done using the tasks ZEROCOMBINE, DARKCOMBINE, and FLATCOMBINE. The combining must be done in the following order since the processing of later calibration data requires the preceding calibration data. ms> zerocombine *.fits ms> darkcombine *.fits ms> flatcombine *.fits Each of these tasks search all the exposures for a particular type so it is fine to specify all files, though if the file names code the type, such as "dflatNNN", then one can use that as the wildcard to shorten the search of all the data. Also FLATCOMBINE has the feature that it will combine the data separately for each filter. However, you can use explicit file lists, templates, or @files to limit the input files. The output combined names have standard default values which the above settings for CCDPROC use. It is a good idea to first check that the different calibration types and filters are correctly identified by the software. This is done using the CCDLIST command ms> ccdlist *.fits Unless you change the parameters "mscred.backup" and "mscred.bkuproot" the original raw files will be saved in the subdirectory "Raw/". If you want to start over, delete the processed files and copy the raw files back to the working directory. If disk space is a concern and you are satisfied with the combined calibration files you can delete the individual processed calibration files. There is a parameter in the combining tasks that will delete the individual files automatically after processing and combining. 4.2.3 PUPIL IMAGE REMOVAL FROM FLAT FIELDS NOAO Mosaic data taken at the Mayall (4meter) telescope include a pupil image caused by reflections off the corrector. The magnitude of this image is a function of the filter and the state of the anti-reflection coatings on the corrector. It is also a function of the total light, including from outside the field of view, and somewhat on the location of bright stars. It might appear at first that one simply divides the object exposures by the flat field as is done for the OTF display calibration. However this is not photometrically correct because the pupil image is an additive light effect and not a detector response. Instead the pupil image must first be removed from the flat field before applying it to the object data. The object data is then corrected after flat fielding by subtracting the extra light from the pupil image. The pupil image is removed from the flat field by dividing by an estimate of the pupil image pattern. The challenge is to determine the pupil image contribution in the presence of other flat field structure. There are two current approaches to obtaining the pupil image pattern for removal from the data. One is to use data from another source where the pupil pattern is more easily separated from the flat field pattern. The second is to derive the pattern from the data assuming something about the form of the pattern. In particular, to use the difference in scales between the larger pupil pattern and the smaller flat field pattern. The first approach is preferable since it better preserves fine structure in the pupil image but the second is needed when no other data is available. 4.2.3.1 BROADBAND DATA For broadband data the recommended procedure is to obtain a narrowband flat field exposure. This narrowband exposure will have a stronger pupil image relative to the flat field pattern and, when the pupil image is scaled down to match the broadband image flat field, the errors from the flat field response will be diminished. The pupil image is extracted from the narrowband flat field using the task MSCPUPIL. This task determines the background levels in a ring inside and outside the main pupil image and subtracts this background to produced the pupil image template. Outside the outer background ring the template is set to zero. In effect this is like "scrapping off" the pupil image from the exposure. The relevant parameters are I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = mscpupil input = List of input images output = List of output images (masks = BPM) List of masks (type = data) Output type (xc = 27.) Pattern center offset (pixels) (yc = 9.) Pattern center offset (pixels) (rin = 300.) Radius of inner background ring (pixels) (drin = 20.) Width of inner background ring (pixels) (rout = 1500.) Radius of outer background ring (pixels) (drout = 20.) Width of outer background ring (pixels) (funcin =chebyshev) Inner azimuthal background fitting function (orderin= 2) Inner azimuthal background fitting order (funcout= spline3) Outer azimuthal background fitting function (orderou= 2) Outer azimuthal background fitting order * (rfuncti= spline3) Radial profile fitting function * (rorder = 40) Radial profile fitting order * (abin = 0.) Azimuthal bin (deg) * (astep = 0.) Azimuthal step (deg) (niterat= 3) Number of rejection iterations (lreject= 3.) Low rejection rms factor (hreject= 3.) High rejection rms factor (datamin= INDEF) Minimum good data value (datamax= INDEF) Maximum good data value (verbose= yes) Print information? The output type is set to "data" to extract the pupil image after background subtraction. The pattern center parameters are offsets from the astrometric center and the inner and outer radii are measured from the pattern center. The default values are for the last measured Mayall pupil image. The fitting parameters marked with an asterisk are not used when extracting the pupil image directly. The pupil image template is scaled and removed from the flat field using the task RMPUPIL. The removal is done with the arithmetic operation I(out) = I(in) / (scale * I(template) + 1) where I(out) are the output corrected pixel values, I(in) are the input pixel values, I(template) are the pupil image template pixel values, and scale is the relative scale factor to be applied. The parameters for the pupil image removal task are I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = rmpupil input = Input mosaic exposure output = Output mosaic exposure template= Template mosaic exposure (type = ratio) Type of removal (extname= [2367]) Extensions for fit (blkavg = 8) Block average factor (fudge = 1.6) Fudge factor (interac= yes) Interactive? (mscexam= no) Examine corrections with MSCEXAM? The "input" is the broadband flat field, the "output" is the corrected flat field, and the "template" is the narrowband pupil image produced by MSCPUPIL. The type of removal for a flat field is "ratio" as given by the equation above. Determining the optimal scaling of the template pupil image to the input pupil image is normally done interactively. The task makes a guess at scaling. If this task is used non-interactively this will be the scale used. When the task is used interactively the input and corrected mosaic exposures are displayed and then a query for a new scale is given. By repeatedly adjusting the scale factor the best visual removal can be obtained. When done the output corrected flat field is created using the last specified scale factor. Note that to quit requires entering dummy special values for the scale factor. A value of zero means to create the final output exposure with the last scale factor and a value of -1 means to quit without producing any output. Because this operation is fairly slow and iterative there are some steps that can be taken to it speed up. The "extname" parameter selects just those extensions to look at. For NOAO Mosaic data the default selects the central four extensions covered by the pupil image. The "blkavg" parameter applies a block average to the input exposure and template. This makes the display and iterative corrections faster. When the best scale factor has been determined the entire input image at full resolution is corrected by the full resolution template to create the output flat field. If one wants to use the facilities of MSCEXAM to evaluate each iterative correction then the "mscexam" parameter can be set. However, the most powerful estimate for the optimal scale factor is viewing the display and possibly blinking between the uncorrected and corrected frames. 4.2.3.2 NARROWBAND DATA For narrowband data the pupil image template must be derived from the data itself. This is done by fitting the data with an axially symmetric pattern. The fitting is performed by MSCPUPIL with the parameters I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = mscpupil input = List of input images output = List of output images (masks = BPM) List of masks (type = ratio) Output type (xc = 27.) Pattern center offset (pixels) (yc = 9.) Pattern center offset (pixels) (rin = 300.) Radius of inner background ring (pixels) (drin = 20.) Width of inner background ring (pixels) (rout = 1500.) Radius of outer background ring (pixels) (drout = 20.) Width of outer background ring (pixels) (funcin =chebyshev) Inner azimuthal background fitting function (orderin= 2) Inner azimuthal background fitting order (funcout= spline3) Outer azimuthal background fitting function (orderou= 2) Outer azimuthal background fitting order (rfuncti= spline3) Radial profile fitting function (rorder = 40) Radial profile fitting order (abin = 0.) Azimuthal bin (deg) (astep = 0.) Azimuthal step (deg) (niterat= 3) Number of rejection iterations (lreject= 3.) Low rejection rms factor (hreject= 3.) High rejection rms factor (datamin= INDEF) Minimum good data value (datamax= INDEF) Maximum good data value (verbose= yes) Print information? Note that this only differs from the previously shown parameters by setting the "type" parameter to ratio. Because the template is derived from the data itself there is no need to use RMPUPIL to iteratively determine a scale factor. The "output" parameter is the corrected flat field. The corrected narrowband flat field will show some artifacts from fine structure in the pupil image. However, a large fraction of the pupil image will be removed. Later reduction steps of applying a sky flat field and combining with dithering further eliminate effects of this approximate solution to the pupil image. 4.2.4 OBJECT EXPOSURE REDUCTIONS At this point you will have some subset of combined zero level, dark count, and flat field calibration data. The calibration data is applied to the object exposures, either in bulk or as observations are completed, using the task CCDPROC. The command is simply ms> ccdproc <files> 4.2.5 PUPIL IMAGE REMOVAL FROM OBJECT DATA The pupil ring image in the object exposures is removed by subtraction since this is excess light. Again this is only required for data where the pupil image occurs, such as from the Mayall telescope. The tasks for modeling and removing the image are the same as for removal from the flat field except that the "type" parameter is set to "difference". 4.2.5.1 BROADBAND DATA Probably the best subtraction will be obtained by using the pupil image template from a narrowband flat field. This would be the same as used for the flat field and extracted from the narrowband flat field using MSCPUPIL with "type = data". The subtraction is carried out using RMPUPIL with "type = difference". An alternative, since the pupil image is weak and the fine structure is unimportant, is to use MSCPUPIL with "type = difference" to determine a smooth large scale ring pattern and subtract it from the data. The iterative sigma rejection and the "datamin" and "datamax" parameters are used to eliminate smaller scale astronomical objects in the field from affecting the background fits and the ring profile fits. For this application the "abin" parameter should be set to a value such as 30 degrees and the "astep" parameter to a smaller value such as 5 degrees. The main advantage of this method is that no iterative scaling is required since the fit is done directly to the data. The difficulty, though, is if there is a bright star or fairly extended object, particularly in the inner background ring, then the fit will be poor and the subtraction will show gross artifacts. The last alternative, and the one to use if there is no narrowband flat field for the template and the field has bright stars which affect fitting directly to the data, is to make a "sky flat" to generate the pupil image template. This is done as described in the section for creating a sky flat. Once the sky flat is created with the pupil image then MSCPUPIL is used to separate the pupil image from the background and RMPUPIL is used to scale and subtract the image from the object exposures. Note that after the pupil image is subtracted then a new sky flat should be created. 4.2.5.2 NARROWBAND DATA For narrowband data the two alternatives described for the broadband data are used. The first is to fit and subtract a smooth ring model from each object exposure using MSCPUPIL. This is the same as described for removing the pupil image from the flat field except the "type" parameter is set to difference. The second is to create a sky flat from disregistered exposures, extract the pupil pattern with MSCPUPIL, and then subtract it from each object exposure using RMPUPIL. 4.2.6 DARK SKY OR TWILIGHT SKY FLAT FIELDS You will notice that there are two flat field corrections which can be performed by CCDPROC. The first one is for an initial flat field such as the dome flat obtained at the beginning of the night, a standard flat field from a previous night or run, or a final combined dome flat and sky flat from some other night or run. The second is for a dark sky or twilight sky flat field prepared from the object exposures after they have been calibrated with the first flat field. Sky flat fields are created by combining object exposures with objects removed by using data in each pixel that is only sky. In principle one could use exposures of the twilight sky but our experience is that these do not work well. You are welcome to take some exposures and try using them. We have found that dark sky flat fields derived from the object exposures do work quite well. Mosaic observations already typically dither a field. One will do even better by combining observations from other fields. The more data used the better the resulting sky flat will be. The main criterion for including data is to avoid observations contaminated by varying background light from the moon or scattered light from bright stars off the field. Of course, another factor that has to be considered is whether a field has a very large extended object which appears in many of the observations. These will not be useful. The sky flat field is created using the task SFLATCOMBINE with parameters selected to reject objects appearing above a median. We don't have much experience with creating sky flats currently so some experimentation with parameters may be required. Below is one possibly set of parameters. I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = sflatcombine input = List of images to combine (output = Sflat) Output sky flat field root name (combine= average) Type of combine operation (reject = avsigclip) Type of rejection (ccdtype= object) CCD image type to combine (subsets= yes) Combine images by subset parameter? (scale = mode) Image scaling (statsec= ) Image section for computing statistics (nkeep = 1) Minimum to keep (pos) or maximum to reject (neg) (nlow = 1) minmax: Number of low pixels to reject (nhigh = 1) minmax: Number of high pixels to reject (mclip = yes) Use median in sigma clipping algorithms? (lsigma = 6.) Lower sigma clipping factor (hsigma = 3.) Upper sigma clipping factor (rdnoise= rdnoise) ccdclip: CCD readout noise (electrons) (gain = gain) ccdclip: CCD gain (electrons/DN) (snoise = 0.) ccdclip: Sensitivity noise (fraction) (pclip = -0.5) pclip: Percentile clipping parameter (blank = 1.) Value if there are no pixels (grow = 3.) Radius (pixels) for neighbor rejection This task is a combination of CCDPROC to first process the images, if they have not previously been processed, and COMBINE to combine the offset images with rejection of object pixels. A new feature of this task is the "grow" parameter which now provides a two dimensional circular rejection of pixels around pixels rejected by the rejection algorithm. Whatever rejection algorithm is used it is likely that the best results will be when the clipping sigmas are non-symmetric as shown above. Note that a very low rejection threshold or very large grow radius will make the task quite slow. After producing a good sky flat that has no evidence of objects it may be applied directly to the data by using it as the second flat field correction. ms> ccdproc <files> sflatcor=yes sflat=Sflat* Note that the object exposures used in creating the sky flat will already have been processed except for the application of the sky flat so CCDPROC will only apply the sky flat field calibration. The sky flat field includes corrections at all scales from pixel-to-pixel sensitivity variations to large scale illumination differences. If the signal-to-noise is poorer than the dome flat field you might wish to apply a filtering/smoothing operation to the sky flat data thus relying on the dome flat field for the pixel-to-pixel sensitivity calibration and the sky flat field for larger scale illumination corrections. There are a number of filtering tasks in IRAF. A median is a good filter and there is the choice of a ring median or box median. To apply one of these general filtering tasks you would use MSCCMD to run it on all the CCDs ms> msccmd msccmd: median $input $output 10 10 Input files: SflatV Output files: SflatMedV msccmd: q Because the object exposures are first processed with the dome flat (or other flat field) you would normally run CCDPROC again on the data using the sky flat and any observations that have not been processed at all will use both the dome flat and the sky flat. However, if you want to make a single flat field to apply to raw data, say if starting over or using it for a second night, you can combine the two flat field corrections into a single flat field to be used as the only flat field correction. This is done by multiplying the two flat fields using MSCARITH ms> mscarith FlatV * SflatV FinalflatV 4.2.7 THE VARIABLE PIXEL SCALE AND ZERO POINT UNIFORMITY A key assumption in the traditional reduction of CCD images is that the pixel scale is uniform and that a properly reduced blank sky image will have a uniform and flat appearance. Unfortunately, this is not correct when the pixel scale varies over the field. In the case of the NOAO Mosaic at the Mayall telescope, the pixel scale decreases approximately quadratically from the field center, with the pixels in the field corners being 6% smaller in the radial direction, and 8% smaller in area. Pixels in field corners thus would properly detect only 92% of the sky level seen in the field center, even with uniform sensitivity. At the same time the same number of TOTAL photons would be detected from a star regardless of how many pixels the PSF would be distributed over. Forcing the sky to be uniform over the image has the deleterious effect of causing the photometric zeropoint to vary from center to field corners by 8%. Note that this effect is different from vignetting where the flux actually delivered to the image margins is less than that at the center, an effect that IS corrected by the flat field. In practice, the photometric effect of the variable pixel scale can be ignored provided that the reduced images will be part of a dither-sequence to be stacked later on. As discussed below, prior to stacking the images they first must be re-gridded, which produces pixels of essentially constant angular scale. This is done with the MSCIMAGE task, which re-grids the pixels and has a "flux conservation" option that can scale the pixels photometrically by the associated area change. If this function is disabled, then "improperly" flattened images will have a uniform zero point restored. In short, the flat field adjusted (if inappropriately) for the different pixel sizes, so MSCIMAGE would then do no further adjustment. Stars would be too bright in the corners of the flattened images, but after re-gridding, their total fluxes would be seen to be scaled down to the appropriate values. If the mosaic CCD images are to be analyzed individually, as might be done for standard star fields, then after the flat field reductions are complete the differential scale effects must be restored. At present we are developing a routine in the MSCRED package to do this, without actually re-gridding the image. The correction process is simple; the scale at any point in the Mosaic field is already known from the astrometry so one just calculates and multiplies by the correction. The final image would appear to have a variable sky level, but would be photometrically uniform. 4.3 COORDINATE CALIBRATION For some projects the basic flux calibrated CCD exposures may be all that is required. However, if you want to obtain coordinate information or combine multiple exposures which are dithered on the sky or taken with different filters, you must calibrate the celestial world coordinate system (WCS) of the data. This may be done in an absolute or relative sense; an absolute calibration ties the data coordinates to catalog coordinates while a relative calibration ties multiple exposures to the same coordinates. Determining the WCS from scratch is a complicated business and requires special observations of astrometry fields. However, for NOAO Mosaic data a standard coordinate calibration determined earlier is automatically inserted into your data by the data capture agent. The default coordinate system is sufficiently accurate for most purposes and just requires some small adjustments as described below. To piece a single exposure into a single image that does not require registration to any other data you may use the default WCS and skip the WCS calibration steps. The WCS is a mapping from pixels in the mosaic data to celestial coordinates relative to a reference point on the sky. The reference point, or zero point, is set using the telescope pointing coordinate. The telescope pointing is generally off by a small amount, though it could be completely wrong in some hardware/software error situations. In addition, differential atmospheric refraction introduces small axis scale changes and rotations, which are significant due to the large field of view of the mosaic even during the course of single set of dithered exposured. Putting observations from different filters onto the same coordinate system also requires mapping small scale changes, since currently there is only a single standard WCS solution derived through one filter. [In the future filter dependent solutions will be made available.] The WCS calibration operations consist of adjusting the standard coordinate system calibration to a desired zero point and applying small axis scale changes and rotations. This is done using objects (usually stars) in the exposures. Unlike a full WCS calibration, which requires a high density of stars with accurate catalog coordinates, the adjustments to the default WCS calibration require only a few objects; only one objects is needed to provide a zero point correction. The WCS adjustments are determined by specifying coordinates for one or more objects in the data. The coordinates can be obtained from a reference catalog or, more commonly, by measuring coordinates from one reference exposure to which other exposures are to be "registered". A combination of using a catalog coordinate for one object in the field to set the zero point in a reference exposure and then measuring the positions of other stars in the reference image based on that zero point calibration may also be done. The two tasks you will use are MSCZERO and MSCCMATCH. MSCZERO is used to interactively set the zero point of the coordinates, register multiple exposures closely, and generate a list of coordinates in a reference exposure to which other exposures in a dither set are registered. MSCCMATCH finds objects at the positions specified by a list of coordinates and determines corrections for the zero point, axis scale change, and axis rotation. 4.3.1 SETTING COORDINATE ZERO POINTS AND MEASURING COORDINATES MSCZERO is an interactive display task for mosaic exposures that allows measuring coordinates and adjusting the WCS zero point. The task parameters are shown below. The last set of parameters (starting with "ra") are for the task to query and maintain lists. I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = msczero images = List of mosaic exposures (nframes= 2) Number of frames to use (logfile= default) Log file for measurements ra = RA (hours) dec = DEC (degrees) update = yes Update WCS zero point? (fd1 = ) (fd2 = ) The task displays each exposure in the list, in turn, and responds to cursor key commands. You can go forward and backward through the input list or quit at any point. The exposures are displayed by cycling through the specified number of frames starting with the first frame. As an aid to efficiency, if the exposure is already loaded in the appropriate frame then the display step is skipped. This task has several uses (type '?' to get the list of command options): 1. Set the WCS zero point by specifying the coordinate of a star. 2. Create a list of coordinates for use with MSCCMATCH and MSCIMATCH. 3. Report coordinates at the cursor position. It may be that the WCS zero points, based on the telescope pointing coordinates, are accurate enough that you can use this task on only a reference exposure to generate a list of coordinates for use with MSCCMATCH and MSCIMATCH. However, because it is fairly quick to explicitly check and set the zero point of all the exposures in a dither set to the same coordinate for a common reference star, it is recommended you do this first. To check and set the zero points for a set of dithered exposures run MSCZERO with a list of the exposures ms> msczero @field1 After the first exposure is displayed either find a reasonably bright unsaturated star which will be in all the exposures or find a star whose coordinate is known from a catalog such as the HST Guide Star Catalog. Move the cursor to the star and type 'z' (zero) to invoke a centering algorithm. Note that even though the exposure may be displayed at lower resolution the centering is done with the full resolution data. The task will then tell you what it thinks the coordinate is and ask you for a new coordinate. The first time 'z' is typed it will prompt with the measured coordinate and thereafter it will prompt with the last entered value. If you are referencing all the exposures to the first exposure in the list accept the measured coordinate (and write the value down in case you need it later) otherwise enter the desired coordinate. Note that all further measurements of the image will automatically apply the measured zero point correction but the exposure WCS is not actually updated until you type 'n' (next) or 'q' (quit). If you want to print coordinates without changing the zero point correction use the space bar or 'c' (center) to center on an object and print the centered coordinate. If you changed the WCS zero point you will be shown the zero point offsets and given the option to update the WCS in the data file when you type 'n'. Then the next exposure in the list will be displayed. Find the same star and type 'z' again. Since it will retain the last entered coordinate you should only need to accept the prompted coordinates. When you have done this for all the exposures their coordinate systems will be registered at least at that point. The WCS in the dither set may still not be registered over all the field due to refraction effects. Also the intensity scales of the dithered exposures may not be the same due to changes in transparency and sky brightness. These effects are calibrated by matching objects throughout the field in position and brightness. This requires a list of coordinates tied to one of the dithered exposures as a reference. Usually the first exposure in the set is used as the reference. MSCZERO is used to create a list from objects in the reference exposure. ms> msczero obj021 Select objects, usually stars, throughout the field and type 'x' for each one. This will center on the object and and record the coordinate in a logfile. The default logfile name "default" creates a log file beginning with "Coords." and followed by the name of the exposure. In the example this will be "Coords.obj021". To be useful for coordinate matching this list should have a good number of stars, say three or four from each CCD, with emphasis on the field edges but allowing for the dithering. For the intensity matching you want to have stars with a range of brightness (though not saturated or extremely faint) and which are mostly isolated so that a region around them may be used for sky. The lists for the coordinate and intensity matching do not have to be the same but it is reasonable to just create one list. 4.3.2 MATCHING COORDINATE SYSTEMS The task MSCCMATCH determines and applies a linear correction to the WCS to match objects, generally stars, in an exposure to a set of reference celestial coordinates. This correction maintains the detector geometry and optical distortions while adjusting for changes in apparent sky position such as produced by atmospheric refraction and telescope pointing errors. The linear correction consists of a zero point shift, scale changes in the right ascension and declination axes, and rotations of the axes. To use this task you need a list of reference celestial coordinates, right ascension in hours and declination in degrees, and the mosaic exposure coordinate system must be relatively close to the reference coordinate system. The default WCS plus telescope pointing may be close enough, but if not you would use MSCZERO to register the zero points at some point in the exposures. Since it is relatively simple to register a set of dithered exposures to a common star with MSCZERO this is recommended procedure before using MSCCMATCH. The reference coordinates should cover all of the mosaic field of view to be sensitive to the small rotation and scale effects. The coordinate list might be obtained from a catalog or measured from one of the exposures to which other overlapping exposures will be matched. For the purposes of making a well aligned stacked image from a set of dithered exposures one generally uses one of the exposures as the source of the reference coordinates. MSCCMATCH operates on a set of input mosaic exposures; each in turn. For an exposure it converts each input celestial coordinate to a pixel coordinate in one of the extensions using the current WCS. If the coordinate does not fall in any extension the coordinate is not used. The pixel coordinate is used as a starting point for the APPHOT.CENTER task. If the centering fails for some reason, such as the object being too near the edge or the final position being too far from the initial position, the coordinate is not used. For those objects successfully found a fit is made between the original celestial coordinates and the measured coordinates expressed as arc seconds from the exposure tangent point. The fit is constrained to yield some combination of shift, scale change, and rotation for each of the celestial coordinate axes. These parameters are then used to update the exposure WCS so that the adjusted measured coordinates best agrees with the reference coordinates. The task parameters are shown below. I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = msccmatch input = List of input mosaic exposures coords = Coordinate file (ra/dec) (nfit = 4) Min for fit (>0) or max not found (<=0) (rms = 2.) Maximum fit RMS to accept (arcsec) (maxshif= 5.) Maximum centering shift (arcsec) (fitgeom= rxyscale) Fitting geometry (update = yes) Update coordinate systems? (interac= yes) Interactive? (fit = yes) Interactive fitting? (verbose= yes) Verbose? accept = yes Accept solution? The input is a list of mosaic exposures and a file of reference celestial coordinates. The exposures should all include a significant number of objects from the list of coordinates. The task can be run interactively or non-interactively based on the "interactive" parameter. In interactive mode you can graphically interact with the fitting (selected with the "fit" parameter) and accept or reject a fit based on the printed fit parameters. The fitting is done using the task GEOMAP and the interactive mode allows you to view the distribution of coordinates, residuals verses the input coordinates, delete bad values, and possibly change the fitting constraints (see the help for GEOMAP for more information). The linear transformation may be constrained by the "fitgeometry" parameter as described in the help for GEOMAP. This may be desirable if there are only a few coordinates or if you want to impose some physical assumption. Note that the effects of atmospheric refraction actually do cause independent scale changes and rotations in the two axes so the default "rxyscale" should be used. There are some constraints which are placed on the task. The "maxscale" parameter limits how far the objects may be found from the initial coordinates. This constraint protects against incorrect identifications and tells the centering routine how much of the image to look at. This parameter should be as small as possible consistent with the errors in the WCS. If you first zero the coordinates then the objects should be found quite close to the initial coordinates. When the "verbose" parameter is set the results of the centering will be printed consisting of the image extension name, the final pixel coordinates, the shift in pixel coordinates from the initial value, and the formal uncertainties in the pixel coordinates. If an error occurs one of the error codes from APPHOT.CENTER will be reported such as "BigShift" for objects with too big a shift from the initial position and "EdgeImage" for objects to near the edge of the image. The "nfit" parameter requires a certain number of coordinates to be included in the fit. If specified as a negative number the parameter is interpreted as a maximum number that may be lost from the input list due to being off the exposure or failing to be centered. The "rms" parameter requires that the final RMS of the residuals about the fit for each axis be less than a certain value. 4.4 PUTTING THE PIECES TOGETHER This section tells you how to make single images from each multiextension exposure and how to combine sets of dithered images into a final deep image free from gaps and artifacts. Obtaining good results depends on having well-flattened data, a uniform sky, a dither pattern that samples the gaps and bad regions of the detectors, and accurately registered world coordinates. Most difficulties are caused by variable sky conditions or scattered light within a dither sequence or the data used to create a sky flat. 4.4.1 REMOVING SKY GRADIENTS Any sky level mismatches when combining dithered exposures produce artifacts in the final image. The three sources of such mismatches are sky gradients, sky level differences between the CCDs, and sky level differences between exposures. While the flat field calibration, particularly with a sky flat, should remove differences in sky levels between CCDs, in practice there may still be small errors. And the flat field will not deal with sky gradients across the large field of view. Exposure-to-exposure sky brightness variations can be dealt with at a later stage but even this is tricky. The best final result is obtained by fitting a low order surface (a plane or quadratic) to the sky and subtracting it from each CCD of each object exposure at this stage. This will force the sky to be zero for all CCDs and all exposures. Note that if one wants to preserve a sky level for statistical reasons it is possible to add a uniform constant after the subtraction to all the data (or add the constant to the final dither stacked image). To fit and subtract a sky and sky gradient the combination of IMSURFIT and MSCCMD is used. With IMSURFIT use the option to fit to medians in large blocks to remove the effects of objects. ms> msccmd msccmd: imsurfit $input $output xo=2 yo=2 type=resid xm=100 ym=100 Input files: obj* Output files: obj* msccmd: q In this example the input and output are the same, replacing the original by the sky subtracted data, but one can create new output files if desired. Note that x and y orders of 2 correspond to a plane and orders of 3 correspond to a quadratic surface. 4.4.2 CONSTRUCTING SINGLE IMAGES Making a single image from a mosaic exposure is done by mapping the pixels from each extension to a single uniform grid on the sky. The WCS calibrations described in previous sections provide this. For making a single image from a single exposure the WCS calibration is not critical and the default WCS is sufficient. For combining multiple dithered exposures all the exposures must be registered to a common coordinate system, either relative to one reference exposure or to a set of catalog stars, and each exposure must be resampled to the same final coordinate system. The task that makes single images from mosaic exposures is MSCIMAGE. Its parameters are shown below. I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = mscimage input = List of input mosaic exposures output = List of output images (referen= ) Reference image (pixmask= yes) Create pixel mask? (verbose= )_.verbose) Verbose output? # Resampling parameters (blank = 0.) Blank value (interpo= linear) Interpolant for data (minterp= linear) Interpolant for mask (fluxcon= no) Preserve flux per unit area? (ntrim = 7) Edge trim in each extension (nxblock= 2048) X dimension of working block size in pixels (nyblock= 1024) Y dimension of working block size in pixels # Geometric mapping parameters (interac= no) Fit mapping interactively? (nx = 10) Number of x grid points (ny = 20) Number of y grid points (fitgeom= general) Fitting geometry (functio=chebyshev) Surface type (xxorder= 4) Order of x fit in x (xyorder= 4) Order of x fit in y (xxterms= half) X fit cross terms type (yxorder= 4) Order of y fit in x (yyorder= 4) Order of y fit in y (yxterms= half) Y fit cross terms type An output image is created for each input mosaic exposure. The output image is created with a coordinate system defined by the specified "reference" image. If no reference image is specified then the first input mosaic exposure is used to define a simple tangent plane coordinate system with optical distortions removed, and that coordinate system is used for all the input mosaic exposures. The important point is that for a set of dithered exposures all the output images must be created with the same coordinate system grid so that they may be combined by simple integer shifts along the image axes. The normal usage is to specify all the mosaic exposures in a dither set as the input, give a matching list of output images, and leave the reference image unspecified. If all the exposures in a dither set are not done at the same time then you must specify one of the earlier output images as the reference image to continue to create the output images on the same coordinate grid. The output images are created with a size that just covers the input data and initially filled with the specified "blank" value. This is the value that the mosaic gaps will have in the final output image. Then each extension is resampled into the appropriate part of the output image. The coordinate mapping is generated by GEOMAP using the geometric mapping parameters which you don't need to change. The resampling is done with the specified interpolation function. The small rotations in the CCDs produce edge effects in the interpolated output pieces so a small trim is required to eliminate these. [At the time this document was prepared the best value for the new science grade NOAO Mosaic had not been determined.] Linear interpolation is the fastest and most straightforward. Other interpolation functions are available. In particular sinc interpolation is now available as an add-on option (see the MSCRED installation instructions). Experience with sinc interpolation shows that it is not overly slow and does provide improved results; particularly with maintaining the statistical characteristics of the sky noise. The "minterpolant" parameter allows using a faster and more local interpolation function for the mask. This is particularly useful when using sinc interpolation of the data to allow flagging only around the actual bad pixels and not extending out as far as the sinc interpolation does. It is useful for the later combining step to make bad pixel masks that reflect the interpolation and resampling from the input data. These may be created by setting the "pixmask" parameter. If this parameter is set and the input mosaic data have bad pixel masks defined through the header BPM keywords (default bad pixel masks are provided in the NOAO Mosaic data) then the masks will be interpolated in exactly the same way as the data. The interpolated masks will appear in the working directory with names related to the output image names and with the output images containing the BPM keyword pointing to these masks. The input bad pixel masks are assumed to have zero for good data and one for bad data and the output masks have zero for good data and values between zero and ten thousand for bad data. The value is the result of interpolation and reflects the relative contribution of good and bad data. The "fluxconserve" parameter applies a pixel area correction if selected. As discussed earlier, standard flat fielding distorts the flux per unit area in pixels of different projected size by making them have the same flux per pixel. In effect this applies half of the flux conservation operation by adjusting the pixel values without adjusting the pixel sizes. MSCIMAGE does the second half by adjusting the pixel sizes. So for standard flat fielded data, the usual route to making a combined dithered image, the flux conservation parameter should not be used to arrive at a proper final flux per unit area in the resampled data. Flux conservation would only be used if the input mosaic data has previously been corrected back to proper flux per unit area through adjustment of the flat field or data for the variable pixel size inherent in the mosaic coordinate system. Below are two examples; one using prepared @files and one illustrating advanced usage of filename templates. ms> mscimage @dither1 @outdither1 pixmask+ ms> mscimage obj02![2-5]* %obj%mos%02![2-5]* pixmask+ In the second example the input template expands to obj022.fits to obj025.fits and the output template matches the input template using the first part of the %% substitution field and then replaces the "obj" with "mos" to give output images mos022.fits to mos025.fits. 4.4.3 MATCHING INTENSITY SCALES When stacking dithered exposures (the single images created in the previous step) to fill in the mosaic gaps and remove bad pixels and cosmic ray events it is critical that the intensity scales of the images match. Otherwise you will see artifacts from the gaps, places with bad data, and around objects as the combined intensity level jumps when data from an exposure is missing or rejected. Also the rejection algorithms require that the image intensities match both at the sky level and in the objects. There are two parameters that must be determined to match the intensity scales. One is a additive offset caused by sky brightness variations. The second is a multiplicative scale change caused by transparency and exposure time variations. Matching the intensity scales for a set of dithered exposures consists of determining values for these two scaling parameters relative to a reference exposure and setting them in the image headers. The actual adjustment of the pixels values occurs when stacking the exposures. The intensity matching values are determined by the task MSCIMATCH. The task parameters are shown below. I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = mscimatch input = List of images coords = File of coordinates (scale = yes) Determine scale? (zero = no) Determine zero offset? (box1 = 21) Inner box size for statistics (box2 = 51) Outer box size for statistics (lower = 1.) Lower limit for good data (upper = INDEF) Upper limit for good data (niterat= 3) Number of sigma clipping iterations (sigma = 3.) Sigma clipping factor (interac= no) Interactive? (verbose= yes) Verbose? The input is a list of images to be matched and a file of celestial coordinates (RA in hours and DEC in degrees) to use in computing the matching parameters. The input images are the single images constructed from the mosaic exposures for a set of dithered observations. The parameters "scale" and "zero" select whether to determine the multiplicative scale, the zero level offsets, or both. If the sky has been subtracted at an earlier stage (as recommended) then only the multiplicative scale difference needs to be determined. The advantage of subtracting the sky earlier is that scale determination becomes better constrained. Also determining the sky from photometry (as done by this task) is less robust than the surface fitting which uses all of the sky data. The scaling parameters are determined by measuring the mean flux in a set of matching regions between each input image. The centers of the regions are specified by their celestial coordinates. The list of coordinates should consist of the positions of objects in the field. These objects should span a range of brightness. As noted previously, you would normally use the same coordinate list as used with MSCCMATCH, which is generally obtained using MSCZERO. However, you can use any IRAF task that produces a list of celestial coordinates from images with a WCS. One possibility is to use RIMCURSOR on one of the displayed single images with the "wcs" parameter set to "world" and the "wxformat" set to "%.2H" to produce right ascension values in hours instead of degrees. The now accurately aligned coordinate systems in the images are used to identify the matching pixel coordinate center in each image. The regions to be measured consist of square boxes of the specified sizes about the pixel coordinate center. There are two boxes, an inner box and an outer box which excludes the inner box. The box sizes are intended to define photometry apertures for the objects and nearby background. It is not critical that they exactly fit the objects or that the objects necessarily be stars but this is usually how they will be set. Because of possible PSF variations the inner box should be large enough include all the light from stars over the whole data set. If the inner box is not fully contained in the input or reference image that box is not used for that pair. Similarly the outer box must be fully contained in the images but if only the outer box is outside one or both images the measurement for the inner box may still be used. In order to exclude regions that include the gaps or bad data in one or both of the pair of images all pixels in a box must have values between the specified good data limits. Those regions with values outside the limits are eliminated from the intensity matching. The mean fluxes in each region are used to simultaneously fit the relations mean_j = A_ij + B_ij * mean_i for all i and j where i and j are any pair of images. These equations are constrained by the fact that the scaling from image i to j, followed by the scaling from image j to k, must agree with the scaling from image i to image k. The final scaling coefficients reported and stored in the image header are A_1j and B_1j, which correspond to the scalings to the first image in the input list. The task will attempt to reject photometry points which are discrepant. If the task is run interactively it will also show plots of the photometry flux in one image verses another. It does this for sequential pairs of images. Points can be deleted in these plots and they will be excluded from the data used to determine the scaling parameters. When the task is done determining the scaling factors they will be printed and a prompt issued to accept or not accept the results. If the scaling parameters are accepted then the keywords MSCZERO and MSCSCALE are recorded in the input image header when the "update" parameter is set. Note that the reference image is assigned values of 0 and 1 for these header keywords. 4.4.4 MAKING THE FINAL STACK IMAGE After MSCIMAGE produces single images of each of the dithered mosaic exposures with a common coordinate system grid, a final image is created with the task MSCSTACK. The task MSCIMATCH is generally used to match the intensity scales of the images before this step as described in the previous section. However, for quick reductions or for other reasons the images may be stacked either with no intensity matching or using the "scale" and "zero" options of MSCSTACK. The task parameters are shown below. I R A F Image Reduction and Analysis Facility PACKAGE = mscred TASK = mscstack input = List of images to combine output = Output image (plfile = ) List of output pixel list files (optional) (combine= median) Type of combine operation (median|average) (reject = none) Type of rejection (masktyp= none) Mask type (maskval= 0.) Mask value (blank = 0.) Value if there are no pixels (scale = !mscscale) Image scaling (zero = !msczero) Image zero point offset (weight = none) Image weights (statsec= ) Image section for computing statistics (lthresh= 1.) Lower threshold (hthresh= INDEF) Upper threshold (nlow = 1) minmax: Number of low pixels to reject (nhigh = 1) minmax: Number of high pixels to reject (nkeep = 1) Minimum to keep (pos) or maximum to reject (neg) (mclip = yes) Use median in sigma clipping algorithms? (lsigma = 3.) Lower sigma clipping factor (hsigma = 3.) Upper sigma clipping factor (rdnoise= 0.) ccdclip: CCD readout noise (electrons) (gain = 1.) ccdclip: CCD gain (electrons/DN) (snoise = 0.) ccdclip: Sensitivity noise (fraction) (sigscal= 0.1) Tolerance for sigma clipping scaling corrections (pclip = -0.5) pclip: Percentile clipping parameter (grow = 0) Radius (pixels) for 1D neighbor rejection This task is a simple variant of COMBINE that registers the images using the coordinate systems and has the default threshold parameters set to ignore values below one DN based on the default "blank" value in MSCIMAGE for the gaps. If you have also generated bad pixel masks for the resampled images you can exclude them as well by setting "masktype" to "goodvalue". The real art in using this task is deciding how to scale and reject bad data not covered by the bad pixel masks. A "combine" of "median" is the simplest but it does not optimize the signal-to-noise for the number of images. If you "average" the data you will probably want to apply a rejection algorithm such as "avsigclip". Careful flat fielding will make each separate image have the same sky level across the different CCDs. However, the sky levels and transparency may still be varying from exposure to exposure. If you simply combine such data you will see imprints of the gaps. So it is generally a good idea to scale the images. This is done using the "scale" and "zero" parameters which can be set to header keyword values, files containing the values, or special values to compute image statistics in a particular region of the data. The recommended method for scaling is to use the intensity matching task MSCIMATCH described in the previous section and use the image header keywords MSCSCALE and MSCZERO produced by that task. An example of using this task to create a final image is given below. ms> mscstack @field1 Field1 combine=average rej=avsigclip
IRAF Project
National Optical Astronomy Observatories
950 North Cherry Avenue
Tucson, Arizona 85719
(602) 327-5511