![[NOAO logo]](/iraf/web/images/noao.gif)
![[IRAF logo]](/iraf/web/images/iraf.gif)
NOAO is operated by the Association of Universities for Research in Astronomy (AURA), Inc. under cooperative agreement with the National Science Foundation
Francisco Valdes, May 8, 2000 (2_1 <-> 1_2 March 20, 2007)
This document defines the basic data structures for NOAO image data. Image data is that produced by regular array detectors such as CCDs and infrared devices. The data structures are created by the data acquisition systems for use by observers, data reduction systems, and archives.
The general definitions given here are intended to be applicable to both optical CCD data and infrared array data. However, the current version of this document only provides detector specifics for CCD data.
The information content and FITS keywords for the image headers is defined through the document Classes Describing Astronomical Observations and the associated Master NOAO FITS Keyword Dictionary.
It is almost universal that astronomical image data from array detectors are recorded as FITS image files; at least for data interchange and archiving. There is also a trend to adopt FITS image files as the data format for disk storage and access. At NOAO FITS image files are used by the Save-The-Bits system, is the recommended user tape format, and is a new IRAF supported disk format. Therefore, the basic image data structures created by future data acquisition systems will be FITS image files.
The basic FITS image format consists of an ASCII "header" and a optional binary "data" array. The combination is called a Header and Data Unit (HDU). The header contains comments and individually identified pieces of information about the data. Multiple HDUs may be combined in a single file using the FITS Image Extensions format.
For CCD data the basic image unit corresponds to data from a single amplifier. CCD observations taken with a single amplifier will be recorded as a single simple FITS image file with one HDU. Observations made with multiple amplifiers and/or multiple CCDs in a "mosaic" of CCDs will be recorded as multiple HDUs in a single file using the Image Extensions format.
When a single observation consists of data from multiple amplifiers and CCDs much of the header information is the same for all amplifiers. The common header information will be recorded in the first header, called the primary header unit (PHU). This header will have no data array associated with it. The following headers, called extension header units (EHU), and data arrays for each amplifier will then contain only information relevant to that amplifier. The logical header for each amplifier is then the combination of information from the PHU and the individual EHU. If an individual amplifier image is extracted from a file with multiple images then the common header keywords will be merged into the new image file.
The connection between individual image units from various amplifiers and CCDs for a single observation is provided in two ways. The primary connection is a unique observation identification that is common to all the image units. The second connection is through the grouping of the image units in a single FITS Image Extensions file by the data acquisition system. However, this is a less fundamental connection since the individual units may be extracted into separate image files at some later time.
Within a set of data from the same observation (all having the same
observation identification) each image unit has a unique identification
number. For CCD data this would be a number identifying each amplifier
uniquely in the detector. Thus data from an observation consists of all
image units with the same observation identification with the image
identification identifying and distinguishing each image. The FITS
keywords that implement this identification information are discussed in section 4.1.
The geometrical (pixel raster) relationships between the various image
units is defined in the image headers by specification of parent raster
sizes, the regions within this raster covered by the subunits, and linear
coordinate transformation between the various pixel coordinate systems. In
a mosaic of CCDs there are three levels. The full mosaic is described by a
logical pixel raster in a "detector" coordinate system of a specified size,
say 8192x8192 for a 4x2 mosaic of 2048x4096 CCDs. The size of the logical
detector raster could also be defined to include gaps between the CCDs, as
a number of missing pixels, but it must be rectangular.
At the next level each CCD header specifies a rectangular region in a
"CCD" coordinate system within the higher level mosaic raster which it
covers. The CCD header also defines its own logical pixel raster size.
This is generally the physical size of the CCD in pixels though for drift
scanning it may be defined to be bigger in the parallel transfer
direction. At the next and final level each amplifier header specifies a
rectangular region that it covers within the CCD raster. Because multiple
amplifiers read the CCD data in different orders, as described in an
"amplifier" coordinate system, there will be relative flips between the
different amplifiers though the pixels may be flipped when they are written
to an image raster by the data acquisition system . The description of the
regions covered by each piece includes specification of the flips so that
the piece is correctly mapped to the higher level pixel raster. This
scheme of specifying sizes and regions of the higher level raster covered
can be extended to greater or lesser depths; i.e. a mosaic of mosaics or a
single CCD with multiple amplifiers.
The pixel raster description can be used by software to piece together
a single image for display and quick-look analysis. For a single CCD with
multiple amplifiers the single image will be geometrically correct with
respect to the CCD array. However, for a mosaic of CCDs it will not
generally be correct due alignment errors between the CCD chips. The
alignment corrections are left to later data reduction and analysis
software.
In this section we define four "pixel" coordinate systems called "CCD"
"amplifier", "image", and "detector". We also define coordinate
transformations between the CCD coordinates and the other three.
All the pixel coordinate systems are continuous with integer coordinate
values for the centers of the pixels. The integer coordinate values then
relate to the raster array indices of the CCD or image.
The CCD coordinate system (Cx,Cy) is defined in terms of the individual
charge wells or "unbinned" pixels. The CCD pixel coordinates start
with one for the first charge well read and run to the maximum number of
pixels which can be read. Note that the maximum might be larger than the
physical size of the CCD for the case of drift scanning. Overscan
(postscan) dummy reads will have CCD coordinates greater than the maximum
and prescan pixels will have CCD coordinates less than one. The choice of
CCD corner defining the origin is arbitrary (except for the case of drift
scanning). However, for a mosaic of CCDs it is recommended that all the
CCD coordinates have the same orientation relative to an image focal plane
regardless of which amplifier is used.
The CCD coordinate system is important for specifying regions of interest
(ROI) and matching these regions in observations to full format calibration
data (such as zero level, dark count, and flat field images), bad pixel
masks, and distortion and world coordinate maps.
The amplifier coordinate system (As,Ap) is based on the order in which
pixels are read. The the serial coordinate is As and the parallel
coordinate is Ap. The coordinates begin with one for the first CCD pixel
(actual charge well) that the controller records or uses in binning and
increments by one for each "unbinned" pixel. If the controller always
skips some initial columns or lines which are never used (such as for
transient settling) then these are not counted. Prescan pixels which are
recorded with the image data will have amplifier coordinates less than
one. The general linear transformation between the amplifier and CCD
coordinates is given by
2.2 Pixel Geometry for Multiple Raster Data
2.3 Pixel Coordinate Systems and Transformations
The CCD Coordinate System
The Amplifier Coordinate System
As = ATM1_1 * Cx + ATM2_1 * Cy + ATV1
Ap = ATM1_2 * Cx + ATM2_2 * Cy + ATV2
Cx = ( ATM2_2 * (As - ATV1) - ATM2_1 * (Ap - ATV2)) /
(ATM1_1 * ATM2_2 - ATM2_1 * ATM1_2)
Cy = (-ATM1_2 * (As - ATV1) + ATM1_1 * (Ap - ATV2)) /
(ATM1_1 * ATM2_2 - ATM2_1 * ATM1_2)
where the ATM values are the amplifier transformation matrix and the
ATV values are the amplifier transformation vector. The transformation
coefficients are recorded in the image header with keywords matching the
above terms. The main purpose of maintaining the amplifier coordinate
system transformation is to allow determining the readout order of
the pixels, particularly when different amplifiers may be used. The
order of the pixel readout is important in defining how pixel binning
is done.
The image coordinate system (Ic,Il) refers to the recorded image pixel array. The image array may include other information such as overscan and prescan. Thus the actual data pixels from the CCD may occupy only a region of the image. However the image coordinate system begins with (1,1) for the first pixel in the image. The first data pixel will begin with some coordinate (Ic1,Il1). The data pixels then increment by one for each "binned" pixel. As with the amplifier coordinate system the general linear coordinate transformation between the CCD and image coordinates is given by
Ic = LTM1_1 * Cx + LTM1_2 * Cy + LTV1
Il = LTM2_1 * Cx + LTM2_2 * Cy + LTV2
Cx = ( LTM2_2 * (Ic - LTV1) - LTM1_2 * (Il - LTV2)) /
(LTM1_1 * LTM2_2 - LTM1_2 * LTM2_1)
Cy = (-LTM2_1 * (Ic - LTV1) + LTM1_1 * (Il - LTV2)) /
(LTM1_1 * LTM2_2 - LTM2_1 * LTM2_1)
[This CCD to image transformation is equivalent to the IRAF concept of a logical to physical coordinate transformation where the physical coordinate system is the CCD pixel coordinate system and the logical coordinate system is the image coordinate system.]
The final coordinate system is the detector coordinate system (Dx,Dy). This is important when multiple amplifiers and/or multiple CCDs are used. The detector coordinate system describes a single pixel raster of "unbinned" CCD pixels into which the multiple CCD/amplifier pixels are mapped to make a single, simple picture. It is used for displaying (and possibly processing) multiple amplifier/multiple CCD data in a consistent fashion with offsets, flips and transposes between the image pixels accounted for.
As with the CCD coordinate system there is some flexibility in how the detector coordinate system is defined. Normally it will be defined to produce a semblance of a picture of the sky or focal plane. For a single CCD with multiple amplifiers the detector and CCD coordinate systems are generally the same. For a mosaic of CCDs the CCD coordinate systems for each CCD should be defined to have the same orientation relative to the focal plane and then the detector coordinate system will have the same orientation but with different origins for each CCD.
As before we define the transformation between the detector coordinates and CCD coordinates as
Dx = DTM1_1 * Cx + DTM2_1 * Cy + DTV1
Dy = DTM1_2 * Cx + DTM2_2 * Cy + DTV2
Cx = ( DTM2_2 * (Dx - DTV1) - DTM2_1 * (Dy - DTV2)) /
(DTM1_1 * DTM2_2 - DTM2_1 * DTM1_2)
Cy = (-DTM1_2 * (Dx - DTV1) + DTM1_1 * (Dy - DTV2)) /
(DTM1_1 * DTM2_2 - DTM2_1 * DTM1_2)
When pixels are binned and written to the image then there is a non-integer relationship between the image coordinate system and the other coordinate systems. The image coordinates of binned pixels are still defined to have integer values at the centers of the binned pixels and the integer values also are indices into the image pixel array. But now the center of a binned pixel transforms to the middle of the extent of the unbinned pixels. For example if the first two CCD pixels are binned to form the first image pixel the center of the binned pixel in the CCD coordinate system would be 1.5 (the extent of the pair of pixels is 0.5 to 2.5). Thus image coordinate 1 (assuming the first binned pixel is written to the first image pixel) maps to CCD coordinate 1.5, image coordinate 2 maps to CCD coordinate 3.5, and so on.
The binning of amplifier pixels is described by four parameters. Two of these are Ns and Np which are the number of serial and parallel pixels summed for each output pixel in the amplifier coordinate system. While we would expect that any real application would have all output pixels be the sum of the same number of single pixels it is possible that the first and last output pixels could be partial sums of fewer pixels. Thus we also define the parameters Ns1 and Np1 which indicate how many amplifier pixels are in the first sum (in the readout order). The number of amplifier pixels in the last sum is implicit in the total number of pixels. The binning information is recorded in the image header with the keyword and format
CCDSUM = 'Ns Np Ns1 Np1'
where Ns1 and Np1 can be omitted if they are the same as Ns and Np.
Note that the nomenclature of these inherently requires them to be
interpreted in the amplifier coordinate system.
Image sections are an array-like notation for defining regions in raster coordinate systems such as occur with CCDs and other digital detectors. The syntax follows that used by IRAF. One will note that the relations between the sections can be considered an equivalent and compact way of specifying the transformations between coordinate systems except for the case where two coordinate systems have a transposition. There are two reasons why the section notation keywords are included. The first is that legacy data and IRAF software make use of these keywords. The second is that it is, in some ways, easier to grasp and read than the transformation keywords defined previously.
The actual pixel data from an exposure are recorded in a section of the image called the "data section". The data section is identified in the image header under the keyword DATASEC. The format is
DATASEC = '[Ic1:Ic2,Il1:Il2]'
where Ic1 and Ic2 are the range of image pixel columns and Il1 and Il2 are
the range of image pixel lines. It is recommended that the section have
Ic1<Ic2 and Il1<Il2. The order determines the order for all image
sections in the sense that once the coordinate conventions and
transformation terms are defined as described previously then flipping any
one image section requires all image sections to be flipped to remain
consistent.
The set of unbinned pixels used to form the pixels recorded in the data section of the image are specified by the similar section definitions in each coordinate system
AMPSEC = '[As1:As2,Ap1:Ap2]'
CCDSEC = '[Cx1:Cx2,Cy1:Cy2]'
DETSEC = '[Dx1:Dx2,Dy1:Dy2]'
The above limits are related to the data section by transforming Ic1-0.5,
Ic2+0.5, Il1-0.5, and Il2+0.5 and taking the nearest integer. In
particular note that the order of the starting and ending values
is determined from the DATASEC limits and so the starting value
may be larger than the ending value.
The various transformation coefficients are derived by considering the mappings between the sections in the various coordinate systems (AMPSEC, DATASEC, DETSEC) to the section of the CCD (CCDSEC) used in the exposure the pixel binning factors given by CCDSUM. In this section we only consider the case of no partial pixel sums (Ns1 = Ns, Np1 = Np).
Since the transformations are all defined in similar terms we derive formulas for the CCD to image coordinate transformation. The other transformations can be derived by substituting the analogous transformation coefficients for the other coordinate systems and replacing Ns and Np by 1.
When there is no transpose between the coordinate systems we have:
LTM2_1 = 0, LTM1_2 = 0 Ic = LTM1_1 * Cx + LTV1 Il = LTM2_2 * Cy + LTV2 Cx = (Ic-LTV1) / LTM1_1 Cy = (Il-LTV2) / LTM2_2 Ic1 = LTM1_1 * Cx1 + 0.5 * (1 - LTM1_1) + LTV1 Ic2 = LTM1_1 * Cx2 - 0.5 * (1 - LTM1_1) + LTV1 Il1 = LTM2_2 * Cy1 + 0.5 * (1 - LTM2_2) + LTV2 Il2 = LTM2_2 * Cy2 - 0.5 * (1 - LTM2_2) + LTV2 Cx1 = (Ic1 - 0.5 * (1 - LTM1_1) - LTV1) / LTM1_1 Cx2 = (Ic2 + 0.5 * (1 - LTM1_1) - LTV1) / LTM1_1 Cy1 = (Il1 - 0.5 * (1 - LTM2_2) - LTV2) / LTM2_2 Cy2 = (Il2 + 0.5 * (1 - LTM2_2) - LTV2) / LTM2_2 Ns = abs (1 / LTM1_1) Np = abs (1 / LTM2_2) LTM1_1 = [flip] / Ns LTM1_1 = (Ic2 - Ic1 + 1) / (Cx2 - Cx1 + 1) LTV1 = Ic1 - LTM1_1 * Cx1 - 0.5 * (1 - LTM1_1) LTM2_2 = [flip] / Np LTM2_2 = (Il2 - Il1 + 1) / (Cy2 - Cy1 + 1) LTV2 = Il2 - LTM2_2 * Cy1 - 0.5 * (1 - LTM2_2)The value [flip] is 1 if the image pixel order is in the same sense as the CCD pixel order and -1 otherwise.
For the case of a transpose between the coordinate systems exchange LTM1_1 with LTM2_1, LTM2_2 with LTM1_2, and Cx with Cy.
1. A 2048x2048 CCD is readout with four amplifiers as shown in the figure below. The arrows show the serial direction. During the readout 32 overscan "pixels" are included in the image. The CCD and detector coordinate systems are defined with the origin at amplifier 1 and Cx increasing to the left and Cy increasing up in the figure.
3 4
+-------+
|<- ->|
| |
| |
|<- ->|
+-------+
1 2
Case a: The full format is readout with no binning and with the pixels written directly in the readout order to the image; i.e. the controller system does not flip the data when writing to the image. In this case the overscan will always be on the right and the first data pixel will be pixel (1,1) in the image.
Amplifier CCD Detector
1 As = Cx Ic = Cx Dx = Cx
Ap = Cy Il = Cy Dy = Cy
2 As = 2049 - Cx Ic = 2049 - Cx Dx = Cx
Ap = Cy Il = Cy Dy = Cy
3 As = Cx Ic = Cx Dx = Cy
Ap = 2049 - Cy Il = 2049 - Cy Dy = Cy
4 As = 2049 - Cx Ic = 2049 - Cx Dx = Cx
Ap = 2049 - Cy Il = 2049 - Cy Dy = Cy
AMPLIFIER 1 2 3 4
------- --------------- ------------------ ------------------ ---------------------
CCDSEC [1:1024,1:1024] [2048:1025,1:1024] [1:1024,2048:1025] [2048:1025,2048:1025]
AMPSEC [1:1024,1:1024] [1024:1,1:1024] [1:1024,1024:1] [1024:1,1024:1]
DATASEC [1:1024,1:1024] [1:1024,1:1024] [1:1024,1:1024] [1:1024,1:1024]
DETSEC [1:1024,1:1024] [2048:1025,1:1024] [1:1024,2048:1025] [2048:1025,2048:1025]
NSUM 1 1 1 1 1 1 1 1
ATV1 0 2049 0 2049
ATV2 0 0 2049 2049
ATM1_1 1 -1 1 -1
ATM2_2 1 1 -1 -1
LTV1 0 2049 0 2049
LTV2 0 0 2049 2049
LTM1_1 1 -1 1 -1
LTM2_2 1 1 -1 -1
DTV1 0 0 0 0
DTV2 0 0 0 0
DTM1_1 1 1 1 1
DTM2_2 1 1 1 1
Case b. The full format is readout with no binning but in this case the controller flips the readout order when writing to the image so that the recorded images have the same orientation to the sky. The overscan is written to the continuing in the same order as the pixels so that the overscan appears in columns 1 to 32 for amplifiers 2 and 4.
Amplifier CCD Detector
1 As = Cx Ic = Cx Dx = Cx
Ap = Cy Il = Cy Dy = Cy
2 As = 2049 - Cx Ic = Cx - 992 Dx = Cx
Ap = Cy Il = Cy Dy = Cy
3 As = Cx Ic = Cx Dx = Cy
Ap = 2049 - Cy Il = Cy - 1024 Dy = Cy
4 As = 2049 - Cx Ic = Cx - 992 Dx = Cx
Ap = 2049 - Cy Il = Cy - 1024 Dy = Cy
AMPLIFIER 1 2 3 4
------- --------------- ------------------ ------------------ ---------------------
CCDSEC [1:1024,1:1024] [1025:2048,1:1024] [1:1024,1025:2048] [1025:2048,1025:2048]
AMPSEC [1:1024,1:1024] [1024:1,1:1024] [1:1024,1024:1] [1024:1,1024:1]
DATASEC [1:1024,1:1024] [33:1056,1:1024] [1:1024,1:1024] [33:1056,1:1024]
DETSEC [1:1024,1:1024] [1025:2048,1:1024] [1:1024,1025:2048] [1025:2048,1025:2048]
NSUM 1 1 1 1 1 1 1 1
ATV1 0 2049 0 2049
ATV2 0 0 2049 2049
ATM1_1 1 -1 1 -1
ATM2_2 1 1 -1 -1
LTV1 0 -992 0 -992
LTV2 0 0 -1024 -1024
LTM1_1 1 1 1 1
LTM2_2 1 1 1 1
DTV1 0 0 0 0
DTV2 0 0 0 0
DTM1_1 1 1 1 1
DTM2_2 1 1 1 1
Case c. A region of interest given by CCD pixels 511 to 2000 by 1001 to 2047 is read with 2x3 binning. As in example b the controller flips the readouts. There are still 32 pixels of overscan which are on the right for amplifiers 1 and 3 and on the left for amplifiers 2 and 4.
Amplifier CCD Detector
1 As = Cx Ic = Cx / 2 - 254.75 Dx = Cx
Ap = Cy Il = Cy / 3 - 333 Dy = Cy
2 As = 2049 - Cx Ic = Cx / 2 - 511.75 Dx = Cx
Ap = Cy Il = Cy / 3 - 333 Dy = Cy
3 As = Cx Ic = Cx / 2 - 254.75 Dx = Cy
Ap = 2049 - Cy Il = Cy / 3 - 344 Dy = Cy
4 As = 2049 - Cx Ic = Cx / 2 - 511.75 Dx = Cx
Ap = 2049 - Cy Il = Cy / 3 - 344 Dy = Cy
AMPLIFIER 1 2 3 4
------- -------------------- --------------------- ------------------ ---------------------
CCDSEC [511:1024,1001:1024] [1025:2000,1001:1024] [511:1024,1025:2047] [1025:2000,1025:2047]
AMPSEC [511:1024,1001:1024] [1024:49,1001:1024] [511:1024,1024:2] [1024:49,1024:2]
DATASEC [1:257,1:8] [33:520,1:8] [1:257,1:341] [33:520,1:341]
DETSEC [511:1024,1001:1024] [1025:2000,1001:1024] [511:1024,1025:2047] [1025:2000,1025:2047]
NSUM 2 3 2 3 2 3 2 3
ATV1 0 2049 0 2049
ATV2 0 0 2049 2049
ATM1_1 1 -1 1 -1
ATM2_2 1 1 -1 -1
LTV1 -254.75 -511.75 -254.75 -511.75
LTV2 -333 -333 -344 -344
LTM1_1 0.5 0.5 0.5 0.5
LTM2_2 0.33333333 0.33333333 0.33333333 0.33333333
DTV1 0 0 0 0
DTV2 0 0 0 0
DTM1_1 1 1 1 1
DTM2_2 1 1 1 1
2. A mosaic of 4 1024x1024 CCDs is readout with four amplifiers as shown in the figure below. The arrows show the serial direction. During the readout 32 overscan "pixels" are included in the image. The CCD and detector coordinate systems are defined with the origin at amplifier 1 and Cx increasing to the left and Cy increasing up in the figure. The controller readouts out 32 overscan pixels and flips the readout order when writing to the image to preserve the orientation relative to the sky in the images. This means that the overscan regions for amplifiers 3 and 4 will be in the first columns of the images.
3 4
+-------++-------+
| ->|| ->|
| || |
| || |
| || |
+-------++-------+
+-------++-------+
| || |
| || |
| || |
|<- ||<- |
+-------++-------+
1 2
Case a. Readout the full format with no binning.
Amplifier CCD Detector
1 As = Cx Ic = Cx Dx = Cx
Ap = Cy Il = Cy Dy = Cy
2 As = Cx Ic = Cx Dx = Cx + 1024
Ap = Cy Il = Cy Dy = Cy
3 As = 1025 - Cx Ic = Cx + 32 Dx = Cx
Ap = 1025 - Cy Il = Cy Dy = Cy + 1024
4 As = 1025 - Cx Ic = Cx + 32 Dx = Cx + 1024
Ap = 1025 - Cy Il = Cy Dy = Cy + 1024
AMPLIFIER 1 2 3 4
------- --------------- ------------------ ------------------ ---------------------
CCDSEC [1:1024,1:1024] [1:1024,1:1024] [1:1024,1:1024] [1:1024,1:1024]
AMPSEC [1:1024,1:1024] [1:1024,1:1024] [1024:1,1024:1] [1024:1,1024:1]
DATASEC [1:1024,1:1024] [33:1056,1:1024] [1:1024,1:1024] [33:1056,1:1024]
DETSEC [1:1024,1:1024] [1025:2048,1:1024] [1:1024,1025:2048] [1025:2048,1025:2048]
NSUM 1 1 1 1 1 1 1 1
ATV1 0 0 1025 1025
ATV2 0 0 1025 1025
ATM1_1 1 1 -1 -1
ATM2_2 1 1 -1 -1
LTV1 0 0 32 32
LTV2 0 0 0 0
LTM1_1 1 1 1 1
LTM2_2 1 1 1 1
DTV1 0 1024 0 1024
DTV2 0 0 1024 1024
DTM1_1 1 1 1 1
DTM2_2 1 1 1 1
Another level of description of the geometry is that each image unit, the amplifier read-outs, may include a world coordinate system (WCS). Whether it is present and useful depends on the calibration of the instrument and telescope and the capabilities of the data acquisition system. Each WCS can map pixels in the image to right ascension and declination on the sky or to some other coordinate system such as fractional column and line coordinates in a mosaic image that includes the alignment corrections. The current types of WCS descriptions allow for rotations and various types of sky projections. In addition it is also possible to include distortion corrections to connect to the sky projections which assume an "ideal" detector. By providing a separate WCS for each amplifier a piece-wise correct astronomical or mosaic coordinate system can be available so that software may use it to report coordinates from an image display and to resample the data into a geometrically correct image.
Examples of the basic CCD geometry keywords (as defined elsewhere) for single CCDs with one or two amplifiers and for a CCD mosaic using CCDs with one or two amplifiers are given elsewhere.