A number of system enhancements are underway as part of integrating IRAF with the Virtual Observatory, in particular is the ability to seamlessly use data tables returned by VO data services as input to IRAF tasks. This has a number of implications I'll write about later, but one early pay-off of this work is an enhanced image template interface in the core system that is now in testing internally. If you aren't sure what an "image template" is, think of the "*.fits" or @files that let you pass lists of images to IRAF tasks for processing.

In the enhanced templates, things like MEF files (and soon, tables) can be expanded automatically using the @-file operator to let any task that takes an image list process the extensions automagically. Additionally, selection of images based on image header keywrods can also be done dynamically using expressions in the template strings. For example, consider the following templates:

These templates could be used e.g. to run IMSTAT on all the extensions of a FITS file with a single command, as in

where before it would have been necessary to expand the extensions explicitly using a second task into a standard @file. Notice how the other templates provide even more refined selection of the extensions, either within a single MEF or across multiple images.

These changes are all fully backward compatible but provide new syntax to give users a powerful and compact way to dynamically build image lists for use by all tasks. These new features will be especially useful for script developers tired of managing temp files of image lists, or those who just need to quickly examine MEF files or a directory of images.

Leave a message or post a comment if you're interested in getting an early peek at these new features. [Note that a small source patch to your system and a relink will be required].

                    Enhanced Image List Template Package

                             April 15, 2011

    The enhanced image list package provides new capabilities for handling
image lists, but remains backwards compatible with tasks currently using  
the IMT interface.  The enhancements allow for expansion of MEF files into
lists of extensions using the @-file operator, as well as selection of    
images within more general lists by means of modifiers (e.g. a simple     
expression such as the extname/extver or explicit extension number, or more
complex boolean expressions to allow selection by header keyword). In      
addition, tables may now take the @-file operator to use a column          
containing image references as an input list.                              

    - Describe syntax for use with tables and selection by row values   
    - Describe remote image specification caching mechanism             

Template Strings

The FNT template package supports the following forms of pattern strings:

    alpha, *.x, data* // .pix, [a-m]*, @list_file, nite%1%2%.1024.imh

i.e. simple filenames, wildcard expansion in filenames, concatenation of
filenames, @files, substitution in filenames, or a comma-delimited list of
the above.  The image template package (IMT) extends these patterns to    
allow image names followed by a cluster index or image section in []      
brackets.  These patterns remain unchanged in the new version of the      
package to allow backward compatability with existing applications. Lists 
of these types represent *explicit* collections of images, i.e. a         
collection based on the image name (wildcards) or as a result of processing
by some task (e.g. expansion of an MEF file to create an input @-file of   
expanded extension specifications).                                        

    The enhanced version of the IMT package further abstracts the concept
of image collections to include data objects such as MEF files or tables 
containing a list of image references that *implicitly* defines the list 
(e.g. the expanded MEF extension specification or the complete column of 
image references).  Further, we allow this list (which might be quite    
broad) to be refined using modifiers or selectors on the list and thus   
dynamically create the list without requiring the use (and management) of
intermediate files.  For example,                                        

  @file*                        expand all files beginning w/ 'file'
  @file//".fits"                append ".fits" to contents of 'file'

  @mef.fits                     expand all (image) extensions of an MEF file
  @mef.fits[SCI]                select SCI extensions from MEF file         
  @mef.fits[SCI,2][noinherit]   select v2 SCI extns, add kernel param       
  @mef.fits[1-16x2]             select range of extensions from MEF file    
  @mef.fits[+1-8]               create a list of extensions for an MEF      

  *.fits[1:100,1:100]           append section to all FITS images
  @@file[1:100,1:100]           append section to expanded MEFs in 'file'

  *.fits[filter?='V']           select images w/ FILTER keyword containing 'V'
  @*.fits[gainfile  [extname]          [expr;...]  [ikparams]
  [@@ | @]  file  [extname,extver][expr;...]  [ikparams]   
  [@@ | @]  file  [index_range]   [expr;...]  [ikparams]   
                    -------- selectors -------  ------ modifiers      

    The file specification may be the name of a file, and image, or a
table.  The behavior of the @-file and @@-file operators will depend on     
the type of file but the @-file usage remains backward compatible when
used with text files.                                                        

    The use of a modifier/selection on an MEF file will automatically
trigger expansion of the extensions in the image and so an '@' operator is
not strictly required, however only those extensions matching the selection
expression will be present in the final image list.  Note that the use of  
index ranges and extname/extver selectors are mutually exclusive, selector 
expressions may be added to either.                                        

@-file Operations

    The @-file operator is unchanged from previous versions when used with
text files of image names.  Modifier/selector expressions however can now 
be applied to the contents of the @-file to select from the list only those
images that match the selector expression, or to augment the name in the   
list with an additional image syntax such as a section or kernel parameters.

@@-file Operations

    The @@-file operator is new syntax meant to allow the contents of an 
@-file to be expanded automaticaly, e.g. as if there were an @-file of   
@-file names.  Primarily this can be used to create a list of MEF image  
names in which an @-file would return the names of the MEF, while the    
@@-file syntax could be used to expand each MEF into individual extension

Extension Indices

    The [index_range] modifier may be used to specify an explicit set of
extensions to be used.  Index ranges are specified as a comma-delimited 
list of strings specifying individual range segments as described in the
RANGES help page.                                                       

    The use of a '+' operator before an index range indicates the range
list should be expanded without checking that the extension exists in the
MEF itself.  Otherwise, only those extensions present in the MEF will be 
included in the list.                                                    

Selection Expressions

    Selection expressions may be used to restrict a template list to only
those images that match some boolean expression, e.g. to provide for     
selection based on a header keyword value.  Expressions follow the same  
guidelines as in the HSELECT task 'expr' parameter (see the help page    
for details).  Multiple expressions may be specified if they are separated
by a semicolon however they are evaluated as a single expression of       
OR'd values rather than as individual expressions.  This is significant   
when considering that expressions may contain keywords not present in     
all images being checked, for instance                                    

        *.fits[filter?='V';gain0.5&&gain foobar.fits
        foo.fits // bar                 ==> foobar.fits

        foo // @file1                   ==> foosif1.fits,foomef1.fits
        @file1 // bar                   ==> sif1foo.fits,mef1foo.fits

Comments (0)