Welcome to iraf.net Monday, August 21 2017 @ 08:13 AM GMT

Enhanced Image List Templates Being Developed

  • Tuesday, May 24 2011 @ 10:50 PM GMT
  • Contributed by:
  • Views: 3,694
IRAFNET

A number of system enhancements are underway as part of integrating IRAF with the Virtual Observatory, in particular is the ability to seamlessly use data tables returned by VO data services as input to IRAF tasks. This has a number of implications I'll write about later, but one early pay-off of this work is an enhanced image template interface in the core system that is now in testing internally. If you aren't sure what an "image template" is, think of the "*.fits" or @files that let you pass lists of images to IRAF tasks for processing.

In the enhanced templates, things like MEF files (and soon, tables) can be expanded automatically using the @-file operator to let any task that takes an image list process the extensions automagically. Additionally, selection of images based on image header keywrods can also be done dynamically using expressions in the template strings. For example, consider the following templates:

      @file* expand all files beginning w/ 'file'
      @file//".fits" append ".fits" to contents of 'file'
      @mef.fits expand all (image) extensions of an MEF file
      @mef.fits[SCI] select SCI extensions from MEF file
      @mef.fits[SCI,2][noinherit] select v2 SCI extns, add kernel param
      @mef.fits[1-16x2] select range of extensions from MEF file
      @mef.fits[+1-8] create a list of extensions for an MEF
      *.fits[1:100,1:100] append section to all FITS images
      @@file[1:100,1:100] append section to expanded MEFs in 'file'
      *.fits[filter?='V'] select images w/ FILTER keyword containing 'V'
      @*.fits[gain==3.0]select image extns where GAIN keyword is 3.0
      *.fits[filter?='V';gain==2.5]select using multiple OR's expressions

These templates could be used e.g. to run IMSTAT on all the extensions of a FITS file with a single command, as in

    cl> imstat @mef.fits
      instead of
      cl> imext mef.fits output=file > list.dat
      cl> if (imext.nimages > 0)
      >>> imstat @list.dat
      cl> del list.dat

where before it would have been necessary to expand the extensions explicitly using a second task into a standard @file. Notice how the other templates provide even more refined selection of the extensions, either within a single MEF or across multiple images.

These changes are all fully backward compatible but provide new syntax to give users a powerful and compact way to dynamically build image lists for use by all tasks. These new features will be especially useful for script developers tired of managing temp files of image lists, or those who just need to quickly examine MEF files or a directory of images.

Leave a message or post a comment if you're interested in getting an early peek at these new features. [Note that a small source patch to your system and a relink will be required].



                    Enhanced Image List Template Package

                             April 15, 2011


    The enhanced image list package provides new capabilities for handling
image lists, but remains backwards compatible with tasks currently using  
the IMT interface.  The enhancements allow for expansion of MEF files into
lists of extensions using the @-file operator, as well as selection of    
images within more general lists by means of modifiers (e.g. a simple     
expression such as the extname/extver or explicit extension number, or more
complex boolean expressions to allow selection by header keyword). In      
addition, tables may now take the @-file operator to use a column          
containing image references as an input list.                              


========================================================================
TODO:                                                                   
    - Describe syntax for use with tables and selection by row values   
    - Describe remote image specification caching mechanism             
========================================================================



Template Strings
----------------

The FNT template package supports the following forms of pattern strings:

    alpha, *.x, data* // .pix, [a-m]*, @list_file, nite%1%2%.1024.imh

i.e. simple filenames, wildcard expansion in filenames, concatenation of
filenames, @files, substitution in filenames, or a comma-delimited list of
the above.  The image template package (IMT) extends these patterns to    
allow image names followed by a cluster index or image section in []      
brackets.  These patterns remain unchanged in the new version of the      
package to allow backward compatability with existing applications. Lists 
of these types represent *explicit* collections of images, i.e. a         
collection based on the image name (wildcards) or as a result of processing
by some task (e.g. expansion of an MEF file to create an input @-file of   
expanded extension specifications).                                        

    The enhanced version of the IMT package further abstracts the concept
of image collections to include data objects such as MEF files or tables 
containing a list of image references that *implicitly* defines the list 
(e.g. the expanded MEF extension specification or the complete column of 
image references).  Further, we allow this list (which might be quite    
broad) to be refined using modifiers or selectors on the list and thus   
dynamically create the list without requiring the use (and management) of
intermediate files.  For example,                                        


  @file*                        expand all files beginning w/ 'file'
  @file//".fits"                append ".fits" to contents of 'file'

  @mef.fits                     expand all (image) extensions of an MEF file
  @mef.fits[SCI]                select SCI extensions from MEF file         
  @mef.fits[SCI,2][noinherit]   select v2 SCI extns, add kernel param       
  @mef.fits[1-16x2]             select range of extensions from MEF file    
  @mef.fits[+1-8]               create a list of extensions for an MEF      

  *.fits[1:100,1:100]           append section to all FITS images
  @@file[1:100,1:100]           append section to expanded MEFs in 'file'

  *.fits[filter?='V']           select images w/ FILTER keyword containing 'V'
  @*.fits[gainfile  [extname]          [expr;...]  [ikparams]
[section]                                                                
  [@@ | @]  file  [extname,extver][expr;...]  [ikparams]   
[section]                                                                
  [@@ | @]  file  [index_range]   [expr;...]  [ikparams]   
[section]                                                                
                                                                                
                    -------- selectors -------  ------ modifiers      
-----                                                                       

    The file specification may be the name of a file, and image, or a
table.  The behavior of the @-file and @@-file operators will depend on     
the type of file but the @-file usage remains backward compatible when
used with text files.                                                        

    The use of a modifier/selection on an MEF file will automatically
trigger expansion of the extensions in the image and so an '@' operator is
not strictly required, however only those extensions matching the selection
expression will be present in the final image list.  Note that the use of  
index ranges and extname/extver selectors are mutually exclusive, selector 
expressions may be added to either.                                        


@-file Operations
-----------------

    The @-file operator is unchanged from previous versions when used with
text files of image names.  Modifier/selector expressions however can now 
be applied to the contents of the @-file to select from the list only those
images that match the selector expression, or to augment the name in the   
list with an additional image syntax such as a section or kernel parameters.


@@-file Operations
------------------

    The @@-file operator is new syntax meant to allow the contents of an 
@-file to be expanded automaticaly, e.g. as if there were an @-file of   
@-file names.  Primarily this can be used to create a list of MEF image  
names in which an @-file would return the names of the MEF, while the    
@@-file syntax could be used to expand each MEF into individual extension
specifications.                                                          


Extension Indices
-----------------

    The [index_range] modifier may be used to specify an explicit set of
extensions to be used.  Index ranges are specified as a comma-delimited 
list of strings specifying individual range segments as described in the
RANGES help page.                                                       

    The use of a '+' operator before an index range indicates the range
list should be expanded without checking that the extension exists in the
MEF itself.  Otherwise, only those extensions present in the MEF will be 
included in the list.                                                    


Selection Expressions
---------------------

    Selection expressions may be used to restrict a template list to only
those images that match some boolean expression, e.g. to provide for     
selection based on a header keyword value.  Expressions follow the same  
guidelines as in the HSELECT task 'expr' parameter (see the help page    
for details).  Multiple expressions may be specified if they are separated
by a semicolon however they are evaluated as a single expression of       
OR'd values rather than as individual expressions.  This is significant   
when considering that expressions may contain keywords not present in     
all images being checked, for instance                                    

        *.fits[filter?='V';gain0.5&&gain foobar.fits
        foo.fits // bar                 ==> foobar.fits

        foo // @file1                   ==> foosif1.fits,foomef1.fits
        @file1 // bar                   ==> sif1foo.fits,mef1foo.fits


Enhanced Image List Templates Being Developed | 0 comments | Create New Account

The following comments are owned by whomever posted them. This site is not responsible for what they say.



Privacy Policy
Terms of Use

User Functions

Login