iraf.net

Hi Lindsey-
At least I figure that Lindsey Davis will probably end up getting this...
can you tell me where to look to find out exactly what the sharpness and
roundness mean (i.e. hwo they are calculated) for daofind.
I can't seem to find the path down to the doafind task in the code...
(or better do you have reference for it????)
thanks
Hank D.
PS HI to every one there...
:-)

Hello Hank, Lindsey would be the best person to reply to your mail, but she
is out of the office until the 17th. Meanwhile, I can point you to the
source files where the roundness and sharpness are calculated. In the
file apfind.x, I see procedures ap_round and ap_sharp. The comments and
code will probably tell you what you want to know. The apfind.x file is
in the noao$digiphot/apphot/daofind directory. If you have further
questions, send more mail to Lindsey (ldavis@noao.edu) and she'll get
back to you after she returns.Suzanne Jacoby

Hello Trudy, In what follows I am assuming that you are running noao.apphot.daofind
under iraf 2.10.2. If instead you are running daofind under iraf 2.10.3 beta
let me know as things are slightly different. You should set threshold to some small number times the standard deviation
of your sky background pixels. If for example the standard deviation in
your sky regions is ~10 counts, then setting threshold to ~30 counts will give
you a 3-sigma detection limit. Most people choose a number from 3-5 times
the standard deviation in the sky background. Much fainter and you will
tend to detect many noise spikes along with stars; much brighter and you
will miss a lot of stars. You will have to determine the best threshold for
your data by trial an error. Make sure the fwhmpsf parameter is set
resonably close to the actual fwhmpsf of stars in your image. The sharpness parameter is the amplitude of the best fitting delta
function divided by the amplitude of the best fitting gaussian at the
position of the detected object. Typical values are ~0.6
for gaussian stars and nsigma=1.5. Hot pixels have values >> 1.0; and
cold pixels have values ~0.0. Therefore reasonable bounds on the
sharpness statistic are 1.0 and 0.2. The roundness statistic is equal to roundness = 2.0 * (hx - hy) / (hx + hy)where hx and hy are the heights of the best fitting 1D gaussians at the
position of the detected object. For round objects roundness ~0.0; for
objects elongated in x it is a large negative number; for objects
elongated in y it is a large positive number. The roundness statistic
is usefull for filtering out bad rows and columns. The output of daofind can be used directly with the phot task.
Lindsey Davis

Hello Greg,Before going on to the subject of how daofind works and what might
be going "wrong" with your data, I would like to make you aware of a bug
in the version of daofind you are currently running (2.10.4 or 2.10.4
patch 1 ?) that may introduce unecessary scatter into the transformations
you later compute with geomap. Basically the fractional pixel component of
the x and y centers was being computed incorrectly (underestimated) resulting
in a modulation of those fractional pixel components, e..g some values were
more common than others. This may show up as a quantization affect in the
residuals of geomap fits and was discovered by a user using daofind for the
same purpose you are. There error was in the standalone daophot package as
well and I unfortunately propagated it to the IRAF version ... This bug
has been fixed in IRAF 2.10.4 patch 2 and our current 2.11 release and in
the addon version of digiphot digiphotx. See our on-line bug log (bug
332) for details.Since you are apparently using various task in the addon immatchx package
I would also like to draw your attention the starfind task which is a
modified version of daofind intended for use in image registration and
astrometry type applications. Starfind computes a density image and
applies a detection threshold like daofind, but after that it does an
imexamine stype moment analysis on the detected objects. It also
has a sepmin parameter which people can use to detect the brightest
object within some distance which is different than the fitting or
convolution radius. The starfind task is part of the latest version of
the addon immatchx package and is part of the images package in IRAF 2.11.>I'm running daofind with a moderately high threshold so as to get a set of
>good stars with which to make coordinate transformations, etc. (between
>CCD and IR images) with linxymatch, geomap and geoxytran. What I'm having
>difficulty with is avoiding daofind detections in the saturated "profiles"
>of overexposed brighter stars on my CCD image. Setting datamax
>appropriately would seemingly solve the problem, but daofind is still
>finding detections in the phony psf's of saturated stars, even when pixel
>values at such a detection site exceed datamax.
>
>For instance, I would set datamax=60000 (and did experiment with other
>values), run daofind, show the detections with tvmark, clearly see bad
>detections, and then check some with imexam. When I'd do an imexam
>contour plot at the precise x-y coordinates daofind came up with (in a
>saturation profile), the high pixel shown in the contour plot was often
>above datamax=60000 counts.
>
>I narrowed other criteria such as roundness and sharpness, but some
>saturation detections still came up for a range of datamax values tried.
>How is the datamax number checked when daofind runs through an image?
>Over what radius from the detection point is the detection supposed to
>free of too-high pixel values?
I took a look at the daofind parameters you set in your last mail and
I have a couple of comments. First it is a good idea to set the gain
and readout noise values to their true values. Although I don't think
this should affect your application significantly, these numbers are
used to tune up the sharpness statistic computation and should be set
to realistic values. Have you sky subtracted your data as IR people
often do? If so it is a good idea to add a representative constant
sky back into the data because daofind makes some assumptions about
the statistics of the data. Again this is a subtle point and may not
affect the results for your data but ... Finally your roundness limits
are pretty restrictive. I know you were probably experimenting with
removing bad objects but you might eliminate some good objects too.Daofind uses the min and max good data values to identify bad data and
remove it from the density image computation and the object position and
statistics computation rather than to remove bad objects as a whole from
the detected list. This because other downstream daophot tasks can
still fit a psf to stars which have saturated cores as long as there
are enough good pixels in the wings. It is sometimes desirable to
fit fainter stars on the wings of brighter saturated stars as well.
Only if the stars is saturated inside a reasonable fitting radius do
you have to throw it away. The psf task even has a provision for
fitting a psf model to the cores of fainter and the wings of brighter
stars although this is a dangerous option and requires care to use well.All the daofind computation, e.g. convolution, sharpness and roundness
statistics computation, x and ycenter computation etc. are made over
a circular region which has a radius ofradius = max (2.0, nsigma * fwhmpsf * 0.42467)Therefore the convolution kernel can never be less than 5x5 pixels. You
can try increasing nsigma (the current value is optimized for detecting
stellar cores) but if you do you will need to modify the sharpness
statistic limits.The datamin and datamax values are used to:1. eliminate bad data from the density enhancement image computation. In
mathematical terms the density enhancement image contains the amplitude
of the best fitting gaussian of fwhmpsf equal to the fwhmpsf value supplied
by the user at each point. Any points with data outside the limits defined
by datamin and datamax are eliminated from this fit. If there are
fewer than 2 data points left after eliminating any bad data then the
density value at that point is set to 0.0 which is equivalent to saying
there is not source there. Most of the left half of the array you sent
should have density values 0.0. However there are a few pixel down
around x=368-370 and y=1474-1476 where the data are less than 5000 counts
and noise fluctions may cause a spurious detection. Row 1474 is interesting
in that regard.2. If the middle data pixel in a potential detection is outside the good
data limits then the sharpness value is automatically set to INDEF and
the sharpness test is eliminated because it is meaningless. If the
middle pixel is on the high side of the limit the saturation flag is set.
I suspect that this is where your INDEF values sharpness value is coming from.3. The remaining statistics are used to compute the sharpness statistics.
The sharpness statistic is set to INDEF if the density at the middle
pixel is 0.0 (this can only happen if the detection threshold is 0.0
which makes no sense) or if all the pixels other than the center one
are bad. If any pixels are greater than datamax the saturation flag is set.3. The datamin and datamax parameters are also used to eliminate bad
data from the x, y, and roundness computations. The roundness test
is eliminated if the object is saturated. Otherwise if there is not enough good
points to compute these statistics then these objects are eliminated from
the list.Note that the latest version of daofind included a second roundess test
based on the density rather than the data values which can be used to
eliminate objects elongated in directions other than the x and y.>
>I don't know if this matters, but I'm running IRAF V2.10.4 where this
>peculiarity has occurred. It does (see beginning of this message) but not for the reasons you
are concerned about.>
>Also, at the end of the daofind output, there's a number given for
>"relerr" which I don't recall seeing with earlier versions of daofind.
>What does "relerr" mean, and how is it determined? The relative error is a number by which the standard error in one pixel
in the orginal image must be multiplied by to obtain the standard error
in 1 pixel in the smoothed / differenced density image. The relative
error is computed directly from the convolution kernel used to compute
the density image. A blank sky region in your input image with a
measured standard deviation of sigma would translate into a region with
a mean value of 0.0 and a standard deviation of relerr * sigma in the
density image. The user supplied detection threshold is multiplied by
relerr in order to obtain the "true" detection threshold that is actually
used.In earlier versions of daophot relerr was computed but the user had to decide
whether or not to actually use it in the definition of threshold which was
in counts. In new version the detection threshold is defined in sigma
and relerr is computed automatically. Relerr is a function of the size
and shape of the kernel and can make a significant difference in the
detection rate.After reading this write back if what you are seeing still does not make
sense. I would be happy to look at your image and do a more detailed
analysis of what daofind in doing, to see if there are any hidden gotchas,
although the results might have to wait a couple of weeks as I leave
on vacation shortly.
Regards,
Lindsey Davis

Hi Lindsey. Thanks for the reply, and the helpful explanations and
suggestions contained therein. Addressing things one at a time:On Thu, 9 Jul 1998 davis@noao.edu wrote:> I would like to make you aware of a bug
> in the version of daofind you are currently running (2.10.4 or 2.10.4
> patch 1 ?) that may introduce unecessary scatter into the transformations
> you later compute with geomap. Basically the fractional pixel component of
> the x and y centers was being computed incorrectly (underestimated) resulting
> in a modulation of those fractional pixel components, e..g some values were
> more common than others. This may show up as a quantization affect in the
> residuals of geomap fits and was discovered by a user using daofind for the
> same purpose you are. There error was in the standalone daophot package as
> well and I unfortunately propagated it to the IRAF version ... This bug
> has been fixed in IRAF 2.10.4 patch 2 and our current 2.11 release and in
> the addon version of digiphot digiphotx. See our on-line bug log (bug
> 332) for details.I guess my version is just 2.10.4, since I don't see "patch" anywhere when
starting up IRAF: NOAO SUN/IRAF Revision 2.10.4 Fri May 26 09:12:57 MST 1995
This is the EXPORT version of Sun/IRAF V2.10.4 for Solaris 2.4.If I feed the daofind coordinates to phot, using the centroid algorithm,
will that mitigate the bug? Does phot's centroid have a similar problem? > Since you are apparently using various task in the addon immatchx package
> I would also like to draw your attention the starfind task which is a
> modified version of daofind intended for use in image registration and
> astrometry type applications. Starfind computes a density image and
> applies a detection threshold like daofind, but after that it does an
> imexamine stype moment analysis on the detected objects. It also
> has a sepmin parameter which people can use to detect the brightest
> object within some distance which is different than the fitting or
> convolution radius. The starfind task is part of the latest version of
> the addon immatchx package and is part of the images package in IRAF 2.11.Thanks for the tip. In the STARFIND help file it says: STARFIND is a modified version of the DAOPHOT package DAOFIND
algorithm. However STARFIND is intended for use with the IMAGES
package image matching and image coordinates tasks and is therefore
configured somewhat differently than the version used in the
photometry packages.So if I feed the coordinates of found objects to a variety of tasks (e.g.
PHOT, LINXYMATCH, and IMEXAM), might I be better off with DAOFIND? As you
can imagine, I'm always a little relunctant to figure out all the nuances
of a new task, if an old failiar task is sufficient. > First it is a good idea to set the gain
> and readout noise values to their true values. Although I don't think
> this should affect your application significantly, these numbers are
> used to tune up the sharpness statistic computation and should be set
> to realistic values. I ususally try to set those, but I wasn't sure of the numbers for the CCD
that was used (LRIS on Keck II). Admittedly I was a little lazy in this
regard. I just now looked up the readnoise and gain on the Keck website,
and inputted them into the datapars set. > Have you sky subtracted your data as IR people often do? I do sky-subtraction with IR array images, but not actually for my CCD
frames (although they are taken at I-band). > Finally your roundness limits
> are pretty restrictive. I know you were probably experimenting with
> removing bad objects but you might eliminate some good objects too.Yes, I was just experimenting and made roundness a restrictive criteria
here. I ordinarily use -0.6 < roundness < 0.6 when I'm pushing down to
5-sigma sources (trying to find my brown dwarfs), and started with -0.5
to 0.5 for this higher-threshold star-finding procedure (for feeding to
LINXYMATCH, etc.).> Daofind uses the min and max good data values to identify bad data and
> remove it from the density image computation and the object position and
> statistics computation rather than to remove bad objects as a whole from
> the detected list. This because other downstream daophot tasks can
> still fit a psf to stars which have saturated cores as long as there
> are enough good pixels in the wings. It is sometimes desirable to
> fit fainter stars on the wings of brighter saturated stars as well.
> Only if the stars is saturated inside a reasonable fitting radius do
> you have to throw it away. The psf task even has a provision for
> fitting a psf model to the cores of fainter and the wings of brighter
> stars although this is a dangerous option and requires care to use well.
>
> After reading this write back if what you are seeing still does not make
> sense. I would be happy to look at your image and do a more detailed
> analysis of what daofind in doing, to see if there are any hidden gotchas,
> although the results might have to wait a couple of weeks as I leave
> on vacation shortly.I understand better now how this all works (your useful tutorial is
appreciated), although it still seems a little strange to me that the
particular daofind detection described (and similar ones) should come up.
I guess phony saturation sources are unavoidable, and that I'll have to
edit the DAOFIND/PHOT output to remove bad sources. It seems that using
TXDUMP boolean expressions such as "merr<0.1" on PHOT output tends to get
rid of many bad sources. Thanks again (and have a nice vacation!),Greg************************************************************************
GREG SCHULTZ
UCLA Astronomy & Astrophysics (310) 825-3172 [office phone]
Box 951562 (310) 206-2096 [FAX machine]
Los Angeles, CA 90095-1562 schultz@astro.ucla.edu
************************************************************************