mscred/ccdproc crashing on xtalk

[travisrector]

I'm having a problem with mscred/ccdproc. I'm running IRAF v2.15.1 and mscred 5.0.2, so I should be up to date. I've done all the usual stuff (init uparm, etc.).

When I run ccdproc set up as:

images = n407052.fits List of Mosaic CCD images to process
(output = ) List of output processed images
(bpmasks= ) List of output bad pixel masks
(ccdtype= ) CCD image type to process
(noproc = no) List processing steps only?

(xtalkco= yes) Apply crosstalk correction?
(fixpix = yes) Apply bad pixel mask correction?
(oversca= yes) Apply overscan strip correction?
(trim = yes) Trim the image?
(zerocor= no) Apply zero level correction?
(darkcor= no) Apply dark count correction?
(flatcor= no) Apply flat field correction?
(sflatco= no) Apply sky flat field correction?
(split = no) Use split images during processing?
(merge = no) Merge amplifiers from same CCD?

(xtalkfi= M11_1011_xtalk) Crosstalk file
(fixfile= M11_1011_bpm.fits) List of bad pixel masks
(saturat= INDEF) Saturated pixel threshold
(sgrow = 0) Saturated pixel grow radius
(bleed = INDEF) Bleed pixel threshold
(btrail = 20) Bleed trail minimum length
(bgrow = 0) Bleed pixel grow radius
(biassec= !biassec) Overscan strip image section
(trimsec= !trimsec) Trim data section
(zero = ) List of zero level calibration images
(dark = ) List of dark count calibration images
(flat = ) List of flat field images
(sflat = ) List of secondary flat field images
(minrepl= 1.) Minimum flat field value

(interac= no) Fit overscan interactively?
(functio= legendre) Fitting function
(order = 1) Number of polynomial terms or spline pieces
(sample = *) Sample points to fit
(naverag= 1) Number of sample points to combine
(niterat= 1) Number of rejection iterations
(low_rej= 3.) Low sigma rejection factor
(high_re= 3.) High sigma rejection factor
(grow = 0.) Rejection growing radius
(fd = )
(fd2 = )
(mode = ql)


I get the following error:

ERROR: bus error
"split=splt, fextn="fits", noproc=noproc)"
line 244: mscsrc$ccdproc.cl
called as: `ccdproc (mode=h)'

I know the problem is somehow associated with xtalk. If I turn off the xtalk correction it works fine. I moved the two necessary mscdb files to the local directory as a test, so that shouldn't be the problem either.

I'm running macintel architecture. Any ideas on what might be wrong?

[valdes]

Hi Travis,

I saw this posting and also got your email. I don't know what is causing this. I will look into it shortly. Sorry for the problem.

Frank

[travisrector]

Thanks!

[valdes]

The problem is a 64-bit issue (which is what the macintel binaries for V2.15 are). I have fixed the problem and issued a new distribution. So you need to update your mscred to V5.03. Sorry for the problem.

Frank Valdes

[travisrector]

Hi Frank-

On iraf.noao.edu in /iraf/extern-v215 it looks like the macosx binaries have been updated but not the macintel or the src files. Am I looking in the right place?

-Travis

[valdes]

Hi Travis,

I forgot the different places where we distribute things. I had updated the automatic V2.15 repository installation and not the older place where people can download binaries directly and install them themselves in the source tree. So I've now updated things in /iraf/extern-v215 where you are looking.

FYI the automatic repository is /iraf/v215/REPO and you can get things manually with ftp as before. The tar packages are slightly different in that in the REPO location the distributions are binary + source (ie one file gets you all you need) while in the extern version you get the source and binaries in two files and untar the source first, cd to the directory, create the bin.<arch>, cd to that, and untar the binaries.

Let me know if you need more information.

Frank

[travisrector]

Ok thanks for the explanation. Indeed this is a much easier install! It appears to be working fine now.

Thanks!

-Travis

[travisrector]

Ok...I seem to have spoken too soon. I'm having what appear to be "math" related crashes. For example, if I run flatcombine as shown below I get the following error:

PACKAGE = mscred
TASK = flatcombine

input = @n4.flat.lst List of flat field images to combine
(output = testFlat) Output flat field root name
(combine= average) Type of combine operation
(reject = ccdclip) Type of rejection
(ccdtype= ) CCD image type to combine
(process= no) Process images before combining?
(subsets= yes) Combine images by subset parameter?
(delete = no) Delete input images after combining?
(scale = mode) Image scaling
(statsec= ) Image section for computing statistics
(nlow = 1) minmax: Number of low pixels to reject
(nhigh = 1) minmax: Number of high pixels to reject
(nkeep = 1) Minimum to keep (pos) or maximum to reject (neg)
(mclip = no) Use median in sigma clipping algorithms?
(lsigma = 3.) Lower sigma clipping factor
(hsigma = 3.) Upper sigma clipping factor
(rdnoise= rdnoise) ccdclip: CCD readout noise (electrons)
(gain = gain) ccdclip: CCD gain (electrons/DN)
(snoise = 0.) ccdclip: Sensitivity noise (fraction)
(pclip = -0.5) pclip: Percentile clipping parameter
(blank = 1.) Value if there are no pixels
(mode = ql)



ERROR: segmentation violation
"pclip=pclip, grow=0)"
line 50: mccdred$flatcombine.cl
called as: `flatcombine (mode=h)'

But if I set scale=none it works ok. Similarly, when I run msccmatch as shown below I get this error:

PACKAGE = mscred
TASK = msccmatch

input = n4071*fits List of input mosaic exposures
coords = !mscgetcat $I $C Coordinate file (ra/dec)
(outcoor= ) List of updated coordinate files
(usebpm = yes) Use bad pixel masks?
(verbose= yes) Verbose?

# Coarse Search
(nsearch= 60) Maximum number of positions to use in search
(search = 60.) Translation search radius (arcsec)
(rsearch= 0.2) Rotation search radius (deg)

# Fine Centroiding
(cbox = 11) Centering box (pixels)
(maxshif= 5.) Maximum centering shift to accept (arcsec)
(csig = 0.1) Maximum centering uncertainty to accept (arcsec)
(cfrac = 0.5) Minimum fraction of accepted centers
(listcoo= yes) List centered coordinates in verbose mode?

# WCS Fitting
(nfit = 50) Min for fit (>0) or max not found (<=0)
(rms = 0.5) Maximum fit RMS to accept (arcsec)
(fitgeom= general) Fitting geometry
(reject = 3.5) Fitting rejection limit (sigma)
(update = yes) Update coordinate systems?
(interac= yes) Interactive?
(fit = yes) Interactive fitting?
(graphic= stdgraph) Graphics device
(cursor = ) Graphics cursor

accept = yes Accept solution?
(mode = ql)



MSCCMATCH:
n407106.fits:
5339 input coordinates
ERROR: arithmetic exception

Could I have somehow botched the mscred 5.0.3 and/or IRAF 2.15.1 installation?

-Travis

[valdes]

Hi,

This is not some kind of installation mistake. The evolution to native 64-bit code can have subtleties that are not always caught automatically and require examining the code. So you have probably found two such instances that need to be debugged. If you could send the original filenames for the flat list I will try to get them from the archive and reproduce the error.

If you get tired of being a tester, you can switch to using the macosx binaries which are 32-bit and will work on your Mac Intel machine. To do that you would install the macosx binaries. The easiest way may be by going to the extern-v215 ftp directory, getting the binaries, and unpacking from the mscred root directory. Then to tell IRAF to use them the simplest thing is to rename bin.macintel to something else such as bin.macintelBAK.

Frank Valdes

[travisrector]

Hi Frank-

I don't mind being a beta tester. The data in question are Mosaic 2 data from the 0.9m, UT 2011-01-22. Hillary was the observer. The flat field file names are n407052-040766. I'm also seeing problems when I try to use msccmatch on the object data, starting with n407106.

-Travis

[valdes]

Hi Travis,

The problem with flatcombine, which was actually in combine, was a simple bug where an argument was missing in a subroutine call. Apparently it is not a problem with many architectures but was for macintel. I could run flatcombine on the data you cited without problem.

The/A problem in msccmatch and a number of other tasks had to do with an instability in the core WCS library. I made a change there for the next IRAF release. But I then included the changes in the mscred macintel binaries which you will download. After that I was able to use mscgetcat and msccmatch. msccmatch did not work well on the file you pointed to. What I found I had to do was use mscsetwcs with a database file mscdb$noao/Mosaic1.1/WCS1101/kp09mVtnx34.db. This has been added to the mscdb distribution you can get with the 2.15 automatic update or from the extern-v215 ftp directory. This WCS is the one that is supposed to be added at the telescope but the first 0.9m commission run had some initial problems (as would be expected) and so the WCS in the header must not be quite right. I did not figure that out. But by using mscsetwcs to reset the WCS to the solution in the file then msccmatch worked fine.

Good luck,
Frank

[travisrector]

Hi Frank-

I've updated to 2.15.1a and I still get the same error. So then I went to iraf.noao.edu in /iraf/v215/REPO and grabbed mscred-macintel.tar.gz. But this appear to be mscred5.0.2. The files in extern-v215 appear to be 5.0.2 as well. Could you have updated the older version?

-Travis

[valdes]

There was some confusion here about what was the master version of mscred. When everything was rebuilt to address a different bug I believe the old mscred was used. I am not completely sure what happened but I have updated mscred again with what I did that fixed the problems you reported. So ... one more try?

Frank

[travisrector]

Ok, I've updated to mscred 5.0.4 and flattombine and msccmatch are now working.

Thanks!

-Travis

[travisrector]

Hi Frank et al.-

Ok, now I'm having a problem with mscimage. When I try to project a mosaic1.1 image I get the following error:


PACKAGE = mscred
TASK = mscimage

input = @ngc5364.lst List of input mosaic exposures
output = y//@ngc5364.lst List of output images
(format = image) Output format (image|mef)
(pixmask= yes) Create pixel mask?
(verbose= yes) Verbose output?

# Output WCS parameters
(wcssour= image) Output WCS source (image|parameters|match)
(referen= ) Reference image
(ra = INDEF) RA of tangent point (hours)
(dec = INDEF) DEC of tangent point (degrees)
(scale = INDEF) Scale (arcsec/pixel)
(rotatio= INDEF) Rotation of DEC from N to E (degrees)

# Resampling parmeters
(blank = 0.) Blank value
(interpo= sinc17) Interpolant for data
(minterp= linear) Interpolant for mask
(boundar= constant) Boundary extension
(constan= 0.) Constant boundary extension value
(fluxcon= no) Preserve flux per unit area?
(ntrim = 7) Edge trim in each extension
(nxblock= 4200) X dimension of working block size in pixels
(nyblock= 2100) Y dimension of working block size in pixels

# Geometric mapping parameters
(interac= no) Fit mapping interactively?
(nx = 10) Number of x grid points
(ny = 20) Number of y grid points
(fitgeom= general) Fitting geometry
(xxorder= 4) Order of x fit in x
(xyorder= 4) Order of x fit in y
(xxterms= half) X fit cross terms type
(yxorder= 4) Order of y fit in x
(yyorder= 4) Order of y fit in y
(yxterms= half) Y fit cross terms type


(fd_in = )
(fd_ext = )
(fd_coor= )
(mode = ql)







WCS reference image is apr20.06657.fits[im7]
Resampling apr20.06657.fits[im1] ...
Resampling apr20.06657.fits[im2] ...
Resampling apr20.06657.fits[im3] ...
Resampling apr20.06657.fits[im4] ...
Resampling apr20.06657.fits[im5] ...
Resampling apr20.06657.fits[im6] ...
Resampling apr20.06657.fits[im7] ...
Resampling apr20.06657.fits[im8] ...
Resampling apr20.06657.fits[im9] ...
Resampling apr20.06657.fits[im10] ...
Resampling apr20.06657.fits[im11] ...
Resampling apr20.06657.fits[im12] ...
Resampling apr20.06657.fits[im13] ...
Resampling apr20.06657.fits[im14] ...
Resampling apr20.06657.fits[im15] ...
Warning: Cannot open file (EMPTY.im15.pl)
ERROR: Cannot open image (tmp1149k)
"nyblock=nyblk, verbose=no)"
line 233: mscsrc$mscimage.cl
called as: `mscimage (mode=h)'

As a test, I also tried running 'mscimage' on a single image (not MEF) and got the following error:

WCS reference image is dummy_t2ka.fits
Resampling 22sep09 ...
Warning: Image and mask have a relative rotation
ERROR: Cannot open image (tmp1952s)
"nyblock=nyblk, verbose=no)"
line 233: mscsrc$mscimage.cl
called as: `mscimage (mode=h)'

This has worked for me in the past. In both cases it is attempting to call a tmp file that doesn't exist.

Ideas?

-Travis