iraf.net

Hello colleagues,I have several spectra of different wavelength range. These spectra look very noisy and sometimes its difficult to tell which is noise and which is real absorption line. I want to combine some of them in order to increase the signal to noise ratio. For example I got nspec5 with 1000-1090 A, and nspec7 with 1015-1100 A. When I doscombine nspec5.fits,nspec7.fits nspec57.fits combine=medianI get okay looking spectra between 1015 - 1090 A, that is the overlapping part. The rest is nonsense. Can you suggest something?Thanks,Siti

Could you post a gif or make some data available along with the command and parameters to produce the "nonsense" values? The non-overlap parts should be ok but with greater noise.Actually and easier first step is to post the header listings for the two input spectra (imhead l+). I can make dummy spectra with the same headers (i.e. dispersion sampling) and see what happens. The output of imstat might also be helpful.Yours,
Frank Valdes

Hello Frank,
Glad to hear from you as I have to present my preliminary analysis next week (I am a graduate student in physics dept.) [quote:5b3c16ec87]Could you post a gif or make some data available along with the command and parameters to produce the "nonsense" values? The non-overlap parts should be ok but with greater noise[/quote:5b3c16ec87]
You mean post an splot of the combined spectra and the data? Would you prefer the data in image.fits or text.dat format? And where shall I post them? The non-overlap parts between ~1009.9-1016.5 A and ~1090.7-1097.0 A are situated well below the normalized continuum. Below 1009.9 A and above 1097.0 A, things look okay. :shock: Here are parameters for scombine:
PACKAGE = kpnoslit
TASK = scombine

input = nspec5.fits,nspec7.fits List of input spectra
output = nspec57.fits List of output spectra
(noutput= ) List of output number combined spectra
(logfile= STDOUT) Log file
(apertur= ) Apertures to combine
(group = apertures) Grouping option
(combine= median) Type of combine operation
(reject = none) Type of rejection
(first = no) Use first spectrum for dispersion?
(w1 = INDEF) Starting wavelength of output spectra
(w2 = INDEF) Ending wavelength of output spectra
(dw = INDEF) Wavelength increment of output spectra
(nw = INDEF) Length of output spectra
(log = no) Logarithmic increments?
(scale = none) Image scaling
(zero = none) Image zero point offset
(weight = none) Image weights
(sample = ) Wavelength sample regions for statistics
(lthresh= INDEF) Lower threshold
(hthresh= INDEF) Upper threshold
(nlow = 1) minmax: Number of low pixels to reject
(nhigh = 1) minmax: Number of high pixels to reject
(nkeep = 1) Minimum to keep (pos) or maximum to reject (neg)
(mclip = yes) Use median in sigma clipping algorithms?
(lsigma = 3.) Lower sigma clipping factor
(hsigma = 3.) Upper sigma clipping factor
(rdnoise= 0.) ccdclip: CCD readout noise (electrons)
(gain = 1.) ccdclip: CCD gain (electrons/DN)
(snoise = 0.) ccdclip: Sensitivity noise (fraction)
(sigscal= 0.1) Tolerance for sigma clipping scaling corrections
(pclip = -0.5) pclip: Percentile clipping parameter
(grow = 0) Radius (pixels) for 1D neighbor rejection
(blank = 0.) Value if there are no pixels
(mode = ql)[quote:5b3c16ec87]Actually and easier first step is to post the header listings for the two input spectra (imhead l+). I can make dummy spectra with the same headers (i.e. dispersion sampling) and see what happens. [/quote:5b3c16ec87]
image header for nspec5.fits:
nspec5.fits[7693][real]:
No bad pixels, min=0., max=0. (old)
Line storage mode, physdim [7693], length of user area 1256 s.u.
Created Wed 18:48:20 17-Dec-2008, Last modified Wed 18:38:13 17-Dec-2008
Pixel file "nspec5.fits" [ok]
EXTEND = F / File may contain extensions
ORIGIN = 'NOAO-IRAF FITS Image Kernel July 2003' / FITS file originator
DATE = '2008-12-17T10:48:20' / Date FITS file was generated
IRAF-TLM= '18:38:13 (17/12/2008)' / Time of last modification
APNUM1 = '1 1 '
WCSDIM = 1
CTYPE1 = 'LINEAR '
CRVAL1 = 997.
CRPIX1 = 1.
CDELT1 = 0.013
CD1_1 = 0.013
LTM1_1 = 1.
WAT0_001= 'system=equispec'
WAT1_001= 'wtype=linear'
DC-FLAG = 0
SFIT = '1 '
SFITB = '1 'Image header for nspec7.fits:
nspec7.fits[7693][real]:
No bad pixels, min=0., max=0. (old)
Line storage mode, physdim [7693], length of user area 1256 s.u.
Created Fri 11:03:28 19-Dec-2008, Last modified Fri 11:03:28 19-Dec-2008
Pixel file "nspec7.fits" [ok]
EXTEND = F / File may contain extensions
ORIGIN = 'NOAO-IRAF FITS Image Kernel July 2003' / FITS file originator
DATE = '2008-12-19T03:03:28' / Date FITS file was generated
IRAF-TLM= '11:03:28 (19/12/2008)' / Time of last modification
APNUM1 = '1 1 '
WCSDIM = 1
CTYPE1 = 'LINEAR '
CRVAL1 = 1010.
CRPIX1 = 1.
CDELT1 = 0.013
CD1_1 = 0.013
LTM1_1 = 1.
WAT0_001= 'system=equispec'
WAT1_001= 'wtype=linear'
DC-FLAG = 0
SFIT = '1 '
SFITB = '1 '
[quote:5b3c16ec87]The output of imstat might also be helpful. [/quote:5b3c16ec87]
imstat nspec5.fits
# IMAGE NPIX MEAN STDDEV MIN MAX
nspec5.fits 7693 0.8347 0.3613 0. 1.532imstat nspec7.fits
# IMAGE NPIX MEAN STDDEV MIN MAX
nspec7.fits 7693 0.8335 0.3815 -0.01138 1.976
thanks.,
Siti

Hi,Everything looks fine with the parameters, headers, and statistics. My only comment is that with only two spectra a median and average are the same.Because these are only 2 1D images they are not too big to send to me as attachments to an email to valdes@noao.edu. Otherwise you can put then in an anonymous ftp directory or a directory visible by http. Or you can transfer them to our anonymous ftp directory at iraf.noao.edu in the pub directory.It also did not occur to me that you could just send the combined FITS file to show me the problem. But I would still want to see the input spectra too.To make a gif from your splot that shows the "nonsense" values you type ":.snap g-gif" and ".gflush" when you have a graphics cursor. This will produce a file sgiNNNN.gif (where NNNN is a number) in your working directory. You can then put that on a web page, send it to me, or include it in a post.Yours,
FrankHere is a gif I made with specplot. The bottom two are the individual spectra and the top one is the result of scombine with your parameters.
[img:f16f882cbc]http://iraf.noao.edu/valdes/sgi6755.gif[/img:f16f882cbc]

Hi Frank,
[quote:2b19ec878e]Because these are only 2 1D images they are not too big to send to me as attachments to an email to valdes@noao.edu. [/quote:2b19ec878e]
I sent the 2 spectra and gif splot to your email. Haven't figured out yet how to upload the file. Yesterday I downloaded from MAST the same observations of the same star which were supposed to have been recalibrated using the latest reduction pipeline. Mine was done using an earlier version with the help of a friend. However I still get the same weird results. Thanks for looking at the spectra. Siti

Hello Siti,The problem is that the spectra are padded at the ends by zero. So where zero overlaps the non-zero data the median is reduced (since the median of two values is the average). In the figure below I show a specplot where the bottom two spectra (a,b) are your individual spectra, the 2nd from the top (c) is what you found with scombine but not doing anything about the zero padding, and the top (d) is what it should be as I'll describe. So in (c) you see a step down when the average of zero and the data is computed. This is your "nonsense" data.There are two solutions I can suggest. The most obvious is you trim the spectra to remove the zero values. This could be done with scopy specifying the wavelengths of the end points of the good data. An easier way is to set scombine.lthresh=0.001. This means that scombine will ignore any values less than 0.001. The potential problem is if the bottoms of the absorption lines actually reach zero. This latter case is what I did to generate (d):[code:1:e1de18ada6]scombine nspec5,nspec7 nspec57 lthresh=0.001[/code:1:e1de18ada6]Yours,
Frank Valdes[img:e1de18ada6]http://iraf.noao.edu/valdes/sgi19445.gif[/img:e1de18ada6]

Hi Frank,[quote:6053df5ab7]The problem is that the spectra are padded at the ends by zero.[/quote:6053df5ab7]
Sorry, for asking such a trivial question. I am not sure what you mean by "zero" as the plots and flux innocently show values different from zero. [quote:6053df5ab7]The potential problem is if the bottoms of the absorption lines actually reach zero.[/quote:6053df5ab7]
Some absorption lines do reach zero, but never mind at least I have some ideas on how to make adjustments to "scombine". I was playing around with the reject, and scale parameters but didn't think about "lthreshold". I really had to convince my supervisor to allow me to use Iraf and I don't think I could survive without this forum. I'm sure she will agree that the pain is worth it. Siti

Hi Frank,
[quote:79205e6da1]Sorry, for asking such a trivial question. I am not sure what you mean by "zero" as the plots and flux innocently show values different from zero.[/quote:79205e6da1]
Yeah The text file of the data do betray the zero values. I should have been more thorough.Siti