iraf.net

Hi all!At certain echelle instruments (e.g. X-shooter, ESPADON or our OES) there is a problem with the tilt of the arc lines which are not prependicular to the central line of an echelle order but the tilt is changing along the order as well. IN the worst case the arc line is even curved. Both ESPADON and X-SHOOTER are using their own pipeline to solve the problem. As far as I understand the IRAF documentation , the solution in IRAF is using the tasks FITCOORDS/TRANSFORM. They map the different parts of an arc line (even curved) to rectangular mesh to which the whole spectrum is converted as well.
This works nicely for longslit (single-order) spectra and has been used for removal of e.g. curved nigt-sky lines.
But the problem is in the dependency on database produced by task IDENTIFY. (and REIDENTIFY)On echelle spectra of such a more complicated instruments it is very difficult to work only with single-order dispersion solutions (and so using Idenitfy) but the echelle relation nad corresponding two-dimensional global dispersion solution implemented in task ECIDENTIFY is essential for the success. Is there a simple way (that could be done in cl scripts) how to join the FITCOORDS/TRANSFORM with ECIDENTIFY/ECREIDENTIFY database format?
If not, would it be possible to extend these corresponding tasks to be able to collaborate easily (perhaps in IRAF 2.15 ?) ?Petr Skoda

Hi Petr,I am well aware of this problem and it is not an uncommon question. The recommended solution is not TRANSFORM but to a feature of the extraction to define an order in the usual way (width, tracing, etc.) but to then extract the order in narrow subapertures (say 1 pixel wide). These are all then wavelength calibrated independenently but without resampling. Then you have a set of spectra for an order that can be added or combined in wavelength space where the curves of the arc lines (or sky lines) are accounted for by the dispersion function.If you search the forum by words like curved, tilted, echelle, and extraction you will find some responses I've made in the past. The bookkeeping is more complicated but there are special features in the tasks that were intended to support this approach to deailing with the problem.Yours,
Frank Valdes

Dear Frank,I am aware of the possibility of subapertures but I am afraid this is only a hypethehtical replacement for the fitcoords and transform. According to my experience with identify/ecidentify difficulties there is a very small chance to get all subapertures with identical features (some will be very faint at the edge of orders) and so the transformation will not be just smooth shifting but there will be spurious jumps in straighted lambda-pixel space making the lines not curved but jagged - so the sky subtraction and summing will not work well enough. There is further a bad experience with narrow aperture, as it makes periodic fluctuation of the flux due to the subpixel handling when the aperture passes next pixel on tilted line - According to my experience with fiber echelle spectra the aperture should be wide enough to overcome this artificial noise fluctiations. And it is a disaster with optimal extraction. So I am very afraid of 1pix wide subaperture (and we have even changing width of orders with order number due to prism in our OES)I feel that the smooth 2D rebinning used in FITCOORDS is more natural - in fact there may be outliers in the reidentify but the TRANSORM makes the smooth resampling following the surface .I did not yet try your suggestion, though, but I am wondering how many people really succeded and got smooth rebinninig done - In most of the mail list threads your subaps suggestion is not followed by final confirmation of success.
OTOH looking in database for id*.ms solution it is a list of fetures with x coordinates and wavelenght with some other parameters followed by description of polynomials.
The ec*.ec is just the same - only the order (relative and absolute) is prepended (and different polynomials written at the end).So if fitcoords uses the id*.ms record and then it works on this information and WCS in the image, I feel it could in principle use the ec file as well and operate on every spectrum extracted by apall with strip option.So I think only some ascii file processing could be used to convert the database formats (some text extraction corresponding to given order with some other polynomials coefficients rewritten).So my naive idea is to use an ascii preprocessing and some bookkeeping to manipulate data to set of order stripes on which the transform task will work as usually but the original wavelength solution is already global with all the advantages of robustness.I am pretty sure that preparing the ecfitcoords/ectransform in new IRAF version would help to lot of people working with modern echelles - my experience is that many astronomers are using IRAF for reprocessing spectra from raw data even that there are instrument pipelines for specific instruments ready (and despite all the effort of the pipeline producing teams)And now there are several instruments requiring the variable tilt and curvature compensation like ESPADON or X-SHOOTER to name the most important today.
Do you consider the idea of 2D echelle strips and database file processing to be feasible ? Could you think about its implementation in future IRAF ?Best regards,Petr Skoda

Hi Petr,You make very good points about the difficulties with narrow apertures. As you say, I have never had feedback from people after explaining the subaperture approach to people. My experience is that usually this means that it works well enough for them rather than them deciding not do this and not replying. I have not looked into what artifacts this might introduce. I wonder if the subpixel edge effects, which cause the wiggles with period related to the trace crossing pixel boundaries, would average out. I think for simple summing this would be the case. As you said, optimal extraction would not be a good idea with narrow apertures.You idea to manipulate the database information is fine. It also reminds me that when doing a dispersion solution based on a neighboring solution there are options to only determine a shift. This would allow finding a full solution from the center, or even on an extraction of a wider aperture. In that case even weak or missing lines will not be as critical.Frank

Hello,I am one more person dealing with this problem. I tried different ways and
finally obtained a satisfactory solution in the way that I describe in the
following. Yet I still have one problem left for which I'll appreciate very much
advice.1) I use an order tracking image like a flat or bright star to identify
extraction apertures with APALL2) using the previously defined apertures as reference I extract the
orders of a Th-Ar spectrum setting the parameter nsubaps equal to the aperture
width in pixels, as suggested. In this way I obtain a number of 1D spectra. The
curvature of the orders is gone because tracked by APALL and I am left
to deal with the curvature of the slit.3) I work on the bunch of 1D spectra but problems start to arise because
of the APNUM keyword of the header.I obtained my best results re-combining with IMSTACK all 1D spectra corresponding to one order in a 2D spectrum, then IDENTIFY, REIDENTIFY, FITCOORD and TRANSFORM.That works great but IRAF starts to complain about APNUM# missing. If I add
those keywords I can go on but in the database/idfile I get something like
this:# Fri 17:42:59 11-Feb-2011
begin identify ord_2 - Ap 0
id ord_2
task identify
image ord_2 - Ap 0
aperture 0
aplow INDEF
aphigh INDEF
units Angstroms
features 6
378.53 4433.72825 4433.838 3.0 1 1 ThAr Line
410.61 4431.04467 4430.996 3.0 1 1 ThAr Line
419.69 4430.27217 4430.189 3.0 1 1 ThAr Line
591.69 4414.58108 4414.486 3.0 1 1 ThAr Line
651.20 4408.68299 4408.882 3.0 1 1 ThAr Line
733.03 4400.17784 4400.096 3.0 1 1 ThAr Line
function legendre
order 3
sample *
naverage 1
niterate 0
low_reject 3.
high_reject 3.
grow 0.
coefficients 7
2.
3.
378.5255126953125
733.0348510742188
4417.666696670034
-16.77520827869601
-0.7136509993230779and so on, and that does not work with FITCOORD! I need to edit the idfile to remove the aperture identification and have this:begin identify ord_2[*,1]
id ord_2
task identify
image ord_2 [*,1]
aperture 0
aplow INDEF
aphigh INDEF
units Angstroms
features 6
378.53 4433.72825 4433.838 3.0 1 1 ThAr Lineand so on, for all the lines (spectra) of the 2D spectrum and finally have FITCOORD work properly.Could you please advice how to fix the problem with the APNUM keyword and get a proper idfile without having to edit it? setting the APNUM1 keyword to null in the initial 1D spectra does not help.I hope this has a simple solution because the final result is really good. Any other method, as working with the individual 1D spectra or combine them differently did not work as well.Thanks,
Leonardo

I am not sure what you mean by problems with the APNUM. Let start with an example. I created a test longslit spectrum and extracted with nsubaps=5. This produced test.ms. If I list the spectra I get:[code:1:12917fd001]
apextract> slist test.ms
test.ms 1 1 1 -1 1. 1. 512 Example artificial long slit image
test.ms 2 1001 1 -1 1. 1. 512 Example artificial long slit image
test.ms 3 2001 1 -1 1. 1. 512 Example artificial long slit image
test.ms 4 3001 1 -1 1. 1. 512 Example artificial long slit image
test.ms 5 4001 1 -1 1. 1. 512 Example artificial long slit image
[/code:1:12917fd001]I did identify and reidentify without problem. But when I try dispcor I getERROR: test.ms - Missing reference for aperture 1001I don't know why this is happening and I will have to investigate. Is this what you are finding?Frank Valdes

Hello,I believe it is related to that. I extract 1D spectra with APALL from each order of the echelle and then combine them again into a 2D spectrum: one 2D spectrum per order.1) when I try to display this new 2D spectrum I get
ERROR: segmentation violation
not a big deal as I can still load the image on DS9, but not nice.2) when I try to run IDENTIFY on this 2D spectrum I get
Warning: Image header parameter not found (APNUM2)to have it works I need to add by hand the keywords APNUM2, 3, 4.... n where n is the number of dispersion lines in the 2D spectrum.3) the real problem occurs when I run REIDENTIFY because in the idfile of the image, written in the directory database, each dispersion line is identified by aperture number, as indicated in the example that I reported previously, not by pixel, so FITCOORDS fails on it.Thanks a lot for any help you may provide.Leonardo

I have not had time to look at this recently. From your last message, I think the quickest way to understand at least one problem is if you can provide me with the spectrum the produces the segmentation error and further produces the messages about the aperture number. If you want to send me the file as an attachment (valdes@noao.edu) or you can put the file by anonymous ftp at iraf.noao.edu/pub and let me know via email.Frank Valdes

Hello Frank,
I am sending two of my spectra: a ThAr lamp and the spectrum of Sirius, both bias subtracted. As you can see the ThAr spectral lines in the low part of the spectrum (small y) are elongated and tilted. That's what I want to correct.My procedure is the following:
1) use the spectrum of Sirius to define the extraction apertures, I identify 47 orders automatically
2) use those apertures to extract the ThAr spectrum with Apall and
format=onedspec
nsubaps=4in this way I obtain 47x4 1D spectra. Now I have 2 options.Option A
I use Imstack to recombine the 4 spectra of each order in a 2D spectrum (1024x4), then use identify, reidentify, fitcoords and transform. In this case when I run identify I get:
Warning: Image header parameter not found (APNUM2)If I add blank APNUM keywords in the header I can continue but finally end into the fitcoords problem described in my post, because reidentify works by aperture and not by pixel.Option B
I work on the 1D spectra, for each order I identify lines in one of the 1D spectra and reidentify the others. The only way I found to have this work was to set the keyword APNUM1 to null otherwise IRAF would keep getting confused by it.Applying a number of tricks I managed to get to the end of both methods and found that Option A provides a better correction than B, at least in my case. By the way, if you could have some more direct way to deal with this it would be really great.Thanks a lot for your help------------------------------------------------------Hi,As with many IRAF tasks there are a number of options and I realize it is hard to be aware of them all. To do what you describe as the better option A you would use the "strip" output of apall.First find the apertures and trace them in your reference image. Set the aperture limits such that you will be able to map the arc lines in the extracted strip (e.g. lower=-2, upper=2). Use nsubaps=1. In other words, the nsubaps feature is not suitable for this approach.Now extract the arc spectrum using a format of "strip" and the reference parameter set to the image that you used to define the apertures. You might specify an output rootname such as "order". This will produce images <rootname>.nnnn where the nnnn will be the aperture number from the reference.Now you can use the longslit package to trace and rectify the 2D slitlets. Note that you probably want to set nsum=1 in identify/reidentify and step=1. You can use either wavelengths or pixels for the reference line (usually the middle). You don't need a spatial fit since the trace in apall will have taken care of that. Finally you use transform to produce the interpolated slitlet with the tilts removed. In this approach I did not encounter any confusion about the aperture number.Frank ValdesA hint, after you get a slitlet which have a small number of lines you can use specplot to display them graphically and, after zooming appropriately, you can visually see how the arc lines align both before and after rectification.