iraf.net

Hi,I received a month ago my first set of Gemini observations. This is GMOS spectra in MOS mode. My first surprise was that I did not find a cookbook or similar to reduce the observations. The second surprise was that the Gemini help desk does not work at all.So I began to reduce the data almost without any help but I have now a big problem that I cannot solve and I would appreciate any help.I'm using IRAF V2.13 and Gemini1.10 package.The problem is the following; I'm using gsreduce to process the CuAr spectra
and the result is that there is an offset of about 15 pixels on the spatial axis (i.e. Y axis), and therefore spectra are cut in half. In other words, let y_i be the y coordinate where spectrum i begins (spectra are about 30 pixels wide on y), then for some unknown reason, gsreduce is cutting the spectra from y=y_i+15 to y=y_i+45 instead from y=y_i to y=y_i+30.I already checked the mask and it's ok. thanks
charly

Hello Charly,Firstly, I'd like to apologise for the unfortunate experience you had with the Gemini HelpDesk. The Gemini HelpDesk is organised into three levels or tiers, as described here: http://www.gemini.edu/sciops/helpdesknew/structure-and-responsibilities. The first points of contact for queries from the community are the National Gemini Offices (NGOs). I can assure you that the Gemini HelpDesk does work, it was just unfortunate that the NGO for your country that was assigned to your ticket has not replied to your request yet.The Gemini IRAF package does have a number of cookbooks to assist PIs in processing their data. After starting IRAF and loading the gemini package and then the gmos package, run the task "gmosexamples". This will provide you with a list of example cookbooks. Type "MOS" at the prompt to see the MOS cookbook example.For me to assist you with your query, please can you provide a list of commands that you have executed up to and including gsreduce, along with the parameters used (type "lpar <taskname>" at the IRAF prompt to list the parameters for a given task <taskname>.Also, we recommend using IRAF v2.14.1 with Gemini IRAF v1.10.I look forward to hearing from you.Many thanks,Emma (Gemini Observatory)

Hi Emma,
thank you very much for your answer.
I sent two mails to the helpdesk (request #000003637, on May 2dn, and request #000003661, on May 31th) and I did not receive any answer.About the gmosexamples, I think that (for a complete ignorant like me) it is pretty obscure. The first time I reduced spectra I used the User's Guide to CCD Reductions with IRAF (Massey, 1997), and I needed no other help to do the work.
Using this guide you understand each single step in the reduction process, which of course is not the case for the gmosexamples cookbook.Ok, about the reduction process I followed, these are the steps:
- gprepare (objects, flats, and arcs)
- gsflat
- gsreduce, for arcs with the following parameters:
- GSREDUCE ga*
inimages = "ga*" Input GMOS images or list
(outimages = "") Output images or list
(outpref = "gs") Prefix for output images
*(fl_over = no) Subtract overscan level
*(fl_trim = yes) Trim off the overscan section
*(fl_bias = yes) Subtract bias image
(fl_gscrrej = no) Clean images for cosmic rays
(fl_dark = no) Subtract (scaled) dark image
*(fl_flat = no) Apply flat field correction
*(fl_gmosaic = yes) Mosaic science extensions
*(fl_fixpix = no) Interpolate across chip gaps if mosaicing
*(fl_gsappwave = yes) Run gsappwave on reduced image
*(fl_cut = yes) Cut slits into separate spectra if mosaicing
(fl_title = yes) Put object id in title of cut spectra (MOS only)
(fl_vardq = no) Create variance and data quality frames
*(bias = "bias.fits") Bias image nameand that's it. The result of this task is an image with the arc spectra stored in image[SCI,n] where n indicates the mask slit. At this point you can observe that spectra were cut with an offset of 15 pixels on the spatial direction. So basically you have half spectrum, half noise background.Emma, thanks again for your help.
charly

Hello again Charly,Sorry, perhaps I should have used the word "example" rather than "cookbook" ... the examples that are in the package are just that ... examples. For more information about GMOS MOS reductions, please look at the gmosinfo and gmosinfospec tasks (type "gmosinfo" and "gmosinfospec" at the IRAF prompt). You can also look at the help files for a particular task for further, specific details (type "help <taskname>" at the IRAF prompt).Ideally, I need exact copies of the commands you executed, along with the full parameter list for *all* tasks. Do you have a large number of slits in your MOS mask? Did you keep the imcombined flat from your call to gsflat (i.e., did you have the "fl_keep" parameter set to yes and did you define a file name in the "combflat" parameter)? Did you use this "combflat" as a reference (specified by the "refimage" parameter) when you called gsreduce (you did not provide the full set of parameters for gsreduce).The "combflat" file has the x and y positions of the slits in its MDF extension. This "combflat" file is often used as a reference to define the slit positions when cutting the arc / science data. It is likely that either you are not using this as a reference when cutting the arc. Another possibility is that your MOS mask contains a large number of slits, which means that the software may have gotten a little confused when trying to detect the slit edges.Please provide me with exact copies of the commands you executed, along with the full parameter list for *all* tasks so that we can figure out exactly what is going on.Many thanks,Emma

Hi Emma,
thanks for your mail.ok, regarding to your questions:>> Do you have a large number of slits in your MOS mask?
I have 44 slits in the mask.>> Did you keep the imcombined flat from your call to gsflat?
No, I did not. FL_KEEP parameter was set to 'no' in the gsflat task. The COMFLAT parameter was left empty. For your information I have only one flat per central wavelength, per nigth (497, 502 and 507 nm).>> Did you use this "combflat" as a reference?
The REFERENCE parameter in the gsreduce task is empty.
Here is the complete list of the gsreduce parameters: inimages = "gtS65.fits" Input GMOS images or list
(outimages = "") Output images or list
(outpref = "gs") Prefix for output images
(fl_over = no) Subtract overscan level
(fl_trim = yes) Trim off the overscan section
(fl_bias = yes) Subtract bias image
(fl_gscrrej = no) Clean images for cosmic rays
(fl_dark = no) Subtract (scaled) dark image
(fl_flat = no) Apply flat field correction
(fl_gmosaic = yes) Mosaic science extensions
(fl_fixpix = no) Interpolate across chip gaps if mosaicing
(fl_gsappwave = yes) Run gsappwave on reduced image
(fl_cut = yes) Cut slits into separate spectra if mosaicing
(fl_title = yes) Put object id in title of cut spectra (MOS only)
(fl_vardq = no) Create variance and data quality frames
(bias = "gS20110329S0110_bias.fits") Bias image name
(dark = "") Dark image name
(flatim = "Flat_562") Flatfield (output of GSFLAT) image
(geointer = "linear") Interpolation to use if mosaicing
(gradimage = "gtS64.fits") Image to use for finding slit edges using the gradient
(refimage = "") Reference image for slit positions
(key_exptime = "EXPTIME") Header keyword for exposure time
(key_biassec = "BIASSEC") Header keyword for overscan strip image section.
(key_datasec = "DATASEC") Header keyword for data section (excludes the overscan)
(fl_inter = no) Interactive cosmic ray cleaning fitting
(rawpath = "") GPREPARE: Path for input raw images
(sci_ext = "SCI") Name of science extension
(var_ext = "VAR") Name of variance extension
(dq_ext = "DQ") Name of data quality extension
(key_mdf = "MASKNAME") Header keyword for the Mask Definition File filename
(mdffile = "") MDF file to use if keyword not found
(mdfdir = "gmos$data/") MDF database directory
(bpm = "") Bad Pixel Mask filename
(gaindb = "default") Database with gain data
(gratingdb = "gmos$data/GMOSgratings.dat") Gratings database file
(filterdb = "gmos$data/GMOSfilters.dat") Filters database file
(xoffset = INDEF) X offset in wavelength [nm]
(yoffset = INDEF) Y offset in unbinned pixels
(yadd = 0.) Additional pixels to add to each end of MOS slitlet lengths
(bpmfile = "gmos$data/chipgaps.dat") Info on location of chip gaps
(key_ron = "RDNOISE") Header keyword for readout noise
(key_gain = "GAIN") Header keyword for gain (e-/ADU
(ron = 3.5) Readout noise value to use if keyword not found
(gain = 2.2) Gain value to use if keyword not found
(sat = 65000) Saturation level in raw images
(key_nodcount = "NODCOUNT") Header keyword with number of nod cycles
(key_nodpix = "NODPIX") Header keyword with shuffle distance
(ovs_flinter = no) Interactive overscan fitting
(ovs_med = no) Use median instead of average in column bias
(ovs_func = "chebyshev") Overscan fitting function
(ovs_order = 1) Order of overscan fitting function
(ovs_lowr = 3.) Low sigma rejection factor
(ovs_highr = 3.) High sigma rejection factor
(ovs_niter = 2) Number of rejection iterations
(nbiascontam = 4) Number of columns removed from overscan region
(biasrows = "default") Rows to use for overscan region
(logfile = "") Logfile
(verbose = yes) Verbose output?
(status = 1) Exit status (0=good)
(scanfile1 = "") Internal use only
(scanfile2 = "") Internal use only
(mode = "ql")you can check the steps I followed from this link:
http://kellecruz.com/gmos/gemini_reduction.htmlFollowing your questions I did gsflat again using fl_keep set to 'yes' and combflat set to 'comb_502'. Then I ran gsreduce setting refimag = comb_502 and I get the following:ERROR - GSCUT: reference comb_502.fits MDF does
not have slit edge info.Please let me know if you would need the images. I can put them somewhere for you to download.Emma, thank you very much for your patience.
charly

Hello Charly,The web page you mentioned in your message is for GMOS longslit reduction, not GMOS MOS reduction. This is probably why you are having problems with your GMOS MOS reduction.There are a number of important differences between reducing GMOS MOS data and reducing GMOS longslit data. Please look at the MOS example provided in the GMOS package (type "gmosexamples MOS" at the IRAF prompt). You need to run gscut after gsflat to update the MDF with the correct slit positions. All the details are provided in the MOS example.If you are still having problems after working through the MOS example that is provided in the package, please let me know.Many thanks,Emma

well, that's really surprising!
I reduced the flux calibration star without problems, but of course, this is a long slit spectrum.So I guess I will begin with reductions again.
Thank you very much for your support.
charly

No problem, Charly

Emma,I must be pretty dumb, since there is no way I can reduce the hold data set with the only help of the gmos example. Would you be so nice to help me with that?regards,
charly

Hi Charly,Of course How far did you get in the reduction process before you had problems? If you got any error messages, please can you provide the full message and I will try and help you ...Thanks,Emma

Hi Emma,thanks for your response.Well, I did not get very far. In fact, I'm quite confused since the first step in the gmos example is to make the flat field. It lists the task parameters and that's it.I only have 3 flats (lamp) one for each central wavelength (497, 502 and 507 nm).
From the header I see the detector was not binned. However, science object frames are binned (2x2). Is this ok?I also have 2 biases. One is binned and the other is not.. I'm worried about the order of the fitting function: n=29! This is quite high. Have the flats such structure? What are the regions I should check to see if the fitting is good?well, I think it's enough by now.regards
charly

Hello Charly,There are some comments in the GMOS MOS example before the call to gsflat that discuss the detection and cutting of the MOS slits.You should process data at different wavelengths separately. The help file for gsflat states "GSFLAT accepts as input either a single flatfield image or a list of images which are combined using IMCOMBINE before the response function is determined".Your flats, biases and science data should all have the same binning. I took a look at your program in the Gemini Science Archive and there are appropriate flats available for your science data. Look for the CCDSUM keyword in the headers of the pixel data extensions (e.g., [SCI,1]).There is some structure in the flats that do require a high order when fitting. You can run gsflat interactively (set the fl_inter parameter to yes) to adjust the fit until you are happy with it.I hope that helps to get you started. Please let me know if you have any other questions.Many thanks,Emma

Ok, thanks to your answer I now have the flats. I've also kept the combflat images, which I called comb_cw.fits (cw=central wavelenght).However, in the next step the gmos example gets darker. I understand that previous to gsreduce the science observations I have to set the resolution to gscrrej (which I did) and then gscut the comb frame created with gsflat.I observe that for this routine inimage = gradimage. In my case comb_502.fits.
But when I run the task I have: ERROR - GSCUT: gradimage must be GMOSAICed.that for me means 'Game Over'.Obviously, I'm missing something but I don't know what.thanks a lot again,
charlypd: what's the parameter in gsreduce that specifies the reference for cutting the slits? Is it refimag or gradima?

Hello Charly, The error from gscut states that the gradimage must be GMOSAICed. The gradimage (comb_502.fits) should have been gmosaiced automatically by gsflat. The only way for gsflat not to gmosaic the data is if the fl_detec parameter in gsflat is set to yes. Please ensure that the fl_detec parameter is set to no.Generally when using the examples in the Gemini IRAF package, it is assumed that all the other task parameters that aren't mentioned in the example are set to their default value. You can ensure this by initialising your uparm directory before starting your reduction (type "rm -rf uparm/*" at the terminal prompt in your iraf home directory).As stated in the GMOS MOS example, in the comments before the call to gsreduce, "'refimage' contains information on the slit locations that was just found by GSCUT"Many thanks,Emma

Hi Emma,I have partial success reducing data. After gsreduce science data I visually checked each spectrum and I noticed that a few (4 out of 40) spectra were not cut properly. I was trying to solve the problem by myself but I could not. So, I'm looking for your help once more For your information I followed each detail specified in the gmos example. I think the problem could be the gscut routine. Please, find below the parameters used. inimage = "comb_497.fits" Input image
(outimage = "") Output image
(secfile = "") Output file for image sections
(fl_update = yes) Update inimage header if no output image produced
(fl_vardq = no) Propagate variance and data quality planes
(gratingdb = "gmos$data/GMOSgratings.dat") Gratings database file
(filterdb = "gmos$data/GMOSfilters.dat") Filters database file
(gradimage = "comb_497.fits") Image to use for finding slit edges using the gradient method
(refimage = "") Reference image for slit positions
(xoffset = INDEF) X offset in wavelength [nm]
(yoffset = INDEF) Y offset in unbinned pixels
(yadd = 0.) Additional pixels to add to each end of MOS slitlet lengths
(w2 = INDEF) Upper wavelength limit of cut spectra (nm)
(sci_ext = "SCI") Name of science extension
(var_ext = "VAR") Name of variance extension
(dq_ext = "DQ") Name of data quality extension
(key_gain = "GAIN") Header keyword for gain (e-/ADU)\n
(key_ron = "RDNOISE") Header keyword for readout noise
(ron = 3.5) Readout noise in electrons
(gain = 2.2) Gain in e-/ADU
(logfile = "") Logfile
(verbose = yes) Verbose
(status = 0) Exit status (0=good)
(mode = "ql") Please let me know any other information you would need.
regards and thanks
charly

Hi Charly,Partial success is better than no success! Your parameters to gscut look good. The solution to your problem can be found in the help file to gscut:"GSCUT prints warnings for slit locations that appear to be mismatched. The warnings are issued if the slit length found from the edge-detection algorithm deviates from the MDF information by more than 3 pixels. ... GSCUT makes a reasonable effort to recover from such apparently mismatched slit locations. The user is urged to inspect the results closely to evaluate if the cutting was done correctly. If the user finds that some of the slit locations are indeed incorrect, the easiest way to recover is to edit the MDF in the output image to contain the correct locations, then use that image at the refimage for a rerun of GSCUT. The MDF can be edited using the tools in TABLES.TTOOLS."You may have noticed the following warning on the 4 problem extensions when running gscut:[code:1:2923884522]
WARNING - GSCUT: Slitlength from gradimage more than 3 pixels different from MDF info
Using lower edge + MDF info to get upper edge
[/code:1:2923884522]
To check the positions of slits, you will need to look at the MDF attached to your "combflat". To do this, you can use the tread task in the tables package (type "tread comb_497.fits[MDF]" at the IRAF prompt). Use your arrow keys to navigate to columns 25, 26, 27, 28. These columns (titled SECX1, SECX2, SECY1 and SECY2, respectively) contain the section that defines each slit in your pre-cut flat. You should check that the coordinates defined in these columns do indeed match with the slits in your pre-cut image.Unfortunately, there is not an easier, less painful solution.Please let me know how you get on,Many thanks,Emma

Thanks Emma, I'm happy now.
I also realized that the flats were a mess. gscut also failed at this point. So I used
the corrected comb_image in the gsflat routine (with fl_keep = no), and now seems to be all ok.Anyway, I would like you to take a look at the reduced spectra. I really do not like the background structure (due to flat field correction).regards,
charly

Hi again,Emma following the gmos example I realized that it assumes you only have one central wavelength, or just one exposure. In my case, I have 3 central wavelengths per science object and I'm wondering at which step I should combine these 3 spectra.I guess I should combine them once they are extracted (1D spectrum), and wavelength and flux calibrated. Is this correct?regards,
charly

Hi Charly,Sorry for the delay in my reply. I have been speaking with the GMOS Instrument Scientist (IS) about your questions and I have some answers for you.Firstly, your flat has been over-fitted. This means that the fringes present in your flat were fitted in addition to the continuum. Only the continuum of the flat needs to be fit. This over-fitting caused the weird features that you noticed in your normalised flat. Due to the fringing present in your flat frames, the IS recommends fitting the flat detector by detector. To do this you need to set the fl_detec parameter to yes (apologies to telling you to set this to no in an earlier post). When fitting the first extension (CCD1) of your flat (i.e., the red end, which is where the fringing is at its worse) you should use a low order so that you don't fit the fringes. An order of 5-7 should suffice. The second extension (CCD2) of the flat has less fringing, but in order not to fit the fringes, use an order of 10-15. The third extension (CCD3) of the flat (i.e., the blue end) shouldn't have much fringing at all. This is when it is ok to use an order of ~29. This will fit the structure (that doesn't include fringing) in the flats as mentioned in a previous post.The result of running gsflat with the fl_detec parameter set to yes means that the output "combflat" will not be gmosaiced, which will cause problems when running gscut (as you have already discovered). To resolve this issue, just run gmosaic on the "combflat" before running gscut. I also spoke to the IS about your combining question. There are two methods you can try; you can use gemcombine to combine your transformed spectra (i.e., the output from gstransform) prior to extraction (set the gemcombine.offsets parameter to "wcs") OR you can extract the individual spectra and then use scombine to combine the extracted spectra. You can test both methods and decide for yourself which you prefer I hope that answers all your questions.Many thanks,Emma

Emma,I sincerely appreciate your support. Thanks to your help I now know the fundamentals on how to reduce GMOS data, and it is pretty clear that this would not be possible with the only help of the GMOS example.
The other (and not less important) conclusion is that Gemini users urgently need clearer and more detailed cookbooks.
I owe you a big one.regards
charly

Hi Emma again!I'm still trying with GSFLAT following your recipe (low order for ccd1 and high order for ccd3) and I still do not like the flat I get. Please, download the flat (f_497.fits) at http://iate.oac.uncor.edu/~charly/ and let me know your opinion.
I still see fringing there. What do you think?regards,
charly

Hello Charly,Sorry, I didn't get an e-mail notification about your message and so I have only just found it.Your flat definitely looks better. Just to re-iterate, only the continuum of the flat needs to be fit ... you should still have fringes present in your processed, normalised flat. However, the orders I specified in my last post work the best for the R400 grating. I realise now that your data uses the B600 grating. The fringes are not as severe with the B600 grating as they are with the R400 grating, so it is ok to use a higher order when fitting the continuum. For the B600 grating, using an order of 15-19 for all detectors should work well.Also, I was just reminded about the talks that are available at http://www.noao.edu/meetings/gdw/ ... in particular, Kathy Roth's talk (http://www.noao.edu/meetings/gdw/files/Roth_NS_Reduction.pdf) shows step-by-step instructions on how to fit a flat for the R400 grating. Remember that since the B600 grating does not show as severe fringing as the R400 grating that you should use higher orders when fitting B600 spectroscopic flats.I apologise for not being able to give you all this information up front ... this has been a learning experience for me as well! Hopefully you now have enough information to create the perfect flat Many thanks,Emma

Hi Emma,once more thanks for your support.charly