iraf.net

Our acquisition script uses tvmark to overlay a simple line on a displayed image indicating the slit position, like so:
[code:1:2d6ed59427]printf ("%9.3f %9.3f 301 s \n", l_x1, l_y1, > tmpin)
printf ("%9.3f %9.3f 301 s \n", l_x2, l_y2, >> tmpin)
tvmark (l_frame, "", commands=tmpin, color=205, pointsize=5, int-)[/code:1:2d6ed59427]
When running IRAF on a remote redhat machine and connecting to the ximtool or DS9 on port 5137, the actual image display is fast, but then drawing the slit on top of it is pathologically slow. This is independent of whether ximtool/DS9 is running locally on the observer's workstation (with the remote IRAF connecting to port 5137 on the observer's machine across the network) or on the remote machine (with IRAF connecting to the local port 5137 and DS9 displaying remotely to the observer's machine) -- either way, the annotation is slow when using the TCP port for image display, taking 1-2 minutes to draw. The same operation is much faster when running both IRAF and ximtool/DS9 locally on the observer's machine (again with port 5137). In that case the annotation takes around 12 seconds, even though this machine is slower than the remote Linux box with which the problem occurs. The problem also goes away if both IRAF and DS9 are run on the remote Linux box, communicating via local FIFO instead of port 5137. I just wondered whether Mike or anyone else has some idea what causes this slowness and how to avoid it?Thanks!James.

PS. It's more convenient for us to run the display tool locally and connect to it over the network from the remote IRAF session if it's possible to speed that up somehow.I also tried testing inet:5137:localhost on the remote machine, instead of its proper name, but that makes no difference.

Hi James,I'll have to investigate it further, but I can reproduce the problem.I think the problem may be on the IRAF side: In the case of TVMARK, the frame buffer is mapped as an image so that normal IMIO interfaces could be used to mark the frame. A series of magical interfaces then carry this through to actual network calls to write to the display, however a simple image display is a more direct write to the socket connection. I'm not sure yet why the unix socket would be any faster but that appears to be the case; as such, running IRAF and the ximtool on the same machine and using X to remotely display the window should still be faster (for display, however contrast scaling will likely be slower).You might look at the CDL library as a quick-n-dirty way to simply write a line to the display (e.g. see the cdl_markLine() procedure). This should be much faster, I can supply an example that takes the endpoints as cmdline args if needed.Cheers,
-Mike

Hi Mike,Many thanks for the reply. I'm glad you were able to reproduce this. The conversion to an image array makes sense, since otherwise there would only be a few bytes of information to transfer.
[quote:05d6345661]You might look at the CDL library as a quick-n-dirty way to simply write a line to the display (e.g. see the cdl_markLine() procedure). This should be much faster, I can supply an example that takes the endpoints as cmdline args if needed.
[/quote:05d6345661]
In the longer term, we have a rewrite of the acquisition script in Python in progress (it's really rather big/repetitive for CL), but unfortunately resources seem to have been diverted from that next year (same old). However, I'd be quite interested if you could post the CDL example whenever it's convenient. Apart from solving this problem quickly, I'd quite like to understand how CDL compares with other options for writing to the display tool in general, whether from SPP or Python.Thanks,James.

James,It appears there's a new issue with ximtool in how the readback is being done (it *used* to work). I'll fix that for the next release along with some other things, but the CDL program I had in mind was simply:[code:1:a3cf103f9e]
#include <stdio.h>
#include <stdlib.h>
#include "cdl.h"/* MARKLINE -- Use the cmdline args to draw a line on the display.
**
** Compile As: cc -o markline markline.c -lcdl
**
** Usage: markline x1 y1 x2 y2
*/
int main (int argc, char **argv)
{
CDLPtr cdl;
int arg=1, lx, ly, ux, uy;
/* Parse the arguments to get the line endpoints.
*/
if (argc < 5) {
fprintf (stderr, "Usage: markline x1 y1 x2 y2\n");
exit (1);
} else {
lx = atoi (argv[arg++]);
ly = atoi (argv[arg++]);
ux = atoi (argv[arg++]);
uy = atoi (argv[arg++]);
} /* Open the frame buffer, use IMTDEV to specify the device.
*/
if ((cdl = cdl_open ((char *)getenv("IMTDEV"))) == (CDLPtr) NULL) {
fprintf (stderr, "Error opeing display device.\n");
exit (1);
} /* Draw a line on the display at those coords.
*/
printf ("Line: from: (%d,%d) to: (%d,%d)\n", lx, ly, ux, uy);
if (cdl_markLine (cdl, lx, ly, ux, uy, C_YELLOW)) {
fprintf (stderr, "cdl_markLine() returns an error\n");
exit (1);
} /* Close the connection.
*/
(void) cdl_close (cdl); return (0);
}
[/code:1:a3cf103f9e]CDL is a C library so it can be wrapped by Python, but it's much more feature-complete than something like numdisplay. Not only does it support a variety of marker types, image operations, text annotation, but also deleting those markers. See the x11iraf$cdl directory, specifically the doc and examples subdirectory.Even though this is currently broken for 24-bit ximtool, you can experiment with DS9 for the time being.Cheers,
-Mike

Great, thanks. That should help a lot to get started. Actually, the ximtool that's installed locally on the observer's workstation is the old one, so maybe it will work. We'll be switching to Linux in a couple of months though, so the timing of the 24-bit ximtool is pretty good for those of us who weren't looking forward to being stuck with DS9 for observing.James.

When I run this example program, it connects to the ximtool but no annotation actually appears there. It seems to do nothing, but if, for example, I insert a "cdl_clearFrame()" that works. Could there be some step missing?I didn't have an x11iraf$cdl directory, but I found the CDL tarball on the NOAO page and got the manual and examples from there, so I can probably figure this out with a bit more time.Thanks,James.

Having investigated this further, the speed seems to depend mainly on the machine where IRAF is running, rather than ximtool, consistent with your comment that the problem is probably on the IRAF/TVMARK side. For some reason, things are slower with IRAF running on a fast Linux box than on an older Sun, independently of whether the ximtool is local or remote and what hardware/OS it is on. This is true for multiple Linux boxes and multiple Suns. A new 3.4GHz 8-core, 16Gb PC cannot send an image + annotation to its [b:2f47683fab]own[/b:2f47683fab] DS9 TCP port as fast as the old Sun can... I wonder what kind of quirk or OS optimization could be responsible for that (or maybe a difference between IRAF 2.12 and 2.14)?I'll try again to get the CDL program working.James.

OK, I might need to skip a few details in the first instance, to keep this reasonably concise.Emma and I have been trying to get your CDL program working, with reference to the manual (desparate times!), and have run into problems that may be related to the image buffer size not being recognized correctly. Moreover, the CDL annotation still seems slow across a TCP socket, just like TVMARK, though I haven't done a careful timing comparison yet.Here are some observations:

I just tried the original (TVMARK) script on a PC running IRAF 2.12 and it was fast! This is with remote display to one of the same machines I tried before. So now I suspect the annotation slowness is new in IRAF 2.14, since both the old Suns and an oldish PC running IRAF 2.12 are much faster than even a top-of-the-line PC running 2.14. To prove this, I'll try to find a machine that I can run both IRAF 2.12 and 2.14 on.Update: Ack, it's just as slow with IRAF 2.12 on my laptop as 2.14. Another thing the fast machines have in common, compared with the slow machines, is an old OS, but that seems like a long shot (some are Solaris, some Linux)...James.

Yep, the script displays fast to both Ximtool and DS9 on my laptop from yet another machine with an old OS (Fedora 2), running IRAF 2.14.Could this be something to do with X libraries that are dynamically loaded on the client side (where IRAF is running)? Those libraries could have changed on both Linux and SunOS between the older machines that send annotations to Ximtool/DS9 quickly and the newer machines that are very slow to do that over a TCP port. After trying probably a dozen permutations of different machines, I really think it must be something like the X library versions. [b:75ca9a93b8][see below][/b:75ca9a93b8]James.

OK, I thought about this some more and there probably are no X libraries involved in sending data from an IRAF task to a TCP port. I believe that's done using the socket library (libsocket in Solaris or just the standard library, glibc, in Linux). I was probably getting a bit carried away hoping I'd figured it out before the weekend.I also found an example where a machine with an old Linux version displays fast to a machine with a new Linux version (much faster than the latter does to itself) but slowly to a Sun that can display fairly fast to itself... so I'm just not sure what is causing the fast vs. slow gear change between different permutations of machines. I'll need to stop spending much time on this though and just change to a remote DS9 with Unix sockets where it's a problem.Cheers.James.

Hi James,Sorry for the slow replies, I've been under the weather lately. The difference between TVMARK and the CDL program is that in the first case the image display is mapped as a file and some internal magic may be subject to buffering to explain the slow updates, whereas in CDL there is a direct command to the server to execute the update. I would not expect there to be any significant difference to using DS9 over ximtool in the first case.However, in the second case: The "old" way to write subrasters to the display was to read-back the entire width of the frame buffer, edit the pixels and write back the change (i.e. to draw a column in imtgmos you'd read 6400 pixels, change one value and write it back out). This is still the way DS9 does it, because it uses the (old) ximtool IIS code. In the new ximtool, a true subraster write is supported (i.e. a column is written without the full-frame readback-edit cycle). The problem I mentioned with ximtool before appears to be in reading back the image subraster, e.g. when you draw a diagonal line you still need to readback the full box represented by the diagonal even if this is a subraster of the full frame buffer. I haven't had a chance to check this out yet, but trust me, when it works it'll be much faster and way cool 8-)In any case, communication to the display is done using standard sockets. Setting a DEBUG_IIS environment variable before you start the display server will show you all the traffic over the socket from the server point of view. Nothing was explicitly changed since v2.12 that might affect this, I'd have to look to see if some of the FIO buffers changed. I'll let you know when I poke around some more.Cheers,
-Mike

Hi Mike,Thanks for the info. I hope you're feeling better.I haven't compared the TVMARK versus CDL speed carefully, but to first order it seems that the TCP port slowness still exists with the latter, so the readback issue is probably dominating the extra buffering time in TVMARK at present.Anyway, it sounds like you're on top of this already. I could post the DEBUG_IIS results, but if you can reproduce the behaviour there is probably no need(?). On the other hand, maybe it will help understand why some combinations of machines are faster than others. I might look into that quickly.James.