iraf.net

Hi, again.

I'm still doing x11iraf archaeology and now I was looking for

https://iraf.net/ftp/iraf/x11iraf/x11iraf-v1.5DEV-src.tar.gz

but the file isn't found. (Link mentioned in several IRAF.net threads.) The ftp server went down earlier today but the the announcement thread said the ftp server should be running again. The 1.5DEV release was never put on ftp://iraf.noao.edu/iraf/x11iraf/ so it's not there either.

I believe the release history was 1.3.1, 1.5DEV (sort of an unofficial release, I guess), and then 2.0BETA. Does my recollection jive with yours on that? Were there any intermediate releases?

Cheers,
Jason

Hi Jason,

All the versions are available from ftp://iraf.noao.edu/iraf/x11iraf, older versions were put into a subdirectory and thus broke the links here. The missing version reflect internal changes that were never released at the time.

Mind if I ask what all the digging is for, or do you just wanna surprise me later?

Cheers,
-Mike

Hi, Mike.

What subdirectory is that? The file https://iraf.net/ftp/iraf/x11iraf/x11iraf-v1.5DEV-src.tar.gz existed at some point (see "Redhat 64-bit and xgterm" thread) but ftp://iraf.noao.edu/iraf/x11iraf doesn't mention anything at all about the 1.5DEV release. I've had a binary version of the 1.5DEV release for years but I never archived the source distribution on my local machine.

At the moment I'm building a dependency graph for X11IRAF so that I can visualize its overall structure. The important part is knowing which external packages/libraries were used and which versions. I'm still working on it but I already have a very nice graph that that includes everything up to right before the building of xgterm, ximtool, and obmsh. I'll send you a copy when I'm happy with it. At the very least it will help people understand the structure of X11IRAF without too much effort.

After I understand the structure myself, I'll try seeing if any external dependencies can be upgraded to more recent packages. Even though attempts to fix it were made, I'm still having varargs.h trouble from HTML-PSformat.c (a file from Mosaic browser!) so I want to fix that. After that I would like trying to switch the build system over to autotools. There's also some other issues too that should be fixed (FWF files are missing their required license file, etc.).

As for the dependency graph, the really problematic directory is ObmW. There are files from two external sources thrown in there: some from the FWF (Free Widget Foundation) and some from libhtmlw from the Mosaic browser. Each of these have two difficult problems. The best I can currently say is that the FWF files in X11IRAF are from before version 3.54. Unfortunately, I cannot find such early FWF tarballs anyway on the web and they may be lost to history. The libhtmlw files appear to be a grabbed from several different releases of Mosaic (or perhaps several independent releases of libhtmlw?) but I haven't identified precisely which ones. All the releases of Mosaic and stand-alone libhtmlw I can find at best only have a few files that are identical, even excluding the obvious NOAO mods. Speaking of mods, both the FWF and libhtmlw files have been modded by NOAO so the history of these files is complicated and may not extend from a single release. If NOAO still has backups from around the time of X11IRAF 1.0, it would be great to grab whatever FWF and Mosaic (or libhtmlw) tarballs were used as original sources. This would allow me to identify which changes are actually NOAO changes.

Anyway, that's what I'm currently tinkering with.

Cheers,
Jason

I'm not sure what happened to the x11iraf-v1.5DEV directory, it was probably lost during a server move at some point. In any case, I've put it back in ftp://iraf.noao.edu/iraf/x11iraf but only as the source tarball.

The other widgets in the ObmW library grew "organically" shall we say. The HTML widget I remember came from the Mosaic browser when it was still an NCSA product (I even got email from a pre-billionaire Marc Andreeson about it), the rest were collected as needed. Given the age of some of this code it would be a major undertaking to get it 64-bit compatible and so the code is essentially frozen at this point. XGterm would be worth keeping around but would likely need to also be re-done as just a newer XTerm with only the GTerm graphics widget, i.e. we'd lose the rest of the GUI capability. Sadly, that's not high on the list either.

For anybody interested, I'm continuing to look at the X11IRAF codebase when the mood strikes me and my spare time allows. Most recently I've been examining the xgterm directory. It took a bit of investigating but I think I have recovered most of the xgterm development history (at least for the 2.0BETA release). At first I thought it was hopeless so it was a pleasant surprise.

The following files appear to be original and written by NOAO/AURA:

gtermio.c
gtermio.esc
gtermio.h
NOTES

while these next files are original but probably just leftover "junk" from development:

ptyx_new.h (appeared in X11IRAF 1.5DEV)
.pure (appeared in X11IRAF 1.3)
_3d (appeared in X11IRAF 1.3)
mkp (appeared in X11IRAF 1.5DEV)
_spool (appeared in X11IRAF 2.0BETA)

The remainder of the files appear to have an X11 origin. The following chart I made is useful to understand and discover this X11 history. X11 used a file versioning system that includes individual file versions and dates. Releases of X11 had patch files to update them. For instance, X11 Release 5 (R5) had 26 public patches and Release 6 (R6) had 13 public patches. The chart below shows which release and patchlevel is consistent with the version and date stamp in the X11IRAF file. As can be seen, most files were unmodified over many patches or even releases.



It's clear that no single release/patchlevel was the origin for all files in the xgterm directory. In fact, no single X11 release was even the origin for all files. It appears development started with R5 and proceeded during the releases of R6 by applying patches selectively and/or partially. Overall, R6 under patchlevels 1,2,3 is most similar to X11IRAF2.0BETA and all equally so.

Right now the only extra complication to this history is that it seems a small amount of (unattributed) code related to color-handling from XFree86 xterm or color_xterm was ported into the X Consortium X11 xterm code. It seems to have affected at least a couple files (button.c and charproc.c) but maybe more. I'm still deciphering this.

The good news is that basically all the files from R5 (which are largely documentation or headers) can be trivially updated to their R6 counterpart. In other words, with very little effort the entire xgterm codebase can be synchronized to R6 patchlevel 3, allowing a single point of origin. This tremendously simplifies the task of understanding and possibly improving the xgterm code for future hackers. I have a git repo evolving collecting a bunch of improvements to X11IRAF. I haven't yet started a branch for xgterm. Mike, if you recall and notice any errors or omissions in what's written above, I'd be happy to know before I make changes in my repo. Also, what exactly what the connection between NOAO and AURA during the development of X11IRAF? Many of the man pages have a AURA copyright notice (this is a minor problem too since they are often based on the X11 man pages). I don't know anything about the connection between AURA and NOAO.

Cheers,
Jason

AURA is the management entity for NOAO (and STScI and Gemini and NSO), in the same way that AUI is the head office for NRAO. Thus, the template copyright statement refers to AURA, although older files may reference NOAO if they were written before the lawyers settled the matter once and for all.

In general terms, XGterm was built with the idea of taking the stock XTerm code and replacing their Tektronix graphics widget with the (IMHO) much improved GTerm widget. GTerm is part of the Object Manager (OBM) interface used to built GUIs, and so various widget toolkits were imported into OBM to provide buttons, layouts, etc. The GUIAPPS package contains some of the GUI-fied IRAF tasks, but you can think of the XGterm plotting window as the simplest form of GUI.

The starting point was whatever was the current version of XTerm. Over the years, Linux and platforms evolved, the pseudo-terminal code changed, and so the text-terminal side of the code was patched as needed to support new machines or OS versions. The program comments warn "There be dragons....", and it's true. At a later point the XTerm code was almost entirely re-written to clean it up, but by then we weren't actively developing XGterm and so creating a new "master" source just wasn't a priority.

At this point my guess is that it would be cleaner to just start with the latest XTerm code and replace the graphics window rather than try to manage the patches. I think it is also worth considering whether it might just not be better to start a new graphics terminal project altogether, i.e. a native OSX terminal with proper Tektronix emulation or a multi-platform terminal built on Qt or some widget framework other than X11. The goal here would be to simply provide a Tek graphics window, we'd just lose the GUI capability in XGterm now (or perhaps re-write the OBM well using the new widget set). And then since XImtool is a standalone GUI based on the same GTerm widget we'd need to revisit that, and would likely also want to build on a different foundation (e.g. optimize for today's gigapixel cameras and not the "large" 512x512 detectors of the early nineties. So, you can see how the project grows to the point where it will probably never happen, and so we'll just keep it running as long as we can.

A Git repo would certainly help community users keep it running, especially if you did the patch merger.

Someday, a volunteer might create a proper successor to X11IRAF based on new widget sets, although it would no longer be X11IRAF. Until then those of us who like IRAF are stuck with X11IRAF.

As I see it, the hurdle to creating a successor is that to an outsider who didn't help develop either xterm or xgterm, it's difficult to know how they differ; that is, what features were added, removed and what code was changed and why. Knowing what makes xgterm xgterm would be required to create an adequate successor and to answer those questions precisely, one must know the current code base well. The main thing was, of course, putting the "g" in xterm (that is, the graphics window) but there's more changes to xterm than just that.[1] So, I think polishing the existing code and simpling the build from 2.0BETA is low time investment but high value because it would ease the barrier for comprehension and thus would allow interested volunteers to get up to speed more easily and quickly. This is what I've been doing in my git repo: mostly just maintenance stuff that should extend the life of X11IRAF itself and no focus on feature changes at all (e.g., making sure built objects are seperated from the non-built ones, adding missing headers, tweaking code to remove trivial compiler warnings, and so on).

I could send my repo to you if you like so you can see the kind of commits I've made. I have your gmail e-mail from past correspondence.

[1] Besides the color functions I mentioned above and that I believe were backports from the XFree86 release of X11, there's also some I18N work that I've yet to determine where it came from. It's not the in MIT, XC, or XFree86 releases, nor other xterm-related releases I've checked. It might be NOAO additions but I wouldn't have expected this to have been done.

Cheers,
Jason

PS The second commit in my repo is to fix a GCC version detection bug that prevents compilation of X11IRAF on modern systems. (The first is merely to remove and stop tracking object and old, unused files that were present in the 2.0BETA tarball.) You give a fix to the GCC bug in the following thread

https://iraf.net/forum/viewtopic.php?showtopic=1465213

This bug is so important that it could warrant an exception to a code freeze and constitute the official 2.0 (that is, non-BETA) release. Assuming my first commit did not introduce any issues, it could even be a release based on my git tree.