iraf.net

Normally this space is reserved for major release announcements, but I was recently at two conferences and was surprised by the level of concern about 64-bit support and future IRAF releases in general and thus a status report seemed in order....

With the IRAF V2.15 release now in preparation you can expect the following: If you are a user, you'll see 64-bit supported and all its benefits (e.g. large file/memory support and a 10-15% speed increase from native binaries); If you are a developer, you (may) have some minor work to do to update your package for 64-bit support, but will, in the end, have more robust code due to stronger type-checking now imposed by the system. See the full story for detailed notes, but for the impatient, here are the highlights:

• The 64-bit port is complete and in testing.
• A planned V2.15 release will allow for user testing and updates needed for external packages. This release features:
• A single linux architecture for 32-bit systems.
• A simplified build and install procedures (1-file download!).
• Compile-time prototype checking of code to reduce errors.
• A new MEMIO interface to detect pointer over/underflows, provide memory garbage collection & usage reporting, and more.
• SVG graphics device for better web-presentation of plots.
• Numerous bug fixes and enhancements.

We plan an extended Beta release sometime after the holidays to resolve any platform integration issues, to allow time for package maintainers to update their code as needed, and to provide a longer test period for the 64-bit applications that may have lingering, undiscovered bugs.

Please post any comments, questions or concerns to the forum or contact us directly.

The port of IRAF to 64-bit platforms is now essentially complete and the system is now undergoing regression testing and platform integration. Work has progressed at a relatively slow pace due to competing priorities and to-date has taken about 4-5 weeks of total effort to get a stable system that meets the desired goal of minimizing required code changes in the core system and external packages. The tests scripts we use to validate new ports indicate there are still a few minor issues to be resolved, and at the moment work is focused on producing a single code base that works well on all 32 and 64-bit systems. We also need to ensure that seldom-used interfaces such as IMFORT still work, that machine independence is still maintained between platforms where it should be, and that image formats/data files and protocols remain backward-compatible with earlier releases so that unmodified packages continue to be useful.

The approach taken for the port was to promote all integers to the same size as pointers (8-bytes), this is known as an ILP64 port (i.e. int/long/ptr all 64-bits) which is somewhat incompatible with the LP64 model used by the GCC compilers where integers remain 32-bit values. However, implicit in much of IRAF code from its inception is an intermixing of int/pointers in applications code; simply promoting integers preserves this relationship at the expense of some extra memory use (and a twinge of guilt for past bad behavior). Because we can control how SPP code is converted to the final compiled language, we can overcome the fact that GCC doesn't directly support the ILP64 model by promoting int to long in the intermediate-to-final languages. Indeed, the majority of the time spent on the port was in modifying C code/interfaces and not SPP. We believe this systematic approach minimizes the changes required for 64-bit support of IRAF/SPP code and will do much of the work of porting external packages that may no longer be actively maintained in order to keep them viable for future releases. We will supply binaries and source patches for as many packages as possible, however actively maintained package developers are expected to update their code since we will provide only minimal and obvious updates and not the proper inspection of code required for true 64-bit compatibility.

The compromise here is that the relative size of real values in data structures has now changed (i.e. single-precision real remains 32-bits, but the size of a structure element has increased to 64-bits). What this means for developers is that the use of Memr in data structures needs to be examined and perhaps changed, however the advantage is that it is only the use of Memr that can be a potential new error and not the many possible interactions of changing all int/pointer occurrences.

To ease the transition for developers, and because datatype-size changes now frequently lead to fatal runtime errors, the compile-time system was updated to provide automatic prototype checking (i.e. are VOS or other SPP procedures called with the proper number of args? Are arguments of the right type? etc.) to identify potential problems. Likewise, error/warning messages are now used to identify uses of Memr that require programmer attention. It is important to note that current 32-bit binaries will continue to run, these changes are required to produce 64-bit binaries or indeed to compile against the V2.15 core system on 32-bit systems. Code changes required will be detailed in the release notes, suffice it to say that even large packages can be properly updated in about a day or less, and runtime errors can quickly be tracked to the use of Memr at the offending line of code. The port itself as well as the runtime testing has revealed a number of long-standing bugs in the system, we hope an extended test period will discover other bugs in applications not covered by port testing.

While the bulk of the work thus far has been on the 64-bit upgrade, this only really makes sense in the context of a new IRAF release. Due to the (sometimes) significant changes to internal system interfaces, a major release rather than a patch is appropriate. This also gives us an opportunity to address issues like the installation process that affect all platforms. Further, in order to properly evaluate output from the 64-bit port, we need to first verify that system changes produce the same results as earlier 32-bit systems, and then that 64-bit port works the same way. So, we will have both 32 and 64-bit versions of V2.15 available for testing anyway.

We expect to have the first public Beta release available shortly after the new year and will issue updates and support for additional platforms thereafter as needed. Because we want testing to reveal as many bugs as possible, all of the optional compile/runtime checks will be enabled as a default. During this time we will be relying on users to report errors they see and we hope they understand the nature of the test phase. For the final release, most of these checks will be made optional and are intended primarily for developers to test their code.

The final release notes will document the full list of changes, however some comments on specific new features are previewed below:

A Single linux Architcture: The SuSE arechitecture was dropped in the V2.14 release, still leaving separate redhat and linux architectures. Initially these were because of differences in Glibc between platforms but this is no longer relevant and the difference now is mainly in dynamically vs statically linked binaries. For V2.15 we plan to have a single linux architecture for all Linux distributions (the 64-bit port will be linux64 since it is incompatible with 32-bit systems). To maintain compatibiity with external packages, the system will still fallback to looking for a redhat binary on appropriate platforms if no linux binary can be found.

A Simplified Install Procedure: The 3-file distribution was done originally in the days of centralized servers where a single IRAF installation might beed to support multiple platforms. Single-user systems are the norm now and so we'll be changing the way the distributions are built so there will be only a single file on each architecture. If time permits, additional changes to allow network-based installs or simplified external package installation might also be implemented.

A Simplified Build Procedure: Although few users build the core IRAF system from pure source, the multi-step process has proven a challenge. Tools we use in porting the system will be made part of the system in order to provide a simple "make"-like facade to compile the system from source. Other utility commands will also be provided.

A New MEMIO Interface: As stated above, the MEMIO interface in V2.15 has been enhanced to not only support 64-bit systems, but modernized to provide better runtime checking and control over its behavior. Changes include:
• Optional clearing of allocated space, i.e. all allocations behave as calloc would. This can be useful for working around bugs caused by unitialized pointer allocations.
• Automatic overflow and underflow detection. Space that goes beyond it's allocated size will trigger an error when freed.
• Attempt to free pointers more than once, or with the wrong data type will be detected.
• Automatic memory garbage collection at task termination. The amount of leaked memory can optionally be reported.
• Optional reporting of memory usage statistics by the task.


An SVG Graphics Device: SVG (Scalable Vector Graphics) is an XML-based format that allows vector graphics to be embedded directly in web pages. As opposed to scaling image data (e.g. a GIF of an iraf plot), an SVG plot can be rendered at almost any size without loss of readability.

Numerous bug fixes and documentation enhancements will also be included in this release. Many of these fixes can be attributed to work contributed by Chisato Yamauchi in his IRAF64 project, and to Jason Quinn for his diligence in correcting and updating documentation.

Please post any comments, questions or concerns to the forum or contact me directly.