Welcome to iraf.net Saturday, September 23 2017 @ 02:42 PM GMT

IRAF v2.15 and 64-bit -- A Status Report

  • Tuesday, December 01 2009 @ 06:20 AM GMT
  • Contributed by:
  • Views: 3,985
IRAFNET

Normally this space is reserved for major release announcements, but I was recently at two conferences and was surprised by the level of concern about 64-bit support and future IRAF releases in general and thus a status report seemed in order....

With the IRAF V2.15 release now in preparation you can expect the following: If you are a user, you'll see 64-bit supported and all its benefits (e.g. large file/memory support and a 10-15% speed increase from native binaries); If you are a developer, you (may) have some minor work to do to update your package for 64-bit support, but will, in the end, have more robust code due to stronger type-checking now imposed by the system. See the full story for detailed notes, but for the impatient, here are the highlights:

  • The 64-bit port is complete and in testing.
  • A planned V2.15 release will allow for user testing and updates needed for external packages. This release features:
    • A single linux architecture for 32-bit systems.
    • A simplified build and install procedures (1-file download!).
    • Compile-time prototype checking of code to reduce errors.
    • A new MEMIO interface to detect pointer over/underflows, provide memory garbage collection & usage reporting, and more.
    • SVG graphics device for better web-presentation of plots.
    • Numerous bug fixes and enhancements.

We plan an extended Beta release sometime after the holidays to resolve any platform integration issues, to allow time for package maintainers to update their code as needed, and to provide a longer test period for the 64-bit applications that may have lingering, undiscovered bugs.

Please post any comments, questions or concerns to the forum or contact us directly.

64-Bit Port Status

The port of IRAF to 64-bit platforms is now essentially complete and the system is now undergoing regression testing and platform integration. Work has progressed at a relatively slow pace due to competing priorities and to-date has taken about 4-5 weeks of total effort to get a stable system that meets the desired goal of minimizing required code changes in the core system and external packages. The tests scripts we use to validate new ports indicate there are still a few minor issues to be resolved, and at the moment work is focused on producing a single code base that works well on all 32 and 64-bit systems. We also need to ensure that seldom-used interfaces such as IMFORT still work, that machine independence is still maintained between platforms where it should be, and that image formats/data files and protocols remain backward-compatible with earlier releases so that unmodified packages continue to be useful.

The approach taken for the port was to promote all integers to the same size as pointers (8-bytes), this is known as an ILP64 port (i.e. int/long/ptr all 64-bits) which is somewhat incompatible with the LP64 model used by the GCC compilers where integers remain 32-bit values. However, implicit in much of IRAF code from its inception is an intermixing of int/pointers in applications code; simply promoting integers preserves this relationship at the expense of some extra memory use (and a twinge of guilt for past bad behavior). Because we can control how SPP code is converted to the final compiled language, we can overcome the fact that GCC doesn't directly support the ILP64 model by promoting int to long in the intermediate-to-final languages. Indeed, the majority of the time spent on the port was in modifying C code/interfaces and not SPP. We believe this systematic approach minimizes the changes required for 64-bit support of IRAF/SPP code and will do much of the work of porting external packages that may no longer be actively maintained in order to keep them viable for future releases. We will supply binaries and source patches for as many packages as possible, however actively maintained package developers are expected to update their code since we will provide only minimal and obvious updates and not the proper inspection of code required for true 64-bit compatibility.

The compromise here is that the relative size of real values in data structures has now changed (i.e. single-precision real remains 32-bits, but the size of a structure element has increased to 64-bits). What this means for developers is that the use of Memr in data structures needs to be examined and perhaps changed, however the advantage is that it is only the use of Memr that can be a potential new error and not the many possible interactions of changing all int/pointer occurrences.

To ease the transition for developers, and because datatype-size changes now frequently lead to fatal runtime errors, the compile-time system was updated to provide automatic prototype checking (i.e. are VOS or other SPP procedures called with the proper number of args? Are arguments of the right type? etc.) to identify potential problems. Likewise, error/warning messages are now used to identify uses of Memr that require programmer attention. It is important to note that current 32-bit binaries will continue to run, these changes are required to produce 64-bit binaries or indeed to compile against the V2.15 core system on 32-bit systems. Code changes required will be detailed in the release notes, suffice it to say that even large packages can be properly updated in about a day or less, and runtime errors can quickly be tracked to the use of Memr at the offending line of code. The port itself as well as the runtime testing has revealed a number of long-standing bugs in the system, we hope an extended test period will discover other bugs in applications not covered by port testing.

IRAF V2.15 Release Planning

While the bulk of the work thus far has been on the 64-bit upgrade, this only really makes sense in the context of a new IRAF release. Due to the (sometimes) significant changes to internal system interfaces, a major release rather than a patch is appropriate. This also gives us an opportunity to address issues like the installation process that affect all platforms. Further, in order to properly evaluate output from the 64-bit port, we need to first verify that system changes produce the same results as earlier 32-bit systems, and then that 64-bit port works the same way. So, we will have both 32 and 64-bit versions of V2.15 available for testing anyway.

We expect to have the first public Beta release available shortly after the new year and will issue updates and support for additional platforms thereafter as needed. Because we want testing to reveal as many bugs as possible, all of the optional compile/runtime checks will be enabled as a default. During this time we will be relying on users to report errors they see and we hope they understand the nature of the test phase. For the final release, most of these checks will be made optional and are intended primarily for developers to test their code.

V2.15 Features

The final release notes will document the full list of changes, however some comments on specific new features are previewed below:

    A Single linux Architcture: The SuSE arechitecture was dropped in the V2.14 release, still leaving separate redhat and linux architectures. Initially these were because of differences in Glibc between platforms but this is no longer relevant and the difference now is mainly in dynamically vs statically linked binaries. For V2.15 we plan to have a single linux architecture for all Linux distributions (the 64-bit port will be linux64 since it is incompatible with 32-bit systems). To maintain compatibiity with external packages, the system will still fallback to looking for a redhat binary on appropriate platforms if no linux binary can be found.

    A Simplified Install Procedure: The 3-file distribution was done originally in the days of centralized servers where a single IRAF installation might beed to support multiple platforms. Single-user systems are the norm now and so we'll be changing the way the distributions are built so there will be only a single file on each architecture. If time permits, additional changes to allow network-based installs or simplified external package installation might also be implemented.

    A Simplified Build Procedure: Although few users build the core IRAF system from pure source, the multi-step process has proven a challenge. Tools we use in porting the system will be made part of the system in order to provide a simple "make"-like facade to compile the system from source. Other utility commands will also be provided.

    A New MEMIO Interface: As stated above, the MEMIO interface in V2.15 has been enhanced to not only support 64-bit systems, but modernized to provide better runtime checking and control over its behavior. Changes include:
    • Optional clearing of allocated space, i.e. all allocations behave as calloc would. This can be useful for working around bugs caused by unitialized pointer allocations.
    • Automatic overflow and underflow detection. Space that goes beyond it's allocated size will trigger an error when freed.
    • Attempt to free pointers more than once, or with the wrong data type will be detected.
    • Automatic memory garbage collection at task termination. The amount of leaked memory can optionally be reported.
    • Optional reporting of memory usage statistics by the task.


    An SVG Graphics Device: SVG (Scalable Vector Graphics) is an XML-based format that allows vector graphics to be embedded directly in web pages. As opposed to scaling image data (e.g. a GIF of an iraf plot), an SVG plot can be rendered at almost any size without loss of readability.

Numerous bug fixes and documentation enhancements will also be included in this release. Many of these fixes can be attributed to work contributed by Chisato Yamauchi in his IRAF64 project, and to Jason Quinn for his diligence in correcting and updating documentation.

Please post any comments, questions or concerns to the forum or contact me directly.

-Mike
IRAF v2.15 and 64-bit -- A Status Report | 9 comments | Create New Account

The following comments are owned by whomever posted them. This site is not responsible for what they say.

  • IRAF v2.15 and 64-bit -- A Status Report
  • Authored by: favilac on Thursday, December 03 2009 @ 02:43 PM GMT
I think is great you will have a unified linux package.
However, which linux distribution will be your testbed? From history I guess it will be RHEL 5 but it is getting old and RHEL 6 seems to be released next year.

Ubuntu is getting more popular (I maintain a script to install IRAF on it and I get a lot of traffic lately) and LTS releases have been quite solid and predictable as they are released every two years.

Anyway, which distro will be the base? :)
  • IRAF v2.15 and 64-bit -- A Status Report
  • Authored by: fitz on Thursday, December 03 2009 @ 04:40 PM GMT
The original reason for the split (long ago) was incompatibilities between the GLibc5 used by RedHat and GLibc4 used by everyone else. This is no longer the case (and hasn't been for a while) and the distribution used doesn't matter so much anymore as long as we're careful. There might be reasons to have both static and dynamically linked binaries at some point, but RHEL vs Ubuntu vs Debian vs whatever are mostly in the user interface and not the bits iraf cares about.

The point about a single linux distro is to simplify the building of IRAF distributions (it takes a while to build 8 PCIX systems from a common code base, especially when I have to scrounge machines or configure virtual machines to do it). Note v2.15 will also have a simpler build-from-source option if you're so inclined.
  • IRAF v2.15 and 64-bit -- A Status Report
  • Authored by: andrew cooke on Wednesday, December 16 2009 @ 02:23 AM GMT
Better type checking sounds really neat. And 64bits.... at last! Congratulations!
  • IRAF v2.15 and 64-bit -- A Status Report
  • Authored by: jturner on Wednesday, January 06 2010 @ 04:28 PM GMT
It's good news that this is nearly done and indeed a bit unexpected (I wasn't aware the work was ongoing in earnest). We'd be happy to take the beta for a spin when it's ready.

Cheers,

James.
  • IRAF v2.15 and 64-bit -- A Status Report
  • Authored by: jturner on Thursday, January 28 2010 @ 07:57 PM GMT
Hi Mike,

Just another thought -- will this new release work with gcc 4.x.y compilers? In the past I've had to specify gcc 3 when building external packages etc. (at least on 64-bit machines), but the gcc3 compatibility packages will probably cease to exist in many Linux distros before too long.

Thanks,

James.
  • IRAF v2.15 and 64-bit -- A Status Report
  • Authored by: fitz on Thursday, January 28 2010 @ 08:01 PM GMT
The port was first done with gcc 4.3 (by accident, I didn't
bother to check the gcc version) but I've now switched to gcc
3.4 for compatibility reasons. I'm hoping the beta period will
reveal issues with shared libs we can fix without having to
resort to static binaries.
  • IRAF v2.15 and 64-bit -- A Status Report
  • Authored by: jturner on Tuesday, February 09 2010 @ 08:26 PM GMT
I was just wondering what platforms you plan to support for 2.15. Are you likely to continue with Solaris SPARC (etc.)? I think our SPARC users have largely evaporated now and we're considering focusing just on Linux and MacOS.

Cheers,

James.
  • IRAF v2.15 and 64-bit -- A Status Report
  • Authored by: fitz on Tuesday, February 09 2010 @ 08:40 PM GMT
All the currently supported platforms will continue to be supported in v2.15, including SunOS/Solaris for Sparc (we still have machines in use at domes that require this). Since the core system is supported, external packages shouldn't be a problem to compile, but I don't see a great need to supply a full set of binaries unless requested.

Where changes may come into play are on Mac systems: Unless I add yet another architecture providing 64-bit Mac Intel binaries is an issue TBD. What I'm looking at is making 'macosx' an i386 universal (PPC and Intel) binary and moving 'macintel' to 64-bit x86_64 binaries. That, or we have both a 32-bit and 64-bit 'macintel' and leave it to users to download the correct binaries. As I said, this is still being decided, rough numbers suggest 1/3 of users still run Tiger so we can't assume everyone has a shiny new Mac.
  • IRAF v2.15 and 64-bit -- A Status Report
  • Authored by: jturner on Tuesday, February 09 2010 @ 10:57 PM GMT
Yes, there is certainly a proliferation of different types of Macs, which had me a bit worried (not least becase I don't use them). It seems that the PPCs are rapidly becoming extinct though, now that (as I understand it) Apple seems to have stopped supporting them properly.

We've recently got away from Solaris for operations.

Thanks,

James.


Privacy Policy
Terms of Use

User Functions

Login