Normally this space is reserved for major release announcements,
but I was recently at two conferences and was surprised by the level of
concern about 64-bit support and future IRAF releases in general and thus a status report seemed in order....
With the IRAF V2.15 release now in preparation you can expect the following: If you are a user, you'll see 64-bit supported and all its benefits (e.g. large file/memory support and a 10-15% speed increase from native binaries);
If you are a developer, you (may) have some minor work to do to update
your package for 64-bit support, but will, in the end, have more robust code due to stronger type-checking now imposed by the system.
See the full story for detailed notes, but for the impatient, here are the
highlights:
The 64-bit port is complete and in testing.
A planned V2.15 release will allow for user testing
and updates needed for external packages. This release
features:
A single linux architecture for 32-bit systems.
A simplified build and install procedures (1-file download!).
Compile-time prototype checking of code to reduce errors.
A new MEMIO interface to detect pointer over/underflows,
provide memory garbage collection & usage reporting, and
more.
SVG graphics device for better web-presentation of plots.
Numerous bug fixes and enhancements.
We plan an extended Beta release sometime after the holidays to resolve
any platform integration issues, to allow time for package maintainers to update
their code as needed, and to provide a longer test period for the 64-bit
applications that may have lingering, undiscovered bugs.
Please post any comments, questions or concerns to the forum or
contact us directly.
64-Bit Port Status
The port of IRAF to 64-bit platforms is now essentially complete and the
system is now undergoing regression testing and platform integration. Work has progressed at a relatively slow pace due
to competing priorities and to-date has taken about 4-5 weeks of total effort to get a stable
system that meets the desired goal of minimizing required code changes in the core system and external packages.
The tests scripts we use to validate new ports indicate there are still
a few minor issues to be resolved, and at the moment work is focused on
producing a single code base that works well on all 32 and 64-bit systems.
We also need to ensure that seldom-used interfaces such as IMFORT still
work, that machine independence is still maintained between platforms
where it should be, and that image formats/data files and protocols remain backward-compatible with earlier releases so that unmodified packages continue to be useful.
The approach taken for the port was to promote all integers to the
same size as pointers (8-bytes), this is known as an ILP64 port
(i.e. int/long/ptr all 64-bits) which is somewhat incompatible with the
LP64 model used by the GCC compilers where integers remain 32-bit
values. However, implicit in much of IRAF code from its inception is an intermixing of
int/pointers in applications code; simply promoting integers preserves this relationship at
the expense of some extra memory use (and a twinge of guilt for past bad behavior). Because we can
control how SPP code is converted to the final compiled language, we
can overcome the fact that GCC doesn't directly support the ILP64 model
by promoting int to long in the intermediate-to-final languages. Indeed, the majority of the time spent on the port was in modifying C code/interfaces and not SPP. We believe
this systematic approach minimizes the changes required for 64-bit support of IRAF/SPP code and will do much of the work of porting external packages that may no longer be actively maintained in order to keep them viable for future releases. We will supply binaries and source patches for as many packages as possible, however actively maintained package developers are expected to update their code since we will provide only minimal and obvious updates and not the proper inspection of code required for true 64-bit compatibility.
The compromise here is that the relative size of real values
in data structures has now changed (i.e. single-precision real remains
32-bits, but the size of a structure element has increased to 64-bits).
What this means for developers is that the use of Memr
in data structures needs to be examined and perhaps changed, however the
advantage is that it is only the use of Memr that can be
a potential new error and not the many possible interactions of changing
all int/pointer occurrences.
To ease the transition for developers, and because datatype-size changes
now frequently lead to fatal runtime errors, the compile-time system was
updated to provide automatic prototype checking (i.e. are VOS or other SPP
procedures called with the proper number of args? Are arguments of the right type? etc.)
to identify potential problems. Likewise, error/warning messages are now
used to identify uses of Memr that require programmer attention. It is important to
note that current 32-bit binaries will continue to run, these changes are
required to produce 64-bit binaries or indeed to compile against the V2.15
core system on 32-bit systems. Code changes required will be detailed
in the release notes, suffice it to say that even large packages can be
properly updated in about a day or less, and runtime errors can quickly
be tracked to the use of Memr at the offending line of code. The port
itself as well as the runtime testing has revealed a number of long-standing
bugs in the system, we hope an extended test period will discover other
bugs in applications not covered by port testing.
IRAF V2.15 Release Planning
While the bulk of the work thus far has been on the 64-bit upgrade,
this only really makes sense in the context of a new IRAF release. Due
to the (sometimes) significant changes to internal system interfaces, a
major release rather than a patch is appropriate. This also gives us an
opportunity to address issues like the installation process that affect all
platforms. Further, in order to properly evaluate output from the 64-bit
port, we need to first verify that system changes produce the same results
as earlier 32-bit systems, and then that 64-bit port works the same way. So, we will have both 32 and 64-bit versions of V2.15 available for testing anyway.
We expect to have the first public Beta release available shortly after
the new year and will issue updates and support for additional platforms
thereafter as needed. Because we want testing to reveal as many bugs
as possible, all of the optional compile/runtime checks will be
enabled as a default. During this time we will be relying on users to
report errors they see and we hope they understand the nature of the
test phase. For the final release, most of these checks will be made
optional and are intended primarily for developers to test their code.
V2.15 Features
The final release notes will document the full list of changes, however some comments on
specific new features are previewed below:
A Single linux Architcture:
The SuSE arechitecture was dropped in the V2.14 release, still
leaving separate redhat and linux architectures.
Initially these were because of differences in Glibc between
platforms but this is no longer relevant and the difference now
is mainly in dynamically vs statically linked binaries. For
V2.15 we plan to have a single linux architecture for all
Linux distributions (the 64-bit port will be linux64 since
it is incompatible with 32-bit systems). To maintain compatibiity
with external packages, the system will still fallback to looking
for a redhat binary on appropriate platforms if no
linux binary can be found.
A Simplified Install Procedure:
The 3-file distribution was done originally in the days of
centralized servers where a single IRAF installation might beed to
support multiple platforms. Single-user systems are the norm
now and so we'll be changing the way the distributions are built
so there will be only a single file on each architecture. If
time permits, additional changes to allow network-based installs
or simplified external package installation might also be implemented.
A Simplified Build Procedure:
Although few users build the core IRAF system from pure source, the
multi-step process has proven a challenge. Tools we use in porting
the system will be made part of the system in order to provide a
simple "make"-like facade to compile the system from source. Other
utility commands will also be provided.
A New MEMIO Interface:
As stated above, the MEMIO interface in V2.15 has been enhanced to
not only support 64-bit systems, but modernized to provide better
runtime checking and control over its behavior. Changes include:
Optional clearing of allocated space, i.e. all allocations
behave as calloc would. This can be useful for working
around bugs caused by unitialized pointer allocations.
Automatic overflow and underflow detection. Space that goes
beyond it's allocated size will trigger an error when freed.
Attempt to free pointers more than once, or with the wrong
data type will be detected.
Automatic memory garbage collection at task termination.
The amount of leaked memory can optionally be reported.
Optional reporting of memory usage statistics by the task.
An SVG Graphics Device:
SVG (Scalable Vector Graphics) is an XML-based format that allows
vector graphics to be embedded directly in web pages. As opposed
to scaling image data (e.g. a GIF of an iraf plot), an SVG plot can
be rendered at almost any size without loss of readability.
Numerous bug fixes and documentation enhancements will also be included
in this release. Many of these fixes can be attributed to work contributed
by Chisato Yamauchi in his IRAF64 project, and to Jason Quinn for his
diligence in correcting and updating documentation.
Please post any comments, questions or concerns to the forum or
contact me directly.
The following comments are owned by whomever posted them. This site is not responsible for what they say.
IRAF v2.15 and 64-bit -- A Status Report
Authored by:
favilac on
Thursday, December 03 2009 @ 07:43 AM MST
I think is great you will have a unified linux package.
However, which linux distribution will be your testbed? From history I guess it will be RHEL 5 but it is getting old and RHEL 6 seems to be released next year.
Ubuntu is getting more popular (I maintain a script to install IRAF on it and I get a lot of traffic lately) and LTS releases have been quite solid and predictable as they are released every two years.
Authored by:
jturner on
Wednesday, January 06 2010 @ 09:28 AM MST
It's good news that this is nearly done and indeed a bit unexpected (I wasn't aware the work was ongoing in earnest). We'd be happy to take the beta for a spin when it's ready.
Authored by:
jturner on
Thursday, January 28 2010 @ 12:57 PM MST
Hi Mike,
Just another thought -- will this new release work with gcc 4.x.y compilers? In the past I've had to specify gcc 3 when building external packages etc. (at least on 64-bit machines), but the gcc3 compatibility packages will probably cease to exist in many Linux distros before too long.
Authored by:
jturner on
Tuesday, February 09 2010 @ 01:26 PM MST
I was just wondering what platforms you plan to support for 2.15. Are you likely to continue with Solaris SPARC (etc.)? I think our SPARC users have largely evaporated now and we're considering focusing just on Linux and MacOS.
However, which linux distribution will be your testbed? From history I guess it will be RHEL 5 but it is getting old and RHEL 6 seems to be released next year.
Ubuntu is getting more popular (I maintain a script to install IRAF on it and I get a lot of traffic lately) and LTS releases have been quite solid and predictable as they are released every two years.
Anyway, which distro will be the base? :)