Welcome to iraf.net Saturday, December 16 2017 @ 01:42 AM GMT


 Forum Index > Help Desk > General IRAF New Topic Post Reply
 Numerical Recipes code
First | Previous | 1 2 | Next | Last
   
olebole
 05/01/2014 03:30PM (Read 3046 times)  
++++-
Regular Member

Status: offline


Registered: 05/01/2014
Posts: 83
Hi,

I just discovered that the IRAF code contains some code from the Numerical Recipes book. As written in http://www.nr.com/licenses/redistribute.html, this code is for personal use only and may not be distributed, even not as GPL. One example is
noao/imred/vtel/mrqmin.x, however there may be more. Comparing the code with the one from the book shows that the code in IRAF really derived from the book's source code.

Could you consider replacing the NR code by something that is free to distribute?

I am asking this since for the inclusion in Debian (and other distributions) it is necessary that the code is really freely distributable.

Best regards

Ole

 
Profile Email Website
 Quote
fitz
 05/01/2014 04:08PM  
AAAAA
Admin

Status: offline


Registered: 09/30/2005
Posts: 3989

We have explicit permission from the authors to use this derived code so long as the following copyright is included in the file:


# Based on Numerical Recipes by Press, Flannery, Teukolsky, and Vetterling.
# Used by permission of the authors.
# Copyright(c) 1986 Numerical Recipes Software.

This appears to be missing from the file in question and will be added for future patches, I'll likewise check the other files I know of (mostly in the XTOOLS lib but also scattered around in task directories) to be sure they have this as well.

Note that similar translations have had to be removed from STSDAS/TABLES source distributions and so it is no longer possible to completely build those packages from source.

 
Profile Email
 Quote
olebole
 05/01/2014 04:14PM  
++++-
Regular Member

Status: offline


Registered: 05/01/2014
Posts: 83
Hi Mike,

Quote by: fitz
We have explicit permission from the authors to use this derived code so long as the following copyright is included in the file


just to make it clear to me: You state that you have the permission to distribute this code under GPL (which would be necessary if IRAF as a whole is distributed as GPL)?
This means that I can also take this code, run it through f2c (or manually convert it to C) and to use it in any other program.

This would be too good to be true... Can you confirm this?

Best

Ole

 
Profile Email Website
 Quote
fitz
 05/01/2014 04:28PM  
AAAAA
Admin

Status: offline


Registered: 09/30/2005
Posts: 3989

We have permission to re-code the same algorithms in SPP for use and distribution in IRAF. This comes from ~1988 before we starting using F2C and likely before GPL itself (and certainly before we started using any other GPL code in the system). I doubt the spirit of granting IRAF permission to use a translation of the algorithms in SPP extends to somebody then recoding it in C to avoid the NR license, and I'm not qualified to say whether this code is now under GPL at all and/or what that might imply.

See the iraf$local/LICENSES directory for licenses which apply to the system. IRAF itself was licensed more openly than GPL but if it is now GPL-tainted and it is something else ...... In any case, as far as we are aware all licensing issues were resolved with v2.16

 
Profile Email
 Quote
olebole
 05/02/2014 08:07AM  
++++-
Regular Member

Status: offline


Registered: 05/01/2014
Posts: 83
Thank you very much for your response. This helps me to provide the copyright information for Debian.

What concerns the GPL, this was my fault; I mixed up the licenses here. I didn't see code yet that would need to put IRAF under GPL.

Best regards

Ole

 
Profile Email Website
 Quote
fitz
 05/02/2014 02:13PM  
AAAAA
Admin

Status: offline


Registered: 09/30/2005
Posts: 3989
The math$slalib and readline code for the ecl/vocl are GPL as well as the cURL code in iraf$vendor/voclient.

 
Profile Email
 Quote
olebole
 05/22/2014 07:33AM  
++++-
Regular Member

Status: offline


Registered: 05/01/2014
Posts: 83
Hi Fitz,

sorry that I bring this up again, but this may still be a problem: Since the version 2.16.1 uses the (GPL) slalib, IRAF as a whole must be offered under the GPL. This at the end includes the Numerical Recipes Code.

Can you confirm that you have the permission to distribute the NR code under an open source license, specifically the GPL?
This is quite serious, since if this is not the case, it is even not allowed to distribute binary packages, since GPL requires to offer the sources as well under their license.

If you don't feel qualified to discuss this: who could I ask?

Best regards

Ole

 
Profile Email Website
 Quote
fitz
 05/22/2014 08:02AM  
AAAAA
Admin

Status: offline


Registered: 09/30/2005
Posts: 3989

We don't distribute NR code, we distribute the SPP implementations of the NR algorithms and we have permission from the authors to do so. I don't believe that puts us in violation of the NR license but, of course, IANAL and you should perhaps consult with the Debian or Gnu folks who oversee such things.

The NR code in STSDAS/TABLES is no longer distributed precisely because they don't have explicit permission to use their SPP implementations, I'm not sure whether any verbatim F77/C code was involved. I don't think these packages are linked against v2.16.1 yet and so the GPL'd slalib wouldn't apply to them either, but you should contact help@stsci.edu for details.

 
Profile Email
 Quote
olebole
 05/22/2014 08:19AM  
++++-
Regular Member

Status: offline


Registered: 05/01/2014
Posts: 83
The GPL allows anyone to take this code (f.e. from noao/imred/vtel/mrqmin.x), change it (f.e. convert it to Fortran, or to C), and to redistribute it under GPL. This means, I can build from the IRAF code a small library that contains all the NR routines that were forbidden to copy up to now. Since GPL is transferrable, I have the same rights on the code as you (and the people who use the result have the same rights as I).

Are you really sure that you can give me these permission? This would be the result of putting IRAF under GPL, which is required since it is linked with GPL code.

The question here is not how to interpret the GPL, but how comprehensive the permissions from NR are. Do they really go so far?
Shall I ask them directly to clarify?

Best regards

Ole

 
Profile Email Website
 Quote
fitz
 05/22/2014 08:24AM  
AAAAA
Admin

Status: offline


Registered: 09/30/2005
Posts: 3989
The only thing I know for sure is that I cannot speak for NR, GPL or NOAO on the matter. However, using the GPL to get around the NR license seems a little skeezy to me personally.

 
Profile Email
 Quote
olebole
 05/22/2014 08:27AM  
++++-
Regular Member

Status: offline


Registered: 05/01/2014
Posts: 83
If IRAF is GPL, then the code in question is this as well. So, I would not "get around" a license but use code with a proper license to replace code with a non-distributable license. This is a normal and legal way to handle these things.

Anyway, I am going to ask NR about the problem. I'll inform you about the answer.

 
Profile Email Website
 Quote
olebole
 05/22/2014 08:29AM  
++++-
Regular Member

Status: offline


Registered: 05/01/2014
Posts: 83
Quote by: fitzThe only thing I know for sure is that I cannot speak for NR, GPL or NOAO on the matter.

Who could I contact at NOAO to clarify this?

Best regards

Ole

 
Profile Email Website
 Quote
fitz
 05/22/2014 08:46AM  
AAAAA
Admin

Status: offline


Registered: 09/30/2005
Posts: 3989
Who could I contact at NOAO to clarify this?


I honestly don't know, the copyright is held by the AURA corporate office. OTOH, this isn't an NOAO problem from what I can tell, and any consequences in a misinterpretation of what you're allowed to do would be yours.

Be aware that whomever you ask at NR probably won't know specifically about our agreement with the authors and you'll likely get a ruling based on the draconian license. You will not be making friends if this settled issue has to be rehashed yet again.

 
Profile Email
 Quote
olebole
 05/22/2014 08:58AM  
++++-
Regular Member

Status: offline


Registered: 05/01/2014
Posts: 83
Who can I ask at AURA then?

Be aware that whomever you ask at NR probably won't know specifically about our agreement with the authors and you'll likely get a ruling based on the draconian license. You will not be making friends if this settled issue has to be rehashed yet again.

Sorry, but this needs to be clarified. I think that the authors are still there, and they should remember that agreement, as you do.

If you confirm that you have the right to distribute this code under GPL, I am fine. I can take the consequences of how I interpret the GPL, but I rely on you that it really is GPL, and I forward any complaints from the NR people back to you.

If you can't do that, I can either contact AURA, or the NR authors.

 
Profile Email Website
 Quote
fitz
 05/22/2014 09:08AM  
AAAAA
Admin

Status: offline


Registered: 09/30/2005
Posts: 3989
Our permission to distribute our NR implementations pre-dates the GPL, until I'm told differently I believe we have permission to distribute that and whatever GPL code is in the system. If your interpretation is that this is then all under the GPL I won't argue, but I'm not qualified to confirm that either way.

 
Profile Email
 Quote
olebole
 05/22/2014 09:10AM  
++++-
Regular Member

Status: offline


Registered: 05/01/2014
Posts: 83
Who at AURA is responsible and qualified to answer these questions?

 
Profile Email Website
 Quote
fitz
 05/22/2014 09:11AM  
AAAAA
Admin

Status: offline


Registered: 09/30/2005
Posts: 3989
I don't know, but I'm sure they'll be thrilled to hear from you.

 
Profile Email
 Quote
olebole
 05/22/2014 09:37AM  
++++-
Regular Member

Status: offline


Registered: 05/01/2014
Posts: 83
I think there is no room for interpretation. The GPL states that

5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions:[...]
c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it.
[...]
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, [...]


Since IRAF is based on SLALIB (in the sense how the GPL defines it), you must license the whole work under GPL. And therefore, I am asking you to get the whole IRAF code under GPL.

 
Profile Email Website
 Quote
olebole
 05/22/2014 10:56AM  
++++-
Regular Member

Status: offline


Registered: 05/01/2014
Posts: 83
Hi Fitz again,

I don't know why you ract so angry here.
This licensing stuff is a time bomb: The code is openly available, and it just a matter of time until someone else finds out that there is NR code and that the license should be GPL.
You will have to face this problem, sooner or later. Just keeping the head in the sand is the worst solution that can be done.

 
Profile Email Website
 Quote
fitz
 05/22/2014 01:02PM  
AAAAA
Admin

Status: offline


Registered: 09/30/2005
Posts: 3989

Fine, so tell me where to get my GPL handstamp and then please followup with all the packages listed at http://iraf.net/article.php?story=20051211115253923 to be sure they are also in compliance with whatever it is you're wanting me to do here. Be sure to also check on those packages which don't distribute source code or distributions built on earlier versions when the system was considered non-free and alert them as well (I'd hate to think you're just finding extra things for only me to do). Seriously, what is it you want done here?

If you know of a specific problem that would prevent us from distributing IRAF legally please let me know and I will address it ASAP. If you're just not happy with the terms of the current scheme, then I eagerly await the community patch that will replace or remove the NR or slalib code to get the configuration of licenses into whichever arrangement you like.

 
Profile Email
 Quote
olebole
 05/22/2014 02:08PM  
++++-
Regular Member

Status: offline


Registered: 05/01/2014
Posts: 83
Hey, it is you who distributes the code (or AURA, or NOAO, if you like). You should know which licenses apply and how to put the bits and pieces together to follow all the rules.

And the rules in question are quite simple here:

1. If you use GPL code, specifically if you link to it, your code must be GPL as well. If you don't like that, or if you cannotfullfill this, then you cannot use GPL code. You cannot use SLALIB, you cannot use Readline in this case.

I would anyway recommend to use ERFA instead of slalib since it provides the same functionality but is BSD-licensed and is actually maintained (leap seconds and so on).

For Readline, you probably have no choice (except writing your own command line editing library). Even if you link to the one provided by the system, the resulting software has to be licensed under GPL (there is a lengthy discussion about that, and be prepared to fight with the GNU foundation if this is ignored).

This applies to the source code as well as to the binaries -- if you offer binaries that use GPL code, you are required to offer the full source code as well (under GPL).

2. If you distribute some code, you should be allowed to do so, specifically under the license you choose resp. that you are required to use. So, if you distribute NR code (and you must do this under GPL as just stated), then you should be allowed to do so. It is your responsibility to check this.

Ofcourse, this all also applies to the parts of the code where you don't provide binaries: class and jar files for the VO stuff. I didn't check that yet, you however must make sure that it is distributable under GPL. And this all applies to all the dependent packages as well: if you put IRAF under GPL, and they are built on top of it (link to it or so), then they must be GPL as well, and ofcourse provide all source code.

I would also question that you are allowed to distribute NR (or derived) source code as part of the sources of IRAF. The NR QA states clearly:
  • [...] Can you include Numerical Recipes routines as part of that source code, including a notice that they are only allowed to be used with your application?
    • Sorry, no. We never give permission for Numerical Recipes source code to be posted on any public server, or distributed with any freeware or shareware package [...]

  • You want to translate some (or all) the Numerical Recipes routines to a different computer language, and then redistribute the resulting translations.
    • [...] However, you can't redistribute the translations in any manner, since they are "derivative works" and still subject to our copyright.

This sounds so clear to me that you should really explicitely state that putting the code into the public IRAF distribution is legal and still covered by the permission you got 25 years ago.

To summarize (and to answer your question): Yes, I think that it is not legal for you to distribute sources or binaries of IRAF in the current form, since you seem to be not able to provide the full sources under GPL, which you would be required to.

If you ask what I want: I want to get IRAF into Debian, and for this I am going through the code and also through the legal issues. I am bound here to the Debian Free Software Guidelines. This is not specific to IRAF, we discussed similar things f.e. for Starlink packages (AST) which at the end converted from SLALIB to erfa.

However, specifically the issue with the Numerical Recipes is quite common among astronomical software packages. Since the problem is so common, it may be worth to start some common effort, either to create a clean-room implementation of them, or to use existing libraries (like the GNU Scientific library) instead. You should just get in contact with the other major software players in our area Smile

 
Profile Email Website
 Quote
fitz
 05/22/2014 02:26PM  
AAAAA
Admin

Status: offline


Registered: 09/30/2005
Posts: 3989
Other than the question about NR code (which I claim we have permission to distribute, perhaps not explicitly stated in the right place, but permission as we understand it), what code are we not distributing that invalidates the GPLness of things? If we don't explicitly compile readline in the source tree but rely on it being in a system shared lib, are we still GPLd?

 
Profile Email
 Quote
olebole
 05/22/2014 02:54PM  
++++-
Regular Member

Status: offline


Registered: 05/01/2014
Posts: 83
what code are we not distributing that invalidates the GPLness of things

You distribute a lot of Java classes and jar files without the sources in vo/java, as well as in vendor/voclient/voclient/classes and other places. There may be source code available; however I am not sure if this is true for all of the packages and classes there.

If we don't explicitly compile readline in the source tree but rely on it being in a system shared lib, are we still GPLd?

Yes. See, for example, the the GNU license FAQ.

This issue lead to the creation of the LGPL; however GNU decided that readline remains under GPL.

 
Profile Email Website
 Quote
fitz
 05/22/2014 03:06PM  
AAAAA
Admin

Status: offline


Registered: 09/30/2005
Posts: 3989

The next release will use a new release of the VOClient code that is all native C, so the jar problem will go away.

 
Profile Email
 Quote
olebole
 04/25/2017 03:59PM  
++++-
Regular Member

Status: offline


Registered: 05/01/2014
Posts: 83
I investigated this a bit further. There are the following files that contain a copyright statement from NUmerical Recipes:
  • pkg/xtools/numrecipes.x
  • noao/onedspec/splot/spdeblend.x
  • noao/rv/numrep.x
  • noao/artdata/numrecipes.x

However, there are some other places that also have NR code without the required copyright (which is against the agreement that fitz told about); I will investigate this further.
The following common procedures from NR are used here, that can probably easily replaced from other sources:
  • gammln(xx): natural log of gamma function.
  • gasdev(seed): normally distributed deviate of zero mean and unit var
  • poidev(xm, seed): Poisson deviates for a given mean.

The following routines are used only in the noao/rv package:
  • four1(data, nn, isign): Replaces DATA by it's discrete transform
  • realft(data, N, isign): Fourier Transform of a set of 2N data points.
  • twofft(data1, data2, fft1, fft2, N): calls four1()
The only non-trivial one here is four(); the two others just call four().

Then we have LU decomposition functions, used in pkg/utilities/stxtools/xtwcs.x:
  • ludcmp(a, n, np, indx, d)
  • lubksb(a, n, np, indx, b)
  • luminv(a, n, np)

And Levenberg-Marquardt nonlinear chi square minimization, used quite commonly
  • mr_eval(x, y, npts, params, flags, np, a, delta, nfit, chisq)
  • mr_invert(a, b, n)
  • mr_solve(x, y, npts, params, flags, np, nfit, mr, chisq)

Maybe this could be replaced by cminpack?

Funnily, several of these functions are defined several times over the code... However it seems that much NR code could be replaced by free code without too much effort, or removed without losing too much functionality. Again, it would help here if there was a central github repository, to provide pull requests and such.

On a long term it is IMO a horrible solution that IRAF has code that is accompanied by the statement
Be aware that whomever you ask at NR probably won't know specifically about our agreement with the authors and you'll likely get a ruling based on the draconian license. You will not be making friends if this settled issue has to be rehashed yet again.


Sorry, but IMO it is not really an acceptable statement.

 
Profile Email Website
 Quote
fitz
 04/25/2017 04:20PM  
AAAAA
Admin

Status: offline


Registered: 09/30/2005
Posts: 3989
On a long term it is IMO a horrible solution that IRAF has code that is accompanied by the statement


There is no such text that "accompanies" IRAF code, it was a personal observation I made on this (a private) site that cannot be construed as official documentation. If there is ever any more than talk about how easy it is to replace the NR functions I'm sure they can be part of a future release once they are shown to work.

 
Profile Email
 Quote
olebole
 04/25/2017 07:09PM  
++++-
Regular Member

Status: offline


Registered: 05/01/2014
Posts: 83
If there is ever any more than talk about how easy it is to replace the NR functions I'm sure they can be part of a future release once they are shown to work.

Again: please create a central github repository for IRAF. This and much more can be discussed and applied easily with pull requests.

In the past, I created some patches for other issues (they are "somewhere" here in this forum), they are going to be forgotten. At least this is how I understand your statement that any repository would not have more than the official release. I have no motivation to put any effort into this unless there is a transparent, well-established development process, like the one implemented by github. There are people which are motivated to help you, but you are just ignoring them for years.

My motivation to look into this is that the NR code prevents IRAF to go into Debian, Ubuntu and other Linux distributions, since despite your better-not-ask-the-other-side statement about that you are allowed to use the code it is not open source software. And I will look into this if there is a chance that we get a solution implemented. There are other problems, like a largely undocumented build process, intransparent major changes between the versions, missing portability etc., where luckily other people would have a look. So, if you would setup a github repository for IRAF (and moderate/apply the pull requests etc.), I would see a good chance for a collaborative improvement. But since you don't, all these efforts are somehow lost in private places. This was asked already three years ago, nothing happened, people got lost and code gets lost.

Back to NR: is there more code from NR than the one I identified above? You stated that according to your agreement the NR code must have the correct copyright, but since there are places with NR code and without NR copyright (pkg/utilities/nttools/stxtools/ludcmp.x as an example), I am afraid that more is hiding somewhere. Could you provide a complete list?

BTW, I know that the statement "You will not be making friends if this settled issue has to be rehashed yet again." is your private statement. But sorry, a "better do not ask them" recommendation is a horrible one. Question is if you are allowed to distribute the source code of NR in the current form, and question is if anyone else is allowed to. We can't just solve this by ignoring the question and keeping silent.

Best regards

Ole


 
Profile Email Website
 Quote
fitz
 04/26/2017 06:04AM  
AAAAA
Admin

Status: offline


Registered: 09/30/2005
Posts: 3989
Again: please create a central github repository for IRAF. This and much more can be discussed and applied easily with pull requests.


Have at it:

https://github.com/iraf

This is the latest version of all code from our dev machine which differs only slightly from the v2.16.1 release because of a general lack of activity.

Note there is absolutely NO implied or express promise that issues and PRs will be dealt with in anything resembling a "timely manner". Major code changes accompanied by test code/results demonstrating they work will receive more attention than ones I need to validate myself. Issues that offer no contribution other than to add to the list of things I have to do will be ignored.

I may still be the person responsible for this project, but that doesn't mean I have a great deal of time for it (especially the remainder of this year). If your schedule can't wait for mine, create a fork.

 
Profile Email
 Quote
olebole
 04/26/2017 06:26AM  
++++-
Regular Member

Status: offline


Registered: 05/01/2014
Posts: 83
Have at it:

https://github.com/iraf


WOW! Great! Thank you very much!
One question/remark: Is there a reason why you encoded the version number in the repository name? This will not match anymore once there is a new release out.

Best regards

Ole


 
Profile Email Website
 Quote
fitz
 04/26/2017 06:30AM  
AAAAA
Admin

Status: offline


Registered: 09/30/2005
Posts: 3989
One question/remark: Is there a reason why you encoded the version number in the repository name? This will not match anymore once there is a new release out.


Seriously? The original plan was to essentially migrate the FTP archive to repos, preserving the history of platforms and branches for different versions, but that takes time. At some later time, or not, that level of organization can be applied, but for now just shut up and enjoy your "official" repo.

 
Profile Email
 Quote
   
First | Previous | 1 2 | Next | Last
Content generated in: 0.77 seconds
New Topic Post Reply

Normal Topic Normal Topic
Sticky Topic Sticky Topic
Locked Topic Locked Topic
New Post New Post
Sticky Topic W/ New Post Sticky Topic W/ New Post
Locked Topic W/ New Post Locked Topic W/ New Post
View Anonymous Posts 
Anonymous users can post 
Filtered HTML Allowed 
Censored Content 
dog allergies remedies cialis 20 mg chilblain remedies


Privacy Policy
Terms of Use

User Functions

Login