Walter Neumann on Fri, 13 May 2005 16:00:20 +0200 |
[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]
Re: erfc() behavior change |
? 0.e-10 %1 = 0.E-38 ? precision(%) %2 = 48 ? %1*10^40 %3 = 0.E1 ? precision(%) %4 = 0 I would expect the answers to be 0.E-48, 48, 0.E-10, 10 respectively. --walter On Thu, 12 May 2005, Karim Belabas wrote:
* Walter Neumann [2005-05-12 17:15]:There does seem to be a related incosistency however: ? erfc(-2^11) %1 = 2.000000000000000000000000000 ? 2-%1 %2 = 0.E-1821573 ? precision(%2) %3 = 1821573 ... ? 0.0 %10 = 0.E-28 ? precision(%) %11 = 28 ? 0.0e-90 %12 = 0.E-91 ? precision(%) %13 = 96 ? 1.e-90 %14 = 1.000000000000000000000000000 E-90 ? precision(%) %15 = 28 Shouldn't precision in %3 and %13 be something like 28?'precision' is not really defined in the documentation (should be...). * For a basic type not involving floating point components, precision(x) is +oo [ well, an absurdly large number, strictly larger than the precision of any inexact object. ] * For a _non_zero_ t_REAL x, precision(x) is the number of significant digits for _non_zero_ x. This is only approximately true: various radix conversions and assorted roundings are involved. * For a _zero_ t_REAL x, precision(x) is max(- expo_10(x), 0). There are no significant digits in this case. * For a t_COMPLEX, precision(x + I*y) is (roughly) precision(|x| + |y|). * For another recursive object, precision(x) is the min of all involved precisions. The precision of X*(1. to 100 significant digits) + 0.e-1000 is 100. X*(1. to 100 significant digits) + 0.e+1000 is 0. Hope this helps, Karim.