Discussion:
precision of FP numbers
(too old to reply)
Eliot Frank
2008-06-13 19:37:55 UTC
Permalink
AnsiString as = "12.34";
double a = as.ToDouble();
It appears that there are only zeros in the double beyond the last
significant digit. Is this always the case?
No. There is NO guarantee that any decimal fraction can be represented
exactly in floating point because internal format is a binary fraction.
Your number may be 12.339999999999999 internally but gets rounded back
to 12.34 on output.
Is there a simple way to determine either the precision or the number of
significant digits of an existing double?
No. You could do it algorithmically, using an algorithm similar to that
AnsiString uses to decide to represent a number.

-Eliot
Eliot Frank
2008-06-13 19:44:04 UTC
Permalink
Is there a simple way to determine either the precision or the number of
significant digits of an existing double?
It would be practically impossible to distinguish between 12.34 which
has 4 significant digits and 12.340000 which has 8. You'd have to track
significant digits separately.

-Eliot
Kevin
2008-06-13 21:06:49 UTC
Permalink
It would be practically impossible to distinguish between 12.34 which has
4 significant digits and 12.340000 which has 8. You'd have to track
significant digits separately.
Thanks, and you've indicated a concept I overlooked -- explicitly entered
trailing zeros.

I was looking for a method to determine where non-zero values ended, as an
indication of significant digits. This seemed like a rather difficult task,
considering the internal format of floating point numbers vs. decimal
formats.

Most of our input data is in string form, so we can readily estimate
"intended" precision.

Thanks again

Kevin
Kevin
2008-06-13 20:51:05 UTC
Permalink
Post by Eliot Frank
No. There is NO guarantee that any decimal fraction can be represented
exactly in floating point because internal format is a binary fraction.
Your number may be 12.339999999999999 internally but gets rounded back to
12.34 on output.
Thanks

While it is correct that math functions do not guarantee exact results
(12.34 / 1.0 may result in 12.339999....), I believe that if I define a
double as 12.34, it will be represented as exactly 12.34 internally, with
all trailing zeros. No?

However, it is not clear how AnsiString ToDouble() converts "12.34".

Kevin
Alan Bellingham
2008-06-13 22:02:45 UTC
Permalink
Post by Kevin
While it is correct that math functions do not guarantee exact results
(12.34 / 1.0 may result in 12.339999....), I believe that if I define a
double as 12.34, it will be represented as exactly 12.34 internally, with
all trailing zeros. No?
No.

Computers work in binary, not decimal. You're expecting 12.34 to be
exactly representable in binary, when it isn't. It requires a recurring
binary fraction.

Alan Bellingham
--
Team Browns
<url:http://www.borland.com/newsgroups/> Borland newsgroup descriptions
<url:http://www.borland.com/newsgroups/netiquette.html> netiquette
Kevin Manuele
2008-06-14 06:02:10 UTC
Permalink
Post by Alan Bellingham
No.
Computers work in binary, not decimal. You're expecting 12.34 to be
exactly representable in binary, when it isn't. It requires a recurring
binary fraction.
Of course. It's been a long Friday.

Thanks

Loading...