divide operation precision is changing

Thread View: alt.pl.comp.lang.csharp

2 messages

[T] Back to Threads [V] Tree View [G] Group Articles

2 total messages Started by Maor Mishkin Tue, 20 Jan 2004 14:58

#431

Author: Maor Mishkin
Date: Tue, 20 Jan 2004 14:58

12 lines
1359 bytes

I have a problem with divide operation,
the example is:
double a =10.0;
double b =220.0;
double divAns = a/b;
the calculation true result is 0.04545454545454545
but for some resone the calculation is changing to get the result
0.0454545468091
if you know how can i get the system to calc the first calculation or if you
know of an object like BigNumber in java that can make
calculation with a specipic precision it could realy help me alot.
Maor

Re: divide operation precision is changing

#432

Author: Marek Janaszewski USUNTO. z adre...
Date: Tue, 20 Jan 2004 23:20

39 lines
2185 bytes

W artykule news:400d4213$1...@news.012.net.il,
niejaki(a): Maor Mishkin z adresu <ju...@omnivee.com> napisał(a):

> I have a problem with divide operation,
> the example is:
> double a =10.0;
> double b =220.0;
> double divAns = a/b;
> the calculation true result is 0.04545454545454545
> but for some resone the calculation is changing to get the result
> 0.0454545468091
> if you know how can i get the system to calc the first calculation or
> if you know of an object like BigNumber in java that can make
> calculation with a specipic precision it could realy help me alot.
> Maor


Hi,


I think it problem with floating point numbers. Floating-point numbers
stores numbers approximately. For example, floating-point types don't
precisely represent 0.1.

Try to use "decimal" type. Decimal type should by used in financial
calculations. You can find more information about floating-point number at:

http://www.research.microsoft.com/~hollasch/cgindex/coding/ieeefloat.html

http://www.mathworks.com/company/newsletter/pdf/Fall96Cleve.pdf

http://developer.intel.com/design/intarch/techinfo/pentium/fpu.htm

PS Sorry for my English.

-- 
Regards,
Marek Janaszewski
[ j_m...@gazeta.pl ]

Thread Navigation

This is a paginated view of messages in the thread with full content displayed inline.

Messages are displayed in chronological order, with the original post highlighted in green.

Use pagination controls to navigate through all messages in large threads.

Back to All Threads