Saturday, December 03, 2022 01:38

Posts Tagged ‘real number precission’

Real types error calculations

Thursday, December 29th, 2016

In calculations with real floating-point data types it is possible to observe strange behavior, because during the representation of a given real number it often happens to lose accuracy. The reason for this is the inability of some real numbers to be represented exactly as a sum of negative powers of the number 2.… Read more


Follow the white rabbit