AngouriMath
DowncastingToleranceAsDouble
Property
Summary
The tolerance within which downcasting rounds a number onto a nearby integer.
Remarks
This is PrecisionErrorZeroRange whenever a caller has set it,
but its default of 1e-16 is far too coarse to be applied blindly: numbers are
evaluated to DecimalPrecisionContext digits, 100 by default, so
1e-16 sits nowhere near the noise floor. Applying it destroyed every legitimate
small value --1e-20 parsed to zero and e^(-40) evaluated to zero.
Residuals genuinely left over by exact cancellation, such as the one from
sin(pi) , land around 1e-99, so half the working digits separates the two
cases with room to spare. Never looser than the setting's own default, so that
lowering the precision cannot silently widen the tolerance.
but its default of 1e-16 is far too coarse to be applied blindly: numbers are
evaluated to DecimalPrecisionContext digits, 100 by default, so
1e-16 sits nowhere near the noise floor. Applying it destroyed every legitimate
small value --
Residuals genuinely left over by exact cancellation, such as the one from
cases with room to spare. Never looser than the setting's own default, so that
lowering the precision cannot silently widen the tolerance.
Summary
DowncastingTolerance as a double, for the cheap check in front of
the exact one: the conversion of the decimal costs more than the check.
the exact one: the conversion of the decimal costs more than the check.
Angouri © 2019-2023 · Project's repo · Site's repo · Octicons · Transparency · 4378 pages online