AngouriMath

Navigation

← Back to list of members

DowncastingTolerance

 Property

Summary

The tolerance within which downcasting rounds a number onto a nearby integer.

Remarks

This is PrecisionErrorZeroRange whenever a caller has set it,
but its default of 1e-16 is far too coarse to be applied blindly: numbers are
evaluated to DecimalPrecisionContext digits, 100 by default, so
1e-16 sits nowhere near the noise floor. Applying it destroyed every legitimate
small value -- 1e-20 parsed to zero and e^(-40) evaluated to zero.
Residuals genuinely left over by exact cancellation, such as the one from
sin(pi), land around 1e-99, so half the working digits separates the two
cases with room to spare. Never looser than the setting's own default, so that
lowering the precision cannot silently widen the tolerance.

























Angouri © 2019-2023 · Project's repo · Site's repo · Octicons · Transparency · 1953 pages online