Product Site

10.1. Precision

Precision is the maximum number of significant digits that can result from an operation. This is controlled by the instruction:

The expression is evaluated and must result in a positive whole number. This defines the precision (number of significant digits) of a calculation. Results are rounded to that precision, if necessary.

If you do not specify expression in this instruction, or if no NUMERIC DIGITS instruction has been processed since the start of a program, the default precision is used. The Rexx standard for the default precision is 9. The default may be overridden on a source-file basis using the ::OPTIONS directive.

NUMERIC DIGITS can set values smaller than nine. However, use small values with care because the loss of precision and rounding affects all Rexx computations, including, for example, the computation of new values for the control variable in DO loops.