(numerical-accuracy)=
# Numerical accuracy

<!--* freshness: { reviewed: '2026-09-18' } *-->

This document summarizes the accuracy of JAX mathematical functions.

## Methodology and terminology

- **Units in the Last Place (ULPs)**: Error is measured in ULPs relative to a
  higher-precision reference: for sub-64-bit types (`bfloat16`, `float16`, and
  `float32`), reference values are evaluated in double precision (`float64`); for
  `float64`, reference values are evaluated using {mod}`mpmath` with 100-bit
  precision. For a target format with $p$ significand bits and normal exponent
  range $[e_{\min}, e_{\max}]$, $\operatorname{ulp}(r) = 2^{\min(\max(\lfloor \log_2 |r| \rfloor, e_{\min}), e_{\max}) - (p - 1)}$.
  A bound of $0.5$ ULP is a correctly rounded result.
  - **Overflow and Infinity**: Reference values at or beyond the
    round-to-nearest-even overflow threshold $2^{e_{\max} + 1} - \frac{1}{2}\operatorname{ulp}(\text{max\_float})$
    are treated as overflowing to $\pm\infty$, so a computed $\pm\infty$ has
    $0$ ULP error. When the reference value is finite in or above the top normal
    binade ($|r| \ge 2^{e_{\max}}$) and the computed result has the same sign,
    rounding across the overflow threshold (either a computed $\pm\infty$ just
    below the threshold, or a finite result such as $\pm\text{max\_float}$ at or
    above the threshold) is measured continuously by placing $\pm\infty$ at
    $\pm 2^{e_{\max} + 1} = \pm(\text{max\_float} + 1\text{ ULP})$.
- **Exhaustive vs. Sampled Testing**:
  - For 16-bit types (`bfloat16`, `float16`) and unary operations on 32-bit
    types (`float32`), bounds are verified by exhaustive testing across all
    bit patterns (up to $2^{32}$ inputs).
  - For binary operations on `float32` and all operations on `float64`, bounds
    are estimated by testing a sample of inputs: values at and near special
    points (such as 0, $\pm 1$, $\pm\infty$, subnormal/normal boundaries, poles,
    and approximation thresholds) along with random floating-point values drawn
    across all exponents. Because sampling cannot guarantee hitting the
    worst-case input, these values are empirical bounds on the maximum error.
- **Flush-To-Zero (FTZ) and Subnormals**: Subnormal floating-point inputs and
  outputs ($|v| < 2^{e_{\min}}$) are typically flushed to zero by default,
  though some operations on types smaller than `float32` (and `float64` on
  GPU) preserve gradual underflow.
  - When gradual underflow is preserved (`ftz=False`), subnormal values are
    measured on the standard number line against the subnormal quantum
    $\operatorname{ulp}_{\text{tiny}} = 2^{e_{\min} - (p - 1)}$.
  - When outputs flush to zero (`ftz=True`), any pair of values with both
    $|c| < 2^{e_{\min}}$ and $|r| < 2^{e_{\min}}$ has $0$ ULP error, and
    distances that cross the subnormal region contract the unrepresentable
    interval $[-(2^{e_{\min}} - \operatorname{ulp}_{\text{tiny}}), +(2^{e_{\min}} - \operatorname{ulp}_{\text{tiny}})]$
    to $0$ so that the adjacent FTZ-representable values $(-2^{e_{\min}}, 0.0, +2^{e_{\min}})$
    are spaced $1$ ULP apart rather than incurring an artificial $(2^{p-1} - 1)$-ULP gap.
- **Hardware Platforms**:
  - **CPU**: x86_64.
  - **NVIDIA GPU**: H100 and B200.
  - **TPU**: TPU v2–v5e, TPU v5p, TPU v6e, and TPU 7x.

---

## `bfloat16` accuracy

Maximum error in units in the last place (ULPs) for `bfloat16`:

| Function | CPU | NVIDIA GPU | TPU v2–v5e | TPU v5p | TPU v6e | TPU 7x |
|:---|:---:|:---:|:---:|:---:|:---:|:---:|
| {func}`~jax.lax.acos` | 1.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.acosh` | 2.0 | 0.5 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.asin` | 1.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.asinh` | 1.5 | 0.5 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.atan` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.atan2` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.atanh` | 1.5 | 0.5 | 1.0 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.bessel_i0e` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.bessel_i1e` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.cbrt` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.cos` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.cosh` | 0.5 | 0.5 | 0.5 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.erf` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.erf_inv` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.erfc` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.exp` | 0.5 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.exp` (`highest`) | 0.5 | 1.0 | 1.0 | 1.0 | 0.5 | 0.5 |
| {func}`~jax.lax.exp2` | 101.0 | 101.0 | 44.0 | 44.0 | 44.0 | 75.0 |
| {func}`~jax.lax.expm1` | 0.5 | 0.5 | 1.0 | 1.0 | 0.5 | 0.5 |
| {func}`~jax.lax.expm1` (`highest`) | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.lgamma` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.log` | 0.5 | 0.5 | 1.0 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.log1p` | 0.5 | 0.5 | 1.0 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.log2` | 2.0 | 2.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.logistic` | 2.5 | 2.5 | 1.0 | 1.0 | 0.5 | 63.0 |
| {func}`~jax.lax.reciprocal` | 0.5 | 0.5 | 1.0 | 1.0 | 0.5 | 0.5 |
| {func}`~jax.lax.rsqrt` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.sin` | 0.5 | 0.5 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.sinh` | 0.5 | 0.5 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.sqrt` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.square` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.tan` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.tanh` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.numpy.deg2rad` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.numpy.hypot` | 1.5 | 1.5 | 1.0 | 1.0 | 0.5 | 1.0 |
| {func}`~jax.numpy.log10` | 2.0 | 2.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.numpy.rad2deg` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.numpy.sinc` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.scipy.special.erfcinv` | 1.0 | 1.0 | 1.0 | 0.5 | 1.0 | 0.5 |
| {func}`~jax.scipy.special.erfcx` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |

---

## `float16` accuracy

Maximum error in units in the last place (ULPs) for `float16`:

| Function | CPU | NVIDIA GPU | TPU v2–v5e | TPU v5p | TPU v6e | TPU 7x |
|:---|:---:|:---:|:---:|:---:|:---:|:---:|
| {func}`~jax.lax.acos` | 1.5 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.acosh` | 2.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.asin` | 1.5 | 1.0 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.asinh` | 1.5 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.atan` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.atan2` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.atanh` | 1.5 | 0.5 | 1.0 | 1.0 | 0.5 | 0.5 |
| {func}`~jax.lax.bessel_i0e` | 1.0 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.bessel_i1e` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.cbrt` | 1.0 | 1.0 | 1.0 | 1.0 | 0.5 | 0.5 |
| {func}`~jax.lax.cos` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.cosh` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.erf` | 1.0 | 1.0 | 0.5 | 0.5 | 1.0 | 1.0 |
| {func}`~jax.lax.erf_inv` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.erfc` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.exp` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.exp` (`highest`) | 1.0 | 1.0 | 1.0 | 1.0 | 0.5 | 0.5 |
| {func}`~jax.lax.exp2` | 14.0 | 14.0 | 7.5 | 7.5 | 7.5 | 7.5 |
| {func}`~jax.lax.expm1` | 1.0 | 1.0 | 1.0 | 1.0 | 0.5 | 0.5 |
| {func}`~jax.lax.expm1` (`highest`) | 1.0 | 1.0 | 1.0 | 1.0 | 0.5 | 0.5 |
| {func}`~jax.lax.lgamma` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.log` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.log1p` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.log2` | 2.0 | 2.0 | 1.5 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.logistic` | 2.0 | 2.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.reciprocal` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.rsqrt` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.sin` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.sinh` | 1.0 | 0.5 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.sqrt` | 0.5 | 0.5 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.square` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.lax.tan` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.lax.tanh` | 0.5 | 0.5 | 1.0 | 1.0 | 0.5 | 0.5 |
| {func}`~jax.numpy.deg2rad` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.numpy.hypot` | 1.5 | 1.5 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.numpy.log10` | 1.5 | 1.5 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.numpy.rad2deg` | 1.5 | 1.5 | 1.5 | 1.5 | 1.5 | 1.5 |
| {func}`~jax.numpy.sinc` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 |
| {func}`~jax.scipy.special.erfcinv` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| {func}`~jax.scipy.special.erfcx` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |

---

## `float32` accuracy

Maximum error in units in the last place (ULPs) for `float32`:

| Function | CPU | NVIDIA GPU | TPU v2–v5e | TPU v5p | TPU v6e | TPU 7x | Notes |
|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---|
| {func}`~jax.lax.acos` | 1.5 | 1.5 | 5.0 | 5.0 | 4.0 | 5.0 | |
| {func}`~jax.lax.acosh` | 4.5 | 2.5 | 4031.0 | 61.0 | 4.5 | 5.0 | |
| {func}`~jax.lax.asin` | 2.0 | 1.5 | 5.0 | 5.0 | 4.5 | 4.5 | |
| {func}`~jax.lax.asinh` | 3.5 | 2.0 | 4034.5 | 62.5 | 3.5 | 3.5 | |
| {func}`~jax.lax.atan` | 4.0–5.5 | 1.5 | 2.5 | 2.5 | 2.5 | 2.5 | CPU bound depends on AMD vs Intel |
| {func}`~jax.lax.atan2` | 0.5 | 3.0 | 3.5 | 3.5 | 3.5 | 3.5 | |
| {func}`~jax.lax.atanh` | 3.0 | 3.5 | 2183.5 | 43.0 | 3.5 | 3.5 | |
| {func}`~jax.lax.bessel_i0e` | 7.0 | 8.0 | 8.0 | 8.0 | 8.0 | 8.0 | |
| {func}`~jax.lax.bessel_i1e` | 11.0 | 15.5 | 15.5 | 15.5 | 15.5 | 15.5 | |
| {func}`~jax.lax.cbrt` | 0.5 | 1.5 | 4.5 | 4.5 | 1.5 | 1.5 | |
| {func}`~jax.lax.cos` | 0.5 | 2.0 | 3.5 | 3.5 | 3.5 | 3.0 | |
| {func}`~jax.lax.cosh` | 25.0 | 2.5 | 93.5 | 99.0 | 59.0 | 59.5 | |
| {func}`~jax.lax.erf` | 7.0 | 6.5 | 7.5 | 8.5 | 1.5 | 1.5 | |
| {func}`~jax.lax.erf_inv` | 65.0 | 65.0 | 427.0 | 65.5 | 65.0 | 65.5 | |
| {func}`~jax.lax.erfc` | 66.0 | 66.5 | 145.0 | 157.0 | 124.5 | 125.0 | |
| {func}`~jax.lax.erfc` (`highest`) | 4.0 | 5.0 | 5.5 | 5.0 | 5.0 | 5.0 | `accuracy=lax.AccuracyMode.HIGHEST` |
| {func}`~jax.lax.exp` | 1.5 | 2.0 | 116.0 | 109.5 | 64.5 | 65.0 | |
| {func}`~jax.lax.exp` (`highest`) | 1.5 | 2.0 | 1.5 | 1.5 | 1.5 | 1.5 | `accuracy=lax.AccuracyMode.HIGHEST` |
| {func}`~jax.lax.exp2` | 68.5 | 69.0 | 141.5 | 133.0 | 90.0 | 90.0 | |
| {func}`~jax.lax.expm1` | 1.0 | 1.5 | 1772.0 | 1357.5 | 3.5 | 3.5 | |
| {func}`~jax.lax.expm1` (`highest`) | 1.0 | 1.5 | 1.0 | 1.0 | 1.0 | 1.0 | `accuracy=lax.AccuracyMode.HIGHEST` |
| {func}`~jax.lax.lgamma` | 4.0 | 4.0 | 4.0 | 4.0 | 4.0 | 4.0 | |
| {func}`~jax.lax.log` | 1.5 | 1.0 | 4030.5 | 62.0 | 2.5 | 2.5 | |
| {func}`~jax.lax.log1p` | 3.0 | 1.0 | 4034.0 | 63.0 | 3.0 | 3.0 | |
| {func}`~jax.lax.log2` | 2.5 | 2.0 | 5159.0 | 57.5 | 2.5 | 2.5 | |
| {func}`~jax.lax.logistic` | 2.5 | 4.0 | 243.0 | 124.0 | 65.5 | 64.0 | |
| {func}`~jax.lax.reciprocal` | 0.5 | 1.0 | 198.0 | 40.0 | 1.5 | 1.5 | |
| {func}`~jax.lax.rsqrt` | 1.0–2.0 | 2.0 | 2.5 | 2.5 | 1.5 | 1.0 | CPU bound depends on AMD vs Intel |
| {func}`~jax.lax.sin` | 0.5 | 1.5 | 3.5 | 3.5 | 3.5 | 3.5 | |
| {func}`~jax.lax.sinh` | 25.0 | 3.0 | 1794.0 | 1332.5 | 59.0 | 59.5 | |
| {func}`~jax.lax.sqrt` | 0.5 | 1.0 | 3.0 | 3.0 | 2.0 | 2.0 | |
| {func}`~jax.lax.square` | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | |
| {func}`~jax.lax.tan` | 0.5 | 3.5 | 6.5 | 7.0 | 5.5 | 5.5 | |
| {func}`~jax.lax.tanh` | 5.0 | 5.5 | 1365.5 | 92.0 | 1.5 | 1.5 | |
| {func}`~jax.numpy.deg2rad` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | |
| {func}`~jax.numpy.hypot` | 1.5 | 2.0 | 3.5 | 3.5 | 2.5 | 2.5 | |
| {func}`~jax.numpy.log10` | 3.0 | 2.5 | 6213.0 | 57.0 | 3.0 | 3.0 | |
| {func}`~jax.numpy.rad2deg` | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | |
| {func}`~jax.numpy.sinc` | 2.5 | 3.5 | 4.0 | 4.0 | 4.0 | 3.5 | |
| {func}`~jax.scipy.special.erfcinv` | 3.0 | 3.5 | 427.0 | 6.0 | 4.0 | 4.0 | |
| {func}`~jax.scipy.special.erfcx` | 3.5 | 4.0 | 4.0 | 4.0 | 4.5 | 4.5 | |
| {func}`~jax.scipy.special.ndtri` | 2.5 | 3.0 | 302.0 | 5.0 | 3.5 | 3.5 | |

---

## `float64` accuracy

Maximum error in units in the last place (ULPs) for `float64`.

```{note}
TPUs have no true float64 hardware. While bounds for 16-bit types and unary
float32 operations are exhaustive, for binary float32 operations and float64
these are empirical bounds.
```

| Function | CPU | NVIDIA GPU |
|:---|:---:|:---:|
| {func}`~jax.lax.acos` | 1.5 | 1.5 |
| {func}`~jax.lax.acosh` | 3.5 | 2.5 |
| {func}`~jax.lax.asin` | 1.5 | 2.5 |
| {func}`~jax.lax.asinh` | 2.5 | 2.5 |
| {func}`~jax.lax.atan` | 3.5 | 2.5 |
| {func}`~jax.lax.atan2` | 0.5 | 1.5 |
| {func}`~jax.lax.atanh` | 2.5 | 3.5 |
| {func}`~jax.lax.bessel_i0e` | 7.5 | 7.5 |
| {func}`~jax.lax.bessel_i1e` | 10.5 | 7.5 |
| {func}`~jax.lax.cbrt` | 0.5 | 1.5 |
| {func}`~jax.lax.cos` | 0.5 | 1.5 |
| {func}`~jax.lax.cosh` | 496.0 | 2.5 |
| {func}`~jax.lax.erf` | 2.5 | 2.5 |
| {func}`~jax.lax.erf_inv` | 500000.0 | 500000.0 |
| {func}`~jax.lax.erfc` | 350.0 | 350.0 |
| {func}`~jax.lax.erfc` (`highest`) | 2.5 | 2.5 |
| {func}`~jax.lax.exp` | 2.0 | 1.5 |
| {func}`~jax.lax.exp2` | 719.0 | 719.0 |
| {func}`~jax.lax.expm1` | 1.0 | 1.5 |
| {func}`~jax.lax.log` | 0.5 | 1.5 |
| {func}`~jax.lax.log1p` | 2.0 | 1.5 |
| {func}`~jax.lax.log2` | 1.5 | 1.5 |
| {func}`~jax.lax.logistic` | 3.5 | 4.5 |
| {func}`~jax.lax.reciprocal` | 0.5 | 0.5 |
| {func}`~jax.lax.rsqrt` | 1.5 | 1.5 |
| {func}`~jax.lax.sin` | 0.5 | 2.5 |
| {func}`~jax.lax.sinh` | 496.0 | 2.5 |
| {func}`~jax.lax.sqrt` | 0.5 | 0.5 |
| {func}`~jax.lax.square` | 0.5 | 0.5 |
| {func}`~jax.lax.tan` | 0.5 | 2.5 |
| {func}`~jax.lax.tanh` | 7.0 | 3.5 |
| {func}`~jax.numpy.deg2rad` | 1.0 | 1.0 |
| {func}`~jax.numpy.hypot` | 1.5 | 1.5 |
| {func}`~jax.numpy.log10` | 2.0 | 2.5 |
| {func}`~jax.numpy.rad2deg` | 1.0 | 1.0 |
| {func}`~jax.numpy.sinc` | 2.0 | 2.0 |
| {func}`~jax.scipy.special.erfcinv` | 3.0 | 3.0 |
| {func}`~jax.scipy.special.erfcx` | 3.0 | 3.0 |
| {func}`~jax.scipy.special.ndtri` | 2.0 | 2.0 |