(I found no category for “LLVM IR semantics”.)
In Rust, we are having a lengthy discussion about what the precise semantics for floating-point operations should be when NaNs are involved. The main problem is that the exact bits in a NaN can differ depending on optimizations and target architectures, which somehow needs to be captured by the language semantics.
During this discussion it was pointed out that WebAssembly basically already solved this problem, in the sense that they precisely defined a reasonable semantics. WebAssembly posits that NaN bits are stable when floating-point values are copied around in memory or between memory and local variables, but that most floating-point operations non-deterministically pick a new NaN payload if the output is a NaN.
Now my question is, what is the behavior of NaNs in LLVM IR? My hope is that it is basically like wasm, in the sense that floating-point values are regular fixed bit patterns (no “magic” weirdness going on when they are just copied around), and that NaN changes may only happen on
fmul et al. This would mean that running
fmul twice on the same input can produce different results, which is something that optimizations need to consider – it would be illegal to turn (using C syntax here but I really mean the underlying LLVM IR program)
float a = ...; float b = ...; float x = a*b; // non-deterministically picks a NaN payload float y = x; // will definitely have the same NaN payload as x
float a = ...; float b = ...; float x = a*b; // non-deterministically picks a NaN payload float y = a*b; // non-deterministically picks a potentially different NaN payload
because in the second program, if
a*b results in
y could have different payload, while in the first program they always have the same payload.
Is LLVM taking this into account for its optimizations? And if not, then what exactly is the semantics of an LLVM IR program where
fmul produces a NaN?
(I should add that my expertise is in formal semantics for programming languages and IRs, but not in floating-point semantics.)