Recently when @stellaraccident and I have been working on npcomp/TCF/TCP, along with other discussions in the context other systems, the need for a
tensor type with an unknown element type has come up.
This is extremely common when attempting to faithfully model frontends with tensor types based on a dynamic language like Python (numpy, PyTorch, and TensorFlow are concrete examples). For example, in the absence of further refinement the Python code
numpy.add(x, y) maps to an op something like
numpy.add %x, %y : (tensor<*xany>, tensor<*xany>) -> tensor<*xany>. The same is true for TensorFlow and PyTorch. Ideally we would like to do that type refinement algorithm inside MLIR so it can be shared across all of them (and also because we generally like MLIR ) .
The fact that element type is currently mandatory I think is just a historical artifact – TensorFlow’s GraphDef protocol buffer representation has no way of expressing an unknown element type for an tensor (even though tensorflow ops can be polymorphic), and I think MLIR just inherited that accidentally.
There’s two obvious ways to dealing with this (would like to hear others):
- Add built-in support to tensor types to know that the element type is unknown, much as we do for dimensions with unknown extents. This could be either allowing getElementType() to return nullptr or have getElementType() internally
assert(hasElementType()), much as getShape() aborts for unranked ShapedType’s today.
- Add an
anytype and have, for example,
tensor<?xany>. (we can bikeshed the name; in npcomp we call it
I tend to lean towards 2, because:
A. It uses an existing extension point. It also scales nicely to in the future e.g. refining
oneof<f32, f64> in a type inference pass.
B. I’m really worried about getElementType() returning nullptr or aborting. That seems ripe for a long tail of random compiler crashes. Pragmatically, ODS predicates like
AnyFloat will need to be rewritten to have null checks, e.g.
$_self && $_self.isa<FloatType>(), which is inconvenient, easy to forget, and a lot of churn.