I think that tensors with a dimension of size 0 are not “just a frontend concern” that is going to boil away before we get lower into the system. I claim that the following PyTorch functions (which have trivial analogs in every tensor programming system) are legal and should be supported by every reasonable compiler for those systems.
>>> import torch
>>> def f(t):
... return t[1:1]
>>> def g(t: torch.Tensor, start: int, end: int):
... return t[start:end]
>>> g(torch.tensor([2,3,4]), 1, 1)
A real-world example where this is relevant is e.g. a program that is maintaining a buffer of accumulated audio samples and a function to get the accumulated audio samples. If 0 samples have been accumulated, the result will be a tensor with a dimension of size 0.
I also don’t see any relevance to the question of whether “0x” in the type is allowed or not. It’s really a question about the semantics of the type, and whether dynamically 0 is allowed (whether we happen to statically know it is 0 is immaterial to the discussion). Examples like the one I showed above suggest that we do need to support the use case of a dimension of size 0 in the builtin tensor type.
Also, as a frontend author, it’s quite inconvenient for me to multiversion my code around every single potentially-zero case. E.g. in the following example, if the size of
tt is not known statically, I would expect that lower levels of the stack should create a matmul kernel that cleanly handles the zero case. Is there some fundamental reason this is hard? It seems like using half-open loops for iteration naturally handles this.
tensor(, size=(0, 1), dtype=torch.int64)
>>> torch.matmul(tt, tt.t())
tensor(, size=(0, 0), dtype=torch.int64)