Linagl VS Affine

Hi, community!

I’m a beginner of MLIR and didn’t have much experience with it.
After reading these docs: Linalg Dialect Rationale: The Case For Compiler-Friendly Custom Operations, ‘linalg’ Dialect, ‘affine’ Dialect, I found quite confusing about the relationship between Linalg dialect and Affine dialect.

  1. I have read that Linalg stands on the HHO level, which is higher than the MHA level which Affine stand, but I think I didn’t understand what it actually means when optimizing a DL op like conv2d. Could you please show me an example of that? And I didn’t quite clear what does the Linalg is intended to complement Affine rather than replace it mean.
  2. I found we have linalg.conv_2d_nchw_fchw in Linalg IR. but after convert-linalg-to-loops, the conv op seems gone. So do we have any method to keep this op until lowering to LLVM, then call it in cudnn.so, or something like that?
  3. Besides, we have a simplified poly form in MLIR: The case for a simplified polyhedral form, so do we use it in Linalg or Affine?

Thanks!

All of these “rationale” docs are outdated to a different extent. We keep them as justification for past design decisions. We should probably add dates to them so people know what to expect.

These levels are purely a mental exercise at dialect layering. What it actually means is Linalg having a possibility to define and use a “conv2d” operation with the attached semantics. So the compiler can do understand that the operation is a 2d convolution. Affine operates on loops, so one would have to spell out the four loops that make up the 2d convolution. At this level, the compiler need to do some non-trivial analysis to understand that these four loops along with the operations in their body are in fact a 2d convolution.

This means that Linalg can be converted to Affine and further transformations can be performed on that representation if desired.

You literally asked the compiler to convert Linalg operations to loops. You got what you asked for – loops.

Not yet. It has been designed for such cases, but nobody actually needed that particular feature so nobody implemented it. It shouldn’t be hard to write as a pattern.

This is a 3+ year rationale doc for what has become the Affine dialect. It is not used in Linalg.

1 Like

Besides, in my view, the features/benefits or design principles of Linalg are codegen-friendly, more expressive and higher-level primitives, composable and declarative transformation, smaller components and more usable, suitability for search/ML methods.
wow, so many benefits! :slight_smile: Did I miss anything?

By the way, I found confusing about the section The Dialect Need not be Closed Under Transformations, could you please explain that a bit more? Thanks!

Thanks very much for your reply!
I’m quite interested in this. Do you mean write a pattern in linalg, then use something like library_call?

Yes, it will be just a RewritePattern that converts the op into either std.call or llvm.call, it is not really a “in linalg”. I suppose this must happen after bufferization though.

okay, thank you! I got that and will give it a try!

This one is a little more elaborate to explain, but basically you can think of it as a answering the question: assuming a set of primitive operands and types; when applying a transformation, should the resulting IR be expressible with the same primitives and types or not ?

This is an important question because it can pull you in different directions in the space of IR design vs analysis and expressiveness vs specialization. One concrete example is a simple linalg.matmul and how to represent a tiled version of that. This is a 3d operation (its iteration space is 3d) and let’s assume we tile each one of the 3 loops; which gives us 6d semantics. Here are 3 possible ways of implementing the tiling:

  • 6 loops over scalar operations: this is what scf.for and affine would naturally do, you can use that if it makes sense for your use case.
  • one single 6d contraction op: this is appealing in principle but has a number of issues esp. related to the type system (statically known vs ?) when mixed with non-divisible boundary conditions. This is a valid design space but it does pull you into designing more and more stuff in the IR and potentially new types (i.e. n-D lists of n-D arrays).
  • 3 loops over a 3d contraction op: this has some nice properties that I won’t go into details here and is what we settled on.

As a consequence, applying a transformation on Linalg does not try to produce a result that is a set of pure Linalg ops: it produces loops and Linalg. This which leads to better compositionality with the rest of the world and avoids bad reflexes of adding more IR design on top of more IR design which is a dangerous slippery slope.

I think I can say historically there has been a strong bias towards systems that try very hard to be closed under transformations and we make the conscious, opposite, choice (at least for now).

Thank you very much for the clear example!

So the unclosed primitives(can we call it like this?) avoid adding more and more primitives in Linalg, as long as they appear in other dialects, and thus have an easier way to interact with other dialects. Then all primitives in all of these dialects can convert to llvm(or bytecode related with PDL dialect?) separately. Did I get that right?

To be honest, I think the current Linalg Dialect is complicated enough. :thinking: :joy:

Here is a minimal test that we have to lower linalg to library calls:

Here is the basic pass:

I would love some effort here, the lack of prioritization on this front has been unfortunate.

Here is a related older thread: Lowering optional attributes in Linalg StructuredOps to Standard dialect

In a good first approximation, there is really one single op in Linalg: linalg.generic.

Many other ops are just configurations of that generic op to which we give a name: i.e. syntactic sugar.
For instance linalg.matmul is a linalg.generic configured to implement the computation: C(i, j) += A(i, k) * B(k, j); many other ops follow the same pattern, are specified with opdsl and are auto-generated from this spec file: core_named_ops.py - llvm/llvm-project - Sourcegraph.
For your particular case of linalg → library, that syntactic sugar name is load-bearing: that named op gets converted to a function name.

When performing transformations (e.g. tiling), we produce scf.for, tensor.extract_slice, tensor.insert_slice and other ops. One possibility would also be to produce a 6-d linalg.generic, which we don’t. In that sense, the Linalg dialect does not try to be closed under transformations and prefers to compose with the rest. In particular, we try hard to

unless they are absolutely necessary.

Exactly.

Thank so much for this example!!! It’s realy helpful! :smile:

wow, thank you! It is really clear and helps me a lot! :smile:

Also relevant to you I think: Linalg to LLVM Lowering - #6 by nicolasvasilache

yeah, that’s helpful! thank you!

I have converted the linalg IR with a library call function to an obj file and written a main() function and _mlir_ciface_print_memref_f32, _mlir_ciface_linalg_fill_f32_viewsxsxf32, _mlir_ciface_linalg_matmul_viewsxsxf32_viewsxsxf32_viewsxsxf32 functions to run it so far.
thank you again for your help!

A lot of things are missing to get this polished and to a point where it can scale and we can properly mix codegen + library calls. They are all quite unambiguous but potentially a lot of work depending on the degree of usability and pluggability one is interested in.

This is actually what I want to implement! But I found that the linalg, std and llvm toolchains have almost done everything!

Indeed, I really want to do something about that and post it as a RFC one day. But actually, I didn’t have a clear mind about it. Could you please give me some advice or what do you think about it?

Specifically, I want to implement something like there’s an interface for users to write their own function or call it from libs. It seems like this: Lowering optional attributes in Linalg StructuredOps to Standard dialect - #2 by nicolasvasilache. Using an attribute sounds good to me. :slight_smile:

But actually, there’s some concept I didn’t fully understand and I guess I need to read them carefully again.