CIRCT as a backend for npcomp

LLVM Incubator CIRCT
#1

In the meeting today, I asked if anyone was interested in targeting CIRCT as a backend for npcomp. I previously brought this up on the npcomp forum here: [npcomp] CIRCT integration. It seems like we can connect the dots between npcomp and CIRCT, enabling high-level synthesis of Python-based array programming frameworks like NumPy, TensorFlow, PyTorch, etc.

Right now, npcomp’s reference backend accepts a mix of std elementwise ops on tensors and linalg named ops on tensors. This level of abstraction is not much higher than what CIRCT can lower into handshake. I’m going to start investigating what needs to happen to pave a path from npcomp’s IR to handshake.

Is anyone else from the CIRCT camp interested in this? This is something I’m planning to allocate some cycles to in the near term, and I’d love to collaborate.

#2

I’d very much like to see this happen. I’d do something along the lines of FINN: https://github.com/Xilinx/finn, but using CIRCT code generation to Verilog, rather than Vivado HLS.

#3

Glad to hear. I dug into FINN a bit when I reached out to the npcomp group previously, and I agree that an integration between npcomp and CIRCT should be similar to FINN at a high level.

#4

This is very interesting, and my apologies for the delay – I literally missed the prior message and found this one when scanning the discourse backlog. I need to look at my notification settings.

So, first - yes! But it does give me some short term pause on two points: if putting this together for real, we’re going to need to spend some time syncing our dev setups and both projects are running very independent right now.

Second - @_sean_silva and @brycearden are making progress on converging to that level of abstraction you want to source from but I’d ideally like to see that be a little more real before introducing hard dependencies.

I think both of those points lead us to some kind of sneakernet prototype, more like a python/shell script that legalizes the frontend part and pipes the asm to your backend.

This would be very nice to see come together! Just a bit cautious on the sequencing of early phase things.

#5

Hi Stella, thanks for the input. I agree with you on both fronts, and I definitely don’t want to create any extra work by trying to connect the dots prematurely. I have some bandwidth, and I’d like to see this happen, so I thought I’d try to get the ball rolling with some investigation on my end.

This is what I had in mind for a proof-of-concept. I’ve been following along with npcomp but I haven’t built it in a long time. My plan right now is to get back up to speed with the dev setup and see what kind of IR I can pipe out today. At that point, I should be able to have more concrete discussions about both the dev setups and the level of abstraction the two projects interface with.

#6

This is really exciting! I agree with Stella that we probably want a sneakernet version of this first, and not take a hard dep until we build out a bit more at these abstraction layers.

In old school mailing lists, OP would just CC circt-dev and npcomp-dev. But alas, I like my emojis :slight_smile: and markdown editor.

#7

I’m away from my computer, but at the next chance, let me find you a more-done-than-not test case to rif off of. Right now there are only a few arithmetic ops going “all the way” – more at the aten layer. I’d rather focus on what is roughly at the right layer and grow from there vs gluing things together that need more work to do right.

Likewise, I haven’t tried to build circt in a while and don’t know anything about your simulators and such. Is the documentation enough to get an outsider going on any of that?

Just had a discussion with the IREE side and trying to bridge that for real in the coming weeks too, so it is shaping up to be a season of backend integrations :slight_smile:

#8

That would be much appreciated!

Not really… there isn’t a ton of documentation outside the README. The diagram is not exhaustive, but it covers a lot of the pieces accurately. Unfortunately, the tests are the best documentation for what passes, tools, etc. can be invoked. If you’d like to explore, I’d be happy to share some examples and a work on a proper getting started guide.

#9

It’s all good. It is somehow mildly reassuring to know that your project isn’t the only one that is indecipherable :slight_smile:

Here is the main example that “does something” e2e for us (the rest is more unit tests): https://github.com/llvm/mlir-npcomp/blob/master/frontends/pytorch/examples/mul_maximum_e2e.py

The prior revision is the more sneakernet/instructive, though (for the frontend): https://github.com/llvm/mlir-npcomp/blob/e60dc2470ecf47ec0a430dd5b0bcb4febe64cc1b/frontends/pytorch/examples/mul_maximum_e2e.py

Right now, lowerings for the backend are in a pipeline that goes all the way to LLVM, which is too far. We should split that up and have examples at a couple of different slice points.

#10

Awesome, thanks. I was looking at the commit where that was added and it is very instructive. I’m almost up to speed with the build setup for NPComp, and then I’ll do some experiments with that example.

#11

Ok. I’m going to be pushing some improvements to the build soon as well (mainly to the CMake options and checks for finding python and pytorch, which is catching a lot of people up).

#12

Cool, that would be very helpful.

Here’s one more data point re: building with Python bindings.

I am on Ubuntu 18.04 and already had a system wide pybind11 install at version 2.0.1, which failed to compile with the current LLVM/MLIR. I installed 2.6.1 for building this, and it works.

Is there a specific version of pybind11 that is required?

I didn’t see any specific version here: https://github.com/llvm/llvm-project/blob/master/mlir/docs/Bindings/Python.md#pre-requisites, but this comment seems to indicate at least some people are using 2.6: https://github.com/llvm/llvm-project/blob/master/mlir/CMakeLists.txt#L85.

Anyway, just a little realtime feedback as I’m going through this for the first time.

#13

Thanks for that update. We’re trying to get this flag flipped on the buildbots now, which I suspect is going to flush out some of these issues and can be a basis for some precise documentation. 2.0.1 is quite old: It has all been developed with something between 2.4 >= 2.6.

1 Like
#14

I was able to get this example going and just printing the IR to stdout. I also commented out the rest of the RefBack lowering pipeline starting here: https://github.com/llvm/mlir-npcomp/blob/master/lib/RefBackend/RefBackend.cpp#L259. With that, I’m able to do some pretty cool stuff, bridging the gap from npcomp to circt using upstream passes.

For example, here’s a sneakernet proof of concept that goes straight down to std and then handshake in circt:

$ python frontends/pytorch/examples/mul_maximum_e2e.py | \
    npcomp-opt --aten-recognize-kernels \
               --convert-aten-to-tcf \
               --numpy-public-functions-to-tensor \
               --canonicalize \
               --tcf-refback-lowering-pipeline | \
    mlir-opt --convert-linalg-to-loops \
             --convert-scf-to-std \
             --canonicalize \
             --cse | \
    circt-opt --create-dataflow \
              --canonicalize-dataflow \
              --remove-block-structure \
              --canonicalize \ 
              --cse \
              --handshake-insert-buffer > mul_maximum_handshake.mlir

This could also go through affine, which circt is gaining support for.

Anyway, this is just the tip of the iceberg, but I thought it was really cool to see both projects working together with mlir in the middle.

Here’s the generated handshake: mul_maximum_handshake.txt (144.2 KB)

2 Likes
#15

Oh wow, this is awesome! And is basically what I ran out of time to try today (we’re plumbing the same sneakernet thing for IREE too, and just having the above saves me a lot of time spelunking on that).

This is really motivating to have so many e2e flows shaping up like this. I don’t think it is going to be super hard to scale it to some of the the more representative complex cases.

#16

Indeed this is incredibly cool :slight_smile:

#17

fyi - as of D91021, v2.6.0 will be required and we’re adding that note to the docs. Thanks for the catch.

1 Like
#18

Wow, this is awesome! (sorry, just seeing it now due to weird notification settings)

Sorry you had to hack up tcf-refback-lowering-pipeline. I just pushed a patch and you should be able to run those isolated passes with -refback-tcf-to-tcp-lowering-pipeline.

#19

No worries, it was good for my own understanding. Thanks for the new pass pipeline option, that’s great.

#20

Resurrecting this thread, since we touched on the NPComp -> CIRCT in our meeting today but it seemed like everyone had to run. To the people interested in compiling numerical Python programs with CIRCT: should we have a follow-up discussion some time? Or hash out some next steps here in the forum?

For now, in the spirit of having something connected end-to-end, I’m trying to see if we can get from actual NPComp output to the Affine dialect using upstream MLIR passes, and then see if we can handle that using the Affine dialect support that is already landed in CIRCT.