Thanks. No argument on what you are trying to do or the utility of it – just iterating on how it will be done.
Emits it to where? I’m guessing the pass takes a parameter like
Is there also an inverse operation, to go from a tosa flatbuffer → ModuleOp? (Or: should there be while we’re in here?)
I don’t generally see i/o pases like this, but maybe its ok (definitely fine to do if practical in an external project but expect more scrutiny in the llvm repo). Still feels like this is more the territory of a tool, which would be more of the usual way of integrating this kind of thing across llvm and works fine with build/testing/etc (as Mehdi says, the translate api is mostly a means to have fewer tools in an MLIR build – not an end in itself).
Regardless of tool vs pass, this seems slightly monolithic: there should probably also be an API with the bulk of the implementation like:
LogicalResult exportTosaFuncToFlatbuffer(FuncOp, Serializer&)
LogicalResult importTosaFuncFromFlatbuffer(Serializer&, OpBuilder&)
Then the tool/translation/Io passes would be the trivial user of these APIs. If you had bidirectional support, I imagine your unit tests of the API would mostly be of the roundtrip variety. E.g:
// RUN: mlir-tosa-convert -export-to-flatbuffer %s | mlir-tosa-convert -import-from-flatbuffer
(A similar pipeline would work with an opt/pass based approach)
If the tool is not bidirectional, we’d need to come up with another mechanism to test it. If history is a guide, just making it bidirectional can be easier (and have utility on its own) than other approaches, but if truly one way, there are things that can be done there (ie. Have a flag to dump the flatbuffer as json), which you then pattern match).
Generally, you want to make sure that however you set up your tool, it supports testing of the API as a first order thing. This also tends to make it a reasonably useful thing for folks to use as well. We’d need to fiddle with it to figure the best way to also test the side npy files but it seems like we can follow-up with that. There are some examples of multi output tools/tests we could look at.
In terms of integration, something like IREE or TF would likely just integrate the c++ API directly in the appropriate production tool (IREE doesn’t ship opt binaries to users generally, because that is a dev tool). They can also pretty trivially build and re-export the tool binary if it is a use to their final product.
If you’d like to post a draft patch with your proposed approach that could be a next step – versus answering all questions of approach here. I’d just advise that based on the discussion here so far, there are likely to be structural comments that necessitate rework, and if you biased towards a rough draft that you haven’t spent much time polishing for initial comment (and note that it is for design feedback in the description), that can help limit the overall cycles and wasted time. For this kind of structure thing, I could also look at an llvm fork if you had a branch somewhere with these changes.