LLVM Discussion Forums

Shared library support

A couple of weeks ago I raised the question on the tensorflow/mlir mailing list that I would like to see
support for building MLIR either with libLLVM.so or as it’s own libMLIR.so. https://groups.google.com/a/tensorflow.org/forum/#!topic/mlir/pKvf1UDpl7s

There is a bigger question whether we should have a separate libMLIR or whether we should embed
the symbols into libLLVM.so.

Pros for the libMLIR.so approach:

  • MLIR right now is dependent but separate from LLVM proper
  • Reduces the size and the symbol count of libLLVM (the latter is already a problem)
  • Allows consumers of builds from LLVM to choose if they want to load MLIR

Cons for the libMLIR.so approach:

  • May make it harder for MLIR to become used in LLVM proper (circular dependency)

Paul referenced https://reviews.llvm.org/D61909 which is for libclang-cpp.so and has some notes on the design thinking there and how one might refactor llvm-shlib.

This created several Phabricator PRs:

(breaking apart because of discourse user limitations)

I’ve broken the circular dependencies:
https://reviews.llvm.org/D73654
https://reviews.llvm.org/D73655

The combination of these patches should allow BUILD_SHARED_LIBS=on.

1 Like

Another wrinkle will be:

In D72822 @rridle wrote:

You are going to run into a much bigger problem given that this depends on ClassOf, which is a static typeid that is used pervasively across the entire code base. Making that work with shared libraries is likely the most important(difficult?) task item.

I must admit that my understanding of C++ is not strong enough to offer any salient thoughts on that.

Sorry, there is a typo in that quote. It should be ClassID.

That class will have to be modified to work correctly in the presence of shared libraries. For Linux I think it should mostly just work as-is(% maybe tweaking the visibility). For Windows DLL, I think we will have to fallback to comparing class names(using llvm::getTypeName?). It would help if we can start clarifying which problems are affecting which systems. This makes it easier as some systems, mainly MSVC, have no problems that can be worked around. I’m more than happy to fix this myself, but it would help to have some repro instructions that can be used for testing the various scenarios.

I plan to create a isolated reproducer next week :slight_smile: right now encountered while experimenting with MLIR in JuliaLang and that setup is convoluted enough that it isn’t a reasonable reproducer.

OK, the patches have landed to allow BUILD_SHARED_LIBS=on to work… Thanks for the help and suggestions.

2 Likes

I’ve been starting to work with @vchuravy 's patch here https://reviews.llvm.org/D72554
With some tweaks, I was able to get this close to working. I think the next major problem is dealing with the structure of the CMakefiles. When building with LLVM_LINK_LLVM_DYLIB, llvm_add_library() rewrites the LINK_LIBS keyword. This means that the current style using target_link_library doesn’t work, instead we have to switch to using the LINK_LIBS keyword to get the proper dependencies.

Similarly, llvm_add_library can also generate new targets, so it needs to know about the dependencies of those targets. The solution is to move the current style with add_dependencies() into the DEPEND keyword.

I think I’m close to getting something working using this.

1 Like