Seperate the value of const op in another file

Is it possible to seperate the value of const op in another file? Or the mlir text will be difficult to view.

1 Like

What would your use case be?

Extern constants have been a topic, but no general solution there yet (and may be different than what you are looking for here). A “local” approach to having constant external is to have an op such as

%1 = "mine.load_from_file"() {filename = "const_a.txt"} : () -> tensor<200x200xf32>

and then give this op a constant folder/canonicalizer that loads and returns the constant. This only solves the “I want my constant in an external file” part of the issue though. This makes the .mlir file no longer self-contained though and yes to see the value one would have to open another file. So this may not be too desirable depending on your use case.

module  {
  func @main(%arg0: tensor<128x28x28x1xf32>) -> tuple<tensor<128x10xf32>> {
    %0 = mhlo.constant dense<0.000000e+00> : tensor<128x11x11x64xf32>
    %1 = mhlo.constant dense<0.000000e+00> : tensor<128x26x26x32xf32>
    %2 = mhlo.constant dense<"0x00000000010..."> : tensor<3x3x1x32xf32>
    %3 = mhlo.constant dense<[4.035740e-43, 4.049750e-43, 4.063770e-43, 4.077780e-43, 4.091790e-43, 4.105800e-43, 4.119820e-43, 4.133830e-43, 4.147840e-43, 4.161860e-43, 4.175870e-43, 4.189880e-43, 4.203900e-43, 4.217910e-43, 4.231920e-43, 4.245930e-43, 4.259950e-43, 4.273960e-43, 4.287970e-43, 4.301990e-43, 4.316000e-43, 4.330010e-43, 4.344030e-43, 4.358040e-43, 4.372050e-43, 4.386060e-43, 4.400080e-43, 4.414090e-43, 4.428100e-43, 4.442120e-43, 4.456130e-43, 4.470140e-43]> : tensor<32xf32>
    %4 = mhlo.constant dense<"0x4001000041010000..."> : tensor<3x3x32x64xf32>
    %5 = mhlo.constant dense<[2.627710e-41, 2.627860e-41, 2.628000e-41, 2.628140e-41, 2.628280e-41, 2.628420e-41, 2.628560e-41, 2.628700e-41, 2.628840e-41, 2.628980e-41, 2.629120e-41, 2.629260e-41, 2.629400e-41, 2.629540e-41, 2.629680e-41, 2.629820e-41, 2.629960e-41, 2.630100e-41, 2.630240e-41, 2.630380e-41, 2.630520e-41, 2.630660e-41, 2.630800e-41, 2.630940e-41, 2.631080e-41, 2.631220e-41, 2.631360e-41, 2.631500e-41, 2.631640e-41, 2.631780e-41, 2.631920e-41, 2.632060e-41, 2.632200e-41, 2.632340e-41, 2.632480e-41, 2.632620e-41, 2.632760e-41, 2.632900e-41, 2.633040e-41, 2.633180e-41, 2.633320e-41, 2.633460e-41, 2.633600e-41, 2.633740e-41, 2.633880e-41, 2.634020e-41, 2.634160e-41, 2.634300e-41, 2.634440e-41, 2.634580e-41, 2.634720e-41, 2.634860e-41, 2.635000e-41, 2.635140e-41, 2.635280e-41, 2.635420e-41, 2.635560e-41, 2.635700e-41, 2.635840e-41, 2.635980e-41, 2.636120e-41, 2.636260e-41, 2.636400e-41, 2.636540e-41]> : tensor<64xf32>
    %6 = mhlo.constant dense<"0x80490000814..."> : tensor<1600x10xf32>
    %7 = mhlo.constant dense<[-0.121695712, -0.449199766, 0.454357237, 0.274700165, -0.304930776, 0.326857656, -0.00982450321, 0.104005054, -0.3422741, 0.717462897]> : tensor<10xf32>
    %8 = mhlo.constant dense<0xFF800000> : tensor<f32>
    %9 = mhlo.constant dense<0.000000e+00> : tensor<f32>
    %10 = mhlo.convolution(%arg0, %2) dim_numbers = [b, 0, 1, f]x[0, 1, i, o]->[b, 0, 1, f], window = {stride = [1, 1], pad = [[0, 0], [0, 0]], lhs_dilate = [1, 1], rhs_dilate = [1, 1]} {batch_group_count = 1 : i64, feature_group_count = 1 : i64, precision_config = ["DEFAULT", "DEFAULT"]} : (tensor<128x28x28x1xf32>, tensor<3x3x1x32xf32>) -> tensor<128x26x26x32xf32>
    %11 = "mhlo.broadcast_in_dim"(%3) {broadcast_dimensions = dense<3> : tensor<1xi64>} : (tensor<32xf32>) -> tensor<128x26x26x32xf32>
    %12 = mhlo.add %10, %11 : tensor<128x26x26x32xf32>
    %13 = mhlo.maximum %12, %1 : tensor<128x26x26x32xf32>
    %14 = "mhlo.reduce_window"(%13, %8) ( {
    ^bb0(%arg1: tensor<f32>, %arg2: tensor<f32>):  // no predecessors
      %32 = mhlo.maximum %arg1, %arg2 : tensor<f32>
      "mhlo.return"(%32) : (tensor<f32>) -> ()
    }) {base_dilations = dense<1> : tensor<4xi64>, padding = dense<0> : tensor<4x2xi64>, window_dilations = dense<1> : tensor<4xi64>, window_dimensions = dense<[1, 2, 2, 1]> : tensor<4xi64>, window_strides = dense<[1, 2, 2, 1]> : tensor<4xi64>} : (tensor<128x26x26x32xf32>, tensor<f32>) -> tensor<128x13x13x32xf32>
    %15 = mhlo.convolution(%14, %4) dim_numbers = [b, 0, 1, f]x[0, 1, i, o]->[b, 0, 1, f], window = {stride = [1, 1], pad = [[0, 0], [0, 0]], lhs_dilate = [1, 1], rhs_dilate = [1, 1]} {batch_group_count = 1 : i64, feature_group_count = 1 : i64, precision_config = ["DEFAULT", "DEFAULT"]} : (tensor<128x13x13x32xf32>, tensor<3x3x32x64xf32>) -> tensor<128x11x11x64xf32>
    %16 = "mhlo.broadcast_in_dim"(%5) {broadcast_dimensions = dense<3> : tensor<1xi64>} : (tensor<64xf32>) -> tensor<128x11x11x64xf32>
    %17 = mhlo.add %15, %16 : tensor<128x11x11x64xf32>
    %18 = mhlo.maximum %17, %0 : tensor<128x11x11x64xf32>
    %19 = "mhlo.reduce_window"(%18, %8) ( {
    ^bb0(%arg1: tensor<f32>, %arg2: tensor<f32>):  // no predecessors
      %32 = mhlo.maximum %arg1, %arg2 : tensor<f32>
      "mhlo.return"(%32) : (tensor<f32>) -> ()
    }) {base_dilations = dense<1> : tensor<4xi64>, padding = dense<0> : tensor<4x2xi64>, window_dilations = dense<1> : tensor<4xi64>, window_dimensions = dense<[1, 2, 2, 1]> : tensor<4xi64>, window_strides = dense<[1, 2, 2, 1]> : tensor<4xi64>} : (tensor<128x11x11x64xf32>, tensor<f32>) -> tensor<128x5x5x64xf32>
    %20 = "mhlo.reshape"(%19) : (tensor<128x5x5x64xf32>) -> tensor<128x1600xf32>
    %21 = "mhlo.dot"(%20, %6) {precision_config = ["DEFAULT", "DEFAULT"]} : (tensor<128x1600xf32>, tensor<1600x10xf32>) -> tensor<128x10xf32>
    %22 = "mhlo.broadcast_in_dim"(%7) {broadcast_dimensions = dense<1> : tensor<1xi64>} : (tensor<10xf32>) -> tensor<128x10xf32>
    %23 = mhlo.add %21, %22 : tensor<128x10xf32>
    %24 = "mhlo.reduce"(%23, %8) ( {
    ^bb0(%arg1: tensor<f32>, %arg2: tensor<f32>):  // no predecessors
      %32 = mhlo.maximum %arg1, %arg2 : tensor<f32>
      "mhlo.return"(%32) : (tensor<f32>) -> ()
    }) {dimensions = dense<1> : tensor<1xi64>} : (tensor<128x10xf32>, tensor<f32>) -> tensor<128xf32>
    %25 = "mhlo.broadcast_in_dim"(%24) {broadcast_dimensions = dense<0> : tensor<1xi64>} : (tensor<128xf32>) -> tensor<128x10xf32>
    %26 = mhlo.subtract %23, %25 : tensor<128x10xf32>
    %27 = "mhlo.exponential"(%26) : (tensor<128x10xf32>) -> tensor<128x10xf32>
    %28 = "mhlo.reduce"(%27, %9) ( {
    ^bb0(%arg1: tensor<f32>, %arg2: tensor<f32>):  // no predecessors
      %32 = mhlo.add %arg1, %arg2 : tensor<f32>
      "mhlo.return"(%32) : (tensor<f32>) -> ()
    }) {dimensions = dense<1> : tensor<1xi64>} : (tensor<128x10xf32>, tensor<f32>) -> tensor<128xf32>
    %29 = "mhlo.broadcast_in_dim"(%28) {broadcast_dimensions = dense<0> : tensor<1xi64>} : (tensor<128xf32>) -> tensor<128x10xf32>
    %30 = mhlo.divide %27, %29 : tensor<128x10xf32>
    %31 = "mhlo.tuple"(%30) : (tensor<128x10xf32>) -> tuple<tensor<128x10xf32>>
    return %31 : tuple<tensor<128x10xf32>>
  }
}

The IR above is an example of my use case. It represents a mnist conv model. When reviewing code, I have to scroll the web page all the way down since the sizes of some const ops are very large.

For debugging purpose I frequently use --mlir-elide-elementsattrs-if-larger=10, but note that it breaks round-tripping.

I’ve been on the verge of doing a POC of some generic infra here (externalizing constants) for a long time, but I always end up having somewhat more important things to be doing and just deal with the various frictions that come from having them inline.

For interactive use, as Mehdi says, the mlir-elide-elementsattrs-if-larger flag helps but be careful if you have multi stage pipelines and such. Also tends to be annoying if using in combination with print-ir-after-all to capture some debugging state as it also modifies the final output.

Thanks for your replies.

I could not enable mlir-elide-elementsattrs-if-larger because I will add the mlir text in my code repo and use it for test afterwards.

Toggling line folding on and off in the editor helps for the large constants. Depending on editor used, one could even add folding support for sections, e.g., vim has a folding based on syntax, so not sure if combining these would work with the mlir syntax file loaded … Well and not sure we made constant arrays a category there, so some extension may be need :slightly_smiling_face:

Thus far switching between line or not has worked reasonably well even with >10 MB constants in there. But interested if you find more convenient ways.

Yeah, toggling line folding on and off in the editor could help. But when it comes to gitlab MR page, there seems not a good way to handle those large constants.

What I want to find is something similar to Tensorflow Saved Model format. Currently I’m busy implement the essential features based on MLIR. Maybe I could take a look at this when I get more spare time. :grinning: