LLVM 13.0.1-rc2 has been tagged plus changes to the upload process

Project Infrastructure Release Testers
1

Hi,

I’ve tagged the 13.0.1-rc2 release now. As discussed on llvm-dev, I would like to try a new process for uploading the tester binaries where we upload the binaries directly to the github release page instead of to the llvm ftp site. Everything else will stay the same. Testers should still email me a cryptographic hash of the binary and then I will still sign the binaries once I receive the hash.

You can use the web interface to upload the binaries, but there is also a python script in tree that you can use:

llvm-project/llvm/utils/release/github-upload-release.py --token --release 13.0.1-rc2 upload --files

The advantage of doing this is that there is less uploading required for the release manager, and it also allows the binaries to be available early to people who don’t care about the signatures.

I will also be adding everyone who submits a binary to the release-testers team on github, and in the future, we may have a GitHub action that deletes uploads by people not on the release-testers team as an extra safety precaution.

Thanks,
Tom

3 Likes
2

Hi Tom,

That sounds good. You say you want us to email you the SHA sums - email or is it fine to post them as a reply to this topic as well?

I just wanted to highlight that if you have the github cli gh you can also upload binaries with that: gh release upload -R llvm/llvm-project llvmorg-13.0.1-rc2 *.tar.xz - since some of you might already have that tool configured it might be worthwhile.

The GitHub action sounds like a good idea!

3

That sounds good. You say you want us to email you the SHA sums - email or is it fine to post them as a reply to this topic as well?

Yes, that should be fine.

I just wanted to highlight that if you have the github cli gh you can also upload binaries with that: gh release upload -R llvm/llvm-project llvmorg-13.0.1-rc2 *.tar.xz - since some of you might already have that tool configured it might be worthwhile.

OK, thanks. I did not know that.

4

Windows is ready. sha256 hashes:

04e6b9b38ccac0a6b19820c8fd7ce1e220d9c6dd4c6227a79069ded7265f54bf LLVM-13.0.1-rc2-win32.exe
c6f5dcf8ef95d34059cabfbadb94c70fe87a4ff9c028e9486b6f02a08ea21eeb LLVM-13.0.1-rc2-win64.exe
5

AArch64 Linux & Windows ready:
c15daa1c695adbcaaeff5308adabd20dd77257517355fa3dc5cef043aa25f80a 13.0.1/clang+llvm-13.0.1-rc2-aarch64-linux-gnu.tar.xz
b8b96154ca4c78f86b3c928d71c235bd49dd7cd82f22e3c2307e770d5ee4a953 13.0.1/LLVM-13.0.1-rc2-woa64.zip

I’m seeing some new failures on Linux:
Profile-aarch64 :: Linux/coverage-linkage-lld.cpp
Profile-aarch64 :: Linux/coverage-weak-lld.cpp
Profile-aarch64 :: instrprof-gc-sections.c
libFuzzer :: gc-sections.test
libFuzzer :: gc-sections.test
libFuzzer :: gc-sections.test

I opened an issue here: AArch64: Linker not removing unused functions (gc-sections related) · Issue #53185 · llvm/llvm-project · GitHub

6

FWIW, I get a compile error in msan on FreeBSD now:

/home/dim/llvm/13.0.1/rc2/llvm-project/compiler-rt/lib/msan/msan_interceptors.cpp:660:46: error: function-like macro '__GLIBC_PREREQ' is not defined
#if SANITIZER_FREEBSD || SANITIZER_NETBSD || SANITIZER_STAT_LINUX
                                             ^
/home/dim/llvm/13.0.1/rc2/llvm-project/compiler-rt/lib/msan/msan_interceptors.cpp:659:50: note: expanded from macro 'SANITIZER_STAT_LINUX'
#define SANITIZER_STAT_LINUX (SANITIZER_LINUX && __GLIBC_PREREQ(2, 33))
                                                 ^
/home/dim/llvm/13.0.1/rc2/llvm-project/compiler-rt/lib/msan/msan_interceptors.cpp:699:46: error: function-like macro '__GLIBC_PREREQ' is not defined
#if SANITIZER_FREEBSD || SANITIZER_NETBSD || SANITIZER_STAT_LINUX
                                             ^
/home/dim/llvm/13.0.1/rc2/llvm-project/compiler-rt/lib/msan/msan_interceptors.cpp:659:50: note: expanded from macro 'SANITIZER_STAT_LINUX'
#define SANITIZER_STAT_LINUX (SANITIZER_LINUX && __GLIBC_PREREQ(2, 33))
                                                 ^
2 errors generated.

This appears to have regressed with [msan] Add stat-family interceptors on Linux · llvm/llvm-project@b40f07b · GitHub. I’ll apply a local workaround, but this should probably be fixed in main first.

7

macOS binaries are ready:

63e048d2eac0f5fd33ec420b4d49c9951f4f0a6c7e06201b2a677cb6bc2ad569 clang+llvm-13.0.1-rc2-x86_64-apple-darwin.tar.xz

The following tests failed:

FAIL: Clang :: Driver/darwin-ld-lto.c (7968 of 91780)
FAIL: Flang :: Runtime/no-cpp-dep.c (31518 of 91780)
FAIL: ORC-x86_64-darwin :: TestCases/Darwin/x86-64/trivial-cxa-atexit.S (78090 of 91780)
FAIL: flang-OldUnit :: Evaluate/folding.test (81202 of 91780)
/Users/tobias/code/llvm-releases/13.0.1/rc2/llvm-project/flang/unittests/Evaluate/folding.cpp:64: FAIL: y1Flushing.IsInfinite() || std::abs(host::CastFortranToHost<R4>(y1Flushing) + 88.) > 2
FAIL: libomp :: affinity/root-threads-affinity.c (88731 of 91780)
FAIL: libomp :: ompt/loadtool/tool_available_search/tool_available_search.c (88798 of 91780)
FAIL: libomp :: tasking/hidden_helper_task/gtid.cpp (88896 of 91780)
FAIL: libomp :: worksharing/for/kmp_sch_simd_guided.c (88940 of 91780)
FAIL: libunwind :: signal_frame.pass.cpp (88994 of 91780)

I have filed the following new issues:

The only one that seems a little like a blocker is Failed test `Driver/darwin-ld-lto.c` in 13.0.1-rc2 · Issue #53200 · llvm/llvm-project · GitHub - I hope someone familiar with that can help sus out what could be the problem there.

1 Like
8

Btw, for the github-upload-release script, what scope(s) should the Personal Access Token have? I think it might be “write:packages (Upload packages to GitHub Package Registry)”, or is that something totally different?

9

armv7 also ready:
875283d664723dc45a7c2a9fa4b4d88fbb83ecd493d52a5f5c9bbe471d536c80 clang+llvm-13.0.1-rc2-armv7a-linux-gnueabihf.tar.xz

Same issue as AArch64, except it only breaks for 3 tests (not sure if the other 3 are run at all though):
Profile-armhf :: Linux/coverage-linkage-lld.cpp
Profile-armhf :: Linux/coverage-weak-lld.cpp
Profile-armhf :: instrprof-gc-sections.c

1 Like
10

Okay, for the Personal Access Token, the scope “repo (Full control of private repositories)” seems to be sufficient to upload. (Provided you are in the right team; for some reason I can’t see the “release-testers” team, but uploading worked so I assume I’m in it. :slight_smile: )

11

There’s a fix that cherry-picks cleanly. Issue at please cherry-pick msan build fixes b4fb42300e39c99ac5bb9d02b304b713fabdec4d · Issue #53211 · llvm/llvm-project · GitHub. (No tags/milestones because I don’t have permissions).

12

Hmm still not much success on FreeBSD 13, the check-all target dies due to something (apparently) in python:

Testing:  0.. 10.. 20.. 30.. 40.. 50.. Process ForkPoolWorker-2:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 131, in worker
    put((job, i, result))
  File "/usr/local/lib/python3.8/multiprocessing/queues.py", line 368, in put
    self._writer.send_bytes(obj)
  File "/usr/local/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes
    self._send_bytes(m[offset:offset + size])
  File "/usr/local/lib/python3.8/multiprocessing/connection.py", line 404, in _send_bytes
    self._send(header)
  File "/usr/local/lib/python3.8/multiprocessing/connection.py", line 368, in _send
    n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/local/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 136, in worker
    put((job, i, (False, wrapped)))
  File "/usr/local/lib/python3.8/multiprocessing/queues.py", line 368, in put
    self._writer.send_bytes(obj)
  File "/usr/local/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes
    self._send_bytes(m[offset:offset + size])
  File "/usr/local/lib/python3.8/multiprocessing/connection.py", line 411, in _send_bytes
    self._send(header + buf)
  File "/usr/local/lib/python3.8/multiprocessing/connection.py", line 368, in _send
    n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
Process ForkPoolWorker-1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 131, in worker
    put((job, i, result))
  File "/usr/local/lib/python3.8/multiprocessing/queues.py", line 368, in put
    self._writer.send_bytes(obj)
  File "/usr/local/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes
    self._send_bytes(m[offset:offset + size])
  File "/usr/local/lib/python3.8/multiprocessing/connection.py", line 404, in _send_bytes
    self._send(header)
  File "/usr/local/lib/python3.8/multiprocessing/connection.py", line 368, in _send
    n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe

and a whole bunch more of these exceptions. It could be that I’m running out of RAM or something, but this didn’t occur with -rc1 …

13

Yep, running out of RAM apparently:

Jan 15 02:02:36 vfbsd130amd64 kernel: pid 61595 (python3.8), jid 0, uid 1000, was killed: out of swap space

VM with 6G of RAM, 10G of swap space, still not enough. :slight_smile:

14

For 13.0.1 rc1, I have built and tested on both FreeBSD 12 and 13. I had to cherry-pick [sanitizer] msan build fix on non linux platforms · llvm/llvm-project@b4fb423 · GitHub to fix the msan build, though.

For the 32-builds I used -no-flang, as flang is currently not 32-bit clean, and I do not expect it will ever be.

Main results on amd64-freebsd12:

  Skipped            :     3 (rc1:     3)
  Unsupported        :  6356 (rc1:  6353)
  Passed             : 91847 (rc1: 91833)
  Expectedly Failed  :   320 (rc1:   320)
  Timed Out          :     2 (rc1:     1)
  Failed             :   298 (rc1:   301)
  Unexpectedly Passed:     2 (rc1:     2)

Test suite results on amd64-freebsd12:

  Passed: 2419 (rc1: 2419)
  Failed:    3 (rc1:    3)

Main results on amd64-freebsd13:

  Skipped            :     3 (rc1:     3)
  Unsupported        :  6355 (rc1:  6352)
  Passed             : 91811 (rc1: 91797)
  Expectedly Failed  :   320 (rc1:   320)
  Timed Out          :     0 (rc1:     2)
  Failed             :   337 (rc1:   337)
  Unexpectedly Passed:     2 (rc1:     2)

Test suite results on amd64-freebsd13:

  Passed: 2419 (rc1: 2419)
  Failed:    3 (rc1:    3)

Main results on i386-freebsd12:

  Skipped            :     3 (rc1:     3)
  Unsupported        :  4741 (rc1:  4738)
  Passed             : 87575 (rc1: 87561)
  Expectedly Failed  :   295 (rc1:   295)
  Failed             :   196 (rc1:   198)
  Unexpectedly Passed:     1 (rc1:     1)

Main results on i386-freebsd13:

  Skipped            :     3 (rc1:     3)
  Unsupported        :  4741 (rc1:  4738)
  Passed             : 87571 (rc1: 87558)
  Passed With Retry  :     0 (rc1:     1)
  Expectedly Failed  :   295 (rc1:   295)
  Failed             :   200 (rc1:   200)
  Unexpectedly Passed:     1 (rc1:     1)

Uploaded:

SHA256 (clang+llvm-13.0.1-rc2-amd64-unknown-freebsd12.tar.xz) = 5999d0d98e05675c464ccc177bf8a945b63e613433141e8d0291df67cfb3c0c0
SHA256 (clang+llvm-13.0.1-rc2-amd64-unknown-freebsd13.tar.xz) = 243c1dc3a8c2c3b7555b4136c8fcbe59fee9ac5b4780f566f6816aac4e1cfde3
SHA256 (clang+llvm-13.0.1-rc2-i386-unknown-freebsd12.tar.xz) = 96d5501dbbdc8722be1e03e6fb769cd8afe7804c81894a501c930e61f13db15f
SHA256 (clang+llvm-13.0.1-rc2-i386-unknown-freebsd13.tar.xz) = 58f3930c771c234c4b96476ebe41f9a45ce339bd0cc92510cf195cbdf4ce731d
15

Maybe we should make Flang disabled for 32 bit by default?

16

Yeah it might be a good idea. Probably best to check with the flang maintainers?

17

Parts of the Flang build are also extremely memory hungry (compilation of single files using one excess of 4.5GB of RAM), it might be helpful to be able reduce parallelism of these builds. (I’m fielding a pretty regular stream of reports from FreeBSD ports users who’s compilations fail on systems without a large amount of RAM).

18

Looks good for me on Debian & Ubuntu

19

PowerPC binaries are ready.

dbcefe7d7eb19c211f02f2eabbc363239ac54770  clang+llvm-13.0.1-rc2-powerpc64le-linux-rhel-7.9.tar.xz
b9daafa70900a676f006c1fd9a9c0b76dc2ffeac  clang+llvm-13.0.1-rc2-powerpc64le-linux-ubuntu-18.04.5.tar.xz

I also saw these new failures:

  • Profile-powerpc64le :: Linux/coverage-linkage-lld.cpp
  • Profile-powerpc64le :: Linux/coverage-weak-lld.cpp
  • Profile-powerpc64le :: instrprof-gc-sections.c
1 Like
20

The SHA256 hash doesn’t match for me with the clang+llvm-13.0.1-rc2-amd64-unknown-freebsd13.tar.xz upload.