LLVM Discussion Forums

Changing clangd index

Hi,

While working with clangd, I found that the index was next to the sources files of the project.
When updating clang, there are clangd errors because I’ve updated from clangd 9 to 10:

E[23:38:12.781] Background-index: Couldn't read [...]\LLVM\lib\clang\9.0.0\include\inttypes.h to validate stored index: no such file or directory

Obviously this folder doesn’t exists as in that folder there is LLVM 10 now (it replaced the previous version).

I see some threads talking about a centralized clangd index for a project, but I think this is not possible for all situation, especially when changing the compiler, the clangd version or using multiple out of sources build directories from one single source directory.

I guess the clangd index should include all included files hash to the hash it puts behind file names like this:
bio.h.858B49E3CCCF6E34.idx

I guess the hash is made on the source file itself only as of now, that would explain why clangd 10 sees reference to headers of clang 9.

Could this be fixed ?

Just found that --compile-commands-dir=<string> also moves where the project’s .clangd directory is created.
Maybe that’s good then, but there is the global cache in the home directory which can get wrong too I guess (for example, because of changing macros like __STDC_LIMIT_MACROS).

Ha, this is an interesting case - the included file has changed location, even though the content of the including file hasn’t. It won’t be reindexed until it changes. Without indexing it we can’t tell this case from others where the header is unreadable for some reason, but reindexing it every time would be bad in those cases.

Very narrowly, this bug can be fixed by including the llvm version number in the index version, so we discard the index after upgrading clangd. That seems like good hygiene anyway. But there’s a more general case, and I don’t know a clean way to fix it.

Yeah, but this has a bunch of other effects. If you want a workaround, just delete /.clangd and ~/.clangd. But this is a real bug (harmless-ish, but I think you’ll see those errors on each startup as long as nothing else triggers reindexing).

This is what’s I’ve done, removing both folders.

I think a complete index per project per build configuration should be ok (+ adding clangd version). And maybe a hash of the command line used to index a source file. So if a macro changes, or an include path changes, or even the toolchain changes, this will be detected.

This is maybe actually what’s done, when opening a file, there is the full compilation command line inside.

Yeah, command-line (or its hash, I forget) is there and can be used to trigger a rebuild of the index. The problem is that inttype.h is a builtin header, and the include path for the builtin headers does not appear on the commandline.

But this can also happen if you have a file built with -I A -I B containing #include "foo.h", and foo.h is moved from A/ to B/.

(Incidentally, the hash in the filename is just the hash of the path of the source file. Inside the index file there is a hash of the contents of the source file, this isn’t part of the index shard file name because if it changes we want to overwrite that index shard.)

I guess that’s ok to have the hash of the source only, so the index isn’t clobbered on every #include changes.

If an index is made to be specific to a <source file, build configuration> combination by having the source file name and hash in the index file name, and the build configuration in the location of the index itself (ie: in the build directory) then I think this is mostly ok.

When clangd version is changed or when a included file changes or is moved, then clangd will detect that (as it logs that) and should reparse the file, and maybe check all other files too.

I don’t know much about how clangd works, but I guess the whole index is checked at startup ?
This seems not as easy to do for the running case, but maybe it doesn’t need to notice it right when a file is moved, but only for opened files that are modified.