Using LibAFL on Windows with LLVM passes
For reasons that are irrelevant, I needed to fuzz a Windows specific codebase recently and ended up neck deep into it. I had source access to the target and wanted to try out the latest and greatest LibAFL with all the good stuff it has to offer.
Since LibAFL is more of a toolbox for building various types of fuzzers, it could already be used in certain ways in Windows. However, I was hoping to use the LLVM passes for coverage, cmplog, etc.
I also got an extra boost in motivation when I saw the open ticket for adding this feature.
Update: the PR has been merged 🎉 !
Changes required to the codebase
For starters, a few changes needed to be made to the codebase of
libafl_cc, one of the projects in the main rust workspace of LibAFL. This projects serves two purposes:
- It implements a clang wrapper such that various arguments are automatically added when compiling your target. If you are used to AFLplusplus, then you are probably used to seeing
afl-clang-fast. In this case,
libafl_ccplays a similar role.
- It builds the associated optimization passes, which at the time of writing this are:
To be fair, the changes to the codebase were not super big and after thinking about it, decided to rebase a bunch of small (sometimes conflicting) commits into a single one. If anything, the best part here was some refactoring to make the codebase (subjectively) more easy to grok.
The biggest change was using
clang-cl instead of
clang for building the optimization passes on Windows. The reason here was to remaing compatible with the existing idea of using
llvm-config to identify libraries, library paths and other flags. While on Unix platforms
llvm-config generates flags for the regular
clang frontend, on Windows
llvm-config.exe generates flags that are compatible with Microsoft's
cl compiler and the
clang-cl compatibility frontend.
At this point, if you are an LLVM on Windows connoisseur, you might be wondering where
llvm-config.exe came from and how optimization passes even compiled. For that, I have to recompile my own LLVM,
clang and while I was at it,
There were mainly one reason to build LLVM myself, namely compiling
llvm-config.exe. Unlike the Unix distributions of LLVM, the one for Windows does not include this binary. It is not fully clear why, but it is considered unstable for regular use. To be fair, I myself noticed some oddities when using it via a shared folder and seeing different outputs in VMs or docker containers versus the host. In any case, by simply building LLVM from source, you get
llvm-config.exe for free.
Another thing I encountered were mentions of an
LLVM_EXPORT_SYMBOLS_FOR_PLUGINS variable in
cmake. I found this bug report which mentions symbols not being exported unless explicitly requested. Apparently these symbols are required for compiling and using plugins on Windows. I tested with and without setting this variable, and things seemed to work just fine in both cases. I am mentioning it here for completeness.
Lastly, I bumped into an issue with the default
link.exe linker. The solution was to compile
lld as well and use that instead.
For reference, the powershell commands I used to build my version of LLVM were
mkdir build; cd build cmake -G "Visual Studio 17 2022" -A x64 ` -DLLVM_ENABLE_PROJECTS="clang;compiler-rt;lld" ` -DLLVM_EXPORT_SYMBOLS_FOR_PLUGINS=ON ` -DLLVM_TARGETS_TO_BUILD=X86 -Thost=x64 ` ../llvm cmake --build . --config Release
If you are following along, remember to also add the bin directory to your path
$env:PATH = "<path to git repo>\build\Release\bin" + ";" + $env:PATH
LLVM runtime library
As a detour, I wanted to write down my notes regarding the default Windows runtime used when building LLVM. While running
cmake, one might notice the specific text
-- Using Debug VC++ CRT: MDd -- Using Release VC++ CRT: MD -- Using MinSizeRel VC++ CRT: MD -- Using RelWithDebInfo VC++ CRT: MD
The four options
RelWithDebInfo refer to the standard options for the
CMAKE_BUILD_TYPE variable. In the above example, the code is compiled with the
The opions for the CRT however refer to the Windows runtime libraries. To (overly)simplify things, there are two main options:
MT for the static library
MD for the dynamic library
msvcrt.lib. An extra
d can be appended to use debug builds of the runtimes. By default, the
MD option is picked up, however if you want to change that using four
With all the above in line, the last step was to use LibAFL itself. I tested things out on StdFuzzer, which is a quick to spin fuzzer, similar to libfuzzer but based on LibAFL. After updating the code to work with the latest version of LibAFL, it was good to go.
The last thing needed here was to tell
rustc to link the code against the static runtime (i.e.
libcmt.lib) instead of the dynamic one (i.e.
msvcrt.lib). This was needed due to some oddities regarding symbols defined in multiple libraries. While goolging for stuff, I found out others had fixed the issue my simply changing the runtime, which also worked for me. To do that, I set up the
RUSTFLAGS environment variable as explained in the docs
$env:RUSTFLAGS='-C target-feature=+crt-static' cargo build --release
There is already a
libafl_cc wrapper in StdFuzzer, which I then used to compile a silly example as
.\StdFuzzer\target\release\libafl_cc.exe ` target.c -o target.exe ` -loleaut32 -lole32 -luserenv ` -O1 -g ` '-Wl,/subsystem:console' ` -fuse-ld=lld
I used the
'-Wl,/subsystem:console' argument to tell the linker what subsystem to generate the final .exe for. This is a
link.exe flag, but
lld understands it as well. An alternative would be to implement the
main() function in the harness.
Also notice the
-fuse-ld=lld argument, which was needed to address an issue with
link.exe. Without the argument,
link.exe would throw an error message saying
fatal error LNK1190: invalid fixup found, type 0x000B.
For those who wish to test things out quickly, there is a
Dockerfile available which sets up a Windows container with
cargo and a few other requirements to compile LLVM and build StdFuzzer.