Tuesday, November 19, 2019

Debug Pybind11 C++ Extension with CLion

OK, so I need to be able to debug C++ part of the code, which is called from Python3 using Pybind11, and I don't want to do it with lldb or gdb, i.e., simple TUI debugger. In fact, I develop C++ extension with CLion extensively, so I want to be able to debug/step within CLion. Here is how to do so.

I'm going to use the Pybind11's cmake-example, since we want to use CMake with CLion.

First, download the repo
$ git clone --recursive https://github.com/pybind/cmake_example.git && cd cmake_example

From now on, I'm going to assume $ROOT is the path for this cmak_example repository.

Next, import the directory with CLion
CLion --> Open --> select $ROOT folder

Add symbols and turn off optimization for debugging by adding the following line to CMakeLists.txt file
cmake_minimum_required(VERSION 2.8.12)
project(cmake_example)
set(CMAKE_CXX_FLAGS "-g -O0")

add_subdirectory(pybind11)
pybind11_add_module(cmake_example src/main.cpp)

Edit Run/Debug configurations for cmake_exmaple as follows:
target: cmake_example
executable: /your/python3/binary
program arguments: tests/test.py
working directory: $ROOT
environment variables: PYTHONPATH=$ROOT/cmake-build

Now, debug with this configuration. You'll probably get version assert error. Let's just comment out that line in tests/test.py.
import cmake_example as m

#assert m.__version__ == '0.0.1'
assert m.add(1, 2) == 3
assert m.subtract(1, 2) == -1

Now, re-run debug with break point at line 4 of src/main.cpp.
CLion should break there!

3 comments:

  1. Hi Unix&Me,
    This post was super helpful, very much appreciated. I'm a captive windows user who runs clion over WSL and SSH to linux backends and I was disappointed in myself to only be able to replicate this setup locally but not remotely - doesn't seem to be an easy way to run a python executable on the remote server and debug the extension via a kick from python that way (e.g. a python script filled with pytest methods calling pybind11 bindings to hit breakpoints in the native code within the local windows clion IDE)? Might be a few too many features layered on top of each other to work, but it would be very slick if possible :-). I would be super grateful for any pointers you might have!
    Thanks

    ReplyDelete
  2. Hi, thanks for your tutorial, I followed your steps, the python test can run and output but can not jump into the checkpoint, do you know why?

    ReplyDelete
    Replies
    1. I managed to find the reason, you need to change the setup.py also:

      cfg = 'Debug' # if self.debug else 'Release'

      comment out the Release configure.

      Delete