Debug Pybind11 C++ Extension with CLion

OK, so I need to be able to debug C++ part of the code, which is called from Python3 using Pybind11, and I don't want to do it with lldb or gdb, i.e., simple TUI debugger. In fact, I develop C++ extension with CLion extensively, so I want to be able to debug/step within CLion. Here is how to do so.

I'm going to use the Pybind11's cmake-example, since we want to use CMake with CLion.

First, download the repo
$ git clone --recursive https://github.com/pybind/cmake_example.git && cd cmake_example

From now on, I'm going to assume $ROOT is the path for this cmak_example repository.

Next, import the directory with CLion
CLion --> Open --> select $ROOT folder

Add symbols and turn off optimization for debugging by adding the following line to CMakeLists.txt file
cmake_minimum_required(VERSION 2.8.12)
project(cmake_example)
set(CMAKE_CXX_FLAGS "-g -O0")

add_subdirectory(pybind11)
pybind11_add_module(cmake_example src/main.cpp)

Edit Run/Debug configurations for cmake_exmaple as follows:

target: cmake_example

executable: /your/python3/binary

program arguments: tests/test.py

working directory: $ROOT

environment variables: PYTHONPATH=$ROOT/cmake-build

Now, debug with this configuration. You'll probably get version assert error. Let's just comment out that line in tests/test.py.

import cmake_example as m

#assert m.__version__ == '0.0.1'

assert m.add(1, 2) == 3

assert m.subtract(1, 2) == -1

Now, re-run debug with break point at line 4 of src/main.cpp.

CLion should break there!

WTF? Fix to "string.h not found" in macOS

OK, I love macOS but I sometimes hate the hassle when it comes to Xcode and its toolchains. Randomly I get errors like "string.h" not found... WTF?

Here is the fix. You probably have the Xcode command line tools installed; Run
open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.1X.pkg

where you want to put the right version yourself. (X = 4 for Mojave, 5 for Catalina, etc)

If you don't have the package, try deleting and re-installing the tools and re-try

sudo rm -rf /Library/Developer/CommandLineTools

xcode-select --install

If that does not work, try
sudo xcode-select -s /Applications/Xcode.app/Contents/Developer

Hope this fixes!

*** EDIT ***

Sometimes with CLion, you may get similar error. For that, the fix is rather simple. Go to Tools --> CMake --> Reset Cache and Reload Project. That's it!

Load Makefile Projects on CLion

If you are like me, there is no better C/C++ IDE than Jetbrain's CLion. I absolutely love it and refuse to use any other IDE.

There is one problem, however. CLion only supports CMake projects. There is a way to import Makefile projects using compiledb, but it was not trivial for projects that heavily relies on GNU toolchains.

In this post, I will go over how to import Makefile projects, such as openfst, that cannot be imported properly following Jetbrain's tutorial and tutorial2. There is one trick; when running make, add -w option, which prints entering/leaving directory. Without this option, the generated compile commands will not locate the true path.

That is, run
$ compiledb make -w

By the way, never use multithreading option -jN here because it will mess up the order of files and compiledb will not be able to reproduce all the make commands.

That's it! Happy hacking.

Compile GNU Coreutils from Scratch on macOS Mojave

Here, I will discuss how to compile GNU coreutils from scratch. You have two options. I recommend Option 2 below.

Option 1:
First, download the source code from its repo. I will use v8.31
$ git clone https://github.com/coreutils/coreutils.git -b v8.31
$ cd coreutils

Next, clone git submodule
$ git submodule update --init

Next, bootstrap
$ ./bootstrap
./bootstrap: line 470: autopoint: command not found
./bootstrap: Error: 'autopoint' not found
./bootstrap: line 470: gettext: command not found
./bootstrap: Error: 'gettext' not found
./bootstrap: Error: 'makeinfo' version == 4.8 is too old
./bootstrap:        'makeinfo' version >= 6.1 is required

./bootstrap: See README-prereq for how to get the prerequisite programs

Well, I need to get prerequisite programs first. Install gettext using brew
$ brew install gettext && brew link gettext --force

Let's try again
$ ./bootstrap 
./bootstrap: Error: 'makeinfo' version == 4.8 is too old
./bootstrap:        'makeinfo' version >= 6.1 is required

./bootstrap: See README-prereq for how to get the prerequisite programs

To check the version, run
$ makeinfo --version
makeinfo (GNU texinfo) 4.8

Copyright (C) 2004 Free Software Foundation, Inc.
There is NO warranty.  You may redistribute this software
under the terms of the GNU General Public License.
For more information about these matters, see the files named COPYING.

So, I do need to update makeinfo. Note that this is the same as texi2any from GNU texinfo package.
$ pushd && cd ~/Downloads
$ wget http://ftp.gnu.org/gnu/texinfo/texinfo-6.6.tar.gz
$ cd texinfo-6.6
$ ./configure
$ make -j4
$ sudo make install
$ makeinfo --version
texi2any (GNU texinfo) 6.6

Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Now, we are ready to bootstrap again
$ popd && ./bootstrap

Option 2:
Download the distribution source code
$ wget https://ftp.gnu.org/gnu/coreutils/coreutils-8.31.tar.xz
$ tar xfj coreutils-8.31.tar.xz && cd coreutils-8.31

------------------------------------------------------------------
The easy parts are left.
$ ./configure

Finally, we should be able to build it
$ make -j4
$ sudo make install

Happy hacking!

Copy to Clipboard in Terminal

Install xclip
$ sudo apt install xclip

Copy to clipboard
$ cat some_file.txt | xclip -selection clipboard

Paste to anywhere!

N-Gram ARPA Model

This post is to summarize how the probability is calculated from ARPA model.

Consider test.arpa and the following sequences of words:
look beyond
more looking on

Let's consider look beyond first. The log10 probability of seeing beyond conditioned upon look, i.e., log10(P(beyond | look)) = -0.2922095. This is directly from the test.arpa file, line 78.

What is, then, the probability of seeing look beyond? Well, this is by the chain rule of conditional probabilities

log10(P(look beyond))
= log10(P(look) * P(beyond | look)) 
= log10(P(look)) + log10(P(beyond | look)) 
= -1.687872 + -0.2922095 = -1.980081558227539, 

which can be verified with python code

import kenlm
model = kenlm.LanguageModel('test.arpa')
print(model.score('look beyond', eos=False, bos=False)


Let's try the next sequence more looking on. Let us start with the chain rule

log10(P(more looking on))
= log10(P(more)) + log10(P(looking | more)) + log10(P(on | more looking))

The first term on the RHS is easy: log10(P(more)) = -1.206319 from line 34

The second term is a bit tricky, because we cannot find the bi-gram more looking from the model. Hence, we use the following formula:
P(looking | more) = P(looking) * BW(more)
where log10(P(looking)) = -1.285941 from line 33, and log10(BW(more)) = -0.544068 is the back-off weight, which can be read off from line 34.

Lastly, the third term is again not present in the model, so we reduce it to
P(on | more looking) = P(on | looking) * BW(looking | more)
where the first term is -0.4638903 from line 80, and the second term is assumed to be 1, because the bigram more looking does not exist in the model

Thus, we get log10(P(more looking on)) = -(1.206319 + 1.285941 + 0.544068 + 0.4638903) = -3.5

For more details, refer to this document. I also find this answer very helpful.

Remote Debugging with Eclipse

I am not familiar with Eclipse, as I prefer to use CLion. However, for projects that do not use CMake build system, I will have to use Eclipse.

In this post, I will discuss how to remote-debug with Eclipse. The setup is as follows:

target (local): running the application from, say terminal
host (Eclipse): debug the program as it is running

First, open up the project with Eclipse. Make sure that Eclipse can build the project.

Next, setup gdbserver on the target:
$ gdbserver :7777 EXECUTABLE ARG1 ARG2 ...

Here, 7777 is the port we will use for remote-debugging, EXECUTABLE is the binary file we are going to debug as it is running, and ARG1, ARG2, ... are appropriate arguments for this program.

Next, we setup Eclipse debugging.
From the menu, select Run --> Debug Configurations... --> C/C++ Remote Application (double click) --> Using GDB (DSF) Auto Remote Debugging Launcher (Select other) --> GDB (DSF) Manual Remote Debugging Launcher --> OK. Basically, we have selected "manual" remote debugging configuration here.

Make sure Project and C/C++ Application fields are properly filled, i.e., you should be able to select the project from the drop down menu if the project import/build is successful, and choose the EXECUTABLE for C/C++ Application.

In the Debugger tab --> Connection tab, change Port Number to 7777.

Finally, click Debug button. You now should be able to remote debug with Eclipse.

Happy hacking!

Debugging Python Module Built with Pybind11

Pybind11 is a wonderful library to build native c++ functions callable within Python. Let's see how we can debug a module built with Pybind11.

For this project, we will use the very simple Pybind11 example module. Let's get started.

Clone the repository
$ git clone https://github.com/pybind/python_example.git
$ cd python_example

Install the package
$ python setup.py install

Run gdb on Python
$ gdb -q python

Break into the desired function, such as
(gdb) b add
(gdb) b src/main.cpp:4

Run the command
(gdb) r tests/test.py

That is it!

Network Visualization with Intermediate Layer Shapes

Network visualization is very important. Looking at the network graph is much easier than reading through the network description. In this post, I will discuss how to create visual graph of a (Pytorch) network using Netron and include intermediate layer shapes as well.

Netron is an excellent tool for network visualization. At the moment, however, Netron does not support Pytorch natively (experimental feature but not stable). The best thing is to convert Pytorch model to ONNX and then use Netron to graph it.

The following is an example code that graphs ResNet50.
import torch
from torchvision import models
import onnx
from onnx import shape_inference

DEVICE = 'cuda:1'
PATH = 'resnet.onnx'
model = models.resnet50().to(DEVICE)
model.eval()
dummy_input = torch.randn(1,3,224,224).to(DEVICE)
torch.onnx.export(model, dummy_input, PATH, verbose=False)
onnx.save(shape_inference.infer_shapes(onnx.load(PATH)), PATH) # this is required for displaying intermediate shapes



To graph it, simply download Netron and run
$ netron resnet.onnx

That's it!

Resolving Bluetooth Issues with Ubuntu 18.04 LTS

So there are a few things that I do not like about Linux in general; quite a lot of things do not work automatically. One of the problems I have faced is that my Microsoft Designer mouse won't work with Ubuntu 18.04 LTS. After some research, I found this solution suggested by Newbuntie to work well, so I'm going to post this here for any other people going through the same trouble.

Basically, enter the following commands and you are good to go.
$ sudo add-apt-repository ppa:bluetooth/bluez
$ sudo apt install bluez

Enjoy!