I was so frustrated with it that I looked for an alternative backend, and here it is: Qt5. In this tutorial, I will go over the method of building OpenCV3 on Mac OS X with Qt5 as HighGUI's backend. For simplicity, I will make use of homebrew.
First, you will need to tap into science:
$ brew tap homebrew/science
Next, install opencv3 with qt5 option:
$ brew install opencv3 --with-qt5
This option will add -D WITH_QT=ON in OpenCV's cmake option, thus linking Qt.
That's it! If you are wondering what are other available options, run
$ brew options opencv3
For more info on the package, run
$ brew info opencv3
That's it! You should now be able to launch HighGUI window via Qt5!