We're going to pin this as we are getting lots of questions about it at the moment.
Tencent Gaming Buddy has a number of different Rendering settings and whilst any of the DirectX or OpenGL options should be compatible with FBX, we've found that some of them can give inconsistent results so the default option may not always be the best to use.
For example, on Windows 7, Tencent Gaming Buddy doesn't seem to actually use DirectXl, even when one of those options is selected. If it isn't using DirectX, FBX cannot detect it to let you record it.
We've found that OpenGL+ is the most reliable, so if you're not seeing the overlay, we recommend that you give that one a try.