Monday, February 15, 2016

Have I found the mysterious anti-debug?

In my previous post I mentioned that running the game under a debugger, would, after a while, force terminate the game.
I speculated either the debugger was being found by an API directly, or indirectly via a timing anti-debug.

I did some experiments. And the evidence points to a timing anti-debug. The time it takes to terminate the game is variable, and it turns out, it only happens if the performance of the game is rather bad. In this case, it was Olly 2's fault. There seems to be some kind of bug in Ollydbg 2.01 whereby all threads of a running application are suspended and resumed constantly, the game runs although with a 30-35% penalty. The timing anti-debug sees this, sees that more ticks are being expended than normal and with careful communication between two threads, it calls NtTerminateProcess by spawning several threads that point to a VM program(only of the thread has a different VM program than the rest)..
In most cases, what Olly is doing is normal behaviour, it's how it's usually done, but not in my case, I've observed olly idling and not doing this suspend/resume thing. The bug seems to disappear if I(at least in my case) I hit a memory breakpoint. Then olly is acting normal, and the game does not terminate, or at least not as fast as before, if the avg grows as time passes, because of small slowdowns, then it will terminate eventually.

I looked at my trace log of one of the obfuscated threads, lo and behold, RDTSC on address 3955DEA9(quick reminder there is no ASLR). The result of RDTSC is stored in EDX:EAX, these values are later used in a loop and are encrypted and stored in a table.
Now that I know what is what, I can better understand the underlying algorithm. One thing is certain, the mystery is solved.

Quick reminder that timing anti-debugs are in my opinion, the most difficult to handle, it isn't as easy returning 0 on GetTickCount.

1 comment:

  1. How to annoy devs: Make the error reporting mechanism concatenate "><img src=randomNicholasCageJpg to a value