It was difficult as my trace was inconclusive, it showed the main thread simply continuing execution to some invalid address. After working on it for days, I found out that my trace was being b0rked by OllyExt, a plugin for Ollydbg 2.01, so I disabled that and found my problem.
The offending code was a huge jump table, I guess at some point execution jumped to that location, but the jump table itself had unfilled addresses to pointers to the FMOD Sound System.
From then on because of my poor pointer arithmetic skills and almost no knowledge of the PE format, it was a whole day before I wrote a small tool to identify the pointers in the original exe to those in the FMOD Sound system DLL(s). This was further delayed by some unknown bug where the index of an exported function did not match the index of the name array, if it was a C unmangled exported function. All C++ mangled/decorated functions's indexes into the name array were correct.
Then I generated my own table of GetProcAddress-es to include in my stub. After that execution of the exe continued, till I stumbled on another problematic area, of some address not being correct, one that is filled by the SecuROM VM before OEP is reached.
So thats going to take some more days.
From then on because of my poor pointer arithmetic skills and almost no knowledge of the PE format, it was a whole day before I wrote a small tool to identify the pointers in the original exe to those in the FMOD Sound system DLL(s). This was further delayed by some unknown bug where the index of an exported function did not match the index of the name array, if it was a C unmangled exported function. All C++ mangled/decorated functions's indexes into the name array were correct.
Then I generated my own table of GetProcAddress-es to include in my stub. After that execution of the exe continued, till I stumbled on another problematic area, of some address not being correct, one that is filled by the SecuROM VM before OEP is reached.
So thats going to take some more days.
No comments:
Post a Comment