That said, I have not worked on SecuROM 7.30 ever, but I believe the VM has changed since then.
Here is an overview of the VM initialization.
When entering the VM, an argument is pushed to the stack. It's a pointer to a pointer to the VM opcodes. I call this argument a "program".
The dummy call after pushfd is used to get the address of the VM.
In the picture above several things happen. A spinlock is created by the thread which enters the VM and will initialize the context, all other threads, if any, will wait till the first thread has finished the initialization.
Then SecuROM uses the loop x86 construct to loop over all 100 possible VM thread contexts, and finds the first free one. The busy flag 0x66666666 indicates if a thread is busy or not. I should note that SecuROM 7.30 only supported up to 10 threads, SC 8.10 supports 100.
After the first free context is found, SecuROM jumps to the following code which sets the busy flag.
lea edx,[ebx+24]
mov dword ptr ds:[edx],66666666
then the VM context is zeroed out, but care is taken not to zero out the busy flag. Afterwards the lock is removed and the jump "je short 38D702FE" takes us to the last step of the initialization.
sub dword ptr ss:[esp],7 subtracts 7 bytes from the VM function address which I mentioned above that it is pushed to the stack with a dummy call so it ends up as 38D70280 in this exe.
This part fills the VM context struct. I've taken the liberty of adding captions next to the instructions which are self-explanatory.
Next is this obfuscated code.
In a nutshell, it fetches the delta to the pointer to the opcodes, adds the VM entry point, and fetches 2 DWORDs(aka 8 bytes).
CPU Disasm
Address Hex dump Command Comments
38D7035A 8B70 04 mov esi,dword ptr ds:[eax+4]
38D7035D 8B00 mov eax,dword ptr ds:[eax]
The first 4 bytes of the opcode is the modifier, the next 4 bytes is the obfuscated address of the handler.
The modifier is used to calculate the next handler address. The one in the VM context is updated with this new one, after some xor and shifts are performed.
The "encrypted" address of the first handler is decrypted with a XOR.
xor esi,48371826It's then copied to eax.
Finally
CPU Disasm
Address Hex dump Command Comments
38D70396 B9 19000000 mov ecx,19
38D7039B 83F1 1D xor ecx,0000001D
38D7039E 01D9 add ecx,ebx
38D703A0 8301 08 add dword ptr ds:[ecx],8 <-- Add 8 bytes to VM EIP
38D703A3 FFE0 jmp eax
The VM EIP(program counter) is incremented by 8 bytes, and we jump to the address of the first handler.
Here comes the juicy part. The jump to first handler goes to this code
Step 1.
First, ebx+20 is updated with the address of the next pseudo handler, for a lack of a better word. And if we follow the jump we end up where the actual first instruction is executed in this particular handler.
Step 2.
and if we follow the jump we end up at what I call the "dispatcher".
Step 3.
The dispatcher adds the VM entry point to the value added in ebx+20 to form the address of the next pseudo handler.
Basically, from what I understood, a single handler which is usually a sequence of instructions has been split into several small pseudo handlers each reached in three steps. In my opinion this is just obfuscation to slow down reverse engineering.
Sometimes in Step 1 there is an additional instruction that moves a value in ebx+400, which is usually used to substract 4 bytes from the stack pointer.
Now, if we follow the each jump and where it leads to, remove all jumps and dispatcher code, the first handler's code is basically this.
mov esi,dword ptr ds:[ebx+4]
add esi,dword ptr ds:[ebx+0C]
add esi,4
push dword ptr ds:[esi]
pop edi
mov dword ptr ds:[ebx+400],4
sub esi,dword ptr ds:[ebx+400]
push dword ptr ds:[esi]
pop esi
mov cl,byte ptr ds:[ebx+10]
push eax
xor eax,esi
xor eax,dword ptr ss:[esp]
add esp,4
shl eax,10
shr eax,18
xor al,2A
add byte ptr ds:[ebx+10],al
mov eax,3BF7C894
push edi
push eax
mov eax,esi
shl eax,18
shr eax,18
ror al,cl
xor al,78
shl eax,2
add eax,ebx
mov edi,eax
pop eax
push eax
pop dword ptr ds:[edi]
pop edi
push edi
pop eax
sub al,cl
xor eax,00B42D00
add dword ptr ds:[ebx+4],8 <-- Update VM EIP with 8 bytes.
In my case, this handler basically calculated the address of another handler.
This isn't an exhaustive analysis of the Securom 8 VM, it's but a scratch of the surface. Furthermore, I have not identified the VM exit procedure.