New Opcode Discussion

[showeqnewb]

Ok it was stated that it was not a change in cryption but more so just changes in opcode structures...

Now before I actually ask the question take this into mind. I cannot nor do I know how to do this myself, so obviously I am not going to be the one to fix it and who ever does fix it obviously knows a heck of a lot more than me about his subject.

Arent these opcode changes the same type of changes that we used to have a fix for in a few hours?

If so did we lose some people that used to rock in this area?

Mind you, I dont care if the answer is either fix it yourself or stfu or a real answer but it is a question that I have been wondering about.

I was tickled to death the other day when we had a real working version with out memory sniffers and my hopes seriously shot up...

[quackrabbit]

Originally posted by showeqnewb
Arent these opcode changes the same type of changes that we used to have a fix for in a few hours?

If so did we lose some people that used to rock in this area?The answers to the above are yes and yes. Unfortunatly the behind the scenes developers of SEQ have moved on to other things - be them real life or other games, it's hard to tell. I wish them the best, and I really do appreciate their hard work.

Ratt has said that eventually, if and when he gets time, he will try to drum up some interest in fixing SEQ, but atm it seems like people who still play EQ are either a) too busy with other things. b) happy with the other products currently available for EQ or c) like myself unable or worse unwilling to contribute.

[fester]

Having not looked at the changes, I can hazard a guess as why it is not fixed.

In the last state, there was a table inside eqgame.exe containing 4 byte words for every opcode. This table contains either 0xffffffff or a size in bytes.

Click link for a list of the last known sizes (items not listed as 0xffff):
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/seq/showeq/src/decode.cpp?rev=1.18&sortby=date&content-type=text/vnd.viewcvs-markup

If the information ShowEq is using is wrong, then ALL FLAG_COMBINED packets will be corrupted (because we jump to the wrong position) and almost half the packets will be missed. The other half could also be invalidated if their structure was changed.

I have maintained a private ShowEQ since Luclin. I have not examined the changes because my system with the tools (IDAPro, MS VS6.0, etc.) suffered a hard disk failure in late March and I haven't bothered to reinstall that system. I also would like to see non ShowEq tools get into wide spread use. This may make it less interesting to change things that break ShowEQ.

To find this location inside eqgame.exe, disassemble and look for "and e.., 00000FFF" in the file. This will be rare. For example in the last disassemble I had around, this was the list:

Valid:
:00554BFD 25FF0F0000 and eax, 00000FFF
:00554CA4 25FF0F0000 and eax, 00000FFF
:005551BD 81E6FF0F0000 and esi, 00000FFF
:0055529C 25FF0F0000 and eax, 00000FFF

Invalid:
:005B7E65 25FF0F0000 and eax, 00000FFF
:005D86BA 81E1FF0F0000 and ecx, 00000FFF
:005DBB56 81E6FF0F0000 and esi, 00000FFF

Look at the VALID ones and ignore the INVALID ones. How do I know the difference? Well valid ones have code like this following the AND instruction:

:00554BFD 25FF0F0000 and eax, 00000FFF
:00554C02 897E0C mov dword ptr [esi+0C], edi
:00554C05 668906 mov word ptr [esi], ax
:00554C08 8D5E08 lea ebx, dword ptr [esi+08]
:00554C0B 0FB7C0 movzx eax, ax
:00554C0E 8B048574218400 mov eax, dword ptr [4*eax+00842174]

Notice eax is used as a index into 00842174 (a 32 bit word since the index is multiplied by 4.)

All one needs to do to get the table is to dump the data at 00842174 with a memory snooper. This table is build during eqgame.exe startup, so it is not trivial to examine a binary to obtain the table. Look for something like this:

:004F6BBF B9FF030000 mov ecx, 000003FF
:004F6BC4 83C8FF or eax, FFFFFFFF
:004F6BC7 BF74218400 mov edi, 00842174

This is somewhat rare (605 "mov edi, 0.*") and many dups (297 uniq "mov edi, 0.*" references) with many of them low (232 below rsrc (Imagebase = 00400000h + Object04: .rsrc Offset: 0023F000 + Size: 00001000 or 00640000h) leaves you with only 65 remaining targets. Very few (only 42) of the pair (or eax, FFFFFFFF followed by mov edi, 00640000 and up) appear.

Someone could write a program to disassemble, sort for these 42 events, them memory dump (from a running eqgame.exe) those 42 locations looking for what resembles the table (large chunk of mostly 0xffffffff values.) This could be used to automate finding the new opcode sizes after a patch. I am too lazy to do so.

This is the only "hard" part of fixing opcodes. Fixing structures is simple (once you have correct opcodes parsing.)

This post may please the people who complain "no one provides locate opcodes and structures lessons," but I still maintain that classes of this type can not make you the next Mvern. They are all self taught.

(BTW this DOES NOT disagree with Ratt's "Opcode changes ... main problem right now is the packet structure changes." It clarifies what he likely means. The structure of the packets to large degree is dictated by this opcode size table. Incorrect sizes for the "packed" opcodes will mean you will not find any of these very common opcodes.)

Archaeopteryx, opcodes used to be a few hours when you ran ShowEQ and watched unknown opcodes until you located the ones you need. This is no longer possible. You must disassemble and read process memory to be able to get to the point of "watching unknown opcodes to pin them down" now. This does only take a few hours, but instead of being able to be done in parallel by many people (perhaps maybe 50 or more), you are down to the few who know how to program in C (to read process memory) and who know how to read x86 assembly (to know where to look in the first place.)

[user387]

ok....

table is located in memory at 0x0076194C

I got the table, I know where the 15 values are being put into memory.

but how do I create a decode.cpp from there? :D

[fester]

The 15 values are the sizes.
The "offset" is the opcode.

Example:

switch (opcode) {
case 0x0019: // 25
return 0x06; // 6 bytes

This entry will look like this in memory:
0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff
0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff
0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff
0xffffffff 0x0006 ...

The 19th entry is 6 (for 6 bytes).

If you wish, post the table (and/or the offsets of the 15) and I can
craft the decode.cpp sub.

[user387]

this is where the table is created :

=========
:0049517A E87A000000 call 004951F9
:0049517F 6A04 push 004
:00495181 C705881C76001F000000 mov dword[00761C88], 0000001F
:0049518B 5A pop edx
:0049518C C705E01B760017000000 mov dword[00761BE0], 00000017
:00495196 6A06 push 006
:00495198 8915941D7600 mov dword[00761D94], edx ; 004
:0049519E 59 pop ecx
:0049519F C705581D760009000000 mov dword[00761D58], 00000009
:004951A9 6A03 push 003
:004951AB 890D38227600 mov dword[00762238], ecx ; 006
:004951B1 58 pop eax
:004951B2 890D3C227600 mov dword[0076223C], ecx ; 006
:004951B8 6A08 push 008
:004951BA A3D0217600 mov dword[007621D0], eax ; 003
:004951BF A3D4217600 mov dword[007621D4], eax ; 003
:004951C4 58 pop eax
:004951C5 A3B01D7600 mov dword[00761DB0], eax ; 008
:004951CA C705DC1D760012000000 mov dword[00761DDC], 00000012
:004951D4 C705F02376000C000000 mov dword[007623F0], 0000000C
:004951DE A32C1E7600 mov dword[00761E2C], eax ; 008
:004951E3 C705841C760088000000 mov dword[00761C84], 00000088
:004951ED A3581A7600 mov dword[00761A58], eax ; 008
:004951F2 8915241C7600 mov dword[00761C24], edx ; 004
:004951F8 C3 ret

added some comments for the values of the registers.

attached a small memdumper. not working as it should be, but it's a start. I plan to extend it so that it can find the offset by itself oneday :)

message board won't let me upload a file with a .c extension, so it has a .cpp extension now, but it's just c

[fester]

This is the guts of the function in decode.cpp:



case 0x0043: // 67
return 0x08; // 8 bytes
case 0x00a5: // 165
return 0x17; // 23 bytes
case 0x00b6: // 182
return 0x04; // 4 bytes
case 0x00ce: // 206
return 0x88; // 136 bytes
case 0x00cf: // 207
return 0x1f; // 31 bytes
case 0x0103: // 259
return 0x09; // 9 bytes
case 0x0112: // 274
return 0x04; // 4 bytes
case 0x0119: // 281
return 0x08; // 8 bytes
case 0x0124: // 292
return 0x12; // 18 bytes
case 0x0138: // 312
return 0x08; // 8 bytes
case 0x0221: // 545
return 0x03; // 3 bytes
case 0x0222: // 546
return 0x03; // 3 bytes
case 0x023b: // 571
return 0x06; // 6 bytes
case 0x023c: // 572
return 0x06; // 6 bytes
case 0x02a9: // 681
return 0x0c; // 12 bytes

[fester]

I have not recompiled showeq with this code yet. I doubt I will have time to do it until the weekend.

It would be cool if someone could test this out.

[user387]

patched, recompiled, tested...

same problem still :mad:

edit: spoke too soon. have to recompile libeq.a

lemme test again.

gcc -c libeq.cpp
ar r ../libEQ.a libeq.o

should give you the needed library.

edit2: new library. make clean make install....

same problems still :mad:

[user387]

k.

I'm having the feeling this was a step in the right direction...

way less error: size > left messages. only seen a couple so far, all from opcode 0x0924 so there must be something special with that..

decode errrors are still there though. :mad:

[Jynx]

Great post fester.

The problems that you guys are having is that along with the new implicit opcode lengths, they also changed the opcode flags. There is a possibility of 5 flags now instead of the previous 4.

FLAG_COMP is now 0x0800
FLAG_CRYPTO is now 0x2000

I have not had enough time to look into what the new flag could be.

This changes fester's search slightly in the previous posts. Instad of looking for "and eax, 00000FFF", you should look for "and eax, 000007FF"

[user387]

v0.2 :)

works after the 14-5-2003 patch.

edit: Blargh! removed the cut&paste.
get the zip file from the other post.

[Amadeus]

Before the questions start, I need to point out that the message board strips out one of \\\s for some reason...so:



for (pCurChar = (pe32.szExeFile + strlen (pe32.szExeFile));
*pCurChar != '\\' && pCurChar != pe32.szExeFile - 1;
--pCurChar)


should be (at least to compile with VisualStudio anyway)


for (pCurChar = (pe32.szExeFile + strlen (pe32.szExeFile));
*pCurChar != '\\\' && pCurChar != pe32.szExeFile - 1;
--pCurChar)



I would also suggest commenting out the following lines just to have a clean decode.cpp:



printf ("found eqgame - pid = %u\n\n", pe32.th32ProcessID);



printf ("scanning for eqgame.exe\n");

[Amadeus]

Ok!! Now we're getting somewhere.....we have great instructions on how to essentually get libeq.a and decode.cpp working in the future (at least if things don't go toooooo crazy).

Now, who is going to give us the tidbit on how opcodes.h and the new structures (everquest.h) is done ;) I've gotten fairly good at determining structures with MQ (ie, eqgame in memory), but this is a whole new ballgame with encryption and packets.... Just need a good example I think :)

[Amadeus]

bah..sorry for spamming...but something else....

This might be a compiler issue; however, with VisualStudio.NET I had to change the following line:



printf ("\t\tcase 0x%4x: // %d\n\t\t\treturn 0x%2x; // %d\n", i, i, data[i], data[i]);


to:


printf ("\t\tcase 0x%x: // %d\n\t\t\treturn 0x%x; // %d\n", i, i, data[i], data[i]);


for the output to be correct.

[Ratt]

I can't do anything about the stripping of the \\\\\\'s because that's a PHP thing...

However, I can add .c to the recognized extensions. However, it might be better to zip up the file and attach it.

Adding .c as valid extension right now.

[user387]

ok, so how do I get it to actually attach the file? :confused:

I click on browse, add the correct file, and click on submit, and no file shows up in my posting, like in this post.

[user387]

new table is located at 0x0076080C

if only I could log in to the EQ servers, and see some packets :)

I updated my other posting with a new version btw.
some bugs fixed, I hope.

[Ratt]

It sure isn't showing up, is it...

I'll have to look into it.

For now, you may want to zip the file.

[user387]

I was testing with a zip file.. doesn't work either....

should be attached to this message again..

edit: haha! doesn't show up, but if I try to attach it again, I get this:

You have already attached this image to a post. Please refrain from uploading duplicate images. If you need to refer to this image it can be found in this thread: New Opcode Discussion

[tamasine]

so it's not just me!
i had exactly that problem on the bazaartracker thread. attachment wasn't visible, but on editing to try and attach again.. it claimed it was there.

i gave up and cut&pasted.

[Ratt]

The problem has been resolved. Attachments should work fine now.

[user387]

yay Ratt

so, another try I guess :)

edit: It worked! It worked!

[user387]

v 0.2 still works..

change addr to 0x007607FC and it works with the latest patch (15-5)

[Amadeus]

Just, for prosperity, I thought I'd attach a Visual Studio .NET project files for this great little resource. Just stick this in a directory, and let VS do the walking :)