Looking for ShowEQ update for Project 1999 after v33 6/25 patch

[amiraldayuk]

After a small update, showEQ is no longer tracking correctly in Project 1999. I doubt that anyone is going to pursue an updated seq client so I'd like to see if I can support a fix. But first, two questions:

1. Up until the patch this week, I had been using the old showeq-5.2.2.0 in an ancient version of Red Hat that used all of the old, required dependencies. I would try one or more of the subsequent versions to see if they worked but they are not available in the seq code archive. Is it possible that one of those may work or did the recent updates create an "alternate reality" that archived versions won't address without doing additional surgery on opcodes?

2. Is it preferable to find the updated opcodes myself and manually patch my current 5.2.2.0? I saw a HOWTO in one of the forums and it looked doable but I don't want to waste the time identifying the opcode changes and updating the code if there is an easier solution.

// A

[BlueAdept]

I added 5.2.3, 5.2.4, 5.2.5 and 5.3.0 (in the repo there is no 5.2.6-9) to the file section. I hope it helps.

[BlueAdept]

If you get it working, please post how you did it.

[amiraldayuk]

Thanks for the archive update, Blue. Unfortunately, none of the 5.2/5.3 archives worked so the update did indeed create an "alternate reality" that can only be resolved by a new branch of opcode/struct fixes. Currently, maps will still properly render for the current zone but SEQ doesn't recognize or render any of the skittles. I suspect that the update changes were minimal (unlike the extensive changes that Sony dropped on big patches back in the day).

Since I didn't generate any traffic logs when SEQ was working in order to establish a baseline for identifying changes, I'll have to use a hex editor to dig through memory for the opcode tables that Fee, Baelang, ksmith, and the others were talking about. I'd like to use the tools that they used years ago to help expedite this process but I suspect that they are out of date and won't help. Starting from scratch is going to suck but I'm kind of looking forward to the challenge. I will post progress updates.

// A

[BlueAdept]

Great. If need be, I can start a branch for 1999. Let me know if you need anything. I might be able to help.

[euler]

fwiw, my suspicion is this change is due to the removal of the con color from the UI. The client likely relied on the mob's lvl or similar, which was changed/removed in order to prevent the client from displaying the color. That'd be the first place I'd check. Haven't looked that deep into the network protocol yet, so treat these thoughts accordingly.

Do protocol changes required an updated dsetup.dll? If so, another approach may be to reverse engineer the dsetup.dll changes in the last patch.

[BlueAdept]

Con colors are based off of your level vs mob level. It used to be a fixed scale, but the blue/lt blue was converted to a sliding scale at some point.

If you are seeing all gray spawns, I would first check to make sure that SEQ is seeing your level and the mobs level correctly. SEQ should automatically color them. If the mobs are showing the right level, but your toon is not, you can set your level (in one of the menus) and it should color the mobs accordingly.

In the debug window, are you getting any struct mismatches or wrong op code warnings?

[seqrox]

Thanks to everyone working on this, I just got seq going again and a week later the patch broke it ;) Most of the NPC types show up as level 83 mermaids. Also seems to be a lot of garbage for pathing directions and the names in the NPC list are garbled. I'm happy to provide any assistance I can

[BlueAdept]

What kind of errors are you seeing in the console?

[amiraldayuk]

I also suspected that the removal of con color may have been a contributing factor but it isn’t something that is easily fixed by manually updating the character level in seq. Not only are the NPC and other player attributes showing up as random garbage, the coordinates are showing up as random locations, some way beyond the constraints of the map, which ruins map rendering. Leaving seq running for a while eventually results in segfault.

For troubleshooting, the console window doesn’t give much that is likely to help in identifying opcodes and structs that are off. The following is one reoccurring series of warnings that is given every time a zone change occurs:

Warning: L Line 14 in map '/usr/local/share/showeq/maps/oggok.map' has 44 points as opposed to the 45 points it specified!
Warning: L Line 42 in map '/usr/local/share/showeq/maps/oggok.map' has 11 points as opposed to the 7 points it specified!
Warning: L Line 44 in map '/usr/local/share/showeq/maps/oggok.map' has 11 points as opposed to the 7 points it specified!
Warning: L Line 59 in map '/usr/local/share/showeq/maps/oggok.map' has 17 points as opposed to the 16 points it specified!
Warning: L Line 62 in map '/usr/local/share/showeq/maps/oggok.map' has 17 points as opposed to the 16 points it specified!

I also turned on all logging and debug dump options but I’m not quite sure what to do with the data that it generates in ~/.showeq. There is quite a bit of this but I am not sure whether it is significant to the problem:

Jul 04 2014 13:49:47:343 [66.55.145.2:7042->client:57631] [Size: 77]
[OPCode: 0x300] [Flags: 5a] [CRC ok]
000 | 00 03 5a 78 5e 93 38 2d 22 c0 c5 60 c3 c9 d8 f1 | ..Zx^.8-"..`....
016 | d9 fc e1 ff ff cf 19 72 ee 1f ff c7 f0 e0 c9 6e | .......r.......n
032 | 01 06 4e 96 5f e7 ad 9d 99 18 18 40 58 9e 49 e2 | [email protected].
048 | b4 c8 3c 6e 06 17 4f f9 07 5f 96 3f d0 62 30 f8 | ..<n..O.._.?.b0.
064 | bf d5 7e 13 58 25 00 6e 15 1b 4e 09 26 | ..~.X%.n..N.&
Jul 04 2014 13:49:47:443 [66.55.145.2:7042->client:57631] [Size: 27]
[OPCode: 0x14cb] [Flags: a5] [CRC ok]
000 | cb a5 14 9c 08 00 00 18 00 a3 05 a0 e0 80 ff df | ................
016 | 00 b5 3f ce fe 00 e0 e4 bb a0 84 | ..?........

In reading the HOWTOs on addressing patch day problems, the reoccurring methods involved finding the opcodes and structs in memory and making sure that the corresponding values in everquest.h were consistent. That may have been an oversimplification of the solution but the forum discussions unfortunately assumed a significant level of familiarity with the code.

So in the spirit of giving a man a fishing pole instead of a fish, my questions would be:


Where would be the best source of log/dump/console data for identifying opcodes/structs that need to be fixed? From the development forum, it isn’t clear whether it is too late to use any of that information because seq isn’t working, as opposed to pulling logs to establish a baseline when seq IS working.
How much of the problem involves pulling opcodes from memory? I can decompile .exe’s and hex dump memory but the forum discussion isn’t clear about the relevancy. Some of the forum solutions discussed don’t involve using a hex editor at all.


Please advise?

Thanks - A

[amiraldayuk]

I'm not sure if I am barking up the wrong tree but I applied some of the techniques that others have been discussing and found a couple of inconsistencies that I'm not sure are relevant. I generated a zone log and then compared the data with structs that are currently in the 5.2.2.0 everquest.h. All but two of the structs that I have found so far are consistent as far as size but there were two exceptions:

First exception: In everquest.h, ServerZoneEntryStruct has a length of 383 octets but the struct was logged as size 385:
(from everquest.h)
/*
** Server Zone Entry struct
** Length: 383 Octets
** OpCode: ZoneEntryCode (when direction == server)
*
* This is just a spawnStruct for the player
*/
struct ServerZoneEntryStruct : public spawnStruct
{
};

(from zone.log)
Jul 04 2014 15:13:20:033 [Decoded] [Server->Client] [Size: 385]
[OPCode: 0x7213]
[Name: OP_ZoneEntry][Updated: 10/27/05][Type: ServerZoneEntryStruct (385) ==]
000 | 00 00 00 00 00 00 00 4d 53 3c 20 22 45 59 39 5a | .......MS< "EY9Z
016 | 3c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | <...............
032 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
048 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
064 | 00 00 00 00 00 00 00 82 00 00 00 00 00 00 00 00 | ................
080 | 00 00 00 00 00 00 4e 29 00 00 00 00 00 00 00 00 | ......N)........
096 | bb 52 ce bb 46 00 2f 00 00 00 00 cd 34 00 00 00 | .R..F./.....4...
112 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
128 | 00 00 00 00 00 00 00 00 00 00 00 49 00 00 00 00 | ...........I....
144 | 00 00 00 00 00 00 00 4b 00 00 00 00 00 00 00 00 | .......K........
160 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
176 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
192 | 00 00 00 00 00 4f 00 00 00 53 00 00 00 31 00 00 | .....O...S...1..
208 | 00 4a 00 00 00 39 00 00 00 4f 00 00 00 53 00 00 | .J...9...O...S..
224 | 00 23 00 00 00 9e 00 00 00 0b 79 33 70 00 da 49 | .#........y3p..I
240 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
256 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
272 | 00 00 00 00 b0 b2 c9 b7 00 00 00 00 40 00 00 00 | ............@...
288 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
304 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
320 | 00 00 00 00 d0 aa d6 73 00 00 00 5b 00 00 4b 32 | .......s...[..K2
336 | 00 00 00 b7 37 55 00 00 00 00 00 00 00 00 00 ae | ....7U..........
352 | 00 00 00 cc 00 00 00 b7 00 00 00 c7 00 00 00 b2 | ................
368 | 00 00 00 ae 00 00 00 cc 00 00 00 b7 00 00 00 c7 | ................
384 | 00

Second exception: In everquest.h, zonePointsStruct has length of 24 octets but is logged with 52:
(from everquest.h)
/*
** ZonePoint
** Length: 24 Octets
** Sent as part of zonePointsStruct
*/

struct zonePointStruct
{
/*0000*/ uint32_t zoneTrigger;
/*0004*/ float y;
/*0008*/ float x;
/*0012*/ float z;
/*0016*/ float heading;
/*0020*/ uint16_t zoneId;
/*0022*/ uint16_t zoneInstance;
/*0024*/
};

(from zone.log)
Jul 04 2014 15:13:31:448 [Decoded] [Server->Client] [Size: 52]
[OPCode: 0x3eba]
[Name: OP_SendZonePoints][Updated: 10/27/05][Type: zonePointsStruct (28) nc]
000 | 01 00 00 00 01 00 00 00 33 f3 cd 44 cd ac 4a 44 | ........3..D..JD
016 | 00 00 74 42 00 00 02 43 2f 00 00 00 00 00 00 00 | ..tB...C/.......
032 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
048 | 00 00 00 00

Is this worth pursuing? It seems on track with the forum guidance regarding inconsistencies in struct sizes. I should also note that the console window didn't produce any useful error messages about inconsistent opcodes or struct sizes during these captures.

Also, there are many entries in the log file of which I'm not sure of the significance. What do these mean?
Examples:
(opcode but zero size)
Jul 04 2014 15:13:15:244 [Decoded] [Server->Client] [Size: 0]
[OPCode: 0x3cdc]

(opcode and size but no label)
Jul 04 2014 15:13:18:256 [Decoded] [Server->Client] [Size: 768]
[OPCode: 0x1234]
000 | bf 00 00 00 03 00 03 00 09 00 09 00 12 00 12 00 | ................
016 | 13 00 13 00 21 00 77 21 39 00 5e 21 3d 00 3d 00 | ....!.w!9.^!=.=.
032 | 3f 00 3f 00 40 00 40 00 48 00 48 00 50 00 50 00 | ?.?.@[email protected].
.... to 768

(opcode and name but no size)
Jul 04 2014 15:13:20:268 [Decoded] [Client->Server] [Size: 0]
[OPCode: 0x7ac5]
[Name: OP_ReqNewZone][Updated: 10/27/05]

(a gigantic one that I'm not sure what to make of - seems to contain opcode information later on)
Jul 04 2014 15:13:31:448 [Decoded] [Server->Client] [Size: 4294967295]
[OPCode: 0000]
000 | 00 00 fe d5 00 00 00 00 00 00 ff ff ff ff 00 00 | ................
016 | 00 00 62 00 09 00 e6 47 0f 00 00 00 00 00 00 00 | ..b....G........
032 | 00 00 00 00 00 fd 01 00 00 31 00 00 00 00 00 00 | .........1......
048 | 00 94 11 00 7f 00 00 00 00 66 66 6c 42 00 80 c1 | .........fflB...
064 | 43 00 00 58 43 49 54 36 33 5f 41 43 54 4f 52 44 | C..XCIT63_ACTORD
080 | 45 46 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | EF..............
096 | 00 00 00 00 00 fe d5 00 00 00 00 00 00 ff ff ff | ................
....

Thoughts?

// A

[Someone]

They are encrypting spawn info, dont think this is something they ever did on live, so you're going to have to get much deeper, I have my doubts its easily fixable.

[BlueAdept]

On live, I believe they encrypted everything (spawns were all unknown and user was level 1). I believe it used to be an 8 bit encryption then they changed it to 32 bit (might have been 64 bit) but for some reason they got rid of it and changed it to an xor. For a long time, there was a libeq.a file, it had all the decryption in it. I believe it is in the libeq.cpp in the versions used by 1999.

I was never very good at the code, I helped occasionally. Structvis used to help me quite a bit. (it is still somewhere on the forums). You will have to take my posts with a grain of salt. It has been MANY years since I really played and even longer since I had done anything with the code.

[purple]

Project 1999 uses the Titanium client? So they can really only do what that client allows them plus whatever they inject into it. I guess it wouldn't be that hard to inject actual encryption into the client, but nothing in what you cut and pasted above screams "This is encrypted" to me. That last packet with the huge payload is an issue, but it could be a seq bug. If you are serious about making it work and they have been serious about stopping network sniffing, your biggest asset will be someone who can disassemble the client and see what it is doing differently. It's a really difficult problem to protect the network stream when the client is by default compromised.

[amiraldayuk]

Hey, purple! Nice to see a legend visiting this new corner!

They were fairly specific about the changes made on the Project 1999 site, none of which mentioned encryption (although I wouldn't be surprised if they excluded encryption from the notes if they had implemented it). The update was in a zip file that we had to unpack into the P1999 directory rather than the automated update that classic EQ used to do. I checked the EQ client (eqgame.exe) and it was identical to before the patch (# md5sum eqgame.exe = a9de1b8cc5c451b32084656fcacf1103) so unless another file is responsible for the change, I suspect that I'll have to check out what it's doing in RAM to identify the changes that need to be addressed.

Code

Haynar: Changed light blue con colors to green.
Haynar: Removed the con color from the target window.
Haynar: Charmed pets who have grids will now guard properly.
Haynar: Mask of Hunter/etc. will no longer fade on zoning.
Haynar: The command for pets to back off, will no longer cause them to ghost if they are moving.
Haynar: Mob movement fixes while pathing between waypoints.
Haynar: Fixes for AC being low.
Haynar: Fear fixes while in water/liquid.

Content

Nilbog: erudnext skiff should arrive at ship from Qeynos sooner, and allow more time for boarding.
Nilbog: Two random npcs in unrest will no longer terrorize the upper levels.
Nilbog: Woven Widow Silk no longer drops
Nilbog: Natasha Whitewater in lakerathe no longer immediately despawns after turn in, but shortly after.
Nilbog: 'Air Tight Box' quest implemented.
Nilbog: Captain Tillin no longer rewards as much experience per gnoll fang.
Nilbog: Highkeep: Captain Boshinko loot updated, spawn time decreased. Chopin and Blayle spawn time decreased.
Nilbog: Prices on Fuligan Soulstone of Innoruuk, Cloudy Stone of Veeshan, Jade Inlaid Coffin, and Tiny Jade Inlaid Coffins have increased.
Nilbog: Skulking Brutling chopper drop %s redone.
Nilbog: A few npcs no longer reside on the roof. Please bug report further oddities with this.
Nilbog: Phin Esrinap's 'Headband' quests are now multiquestable.
Nilbog: Merchant factions in neriakc repaired. Some were erroneously assisting.
Nilbog: Guards in neriaka,neriakb, and neriakc: green aggro applied, increased assist range, and aggro/death messages.
Nilbog: Najena: many npcs now sit when inactive. Added aggro/death messages.
Nilbog: Guards of Neriak Third Gate no longer assist DarkBargainers.
Nilbog: Vials of Swirling Smoke are stocked in limited supply on npc spawn.
Nilbog: Klok Scaleroot now spawns in cabwest. Hooray for iksar alchemy.
Nilbog: lesser spirits for the shaman epic quest require indifferent or better faction. They no longer immediately depop upon giving a gem, but shortly after.
Nilbog: npcs Emissary Glib and Bubbrip are on faction 'OggokResidents'
Nilbog: Wu the Enlightened no longer requests an item he doesn't need.
Nilbog: Avatars spawned during the summoning quests in Paineel now depop after 10 minutes.
Nilbog: Lord Soptyvr always drops his Lens. He also now procs Beholder Dispel.
Nilbog: common evil eyes and Lords Sviir and Syrkl now proc darkness.
Alunova: Reyia Beslin and friends in West Freeport will now return to their spawn point correctly.
Alunova: More FTE shouts will now be zone wide. If I have missed any please post a bug report.
Alunova: A snow orc trooper will no longer roam the Everfrost newbie area.
Alunova: Forsaken Revenants in Plane of Hate will no longer cast charm.
Alunova: Trash mobs in Akanon basement area will no longer walk through walls.
Alunova: Captain Bosec and Guard Bolton in Highkeep will no longer walk through walls and ceilings.
Alunova: Droga jail area redone, some doors are now locked and spawns moved.



kanras edit:

Spells
(c) = Current. The version of the spell currently active on red/blue.
(o) = Old. A version of the spell prior to red/blue timeline.
(a) = All. Affects all versions of the spell including those active on red/blue.
If there's no () notation after the spell, it's assumed to be a change for immediately when Velious is released.

Yaulp IV (Feb2001) updated PAL lvl 255 => 60
Word of Vigor updated from 60 base heal, 583 max heal, x9 heal formula => 70 base heal, 625 max heal, x10 heal formula
Word of Redemption (c) updated mana 1100 => 1200
Wolf Form updated movespeed max 0 => 55
Winged Death updated cast_time 6.0 => 5.0, mana 300 => 350, buffduration 10 => 9, damage -110 => -143
Winds of Gelid (o) updated damage base -1260 => -854, damage max 1260 => 1360
Winds of Gelid (c) updated mana 1000 => 875
Wildfire (o) updated mana 320 => 375, damage 1024 => 1144
Weakening Poison (c) updated name => 'Weakening Poison I', poison counter 1 => 2, str max 0 => 34
Wave of Flame (a) updated stunduration 1000 => 1
Wake of Karana updated recast_time 90000 => 360000
Voltaic Draught (a) updated to 'Voltaic Draugh'
Voltaic Draugh (o) updated MR => CR
Vocerate: Water (o) updated level 54 => 55
Vocerate: Fire (o) updated level 52 => 53
Vocerate: Air (o) updated level 53 => 54
Vocerate: Earth (o) updated level 51 => 52
Visions of Grandeur (o) updated buffdurationformula 11 => 9, atk 15 => 10, attack_speed 158 => 170, recast_timer 15.0 => 18.0, added rune 230pt, removed agi/dex
Vexing Mordinia updated mana 450 => 495, recast_time 2.25 => 0, damage -111 => -122
Vexing Mordinia (c) updated recast_time 2.25 => 10.0
Vengeance of Al`Kabor updated base damage -296 => -352, max damage 896 => 952
Velocity (o) updated level 58 => 51
Touch of Night (o) updated mana 405 => 495, dmg base -720 => -225, dmg max 720 => 825
Torpor (o) updated mana 200 => 100
Torment of Argli updated recast_time 30.0 => 10.0, effect1 -WIS, effect3 -INT, effect4 -HP
Thorny Shield (a) updated DS -4 => -1
Theft of Thought updated mana 10 => 25
Talisman of the Serpent (o) updated mana 350 => 400
Talisman of the Rhino(o) updated mana 350 => 400
Talisman of the Raptor (o) updated mana 350 => 400
Talisman of the Cat (o) updated mana 350 => 400
Talisman of the Brute (o) updated mana 350 => 400
Talisman of Shadoo updated max 40 => 45
Talisman of Jacinth updated max 40 => 45
Syvelian`s Anti-Magic Aria updated range 75 => 100
System Shock (c) updated name to 'System Shock I', max 0 => 149
System Shock II (a) updated stunduration 1000 => 1, stunmin 500 => 1
System Shock III (a) updated stunduration 1000 => 1, stunmin 500 => 1
System Shock IV (a) updated stunduration 1000 => 1, stunmin 500 => 1
System Shock V (a) updated stunduration 1000 => 1, stunmin 500 => 1
Symbol of Marzin (o) updated peridots 1 => 2
Superior Healing (Feb2001) updated DRU lvl 53 => 51, SHM lvl 53 => 51
Sunstrike (o) updated mana 450 => 500, cast_time 7.0 => 7.5, dmg 1615 => 1600
Summon Corpse (Jan2001) updated SK level 255 => 51
Summon Companion (c) updated all lvl except MAG => 255
Strength of Stone updated RNG level 53 => 255
Strength of Nature (Jan2001) updated RNG level 255 => 51
Stream of Acid (a) updated buffdurationformula 7 => 2, buffduration 20 => 0
Storm Strike (a) updated stunduration 1000 => 1
Storm Strength (o) updated RNG lvl 255 => 53
Spirit Tap (Feb2001) updated SK lvl 56 => 55
Spirit of Wolf (Feb2001) updated RNG lvl 39 => 30
Spirit of the Howler updated mana 750 => 850
Spirit of Oak updated mana_regen base 1 => 4, mana_regen formula 119 => 100
Spikecoat (Feb2001) updated RNG lvl 255 => 49
Speed of the Shissar (c) updated buffdurationformula 3 => 10
Speed of the Shissar (Jul2001) updated buffduration 200 => 300
Song of the Deep Seas (a) updated name => 'Song of the Deep Sea's', buffdurationformula => 10, buffduration => 3
Skin of the Shadow updated buffduration 200 => 360, magnification 130 => 115
Skin like Nature (c) updated RNG lvl 59 => 255
Siphon updated damage -85 => -80
Shroud of the Spirits updated max 80 => 100
Shroud of Hate (a) updated atk -20 => -15
Shrink updated shrink 75 => 66
Shock of Steel updated mana 300 => 275, dmg base -300 => -225, dmg max 900 => 825
Shieldskin (Feb2001) updated SK lvl 39 => 34
Shield of Song updated name => 'Shield of Songs'
Seething Fury updated buffduration 60 => 75
See Invisible (Feb2001) updated RNG lvl 255 => 39
Searing Flames moved FR reduction slot 6 => slot 1, FR -20 => -100
Scintillation updated cast_time 3.5 => 3.0, recast_time 7.0 => 6.5, dmg max 618 => 608
Rune V (o) updated peridots 1 => 2
Ro's Fiery Sundering updated buffduration 30 => 50
Retribution of Al`Kabor updated dmg base -248 => -301, dmg max 848 => 901
Resist Magic (Feb2001) updated PAL lvl 255 => 55
Resist Disease (Jan2001) updated PAL lvl 255 => 51
Resist Cold (Feb2001) updated RNG lvl 255 => 55
Rend (o) updated MR => CR
Reckoning updated mana 300 => 250, dmg base -210 => -90, dmg max 810 => 675
Rapture (o) updated mana 425 => 600, recast_time 24.0 => 48.0
Rain of Molten Lava (c) updated dmg -300 => -1000
Rain of Cold (c) updated dmg -300 => -1000
Rage of Tallon (a) added 30 atk
Quivering Veil of Xarn updated buffduration 4 => 3
Pyrocruor updated cast_time 4.0 => 5.5
Pox of Bertoxxulous updated cast time 5.0 => 6.1, buffduration 20 => 18, dmg tick -100 => -111
Pillar of Lightning (o) updated MR => FR
Pillar of Frost (o) updated cast_time 4.5 => 2.7
Pillar of Flame (o) updated cast_time 4.5 => 3.15, dmg base -652 => -592, dmg max 952 => 892
Phantom Plate (o) removed HP regen
Phantom Leather (o) removed HP regen
Phantom Chain (o) removed HP regen
Paralyzing Poison * (a) updated effect 2 95% snare => root
Panic Animal (Feb2001) updated RNG lvl 255 => 22
Nature Walker's Behest (a) updated mana 220 => 600, cast_time 9.0 => 14.0
Nature's Touch (a) updated mana 457 => 400, cast_time 3.75 => 5.5, heal base 891 => 978, heal max 1491 => 978, heal formula 10 => 100
Muscle Lock (c) updated name => 'Muscle Lock I', snare max 40 => 55
Muscle Lock (o) removed poison counters
Mortal Deftness (o) updated dex 60 => 50
Mort Drowsy updated base slow -20 => 90
Minor Summoning: * (Apr2001) updated gem Lapis Lazuli => Malachite
Mask of the Hunter (a) updated Magnification 130 => 115
Maniacal Strength (c) updated buffduration 900 => 1440, str 72 => 68
Maniacal Strength (o) updated name => 'Manicial Strength'
Manaskin updated peridots 2 => 1
Malo updated all max 40 => 45
Mala updated all max 40 => 35
Lure of Lightning (o) updated MR => CR, dmg -680 => -640
Lower Resists (c) updated name => 'Lower Resists I', all max 0 => 10
Liquid Silver (c) updated name => 'Liquid Silver I', max 0 => 250, poison counter 0 => 2
Liquid Silver (c) updated poison counter 0 => 1
Legacy of Thorn (o) updated ds -32 => -24
Legacy of Thorn (c) updated ds -32 => -28
Legacy of Spike (a) updated ds -24 => -20
Largarn's Lamentation updated cast_time 3.0 => 2.5, recast_time 30.0 => 24.0, removed -AGI effect
Jyll's Zephyr of Ice updated dmg 561 => 594
Jyll's Wave of Heat updated dmg base -2 => -48, dmg max 612 => 648
Judgment of Ice (a) updated stunduration 1000 => 500, range 300 => 30
Jonthan's Inspiration updated effect 3 blank, effect 1 min 103, max 166, effect formula 102
Jonthan's Provocation updated effect 3 blank, effect 1 min 103, max 133, effect formula 102
Jolt (c) updated recovery_time 2.25 => 2.0, recast_time 2.25 => 0.0
Jolt (Feb2001) updated RNG lvl 55 => 51
Invisibility to Undead updated WIZ lvl 255 => 39
Improved Superior Camouflage updated DRU lvl 255 => 50
Invert Gravity updated cast_time 10.0 => 12.0
Insight (o) updated wis base 1 => 0, removed int
Insight (c) updated wis max 0 => 14, int max 0 => 12
Injected Poison (c) updated name => 'Injected Poison I', dmg max 0 => 38, dot max 0 => 38
Imbue Ivory (c) updated SHM lvl 29 => 34
Ikatiar's Revenge updated buffduration 6 => 0, tic dmg -20 => -2
Haste (o) updated atk speed max 150 => 135
Hammer of Requital (c) updated PAL lvl 54 => 255
Guard (Feb2001) updated PAL lvl 49 => 39
Growth updated name => 'Grow', grow amt 125 => 133
Group Resist Magic updated buffduration 270 => 360
Grim Aura (Feb2001) updated SK lvl 255 => 22, base atk 5 => 3, max atk 0 => 10, atk formula 100 => 102
Great Divide Portal updated WIZ lvl 36 => 44
Great Divide Portal (Jul2001) updated WIZ lvl 44 => 39
Glamour of Tunare updated buffduration 190 => 0, buffdurationformula 9 => 10
Glamour of Tunare (c) updated buffduration 190 => 0
Glamour of Kintaz (o) updated mana 275 => 350
Glamour of Kintaz (c) updated buffduration 4 => 5, ResistDiff 0 => -10
Gift of Pure Thought updated mana 350 => 300
Fury of Air (a) updated mana 85 => 150
Fufil`s Curtailing Chant (o) updated range 200 => 150
Frost (o) updated cast_time 6.0 => 4.0
Fist of Karana updated dmg 894 => 812
Fist of Air (a) updated stunduration 1000 => 1, stunlimit 500 => 1
Firestrike (o) updated RNG lvl 52 => 59
Firefist (Feb2001) updated RNG lvl 255 => 22
Fellspine (c) updated hate reduction -50 => -150
Feeble Mind (c) updated name => 'Feeble Mind I', max mana 0 => 222
Feast of Blood (c) updated to 100hp instant lifetap
Fascination updated buffduration 5 => 6
Expel Undead (o) updated CLE lvl 54 => 55
Enticement of Flame (a) updated cast_time 4.5 => 6.0
Enforced Reverence updated mana 240 => 200
Endure Magic (Feb2001) updated PAL lvl 255 => 30
Endure Cold (Feb2001) updated RNG lvl 255 => 22
Enchant Steel (Apr2001) removed gem component
Enchant Silver (o) updated mana 60 => 80
Enchant Mithril (Apr2001) removed gem component
Enchant Brellium (Apr2001) removed gem component
Enchant Adamantite (Apr2001) removed gem component
Elemental Maelstrom (a) updated cast_time 3.0 => 7.0
Elemental: * (Apr2001) updated gem Lapis Lazuli => Malachite
Dyzil's Deafening Decoy updated cast_time 10.0 => 8.0, recast_time 0.0 => 24.0
Dyn`s Dizzying Draught updated recast_time 50.0 => 35.0
Draught of Jiva (o) updated MR => CR
Draught of Ice (o) updated dmg base -431 => -388, dmg max 731 => 688
Drain Spirit (Feb2001) updated SK lvl 60 => 57
Drain Soul (Feb2001) updated SK lvl 255 => 60
Dragon Charm (a) updated buffduration 205 => 0
Dizzy (c) updated name => 'Dizzy I', poison counters 1 => 2, agi max 0 => 25, ac max 0 => 50
Divine Light updated heal base 635 => 350, heal max 875 => 910, heal formula 105 => 10
Divine Intervention updated buffduration 60 => 100
Divine Aura (o) updated PAL lvl 55 => 58
Dictate (o) updated buffduration 8 => 5
Diamondskin (c) updated SK lvl 59 => 255
Devouring Darkness updated buffduration 15 => 13, tick dmg -107 => -123
Denon's Bereavement (a) updated stunduration 1000 => 1, stunlimit 500 => 1
Deliriously Nimble updated agi 50 => 52
Death Peace (a) updated NEC lvl 60 => 255
Death Pact updated recast_time 60.0 => 15.0, buffdurationformula 10 => 11, buffduration 60 => 600
Dance of the Blade (a) updated buffduration 30 => 25
Crystallize Mana (a) removed Emerald component
Corporeal Empathy updated name => 'Corpal Empathy'
Contact Poison (c) updated name => 'Contact Poison I', max 0 => 141
Combust (o) updated RNG lvl 255 => 52
Clarity II (o) updated mana 115 => 125, ENC lvl 54 => 52
Clarity II (c) updated buffduration 330 => 350
Circle of Winter updated max 40 => 45
Circle of Summer updated max 40 => 45
Chloroplast (o) updated RNG lvl 55 => 255
Chilling Embrace removed poison counters, resist CR => PR
Charm Animals (o) updated SHM lvl 34 => 255
Char updated mana 245 => 260, dmg base -131 => 140, dmg max 711 => 702
Chaos Breath updated mana 0 => 337
Cessation of Cor updated cast_time 6.0 => 5.0, str base -15 => -25
Cannibalize III (Apr2001) updated hp -100 => -74, mana 36 => 26
Cannibalize II (Apr2001) updated mana 18 => 20, mana formula 109 => 110
Cacophony recast_time 0.0 => 12.0
Brilliance (o) removed WIS
Breath of Ro (c) updated mana 225 => 250, tic dmg -103 => -92
Breath of Karana (c) updated dmg base -72 => -212, dmg max 738 => 812, dmg formula 103 => 10
Bramblecoat (Feb2001) updated RNG lvl 49 => 39
Boon of the Clear Mind (o) updated ENC lvl 52 => 54
Blessing of the Grove (a) added atk speed 140, max 140
Blanket of Forgetfulness updated cast_time 5.0 => 4.0
Banshee Aura (o) updated buffdurationformula 9 => 7, buffduration 90 => 30, dmg base -6 => -8, dmg max 12 => 0, dmg formula 110 => 100
Bane of Nife (o) updated cast_time 5.0 => 6.0
Avatar updated recast_time 360.0 => 180.0, mana 375 => 325, removed grow
Aura of * Petals moved all resists into effect1
Augment Death updated mana 100 => 200, buffdurationformula 8 => 10
Asphyxiate updated tic dmg -45 => -48
Armor of Faith (Feb2001) updated PAL lvl 53 => 49
Antidote updated poison remove -9 => -16, disease remove -9 => -16
Angstlich's Assonance (a) updated dmg -5 => -25, dmg formula 121 => 100
Ancient Breath (a) updated recast_time 60.0 => 12.0, atk slow 60 => 40
Aegis of Ro updated cast_time 5.0 => 9.0

Resistant Discipline updated WAR lvl 255 => 30, MNK lvl 255 => 30, ROG lvl 255 => 30, BRD lvl 255 => 51, SK lvl 255 => 51, RNG lvl 255 => 51, PAL lvl 255 => 51
Fearless Discipline updated WAR lvl 255 => 40, MNK lvl 255 => 40, ROG lvl 255 => 40, BRD lvl 255 => 54, SK lvl 255 => 54, RNG lvl 255 => 54, PAL lvl 255 => 54
Resistant Discipline (Jan2001) updated discduration 60000 => 300000
Holyforge Discipline updated PAL lvl 255 => 55
Holyforge Discipline (Jan2001) updated discduration 120000 => 300000
Weapon Shield Discipline updated RNG lvl 255 => 60
Weapon Shield Discipline (Jan2001) updated discduration 15000 => 20000
Leechcurse Discipline updated SK lvl 255 => 60
Leechcurse Discipline (pre-Feb2001) removed HitChance component
Leechcurse Discipline (Jan2001) updated discduration 15000 => 20000
Deftdance Discipline updated BRD lvl 255 => 55
Deftdance Discipline (Jan2001) updated discduration 10000 => 15000
Puretone Discipline updated BRD lvl 255 => 60
Puretone Discipline (Jan2001) updated discduration 120000 => 240000
Sanctification Discipline updated PAL lvl 255 => 60
Sanctification Discipline (Jan2001) updated discduration 10000 => 15000
Trueshot Discipline updated RNG lvl 255 => 55
Unholy Aura Discipline updated SK lvl 255 => 55
Unholy Aura Discipline (Jan2001) updated focus 25 => 50


*Lazy updates* - Most of these are post-50 scaling clamps on pre-50 spells that likely went live very shortly after kunark release.
Weakness updated max
Voice of the Berserker updated max
Surge of Enfeeblement updated max
Valor updated max
Talisman of Altuna updated max
Upheaval updated dmg max 738 => 725
Spirit of Scale updated max
Skin like Steel updated max
Skin like Nature updated max
Skin like Diamond updated max
Shield of Spikes updated max
Selo`s Accelerando updated max
Seeking Flame of Seukor updated dmg base/max 990 => 1024
Scale of Wolf updated max
Savage Spirit updated all max
Resolution updated max
Renew Bones updated max
Rampage updated max2
Purifying Rhythms updated all max
Pack Spirit updated max
Listless Power updated all max
Intensify Death updated max2, max3
Insipid Weakness updated max
Insidious Malady updated max2
Insidious Fever updated max2
Incapacitate updated all max
Illusion:Werewolf updated max6
Guardian Rhythms updated max2
Group Resist Magic updated max
Greater Wolf Form updated max2
Gasping Embrace updated max2, max3
Form of the Great wolf updated max2
Enveloping Roots updated max1
Ensnaring Roots updated max1
Engulfing Roots updated max1
Elemental Rhythms updated max1, max2, max3
Dooming Darkness updated max2
Divine Barrier updated max2
Dead Man Floating updated max7
Cassindra`s Chorus of Clarity updated max
Cascading Darkness updated max2
Bravery updated max2, max3
Augmentation updated max1,max2,max3


// A

[Someone]

Its dsetup.dll in their patch that does this: http://www.project1999.com/files/P99Files33.zip

[ohhello]

I think that giant opcode is the result of a memory leak in SEQ due to the struct sizes being wrong. If you do zone log file you can clearly see that the spawns and such are not encrypted, including location and name in clear text, and if you run the client in gdb and it doesnt crash, you can con mobs and gray skittles will be drawn in the correct place. I'd try correcting the structs too even it if just means creating dummy fields at least to keep the client sane.

the reason i think it's a memory leak is because on my vm i could see text such as file paths, etc that clearly would not be transmitted from the game client.

[BlueAdept]

Even though I have not looked, I still feel that it is the struct size mismatch that is causing a lot of the problem. I really do not think it is a new encryption since you are getting some things.

[ohhello]

I'm not sure if I am barking up the wrong tree but I applied some of the techniques that others have been discussing and found a couple of inconsistencies that I'm not sure are relevant. I generated a zone log and then compared the data with structs that are currently in the 5.2.2.0 everquest.h. All but two of the structs that I have found so far are consistent as far as size but there were two exceptions:

First exception: In everquest.h, ServerZoneEntryStruct has a length of 383 octets but the struct was logged as size 385:
(from everquest.h)
/*
** Server Zone Entry struct
** Length: 383 Octets
** OpCode: ZoneEntryCode (when direction == server)
*
* This is just a spawnStruct for the player
*/
struct ServerZoneEntryStruct : public spawnStruct
{
};

(from zone.log)
Jul 04 2014 15:13:20:033 [Decoded] [Server->Client] [Size: 385]
[OPCode: 0x7213]
[Name: OP_ZoneEntry][Updated: 10/27/05][Type: ServerZoneEntryStruct (385) ==]
000 | 00 00 00 00 00 00 00 4d 53 3c 20 22 45 59 39 5a | .......MS< "EY9Z
016 | 3c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | <...............
032 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
048 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
064 | 00 00 00 00 00 00 00 82 00 00 00 00 00 00 00 00 | ................
080 | 00 00 00 00 00 00 4e 29 00 00 00 00 00 00 00 00 | ......N)........
096 | bb 52 ce bb 46 00 2f 00 00 00 00 cd 34 00 00 00 | .R..F./.....4...
112 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
128 | 00 00 00 00 00 00 00 00 00 00 00 49 00 00 00 00 | ...........I....
144 | 00 00 00 00 00 00 00 4b 00 00 00 00 00 00 00 00 | .......K........
160 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
176 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
192 | 00 00 00 00 00 4f 00 00 00 53 00 00 00 31 00 00 | .....O...S...1..
208 | 00 4a 00 00 00 39 00 00 00 4f 00 00 00 53 00 00 | .J...9...O...S..
224 | 00 23 00 00 00 9e 00 00 00 0b 79 33 70 00 da 49 | .#........y3p..I
240 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
256 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
272 | 00 00 00 00 b0 b2 c9 b7 00 00 00 00 40 00 00 00 | ............@...
288 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
304 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
320 | 00 00 00 00 d0 aa d6 73 00 00 00 5b 00 00 4b 32 | .......s...[..K2
336 | 00 00 00 b7 37 55 00 00 00 00 00 00 00 00 00 ae | ....7U..........
352 | 00 00 00 cc 00 00 00 b7 00 00 00 c7 00 00 00 b2 | ................
368 | 00 00 00 ae 00 00 00 cc 00 00 00 b7 00 00 00 c7 | ................
384 | 00

Second exception: In everquest.h, zonePointsStruct has length of 24 octets but is logged with 52:
(from everquest.h)
/*
** ZonePoint
** Length: 24 Octets
** Sent as part of zonePointsStruct
*/

struct zonePointStruct
{
/*0000*/ uint32_t zoneTrigger;
/*0004*/ float y;
/*0008*/ float x;
/*0012*/ float z;
/*0016*/ float heading;
/*0020*/ uint16_t zoneId;
/*0022*/ uint16_t zoneInstance;
/*0024*/
};

(from zone.log)
Jul 04 2014 15:13:31:448 [Decoded] [Server->Client] [Size: 52]
[OPCode: 0x3eba]
[Name: OP_SendZonePoints][Updated: 10/27/05][Type: zonePointsStruct (28) nc]
000 | 01 00 00 00 01 00 00 00 33 f3 cd 44 cd ac 4a 44 | ........3..D..JD
016 | 00 00 74 42 00 00 02 43 2f 00 00 00 00 00 00 00 | ..tB...C/.......
032 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
048 | 00 00 00 00

Is this worth pursuing? It seems on track with the forum guidance regarding inconsistencies in struct sizes. I should also note that the console window didn't produce any useful error messages about inconsistent opcodes or struct sizes during these captures.

Also, there are many entries in the log file of which I'm not sure of the significance. What do these mean?
Examples:
(opcode but zero size)
Jul 04 2014 15:13:15:244 [Decoded] [Server->Client] [Size: 0]
[OPCode: 0x3cdc]

(opcode and size but no label)
Jul 04 2014 15:13:18:256 [Decoded] [Server->Client] [Size: 768]
[OPCode: 0x1234]
000 | bf 00 00 00 03 00 03 00 09 00 09 00 12 00 12 00 | ................
016 | 13 00 13 00 21 00 77 21 39 00 5e 21 3d 00 3d 00 | ....!.w!9.^!=.=.
032 | 3f 00 3f 00 40 00 40 00 48 00 48 00 50 00 50 00 | ?.?.@[email protected].
.... to 768

(opcode and name but no size)
Jul 04 2014 15:13:20:268 [Decoded] [Client->Server] [Size: 0]
[OPCode: 0x7ac5]
[Name: OP_ReqNewZone][Updated: 10/27/05]

(a gigantic one that I'm not sure what to make of - seems to contain opcode information later on)
Jul 04 2014 15:13:31:448 [Decoded] [Server->Client] [Size: 4294967295]
[OPCode: 0000]
000 | 00 00 fe d5 00 00 00 00 00 00 ff ff ff ff 00 00 | ................
016 | 00 00 62 00 09 00 e6 47 0f 00 00 00 00 00 00 00 | ..b....G........
032 | 00 00 00 00 00 fd 01 00 00 31 00 00 00 00 00 00 | .........1......
048 | 00 94 11 00 7f 00 00 00 00 66 66 6c 42 00 80 c1 | .........fflB...
064 | 43 00 00 58 43 49 54 36 33 5f 41 43 54 4f 52 44 | C..XCIT63_ACTORD
080 | 45 46 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | EF..............
096 | 00 00 00 00 00 fe d5 00 00 00 00 00 00 ff ff ff | ................
....

Thoughts?

// A



Looking at the two opcodes you mentioned it doesn't look like either of them are the wrong size. The first struct is a spawnStruct which is 385 bytes (the comment in the header must be off because spawnStruct does appear to be 385.) The second opcode is actually zonePointsStruct (not zonePointStruct) which looks to be a wrapper around zonePointStruct for the 52 byte packet.

Something is causing a memory leak which makes the client crash eventually, i'd wager it's struct changes from the con color updates.

[ohhello]

So, bad news.

Just looked at it again and I do have some zone logs from pre-patch.

Before:



May 18 2014 xx:xx:xx [Decoded] [Server->Client] [Size: 38500]
[OPCode: 0x2e78]
[Name: OP_ZoneSpawns][Updated: 10/27/05][Type: spawnStruct (385) %]
000 | 00 00 00 00 00 00 00 6f 72 63 5f 63 65 6e 74 75 | .......orc_centu
016 | 72 69 6f 6e 30 30 31 00 00 00 00 00 00 00 00 00 | rion001.........
032 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
048 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
064 | 00 00 00 00 00 00 00 00 00 00 00 00 00 e0 40 00 | ..............@.
080 | 00 00 00 01 00 00 64 64 00 00 00 00 00 00 00 fc | ......dd........
096 | 48 01 df 1c 00 00 53 fe 07 00 00 80 a6 00 00 00 | H.....S.........
112 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
128 | 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 | ................
144 | 01 00 00 00 00 00 00 09 00 00 00 00 00 00 00 00 | ................
160 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
176 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................


Now



Jul 02 2014 xx:xx:xx [Decoded] [Server->Client] [Size: 38500]
[OPCode: 0x2e78]
[Name: OP_ZoneSpawns][Updated: 10/27/05][Type: spawnStruct (385) %]
000 | 00 00 00 00 00 00 00 53 09 39 55 40 2d 24 35 47 | .......S.9U@-$5G
016 | 30 6d 34 39 41 42 2d 7a 6b 02 00 00 00 00 00 00 | 0m49AB-zk.......
032 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
048 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
064 | 00 00 00 00 00 00 00 00 00 00 00 00 00 b2 17 00 | ................
080 | 00 00 00 4b 00 00 27 56 00 00 00 00 00 00 00 86 | ...K..'V........
096 | 7e 33 b5 54 00 00 aa 42 00 00 00 d2 8d 00 00 00 | ~3.T...B........
112 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
128 | 00 00 00 00 00 00 00 00 00 00 00 4a 00 00 00 00 | ...........J....
144 | 5b 00 00 00 00 00 00 16 00 00 00 00 00 00 00 00 | [...............
160 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
176 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................

[BlueAdept]

I guess I am wrong.

[ohhello]

yup, spawn data is surely encrypted now. dsetup.dll ballooned by almost 1MB since last release and it already foils debugger hooks so unless someone can deobfuscate and disassemble the dll and inspect the encryption, seq is dead on p99.

[BlueAdept]

I do not know if anyone who knows how the old encryption worked is still around (I believe it was just an XOR (might have been ROT13 cant remember) and since everything still lines up correctly, might still be an XOR but just changed a little). I know purple is still around (since he posted) but I do not know if he worked on any of the encryption in the past or would be willing to look at it. Even though IEATACID doesn't show up here much, he is still at the MQ2 boards. Since he is still current on things, he might be your better shot at fixing it.

[wanlor]

Using a zone log where we know what the spawn names should be, XOR encryption could be cracked using frequency analysis (in theory).

Or if we know what the plaintext of an encrypted string is, we could XOR them together:

encrypted = plaintext XOR key
key = encrypted XOR plaintext

[BlueAdept]

After a little research, I verified that everquest used XOR. Since the client is titanium, I would assume that they are still using XOR but changed the key/method. If anyone is able to fix it, I can start a new section and branch for 1999 if there is interest.

[wanlor]

Well, there is definitely some XOR going on. Just figured out an XOR key that can decrypt my character name. Going to see if it generalizes to other things.

[BlueAdept]

I do not know if this will help anyone but thought I would post it.

This kind of encryption is trivial to break, even without computers [587,1475]. It will only take a few seconds with a computer. Assume the plaintext is English. Furthermore, assume the key length is any small number of bytes. Here’s how to break it:

Discover the length of the key by a procedure known as counting coincidences [577]. XOR the ciphertext against itself shifted various numbers of bytes, and count those bytes that are equal. If the displacement is a multiple of the key length, then something over 6 percent of the bytes will be equal. If it is not, then less than 0.4 percent will be equal (assuming a random key encrypting normal ASCII text; other plaintext will have different numbers). This is called the index of coincidence. The smallest displacement that indicates a multiple of the key length is the length of the key.
Shift the ciphertext by that length and XOR it with itself. This removes the key and leaves you with plaintext XORed with the plaintext shifted the length of the key. Since English has 1.3 bits of real information per byte, there is plenty of redundancy for determining a unique decryption.

I am by no means an expert here, I never worked on the encryption nor dealt with a lot of the code. The only thing I can provide is information that I remember being talked about and hopefully it will help.

Back in the libeq.a days, I remember being told that the key is sent upon zoning. I do not think it was the key for that zone, but for when you zone again. If you figured out that key for your name, you might want to search previous packets for that key. As I said, I believe it changes each time you zone.

I believe the decoding of the packets comes from packetinfo.cpp and packetinfo.h, as always, I could be wrong. I only glanced at the code.

I havent programmed anything in C since college (over 25 years ago). I was rusty when I was helping with the structs, now it looks like greek to me now. I, unfortunately, will not be much help.

[wanlor]

Some good news:

Same key that can decrypt my user name works for names in the spawn struct:



'a_Krag_Chick003',
'a_rock_spiderlind@8',
'Glisse_Bluesea000',
'a_goblin_whelp039',
'\u0002N\u001ei\u0016\u0007\u000b',
'\u0003E\u0010t\u0003\n\u0012\u0015\u001botw',
'H\u001de\n\u0006\u001f\u0011',
'a_dwarf_skeleton003'


I'm doing this somewhat manually so it's not working 100% (as you can see above). And, it doesn't seem to work for the two structs I tried it on in this thread (that others posted). However, some of that might be choosing an incorrect offset to start reading the encrypted name.

Here's how I got this far in broad strokes:

1. turn on zone logging
2. cast spell on myself to see what my encrypted name is in ascii (let's pretend it's e8wR&!)
3. search the zone log for the encrypted ascii (cat zone.log | grep e8wR&! -a2)
4. pull out the encrypted bytes for that ascii [65 38 77 52 26 21]
5. since I know my name is "Muffin" [4d 75 66 66 69 6e] the key = [4d 75 66 66 69 6e] XOR [65 38 77 52 26 21]
6. we now have the key: [28 4d 11 34 4f 4f]

After doing this I realized that the key found this way wasn't long enough, so I made a character with a longer name to find a longer key. The longer key shared the first 6 bytes with the shorter key, had for new bytes, and then started repeating. Looks like it's a 10 byte key truncated or repeated to fit the string length.

Any thoughts on where we should go from here?

[BlueAdept]

Nice work. You get an Attaboy for that.

Just need to find someone who can figure out how to change the source now.

[wanlor]

Blue, thanks for the help! Missed this post when I commented.



Back in the libeq.a days, I remember being told that the key is sent upon zoning. I do not think it was the key for that zone, but for when you zone again. If you figured out that key for your name, you might want to search previous packets for that key. As I said, I believe it changes each time you zone.


It doesn't look like the key changes on every zone as I was able to use the same key to decode the character names for different characters in different zones. It might be possible that different accounts/clients get different keys, as that would explain me not being able to decode the structs from earlier in this thread, but probably too soon to make that assumption.

[ohhello]

I was going to try your XOR method but I can't seem to keep SEQ from crashing before the client finishes loading. Have you checked world and zone data for the presence of your key in any packets? They must calibrate the client somehow if it's not the same key for everybody...

[wanlor]

I had one character that kept crashing SEQ on zone, but the other younger characters kept SEQ somewhat stable.

[seqrox]

How's it going with the decoding? Is there a process I can follow to update the client with the key based on knowing my char name? I miss my alerts when the boats are zoning in and whatnot :D

[ohhello]

So it's pretty easy to get your session key from a combination of ClientZoneEntryStruct which is in cleartext and the ServerZoneEntryStruct which contains the player name with the XOR encryption. However, if the player's name is not 10 characters long or longer you don't get the entire 10 byte key.

[ohhello]

Also of note, between the time that the client sends the ClientZoneEntryStruct in plaintext and the server sends back CharInfoStruct, the server now sends a packet with opcode 1234 that did not exist in my older logs. The content of this opcode appears to be static, and also appears to be the last thing the server sends to the client before the encryption starts.

[ohhello]

news flash

the client cleartexts the key to the the server in the login packet.

Jul 09 2014 13:51:22:888 [Decoded] [Client->Server] [Size: 464]
[OPCode: 0x4dd0]
[Name: OP_SendLoginInfo][Updated: 10/27/05]
000 | 31 32 33 34 35 36 00 AB EE CD 4a FF 32 44 FF 56 | 123456.38ICZ2XYV
016 | 4b 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | K...............
(values obfuscated intentionally)

The 10 digit ascii above is the encryption key for this session, sent by client to server. It's a cyclic xor encryption like everyone suspected. They put it into the "password" field. Special thanks to another friend who is working on this as well.

[ohhello]

This puzzle has been solved. Perhaps not elegantly, but it works.

here's a patch for showeq 5.2.2.0:
http://s000.tinyupload.com/?file_id=82338680233971885187 (http://s000.tinyupload.com/index.php?file_id=82338680233971885187)

Be sure to do a 'make install' to copy the opcode xml changes to /usr/local/share/showeq

I don't know if logging is still crashing the client or not but honestly once I got the skittles back I didn't care about logging anymore.

Also as an added bonus staticspells.h has been updated with their latest version of us_spells.txt.

Again special thanks to a buddy who helped figure out the decryption and to wanlor for pointing everybody in the right direction.

[BlueAdept]

Nice work. I will make a new zip when I can and put it in the file section.
If you think we should make a new branch, I will set that up but will need someone to maintain it.

[purple]

The only real issue with the diff to me (except it is kinda noisy but whatever) is that you're keeping a list of opcodes that need to be decoded in a case statement hiding in packetstream.cpp. It would be better to either add a decode element to opcodes.xml (default to false and then add decode="true" for the ones you want decoded) or find something in the network stream that indicates the opcode needs decoding. That's probably just being nitpicky though and working is working.

Nice job.

[BlueAdept]

Just an FYI. I found this protocol information file. I know the solution has been found and that this is probably older than what the 1999 version is, but it might be good reading for those who want to understand what is going on.



Everquest Protocol Layer

This is a rough draft document for the Everquest protocol used by
Verant Interactive. It is a work in progress, and as such, if you
would like to contribute information please email me at
[email protected].

General Information

The EQ layer is used over the UDP layer. UDP is a connectionless
protocol, and packets sent using UDP are not guaranteed to reach
their destination. The EQ layer adds error checking, acknowledgements,
sequencing, and fragmentation.

Header Flags Map

Each packet start with a header containing various flags and sequence
information. The first 16 bits of each packet containes the following
flags (broken into 4 4-bit nibbles):

/=+=+=+=\/=+=+=+=\ /=+=+=+=\/=+=+=+=\
|8|4|2|1||8|4|2|1| |8|4|2|1||8|4|2|1|
\=+=+=+=/\=+=+=+=/ \=+=+=+=/\=+=+=+=/
(8) SEQEnd -/ | | | | | | | |
(4) Closing --/ | | | | | | |
(2) SEQStart ---/ | | | | | |
(1) ASQ ----------/ | | | | |
============= | | | | |
(8) Fragment --------/ | | | |
(4) Closing -----------/ | | |
(2) ARQ -----------------/ | |
(1) ? | |
============= | |
(8) ? | |
(4) ? | |
(2) ? | |
(1) ? | |
============= | |
(8) ? | |
(4) ARSP ---------------------------------/ |
(2) ? |
(1) SpecARQ ----------------------------------/

Full Packet Map

The full header of each packet can be of varying sizes, depending
on which bits are set in the Flags-word.

/=======+=======+=(ARSP?)=+=(ARQ?)=\ /==========(Fragment?)=========\
| Flags | dwSEQ | dwARSP | dwARQ | ... | dwFragSEQ | dwCurr | dwTotal | ...
\= 16b =+= 16b =+== 16b ==+= 16b ==/ \=== 16b ===+= 16b ==+== 16b ==/

/==(ASQ?)==+=(ASQ && ARQ?)=\ /=(!ACKSize?)=\
... | ASQ_high | ASQ_low | ... | dwOpCode | ...
\=== 8b ===+===== 8b ======/ \==== 16b ====/

/==(!ACKSize?)==\ /=======\
... | ..(data).. | ... | CRC32 |
\===== ?b ======/ \= 32b =/


Mapping Information

As you can see, some header data is not present, depending on which flags
are set.

dwSEQ: Seqence number.

This is always sent. If the SEQStart bit is set, then this value
is used as the new current sequence number. If not, then this can
be used to see if the packet is a resend.

dwARSP: ACK Response.

This is only sent if the ARSP bit is set in the Flags word. This should
equal the 16-bit dwARQ of the packet being responded to.

dwARQ: ACK Request.

This is only sent if the ARQ bit is set in the Flags word. This 16-bit
value should be used as the dwARSP in the packet used to respond.

dwFragSEQ, dwCurr, dwTotal: Fragmentation Information.

These three 16-bit values are only set if the Fragment bit is set
in the Flags word.

dwFragSEQ - Fragment Sequence this packet belongs to.
dwCurr - Fragment number in this fragment sequence.
dwTotal - Total number of fragments in this fragment sequence.

ASQ_high: ?.

This is only sent if the ASQ bit is set in the Flags word. I'm still
trying to figure out exactly what this is used for.

ASQ_low: ?.

This is only sent if the ASQ bit and ARQ bit are set in the Flags word.
I'm still trying to figure out exactly what this is used for.

dwOpCode: Op Code (Used for dispatching).

This is only sent if the packet is not a pure acknowledgement or a
pure request.

CRC32: 32-bit CRC check.

This is always sent. It is used for error checking in tranmissions.
See the Integrity_Task for more information.


Pure ACKnowledgements

A pure acknowledgement is sent if a ARQ is pending, but no packet is sent
in a certain amount of time (around 0.1 seconds I think). If a packet is
sent out before this timeout has occured, the ARSP bit, and dwARSP is set
to dwARQ from the requesting packet. If no packets are sent out before
the timeout occurs, then a pure acknowledgement packet is sent with only
ARSP and dwARSP (no OpCode or data).

Pure Requests

A pure request seems to be used as a sort of 'ping' to see if the other
side of the connection is still there. In this case, ARQ, dwARQ, and
SpecARQ are set. When SpecARQ is set, this requests that a reponse
be sent immediately, without waiting for a timeout.


Packet Resends

If an ARQ is sent, then a ARSP is expected to be received in a certain
amount of time (again not sure but I think its 0.1 seconds or so). If
no ARSP is receive when the timeout occurs, the packet is resent.


Example Session

Here is an example of a session between the client and the login server.

(1) Client requests version information.
23:57:05.755261 client.1538 > server.10002: 14
0x0000 3200 0000 09b5 0100 5900 a361 6da7 2.......Y..am.

Flags set:
SEQStart - This is the start of the client sequence.
ASQ - Not quite sure yet.
ARQ - The client is request the client acknowledgement receipt of packet.

Header Data:
dwSEQ - 0x0000: The client is starting its sequence at 0x0000.
dwARQ - 0x09bd: Use this word as dwARSP when responding.
ASQ_high - 0x01: Not sure.
ASQ_low - 0x00: Not sure.
OpCode - 0x5900: Request Version.
CRC32 - 0xa3616da7: Used to check for errors in transmission.

(2) Server sends version information.
23:57:05.756179 server.10002 > client.1538: 32
0x0000 3204 0000 09bd 339a 0100 5900 322d 3234 2.....3...Y.2-24
0x0010 2d32 3030 3120 3133 3a31 3700 5bee b95a -2001.13:17.[..Z

Flags set:
SEQStart - This is the start of the server sequence.
ASQ - Not sure.
ARQ - The server is requesting ACK.
ARSP - This is a response to an ARQ sent by the client.

Header Data:
dwSEQ - 0x0000: The server is starting its sequence at 0x0000.
dwARSP - 0x09bd: Respond to ARQ 0x09bd.
dwARQ - 0x339a: Use this word as dwARSP when responding.
ASQ_high - 0x01: Not sure.
ASQ_low - 0x00: Not sure.
OpCode - 0x5900: Send Version.
CRC32 - 0x5beeb95a: CRC check.

Data:
This sends a null terminated string: "2-24-2001.13:17" as its version.


(3) Client sends login info.
23:57:05.842104 client.1538 > server.10002: 61
0x0000 1204 0001 339a 09be 0101 0100 4669 7a62 ....3..$....Fizb
0x0010 616e 3100 1db5 28f1 02a5 cde2 a513 23da an1...X2..b.V..!
0x0020 19d5 5dae b12d e6af e53b ed50 6e6f 6e65 ...O.,...e..none
0x0030 0000 0000 0000 0000 00e4 a6e1 e2 ...........m.

Flags set:
ASQ - Not sure.
ARQ - ACK Request.
ARSP - ACK Response.

Header Data:
dwSEQ - 0x0001: This is seqence number 0x0001 from the client.
Upon receiving, ignore any further packets <= dwSEQ.
dwARSP - 0x339a: Response to ARQ 0x339a.
dwARQ - 0x09be: Use this for responding.
ASQ_high - 0x01: Not sure.
ASQ_low - 0x01: Not sure.
dwOpCode - 0x0100: Send Login Info.
CRC32 - 0xe4a6e1e2: CRC Check.

Data:
The first thing sent is a null terminated username ("Fizban1" in this case).
After that, a 24-byte password hash is sent. The rest of the packet
doesn't seem to change between logins (not sure what 'none' is for).

(4) Server sends session id.
23:57:05.84957 server.10002 > client.1538: 33
0x0010 1204 0001 09be 339b 0101 0400 3132 3334 ......3.....1234
0x0020 3536 3738 3900 756e 7573 6564 00b9 85f1 56789.unused....
0x0030 98 .

Flags set:
ASQ - Not sure.
ARQ - ACK Request.
ARSP - ACK Response.

Header data:
dwSEQ - 0x0001: This is seqence number 0x0001 from the server.
Upon receiving, ignore any further packets <= dwSEQ.
dwARSP - 0x09be: Response to ARQ 0x09be.
dwARQ - 0x339b: Use this for responding.
ASQ_high - 0x01: Not sure.
ASQ_low - 0x01: Not sure.
dwOpCode - 0x0400: Send Login Info.
CRC32 - 0xb985f1: CRC Check.

Data:
The first thing sent is a string which contains the session id.
In this case it is "123456789". The string 'unused' is always
sent, it seems.


(5) Client sends pure acknowledement.
23:57:06.293296 client.1552 > xylor.10002: 10
0x0000 0004 0002 339b d470 fcf1 ....3..p..

Flags set:
ARSP - This is only a response to an ARQ.

Header Data:
dwSEQ - 0x0002: This is client sequence number 0x0002.
dwARSP - 0x339b: This is a response to ARQ 0x339b.
CRC32 - 0xd470fcf1: CRC Check.

dwARSP - 0x339a: Response to ARQ 0x339a.
dwARQ - 0x09be: Use this for responding.


(6) Client requests update.
23:57:06.578329 client.1552 > xylor.10002: 14
0x0000 1200 0003 09bf 0102 5200 2346 be93 ........R.#F..

Flags set:
ASQ - Not sure.
ARQ - ACK Request.

Header Data:
dwSEQ - 0x0003: This is client sequence number 0x0003.
dwARQ - 0x09bf: Use this for responding.
ASQ_high - 0x01: Not sure.
ASQ_low - 0x02: Not sure.
dwOpCode - 0x5200: Request update.
CRC32 - 0x2346be93: CRC Check.

Data:
N/A

(7) Client requests server list.
23:57:06.578861 client.1552 > xylor.10002: 18
0x0000 1200 0004 09c0 0103 4600 0000 0000 365f ........F.....6_
0x0010 e8ce ..

Flags set:
ASQ - Not Sure.
ARQ - ACK Request.

Header Data:
dwSEQ - 0x0004: This is client sequence number 0x0004.
dwARQ - 0x09bf: Use this for responding.
ASQ_high - 0x01: Not sure.
ASQ_low - 0x02: Not sure.
dwOpCode - 0x4600: Request server list.
CRC32 - 0x365fe8ce: CRC Check.

Data:
Not sure.


(8) Server sends update.
23:57:06.586646 xylor.10002 > client.1552: 47
0x0000 1204 0002 09bf 339c 0102 5200 0100 0000 ......3...R.....
0x0010 4f70 656e 5175 6573 7420 456d 756c 6174 OpenQuest.Emulat
0x0020 6f72 202d 2058 796c 6f72 0076 07fb 58 or.-.Xylor.v..X

Flags set:
ASQ - Not sure.
ARQ - ACK Request.
ARSP - ACK Response.

Header Data:
dwSEQ - Server sequence number 0x0002.
dwARSP - Responding to ARQ 0x09bf.
dwARQ - Use this for responding.
ASQ_high - 0x01: Not sure.
ASQ_low - 0x02: Not sure.
dwOpCodes - 0x5200: Send update.
CRC32 - 0x7607fb58: CRC Check.

Data:
First 32-bits seems to be flags of some sort. After that is a
null terminated Banner that is shown on the server selection screen.

(9) Server sends server list.
23:57:06.587935 xylor.10002 > client.1552: 63
0x0000 1204 0003 09c0 339d 0103 4600 0100 0000 ......3...F.....
0x0010 4f70 656e 5175 6573 7420 5465 7374 2057 OpenQuest.Test.W
0x0020 6f72 6c64 006f 7065 6e71 7565 7374 2e64 orld.openquest.d
0x0030 6873 2e6f 7267 00e8 0300 007f 93b9 cd hs.org.........

Flags set:
ASQ - Not sure.
ARQ - ACK Request.
ARSP - ACK Response.

Header Data:
dwSEQ - 0x0003: Server sequence number 0x0003.
dwARSP - 0x09c0: Reponding to ARQ number 0x09c0.
dwARQ - 0x339d: Use this for responding.
ASQ_high - 0x01: Not sure.
ASQ_low - 0x03: Not sure.
dwOpCode - 0x4600: Send server list.
CRC32 - 0x7f93b9cd: CRC Check.

Data:
A list of servers to list on the selection screen. The chat server
also seems to be sent in this packet. The format seems to be
32-bits first (number of people on server?), null terminated
server-name string, null terminated hostname, and 32-bit value (flags?).

(10) Client sends pure acknowledement.
23:57:06.633771 client.1552 > xylor.10002: 10
0x0000 0004 0005 339d 385c 4f41 ....3.8\OA

Flags set:
ARSP - ACK Response only.

Header data:
dwSEQ - Client sequence number 0x0005.
dwARSP - Responding to ARQ 0x339d.
CRC32 - 0x385c4f41: CRC Check.

Data:
N/A

[purple]

That's the original protocol BA. Too old.

[BlueAdept]

Ahh.. Well I wasn't sure. I was hoping it might be of some help. C'est la vie.

[BlueAdept]

I start vacation at the end of the day. I should have some time to update the file section with the patch for 1999 in the next day or 2.

[ohhello]

i worked around the other crash bugs ive run into, which includes logging and exp-based crashes. I also fixed the exp window so it will calculate exp/hour and such. I started adding in support to specify the key at the command line for starting a session without logging all the way out but that was back burner.

putting the opcodes to decrypt into the xml makes good sense, perhaps ill look into it down the road.


http://pastebin.com/raw.php?i=5EfBiFuH

(note, this would be applied after the previous one)

[Someone]

Thanks for continuing to work on this! I am having an issue with this last patch, any ideas?

patching file src/interface.cpppatching file src/main.cpp
patching file src/packetstream.cpp
patch: **** malformed patch at line 73: diff --git a/src/player.cpp b/src/player.cpp

Line 73 looks to be the start of your exp changes.

Ive tried applying with fresh 5.2.2 src after applying your previous patch (and before), with no luck.

Thanks again.

[iluvseq]

Thank you very much for the hard work on this. Still a few issues however. Corpses show up as the wrong thing (blue + instead of yellow squares) and session tracking doesn't work. Does anyone have some pointers on where to start looking to fix those issues?

[Toxn]

Hey all. Thanks for the hard work! Is ShowEQ currently working on p99? Whats the status on dsetup.dll? I was banned about a year ago for ShowEQ use and would like to avoid if possible.

[Ben8472]

sorry but i highly doubt that you were banned for using ShowEQ which is not even running on your windows box.

[Glycerine]

Any updates here with anyone currently able to get it working in some fashion on p1999? Tried using all the instructions/patches from all available thread on the forums but getting the garbled NPC list and program crash on every Zone.

[typhoon]

If you are trying it with the recent v39 patch for the release of Velious, it will not work until someone can decode what the op codes are again. If you read through the thread you will see the info on how they did it in the past. But look at the new dsetup.dll in the new patch, it has become even bigger and may not be some easy to figure out.

[rasthan]

Anybody happen to have the updated opcodes yet? :D

[VallonZeker]

Any updated opcode for the current version on P1999 ?

[Darkstar]

Bump. Playing EQ again on p1999 and missing this terribly. Thanks to all you guys for the work you've done on this over the years!

[awdt321]

Bump! Also would love this to work with current P1999 version. Any way we can help?

[Fizzl]

Bump!

Does anyone have a tarball of the latest effort with all known patches? One of the patches has vanished from the hosting.

I'm willing to work on this. I used to use ShowEQ back in the good old days.

[vash23]

https://ufile.io/atc6k works for me. Don't click on or /target anything that is not really there. They have monitoring setup for that (like the "trees have no need for money" in Nro or the "SEQ Users Suck" invis NPCs in Hate). Not sure how else they are monitoring, but they have found and banned accounts for using MySEQ (ShowEQ as they call it).

[BlueAdept]

Yea, showeq is not detectable. There are a few exceptions. If you run SEQ in vmware (or similar) they can detect those processes (People have been banned from EQ for running vmware). Rumor also has it that people have been banned from EQ just because their network adapter was in promiscuous mode. I still have a hard time believing that anyone has been banned for those two things, but people claim they have been.
MySEQ, on the other hand, is easily detected.

PLEASE NOTE: Everyone please be careful, while I appreciate vash for linking that, we can not guarantee the file he posted and would not suggest that anyone use that version as to the unknown origin of the files. It could be malware, worse or even an account stealer.

[Jar Mason]

https://ufile.io/atc6k works for me. Don't click on or /target anything that is not really there. They have monitoring setup for that (like the "trees have no need for money" in Nro or the "SEQ Users Suck" invis NPCs in Hate). Not sure how else they are monitoring, but they have found and banned accounts for using MySEQ (ShowEQ as they call it).

Looks like this link is dead, anyone happen to have the latest patches?

[grump]

Looks like this link is dead, anyone happen to have the latest patches?

Also wanted to see if anyone had a link for the latest patches or at least the one that dissapeared earlier in this thread. I'm stuck on trying to work around decrypting the spawn info now and not very good at C++ but happy to share if I can get it figured out. I've read all the medium posts about it by Eratosthenes and gotten it to build only all the spawn info is still decrypted and haven't been able to do the following that's required per https://medium.com/@Packet99/decrypting-project1999-spawns-7248acb1797b

Modify the ShowEQ Configuration Op Codes document to expect the additional data.
Modify the ‘struct’ to contain the decryption key.
Store the key in memory for the session and update when necessary.
Modify the spawnStruct handling functions to decrypt the entire packet before further processing.