Segment Faults - after XML Opcode update - 6/16/04 patch

[wiz60]

I have noticed a sharp increase in segment faults after making the changes to the XML opcode file.

It's a bit early - but seems to be related to respawn events.

Anyone else have the issue?

[h0bbit]

same here ~

[Hobo]

Well since I used both the cvsdiff.patch and the cvs.patch I haven't had a single seg fault. Granted I've only played about 6 hours since applying both patches, but no problems so far.

[reaver]

No segfaults last night after updating the zoneopcodes.xml file.

Was on about 7 hours

-reaver

[mudtoe]

I just started getting seg faults all of a sudden. Have been playing off and on with the 6/16 xml updates and everything was fine until this afternoon. Had been playing for a couple of hours when all of a sudden it seg faulted. Everytime I restart it and zoned it seg faults again within 30 seconds. Tried about five times and everytime it crashed. Also tried completely leaving the game and coming back, and after less than 30 seconds it seg faulted again.

Not sure what's going on now. The only odd message I'm getting in the terminal window is the one that other's have talked about concerning OP_Spawndoor. Can't figure out why it would work all this time and suddenly start seg faulting constantly.


mudtoe

[Cryonic]

Well, due to the lack of useful information in this post it is very hard to help fix what is wrong...

run showeq with gdb and post a backtrace, else, NOTHING WILL BE DONE TO HELP.

[RebelWithCause]

I am having the same problem. I have been running fine since the last patch with the zone opcode changes until today. Suddenly, I'm getting seg faults every time I run SEQ.

Here is a backtrace:

#0 calcCRC32 (p=0xc0000000 <Address 0xc0000000 out of bounds>,
length=4294401191) at util.cpp:960
#1 0x0808bd8f in EQPacket::dispatchPacket(int, unsigned char*) (
this=0x954a320, size=46, buffer=0xbff75c8e "E") at packetformat.h:255
#2 0x0808ba15 in EQPacket::processPackets() (this=0x954a320) at packet.cpp:474
#3 0x0808d9c9 in EQPacket::qt_invoke(int, QUObject*) (this=0x954a320, _id=2,
_o=0xbff77d40) at m_packet.cpp:518
#4 0x03bacdd7 in QObject::activate_signal(QConnectionList*, QUObject*) ()
from /usr/lib/qt-3.1/lib/libqt-mt.so.3
#5 0x03bacc7d in QObject::activate_signal(int) ()
from /usr/lib/qt-3.1/lib/libqt-mt.so.3
#6 0x03e96fbb in QTimer::timeout() () from /usr/lib/qt-3.1/lib/libqt-mt.so.3
#7 0x03bcdad2 in QTimer::event(QEvent*) ()
from /usr/lib/qt-3.1/lib/libqt-mt.so.3
#8 0x03b4e124 in QApplication::internalNotify(QObject*, QEvent*) ()
from /usr/lib/qt-3.1/lib/libqt-mt.so.3
#9 0x03b4d902 in QApplication::notify(QObject*, QEvent*) ()
from /usr/lib/qt-3.1/lib/libqt-mt.so.3
#10 0x03b28c35 in QEventLoop::activateTimers() ()
from /usr/lib/qt-3.1/lib/libqt-mt.so.3
#11 0x03b0676d in QEventLoop::processEvents(unsigned) ()
from /usr/lib/qt-3.1/lib/libqt-mt.so.3
#12 0x09477070 in ?? ()
#13 0x094771f4 in ?? ()
#14 0x09477378 in ?? ()
#15 0x03fbd37c in qt_wait_timer_max () from /usr/lib/qt-3.1/lib/libqt-mt.so.3
#16 0x003af838 in __libc_ptyname2 () from /lib/tls/libc.so.6
#17 0x0028a428 in ?? () from /lib/tls/libc.so.6
#18 0xbf5544e8 in ?? ()
#19 0x095490a8 in ?? ()
#20 0x0000000d in ?? ()
#21 0x003bc998 in __DTOR_END__ () from /lib/tls/libc.so.6

Thank you in advance for any help. If there's any additional info I can provide, let me know.

[Cryonic]

Looks like something from the past has reared its ugly head:
http://seq.sourceforge.net/forums/showthread.php?s=&threadid=4604&highlight=0xc0000000

From reading through what Zaphod has said there, the issue isn't with SEQ per se, but with something on the network (namely the size of the packet that SEQ is getting seems to be larger than it could ever truly be...)

[RebelWithCause]

Thanks for the info Cryonic. After reading your post and doing some thinking, I realized that I had been playing with a freeware SIP softphone on one of my other machines, and left it running. I think it must have been broadcasting some kind of discovery packet that was crashing SEQ, because after I shut it down everything works again.

Mudtoe, it's gotta be something else on your network sending packets.

Thanks again. :)

[mudtoe]

Hmmm.... That's a thought. I've got a ReplayTV on a lan segment that connects to the main network through the same hub that ShowEQ and my EQ machine share. It's hooked up using one of those Netgear phone line bridges, and I had it going into the hub because its max speed was only 10mb, and I didn't want to waste a 100mb full duplex port on the main switch when I could use one of half duplex ports on the hub. I didn't check to see if the light on the bridge was flashing (indicating activity through the bridge) when ShowEQ crashed.

If this is indeed the problem, it's awful strange that we both had the same thing happen on the same day. Also, if this is indeed the problem, doesn't it indicate that there is a problem in the order that ShowEQ does its processing? By that I mean even if there is a packet too large for ShowEQ to deal with, shouldn't it have not been trying to disect it in the first place, other than just the IP header, because it wasn't from the IP address of the machine being monitored (I have ShowEQ setup to monitor a specific IP address rather than looking for the next client seen).

mudtoe

[Cryonic]

Until you get a backtrace of SEQ crashing you can speculate all you want on the why's, whatnot's and wherefore's.

I see 5 IDs complaining of crashing, but only one that posted anything useful to help figure out why.

[reaver]

Crashes often happen from someone not updating to the latest eqstr_us.txt & spells_us.txt files from the patcher.

-reaver

[mudtoe]

Well, the problem wasn't the ReplayTV on the same segment. I took everything off the hub except the EQ machine and the ShowEQ machine. Worked fine for a day, and all of a sudden it started crashing again every few minutes. Here's a trace that I managed to collect:


#0 EQPacketStream::decodePacket(unsigned char*, unsigned, unsigned short) (
this=0x82a6710, data=0x0, len=5, opCode=23808) at packetstream.cpp:528
#1 0x08086e39 in EQPacketStream::processPayload(unsigned char*, unsigned) (
this=0x82a6710, data=0xbfffc454 "\b?\004", len=65) at packetstream.cpp:629
#2 0x08087528 in EQPacketStream::handlePacket(EQUDPIPPacketFormat&) (
this=0x82a6710, packet=@0xbfffc3c0) at packetformat.h:264
#3 0x0809107f in EQPacket::dispatchPacket(int, unsigned char*) (
this=0x82b8308, size=1356, buffer=0xbfffc42e "E") at packet.cpp:658
#4 0x08090e29 in EQPacket::processPackets() (this=0x82b8308) at packet.cpp:474
#5 0x08092ff5 in EQPacket::qt_invoke(int, QUObject*) (this=0x82b8308, _id=2,
_o=0xbfffe4e0) at m_packet.cpp:518
#6 0x4027c3b5 in QObject::activate_signal(QConnectionList*, QUObject*) ()
from /usr/local/qt/lib/libqt-mt.so.3
#7 0x4027c1b5 in QObject::activate_signal(int) ()
from /usr/local/qt/lib/libqt-mt.so.3
#8 0x4059e17c in QTimer::timeout() () from /usr/local/qt/lib/libqt-mt.so.3
#9 0x4029cc3f in QTimer::event(QEvent*) ()
from /usr/local/qt/lib/libqt-mt.so.3
#10 0x4022349d in QApplication::internalNotify(QObject*, QEvent*) ()
from /usr/local/qt/lib/libqt-mt.so.3
#11 0x40222b80 in QApplication::notify(QObject*, QEvent*) ()
from /usr/local/qt/lib/libqt-mt.so.3
#12 0x4021230f in QEventLoop::activateTimers() ()
---Type <return> to continue, or q <return> to quit---
from /usr/local/qt/lib/libqt-mt.so.3
#13 0x401d44b4 in QEventLoop::processEvents(unsigned) ()
from /usr/local/qt/lib/libqt-mt.so.3
#14 0x40234bd7 in QEventLoop::enterLoop() ()
from /usr/local/qt/lib/libqt-mt.so.3
#15 0x40234a94 in QEventLoop::exec() () from /usr/local/qt/lib/libqt-mt.so.3
#16 0x402236fc in QApplication::exec() () from /usr/local/qt/lib/libqt-mt.so.3
#17 0x08067c7b in main (argc=2, argv=0xbffff5f4) at main.cpp:688
#18 0x420158f7 in __libc_start_main () from /lib/i686/libc.so.6



This seems to run in spurts for some reason. I can play for hours with no problem, but once it starts crashing it just keeps doing it over and over. If it's a bad packet length thing again, then whatever is causing it is running in the background on one of the two machines. WinXP has lots of services running, and I had some network shares open, but I don't think anything was running that hadn't been running in the past all along.

[UnGod]

#0 EQPacketStream::decodePacket(unsigned char*, unsigned, unsigned short) (
this=0x82a6710, data=0x0, len=5, opCode=23808) at packetstream.cpp:528

data=0x0...

On line 528 we see:

if (dptr[0] == 0xff) {

Looking up a bit, we see:

uint8_t *dptr = data;

Trying to view a specific position of a null value is generally a bad thing.. ;)

I do not know the decoding part of SEQ well, so I can only give you a possible temporary fix for this, and not actually shed much light on the true cause.

On line 477 of packetstream.cpp, we see the following lines:

if (opCode & FLAG_CRYPTO || opCode & FLAG_COMP)
{
data = decodeOpCode (data, &len, opCode);
if (data == NULL)
return;
}

And thus it seems not hitting the "if (data == NULL)" line due to not being flagged encrypted/compressed.

I would assume this traffic is from something other than EQ.

So change that block of code starting at line 477 from the above to:



if (opCode & FLAG_CRYPTO || opCode & FLAG_COMP)
{
data = decodeOpCode (data, &len, opCode);
}
if (data == NULL)
return;


This is just a quick hack, it does NOT solve the issue, only will keep you from crashing atleast ;)

PS-I cannot guarentee that will even keep you from crashing, or that it won't make it happen more, but for the trace you posted, it wouldn't have crashed if the code looked like that.

PSS-This is directed towards mudtoe's trace.

Enjoy,
Belith/UnGod

[mudtoe]

I played for several hours today with your source code change, and so far it hasn't crashed at all. I'm going to put the ReplayTV back on the hub and leave ShowEQ running all night to see if it crashes. Before, if I left it running without an EQ session going, it would frequently crash overnight, so if it makes it till morning, that's a good sign.

Thanks for your code change.


mudtoe

[mudtoe]

Well, it ran for over 24 hours without crashing, so that's a big improvement. The only thing which did make it crash was starting EQIM on the EQ machine, and as I recall it always crashed when you started EQIM. I'm including the trace for the EQIM crash here, as there is obviously something wrong that is making it crash, but it's not a big deal as long as EQIM is the only thing that can cause it. Thanks for the code change Ungod.


mudtoe

-----------------------------------

EQIM Crash Data:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 16384 (LWP 2682)]
calcCRC32 (p=0xc0000000 <Address 0xc0000000 out of bounds>, length=4294947143)
at util.cpp:960
960 crc = crctab[(crc ^ *(p++)) & 0xFF] ^ (crc >> 8);
(gdb) bt
#0 calcCRC32 (p=0xc0000000 <Address 0xc0000000 out of bounds>,
length=4294947143) at util.cpp:960
#1 0x080911ef in EQPacket::dispatchPacket(int, unsigned char*) (
this=0x81ebd80, size=-534054926, buffer=0xbfffb12e "E")
at packetformat.h:255
#2 0x08090e39 in EQPacket::processPackets() (this=0x82f7728) at packet.cpp:474
#3 0x08093005 in EQPacket::qt_invoke(int, QUObject*) (this=0x82f7728, _id=2,
_o=0xbfffd1e0) at m_packet.cpp:518
#4 0x4027c3b5 in QObject::activate_signal(QConnectionList*, QUObject*) ()
from /usr/local/qt/lib/libqt-mt.so.3
#5 0x4027c1b5 in QObject::activate_signal(int) ()
from /usr/local/qt/lib/libqt-mt.so.3
#6 0x4059e17c in QTimer::timeout() () from /usr/local/qt/lib/libqt-mt.so.3
#7 0x4029cc3f in QTimer::event(QEvent*) ()
from /usr/local/qt/lib/libqt-mt.so.3
#8 0x4022349d in QApplication::internalNotify(QObject*, QEvent*) ()
from /usr/local/qt/lib/libqt-mt.so.3
#9 0x40222b80 in QApplication::notify(QObject*, QEvent*) ()
from /usr/local/qt/lib/libqt-mt.so.3
#10 0x4021230f in QEventLoop::activateTimers() ()
from /usr/local/qt/lib/libqt-mt.so.3
#11 0x401d44b4 in QEventLoop::processEvents(unsigned) ()
from /usr/local/qt/lib/libqt-mt.so.3
#12 0x40234bd7 in QEventLoop::enterLoop() ()
from /usr/local/qt/lib/libqt-mt.so.3
#13 0x40234a94 in QEventLoop::exec() () from /usr/local/qt/lib/libqt-mt.so.3
#14 0x402236fc in QApplication::exec() () from /usr/local/qt/lib/libqt-mt.so.3
#15 0x08067c7b in main (argc=2, argv=0xbfffe2f4) at main.cpp:688
#16 0x420158f7 in __libc_start_main () from /lib/i686/libc.so.6

[mudtoe]

<sigh>


All of a sudden I'm starting to get a new segfault on only one character (my main one of course). It happens everytime this character enters the game or zones if I started SEQ with char already in the game, after most of the zoning stuff is complete right after it shows what your character ID in the zone is. All my other characters can get in and use it just fine, so I'm assuming I must have done something with my inventory or something else with the attributes of the character that SEQ doesn't like anymore. Here's the trace data. I've included the SEQ stuff before the fault so you can see where in the zoning process it happens. It appears from what I can gather from the trace is that the problem has something to do with guildtags. In a way it makes sense because my other characters aren't in a guild. Is there anyway to turn off SEQ's processing of guild related packets? I really don't use the tags anyway. I tried turning off some of the guild displays on the spawnlist, but that didn't fix it. I'm tempted to go into the zoneopcodes file and comment out the ones related to guild stuff as an interrim fix.

======================================

Q Client 192.168.0.250
Info: Loaded map: '/usr/local/share/showeq/maps/Potranquility.map'
Info: Loading Zone Filter File: /usr/local/share/showeq/filters/potranquility.xml
Zone: Zoning, Please Wait... (Zone: 'potranquility')
Player: Name: 'xxxxxxxx' Last: 'xxxxxxxx'
Player: Level: 65
Player: PlayerMoney: P=10 G=31 S=2 C=3
Player: BankMoney: P=0 G=1 S=3 C=7
Player: CursorMoney: P=0 G=0 S=0 C=0
Player: SharedMoney: P=138555
Player: Exp: 1,175,247,146
Player: ExpAA: 120
Player: You have buff Force Shield duration left is 742 in ticks.
Player: You have buff Shield of Maelin duration left is 614 in ticks.
Player: You have buff Blessing of Reverence duration left is 152 in ticks.
Player: You have buff Dead Man Floating duration left is 432 in ticks.
Player: You have buff Virtue duration left is 1254 in ticks.
Player: You have buff 141a duration left is 98 in ticks.
Debug: charProfile(774.000000/-1463.000000/-878.000000 - 131.000000)
Debug: Player::backfill(): Pos (774.000000/-1463.000000/-878.000000) Heading: 131.000000
Debug: Player::backfill(bind): Pos (278.037292/353.624573/-119.757477) Heading:
0.000000
Player: ExpAA: Set: 120 total, with 0 aapoints
Time: Mon Jul 25,3183 - 01:10 am
Warning: OP_SpawnDoor (0x291) (dataLen: 86313) doesn't match: modulus of sizeof(doorStruct):64
Debug: Welcome to lovely downtown 'The Plane of Tranquility' with an experience
multiplier of 1.000000
Debug: Safe Point (-1507.000000, 701.000000, -878.000000)
Info: Loading Zone Filter File: /usr/local/share/showeq/filters/potranquility.xml
Zone: Entered: ShortName = 'potranquility' LongName = The Plane of Tranquility
Zone: NewCode: Zone: PoTranquility (The Plane of Tranquility)
Player: Exp: Set: 1175247146 total, with 79485226 (137/330) into level with 33102854 left, where 1/330 = 341176
Info: Your player's id is 464

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 16384 (LWP 6928)]
0x405455ca in QString::fromUtf8(char const*, int) ()
from /usr/local/qt/lib/libqt-mt.so.3
(gdb) bt
#0 0x405455ca in QString::fromUtf8(char const*, int) ()
from /usr/local/qt/lib/libqt-mt.so.3
#1 0x081969b3 in NetStream::readText() (this=0xbfffb960) at netstream.cpp:154
#2 0x08196c87 in GuildMember (this=0x8495778, netStream=@0xbfffb960)
at guildshell.cpp:32
#3 0x08197ee6 in GuildShell::guildMemberList(unsigned char const*, unsigned) (
this=0x82abc68, data=0x8234e20 "xxxxxxxx", len=10328) at guildshell.cpp:170
#4 0x0819874f in GuildShell::qt_invoke(int, QUObject*) (this=0x82abc68,
_id=2, _o=0xbfffba30) at /usr/local/qt/include/private/qucom_p.h:312
#5 0x4027c3b5 in QObject::activate_signal(QConnectionList*, QUObject*) ()
from /usr/local/qt/lib/libqt-mt.so.3
#6 0x0808edd5 in EQPacketDispatch::signal(unsigned char const*, unsigned, unsigned char) (this=0x844c7d8, t0=0x8234e20 "xxxxxxxx", t1=10328, t2=2 '\002')
at m_packetinfo.cpp:99
#7 0x08086f50 in EQPacketStream::dispatchPacket(unsigned char const*, unsigned, unsigned short, EQPacketOPCode const*) (this=0x82b0e60,
data=0x8234e20 "xxxxxxxx", len=10328, opCode=89, opcodeEntry=0x82b44b8)
at packetstream.cpp:680
#8 0x08086d79 in EQPacketStream::decodePacket(unsigned char*, unsigned, unsigned short) (this=0x82b0e60, data=0x8234e20 "xxxxxxxx", len=10328, opCode=89)
at packetinfo.h:300
#9 0x08086a52 in EQPacketStream::processFragment(EQPacketFormat&) (
this=0x82b0e60, pf=@0xbfffbc40) at packetstream.cpp:629
---Type <return> to continue, or q <return> to quit---
#10 0x08087544 in EQPacketStream::handlePacket(EQUDPIPPacketFormat&) (
this=0x82b0e60, packet=@0xbfffbc40) at packetstream.cpp:811
#11 0x0809108f in EQPacket::dispatchPacket(int, unsigned char*) (
this=0x8304170, size=1391, buffer=0xbfffbcae "E") at packet.cpp:658
#12 0x08090e39 in EQPacket::processPackets() (this=0x8304170) at packet.cpp:474
#13 0x08093005 in EQPacket::qt_invoke(int, QUObject*) (this=0x8304170, _id=2,
_o=0xbfffdd60) at m_packet.cpp:518
#14 0x4027c3b5 in QObject::activate_signal(QConnectionList*, QUObject*) ()
from /usr/local/qt/lib/libqt-mt.so.3
#15 0x4027c1b5 in QObject::activate_signal(int) ()
from /usr/local/qt/lib/libqt-mt.so.3
#16 0x4059e17c in QTimer::timeout() () from /usr/local/qt/lib/libqt-mt.so.3
#17 0x4029cc3f in QTimer::event(QEvent*) ()
from /usr/local/qt/lib/libqt-mt.so.3
#18 0x4022349d in QApplication::internalNotify(QObject*, QEvent*) ()
from /usr/local/qt/lib/libqt-mt.so.3
#19 0x40222b80 in QApplication::notify(QObject*, QEvent*) ()
from /usr/local/qt/lib/libqt-mt.so.3
#20 0x4021230f in QEventLoop::activateTimers() ()
from /usr/local/qt/lib/libqt-mt.so.3
#21 0x401d44b4 in QEventLoop::processEvents(unsigned) ()
from /usr/local/qt/lib/libqt-mt.so.3
#22 0x40234bd7 in QEventLoop::enterLoop() ()
---Type <return> to continue, or q <return> to quit---
from /usr/local/qt/lib/libqt-mt.so.3
#23 0x40234a94 in QEventLoop::exec() () from /usr/local/qt/lib/libqt-mt.so.3
#24 0x402236fc in QApplication::exec() () from /usr/local/qt/lib/libqt-mt.so.3
#25 0x08067c7b in main (argc=2, argv=0xbfffee74) at main.cpp:688
#26 0x420158f7 in __libc_start_main () from /lib/i686/libc.so.6
(gdb)

[mudtoe]

OK. I got it to work by removing the guildmemberlist opcode and the updateguildmember opcode from zoneopcodes.xml. Obviously this isn't a real fix, but in lieu of knowing what's really wrong, it was the only thing I could think of. Also have no idea why it started happening all of a sudden. Only thing I can think of is that something changed in the list itself, although we aren't at our alltime high in regards to number of members, so the size of the guild shouldn't be a problem. Maybe somebody put a really strange character in one of their character descriptions.