New Netcode Revision 4

[purple]

Ok, here's my last drop hopefully of new netcode patches for testing. Again, this is against latest CVS. Please check this out and if all seems well, this will go into CVS. This finished my todo list, except opcodes that I'm not horribly motivated to work on any longer ;)

New in this one:
1) Session Tracking is implemented. If your seq seems to not be picking up anything, you might have session tracking on and it got confused and locked on the wrong port. To fix this, go to Network->Monitor Next EQ Client seen and then zone. That should pick things back up. If there's any weirdness in Session Tracking, or if you needed Session Tracking before, but it doesn't work for you now, please post! I did my best to make it work, but I've never used it before so....

2) CRC checking is implemented. CRC checking under the new protocol is session based, so you might see CRC error messages when session responses get messed up. It doesn't mean you have a bad netcard, it just means that seq doesn't know the correct session key in order to check the CRC. Under normal circumstances, you shouldn't see CRC errors. If things get really screwed up, zoning starts a new session which should help. I did my best to suppress the errors pre-session initialization.

3) More opcodes done. I'd wager more opcodes are correct now than before 1/26 really.

4) Updated version to 5.0.0.17, updated Changelog, News.

As usual, please post any problems you see. Hopefully we can get them sorted out. If you are getting random fragments overwriting buffers, try session tracking first, please.

Attached: 02082005.patch.gz.txt (remove .txt, gzipped patch)

[bonkersbobcat]

Committed to cvs as version 5.0.0.17

Please make any further patches from this version.

Note that for non-developers using anonymous cvs (almost everyone here), It takes a day before these changes are visible. (The anonymous cvs is a delayed batch job copy of the "real" cvs.)

Thanks everyone for the hard work!

BB

[Mikey]

Patched and ran fine. I had skittles for about 5 mins then got:

Warning: !!!! EQPacketFragmentSequence::addFragment(): buffer overflow adding in new fragment to buffer with seq 0264 on stream 3, opcode 0164. Buffer is size 103392 and has been filled up to 103074, but tried to add 506 more!

I'm not sure if it's related or not, but just before this occurred, I got:

Debug: SEQ: Giving up on finding arq 010b in stream zone-client cache, skipping!
Debug: SEQ: Giving up on finding arq 0119 in stream zone-client cache, skipping!
Debug: SEQ: Giving up on finding arq 011c in stream zone-client cache, skipping!
Debug: SEQ: Giving up on finding arq 011d in stream zone-client cache, skipping!
Debug: SEQ: Giving up on finding arq 011e in stream zone-client cache, skipping!


Also, I received a CRC error every time I zoned.

Warning: INVALID PACKET: Bad CRC [199.108.4.155:9000 -> 192.168.1.1:3175] netOp 0700 seq e88f len 42 crc (be93 != 16b7)

This only occurred once each time I zoned. The netOp and len were the same every time, the other values changed from zone to zone.

The bad CRC showed up right after

Info: EQPacket: SEQClosing detected, awaiting next zone session, pcap filter: EQ Client 192.168.1.1

but before the MOTD.

Also, I'd like to give a BIG thank you to purple for taking this on and giving us our skittles back. :D

[purple]

Warning: !!!! EQPacketFragmentSequence::addFragment(): buffer overflow adding in new fragment to buffer with seq 0264 on stream 3, opcode 0164. Buffer is size 103392 and has been filled up to 103074, but tried to add 506 more!

I'm not sure if it's related or not, but just before this occurred, I got:

Debug: SEQ: Giving up on finding arq 010b in stream zone-client cache, skipping!
Debug: SEQ: Giving up on finding arq 0119 in stream zone-client cache, skipping!
Debug: SEQ: Giving up on finding arq 011c in stream zone-client cache, skipping!
Debug: SEQ: Giving up on finding arq 011d in stream zone-client cache, skipping!
Debug: SEQ: Giving up on finding arq 011e in stream zone-client cache, skipping!


When you see these, it means that it somehow missed the last packet in an oversized packet. My first guess at why this would be happening is that you have two EQ machines and it is getting confused between them, seeing packets for the other machine and trying to cache packets until those packets make sense. This causes the cache to fill up with packets it shouldn't be filling up with. Once the cache gets over 1024 packets, it starts giving up on seeing the packets it needs.

So you're either seeing packets really really out of order (this can happen I'd guess while zoning when packets come fast and furious) or you're getting cross talk between two machine (session tracking might help this).

I haven't seen this at all, but if you can make it consistently happen, it would help if you would turn on Networ->Log->Zone Log and Raw Log and then somehow send that to me. Maybe pm me an URL or get on IRC. There are a handful of people reporting this, but I haven't reproduced it yet. So this is all just my best guess.


Warning: INVALID PACKET: Bad CRC [199.108.4.155:9000 -> 192.168.1.1:3175] netOp 0700 seq e88f len 42 crc (be93 != 16b7)

This only occurred once each time I zoned. The netOp and len were the same every time, the other values changed from zone to zone.

That looks like an out of order packet which is for the previous session but comes after the session has received the key to the new session. You can safely ignore that (0700 is network status - not important, it actually gets dropped right after the crc check!).

[tanner]

Committed to cvs as version 5.0.0.17

Stripped out and completely(?) replaced the network code and only get a minor-point release increase?

At minimum I'd think this would be 5.0.1, if not 5.1

[quackrabbit]

Anonymous CVS now has 5.0.0.17

[purple]

I got no control over versioning, Tanner. I just did what BB said he was gonna do when I was done. If you really think it's a problem, start another thread and hopefully a real dev will chime in *grin*

[Snoochie]

Is there some trick to using the CVS?

I haven't had a connection that didn't time out in over six months, and only one-in-a-hundred that didn't time out over the past couple years. Normally it isn't a big deal because manually patching isn't too hard. This looks like it would be a very tedious edit though.

[purple]

Yeah, the secret is to use the correct CVSROOT.

[Snoochie]

From http://www.showeq.net/forums/faq.php?


Where/How do I download ShowEQ?
Make sure you are logged in as the user 'root'. From a terminal window type the following, pressing the return key when prompted for a password (it is blank):
# cd /
# cvs -d:pserver:[email protected]:/cvsroot/seq login
Logging in to :pserver:[email protected]:2401/cvsroot/seq
CVS password:
# cvs -d:pserver:[email protected]:/cvsroot/seq co showeq
This will download the source and maps for ShowEQ into the directory "showeq" where ever you ran the command from (usually ~/).

What do I need to do differently?

EDIT: Nevermind. I'd already run into and fixed this problem before and found my notes file. For anyone else trying to use the FAQ's instructions:

cvs -d :pserver:[email protected]:/cvsroot/seq login

[purple]

s/cvs\.seq/cvs/

[eqhunter]

Anonymous CVS now has 5.0.0.17


sorry guys but cant find it :confused:

anyone have a link.. its not here here (http://sourceforge.net/project/showfiles.php?group_id=10131)
or here (http://cvs.sourceforge.net/viewcvs.py/seq/#dirlist)
sorry if Im being rude for posting here in the dev area

hunter

[uRit1u2CBBA=]

Keep digging -- the steps for pulling pre_5_beta code from cvs has been posted at least twice in the last week.

[eqhunter]

Keep digging -- the steps for pulling pre_5_beta code from cvs has been posted at least twice in the last week.


ok I know where that is but I thought it would have been named 5.0.0.17

[Leetlewon]

export CVSROOT=:pserver:[email protected]:/cvsroot/seq
cvs login
<return>
cvs co -r pre_5_0_beta showeq

[Fatal]

Hey purple..

DO you have a paypal account?

[purple]

I fixed stuff up because I wanted seq to work and had the time to fix it, not for money. Thanks though.

[Leetlewon]

Not really sure if this should go in the help forum instead, but it seems associated to the new code so I'm posting here.

I got everything compiled and working - skittles correct, zoned and loaded maps fine. Then, showeq crashed with the message:

Warning: !!!! EQPacketFragmentSequence::addFragment(): buffer overflow adding in new fragment to buffer with seq 028f on stream 3, opcode 0164. Buffer is size 115114 and has been filled up to 114884, but tried to add 506 more!

[Leetlewon]

OK I now see the "cann" has seen the same exact error in previous revisions.

[Dedpoet]

Let me preface this by saying that I am not a network code guy - not with respect to Seq or otherwise. That being said, I think this bug may be related to network latency. On my server, and presumably others, something is weird with Natimbi specifically, as was mentioned in another thread. In that zone I get ridiculous packet loss, spell lag, and zoning into KT takes literally 3+ minutes since a couple of patches ago. Seq crashes on me with an overflow error more often than not while in that zone.

I have also been getting overall latency issues with my ISP, which has been solid for the past 2 years until about 3 weeks ago, when it has suddenly gone to shit. It's not just EQ, it's everything. When my latencies spike high, Seq invariably crashes with an overflow error. I think it was purple who mentioned that if Seq misses an expected packet, it starts buffering the incoming data, waiting for the lost one. If latency gets high enough, that buffer could easily fill up before the missing packet is ever received - boom.

The ISP issues seem to be prety broad, as some of my rl friends who I play with are getting them too. Whatever is going on with Natimbi is unrelated, but seems to be causing the same end result. I know it's not exactly a solution, but maybe this will provide some insight.

Edit: I didn't want to do a whole new post for this, but I just wanted to say thanks to purple, and to the others who have helped get Seq back on its feet. You guys are fantastic.

[BlueAdept]

This is a shot in the dark, try it with --itemdb-disable switch. See if that helps.

[purple]

Compression was turned off when the new netcode went live, which hurts raw packet throughput a lot, especially during zoning when your computer is being hammered mericilessly with 512 byte packets over and over and over again.

Compression is turned back on on test. I'm hoping this fixes the problem with the fragmentation overflows. I've walked log files of two people nice enough to send them to me and they both look the same, that packets are being lost somewhere, either because they are outside the window of when seq accepts them (though I've seen no proof of this from logs) or just are being dropped by the pcap machine or just never getting in (this can't be true because the EQ client still works I'd imagine).

So wait a day + however long it takes for seq to be fixed post expansion is what I'd recommend for the fragmentation problems.