Network data changes [Archive] - ShowEQ Open Source Project Message Forums

sea4th

02-04-2003, 04:39 PM

Originally posted by fryfrog
I would also like to say that while i am NOT experiencing the CTD, i do get weird ghosting more so than before. i think since they are combining packets, mobs will do odd things.

like, when i mez a running mob it will keep running and then poof back about 10 steps. or, i'll pull something and it isn't showing to be following me but its smacking on me. stupid things like that. i say combine some packets, but please don't fuck with movement update packets. people depend on knowing where a mob is to do ANYTHING and when that movement becomes unpredictable and unstable, it makes things suck.

Is this movement behaviour or "ghosting" described by any other terms that the search feature can narrow down or become selective too? I see the same behaviour when fear kiting in Skyfire. A mob appears to be walking away and out of range so I chase it to get in range, cast snare and get whacked. I have noticed this in the Grey as well but was thinking it was a warping cabability of the mobs.

Any other search lingo terms offered to describe this behavior? Warping seems to match the behavior better than ghosting as ghosting suggests residual images- and I was thinking this is a designed in behaviour for EQ.

Sorry this is NOT an SEQ question / issue but more along EQ behaviour but the comment did stimulate an "oh yeah! Me too" recognition.

S_B_R

02-04-2003, 04:50 PM

Yes, what you are describing is mostly normal lag behavior. if a mob is some distance away from you it magnifies that effect. What I've been experiencing since the patch is more pronounced and is occurring more frequently.

Poncho

02-04-2003, 05:58 PM

I've been getting this alot more since the latest patches as well. Maybe just a coincidence though, who knows

fryfrog

02-04-2003, 06:18 PM

there is a difference between ghosting and warping:

ghosting - when you alter some aspect of the creatures run speed, the server has to update this and report it back to you. in the old system, i assume that this happened fairly quickly but now that packets are combined it seems to happen less quickly. an example of this would be right when you land snare on a mob, it continues running quickly for a very short time (probably less than a second) then ghosts back to where it SHOULD be. as another example, recently my enchanter and my druid gf have been charm duo'ing the stalkers in pon. standing right next to eachother, the druid snare pulls and i can watch the mob running twards us on her screen but it stays at its spawn point on mine. this USED to happen, but only for a brief few seconds. now, i have seen the druid miss pull like 3 or 4 mobs and watched them ALL run into the camp and still not be able to see them on MY screen. it makes it really hard to start mezing things when they don't even show up for a few seconds.

warping - when you get to far away from a creature that is stuck on something (like a house) it may appear to be jogging in place to YOU but it really IS coming after you and eventually just "appears" in front of you. this happens a lot in velks i know, tag a mob and run up a bunch of the stair things... eventually it just pops on top of you :)

bonkersbobcat

02-04-2003, 07:09 PM

It struck me funny when after all these years Verant decided to beat their netcode with an Efficency stick... Are you saying that SOE Nerfed their net code?

csmith12

02-05-2003, 12:30 AM

Yes I too have seen an increase in warping...

As a bard I play with many mobs at once... (swarming)

When I solo I pull trains normally....

I have seen my mobs just take off running in any given direction. That I can handle, but the warping on top of me sucks. :(

Stun..... hit, hit, hit by all in the train = Death

Some zones are still ok, the PoD is a deathtrap for warps now for me at least.

I think they had a good idea when it comes to the patch changes, but what works in real life and in theroy is sometimes two different things.

So am i correct here?

Before patch
1. send update mob speed, direction and destination packet
2. send more packets
3. send more packets
4. send more packets

After packet
1. send update mob speed and direction packet
2. send more packets
3. send more packets
4. send mob end placement packet (warp condition)

Thanks

Manaweaver

02-05-2003, 01:10 AM

I will admit things have been a bit funky since teh patch. I quad kite... I've noticed that mobs have a tendancy to not update the proper positions(making it look like they scattered) but once I finish my spell and it lands, they update back into their tight little formation. Mob warping... I think SOE has added some more code into it about when mobs have bad pathing. If a mob stays at a LOC for too long without a root on, then they get dropped on the player. I've noticed this happens whenever I circle around the mobs really tightly.

CTD's generally happen when using a custom UI... nothing fancy mind you, just any custom UI without the proper updates.

Copy the following files into your custom UI folder.

EQUI_StoryWnd.xml
EQUI_BodyTintWnd.xml
EQUI_JournalCatWnd.xml
EQUI_JournalTextWnd.xml
EQUI_JournalNPCWnd.xml
EQUI_BigBankWnd.xml

Then open the EQUI.xml from your custom UI folder with Word or notepad.

Add the following lines and or any other xml files that are not listed. Be sure to add them in with the list that has all of the other xml files.

<Include>EQUI_StoryWnd.xml</Include>
<Include>EQUI_BodyTintWnd.xml</Include>
<Include>EQUI_JournalCatWnd.xml</Include>
<Include>EQUI_JournalTextWnd.xml </Include>
<Include>EQUI_JournalNPCWnd.xml </Include>
<Include>EQUI_BigBankWnd.xml </Include>

Hopefully that helps. I've noticed my other computer(which is definately a lot more low-end machine) CTD's more often since the patch and also... it can't even ALT+Enter or it'll CTD...

datadog

02-05-2003, 01:19 AM

I just logged of of EQ. It seems after the patch this AM, that the problem fryfrog described (mez'd mobs keep running, etc...) has been corrected... or at least is it much better...

I have been at the same camp the last few nights, and it was a heckuva lot better tonight than it was last night...

Also the problem with HP bars not updating for folks is better as well..

Yesterday, my cleric would show the ranger at full HP, but on the rangers screen he was at 15%.... Today they updated almost simultaneously..

AND.. packet rates were still pretty good...

Poncho

02-05-2003, 02:39 AM

Manaweaver -

No need to copy those into a custom UI folder if you are using a custom UI. New patches that update the UI will put the new EQUI file in the default folder, and your custom UI will read that from the default folder. I am assuming/hoping that you dont have a custom UI that has already modified this file. If you do, you will most likely crash when new updates are made.

The safest way to run ANY custom UI is for your folder to ONLY contain the modified files. The rest will be read from the default folder if not found. This has been effective since about september or october.

I realize that this forum has nothing to do with UI skins, but I felt the need to clear that up. Regardless, Ghosting and the other problems explained in this post have NOTHING to do with your UI. Custom UI's are only for looks and have no other bearing (besides some slight memory loss if you are using huge or sloppy XML's)

Poncho

fee

02-05-2003, 02:52 AM

I have hesitated to discuss these changes on public forums untill now because I really really needed to see someone else self discover the inherit problems with SoE's new network code. What Protector states is absolutly right on.

Protector wrote:
There are definite theoretical improvements in the design, ie combining small packets together and compression that make sense (these were in place before Jan. 28)

However running each opcode through a bit scrambler then feeding it through a lookup table just to get back to the old switch statement is not one of them.

A small portion of the changes to the EQ network code make a lot of sense. Here are a list of Pros and Cons of SoE's implementation as I see them.

Pros:
Combining payloads into a single packet reduces header (IP/UDP/EQ_PROTO) overhead. Total packets is directly proportional to bandwidth, duh ;)

Compressing payloads that have repeating patterns will reduce bandwidth.

Cons:

CPU Overhead; Unfortunatly all of the Pros listed above have a consequence. SoE is essentially trading network bandwidth for Client and Server CPU. Deflating and Inflating these small (rarely large) blocks of data will over time take al chunk of CPU time. To their credit SoE is using a fast compression level (FLEVEL=1), however I wonder how many players have the spare CPU time to accomodate this, I know my CPU sits at 100% while playing. You may actually notice a pause in the client when you see the "character saved" message. When the character is saved a block of data (approx 13K) is compressed and sent from your client to the server. The zlib overhead to prepare and Deflate the data is enough to produce this small stutter.

Non self deterministic data sizes; The new network packet processing scheme requires additional calculations to determine where each payload, in a multi-opcode packet, begins and ends. This is a very minor issue, but again it requires extra CPU that does add up over time.

Stream encryption (for lack of a better term): Some people have questioned whether or not any of the recent changes were aimed at showeq. The answer is definitively yes! Without discussing the details of the mechanism in detail, I can say that this must have been designed to thwart pre-school children from parsing the packets, because thats the only audience I can think of that wouldn't "get it". So the result here is ALOT of extra processing on the packets before the client can use the data. This of course results in CPU overhead.

More implementation faults; The programmer(s) who designed this scheme forgot to take into consideration the already compressed and encrypted packets. The result here is compressed, encrypted, compressed data. Anyone that has ever tried to double zip a file knows the result, thats right it gets BIGGER! So much for some of the bandwidth savings, and hello! MORE CPU overhead.

Latency increase; No i'm not talking about network ping times. As several people have pointed out on this board, there is an increase in "ghosting" of spawns. This is the result of the servers wraping multiple updates together in combination with other types of opcodes. I can only speculate on this but logically it makes sense, atleast to me. Since the spawn updates are being delayed prior to transmission any updates that are recieved by the client are late and may not be in sync with where the server considers the spawns to be. Combined with the issue of non self deterministic data sizes, the latency increases a bit. A possible solution to this would be to increase the frequency at which the server updates the client about spawn movements and the like. Unfortunatly increasing the frequency of updates sort of negates the entire scheme. Damn double edge swords ;) A by-product of this latency is that you may get less updates over time which would result in a decrease in bandwidth. It would be interesting to determine if this is why 70 people raid encounters do not lag you out so much.

My overall rating for this scheme would be Half-Assed. As we saw in January SoE went through 3 attempts at reducing bandwidth finally settling on the thrid attempt. The first attempt was pretty slack and lots of problems resulted. The second attempt was actually decent. But the current implementation is leaving alot to be desired.

I think it is in rather poor character to use a multi-million dollar prodction service as a test bench for a half contrived, poorly implemented "idea", is not this what the Test server is for?

Disclaimer
the above opinions are based on my personal experiences and observations while playing the game and analyzing my network data and computer cpu usage Prior to the February 4 patch. Various play styles such as raiding or sitting in small unused zones will give you drastically differing views the performance issues I have discussed.

To SoE managment/staff: This is the only time I will ever provide any type of analysis to aid your product. I'm not getting paid for this, you are. Random Tom, Dick, and Harry customer should not have to provide this pro bono(free).

References:
http://zlib.org/
http://www.gzip.org/zlib/feldspar.html
ftp://swrinde.nde.swri.edu/pub/png/documents/zlib/rfc-zlib.html

Fee
ShowEQ Developer
[email protected]

Manaweaver

02-05-2003, 04:54 AM

I wasn't aware of that poncho... thank you for the advice... I'll revise my UI tonight so its that way. Should save a TON of problems =)

wfj5444

02-05-2003, 08:38 AM

Very good post Fee! Nice to get some inside info on the way things are happening.

Thanks greatly!

cbreaker

02-05-2003, 08:56 AM

Unfortunately, this probably means that in the future they will, again, completely revamp the system.

Some new ace programmer will sit down with the net code, and say "WTF is this crap?" and code out something that we hope will be good. Unfortunately this would break seq again heh. I mean, sure, companies keep crap code in their software all the time, but for something like an online-only game, it doesn't seem too likely that they will keep "Half-Assed" network code in there forever.

Chunking the data together is a good idea; and in a raid situation it helps quite a bit. When you have large amounts of data coming in, this is where it performs very well.

The only way I can see that they could keep chunking data and at the same time keep movement updates relatively smooth, is to have a seperate connection for the stuff that you want to be updated asap, which wouldn't be chunked so much. But I ain't no expert. I'm sure there's an easier way.. one of them being to stop with all this fancy-dancy stuff to thwart packet sniffing.

I guess it's the same old arguement about copy-protection. No matter how hard you try to keep a peice of software protected, someone will crack it. So does that mean you stop trying to protect your software? IMO yes, stop it's a waste of money. But for the company brass, they CAN'T stop because they need to keep collecting paychecks..

S_B_R

02-05-2003, 09:33 AM

Thanks for the info Fee. I just wish someone from Sony would actually take what you've said and do something with it....

cattj

02-05-2003, 01:50 PM

An overall evaluation of the network protocol ( ie how often opcodes are sent vs. the size of the opcode) added to the priority of the opcode being sent would give you a good idea of how beneficial it is to have this packet wait for others in order to send itself in a chunk. Using this data the queueing mechanism for setting up these chunks can take opcode priority into account.

Warping is going to occur in any case even if they sent every location update packet immediately. The client is attempting to simulate something that is happening in real time on the server. There is no network protocol in existance that can achieve this over a real network.

Now I haven't looked at the network protocol for EQ in ages so I don't claim to be an expert on the exact implementation in question here. Are they chunking together multiple loc updates for a single mob? Or is the server doing the logic where by an old loc is really useless therefore not included?

In the easiest multipacket protocol you would essentially have a time and/or a size threshold. If this packet reaches x bytes then send it and/or if this packet hasn't been sent in y seconds send it. Both of these are however, quite stupid when it comes to the actual payload of the multipackets. Knowing the protocol allows you to weed out redundancy and set priorities.

Aside from the disconcerting feeling of mobs jumping around the server is still running its checks the same as it always has and so gameplay (aside from aesthetics) should not be changed. I think SOE was trying to trade off lower bandwidth for smooth gameplay and it seems from most reaction that they moved the bar a bit too close to the lower bandwidth.

Priority queuing on an opcode basis would be help(do they do this already?) where by they allow themselves to alter the x and y in the above chunking example to better reflect the urgency of the payload.

Amadeus

02-05-2003, 02:59 PM

I hate to put such a simple viewpoint in a great, technical discussion (gratz Fee!)...but, am I the only one that doesn't notice any great difference other than negative changes?

Things worked just great for me before the patch, even with huge raids and in the bazaar. It's more of the same now, except with a lot more ghosting and weird screen flickering.

I just wish they'd fix bugs in the game, or add content -- adding more bugs to the game really doesn't make any sense.

bonkersbobcat

02-05-2003, 03:18 PM

For me the problem hasn't been the performance of the network protocol, it has been the performance of the game itself. I have DSL and and when playing EQ I never max out my network feed. I always, however, max out my CPU when playing EQ. IMO it would be effort better spent by SOE to improve the performance of the base game rather then screw with the network.

darkgrue

02-05-2003, 03:36 PM

IMO it would be effort better spent by SOE to improve the performance of the base game rather then screw with the network.

/agree

The lastest patch not only messed with the network protocol, it also made some changes to the DirectX interface to the game, which manifested itself with The Return of the Stupid Phatom Mouse Cursor, and tons more desktop bleed-through. How that sort of garbage could get through even the most rudimentary test mystifies me, unless it's unique to NVIDIA cards, and Verant doesn't test on probably the single most popular gaming video card...

The rendering engine alone needs help. All you need to do is do a quick (err, well maybe not-so-quick) 360 in the Bazaar to see that it's thinking really hard about a lot of stuff that you can't see. Which isn't trivial optimization, but it certainly would be an effective improvement to the game's performance! Luclin was supposed to improve the renderer with the inclusion of the Umbra engine so that the game could avoid rendering hidden surfaces, but whatever method that engine uses doesn't seem to be terribly effective.

S_B_R

02-05-2003, 03:43 PM

Originally posted by Amadeus
I hate to put such a simple viewpoint in a great, technical discussion (gratz Fee!)...but, am I the only one that doesn't notice any great difference other than negative changes?

Things worked just great for me before the patch, even with huge raids and in the bazaar. It's more of the same now, except with a lot more ghosting and weird screen flickering.

I just wish they'd fix bugs in the game, or add content -- adding more bugs to the game really doesn't make any sense. I've only seen negative results as well.
Originally posted by bonkersbobcat
For me the problem hasn't been the performance of the network protocol, it has been the performance of the game itself. I have DSL and and when playing EQ I never max out my network feed. I always, however, max out my CPU when playing EQ. IMO it would be effort better spent by SOE to improve the performance of the base game rather then screw with the network.I also couldn't agree more!

uncleubb

02-05-2003, 04:54 PM

I find it fascinating that a person who is a "mere" customer obviously has a better understanding of the product than the product maker.

If my development team was put to shame like that by a consumer, I would be extremely embarrassed. It would be time for firings and hirings... in that order.

I know fee is obviously very smart and knowledgeable, but he can't possibly be the only person with this kind of networking savvy... (in fact it appears to me that numerous such people regularly participate in this forum).

If SOE did this to spite SEQ, they really ought to consider serving the VAST majority of their customers before targetting a very small minority. Macy's doesn't pat down every shopper just to make sure they didn't steal anything...

It's a simple matter of quality and service.

And just in case SOE has forgotten, or thinks these terms are some unknown foreign speakease (and because I love to post definitions):

quality

adj 1: of superior grade; "choice wines"; "fine wines" "prime beef"; "prize carnations"; "quality paper"; "select peaches" [syn: choice, fine, prime(a), prize, select]

service

n 1: work done by one person or group that benefits another; . . . 13: the act of mating by male animals [ok, not that type of service please!];

-uu

Amadeus

02-05-2003, 05:44 PM

If my development team was put to shame like that by a consumer, I would be extremely embarrassed. It would be time for firings and hirings... in that order.

I was watching the most facinating episode of NOVA last night on the development of the latest fighter jets by Lockheet Martin and Boeing (the competition)...and all I can do is shudder to think if engineers there were as sloppy as most of the ones we see in the computer entertainment software industry. And HONESTLY, I don't think it's the little guys, I think it's poor ass leadership letting this get by in the first place. I can't even imagine what kind of disaster the console version of EQ is going to be when they can't patch it every week ;)

I swear the test server is just a coffee table in California somewhere.

CybMax

02-06-2003, 05:20 AM

After the last patch there has indeed been a great deal better.. Still have some random warping and similar, but better than it was..

Now.. I too have no problems getting WAY more data inbound on my net, and so do a lot of others, but it might be a problem outbound from SoE serverpark..

So.. to the real fun.. The "lag" you get (not spawn warping++) but the LAG.. the less than 10 fps lag.. that is purely graphics engine coding.. The lag in bazaar is not from cluttered network traffic, but from a bad graphics engine..
Just step into a totally empty zone, go up in a corner all by yourself and check what fps you got.. Now add in a couple of spellcastings, a few friends and watch the fps go to a near halt.. It is not network traffic that makes THAT lag..

seq_not_i

02-06-2003, 09:28 AM

Amadeus: In the electronics arena, engineers are that sloppy, even in aerospace. I did some work for Lockheed and after 5 prototype circuit boards, the system still had incorrect parts called out.

I'm not an engineer, I'm a mechanical type that translates electronic engineering ideas into a chunk of fiberglass with components on it but even I was able to catch engineering mistakes.

I hate to say it, but in general, the US companies I do work for are inherently worse in getting to production than overseas companies.

/kick US engineers

Now that I ripped my country's engineers let me add to that with a positive. The US, in general, is quicker on new design ideas and some of the new stuff I'm currently working on will make you hardware geeks /cheer when it is released.....Hell, I'm cheering now!

old_fart

02-06-2003, 12:31 PM

If the zone and character info is encrypted we may need to automate key retrival so that SEQ has it before the zone info is sent. Or get the zone info from the client...

high_jeeves

02-06-2003, 12:36 PM

SEQ currently just caches the encrypted data until it receives a key.. then it decodes it and moves on.. this is already taken care of.. The key can be received 1 second, or 10 minues after the zone-in, and SEQ will correctly handle the original zone info.

--Jeeves

cbreaker

02-07-2003, 12:48 AM

Originally posted by bonkersbobcat
I always, however, max out my CPU when playing EQ.

I've seen lots of people say this. It's totally not valid =)

Would you want EQ to not use 100% of your CPU and thus run slower? *any* fairly modern 3D game will run your CPU at 100%. Why save clock cycles, when you can put in that fancy texture or detail? I dunno about you, but over here, EQ is pretty graphically intensive, there's a lot to put on the screen. The models themselves are some of the most detailed models I've ever seen in a game.

Not to say that the EQ engine is top-notch, it's got a lot to be desired. However, even if they make it the most efficient 3D engine out there, the game will happily put those clock cycles to use elsewhere.

I expect any modern game I run to completely maximize it's usage on my computer. If I see 60% CPU usage, something is amiss.

Just my off-topic 2.

S_B_R

02-07-2003, 08:35 AM

I think you are slightly missing the point cb. It's not that EQ should use less than 100% of the CPU. It's that it shouldn't waiste CPU cycles on this stupid pseudo-encryption of opcodes, and compressing compressed data...

Hlapmoop

02-07-2003, 09:39 AM

I'd rather have smooth gameplay than pretty graphics. My gmeplay now is laggy when in any real situation where it would matter and I already got all the graphics turned down. You got a point, the game should be snarfing up all the resources it can to run. However, it shouldn't be taking up all those resources because it's trying to do too much in too little time.

baelang

02-07-2003, 11:01 AM

Originally posted by seq_not_i
Amadeus: In the electronics arena, engineers are that sloppy, even in aerospace. I did some work for Lockheed and after 5 prototype circuit boards, the system still had incorrect parts called out.

I used to work at Boeing. don't even get me started on some of the software i saw there.

Hobo

02-07-2003, 11:38 AM

ROFL...I still work at Boeing and I know exactly what you mean.

:)

fester

02-07-2003, 11:54 AM

Darkgrue, one thing of note.

Evidently (from my testing) EQ does not do much 3D processing.

For real performance (at this time) you need to get a system with on board video and PC2100 DDR or better memory.

AGP 8X video cards (like the top Radeon 9700) may score 11 times as faster than onboard on the 3DMark 2001, but they are slower when I ignorantly added the Radeon 9700 to my systems.

I would suggest removing all video cards (Radeon and Nvida) from my EQ systems and are running on the onboard video. DDR is required, because one of my systems had PC133 ram and was better served by a Radeon 9700.

This (of course) does not hold true for just about every other 3D game (especially things like unreal or quake) so keep the 3D cards handy or make a special EQ system to play EQ.

devnul

02-07-2003, 01:20 PM

For regular play EQ is much deproved. Rampant ghosting and stuttering. Right now game quality is the worse it's been in a while.

In a large raid environment, and the nexus, things seem a little better than before.

It's certainly not impossible to accomodate both of these.

Compressing compressed packets is just pathetic in a product like this.

dn

Crux

02-07-2003, 10:09 PM

cbreaker:

my cpu cruises at 49-51% usage in a relitivly dead zone, and upwards of 70-80% on a large guild raid now with eq. i suspect my video card is pulling alot of the load from the proccie though.

p4 3.06ghz
1.5gb ram
ati 9700

bonkersbobcat

02-08-2003, 04:09 AM

Originally posted by S_B_R
I think you are slightly missing the point cb. It's not that EQ should use less than 100% of the CPU. It's that it shouldn't waiste CPU cycles on this stupid pseudo-encryption of opcodes, and compressing compressed data... Agreed. When tuning a system you look for the bottleneck and improve it. If you don't tune to the bottleneck you don't do any good. I have a decent system. (P4 2.4Ghz, 1GB RAM, Nvidia Ti 4600) When I play EQ my processor is always maxed. This means that my particular bottleneck is the CPU. My point here is that improving the network protocol is not going to help me because the network protocol is not my bottleneck. I don't think I am alone.

You might argue that the network is really still the bottleneck and that EQ is just using the extra avaiable CPU time to draw faster or with more detail and is not really waiting on the CPU. I don't think this is the case because the amount of data that goes over the network is not siginficantly different when I am in the Bazaar then in other zones. My performance in the Bazaar, however, is significantly different then in other zones. This tells me that it's not a network thing, but rather a CPU or video thing.

I have tried swaping out a couple of different CPUs and Video cards I have noticed that my performance difference is pretty linear as I change my CPUs and doesn't have much variance between my different video cards. This tells me that my bottleneck is not my video, leaving the CPU.

Given that CPU is my bottleneck, the best thing that SOE could do for me is to improve their use of available CPU time. Burning extra cycles to do a funky encryption on the network (that will get broken anyway) makes things worse, not better for me.

As a disclaimer, I do run at 1600 x 1200 with everything enabled (It's winter and I have to keep the house warm.)

Edit: spelling

fester

02-08-2003, 01:19 PM

CPU is cheap now. I am quite sure they upgraded their servers CPU speed to handle the new encryption (and possible other improvements.)

It is only a matter of time before all the player base will be on 2ghz or better systems (considering cheap ones are 2.4ghz for $499) and then the few cpu cycles used for gzip will not matter.

I would suspect that gzip -1 uses little cpu when used against the low bytes per second rate (2 - 9 k in a raid) that EQ generates.

I would even bet that seq uses more CPU (before Jan 28) to display and animate the world than the CPU in the client to decode these compressed/encrypted packets. If I can run two sessions of seq with reasonable success on a 333mhz celeron, then that would me it uses (on average) 167 mhz to process packets. Sub seq's GUI for the compress/decompress and that means that about 200 mhz of your 2000mhz processor will be dedicated (or 10 percent).

In my mind 10 percent is acceptable if they can manage to make it less laggy while retaining the compressed payload.

EDIT: My main point (which I left out) is that the bandwidth savings for them is worth a considerable amount (since the nearly half the required bandwidth for their user base.) This means that if they keep their current bandwidth allocation, they are good until maybe 200k users. I would bet bandwidth is their largest expense (aside from maybe staff) and this is no small amount of money. Use 1.5 kb/s as an average savings per user, take 110k users, is 161mb/s (or an OC3). I am not sure, but I know a T1 costs about $700 a month so say OC3's are half off the rate, that is a savings of $36,000 a month (for them.)

kleenburn

02-08-2003, 01:39 PM

...they reduce their bandwidth charges at the expense of their customers?

baelang

02-08-2003, 02:02 PM

Originally posted by kleenburn
...they reduce their bandwidth charges at the expense of their customers?

Everythign they do is at the expense of their customers, right?
i mean who else pays the bills?

you are going to pay (part of) their network bill (and every other bill) one way or the other.

dirfrops

02-08-2003, 02:57 PM

Yes, you pay for their bandwidth, their booze and drugs, you even pay their taxes.

That's what business is all about.

The end consumer pays for the world to go round. :p

kleenburn

02-08-2003, 06:25 PM

Yea, the issue seems to be why customers should pay for their bandwidth at the expense of game graphics quality...

BlueAdept

02-08-2003, 07:12 PM

I noticed that the longer you seem to play EQ the worse the stuttering becomes. When I log in and am in PoK, it isnt that bad. I go to a few zones, play for a while, go back to PoK and get the stutter. I log totally out of EQ and go back in, it is back to normal until I play for a bit. I am getting the stutter in zones that NEVER had it before they started all this new crap.

It got so bad that I was having to turn down my clip plane down to like 20% so I wasnt getting the graphics lag.

dirfrops

02-08-2003, 07:53 PM

Are you on broadband or dialup BlueAdept?

I dont have that problem at all...actually I have more stuttering/lag problems when i first log on, but after being on for 10 minutes I am fine.

mudtoe

02-09-2003, 12:30 AM

Blueadept:

I have the exact same problem you do regarding the performance getting worse and worse the more zones you go through. I'm thinking it's a graphics thing rather than a bandwidth one. My guess is that somehow graphics objects from the zones are being kept in memory somehow, and the more zones you go through the worse the problem gets. Since the memory footprint of EQ doens't seem to get larger and larget as time goes by, it points to the video memory. BTW I have a broadband connection and have the datarate.txt set to 12.0.

I'm a mainframe assembler programmer ( TP monitor development) by trade and this behavior almost looks like what happens when you get a table with too many items in it, coupled with an inefficient search algorithm. It works OK until you get more than a certain number of rows, and then the performance goes down the tubes quick after that. I'm wondering if EQ is keeping so many graphics objects defined to the video card, and not getting rid of them when you zone, so that after a while if you are in a zone where there are lots of objects to display, the search through the now too large table of objects starts to degrade performance (I think the viceo card processor is what's getting saturated). It fits because the problem becomes noticable first in zones with lots of graphics objects to display at once (i.e. zones with lots of players in it, and especially if the lots of players are all close together so they have to be rendered at the same screen).

If you want to see the problem in operation go to nexus at the foot of the pyramid during early evening when there are a number of players there. Log out of the game and come back to "clear the cache". Now spin a 360 turn. You may notice a slight slowdown as your view passes by the platform where the players are standing. Note how long it takes. Now zone into Bazaar, PoK, PoD, and back to Nexus. Repeat the spin test. Notice how much worse the "hitch" is when your view passes the platform.

I've gotten in the habit of logging out of the game and coming back in after I've gotten to the zone we are going to fight in, just to clear the accumulated trash from all the zones I traveled through to get there. Most annoying.

mudtoe

BlueAdept

02-09-2003, 08:52 AM

I have broadband. A geforce 4 4400 w/128 megs a p 1.5ghz w/ 1.5 gigs of memory running xp and I have my data rate set to 9.

My meter usually stays around 150 0.0% 0.0% 250 but I still get the stutter and the bazaar is almost unusable now. I have to turn my visability down to 5% to be able to do anything in the zone. Otherwise it takes like a minute for anything I do to actually happen, like mouse movement.

They really messed things up for me. It was perfect before these stupid patches.

Spook

02-10-2003, 12:48 AM

Those "hitches" I call graphics caching. If you are of the mind, you can break out a profiler and monitor traffic on the buss or even what is being called when in the system (everything slows down with this sort of monitoring btw)I generally spin once on zone entry to cache the graphics and things run fine for me with only minimal hiccoughs when new characters enter the zone or I see a new spawn with much different graphics than the rest. As of the so-called network optimization patch it seems as if something is happening with that way graphics are loading - even perhaps causing the zone to be reloaded instead of just updated. This is making things turn jerky at times for me. Off in a remote zone where other players don't zone very much - even though there may be a lot of them things run smoother than before. So, I'm of the mind now that some sort of trigger may be happeneing with the new network compression causing graphics to load more so at certain times /shrug

A note about watching the CPU load graph you can get from ctrl-alt-del Task Manager-Performance; it is deceptive. I can write a real short program that runs in a tight loop but with a call to do system events inside it. The perfmon will show 100% CPU usage which is just showing inverse idle time. The system runs just fine like that. So, some programs may be using that sort of threading or looping instead of putting that application to sleep for a while which will show the large but deceptive CPU usage.

throx

02-10-2003, 11:13 AM

We're not going to start with the fiction that the decompression/decryption of a 2k/sec byte stream is taking any significant portion of CPU time again are we?

Test it out sometime with gzip. My P3/900 can decompress around 2,000k/sec (decompressing to /dev/nul) and that's limited by HDD speed more than CPU time. That means on a 2k/sec stream you're using at most 1% of CPU, more like 0.1% CPU on the 250 bytes/sec that most people are getting now on non-raid situations. Those numbers are on a P3/900 so scale them down appropriately for faster CPUs.

I do agree that EQ's graphics engine is pretty inefficient and is far, far more CPU dependant than Vid Card dependant. A P4/1500 will definitely struggle with new models and lots of people in a zone. Remember, if facing a wall improves things then it's the rendering engine that's at fault and not the network layer.

Ratt

02-10-2003, 12:16 PM

We're not going to start with the fiction that the decompression/decryption of a 2k/sec byte stream is taking any significant portion of CPU time again are we?

Test it out sometime with gzip. My P3/900 can decompress around 2,000k/sec (decompressing to /dev/nul) and that's limited by HDD speed more than CPU time. That means on a 2k/sec stream you're using at most 1% of CPU, more like 0.1% CPU on the 250 bytes/sec that most people are getting now on non-raid situations. Those numbers are on a P3/900 so scale them down appropriately for faster CPUs.

The real issue here is the compression/encryption, not the decompression/encryption.

The additional load on the servers must be awesome, to have to compress/encrypt the data for 2000 users at a time.

Obviously, there's only 2 or 3 zones per computer, some more, some less... but still, on a large, large scale raid, that's gotta suck up major resources on the given box with 500 people in the zone.

Resiliant

02-10-2003, 12:19 PM

Unless hardware assisted.

EnvyEyes

02-10-2003, 12:28 PM

Off topic....
Love the quote, Ratt. That has to be one of my all-time favorite movie lines. Brian Cox was awesome in that role.... not surprising that he played a 'mirror role' in the newest flic to copy "The Long Kiss Goodnight"'s theme (The Recruit- CIA spy/assassin and deep corruption of the powers that be).

Sorry for straying off topic, but I couldn't resist commenting when I read that.

throx

02-10-2003, 05:16 PM

The real issue here is the compression/encryption, not the decompression/encryption.

I agree, though I haven't noticed significant problems on raids with the code. In fact, everyone seems to be raving about how wonderful it is now that /serverfilter works properly and is automatic (*sigh*).

This could certainly be an issue in the bazaar though and causing the time delay noticed for the server to respond.

I wouldn't imagine the encryption takes up any more CPU time than decryption but compression is certainly more time consuming than decompression. I wonder if they've just done another round of server upgrades and I wonder how much CPU time the halved data rate has saved in comparison to the CPU time required for zlib compression?

Raistlin

02-10-2003, 05:54 PM

Very interesting discussion...

I've been out of town for a few weeks on and off. On my main machine I've not noticed any real problems to speak of (PoK and Nexus now lag on me a bit when they didn't use to).

HOWEVER since i've been out of town i've been on my PIII 800 laptop with 3xx meg of PC133 memory and a dial in connection.

And all of a sudden this thing is almost completely unplayable. I have all graphics and models off, I have clip plane set to 0, I'm running in 800x600 mode for gods sakes...and i'm still lagging like i'm running on a 386.

I was just going (after last night) to attribute it to heat (the longer I played the slower it got...and the hotter my laptop gets as the fan is a bit flakey and tends to cut out) but it's sounding like this might not be the case. I THOUGHT I noticed that alt-entering did in fact help speed things up for a short while and if in fact there are too many objects stored in the graphics memory, then possibly this could be a partial fix since switching modes like that MIGHT cause some or all of those stored graphics to have to be cleared out of the cache.

Anyone found any way to either stop this or "reset" this?

wiz60

02-11-2003, 08:50 AM

This topic has evolved into an area that I have been curious about for several years. I had a chance to spend a bit of time with the former head of EQ Customer Support and while he was forthcoming on many things abou the game - he was very tight-lipped about these details.

What is the physical architecture of an EQ server?

I realize several things (these are guesses btw):

1. Multiple computers make up a "server". I suspect that these are not all the same.

2. A "zone" runs on a computer as a task. Probably several zones on a computer - and the mapping of zones to computers is probably very flexible.

3. I think they run linux or a unix variant. And I was told the database is homegrown.

Ratt's comment sparked an interest. He concluded that compressing packets for 2000 users would be quite a load on the server. From that I am concluding that he feels all the packet traffic for a "EQ server (ie: several computers)" is routed to a single machine. I suspect that your closeness to the packet mechanisms give you some valid reasons to conclude this structure. I would have guessed each zone communicated to "its" users.

The notion that they might be consolidating the IO for a server to one machine is interesting. I would suggest that perhaps the individual computers forward compressed packets - thus distributing the load. The workload on any computer would be the people in the zones it runs. Certainly thats not 2000 people.

Note: I understand what servers are - and have tried to stay away from that term to avoid confusion with the logical notion of an EQ server.

LordCrush

02-11-2003, 09:13 AM

Wiz60,

perhaps you might want to look here for interest ...

http://www.eqemu.net

(How it maybe works)

quackrabbit

02-11-2003, 10:02 AM

Originally posted by wiz60
3. I think they run linux or a unix variant. And I was told the database is homegrown. While I don't know for sure what O/S the servers run on, Brad McQuaid has said in the past that the database back end for LIVE servers are dumps to text files (for fast performance) that are loaded into the servers memory.

He also went on to say that these text files are created from dumping SQL tables - so one would assume they do use some sort of relational database for manipulating data, data entry and research - and this would also explain why a zone has to come down for a database change to be propagated to the zone servers.

Please note: Brad has been gone from SoE for a long time, and these facts may no longer be relevant. Also, while I did try to search for his post, it may have been on the long since wiped EQ Developers Corner message board - so I can not find the source of this message - so flame me if you want. ;)

casey

02-11-2003, 10:07 AM

an EQ world is 22 servers running windows NT.

circa 1999 hardware. every server before sullon zek was a copy of the original harware specs of the first server, sullon was the first to take advantage of newer technology.

edit: my info is dated ! yay

i bet you can convince quakrabbit to give you the hard facts if you ask nice. !

quackrabbit

02-11-2003, 10:26 AM

Originally posted by casey
an EQ world is 22 servers running windows NT.There are 34 servers per world now. (1 world server, 33 zone servers) :)

Resiliant

02-11-2003, 11:16 AM

Wanna bet the 'bazaar' zone is on its own server? :)

I can visualize this, actually. I bet they are 1u high server blades implemented in x86 technology running NT2K, connected to a Cisco router that handles back-end packet switching. Each EQ 'server' is, in fact, a single rack.

It would be SO cool to see this machine room.

bonkersbobcat

02-11-2003, 02:14 PM

Guy I talked to from SOE at one of the fan events told me that EQ servers were running NT 4. Yes 4. He also said that each EQ "Server" is a bank of NT4 servers each hosting a number of zones. The number of zones per physical box varies depending on the size and popularity of zones. Zones are not split across physical boxes.

sam

02-11-2003, 02:29 PM

Originally posted by Resiliant
I bet they are 1u high server blades implemented in x86 technology running NT2K, connected to a Cisco router that handles back-end packet switching. Each EQ 'server' is, in fact, a single rack.

I'm not sure why people think the servers are running windows. Windows has never and will never run as stable as the everquest servers have always run. The servers have a great track record for stability.. with the exception of a few patch days that caused server crashes. But overall, for a server that gets so much network traffic, and is so memory/cpu intensive, they have been great! Windows has problems with large armounts of memory, windows has problems with tcp/ip (especially when having to handle more than one local IP), windows has HUGE security holes, windows is very expensive, and windows is just all around unstable for any major server environment.

These guys are running some sort of *nix OS. It probably doesn't run on x86 hardware because then if the binaries leaked out, everyone and his brother would have a working copy.

With that in mind, the hardware for these very stable and very resource intensive boxes is probably very expensive. So they dont buy one server for one zone... they would be loading up multiple zones on one server depending on how much average load that zone receives. For example, all the starting town zones have probably been moved off to one server since no one ever goes back to their starting town.

quackrabbit

02-11-2003, 03:56 PM

Originally posted by sam
I'm not sure why people think the servers are running windows. Windows has never and will never run as stable as the everquest servers have always run.I have seen and heard enough to be 100% convinced that bonkersbobcat's post is correct.

sam

02-11-2003, 04:20 PM

I may be completely wrong, but if the servers are running on a windows platform I will be stunned and amazed.

mvern

02-11-2003, 04:28 PM

The servers has/had a little habit of sending unitialized memory in some network packets. Seeing "WINDIR=C:\WINNT" show up in the trash section of a packet more then once was enough to convince me that they're running NT ;) I havent seen it lately tho, they probably atleast partly fixed it at some point.

throx

02-11-2003, 04:31 PM

I'm not sure why people think the servers are running windows. Windows has never and will never run as stable as the everquest servers have always run. The servers have a great track record for stability.. with the exception of a few patch days that caused server crashes. But overall, for a server that gets so much network traffic, and is so memory/cpu intensive, they have been great! Windows has problems with large armounts of memory, windows has problems with tcp/ip (especially when having to handle more than one local IP), windows has HUGE security holes, windows is very expensive, and windows is just all around unstable for any major server environment.

You've been reading too much Slashdot. My last job involved server computers which controlled the traffic signals for an entire city. They ran NT 4 and had uptimes of around 180 days, though some we managed to get to 400 or so (cause we were sceptical about putting service packs on - these weren't public machines). Windows NT runs just find in a controlled environment and will give you incredible uptimes if you have half a clue about administering them.

I did read somewhere that EQ ran on Windows NT (which also surprised me). I have a feeling that EQ2 will be a Linux based system though, from the skill requirements for jobs listed at SOE.

If you want to look at the server room, there's a very quick glimpse on the CNN video on EQ's main site.

http://everquest.station.sony.com/movies/cnn.jsp

bonkersbobcat

02-11-2003, 04:38 PM

Originally posted by sam

I'm not sure why people think the servers are running windows. Windows has never and will never run as stable as the everquest servers have always run. This can get into religion and can feed the fires of much flaming so I am only going to make one comment on this topic.

On stable hardware, in a stable and structured administrative environment, Windows (NT and beyond) is just as stable as unix. Like there are things that have to be done in a unix environment to make it production worthy, there are similar things that are done in Microsoft environments. People get caught up in religion, but properly deployed, both environments are production-worthy.

Microsoft tends to win with ease of administration for less technical (more cost effective) support staff.

Microsoft tends to loose in areas of online changes (ie not rebooting or shutting down for upgrades or patches)

When putting two single boxes side by side in a 24 x 7 situation, unix will win, however in an environment with multiple machines in a partitioned or distributed archiceture the issue becomes more cloudy.

Pigeon

02-11-2003, 04:53 PM

It's not as if EQ's servers are all that stable anyway. A zone on my server has crashed at least 3 times in the past week, (3 different zones, all non-connected) and no, I'm not refering to all the zones repopping or the servers coming down for a patch.

For the record, I agree with bonkers. If the sysadmins are well trained and know what they're doing, the end-user should be unable to tell what OS a server is running.

sam

02-11-2003, 05:11 PM

There are so many reasons I can think of why windows is a bad OS for anything large scale. But I will no longer discuss this and please, no one else discuss this in this thread. I'll head over to the general discussion forum and post a thread there... it shouldn't be hard to miss and we can fight it out there.

joojooga

02-12-2003, 08:32 PM

From reading this thread I would think they're NT too. I'm sorta shocked though, not from a stability standpoint, but from a cost standpoint. You figure they bought the 5 Cal cheapie version of NT, that's still like 499.00 a pop. And if they ever upgraded to 2K, then they're prolly gonna use Microsofts Open "rent-a-licence" scheme. Linux woulda been cheaper.

I've heard that the patcher program is just HTTP over a non-standard port. If that's the case, hope they're not using IIS LOL.

-Joojooga

throx

02-12-2003, 10:01 PM

Originally posted by joojooga
From reading this thread I would think they're NT too. I'm sorta shocked though, not from a stability standpoint, but from a cost standpoint. You figure they bought the 5 Cal cheapie version of NT, that's still like 499.00 a pop. And if they ever upgraded to 2K, then they're prolly gonna use Microsofts Open "rent-a-licence" scheme. Linux woulda been cheaper.

I've heard that the patcher program is just HTTP over a non-standard port. If that's the case, hope they're not using IIS LOL.

They have over 1000 machines - they'd be using a Select license, or even have a direct deal with MS. You can't tell how much per box it would be, but the last Select license I worked with was a *lot* cheaper than the "off the shelf" prices.

You also can't compare Linux costs to NT costs just from 'off the shelf' prices. You end up looking at the long term pricing and probably want to add in the cost of telephone support from whoever they buy the Linux distro from (most companies don't do a 1000 machine roll out from an unsupported distro).

The patch servers use Apache. Just try to fetch a page and look at the error message.

S_B_R

02-12-2003, 11:36 PM

As I recall, they had some job postings for several Unix Admin postions with "Solaris experience a plus"... that was back around the time of SoL.

Klandis

02-13-2003, 04:19 AM

> The patch servers use Apache. Just try to fetch a page and look at the error message.

Server: Mathopd/SOE

"Mathopd is a very small, yet very fast HTTP server for UN*X systems."

Vertigo1

02-13-2003, 08:19 AM

1.) We've seen SoE get "shown up" by a mere consumer before... EQW? and their now poorly implemented Windowed EQ mode.

2.) I turn off all the fancy stuff... 640x480... base EQ.. feels like I am back in the pre-kunark days... and my 2.0Ghz Pentium 4 will be at 100% CPU usage.. Granted EQ runs fast as hell at this setting.. it is not the game I like to play..

I've popped out into Windowed mode and opened Task Manager in Windows XP and it is pegged at 100% CPU usage.

I gotta agree with fee here.. The have constantly used regular PAYING customers for beta testing their shit. They have no QA, they have no real code/game testers.. They implement it half assed and release it with enough hype of the second coming..

I wish I knew more of network programming and packet structure to help fee and the devs with this. Thanks for all the work guys, and will be waiting for fixed SEQ in the coming weeks if possible. Good work and keep it up.

BlueAdept

02-13-2003, 08:26 AM

You know with the processor pegged at 100% all the time, what good is windowed mode?

I tried to load IE and it just sat there for like 3 minutes loading. To go to web pages, I would click on the link, switch back to the game for a couple minutes and then go back to the page. They arent giving enough time slices to other processes.

Ive ended up putting my old 486 laptop on the network just so I can view web pages faster.

KaL

02-13-2003, 09:09 AM

I run my trader 24x7 all the time in windowed mode. I run Winamp, IE, windows Explorer, image viewers, SSH client, etc., while running my trader.

That's on a 1ghz machine, 256mb RAM, 32mb geforce 2 gts.

My main machine, 1.4ghz, 512mb RAM, 64mb Geforce 4 Ti4200, I can run those things while actively playing, not just in trader mode.

Dunno why you have so many problems.

Oh yeah, I'm running XP on both machines.

wfj5444

02-13-2003, 09:33 AM

My main is about the same Kal, a bit beefier..

Athlon 2000+ 768MB of Ram, GF4 4400, WinXP... etc

and I have no issues using EQW in 1024X768 with a screen rez of 12X10

/shrug. Could be that I keep my page file on another physical drive or that I keep my machine defragged often.

I am also running Norton AV as well.

cryptorad

02-13-2003, 09:36 AM

For you guys saying you have NO problems running in windowed mode.

Are you running EQW?? Or are you running SOE's windowed mode of EQ itself??

Klandis

02-13-2003, 11:59 AM

(win98)
I have no problems with applications if I have them running before I start the game. EQ likes to allocate all memory and anything extra started after that enters swap hell for a while.

sam

02-13-2003, 12:45 PM

I expect to get flamed, but oh well...

Can we please go back to posting about Network data changes in this "Network data changes" thread? The topic has been off target for a lot of the posts. The reason being, I'm really curious about what new changes the devs find out and I'm not as interested in OS wars and how bad the windowed version sux... lots of other threads/forums to post these in :)

Lyenu X`Arie

02-13-2003, 12:56 PM

I agree with you Sam, and I am very interested in seeing what the devs have found out as well.

throx

02-13-2003, 01:17 PM

You know with the processor pegged at 100% all the time, what good is windowed mode?

Yeah, and if I could kill that pesky "System Idle Process" that normally takes all my CPU I'm sure my machine would be like ten times faster.

A game will always use 100% of the CPU because that's the way games are designed. They run an event loop the entire time as fast as they can so that improvements in your machine translate directly to improvements in game performance. Most people would be rather upset if they upgraded their machine to find that they got no improvement in the game's performance, just some more idle time for their system.

If you want better performance from other applications, just go to task manager (assuming you are running XP/2k) and change the scheduling priority of eqgame.exe. Because NT uses a pseudo-realtime scheduler though, I'd not recommend running anything CPU intensive at a higher priority than EQ if you don't want to go LD a lot.

fester

02-14-2003, 11:55 AM

Well, I am not a dev, but when I read the post by Protector that created this thread, I thought "well duh".

I knew within 48 hours the transforms made to the majority of the packets. I am QUITE sure the devs did within that time also. There was even a poll on the irc channel of the likes of "spend time to release a working one now or wait until after LoY as they are likely to break it again" (close but not exact quote.)

Most people opted for the "fix it once" model. I personally don't want to spend more time on this than I have to (when a fix is released, I only want to install it once.) I was able to get seq working for much of the packets to which we didn't need another key (did this more as a curiosity, as I had never seen them put effort into breaking GPS before.) The key which is init'd during zone, they know how to derive that without snooping the client (if present during zone) and at least I know how to snoop for it IF I was not not sniffing during a zone (it can be reversed from the packet stream as it is a simple repeating xor).

I think why no one answered this thread (at least why I didnt until now), is that I didnt think it was anything more than a complaint and didn't deserve an answer. Especially since this thread has ALREADY BEEN answered in the form of posts during the first couple of days after it broke.

(Edit: it was early and I corrected some spelling)

Cryonic

02-14-2003, 02:05 PM

Actually EQ doesn't keep my CPU pegged to 100% (at least not when run Full-screen). I know this because I run distributed.net in the background and if the CPU was fully busy, then distributed wouldn't get any time from the system to crunch keys. However in windowed mode (non-EQW) it does peg the CPU.

The Mad Poet

02-14-2003, 09:37 PM

Actually it's the engine that has a hard time - I'm not sure how they draw the world - but I can tell ya it's the suck :P

Ever wonder why the lag in the bazaar is bad? It's not the number of people - look at the ground and everything is fine - data packets are not dragging your comp down.

The problem is when you look at a wall not only the wall is rendered - but everything behind the wall also.

Every *new* model also has separate models for the teeth, eyes and such - all the armor is *over* the skin of the model that is drawn every time you look at someone even though you can't see it.

*That* is why you need such a good system to run new models on.

The old models used skins that actually changed the skin of the model when you wore armor - that's why they were alot less laggy.

If they added the code - or fixed it - so when you *can't* see something it isn't drawn - everyone in EQ could play with all new models in the bazaar and never lag - even with an old geforce1.

The actuall polygons are not *that* detailed - it's the layer upon layer with the stuff drawn behind the walls that causes such grief.

Currently the only way to stop drawing stuff behind a wall is to narrow your clip plane down to where everything is in a fog.

mudtoe

02-15-2003, 12:30 PM

high_jeeves

02-15-2003, 01:38 PM

If I'm not mistaken, isn't that what the 3D graphics cards do for you, that is do all the drawing of objects for you? I don't believe that EQ program has to take into account your position relative to all the other objects and draw what is appropriate. My understanding is that the EQ program simply tells the graphics card the position of all the "objects" and your current position, and the driver and firmware for the card take care of the rest, including knowing which objects are behind other objects and therefore shouldn't be rendered on the screen.

You are mistaken... read any book on 3D graphics.. when graphics cards can do realtime object occlusion, the world will be a wonderful place... For now, they do z-buffering to decide what objects are in front, but in order to do that, they still have to render (atleast partially) all of the triangles in the scene...

--Jeeves

The Mad Poet

02-15-2003, 03:52 PM

there is a process called bliting (spelling - heck it's been a few years)...

This is where the software determines if a pixel is behind another pixel and if so doesn't draw the stuff behind - this is the basic premise behind a ray-engine such as Doom and Quake use.

it would be more processor intensive to check the position of objects for obstruction - but I think that the overhead would be minimal compared to the processor time spent sending the data/textures/etc to the graphics card to draw - not to mention the drain that your graphics card takes...

high_jeeves

02-15-2003, 05:48 PM

I do quite a bit of DirectX development in my spare time, here is the way it generally works:

Object culling is done (maybe, maybe not in EQ.. no idea). This checks for OBJECTS (not polygons) that are obstructed by other OBJECTS, note: in EQ, the entire zone is an object.

Backface culling is done (so polygons facing the wrong direction arent transformed, lit, or rendered)

view frustum clipping is done, so objects outside of the view frustum arent rendered (think clipping plane)

polygons are sorted by distance from the eye point (not strictly necessary on anything except for transparent polygons, but can save alot of time on the graphics card).

rendering pass 1:

Render all opaque polygons from closest to farthest from the eye point.. the video card (or software driver, if vid card doesnt support hardware z-buffering, which i think all do these days) figures out which polygon each pixel should take its color from... and does texturing, etc..

rendering pass 2:
Render all transparent polygons from farther to closest to the eye (so that multi-transparency works correctly)..

this is the basic premise behind a ray-engine such as Doom and Quake use.

EQ and any modern 3D games are NOT ray-cast rendered, that is an old technology used in games like wolfenstien, doom, quake 1, etc...

This is where the software determines if a pixel is behind another pixel and if so doesn't draw the stuff behind

This is called z-buffering, and it is done on the graphics card... and note, z-buffering works on a PIXEL level, not a POLYGON level...

To see an example of what the z-buffer does, drop down to 16-bit color in EQ... if you have a card that decreases your z-buffer to 16-bit (instead of 32-bit), you will start to see jagged, moving, sawtooth edges where polygons intersect, particularly when the polygons are large... (go to Plane of Tranquility, and look at the water's edge..)

If people are interested in these topics, there are some excellent books out there... here is one I liked in particular:

http://www.amazon.com/exec/obidos/ASIN/0201619210/qid=1045352836/sr=2-1/ref=sr_2_1/103-7651110-9650202

Oh, and blitting is moving image data from one area of memory (video or otherwise) to another area of memory, with some action going during the copy (could be ANDing the source memory by destination, ORing, XORing, alpha blending, etc, etc, etc..)

--Jeeves