Network data changes

[wiz60]

This topic has evolved into an area that I have been curious about for several years. I had a chance to spend a bit of time with the former head of EQ Customer Support and while he was forthcoming on many things abou the game - he was very tight-lipped about these details.

What is the physical architecture of an EQ server?

I realize several things (these are guesses btw):

1. Multiple computers make up a "server". I suspect that these are not all the same.

2. A "zone" runs on a computer as a task. Probably several zones on a computer - and the mapping of zones to computers is probably very flexible.

3. I think they run linux or a unix variant. And I was told the database is homegrown.

Ratt's comment sparked an interest. He concluded that compressing packets for 2000 users would be quite a load on the server. From that I am concluding that he feels all the packet traffic for a "EQ server (ie: several computers)" is routed to a single machine. I suspect that your closeness to the packet mechanisms give you some valid reasons to conclude this structure. I would have guessed each zone communicated to "its" users.

The notion that they might be consolidating the IO for a server to one machine is interesting. I would suggest that perhaps the individual computers forward compressed packets - thus distributing the load. The workload on any computer would be the people in the zones it runs. Certainly thats not 2000 people.

Note: I understand what servers are - and have tried to stay away from that term to avoid confusion with the logical notion of an EQ server.

[LordCrush]

Wiz60,

perhaps you might want to look here for interest ...

http://www.eqemu.net

(How it maybe works)

[quackrabbit]

Originally posted by wiz60
3. I think they run linux or a unix variant. And I was told the database is homegrown.

While I don't know for sure what O/S the servers run on, Brad McQuaid has said in the past that the database back end for LIVE servers are dumps to text files (for fast performance) that are loaded into the servers memory.

He also went on to say that these text files are created from dumping SQL tables - so one would assume they do use some sort of relational database for manipulating data, data entry and research - and this would also explain why a zone has to come down for a database change to be propagated to the zone servers.

Please note: Brad has been gone from SoE for a long time, and these facts may no longer be relevant. Also, while I did try to search for his post, it may have been on the long since wiped EQ Developers Corner message board - so I can not find the source of this message - so flame me if you want.

[casey]

an EQ world is 22 servers running windows NT.

circa 1999 hardware. every server before sullon zek was a copy of the original harware specs of the first server, sullon was the first to take advantage of newer technology.

edit: my info is dated ! yay

i bet you can convince quakrabbit to give you the hard facts if you ask nice. !

[quackrabbit]

Originally posted by casey
an EQ world is 22 servers running windows NT.

There are 34 servers per world now. (1 world server, 33 zone servers)

[Resiliant]

Wanna bet the 'bazaar' zone is on its own server?

I can visualize this, actually. I bet they are 1u high server blades implemented in x86 technology running NT2K, connected to a Cisco router that handles back-end packet switching. Each EQ 'server' is, in fact, a single rack.

It would be SO cool to see this machine room.

[bonkersbobcat]

Guy I talked to from SOE at one of the fan events told me that EQ servers were running NT 4. Yes 4. He also said that each EQ "Server" is a bank of NT4 servers each hosting a number of zones. The number of zones per physical box varies depending on the size and popularity of zones. Zones are not split across physical boxes.

[sam]

Originally posted by Resiliant
I bet they are 1u high server blades implemented in x86 technology running NT2K, connected to a Cisco router that handles back-end packet switching. Each EQ 'server' is, in fact, a single rack.

I'm not sure why people think the servers are running windows. Windows has never and will never run as stable as the everquest servers have always run. The servers have a great track record for stability.. with the exception of a few patch days that caused server crashes. But overall, for a server that gets so much network traffic, and is so memory/cpu intensive, they have been great! Windows has problems with large armounts of memory, windows has problems with tcp/ip (especially when having to handle more than one local IP), windows has HUGE security holes, windows is very expensive, and windows is just all around unstable for any major server environment.

These guys are running some sort of *nix OS. It probably doesn't run on x86 hardware because then if the binaries leaked out, everyone and his brother would have a working copy.

With that in mind, the hardware for these very stable and very resource intensive boxes is probably very expensive. So they dont buy one server for one zone... they would be loading up multiple zones on one server depending on how much average load that zone receives. For example, all the starting town zones have probably been moved off to one server since no one ever goes back to their starting town.

[quackrabbit]

Originally posted by sam
I'm not sure why people think the servers are running windows. Windows has never and will never run as stable as the everquest servers have always run.

I have seen and heard enough to be 100% convinced that bonkersbobcat's post is correct.

[sam]

I may be completely wrong, but if the servers are running on a windows platform I will be stunned and amazed.

[mvern]

The servers has/had a little habit of sending unitialized memory in some network packets. Seeing "WINDIR=C:\WINNT" show up in the trash section of a packet more then once was enough to convince me that they're running NT I havent seen it lately tho, they probably atleast partly fixed it at some point.

[throx]

I'm not sure why people think the servers are running windows. Windows has never and will never run as stable as the everquest servers have always run. The servers have a great track record for stability.. with the exception of a few patch days that caused server crashes. But overall, for a server that gets so much network traffic, and is so memory/cpu intensive, they have been great! Windows has problems with large armounts of memory, windows has problems with tcp/ip (especially when having to handle more than one local IP), windows has HUGE security holes, windows is very expensive, and windows is just all around unstable for any major server environment.

You've been reading too much Slashdot. My last job involved server computers which controlled the traffic signals for an entire city. They ran NT 4 and had uptimes of around 180 days, though some we managed to get to 400 or so (cause we were sceptical about putting service packs on - these weren't public machines). Windows NT runs just find in a controlled environment and will give you incredible uptimes if you have half a clue about administering them.

I did read somewhere that EQ ran on Windows NT (which also surprised me). I have a feeling that EQ2 will be a Linux based system though, from the skill requirements for jobs listed at SOE.

If you want to look at the server room, there's a very quick glimpse on the CNN video on EQ's main site.

http://everquest.station.sony.com/movies/cnn.jsp

[bonkersbobcat]

Originally posted by sam

I'm not sure why people think the servers are running windows. Windows has never and will never run as stable as the everquest servers have always run.

This can get into religion and can feed the fires of much flaming so I am only going to make one comment on this topic.

On stable hardware, in a stable and structured administrative environment, Windows (NT and beyond) is just as stable as unix. Like there are things that have to be done in a unix environment to make it production worthy, there are similar things that are done in Microsoft environments. People get caught up in religion, but properly deployed, both environments are production-worthy.

Microsoft tends to win with ease of administration for less technical (more cost effective) support staff.

Microsoft tends to loose in areas of online changes (ie not rebooting or shutting down for upgrades or patches)

When putting two single boxes side by side in a 24 x 7 situation, unix will win, however in an environment with multiple machines in a partitioned or distributed archiceture the issue becomes more cloudy.

[Pigeon]

It's not as if EQ's servers are all that stable anyway. A zone on my server has crashed at least 3 times in the past week, (3 different zones, all non-connected) and no, I'm not refering to all the zones repopping or the servers coming down for a patch.

For the record, I agree with bonkers. If the sysadmins are well trained and know what they're doing, the end-user should be unable to tell what OS a server is running.

[sam]

There are so many reasons I can think of why windows is a bad OS for anything large scale. But I will no longer discuss this and please, no one else discuss this in this thread. I'll head over to the general discussion forum and post a thread there... it shouldn't be hard to miss and we can fight it out there.