Viewing By Entry / Main
November 14, 2008

ArcGIS Server Sites Under Heavy Load

I have done several viewers for a client of ours using the ArcGIS Javascript api. Point is, this server has been in use for several months now with no problems at all. Its a decent, dual quad core box, and has been handling the load quite nicely. Then came the release of our 'Historical Flight Tracker. Its a site geared towards the the aviation industry. It allows pilots to see where they have flown in the past 30 days. Its a neat little app that has lots of cool asynchronous database calls using coldfusion (cfajaxproxy). Thats another blog post if anyone is interested...

Anyway, shortly after release, we noticed that the lsass.exe (in task manager) process was starting to eat lots and lots of memory and the cpu was ranging upwards of 30-40% at times. Eventually, the entire server would be become totally unresponsive and needed a hard reset. Well, if I was smart, I would have searched the ESRI support center first, but I didnt. I hit the Google and tried every 'fix' for 'high cpu lsass.exe' I could find. Well, after no luck, I finally took a look at the ESRI support site, and it turns out that this is a known issue for ArcGIS Server sites that are under a heavy load, which this one certainly is. Its on the front page (on the right side of the page) FltPLan.com, which is one of the top destinations for the aviation industry.

So, if you have a site that is under heavy load, and you notice the lsass.exe processes getting a little carried away with itself, be sure to have a look at this Tech Article from esri. The long and short of the issue is this:

"Every time a Web service or application that uses impersonation handles a request, the underlying ASP.NET worker process must use the Local Security Authority Subsystem Service process (lsass.exe) to authenticate. Under normal load conditions, this authentication operation is insignificant.

When a Web service or application that is impersonating is under heavy load (more than 25 simultaneous connections per second) for extended periods of time, the per request authentication operations begin to severely affect the memory and processing footprint of the lsass.exe process."

Comments

jason,

so you're running the DOT NET server on the backend, not the java server? any particular reason?


Ouch. You caught me. Paul, I went for the most supported platform available. I did need a platform that I can manage and would suit the client, so in this case we went that route. I'm certainly not going to touch the .net web adf.


Its probably better you did use the dot net server. Server runs faster on a windows machine than a non-windows machine.
When server runs on linux or on other OSs, it actually uses a piece of software provided by MainSoft that interprets COM libraries to Java. I'v heard that extra layer slows down Server.


@jason,

i'm just not sure which way to jump (or even if we should just finally jump ship). lately we mostly need what google maps offers (more or less) with occasional heavy custom/local layers. since ESRI pulled the plug on AWS (at least the free bits), not sure what will happen with the on-line arcGIS.

and you can talk to the DOT NET thing w/cf ok (cf8 dot net integration like we talked about last year or so??)?


@donny,

you mean the java server's not native java?


@Paul

9.3 is a different beast than anything previous. I (me personally) dont do a single thing with .net and have no plans to do so. But, you are right, should I have the need, Coldfusion 8 will talk to .net with no problems. 9.3 included the new REST and JS api's and they will work with both the .net and java flavor of server. Meaning, both java and .net ArcGIS server include the new js and REST api's.

Now, as to your question, is it time to jump ship? Short answer: Yes. The AWS has morphed into ArcGIS online. Its essentially the same, but better. I gotta tell ya, I have been so impressed with all the new stuff that esri has been putting out with 9.3. The javascript api, as it matures, will become the killer app. Using it with all the goodies that come with Coldfusion 8, you can really do some serious damage. No more clunky web adf or soap calls. So, yes, make the leap. Start <a href="http://resources.esri.com/arcgisserver/apis/javascript/arcgis/index.cfm?fa=home">here</a>.


@PaulH
I'm not exactly sure how the whole thing works. But what I can gather is that the MainSoft software acts like an interpreter between C++ and Java. That makes server able to run on Linux and Unix. It supposedly decreases performance.

The MainSoft site should have more info.


http://www.mainsoft.com/


q4C2x4 <a href="http://ogkuqoeszcaq.com/">ogkuqoeszcaq</a>, [url=http://gdzuqycbugyv.com/]gdzuqycbugyv[/url], [link=http://qihntrxyaytv.com/]qihntrxyaytv[/link], http://rlpurrfnvvrl.com/