ROK Technologies ESRI Developer Blog - ArcGIS Server

ArcGIS Server Map Cache, Annotation, and Antialiasing

Wed, 23 Jun 2010 10:45:00 -0400

Here is another quick tip that we hope will save other folks some time. When caching a 'basemap' layer for county/city/municipal, we'll often include Streets, Annotation, and Property Parcels. Of course, these folks almost always have a really nice imagery, and anything that uses antialiasing looks terrible. Just in case you dont know what antialiasing is, where is a definition from ESRI: " Anti-aliasing is a graphics technique that blends foreground and background pixels near edges of objects to trick your eye into seeing smoother borders. You can use this option if unwanted artifacts appear in your map displays -- for example, jagged lines, wavy lines or bands, and moiré patterns. " Well, its easy enough to turn off antialiasing in your map cache (check your MSD properties from the toolbar in ArcMap and Create Map Cache screen). However, we have run into several instances where these settings were not being honored - most of the time it was with annotation layers. No matter what we tried, the annotation layers were being antialiased. Crazy right? Well anyway, turns out, in our case, it was an easy fix. Simply change the cache image type from png8 to png24. After that, no antialiasing in the anno. The image tiles are slightly larger (file size), but not too bad.

ArcGIS Server Identify Operations Slow?

Fri, 04 Jun 2010 12:24:00 -0400

Here is a quick tip for my fellow ArcGIS Server developers. While I have never been a big fan of using the Identify operation, there is no doubt that it can be quite useful at times. Its strange though, sometimes I have noticed that it was terribly slow (I have seen 8 sec+ response times on a simple 4 layer Identify). So, I'd do the normal thing, and remove some layers, and try again. It would usually end up being one specific layer that was causing the huge slowdown. I would just chalk it up to a bad layer and move on. Well, after running into this issue a few times, I couldn't really ignore it and blame it on a 'bad layer'. It turns out that these layers had something in common...The all had complex / detailed geometries. I'm not entirely sure why a complex geometry would effect an identify operation, but it does. So, my simple fix was to use ArcToolbox and perform a generalization operation against the layer. You GIS folks should be familiar with that type of thing. In short, it removes superfluous geometry in the shape. Instead of 1000 points along a curve, it can reduce it to say, 100. Visually, you can normally not tell the difference. But, to be on the safe side, I left the original layer alone, and just made a copy. Then, I use that copy in all all my identify operations instead of the original. Yes, it does create extra overhead and some additional logic in your app, but the run time processing saving is huge. My 4 sec response times have been cut to under 100 millisecs. Yeah, its that big of a difference. So, in my mind, totally worth the extra effort.

ArcGIS Server Map Cache Creation

Fri, 09 Apr 2010 13:10:00 -0400

It seems that more and more folks are looking to host their ArcGIS Server map services off site recently. From what we can tell, its due to a couple of factors. First, would be the total cost of ownership (think map servers, bandwidth, cache servers, web servers, database, personnel). Next in line is probably the complexity for keeping everything running smooth. Having done this for such a long time (MoIMS, ArcIMS alpha, to ArcGIS Server 9) we really know how to make these things hum pretty smooth. But, there is always something new to learn and figure out. The big change with 9.2, but more so with 9.3 (wont even get into the changes at 10 - yet) was map cache. Map cache can be a real pain to create, but the payoff is huge. If done right, your sites will scream (shameless plug for our Map Cache Hosting service here and Map Service Hosting here). But, a few months ago, ESRI threw a monkey wrench into all those map caches we have been creating for 2 or 3 years now. The tiling scheme changed for their free online map services. To better align themselves with Google & Bing, they standardized their tiling scheme to web specific Mercator projection. I find it pretty good, for the most part. And, to ESRI's credit, they gave everyone ample time to start migrating. But, for us, it has been a bit tougher....We have many many (many) map caches to migrate over. So, a couple of things that we have run into that I thought I would share. The new default tile size is 256 x 256. In a perfect world, that would be great. Smaller tiles give you that appearance of loading much quicker, which is great for the end user experience. However, if you are the person at the helm, creating this map cache, it can become a nightmare. We deal with nationwide, and even world wide map caches. using the new default 256 x 256 size creates 4 times the 'normal' 512 x 512 tile size. This adds up to a huge additional overhead, when moving cache around and processing it in general. So, as many of us know, cache creation is very finicky. If you don't get it right, you could waste a day of work. I was curious to know if you could mix and match tile sizes (my custom ones at 512 x 512 and ESRI's at 256 x 256) when working with your own cache, and one of the new ESRI map service. While this may have been obvious to some, I didnt even think to try it, until today. To my pleasant surprise, the answer was yes! Thats going to save me a ton of time in the long run. But, before embarking on one of these big cache migrations or creations, make sure to read ESRI's posts on the subject: http://resources.esri.com/help/9.3/arcgisonline/about/Content/migrating_tiling.htm http://resources.arcgis.com/content/kbase?fa=articleShow&d=37329 Have you run into the dreaded 'Failed to generate cache. All server contexts failed to cache map' error when creating your map cache? While I don't know exactly why, but it often seems due to high memory consumption. I recently was creating a big cache whose base data were lots of small mr sid rasters. Well, it didnt like it. It failed every single time. From what I could tell, the RAM required by each ArcSoc was in excess of 500 mb, and since I was running it with 8 instances, it just didnt have enough RAM to do the job. I wish the error was more descriptive though. If this tends to happen to you a lot, try lowering the number of instances it runs. Might help. Lots of very exciting changes are coming with ArcGIS 10, so stay tuned.

ArcGIS Server Hosting

Tue, 09 Feb 2010 13:20:00 -0400

As a general rule, I don't post things here that are ROK Technologies specific. Yes, I do post on occasional demo or recently released site if I think it merits a mention. I usually stick to tips and tricks that I find when using ESRI based products. So, this is my rare exception to that rule. We have been hosting ArcGIS Server & ArcIMS applications for a long time now - but usually for our existing clients for whom we have developed an application for. However, as time passed, it became clear to us that there was a real need for better, widespread ArcGIS Server hosting. ArcGIS Server, especially with 9.3 has opened up GIS applications to the masses. That sounds very cliché I know, but it really is true this time. These ArcGIS Server services can be published and used in so many different types of clients (desktop, web, etc). But, we are also aware, that setting up a full fledged ArcGIS server environment is not an inexpensive proposition. There are some serious hardware (think multiple quad core, disk space for cached maps), software (think SQL Server & ArcGIS Server), and bandwidth requirements (more than you would think, especially for tiles!) to be addressed. The costs for implementation add up quickly. We won't even get into the ongoing maintenance required. Some folks just want to publish a service and be done with it. And we understand that. Being responsive and always available to our clients is our mantra. If you call us, we pick up the phone, and get it done. So, that being said, I'd like you to know about ROK Technologies ArcGIS Server hosting options. We have some serious infrastructure inplace (see below) and can handle any sized site you require. Some folks may just want to host services to consume in their self hosted applications. Some may want to offload their map cache tiles onto our high speed fiber network to free up their machines for serving their dynamic map services at full capacity. Do you have some data, but not quite sure what to do with it? We can provide a turnkey (think tax assessor site, data browser, and parcel viewer) website that's all inclusive (ArcGIS License, hosting & development). Yes, we can do it fast. Send us your data, and well send you your URL the next day. Or, maybe you need the soup to nuts solution - geodatabase replication, custom application development/hosting, and map cache management. We have you covered there too. Take advantage of our infrastructure. Think about freeing yourself up a bit. Our years and years of experience can help you get your site up and running fast (and right). Please do get in touch if you have any questions. info@roktech.net 888.898.3404 Click for a fullsize shot:

Using ESRI's ArcGIS Server To Help Get Relief Flights Into Haiti

Thu, 21 Jan 2010 14:59:00 -0400

One of our long time clients, FltPlan.com is in the business of helping private and corporate pilots file flight plans with the FAA. Filing flight plans, along with navigating customs can be a daunting task. Fltplan.com has really simplified this process by storing aircraft and pilot profiles. Access to this data makes these filing procedures a snap. We have worked with them over the years to help enhance their service with GIS. First with ArcIMS, and for the past 3 years, ArcGIS Server. When the Haitian earthquake occurred and humanitarian efforts began, FltPlan asked themselves, how could they help? With the largest online community of corporate and private pilots on the internet it was a natural step to offer a clearing house to share information about flying in and out of Haiti to assist with the relief efforts. Fltplan.com has a huge database of airports, waypoints and other flight related data. However, to date, none of this Haitian flight related data, had been available on a internet based map. That's where we (ROK) came in. We took the data and did our GIS thing that we do so well, and the result is a Javascript API based, ArcGIS Server application dedicated to displaying information about flying in and out of Haiti and the Dominican Republic. If you are interested, check out what a pilot would see when planning a flight from Ft. Lauderdale to Port-au-Prince Haiti with this ArcGIS Server site Now, the legions of private and corporate pilots can better assist these ongoing relief efforts and get their full planes of supplies and doctor on the ground where they are needed.

ArcGIS Server Security::Token Based Authentication w/ JavaScript API

Thu, 17 Dec 2009 17:39:00 -0400

Securing services for ArcGIS Server is not as difficult as one would think. The ArcGIS Server Manager works as a great tool to lock down services, create and manage a security database, and grant permissions for specific services and or folders.

Check back later for screen grabs.

Step 1: Before you lock down your server, you will need to create a database to manage the security.
Log in into the ArcGIS Server Manager and click on the "Security" tab. Then click on "Settings".

The settings window will have a wizard that allows you to connect to a database server. The wizard will then create a new database with the needed tables to store permissions. As you are going through the wizard, be sure that you check the box that automatically creates a user and role for "Everyone." We will use this later.

Step 2: Create roles and users
Permissions for the services are managed via roles. After you successfully create a role, you can create a new user or add existing users to that role. Creating users and roles can be done easily via the ArcGIS Server Manager wizard.

Step 3: Lock down your server
Now that you have a database set up, you are ready to lock down your services.
Go back to the "Setting" section of the "Security" tab.
You will see a section that says: "Security for GIS Services is: Not Enabled". Beside this text is a button that says "Enable"

Note: Once you Enable security, you cannot undo this via the ArcGIS Server manager. Don't get frightened away yet, you will see in the next step that this is not a big deal.

Step 4: Grant permission for "Everyone" at the root level
Now that your site is locked down, you need to grant permissions to your services.

The next few steps are going to assume you have some public services and some private services. Even though security is turned on, we want to make sure that our public services remain public, not requiring a username / password.

Click on the "Services" tab. It should automatically load the "Manage Services" section. If you are not already there, change the "Services In" pulldown value to your root directory. Then Click Manage Folders -> Permissions. At the root level, grant "Everyone" permission by adding the role to the Access List.

Now security is turned on, but all of your services are public. I recommend this set up for any initial installation of ArcGIS Server. This will allow you to quickly add permissions when needed.

Step 5: Lock down a folder in your ArcGIS Server
Now that all of our sites are public, we want to lock down specific services.

If your services are organized into folders, this can be done easily. Let's assume you have a folder called "Private". Switch the "Services in" value to your private folder. Then, click Manage Folders - > Permissions. Add the role you created to this folder. Be sure to remove "Everyone" from the Access list.

Now your services in this folder will require the a username and password that is associated to that role.

Step 6: Access the Secure Folder via REST interface.
If you're developing applications with the ArcGIS Server Javascript API, the REST interface should be your best friend. However after you lock down a folder, you will notice the interface for that folder is gone from your root services directory:
http://<servername>/ArcGIS/rest/services

To access the "Private" folder, simply type in the URL for your secure folder:
http://<servername>/ArcGIS/rest/services/Private

This will open up a login screen. You can now access the folder with a username / password.

Step 7: Using a secure service in your ArcGIS Server JavaScript API application
The following setup will work great if you would like to have a public ArcGIS Server Javascript API application with secure services embeded into the application. As a result, users will not be able to grab your service URLs and use them in their own applications.

To keep this blog from getting to long, we are going to assume you already have a proxy page set up in your application. If you do not, that probably means 2 things. You are not printing your maps, and you are not receiving data back in IE where the queries are longer than 2000 characters. Here is a link to setting up your proxy: http://resources.esri.com/help/9.3/arcgisserver/apis/javascript/arcgis/help/jshelp/ags_proxy.htm#webprint

Since we're assuming you already have a proxy set up, the first step is going to be an updating your settings for the proxy in your JavaScript.

The sites are now secure, so you will need to use the proxy page for each request. If you have not done so already, change the "alwaysUseProxy" setting to "true" in your application.
esri.config.defaults.io.alwaysUseProxy = true;
Additionally, change your http references to https for your secure servcies.

Step 8: Token Based Authentication
Token based authentication is used for authorization of ArcGIS Server Services via the JavaScript API. You have two options: 1. Create a token via the Token generation page. 2. Create a token Dynamically

Option 1. Create a token via the Token generation page
After you turn on security for your ArcGIS Server installation, you will have access to a token generation page. http://<servername/ArcGIS/Tokens/ gettoken.html (You can also get to this page from the REST interface.) Type in a valid user name/password and it will return a token value.

Change the http reference in the proxy.config to https. Copy and paste the token into the service url references in your proxy.config file.
Sample from ESRI's Help:
<serverUrl url="https://www.example.com/arcgis/rest/services/MyMapSecureService/MapServer" matchAll="false"token="5fFo4%2fI4Tv8IGSqSYbpUNORRD%2fYxXMSPo6NEHNNGMpt9CMknpXIjEVqYGm3uuQnU" ></serverUrl>

You will notice on the token generation page that by default this token is only good for 10 days. You can change this if needed in the ArcGIS Server Manager. Optionally, you can set up your services as done below.

Option 2. Create a token dynamically:
If you wish to create tokens dynamically, you will need to download a new proxy.ashx page.

We posted a question on ESRI's users forums about this setup. Thang Le from ESRI Support, provided us with the new proxy file. You can download it from the user forums here:
http://forums.esri.com/Thread.asp?c=158&f=2396&t=297001#926770

After you download the zip file, replace your existing proxy.ashx page with the new one. Then, you will need to update your server url references in your proxy.config file. Change the http reference to https. Add the following new attributes:
dynamicToken="true"
host=""
userName=""
password=""

ArcGIS Server JavaScript API version 1.5 - Bug with Drawing Toolbar deactivation?

Thu, 12 Nov 2009 15:29:00 -0400

After upgrading our ArcGIS Server JavaScript API sites to version 1.5, we have noticed some strange behavior with our Drawing Toolbars. We use a Drawing Toolbar to allow users to draw a line or polygon to select features on the map. The geometry from the drawing toolbar is used as the geometry in the Query for the QueryTask.

At version 1.5, we noticed the last segment of the line or polygon was not being used in the QueryTask results. Consequently, no features in the last segment of the geometry were selected on the map. All other segments of the geometry were working fine.
User Selection:

Parcel Results: (Notice the last segment did not return features)

Bug:
We use 3 different drawing toolbars on our site for measuring distances, selecting features, and drawing graphics.

The problem came when we activated 1 drawing toolbar, then deactivated another toolbar after the activation.

//switch to select toolbar
selectToolbar.activate(esri.toolbars.Draw.POLYLINE);drawingToolbar.deactivate();
Soultion:
At version 1.5, you must deactivate any toolbars before you activate one.

//switch to select toolbar
drawingToolbar.deactivate();selectToolbar.activate(esri.toolbars.Draw.POLYLINE);

At previous versions of the ArcGIS Server JavaScript API this was not a problem. I'm not sure if this falls into the category of a bug, but it is certainly something to be aware of when you migrate your applications to Version 1.5.
The ESRI Fourm blog referencing this issue can be seen here. Thanks to xiaoming - Prince William County for finding the work around.

Trent Tinker
Developer - ROK Technologies

Hey You, Dummy, Read The Docs

Tue, 10 Nov 2009 12:36:00 -0400

I just wasted a good 2 hours trying to debug an issue with a AGS Flex application I am doing. I could not, for the life of me, figure out how I was able to deactivate a Draw Toolbar without explicitly calling drawToolbar.deactivate. It was driving me crazy. So, I decided to revert back to the Flex 1.2 API and sure enough, it was necessary to explicitly call drawToolbar.deactivate to stop drawing. However, I reason I had to revert back was because I think I may have found a bug with the ArcGISTiledMapServiceLayer tileServers. It was just coincidence that I tried 1.2. Turns out, the big dummy that I am, this is new (and very welcomed) behavior as of 1.3. I should have stopped, and read the docs and could have saved myself some time and frustration.

Combining ArcGIS Server Images - Serverside

Fri, 30 Oct 2009 15:05:00 -0400

Printing aint what it used to be. The days of sending that request off to ArcIMS and getting back a print - ready image are long gone. In fact, I can safely say that one of the few things that ArcIMS does much better than ArcGIS Server is printing. These days, we have multiple services, markup, etc all on different layers being combined (layered), client side to create the illusion of a nice neat merged image. That illusion doesn't translate very well when it comes to printing. There have been a few hackish (in my opinion) attempts working around this by combining these different layers of images serverside. Personally, I it looked like a mess, and it was difficult to configure. I set out a few months back to try and figure out an easy way to combine these images using my serverside language of choice, Coldfusion. I was extremely surprised to find out that I couldn't not do it. I tried about everything in Coldfusion's image processing function library, and I was just stumped. I asked around and I could get anyone that could get this to work either. Until today. The other day, I got an email from Ann, and she was looking for some advice on using REST calls to ArcGIS server from serverside. Then more specifically, she asked about combining images from a cached service and a dynamic service (or really any two services): "Is it possible to use arcGIS to produce static images which consist of one base layer (cached) and an overlay? " I, having tried this already, quickly replied no way. To my surprise, this morning, she sent over the following code (modified slightly for example purposes) that's using the Coldfusion ImagePaste function:

I cant believe that I missed something that easy. Here is the example: Image Merge Demo Thanks Ann!

Article on Map Cache Hosting in ArcUser

Thu, 22 Oct 2009 11:20:00 -0400

Those of you that get ArcUser, make sure to check out the Article 'Tiles On A Cloud'. Its a good rundown of what we have been doing for our clients that need to offload some bandwidth for their cached tile-intensive applications. I have blogged about this service before, but this article does a good job in putting it in laymans terms.

ArcGIS Sever In The Cloud Gets A Little Closer

Wed, 26 Aug 2009 10:49:00 -0400

I just opened up my inbox to an announcement from Amazon. Looks like they are continuing to push their cloud computing services. Now this one can make all the difference for us GIS nerds. Its called Virtual Private Cloud. Giving it a quick read, it looks like you'll me able to add an Amazon Cloud based server to your existing network, via a standard VPN connection. Here are the details from Amazon: * Create a Virtual Private Cloud on AWS's scalable infrastructure, and specify its private IP address range from any block you choose. * Divide your VPC's private IP address range into one or more subnets in a manner convenient for managing applications and services you run in your VPC. * Bridge together your VPC and your IT infrastructure via an encrypted VPN connection. * Add AWS resources, such as Amazon EC2 instances, to your VPC. * Route traffic between your VPC and the Internet over the VPN connection so that it can be examined by your existing security and networking assets before heading to the public Internet. * Extend your existing security and management policies within your IT infrastructure to your VPC as if they were running within your infrastructure. The first thing that came to my mind was spinning up ArcSOC servers to handle additional load if your main box (on your own physical network) gets swamped. Just add a new SOC server (an EC2 instance) to your existing ArcGIS Server. I have no idea if the network latency would be an issue in a distributed environment like this, but I suspect it will be. I'm going to test this out as soon as I can and will report back. You could move your entire ArcGIS server operation up to the cloud, but as we have discussed before, it can be prohibitively expensive to do so. With this new method, you can bring them online only when you really need them. I'm sure James Fee has 'ArcGIS Server Cloud' setup as a Google Alert...so what do you think James?

Geoworld - Resource Of The Month

Thu, 13 Aug 2009 11:42:00 -0400

Our recent HivAtlas application (ArcGIS Server Javascript API site) was just highlighted in the latest issue of GeoWorld. Its on page 9 - "Resource Of The Month". I always find it strange when the folks who publish these articles never give you a heads up. Anyway, its nice to be recognized. Lots and lots of work went into that application. And - its one of the many ArcGIS Server sites that are hosted here at ROK...If you would like any information on our ArcGIS Server hosting services, drop us a line

We are going to need a bigger boat.

Mon, 29 Jun 2009 11:59:00 -0400

That clip is the exact reaction I had when I walked into my office last Monday morning. Beginning last week, we got absolutely crushed/overwhelmed with traffic. A site that we developed and hosted was suddenly 'discovered' by the media (fox, associated press, USA Today, front page of DIGG, etc). Check out the sheer number of stories on it on google news here It continues today, like right now....currently our site is featured on the front page of CNN. Its a site that shows HIV disease prevalence across the country by county (and in some cases zip code) maphiv.org. I will warn you that registration (its a quickie) is required if you are interested. Its an ArcGIS Server javascript api based site running on 9.3.1 using msd's & mxd based services with multi layer caches. We plan an extensive series of posts when things calm down a bit that will focus on the many lessons that we learned from this overwhelming amount of traffic. QUICK UPDATE 12 noon edt: Our ArcGIS server is taking close to 20 requests per second and still holding strong.

New Site Using Remote Cache Hosting Service - ArcGIS Server in The Cloud

Fri, 19 Jun 2009 12:58:00 -0400

Hey folks...Just wanted to share a site that is now using our remote cache hosting service to serve cached tiles. I previously showed several Javascript API sites uing this service, but this one is using Flex api for ArcGIS Server. This really frees up bandwidth to let the dynamic layers be served quickly. Those aerial images can be very bandwidth intensive, so hosting these remotely them makes a lot of sense when bandwidth is limited, or you have a high volume site. Here is is: Santa Rosa Planning and Zoning Viewer. The layer that is being served remotely is the 'Aerial Photos' layer. If you are a developer geek like me, check out the site with firebug enabled so that you can see the Aerial imagery being served remotely - meaning a different location that the ArcGIS server itself. Cool stuff.

Part 1 - Whats Is A Geoprocessing Task And Why Should I Care?

Wed, 17 Jun 2009 13:11:00 -0400

So, lets begin this discussion of Geoprocessing with ArcGIS Server that I promised. I guess the first question is what is geoprocessing and why should I even care? The big promise of ArcGIS Server has always been this: "Whatever you can do in ArcMap and ArcCatalog, you can now do with ArcGIS Server". I have often heard ESRI reps say 'You can even rebuild ArcMap if you want to'. Of course, no one would ever want to do such a thing, but wouldn't it be nice to have all that functionality available to you in a web based application? In the old days of ArcIMS and MapObjects IMS we were pretty limited in what types of geoprocessing we could do on the web. I can recall all sorts of crazy tricks with buffers that we used to write to really push ArcIMS to its limits. But, at the end of the day, it was still just a buffer, and there is only so far it will take you. How about a select by location, or a union, or a clip? Sorry folks, you were out of luck if you needed to do that on the web. But then, ArcGIS Server came around, and suddenly doing real geoprocessing became a reality. So, what is a geoprocessing task? In short, it is a set of server side tools called from a web based client. Users create a model using Model builder, and then publish that model to the web as a geoprocessing task. The output parameters of the model are then streamlined to your web application. Lets take for example a recent Flex application that I wrote (Check it out here). The purpose of this application is to help citizens locate their property and then return information about the zoning on that property. It sounds easy enough, right? I'll bet you are thinking, why not just do an Identify, and return results from the zoning layer? Ah, if it were only that easy. There can be many, multiple types of zoning found on a single parcel, so a simple identify by point will not work. We need to do a spatial analysis to get the property boundary that the map click point or address point resides in. Once we get the property boundary, we need to do a second spatial analysis to determine which zoning layers intersect that property boundary. So, how can we use a geoprocessing task to help solve this problem? I started by simply going into ArcMap and writing down the steps that it takes to get the results I need. Then, I used model builder to string together these tasks into a single workflow. Finally, I published this model to ArcGIS Server so I could access it from the web. The, the final product takes in a point, selects the containing parcel for the point, and returns the zoning types that are present in the parcel. Now, this is a pretty straightforward example. We could take this example much further. We could actually clip that zoning layer and determine precise percentages of each zoning type on that parcel. For parcels with multiple structures, we could determine the zoning district for each structure. List goes on. This scenario, while certainly not impossible to do by conventional means (thinking FIND, QUERY, and IDENTIFY tasks), becomes much, much easier to solve by using a geoprocessing task. It also combines all the steps into one task. Instead of writing a complex combination of code to perform these steps in javascript or actionscript (flex) on the client, I have created a new, reusable tool that does all of these steps for me, serverside. I hope this clears up what a geoprocessing task actually is and why you should care. Next up in the series, we'll tackle the technical aspects of actually creating and executing one from your web application.