Tuesday, April 01, 2008
Posted on Tuesday, April 01, 2008 6:58:05 AM (Mountain Daylight Time, UTC-06:00)  Comments [1] | 
Categories: ArcGIS Server | Podcast

I did a podcast interview while out at the Developer Summit. It's focused on the ArcGIS Server Tile Server project that I submitted to the code challenge. I give a high-level overview of the project, and talk a little about my ideas on designing for ArcGIS Server - big ups for the SOAP API, and the 9.3 REST API.

You can check it out here.

Monday, March 24, 2008
Posted on Monday, March 24, 2008 7:35:15 PM (Mountain Daylight Time, UTC-06:00)  Comments [1] | 
Categories: ArcDeveloper | ArcGIS Server

Could anyone who is using the ArcGIS Server Virtual Earth Tile Server code please shoot me an email or drop a note in the comments. ESRI is preparing a press release about the Code Challenge winners, and they asked for a list of people/agencies etc using it. Some free publicity if you answer quickly!

Thanks!

Thursday, March 20, 2008
Posted on Thursday, March 20, 2008 8:45:22 AM (Mountain Daylight Time, UTC-06:00)  Comments [2] | 
Categories: .NET | ArcGIS Server

Official Line: "The .NET ADF is a hybrid platform - mix of server side and client side development environment."

Built on ASP.NET Ajax, but still supports the ASP.NET Callback model for backward compatibility.

As a quick aside - Art Haddad was running a Mac...

Core object model is relatively the same as at 9.2

Core controls are now scriptable - they work with the MS Ajax client side libraries. The javascript is now "public" and documented. This is great - in the past we had used some of the client side javascript (not supported) and found that it changed across service packs - doh!)

Common Data Source API - standard access across the spectrum of data sources. The common data source API will expose different functionalities based on the type. You can get down the the ArcObjects if need be *I'm assuming this is the same model as at 9.2 but they were moving through stuff very quickly)

This ADF has a Visual Studio design experience that provides an option to do things without getting your fingers in javascript. That said, the javascript is public, so just like the Ajax Control Toolkit, you can just jump in and use it like the "Javascript API" (which brings up the fact there are two javascript APIs - one based on Dojo and one based on MS Ajax - choice is good).

Task Framework is now extensible in the Application Manager via web controls (dlls). This will allow you can publish new tasks, and users can then include the tasks in a published application. Nice feature if your environment is involved in the author-publish-consume workflow.

They are releasing tasks into the code gallery, will allow a faster release cycle.

More "comprehensive" class library documentation, discussions of Ajax patterns, migration from 9.2 to 9.3. Lots of work on the Resource Center to help people get rolling quickly.

Can create user controls for the task frameworks (ascx) - much simpler than creating full blown web controls in dlls.

Performance

Blending at 9.2 used a single tiling scheme. At 9.3 each resource has it's own tiling scheme, and the "blending has been massively improved"

Use of HTTPHandlers used instead of Pages so the page lifecycle is avoided, so it's faster.

Javascript is public, documented, JSON based, and object oriented.

Better activity indicator so you know that something is going on. 

Demo of 9.2 vs 9.3 performance. Definitely much faster. Eliminated lots of callbacks, httphandlers improve performance because the control tree is not re-created and destroyed.

Webcontrols

MapControl - scriptable directly. This is provided by implementing IScriptControl, and just works. The overview map had been improved a lot. The logic has been streamlined, so it's much simpler. You can enable the "intelligent" mode. At 9.2 there were 12 callbacks to get the overview. At 9.3 it's 4 and then it's just client side.

The resource manager can be managed in javascript, and has lots of configuration - i.e. layer alises, fields to show, map tips etc.

Results Viewer - bi-directional high light. The task can have different results behavior (fields, maptips etc)

Additional web controls, additional AjaxExtenders - DockExtender, HoverExpandExtender (pin window type of thing)

Supports .NET 3.5, but not required.

By final, there will be javascript intellisense for Visual Studio 2008 (you likely need to get a Visual Studio service pack as well as the ADF)

Better templates, but if you use these, be sure to send feedback to the team during the beta to get your features in the list.

To use the "Client side geometry model" - Just set the graphics layer RenderOnClient property to true. Sweet.

Automated migration for the Manager applications (author-publish-consume  three-click workflow)

There is a utility to help convert the template apps to 9.3

To do it manually - add reference to the System.Web.Extensions.dll, upgrade assembly version reference. Script manager is not required. If you want it to work with the Ajax partial postback model, you need to add a script manager.

Can use an update panel with non-Ajax controls and use triggers between the .NET ADF controls which are already Ajaxified.

Do not put the .NET ADF Controls in an update panel. This will kill your performance.

 

Questions:

Client geometry sent in JSON... GeoJSON, same JSON as REST API? It's ADF JSON - basically serialized versions of the server side .NET classes. Standard .NET design pattern

Have you worked with the controls in a javascript UI layout framework like Dojo, ExtJS, or YUI? They have done some testing with YUI components, and it's fine. All the client side code is namespaced, so there should be few issues.

Compressed / minified Javascript? By final

Thoughts:

Sounds like we can work directly with the MS Ajax Javascript API, which I personally like. The hybrid approach is very compelling for many apps which need some form of server state, but still require a very responsive interface.  Use of the ADF vs the REST/Javascript API may come down to a question of licensing.

Wednesday, March 19, 2008
Posted on Wednesday, March 19, 2008 9:51:50 AM (Mountain Daylight Time, UTC-06:00)  Comments [3] | 
Categories: ArcGIS Server | Life

So, despite completely spacing the voting, I just walked into the Keynote session to see my name up on the big screen winning second place. Wow. I'm totally surprised, and honored. Thanks to everyone who voted for the Virtual Earth Tile Server, and if you have used it, and made changes - please be sure to submit the fixes / updates / patches etc. either to me, or directly into the subversion repository. To get write access into the repository you need to be a member of the project @ Assembla.com - just shoot me an email (contact link above), and I'll send you an invite.

I would list the other two winners, but honestly I was too floored to take notes.

Again - thanks to everyone who voted!

Posted on Wednesday, March 19, 2008 8:27:15 AM (Mountain Daylight Time, UTC-06:00)  Comments [0] | 
Categories: .NET | Ajax | ArcGIS Server | ASP.NET

Yesterday I focused on the REST, and Javascript API's, and today's task is to get the skinny on the re-vamped .NET ADF.

In talking to some folks, apparently the .NET ADF includes a Javascript API based on the Microsoft Ajax client library. In contrast to the relatively simple Dojo based API, apparently this one has more depth, and with that (one would assume) comes more fine grained capabilities. So, we'll see what some pointed questions can drag out.

At a high level, I think that the .NET ADF may have a tough sell moving forward. The REST stuff is clean and elegant. It may be that on paper it can't do as much, but I believe that there are creative solutions that can step around any limitations. Secondly, everyone I've talked to who has used the 9.2 .NET ADF, well, I'll be charitable and say they are "luke warm" on it. The difficulties they faced in getting good performance, and implementing seemingly simple functionality is what will drive them to try the REST API before the .NET ADF. And if the licensing story for the .NET ADF remains the same, and the REST API is gratis, well, money talks.

I hope that the 9.3 ADF does more than support the ASP.NET Ajax "extender" model (this is how the Ajax Control Toolkit works). From my work with the toolkit (which has been restricted to the ModalPopupExtender and the ListSearchExtender), it's a nice model if you really want to keep your fingers out of the client side code. The difficulty is that since the extenders are setup as part of the page lifecycle, they seem to need everything they interact with to be "runat=server". For reasons I'll skip right now, this is not always easily done. Since all this behavior is implemented in the client, theoretically you can wire up events and use the client side code however you like, but in the Control Toolkit, this is not exactly documented (please send me details if you know otherwise!).

Getting back to the .NET ADF, I hope that the Javascript libraries are designed to be directly consumed and manipulated. Personally I like the Microsoft Ajax client library, and working with server side services is just a slick developer experience. So - I have high hopes.

Tuesday, March 18, 2008
Posted on Tuesday, March 18, 2008 5:13:52 PM (Mountain Daylight Time, UTC-06:00)  Comments [0] | 
Categories: Ajax | ArcGIS Server | Web Mapping

The Javascript API is based on Dojo. ESRI has created some dijits for a map, and the basic pattern looks really similar to the Virtual Earth maps.

The Google Maps extended is just more javascript which extends the base GoogleMaps API to play nice with the ESRI services.

2D and 3D in Virtual Earth is also fully supported - again via javascript extensions.

Slightly different syntax across the APIs, but same basic patterns.

Embed maps and tasks into sites

Use ArcGIS Online or custom (ArcGIS Server) data sources.

Any supported projection. So you polar explorers are now in luck.

Javascript can be hosted by ESRI (like VE or Google Maps) and is in the Akami cloud (think fast, and globally cached - a nice touch).

Samples will be hosted in a live server. Much easier to dig into the samples because they are live. Can copy paste code into your html/aspx/??? file and start modifying. Very streamlined development cycle.

Examples working with a Dojox.grid (Nice because it takes a DIV as a target so the page layout is pretty smooth)

http://maps.esri.com/agsjs/surfaceprofile.html - surface profile demo

InfoWindow can contain whatever you want - i.e. another map

Symbology Model

Really nice demo of a symbol manager based on dojo controls. Incidentally it was running in FireFox on a Mac. Lots of options re: how client side features are displayed. Should be relatively easy to allow client symbolization. Again - looks very performant. They also have symbol sets. They will have a code-gen tool that will allow you to use this tool to setup your symbology, and then export the code to drop into your app. Also running in Safari. Only Solid_Fill supported in FireFox 2. All the saucy picture fills are supported in FireFox 3.

Again stressing the simple, but you'll need to have your CSS, HTML and Javascript chops in shape.

Virtual Earth Extender

Only Tiles - no dynamic layers, but all results can be sent back as VEShapes. Also an interative SDK. Looks just like the VE Interactive API. Javascript hosted by ESRI.

Google Maps Extender

Tiled layers or dynamic layers (via GGroundOverlay). The extender allows you to easily pump results back into GM. The Extender removes the need to convert between the native classes in the base API and the ESRI javascript API - i.e. you can pass a GM LatLongBound into a query. This should be a very smooth developer experience. 

Mapplets

Run within Google Map web site. Can be published to Google Maps. Pretty cool way to push stuff out.

Thoughts:

Again - this is really great stuff. Simple stuff is really easy, and I'm betting that the hard stuff is still difficult, but I'd wager it's miles better than the 9.2 ADF experience. We'll see what the 9.3 .NET ADF has in store at tomorrows session...

The information flow here is very important - design the application to show focused data in small chunks. This allows very small, fast transactions. That said, if you try to push tons of data into your browser, it will bog down.

Since the Javascript is hosted by ESRI, presumably anyone can just include it on a page. From there, you can consume the free ArcGIS Online services, and of course you can mash-in anything else you want. So it would see that there is a world of good free fun just waiting to be explored...

Posted on Tuesday, March 18, 2008 11:59:34 AM (Mountain Daylight Time, UTC-06:00)  Comments [0] | 
Categories: ArcGIS Server | Dev Summit

I don't have the WEP key yet... so not on the fly updates... This is just a dump of notes taken on the fly - I'll be talking to people later and getting more details on a lot of this. I'm really looking at the REST stuff, so if you have specific questions, drop them in the comments and I'll see what I can do to get an answer...

Plenary Intro...

ESRI using Coverity for static analysis - lots of bugs fixed as a result. Lots of hype for Coverity.

Resource center being expanded from ArcExplorer to the entire line.

ArcGIS Server

Ismael Chivite - AGS Product Manager

<koolaid>

  • "Making GIS knowlege available on the web"
  • "Complete Server GIS"
  • "Core Services"
  • "Replication and Federated GIS Systems"
  • "Out of the box applications access core services"
  • "Use Google or Microsoft Virtual Earth to connect to the Services"

</koolaid>

Performance Improvments

What we've heard before - partial caching, on demand caching.  Nothing about improved performance for creating the caches

Web apps - lots of effort in improving the .NET ADF. Less round trips to the server. Built on MS Ajax.

AGS Image Service = Image Server, with smoother publishing workflow. Drag and drop raster in Catalog to publish it. Nothing on how you setup the overviews, so maybe that's automated with defaults.

Security - many more options - ASP.NET membership is an option, and you can implement your own provider.

Developer Center Resource Center - portalized content, but do they have good search? Also have web services that can be included for free (interesting - wonder what the TOS are on this?)

Apparently lots more content, better organized help. Will there be any FooClass.DoThing() --> Does a thing API documentation

Code Gallery - more tasks etc will be incrementally released. Nice touch. Will include source code. Will there be stuff beyond "tasks"? The community will also be able to upload code - nice.

Rex Hansen Demo

Using the Resource Center to find info on the Microsoft ASP.NET Ajax framework. Many samples - showed a Partial Postback sample. Personally I'm more into using the lower level ASP.NET Client library with JSON Services than the update panel, but that's just me. Moved to creating a custom web part, that can then be pushed into SharePoint. Points pulled from a CRM (unclear on how they are linked), with a ArcGIS Online Street map. I'll be playing with this to see how the cascade performs - i.e. does the client make the street map request directly to ESRI, or does the Server do it? What's the lag time? Are the points geometry in the browser a la VEShape?

Rex also talked about the Java ADF...  did some editing tasks - attributes shown in a pop-up instead. Not sure if it was for simplicity but the demo showed changing a building permit type - defined as a text field - wouldn't this be a Domain? Maybe the domain pull downs are not cooked yet? Or this is just a very simple demo...

Javascript and REST API

This is one of the few areas with "NEW" functionality. Can you say Mashups? Ismael compared Javascript to Avenue or Python... "it's very easy to create applications with the javascript API"... um yeah. It's just object based Javascript, REST, JSON, angle brackets, and CSS but it's very easy. "Easier" - maybe, "easy" - only if you've already cut your teeth in these technologies.

Jeremy Bartley Demo

No web server licensing for any of the REST/Javascript API's. This is great news.

Interactive site for the Javascript API, with line by line review of the sample apps. Showed the simple markup for the "basic map". Very VE like in terms of setting up a map. Nice looking API with clean documentation - I will likely be spending quite a bit of time here. Also a community area.

Samples - lots of them covering all(?) aspects of the API. All samples use ESRI services, so they are easy to get up and running locally. Nice.

Talked briefly about using the geometry service. Looked like a couple of round trips, but seems very fast.

Geoprocessing task - summarize population within a geometry - pretty simple. There is a GP Explorer that lets developers poke around on a server - includes an image of the model (shudder). Pass arguments into the service as JSON, so should be interesting.

Nice demo using the Dojo Chart API. All rendered on the client. Very sweet.

Google Maps and Virtual Earth Integration

Nice demo that takes a Google Route, and generates a profile.

Virtual Earth sample that gets hurricane tracks, and draws them on the map (not clear where the tracks came from) but they were then passed to a GP service to buffer them. Are the extenders capable of taking VEShapes directly and shooting them to the GP Task?

<koolaid>

"I have not met anyone who could not create a map application in 15 minutes" - Ismael

</koolaid>

Mobile ADF

Pretty cool stuff, and I hope to get into this. I'm sure there are improvements, but not having done more than a very very simple prototype back in the 9.2 beta, I'm not really one to say much on this.

One thing of note - there was a demo that showed a "workflow driven" application. Hurray! It's not always all about the map! This is what a lot of people need to understand - just because you can show stuff on a map, does not mean that it's the best way to interact with the an application. Very nice.

ArcEngine & Desktop Review (fast and furious by Euan)

More editing options - SQL Server express mentioned, but thin on details. More "use GP" talk. Better API documentation, better indexing of the documentation. Eclipse add-in for better java debugging.

Desktop has some Z-Value editing (??) HTML popup window in desktop for simplified identify type functions. VBA 6.5 will be in final, to make things easier to deploy on Vista and interop with MSOffice

Error report demo - ESRI will provide debug symbol servers. Helps with sleuthing "random crashes"

ArcGIS Explorer

Rapid evolution, releases every few months. Version Neutral (not tied to the rest of the ArcObjects API), local and remote data support.

New symbols in build 450 will also be in 9.3 - basically some prettier icons. Ability to control the look of the attribute popup via CSS. Ability to add Google 3D warehouse via KMZ/KML. Nice demo showing Youtube video integration, live-traffic cams etc. Nice, and really just catching up to Google Earth.

Tasks is where things diverge from Google Earth. More pre-cooked tasks available online. GeoNames.org demo piping one task into another task. Focusing on integration and... yep... mashups.

User Showcase for community tasks. Looks like ArcScripts' days are numbered.

I'm going to have to play with Explorer more - looks like you can do a lot of stuff. Nice demo of vertical slicing.

Future builds will have multi-thread tile fetching, easier task updating (push? version check/update?)

Even further out - more Ux updates, integrated 2D and 3D support. Markup and collaboration (sweet), and developers can use the 2D and 3D controls in our own apps! Woot!

Andy MacDonald showed off a "build 600" version with a Ribbon UI - context driven UI like office. Nice basemap "gallery" integration directly into the UI (instead of opening a browser first). Flyout/self-docking windows. I think this is later this year?

Explorer SDK...

Available with Build 600 & 700. Map Control. Sweet. Super Sweet. It's certainly not a MapObjects replacement, but it sure is nice. API has been expanded - easier to use - lots of overloaded constructors. Less "interface" driven API - easier API exploration via intellisense. More helper methods to simplify verbose tasks. Collections now support enumerations so looping is smoother. Licensing?

Scott Moorehouse Big Picture

"Web is now the center of ArcGIS."

At 9.3 the focus was on finishing concepts introduced at 9.2. "Federated" - everyone take a sip. Web = Easy to Use. Get things done. ArcGIS is part of a broader community of products - i.e. we'll play nice with other stuff.

9.4 will be an incremental release. Nothing crazy new. Will be modularizing more aspects of the platform, which will allow ADF's and Web APIs will have faster release cycles. Also more Linux support. Complete re-write of the dynamic map engine (sounds like new rendering engine for better performance?) for server. Silverlight SDK coming. More Geodatabase APIs, better web APIs for GDB transactions.

Wow this is long session...

EDN Community - Jim Barry

Growing online resources. Team has been expanded. Little bit of "ESRI and You lovefest". ESRI has blogs. woot. Apparently there will be more blogs. woot. Forums *may* be updated. They had some new better forums during the 9.2 beta, but apparently that did not go anywhere. Jim asked for feed back on the forums so here goes:

If nothing else, add RSS feeds into the forums. 

 

Although I missed the first few minutes, no mention of "author publish consume"... wonder if that's because this audience may be a little more skeptical of the three-clicks to greatness story line.

Found the WEP key, so more updates later...

Posted on Tuesday, March 18, 2008 8:32:20 AM (Mountain Daylight Time, UTC-06:00)  Comments [0] | 
Categories: Ajax | ArcGIS Server | Dojo

Based on the Super Tuesday sample site, we got a preview of what ESRI is planning in 9.3.

Map Widgit

Based on Dojox, ESRI has created some wijits (are they dijits if they are based on dojox?) of their own, which we can see in the dojo.require statements...

 dojo.require("esri.map");
 dojo.require("esri.tasks.query");

I'd assume there are more packages lurking around, but this is what I found in a quick look around.

I also noticed something interesting in the constructor for the map - usePlainJson - could this indicate that they have a compressed/encoded Json format? That would be really good, as you can get a lot of bloat when you start shipping around un-compressed JSON geometries. Google uses an encoding scheme, which John O'Brien has written up in an article, and implemented for Virtual Earth so that may also be in the mix.

You can see the javascript for these components here: (it's minified, so it's not exactly readable, but is publicly available) http://serverapi.arcgisonline.com/jsapi/arcgis/?v=1

 

Vectors in the Browser: Dojox.gfx

Dojox is the next generation of Dojo dijits which are pushing the edge of what's possible. The Dojox.gfx package provides an abstraction layer for dealing with vector geometries in the browser - it renders SVG or VML depending on the browser.

I'm glad that we chose to start working with Dojo, as it looks like it will come in handy really soon ;-)

 

In my opinion, the REST services and the Javascript API are "the" 9.3 features, and I'll be focusing most of my attention on this over the next few days.

Monday, March 10, 2008
Posted on Monday, March 10, 2008 10:57:24 AM (Mountain Daylight Time, UTC-06:00)  Comments [1] | 
Categories: ArcDeveloper | ArcGIS Server

I've mentioned the ArcDeveloper project in the past, and called out the community to "just do it" if we really wanted an ArcGIS Server REST API.

Well, the good news is that it's now alive and kicking. At this point it's lean and mean - you can:

  • Get a single feature by id (must be OBJECTID, ugh)
  • Query multiple features with a where clause and/or a bounding box

The requests look like this:

http://65.101.234.201/rest/rest.svc/TestService/Flyways/10?g=true

(go ahead and try it - it's live and will return GeoJSON)

Currently there is only one format - GeoJSON, and it's not a 100% implementation yet, but it's enough to get the geometries and attributes.

The project is designed around a plug-in model - so as the need arises, we can write different formatters (think ESRI 9.3 REST compatibility), and different data services (think SQL Server 2008).

Huge props go out to Glenn Goodrich, who wrote the vast majority of the existing code. He has posted an Intro to the project that talks about how to get the code and the requirements, as well as a second article about how the end point works. I don't want to speak for him, but it looks like he will be posting more about the code, so go ahead and subscribe now.

For those with a short attention span who want to see the code, here's the subversion repository:

http://svn2.assembla.com/svn/arcdeveloper/ArcDeveloper.REST/trunk/

Demo Site

I am using this code on an active project, and I'm hosting a test service @ work. The project comes with a demo page, that uses prototype with OpenLayers. You can see a live version here: http://65.101.234.201/rest/ 

rest-demo

The demo is super simple at this point, but we'll be adding more functionality as we have time.

Comments / Bugs / Issues / Ideas?

We have setup a Google Group for the project - pop on over and let us know what you think. http://groups.google.com/group/arcdeveloper-dev

Tuesday, March 04, 2008
Posted on Tuesday, March 04, 2008 7:11:48 PM (Mountain Standard Time, UTC-07:00)  Comments [0] | 
Categories: ArcGIS Server

Just a quick note that anyone working with Server Object Extensions, or needing to tweak the settings of your ArcGIS Server, should take a look at the SOE Xplorer. Vish cooked this little tool up to simplify registering Server Object Extensions, but then went hog wild and added in all sorts of other goodness - basically allowing you to edit any server configuration  attribute.

I used it to pump up the MaxRecordCount for a service - which defaults to 500 in case you don't know.

Get the full skinny on the tool and the download links (both 32-bit and 64-bit versions are available for download) or check out the source code in Assembla http://svn2.assembla.com/svn/SOEXplorer 

 

He has also entered this into the ArcGIS Server Code Challenge, so it's up on ArcScripts as well.

Friday, February 29, 2008
Posted on Friday, February 29, 2008 4:49:16 PM (Mountain Standard Time, UTC-07:00)  Comments [1] | 
Categories: .NET | Ajax | ArcGIS Server | DNN | Dojo

I've been diggin in deep this week - at 4:30pm on Friday this was my desktop...

desktop

(click to view full res -500k)

I've got the ArcDeveloper REST project up, a testing site where I'm furiously mixing the Dojo Framework with MS Ajax, and a DotNetNuke module where I'm bringing it all together...

wme1

This is just the start, but I've got Virtual Earth in a Dojo Split Container, (there will be a Dojo Tree in the lovely light blue area) with a Dojo Toolbar (icons to be changed). Not shown but also working are some Dojo dialog forms. The orange blob is a feature brought into VE via the ArcDeveloper.net REST service. I still have to wire in some point clustering, and the tree view, but given it's been a productive week.

Wednesday, February 06, 2008
Posted on Wednesday, February 06, 2008 4:56:10 PM (Mountain Standard Time, UTC-07:00)  Comments [0] | 
Categories: ArcGIS Server | ESRI

yeta-rest-api Ok, it's a little grainy, but if you look closely you can see the shiny new ArcGIS Server REST API behind all that fur. ;-)

Kudos to ESRI for actually showing some beta software. Typically this sort of stuff is locked down until the final release. Does this mean that people in the beta program will be able to stage up applications for the public to view? Will they be able to talk / blog about it? And most of all - does this mean that we'll see a release before the User Conference?

Sunday, February 03, 2008
Posted on Sunday, February 03, 2008 3:32:04 PM (Mountain Standard Time, UTC-07:00)  Comments [6] | 
Categories: ArcDeveloper | ArcGIS Devt | ArcGIS Server | Community

Starting from the little bit of hacked up code that I cooked up a while back, I've created an extensible, interface based, adaptive caching Virtual Earth Tile Server that can be extended to use any map rendering engine. The initial release has one Tile Provider which uses the ArcGIS Server SOAP API. In the coming weeks I will be adding providers for both WMS and ArcIMS services.

Project Features:

  • HttpHandler (.ashx)that responds to Tile requests from Virtual Earth
  • Supports multiple layers/map services through the same handler
  • ArcGIS Server Tile Provider uses the AGS SOAP API (fast!)
  • Projects the data - can be used with any ArcGIS Server map service
  • Extensible design allows for additional Tile Providers - i.e. ArcIMS, Image Server
  • Extensible design allows for additional storage providers - i.e. Amazon S3, DBMS
  • Easy to use - just edit the web.config file.
  • Includes a demo web page to get you up and running quickly.
  • API Documentation compiled using DocProject is at http://doc.arcdeveloper.net/tileserver (coming soon!)
  • Leverages the Castle Windsor Inversion of Control Container.
  • Exceptions are written into a tile image so it's easy to debug your configuration.

Design

The Virtual Earth Tile Handler uses a Virtual Earth Tile Service Manager to actually do all the work. The Tile Service Manager contains a collection of Tile Services, keyed by a name. This allows a single instance of the Tile Handler to host multiple tile services. The handler determines which Tile Service to use by extracting the service name from the url of the request. Assuming your tile handler lives in a subdirectory called "tileHandler", a request to http://myserver/tileHandler/tahoe/<vetileindex>.ashx will look for the tahoe tile service, and use it's settings to create the map.

Tile Service is simply a named container for a Tile Provider (a class which creates map image tiles) and a Storage Provider (a class which can cache the images on disk)

The logic in the system is quite simple as shown below:

 TileServer-logic

How to Configure Tile Server:

There are 2 main things you need to do to configure this for your site - add the layer to your Virtual Earth map control, setup a TileService in the web.config.

Adding a Tile Layer to the Virtual Earth Map Control:

Using version 6 of the Map control this is very easy (the Default.aspx file in the attached zip file has this code in it)

First Create a bounding box that will limit the area that Virtual Earth will make tile requests. This box is for Colorado.

var bounds = [new VELatLongRectangle(new VELatLong(40,-109),new VELatLong(37,-102))];

Next, create a Tile Source Specification. You will need to specify the full Url to the handler if your Virtual Earth page is not located in the same web application. What's important to note is that the name of the TileService must be in the url - in this case it's "colorado". You will need to change this to the name of the TileService you want this layer to pull from.
var tileSourceSpec = new VETileSourceSpecification("agssoap", "./colorado/%4.ashx");

From here, it's just setting some simple tile source properties...
tileSourceSpec.NumServers = 1;
tileSourceSpec.Bounds = bounds;
tileSourceSpec.MinZoomLevel = 10;
tileSourceSpec.MaxZoomLevel = 18;
tileSourceSpec.Opacity = 0.5;
tileSourceSpec.ZIndex = 100;

Finally you want to add the layer to your map
map.AddTileLayer(tileSourceSpec, true);

 

Setting up a Tile Service in Web.config

A Tile Service is a container for a Tile Provider and a Storage Provider, so we are going to set up these three things, and link them together. The web.config file in the download already has these sections in it - you will just need to change a few things.

Step 1: Configure the Tile Service

Change the service name to match what you put into the VETileSourceSpecification path, as this is the linkage between the Tile Service and the Url that the request is coming from. For now, leave the other elements alone.

<!-- Colorado Tile Service-->

      <component id="ColoradoVETileService"

                service="ArcDeveloper.TileServer.Interfaces.IVirtualEarthTileService, ArcDeveloper.TileServer.Interfaces"

                type="ArcDeveloper.TileServer.Core.VirtualEarthTileService, ArcDeveloper.TileServer.Core" >

        <parameters>

          <!-- Service Name - must be specified in the url parsed by the handler -->

          <servicename>colorado</servicename>

          <!-- The tile provider for this service is AGSColorado, with it's settings defined lower down -->

          <tileProvider>${AGSColorado}</tileProvider>

          <!-- The storage provide for the cache is the default FileStorage Provider, but this could be changed as needed -->

          <storageProvider>${FileStorageProvider}</storageProvider>

        </parameters>

      </component>

The other sections specify the TileProvider and StorageProvider that this Tile Service will use. If you want to use different or differently configurated providers, just copy/paste the sections and rename them. The configuration syntax here is based on the Castle Windsor project, and you can get more information about how this works by reading Simone Busoli's articles here: Part 1, Part 2, Part 3, and Part 4.

Step 2: Specify the Map Service URL

Locate the following section in the web.config

<component id="AGSColorado"

                service="ArcDeveloper.TileServer.Interfaces.ITileProvider, ArcDeveloper.TileServer.Interfaces"

                type="ArcDeveloper.TileServer.ArcGIS.ArcSoapTileProvider, ArcDeveloper.TileServer.ArcGIS">

        <parameters>

          <!-- Make sure this Url is accessible from this web server or you'll get an exception -->

          <webServiceUrl>http://YOURSERVER/arcgis/services/YOURSERVICENAME/MapServer</webServiceUrl>

        </parameters>

      </component

 

Step 3: Change the StorageProvider location

The Storage Provider is used to cache the tiles as they are created. Thus the location you specify must have write privileges granted to the identity that the ASP.NET application runs as (ASPNET on Windows XP and NETWORK SERVICE on Windows Server 2003)

<component id="FileStorageProvider"

                service="ArcDeveloper.TileServer.Interfaces.IStorageProvider, ArcDeveloper.TileServer.Interfaces"

                type="ArcDeveloper.TileServer.Core.FileStorageProvider, ArcDeveloper.TileServer.Core">

        <parameters>

          <!-- Make sure this path exists, or you'll get an Exception instantiating the FileStorageProvider -->

          <cacheFilePath>c:\tiles\</cacheFilePath>

        </parameters>

      </component>

That's it.

Known Issues:

Occasionally empty tiles are returned from ArcGIS Server. I am still looking into why this occurs, but since the SOAP API actually returns an image, there is no way for the code to know that it's a blank image, and thus it is still added to the cache.

Do not use labeling in the map service. This is because the images are requested at their native tile size of 256 x 256. Thus the labels will be cramped into the tile, and you will have a lot of repeated labels across your map. In the future we may build out a tool that will pre-cook a tile cache, and when doing that you can request larger images which will not have this issue. For now, just turn your labels off.

Feedback / Bugs / Issues

Please use the ArcDeveloper.net forums to ask questions or raise issues. This way everyone will benefit, and it's less one on one emailing.

Source Code:

All the source code for this project is available at the ArcDeveloper project hosted at Assembla.com. You can get the source code from http://svn2.assembla.com/svn/arcdeveloper/tileserver/trunk

If you would like to contribute to the project, use the contact link at the top of this page and I will send you an invitation to join the project team.

Download

Download ArcDeveloper.net Tile Server v0.1 (from ArcScripts). This zip file contains the source code current as of February 3,2008.

Vote!

I also entered this code in the ESRI Code Challenge . If you are an EDN subscriber or are going to the 2008 Developer Summit, and find this code useful, I'd appreciate your vote!

Thursday, January 31, 2008
Posted on Thursday, January 31, 2008 7:59:08 AM (Mountain Standard Time, UTC-07:00)  Comments [5] | 
Categories: ArcGIS Devt | ArcGIS Server

Looking at the comments in James Fee's post summarizing Doron Yaakobi's feelings about the ArcGIS Server ADF, this seems to have struck a chord with many other people building solutions with ArcGIS Server.

The question I have is - will we do anything about it?

By that I don't mean writing more angry blog postings, canceling our EDN subscriptions or other symbolic, by really meaningless actions. I mean lets write the APIs we want to have. This is akin to the Alt.Net movement (if you can call it a "movement"). We're not saying we're ditching ESRI for GeoServer and uDIG, but we are going to play the game by our rules. We will use ArcGIS Server for what it's good for, and skip the rest. If we don't like the ADF, lets cook up something better.

To show that this is not all talk, shortly I will be releasing a much more solid version of the ArcGIS Server Adaptive Virtual Earth Tile Cache I posted about last week. I'll post more about it, but suffice to say it's very flexible and extensible. It will be put up in a public Subversion repository on Assembla for all to have and play with.

They key thing is working together on this stuff - back when  ArcIMS 9.0 shipped and ESRI did a half baked job with the ActiveX/COM/not-dot-net connector, dozens of people went out and wrote their own .NET ArcIMS connectors. How many 1000's of hours were wasted with people re-doing that same stuff? Wouldn't hindsight suggest that we not all roll our own "Not-ADF" solutions?

So - if we want a REST API, lets built it. There's not really a whole lot to it if it's designed correctly. Heck if nothing else we can use what has been done on FeatureServer.org as a road map. Suppose we cook up a server object extension (SOE) that can do all the low-level stuff in raw ArcObjects. Then we write a httphandler that can take the requests, parse the Urls and make calls back to the SOE. If we but the JSON/GEORSS/KML/whatever conversion in the SOE, we're very light-weight on the DCOM stuff, and can get away with little to no ADF. And if we run this all directly on the ArcGIS Server box, we're good in terms of licensing.

Once we have that out of the way, then integrating with OpenLayers or Virtual Earth just got a lot easier. Sure it's not full featured editing, but who cares? We can leave that to the ADF.

So how about it? What to see something change? Want a REST API before September? Want to be able to fix it if something does not work?

If so, drop me a message (contact link above) and for those attending the Developer Summit, lets plan on having a meeting (over beers) to get things rolling.

Tuesday, January 29, 2008
Posted on Tuesday, January 29, 2008 6:37:22 PM (Mountain Standard Time, UTC-07:00)  Comments [4] | 
Categories: ArcGIS Server

We just ran into a issue with the ADF - namely that the ADF Geometry model does not support curves (aka "True Arcs"). Instead, it just puked up an exception stating that "curves are not supported". After doing some searching we found this thread in the forums, and this little gem stood out...

I just want to add one more thing to what Kevin said: the SP3 will handle the exception gracefully when identifying on a true arc (i.e. a message stating "Curves are not supported") instead of throwing an unhandled exception. The goal is to get this functionality fully working at 9.3.

I take this to mean that we should be grateful we ran into this now and not pre-SP3 when we would have gotten a more generic exception. Nice.

Obviously we are not the first people to run across this. There is a write up about the issue in the Knowledge Base: Bug:  Web Application Developer Framework (ADF) does not support non-linear geometry (curves)

As I noted above, it seems that the issue is that the Web ADF geometry classes can't handle curves. Fine - I can understand that it may be difficult to work with curves. Maybe implementing real curves will need to wait for v9.3 or v10 or whatever, but how about this - if a curve is sent into the Web ADF geometry how about you densify it for me? It may not be as "accurate", but "close" is a whole lot better than an exception - even if it has a nice message.

The thing is that this is the suggested solution. If you have curves, don't use the IQueryFunctionality. Instead use the IMapServer.Identify and IMapServer.Query* methods. Loop over the returned features, check if the geometries are curves, if so then densify the geometries before stuffing them into a graphics layer.

Of course it's nontrivial if you're not into the ADF in a big way, and you are hosed if you are not using a local ArcGIS Server data source, but hey what do you expect.

ESRI has put some code up on ArcScripts that shows how to deal with this for the Identify tool, but you are on your own for anything else. Fun Fun Fun.

Monday, January 21, 2008
Posted on Monday, January 21, 2008 6:36:06 PM (Mountain Standard Time, UTC-07:00)  Comments [3] | 
Categories: ArcGIS Server | ASP.NET

I keep getting more requests for the code, so I'm just going to put it up here. I sent the code out to a bunch of people over the weekend, and I thought I should put the same disclosure here...

As I said in the posting - this is really hacked up, pure "can it work" research code – it’s based on Rob Blackwell’s VE WMS Tile Server sample, with some “create a map” stuff from the AGS SOAP documentation added, and a few lines of projection stuff smashed in.

Known Issues:

Currently the projection is not exactly right, the caching stuff is just plain ugly, and requires that ASPNET (or whatever identity you run the handler as) have write access to the hard coded path. All the tiles are stored in the same folder, which clearly won’t scale. The Handler matches *.png, which means you need to map that extension to the aspnet_isapi.dll for the web application you drop this into. The SOAP end point is poorly named. And there are no comments of any use.

So – I think that about covers it – you have been warned! That said have fun with it, and let me know how it goes.

Download

Posted on Monday, January 21, 2008 5:25:54 AM (Mountain Standard Time, UTC-07:00)  Comments [0] | 
Categories: .NET | ArcGIS Server

So I've been playing more with the AGS SOAP API, and have found a few things...

Generating Proxy Classes

While you could pass Xml back and forth, most people will opt to use some kind of proxy classes. Visual Studio will generate these for you when you add a web reference, and this is a good way to get up and running quickly.

There is one downside however - it seems that the WSDL returned by ArcGIS Server for any particular service injects the service name into itself. So if you were adding a web reference to http://myserver/arcgis/services/mytestService/MapServer?wsdl then you'd get a class that was named mytestService_mapserver. While you can change the actual service Url to point to another Map Service, you'll still have  "mytestService_mapserver" littered all over your code. To me this is ugly and a code smell if you are trying to write something which is not bound to a single service. I guess the idea is that it helps a developer figure out which web service they are talking to, and I suppose that if you needed to write an app that used multiple connections to multiple mapservices, then it may *help*, but I doubt it. Any insight on why this was done would be great.

Creating Manual Proxies: wsdl.exe

In addition to simply using Add Web Reference, you can take a more manual route and use wsdl.exe to create the proxies. The wsdl.exe tool that ships with Visual Studio 2005 and it offers a little more control. The syntax is pretty simple - here's what I used:

wsdl.exe /nologo /l:CS /n:ArcDeveloper.AGS.SOAP.Proxies /o:c:\wsdl\proxies /urlkey:MapServerUrl MapServer.wsdl

So this creates a C# proxy class for the MapServer.wsdl file, in c:\wsdl\proxies, in the ArcDeveloper.AGS.SOAP.Proxies namespace.

However, if you run this against the WSDL you download from the MapServer WSDL page, you'll get an error:

Warning: This web reference does not conform to WS-I Basic Profile v1.1.
R2028, R2029: A DESCRIPTION using the WSDL namespace and the WSDL SOAP binding namespace MUST be valid according to the XML Schemas found at http://schemas.xmlsoap.org/wsdl/2003-02-11.xsd and http://schemas.xmlsoap.org/wsdl/soap/2003-02-11.xsd.
  -  Warning: The 'name' attribute is invalid - The value '#NAME#' is invalid according to its datatype 'http://www.w3.org/2001/XMLSchema:NCName' - The '#' character, hexadecimal value 0x23, cannot be included in a name. Line 2465, position  11.
  -  Warning: The identity constraint 'http://schemas.xmlsoap.org/wsdl/:service'  validation has failed. Either a key is missing or the existing key has an empty  node. Line 2465, position 3.

For more details on the WS-I Basic Profile v1.1, see the specification at http://www.ws-i.org/Profiles/BasicProfile-1.1.html.

Writing file 'c:\wsdl\NAME.cs'.

The errors are not critical, since a file still gets written, but now the proxy class is named "NAME", which is worse than the <mapservicename>_mapservice that we got using Add Web Reference. If we look at the wsdl file, the culprit is in the last few lines of the file - we see this:

<service name="#NAME#">
  <port name="MapServerPort" binding="e:MapServerBinding">
   <soap:address location="#URL#"/>
  </port>
 </service>

I'm betting that the #NAME# and #URL# are tokens which get replaced when you get the WSDL from a live service (i.e.  http://myserver/arcgis/services/mytestService/MapServer?wsdl ). So - the fix is pretty simple - just edit your copy of the wsdl file, and assign the name to whatever you want the proxy class to be called. In my case, I set the name to "MapServiceProxy" and the Url to http://localhost/use/app.settings to remind myself to always configure the url in the app settings (the "key" is part of the wsdl command line --> /urlkey)

Note: I think you can use Add Web Reference and just point to your own local "fixed" copy of the WSDL file, and this will produce nicely named proxy classes - I did not test this since I'd already baked my own using wsdl.exe ;-)

Anyhow - I've created a set of proxy classes (VB.NET and C#), and a batch file for cooking them yourself. Grab it here: ags-proxy.zip  [UPDATE - link now works]

Wednesday, January 16, 2008
Posted on Wednesday, January 16, 2008 10:14:21 PM (Mountain Standard Time, UTC-07:00)  Comments [2] | 
Categories: .NET | ArcGIS Server | Virtual Earth

Just doing some research into pulling ArcGIS Server map services into Virtual Earth, and thought I'd share a some of what I'd found.

ArcGIS Server WMS Layer in Virtual Earth

Using the "Accessing WMS from Virtual Earth" article and code as a starting point, I was able to add an ArcGIS Server WMS layer pretty quickly. The only problem I had was divining the correct values for the WMS "LAYER" element - apparently the default AGS WMS service expects the LAYERS to be specified as numbers - 1, 2, 3 etc. Maybe this is standard, but I'm not a WMS guru by any means. Anyhow, the performance was good, but I will say that I've got some beefy servers - the AGS box is a Dual Quad Core box, with 8 GB RAM. The SDE box has one Quad Core. Neither have any appreciable load, so you're mileage may vary.

ArcGIS Server SOAP API

WMS is nice, but I'm going to need to do some more crafty stuff - adding things into graphics layers, drawing selections etc. This would normally mean using the ADF, but I want to be able to spread this across a web farm with out spending a zillion bucks, so the  ArcGIS Server SOAP API seemed to be the ticket.

The API itself is pretty easy to use, and the example on the AGS SOAP API page was about all I needed to get started. Where things became problematic was with the projections. Morten has a great rundown of the VE/Google Earth projection stuff which was very helpful.

The WMS requests specify a SRS which tells ArcGIS Server how to project the map so that it will match up with the Virtual Earth tiles. Going at the SOAP API, you need to do this yourself.

Armed with the Well Known Text definition of the Geographic Lat/Long system used in the VE front end, I was able to set the SpatialReference for the zoom envelope as well as the map.

wsMapService.SpatialReference sr = new wsMapService.GeographicCoordinateSystem();
sr.WKT = "GEOGCS[see Morten's post...]";
env.SpatialReference = sr;

I also found that I needed to tell the map to be projected into the same space...

mapdesc.SpatialReference = sr;

Update: The tiles are still a little off. Need to do more messing with projections...

Overall, I like the SOAP API - and until the REST API shows up at 9.3 whenever that ships, this is a pretty cool option.

Missing Tiles

Ok - with that out of the way I started to get some tiles showing up, but I also got some run-time exceptions. Running in debug mode, I found that sometimes the SOAP API would return a Url to a map image which did not exist. I added in some code which would catch this, and shoot back a "Missing Tile" image.

 

At this point the behavior is seems random - different tiles at different scales will draw. When I refreshed the page, different tiles would be drawn. My initial thought was that I was overwhelming the Server Object Containers with requests.  I had been running with just two SOC's, so I cranked it up to 10. Same results. And nothing in the logs. Next I thought that maybe it's an issue of the Url being sent back before the image is written out - so I added a delay before fetching a tile. Still the same thing. Any insights on this would be great.

Solution: esriImageReturnMimeData

Since this was bugging me, I went in an changed the return type to esriImageReturnMimeData - which would return me a byte array containing the image. A little bit more hacking and it's working just fine. Now, every tile draws every time. So - the issue seems to be AGS not writing out the files, but still sending back a url.

Adaptive Caching

Since I'm already throwing images around in the httpHandler, it was nothing to write the final images out to files, and put a quick check in up front to see if it exists before going out to AGS to create it.  Works great, but I need to add a little more logic so I don't end up with one folder containing 50,000 files.

The Code

This is horrid evil messy hacked up "research" code that likely has substantial performance issues, high-cholesterol, security holes, and lice but if you really want it, shoot me an email (Contact link above) and I'll zip it.

Posted on Wednesday, January 16, 2008 6:28:27 PM (Mountain Standard Time, UTC-07:00)  Comments [0] | 
Categories: ArcGIS Server | Dev Summit

Just got the notice in via email - wow.

code-challenge

I may have to polish up that adaptive cache code!

Thursday, January 10, 2008
Posted on Thursday, January 10, 2008 10:49:38 PM (Mountain Standard Time, UTC-07:00)  Comments [0] | 
Categories: ArcGIS Server | ASP.NET

When writing a web application, it can be useful to store information between client requests, and this is exactly what "session state" is used for. By default, ASP.NET stores this information in the IIS process, in a mode called "InProc".

However, when you need to run a web site across a number of web servers this model can become problematic. Right now we are working with a client who needs to run the application we are building on two web servers (for redundancy). In front of these web servers is a hardware load-balancer. This particular environment does not support client affinity (i.e. your first request may go to server X, a subsequent request may go to server Y).

In this environment, we can not store the session state on the web server itself. Conveniently ASP.NET supports  two other modes - StateServer and SQLServer. This is configured in the web.config file using the <sessionstate> element.

With the StateServer option, the session state from all the servers in the farm are stored in the RAM of a single server that's running the ASP.NET State Service.

Since we did not have the option of adding another server into our client's architecture (and it would be a single point of failure), we had to go with the SQLServer option.

Setting up SQL Server for ASP.NET Session Storage

There are two options for how the session state is stored - either in tempdb or in a persisted mode in a database. The persisted mode allows sessions to persist through a SQL Server re-start. I opted for the tempdb option.

Despite being a built in part of ASP.NET, it was not obvious how to actually setup SQL Server for this - the documents indicate that you should just run aspnet_reqsql.exe without command line options, and use the wizard. However, this does not install the State stuff. You need to run aspnet_regsql.exe with the -ssadd switch.

This is what I used:

aspnet_regsql.exe -S <server> -ssadd -sstype t

(fyi aspnet_regsql.exe is in C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727.)

This will create an ASPNETDB database where the stored procs are stored, and the actual state will be stored in tempdb.

ADF Support

I did a lot of Googling and searching on EDN, and the ArcGIS Server Forums, and could not find anything that authoritatively said that the ADF would play nice with ASP.NET session state models other than "InProc". The quick answer is that it does work. We are able to run all the basic out of the box stuff without issue.

Friday, November 02, 2007
Posted on Friday, November 02, 2007 3:15:48 PM (Mountain Daylight Time, UTC-06:00)  Comments [0] | 
Categories: ArcGIS Server | ArcSDE | SQL Server
Our EDN licenses expired on Oct 30th and while I knew this was coming, I did not get to installing our new 2008 licenses until they had timed out on our ArcSDE server(s).

While you can use the ArcSDE Post Install tool to authorize ArcSDE using your new keys, if you have more than a few databases, it's much faster and easier to apply the license manually. (I have >10)

The license information (at least for SQL Server) is stored in the SDE_Server_Config table. Just copy arcsdeserver line of your ECP file (looks something like this: arcsdeserver,92,ecp123457890,30-oct-2008,S0M3H45HTH1NG) into the CHAR_PROP_VALUE column for the AUTH_KEY row.

Alternatively you can use this sql:

UPDATE    SDE_server_config
SET char_prop_value = 'arcsdeserver,92,ecp123457890,30-oct-2008,S0M3H45HTH1NG'
WHERE     (prop_name = 'Auth_Key')

If you have 100's of databases, get crafty with some T-SQL to iterate over all the databases and run the query...


Friday, October 12, 2007
Posted on Friday, October 12, 2007 5:29:13 PM (Mountain Daylight Time, UTC-06:00)  Comments [3] | 
Categories: ArcGIS Server | ArcMap

I'm trying to get some maps published in ArcGIS Server, and I've run into some weird problems. When I try to publish a map, I get an error stating that the Server Object Creation failed, and that the map file can not be opened.

My first thought was security, and there were some issues there, but that did not solve everything - I still got the errors. So, as I posted the other day, I started up ArcMap as the ArcGISSOC user. The thing is that even though I can browse to the map as that user, it fails to load the map under most conditions.

I can load the mxd file into ArcMap if I first preview it in ArcCatalog (also running as ArcGISSOC). If I just try to open the mxd with ArcMap, the app just disappears. So now I'm thinking it's something with the file being corrupted (works fine as a normal user, but who knows). So I try to create a fresh new map - I fire up ArcMap as ArcGISSOC, connect to a layer, and save the file. When the save finishes, I get this helpful message

best-error-ever

Apparently no error occurred. Thanks ArcMap, but I'm still somewhat suspicious about that.

Turns out that the error is related to security. I'm working on a client's system, that's really locked down, and instead of trying a 1000 little things to see if it was security related, I took out the big hammer - I added ArcGISSOC to the local admin group. Shazam - we got maps! Now that I know it's permission based, I need to dig out the Admin guide again and review exactly what's needed

Wednesday, October 10, 2007
Posted on Wednesday, October 10, 2007 11:16:51 PM (Mountain Daylight Time, UTC-06:00)  Comments [0] | 
Categories: ArcGIS Server

Today I had to try and figure out why I could not publish a map file to ArcGIS Server. The message I got was

"Server Object instance creation failed on machine foomachine. Document D:\arcgisserver\maps\mymap.mxd can not be opened."

When I checked the permissions on the file all looked good - the ArcGISSOC user had access to the file. So where's the beef?

Instead of messing around in the security settings for hours on end, I simply fired up ArcMap as the ArcGISSOC user. It's easy to do - just drop this into a batch file, and you're off and running...

runas /user:youdomain\arcgissoc /profile /savecred "C:\Program Files\ArcGIS\Bin\ArcMap.exe"

Then, try to browse to the file in question and open it. In my case, the problem was ArcGISSOC could not access the D: drive, which seemed to override the explicit permission I granted on that file.

Monday, June 18, 2007
Posted on Monday, June 18, 2007 4:06:18 PM (Mountain Daylight Time, UTC-06:00)  Comments [0] | 
Categories: ArcGIS Server | ArcSDE | ESRI | Mobile ADF
Some quick items from the plenary that were interesting...

ArcGIS Server
More mentions of supporting ArcSDE on PostGIS. Mild applause from the crowd, but it seems like this is still on track to ship at 9.3
Support for ArcSDE on Oracle Express
Image Server will be integrated into Server. Not clear if it will be an extension or part of the core.
On the Mobile ADF side of things, there will be an out of the box Mobile app that ships with 9.3 - so it's no longer just a developer toolkit - I think it's now called ArcGIS Mobile...
Javascript API and REST services for the mashup crowd.

ArcGIS Explorer
Really good demo of ArcGIS Explorer and ArcWeb Services. The data was really really fast - not clear if this was running "live" or from local servers, but it was screaming fast (maybe it was all pre-cached?)
Also notable was the ESRI Content Sharing service - apparently they will host your data for you if you agree to share it. And I may have the name wrong, but I think it's called ArcDataServices which gives you the ability to request (buy? was not clear) data, which is re-projected/extracted etc for you. What was cool is that it's powered by i-cubed's DataDoors on the back end. Kudos to the whole team at i-cubed!

Support Issues
The spin on the current huge backlog of support issues was that it's due to the massive and rapid adoption of 9.2, which has overwhelmed support. That may be, but it's also pretty buggy. Anyhow - the plan to address these issues was laid out. Over the last year they have been re-structuring their whole lifecycle to include "support" as an active element. Support personnel have been physically co-located with the developers. Additionally they said they will open up the entire issue database. Total transparency. What they see, we will see. This got a lot of applause, but we will see...

There were also some cool demos, and the requisite Jack Dangermond "grand vision" segments, but overall a very good opening session. I'm actually excited to talk to the ESRI people about some of the new stuff and some of the 9.2 stuff that I had not known about.

Sunday, April 29, 2007
Posted on Sunday, April 29, 2007 8:48:26 PM (Mountain Daylight Time, UTC-06:00)  Comments [1] | 
Categories: ArcGIS Devt | ArcGIS Server | ArcMap | Geodatabase

I recently recieved an email from a reader asking about role based security & ArcGIS Server. Specifically they were interested in restricting access to columns, based on the user's credentials.

I'd been doing some thinking on similar issues related to row based edit permissions, and I thought I'd share my ideas on possible solutions to these problems.

Fine Grained Access Permissions

Essentially, you can't use the DBMS security capabilities to restrict user access to columns or rows because ArcSDE does not support security at this fine grained level (FeatureClass, ObjectClass or FeatureDataset only). There may be some hacks, but as far as I know nothing like this is supported directly.

Next, and possibly more important, regardless of how you actually create the connections (i.e. store the connection properties in the MXD file, or use pass-through authentication and grant the SOC process permission to connect into the database) all interactions with features are done as the SOC Service user. The identity of the front end user  is not passed back to the SOC. Thus, regardless of who is running the application, the SOC identity is always the same. The diagram below shows how far you can pass back the end user's identity (assuming you setup impersonation in your web app).


 

The SOC always runs as the SOC Service user - so any use of the end user's identity to control any sort of access is not going to work.

Possible Solutions - Restricting Access to Columns

There are a variety of ways to tackle this, but none are particularly flexible - you will need to define the roles up front, and then setup the application to respect those roles explicity. Adding additional roles will require code changes. Four options come to mind

1) Let the SOC return all the data and apply the permissions in the client application. This is not a bad solution if you control the web app. It basically means re-writing a lot of the out of the box tools so that they will respect this additional business logic.

2) Setup the spatial data so that it has the minimal set of field that all users can see (including ObjectId). When doing an inquiry, the web app would get the base set of fields from the featureclass, and then use the ObjectId  to run a second query against a multi-version view to retreive the other "secured"  attributes. There would be a multi-version view for each role in the application, and the columns would be restricted as needed. The application logic would need to handle the mapping of the users role to the correct view.

3) Use Spatial Views. Similar to #2, but the SOC would load layers based on the user's role, and the layers would actually be Spatial Views with only the columns that are visible for the particular role of the user. I suspect this would be slower than the previous two options, as the SOC would need to load layers all the time, but if the application is using a non-pooled SOC anyhow, it may not bemuch more of a performance hit - just start with a minimal map (just the layers which do not have any restrictions on them). The upside is that you can likely use most of the out of the box tools since the column restrictions are being applied by the view, at the ArcSDE level. Another upside of this model is that the restricted data is never available - either in the SOC, or in the web application tier.

4) Similar to #3, you could separate out the restricted fields into separate tables, and apply joins on the fly. Same benefits as #3, but you'd still want to work with non-pooled SOCs. Not sure how well this would play with the out of the box tools, but it may be ok.

I'm sure there are other options, but I suspect they are all variations on this same theme. If the application was super simple (like 1 ID tool), you could just limit the fields returned by setting the IQueryFilter.SubFields property.


Row Based Edit Permissions

For our applications, there are typically business rules which restrict the area in which a user can edit features, and related attribute tables. In the past we have implemented this logic in feature class extensions. In the extension, we simply get the current principal (the user's Windows Login identity) lookup their access permissions from our repository and allow/block the edit. We have logic in the extension which allows for editing of shared boundaries (which I believe is one of the issues with implementing this sort of logic at the DBMS level)

This works just fine for Desktop and ArcEngine applications, but is more problematic for web applications. As I noted at the start of this post, all connections into ArcSDE from ArcGIS Server run as the SOC user. Thus we have a problem - if we apply a class extensions to the geodatabase, it's logic always fires - regardless of the client appliction. But, if all the edits from everyone using the web app effectivly run in the same context, (agssoc or whatever you call it on your system), then this logic is both moot, and a waste of processor time. But we still want to use this logic for the desktop apps, so we had a bit of a problem.

Our solution is to create feature class extensions which are themselves extensible. Basically we use  the Provider Model to add in various security providers at run-time. When the class extension spins up, it uses the ConfigurationManager to check the configuration file (ArcMap.exe.config for desktop, web.config for web apps) for "SecurityProviders". In the desktop apps, there will be a provider configured, the class will be instantiated, and the logic applied to all the edits. In the web app, we do not configure a security provider, thus there is no logic applied. 

This still begs the question of "how does the row based edit permission get applied in the web app?". At this time, there is no (easy) way to pass the user identity back into the SOC, let alone get the SOC to impersonate a user such that the class extension would be running in the context of the front end user. So, for now, the answer is - the web application must handle this logic itself. While this is not idea, at least the class extensions can be used in the desktop apps - which is where the vast majority of the edits will occur anyhow.

Thursday, April 05, 2007
Posted on Thursday, April 05, 2007 2:57:01 PM (Mountain Daylight Time, UTC-06:00)  Comments [7] | 
Categories: ArcGIS Server

So I was just playing around with a simple ArcGIS Server app, seeing what I could do to reduce the amount of data sent to the client and at the same time speed things up a little. I took a look at the PreferredTileHeight and Width properties of the Map control. By default, these values are set to zero, which means that each tile will be the size of the entire map control. But, if you have a pretty large map, a small pan can result in a lot of data being s