Over the years I've seen a few "developer" surveys put out by various GIS organizations, and they have all been pretty bland. After talking to a few people at the Developer Summit about this, I thought I'd see if we could do a little better - thus the launch of the 2008 Geospatial Developer Survey.

Why are you doing this?

The main reason is that I'm interested in knowing more about how other people doing GIS/Geospatial development work. How many people view themselves as software engineers vs. occasional coders who hack some stuff together as needed. I want to know if people are exploring and adopting the tools, patterns and practices that are common place in main stream software development. And if not, why.

And beyond my curiosity, I'm also looking for some insight as to what topics people want to know more about. Although I have some ideas, lets hear from you. I get a fair number of requests to write more "fundamentals" type posts, and at the same time get requests for more on patterns, Inversion of Control and Unit testing. Since I don't write this blog for a living, I need to focus my efforts, so I hope that this survey will help quantify which way I should lean in future postings.

Who should take the survey?

Anyone who writes any code that is geospatial in nature. From the Google Maps Mashup dudes, to those running some geoprocessing tasks with python. If you write code that has anything to do with a map, you're the perfect person to take the survey. Be sure to pass this url around to everyone you work with who is a geospatial developer - the more data points we get, the better. Who knows - we may even be able to influence the content at some conferences...

How?

It's simple - just follow this link to SurveyMonkey, and fill out the questions. Yes there are 30 questions, but they should be pretty quick - I'd guess no more than 10 minutes.

What Next?

I'll be leaving they survey open until May 9th 2008. Then I'll crunch the numbers and do a post or two about the results.

Link (for copy pasting to all your developer buddies):

http://www.surveymonkey.com/s.aspx?sm=cugzupf5rAG_2bIu5WJ_2f7rxQ_3d_3d

UPDATE 4/8 @ Noon: After recieveing some feedback on the questions, I have added more "Not Applicable" and "Other" options.

Starting from the little bit of hacked up code that I cooked up a while back, I've created an extensible, interface based, adaptive caching Virtual Earth Tile Server that can be extended to use any map rendering engine. The initial release has one Tile Provider which uses the ArcGIS Server SOAP API. In the coming weeks I will be adding providers for both WMS and ArcIMS services.

Project Features:

• HttpHandler (.ashx)that responds to Tile requests from Virtual Earth
• Supports multiple layers/map services through the same handler
• ArcGIS Server Tile Provider uses the AGS SOAP API (fast!)
• Projects the data - can be used with any ArcGIS Server map service
• Extensible design allows for additional Tile Providers - i.e. ArcIMS, Image Server
• Extensible design allows for additional storage providers - i.e. Amazon S3, DBMS
• Easy to use - just edit the web.config file.
• Includes a demo web page to get you up and running quickly.
• API Documentation compiled using DocProject is at http://doc.arcdeveloper.net/tileserver (coming soon!)
• Leverages the Castle Windsor Inversion of Control Container.
• Exceptions are written into a tile image so it's easy to debug your configuration.

Design

The Virtual Earth Tile Handler uses a Virtual Earth Tile Service Manager to actually do all the work. The Tile Service Manager contains a collection of Tile Services, keyed by a name. This allows a single instance of the Tile Handler to host multiple tile services. The handler determines which Tile Service to use by extracting the service name from the url of the request. Assuming your tile handler lives in a subdirectory called "tileHandler", a request to http://myserver/tileHandler/tahoe/<vetileindex>.ashx will look for the tahoe tile service, and use it's settings to create the map.

Tile Service is simply a named container for a Tile Provider (a class which creates map image tiles) and a Storage Provider (a class which can cache the images on disk)

The logic in the system is quite simple as shown below:

 

How to Configure Tile Server:

There are 2 main things you need to do to configure this for your site - add the layer to your Virtual Earth map control, setup a TileService in the web.config.

Adding a Tile Layer to the Virtual Earth Map Control:

Using version 6 of the Map control this is very easy (the Default.aspx file in the attached zip file has this code in it)

First Create a bounding box that will limit the area that Virtual Earth will make tile requests. This box is for Colorado.

var bounds = [new VELatLongRectangle(new VELatLong(40,-109),new VELatLong(37,-102))];

Next, create a Tile Source Specification. You will need to specify the full Url to the handler if your Virtual Earth page is not located in the same web application. What's important to note is that the name of the TileService must be in the url - in this case it's "colorado". You will need to change this to the name of the TileService you want this layer to pull from.
var tileSourceSpec = new VETileSourceSpecification("agssoap", "./colorado/%4.ashx");

From here, it's just setting some simple tile source properties...
tileSourceSpec.NumServers = 1;
tileSourceSpec.Bounds = bounds;
tileSourceSpec.MinZoomLevel = 10;
tileSourceSpec.MaxZoomLevel = 18;
tileSourceSpec.Opacity = 0.5;
tileSourceSpec.ZIndex = 100;

Finally you want to add the layer to your map
map.AddTileLayer(tileSourceSpec, true);

 

Setting up a Tile Service in Web.config

A Tile Service is a container for a Tile Provider and a Storage Provider, so we are going to set up these three things, and link them together. The web.config file in the download already has these sections in it - you will just need to change a few things.

Step 1: Configure the Tile Service

Change the service name to match what you put into the VETileSourceSpecification path, as this is the linkage between the Tile Service and the Url that the request is coming from. For now, leave the other elements alone.

The other sections specify the TileProvider and StorageProvider that this Tile Service will use. If you want to use different or differently configurated providers, just copy/paste the sections and rename them. The configuration syntax here is based on the Castle Windsor project, and you can get more information about how this works by reading Simone Busoli's articles here: Part 1, Part 2, Part 3, and Part 4.

Step 2: Specify the Map Service URL

Locate the following section in the web.config

 

Step 3: Change the StorageProvider location

The Storage Provider is used to cache the tiles as they are created. Thus the location you specify must have write privileges granted to the identity that the ASP.NET application runs as (ASPNET on Windows XP and NETWORK SERVICE on Windows Server 2003)

That's it.

Known Issues:

Occasionally empty tiles are returned from ArcGIS Server. I am still looking into why this occurs, but since the SOAP API actually returns an image, there is no way for the code to know that it's a blank image, and thus it is still added to the cache.

Do not use labeling in the map service. This is because the images are requested at their native tile size of 256 x 256. Thus the labels will be cramped into the tile, and you will have a lot of repeated labels across your map. In the future we may build out a tool that will pre-cook a tile cache, and when doing that you can request larger images which will not have this issue. For now, just turn your labels off.

Feedback / Bugs / Issues

Please use the ArcDeveloper.net forums to ask questions or raise issues. This way everyone will benefit, and it's less one on one emailing.

Source Code:

All the source code for this project is available at the ArcDeveloper project hosted at Assembla.com. You can get the source code from http://svn2.assembla.com/svn/arcdeveloper/tileserver/trunk

If you would like to contribute to the project, use the contact link at the top of this page and I will send you an invitation to join the project team.

Download

Download ArcDeveloper.net Tile Server v0.1 (from ArcScripts). This zip file contains the source code current as of February 3,2008.

Vote!

I also entered this code in the ESRI Code Challenge . If you are an EDN subscriber or are going to the 2008 Developer Summit, and find this code useful, I'd appreciate your vote!

Looking at the comments in James Fee's post summarizing Doron Yaakobi's feelings about the ArcGIS Server ADF, this seems to have struck a chord with many other people building solutions with ArcGIS Server.

The question I have is - will we do anything about it?

By that I don't mean writing more angry blog postings, canceling our EDN subscriptions or other symbolic, by really meaningless actions. I mean lets write the APIs we want to have. This is akin to the Alt.Net movement (if you can call it a "movement"). We're not saying we're ditching ESRI for GeoServer and uDIG, but we are going to play the game by our rules. We will use ArcGIS Server for what it's good for, and skip the rest. If we don't like the ADF, lets cook up something better.

To show that this is not all talk, shortly I will be releasing a much more solid version of the ArcGIS Server Adaptive Virtual Earth Tile Cache I posted about last week. I'll post more about it, but suffice to say it's very flexible and extensible. It will be put up in a public Subversion repository on Assembla for all to have and play with.

They key thing is working together on this stuff - back when  ArcIMS 9.0 shipped and ESRI did a half baked job with the ActiveX/COM/not-dot-net connector, dozens of people went out and wrote their own .NET ArcIMS connectors. How many 1000's of hours were wasted with people re-doing that same stuff? Wouldn't hindsight suggest that we not all roll our own "Not-ADF" solutions?

So - if we want a REST API, lets built it. There's not really a whole lot to it if it's designed correctly. Heck if nothing else we can use what has been done on FeatureServer.org as a road map. Suppose we cook up a server object extension (SOE) that can do all the low-level stuff in raw ArcObjects. Then we write a httphandler that can take the requests, parse the Urls and make calls back to the SOE. If we but the JSON/GEORSS/KML/whatever conversion in the SOE, we're very light-weight on the DCOM stuff, and can get away with little to no ADF. And if we run this all directly on the ArcGIS Server box, we're good in terms of licensing.

Once we have that out of the way, then integrating with OpenLayers or Virtual Earth just got a lot easier. Sure it's not full featured editing, but who cares? We can leave that to the ADF.

So how about it? What to see something change? Want a REST API before September? Want to be able to fix it if something does not work?

If so, drop me a message (contact link above) and for those attending the Developer Summit, lets plan on having a meeting (over beers) to get things rolling.

I had originally setup a SourceForge project for ArcDeveloper.net, but the whole SourceForge experience is clunky and web 1.0. So when I found out about Assembla, I immediately setup a project.

Assembla is a really cool, free Subversion + TRAC + wiki + Scrum development tool / service. It's really amazing. It's totally free for projects up to 200Mb in size (which is a MOUNTAIN of source code).

ArcDeveloper Projects

A repository and some tools is nice, but a bigger question is what projects will be there. Well, my goal for ArcDeveloper is to provide a set of code resources that will make unit testing on ArcGIS projects easier. Since software quality is something that all developers must address, and since it's the same regardless of your vertical market, I think that few organizations would object to using and contributing to this effort.

Getting Access

The repository is open for all users, just point your Subversion client to http://svn2.assembla.com/svn/arcdeveloper and you can get latest on everything. If you are not familiar with Subversion, Assembla has some handy starter info here, and you can Google for mountains of information.

Project Status (TRAC)

The project is managed in TRAC - since it's a web based tool, you can access it here: http://trac2.assembla.com/arcdeveloper/wiki. There's not a whole lot in there yet, but I'll be posting about the inital project shortly.

Bug Reporting / Feature Requests / Contributing

You'll need to be a member on the project to add a TRAC ticket. The easiest way to do this is to contact me, and I'll send you an invite directly from Assembla. If you are already an Assembla member (sigh-up here - takes 30 seconds and it's free), you should be able to contact me through the project's team page. Either case, as far as I can tell, I have to invite you into a project - I don't think you can "apply" to join.

Once you are a member of the project, you will also be able to contribute code.

So that's it for now - as I said, I'll be posting about the initial code base shortly...

I wanted to raise the coverage numbers noted in my last posting, so I jumped onto the code, looked at the coverage, and realized that I needed another test. I added this is, and my coverage jumped from 35% to ~70%. Good I thought - and went to look at the coverage output to see what other corner case I had not covered.

What I saw was very interesting. All the code was highlighted  as being run. Wha?

The SecurityManager.CanUserEditObject method basically does 3 levels of security checks - it checks the layer's "default policy", a layer-wide editor role, and finally a per-user, per-feature policy. Thus, the code is structured around 3 nested select blocks... (simplified example below)

 

What was interesting was that by directly returning from inside the case statements was causing the code coverage numbers to be lower than you would expect. Since this is a bit of a code-smell, I refactored the code to only have one return statement at the end. That alone got me 10% more coverage - no more actual code was run, but the coverage algorithm liked it better. Below is an image of the code-coverage highlighting.

 

Even at this point, the Visual Studio code coverage tool is reporting only 87.5% coverage for this block.

Anyhow - the take home here is that the code coverage algorithms are fickle, and even though you tests hit all the code, you may not get 100%.

In response to my last post on Refactoring and Unit Testing, Morten posted a comment with some very good questions...

1. Should you really have multiple test assertions in the same test? (I know is convenient to do, but also very messy from a test driven development perspective).
2. What is the code coverage of the unit tests.

Instead of just answering in comments, I thought I'd provide some more information as a separate post.

Multiple Assertions in a Test

It's true that this is a bit of a code-smell, but I view this as an acceptable trade-off between making tests easy to author (ergo you actually write and maintain them), and "best practices". As long as the asserts have good messages, what you get back is informative. Given that the class I'm focusing most of this testing on is a singleton, running multiple tests in a row is  realistic in terms of run-time usage.

Code Coverage

I had not setup code coverage when I wrote the original post. After setting that up, here's what I found:

At first glance, 73% is not bad coverage. But let's look at the class model for the SecurityManager class.

Looking at the ISecurityManager interface, we can see that the most critical method is CanUserEditObject, and it's only at 35% coverage! It's good to know that the private methods have high coverage numbers - 100%, 94% and 72%, but the actual method that is used by the client objects still has a ways to go in terms of good coverage.

When authoring the tests, my gut feel was that I had pretty good coverage - certainly shows the value of running coverage! I'm going to add more tests to up the coverage on this method, and I'll be posting more about how I created the tests later this week.

I just finished off some burly refactoring of our security management code base so that it could be better unit tested. This subsystem essentially controls what tools a user has access to, as well as what layers / features a user can edit, so it's a pretty critical part of the infrastructure.

I'm working on some posts about how I did this, and should get them out next week, but for now, I'll just show this...

Although there are only 16 tests listed, many of these methods actually contain a number of individual test assertions, and there are more than 80 tests that are actually run.

We put a lot of business logic into Class Extensions tied in on the OnCreate, OnChange, and OnDelete Object Class Extension events. If an edit violates business logic, it is rolled back and the user is informed about what was wrong. This has been working quite smoothly until today when we started creating our Object Inspectors (see previous post).

What we found was that attribute edits done through our Object Inspector (and consequently ArcDAL) were not respecting the business logic. I dropped some breakpoints into the OnChange event in one of the class extensions, and found that it was not hit when we saved changes in the Object Inspector. Yet, when I opened the table editor I would see the changes, so I knew something was happening. Here's the series of events graphically...

1) Original values in Object Inspector...

2) District Name updated and saved

3) Table editor shows the updated name, but OnChange event never fired!

4) When I try to edit the District Name via the table editor...

the OnChange event does fire, and the edit is rolled back because I do not have edit rights to that feature.

We traced it back to using update cursors. Turns out that updating features via FeatureClass.Update in an edit session does not store changes to the base table until the edit session is saved. Thus, the ObjectClassExtension events do not fire until you "save" the edits and end the session. We need these events to fire as the edits occur so the user is informed when business rules are broken.

The solution was to just change back to using IFeature.Store - which correctly raises the events. This was just 4 simple edits, but we have 25 feature classes in our geodatabase model. Thanks to code-generation, we just made the edits in the templates, and re-generated all the classes.

Ok, just had to share this because I think it's so darn cool. I've posted about our internal tool "ArcDAL" in the past (follow that link for details), but the quick review is that we have created a set of code-generated data access classes which wrap the geodatabase API. These classes fully support data binding, and because they are code generated, we can change the geodatabase model with limited impact to our code base.

Anyhow - while I was involved with the design and initial template generation, this is my first time actually using it in a project (This is the second major project using ArcDAL). It's just so fun to see this all come together, I thought I'd share...

So - here's how we make Object Inspectors (little UI's that "snap" into ArcMap's Attribute Editor)...

1) Create an inherited user control in the project. We created a base control to ensure consistent look and feel, so that simplifies things. Here's the new control...

2) Create an object data source from the ArcDAL List related to the feature class of interest - in this case ForestDistrictList

3) In the data source window, Set the display style for the ForestDistrictList data source to "Details"

4) Set the Display Style of fields we do not want visible to "None"

5) Drag the List data source onto the user control... and watch it layout the controls for you

Once the controls are on the form, you're almost done. For all the fields that are automatically populated, set the control to Read-Only. While this example does not have domains, those are handled and will automatically be dropped in as dropdown lists. Once the form is all laid out, we just have a minor bit of coding (~20 lines) to convert the inbound IObject into an ArcDAL object, and add it to the binding list, and handle some eventing so that changes are automagically saved.

I may have to create a little screen cast because it's just too cool to have it lay things out for you.

From a productivity stand point, this is amazing. And since it's data bound and code generated, even substantial changes to the geodatabase schema are relatively painless propogate up through the application.

A couple of times I've seen people asking about how to share information between forms/commands/tools etc. in ArcMap and I thought I'd create a simple example of how I've been doing it.

Problem

At a high level, the problem is pretty simple - one part of your application needs to know about the state of another part. One example could be a combo box tool control that allows the user to select an Active layer, and you have other tools/controls/forms etc that need to know what the active layer is, and need to be informed when the active layer changes.

Solution

This is an obvious case where custom events are going to come into play, since they are the standard way to provide notification of a state change to a set of "subscribing" or listening classes. In standard .NET applications, you'd create an event right on the actual source - in this case the tool control. However, since we are working in ArcMap, we have a slightly different situation. Since we (developers) do not control the load order of tools/commands etc in ArcMap, it's rather difficult to effectively sink up event handlers and sources if they are all located in your customization classes (ICommand/ITool etc). This is because we don't really know when either class is actually instantiated during the load process, and once it's instantiated, it's a pain to actually get a pointer to it.

The way to solve this is to route the events through something that's always loaded, and easily accessed - such as an application extension (IExtension).

Since application extensions (with the exception of JIT Extensions) are always loaded when ArcMap is starting up, we know it's always present. Regardless of when the event source tool / form / command is initialized, it will always be able to call a method on the extension, which then raises an event. The same it true for the classes handling the event - since the extension is loaded and accessible, they can wire up an event handler during their initialization, or any time after that.

While there are other ways to pass "state" around, using an event allows a whole set of classes to then subscribe to the event and be notified when changes occur.

Once you think about it, this is quite obvious, and it is a small scale mirror of much of the ArcObjects customization API - write a class, wire it into ArcMap, and have it respond to ArcMap events. Except in this case, it's our own custom event.

Sample

The sample is in C#, and is kept very simple, so the focus is on wiring the events.

Extension:

The extension has a public SetSelectedItem method, which in turn raises the SelectedItemChanged event.

Commands:

There are two commands - EventSourceCommand and EventSinkCommand. In the onCreate function, both commands get pointers to the event extension, and pass this into the IntializeUI methods on the forms. The Event Source Form has a simple combo box listing 5 fruits. In the OnChange event of the combo box, the form calls SetSelectedItem on the extension. The Extension raises the SelectedItemChanged event, to which the Event Sink Form has  subscribed, and a label is updated to show the currently selected fruit. Although it's a bit complex when you read it, this is very simple when you look at the code, and run it.

Since the sample is in C#, it uses event wiring involving delegates. If you are new to delegates, O'Reilly has a good article - Writing C# Custom Events, which I would recommend. The syntax is much simpler in VB.NET, and here's a CodeProject article on that. Of course you can use delegates in VB.NET if you want to, so for completeness sake, here's an article on events using delegates in VB.NET. Moving on...

To run the sample, download the code, compile it, and run ArcMap. You should get a pop-up box as ArcMap is loading which tells you that the Event Extension is loading. Once ArcMap is started, open up the Customize dialog, and look in the DaBo.Samples category.

Drag the Event Sink and Event Source command onto a toolbar, and then run them. Both commands open forms. Use the pulldown on the Event Source Form to update the label on the Event Sink Form.

When you dig into the code you'll see that I've used the out of the box templates for the commands and extension. The only other "cool" thing I threw in was the ArcMapWindow class I wrote about previously.

Have fun wiring up your events!

Download Sample

While I was at home, away from my source tree at work, I was looking to correctly parent a form to ArcMap. By parenting a form to ArcMap, it will be minimized with ArcMap, and will always stay in front of ArcMap - important if you want to have multiple forms up at the same time, and what them all to be visible!

In the past we've always used the Win32 API 's SetWindowLong function - which we wrapped up a little in a helper class. Not having the source to look at, or the helper class to use, I needed to figure out the syntax. To complicate things, I was also writing the code in C# - it's those subtle changes that will always mess you up. After finding the PInvoke site, and locating the SetWindowLong function there, and being somewhat confused about how to get an IntPtr from Application.hWnd, I did a search in the ESRI forums. Low and behold I found this recent post (July 31, 2007) by Berend Veldkamp in which he provides a very simple way to parent a form to ArcMap.

In .Net 2.0 it's possible to pass a parent window to the Show method of your form. This should be an object implementing IWin32Window, here's a basic example of how to do that in VB.Net 

Doh! Amazing what you overlook because you're used to doing things the old way - I had no idea that Form.Show had a parent option. Anyhow, I'll let you get the VB.NET version from the forum post - here's the C# version: 

Now, to show a form parented to ArcMap, all I need to do is...

This is much simpler and avoids the Win32 API call. Thanks Berend!

We've been having some issues where Visual Studio 2005 crashes when building some of our projects. Nothing graceful here - the IDE just disappears. To make things more fun, it's inconsistent, but for some projects it's happening about 80% of the time. As you can imagine this is wasting a ton of time and is incredibly frustrating.

Today I was working on enhancing our MSBuild scripts to do something more elegant than just building one master solution and I ran into the same problem - only this time I could see an error message:

    Target UnmanagedRegistration:
        C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Microsoft.Common.targets : warning : Type library exporter warning processing 'Sanborn.ArcEnterprise.Productivity.Query.sbenumQueryStepTypeCollection, ArcEnterprise.Productivity.Query'. Warning: Type library exporter encountered a type that derives from a generic class and is not marked as [ClassInterface(ClassInterfaceType.None)]. Class interfaces cannot be exposed for such types. Consider marking the type with [ClassInterface(ClassInterfaceType.None)] and exposing an explicit interface as the default interface to COM using the ComDefaultInterface attribute.

Shazam! The error message actually tells you what's wrong and how to solve it! Only you can't see this message when building in the IDE if the IDE crashes.

The problem in a nutshell is that there are no COM types which can be mapped to .NET Generics, and when the type exporter tries to export them, it sometimes causes the compiler and Visual Studio to bomb. (For those not using generics, I recommend checking them out - here's an intro)

Since we are doing ArcGIS Development, most of our assemblies are marked to be registered for interop when they build.

By default, the type exporter tries to export all classes in the assembly for COM interop. To avoid this, you just decorate your classes with some attributes which stop the class from being exported.

Specifically, you should decorate any classes which inherit from generics, or have a generic type in a method signature, or implement an interface that has a generic in a method signature. The example below shows the syntax in VB.NET

You'll also need a using/import statement for System.Runtime.InteropServices.

If you have defined interfaces which have generics in the method signature, just use the <ComVisible(False)> attribute.

Imports System.Runtime.InteropServices

    <ComVisible(False)> _

    Public Interface ICreateViewScriptsPluginView

Once I made these changes, MSBuild was stable, and it seems like builds in Visual Studio are now stable as well.

We've been having some issues where Visual Studio 2005 crashes when building some of our projects. Nothing graceful here - the IDE just disappears. To make things more fun, it's inconsistent, but for some projects it's happening about 80% of the time. As you can imagine this is wasting a ton of time and is incredibly frustrating.

Today I was working on enhancing our MSBuild scripts to do something more elegant than just building one master solution and I ran into the same problem - only this time I could see an error message:

    Target UnmanagedRegistration:
        C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Microsoft.Common.targets : warning : Type library exporter warning processing 'Sanborn.ArcEnterprise.Productivity.Query.sbenumQueryStepTypeCollection, ArcEnterprise.Productivity.Query'. Warning: Type library exporter encountered a type that derives from a generic class and is not marked as [ClassInterface(ClassInterfaceType.None)]. Class interfaces cannot be exposed for such types. Consider marking the type with [ClassInterface(ClassInterfaceType.None)] and exposing an explicit interface as the default interface to COM using the ComDefaultInterface attribute.

Shazam! The error message actually tells you what's wrong and how to solve it! Only you can't see this message when building in the IDE if the IDE crashes.

The problem in a nutshell is that there are no COM types which can be mapped to .NET Generics, and when the type exporter tries to export them, it sometimes causes the compiler and Visual Studio to bomb. (For those not using generics, I recommend checking them out - here's an intro)

Since we are doing ArcGIS Development, most of our assemblies are marked to be registered for interop when they build.

 

By default, the type exporter tries to export all classes in the assembly for COM interop. To avoid this, you just decorate your classes with some attributes which stop the class from being exported.

Specifically, you should decorate any classes which inherit from generics, or have a generic type in a method signature, or implement an interface that has a generic in a method signature. The example below shows the syntax in VB.NET

You'll also need a using/import statement for System.Runtime.InteropServices.

If you have defined interfaces which have generics in the method signature, just use the <ComVisible(False)> attribute.

Imports System.Runtime.InteropServices

    <ComVisible(False)> _

    Public Interface ICreateViewScriptsPluginView

Once I made these changes, MSBuild was stable, and it seems like builds in Visual Studio are now stable as well.

Just a quick note that I've added 5 feeds to the ArcExperts.net aggregator. They are:

• Chris Spagnuolo's GeoScrum - I work with Chris and his blog is really his insights into project management and lessons learned by our team as we adopt Scrum.
• ADC Develpers Blog - looks to be a great technical blog written by and for developers.
• Raba Shani's Human-Debugger blog. Kudos for also running dasBlog!
• Flat out GIS blog with Shane
• Jeff Germain's blog on ArcGIS development in .NET

Just a reminder - if you write a GIS Development blog and want to see it in the ArcExperts aggregator, just use the contact me link at the top (and I promise to add feeds faster in the future!)

As I sit here looking at my little $10/month hosting account, I'm seeing that ArcDeveloper.net is taking up a lot of space, and getting relatively little traffic. In addition, I believe that the vast majority of the "1279" the registered users are actually spam accounts which never activated because their email addresses were bogus. And you can bet I'm enthusiastic to dig in there and clear them all out. 

On the bright side, Google Analytics shows a consistent 100 or so visits a day, with 70% of the traffic coming from Search Engines - at least it shows that people are searching for what the few participants are discussing.

And I know from talking to some of you that it has been a good resource from time to time. But, as ESRI is now fully supporting .NET 2.0 (a primary reason for starting this up), and their forums are threatening to add RSS feeds any day now, perhaps this site's time has come and gone.

The question for me (and I guess for you) is - should I keep it going? I've got a lot of other stuff on the go, and so promoting/maintaining the site is just not happening. The wider ESRI Developer crowd has not jumped in either - I'm guessing it has too much overlap with the main support forums.

If people think it's still useful, please let me know in the comments, or via the Contact link at the top of this page. Is there anything else you'd like to see on the site? Different forums? Should it be running on different software (Community Server does many things "ok", but nothing really well in my opinion - and it's a pain to administer).

What would drag me in is one or two simple forums:

1) Agile Development and ArcGIS - which would cover things like Test Driven Development, Continuous Integration, Automated builds, WiX, Sandcastle, and all the other good "goo" that goes around a successful development team.

2) Quick Hits - killer questions and  simple solutions. Kinda like a light-weight community blog where you can post questions that just are not going to get any play at the ESRI forums, and where people without blogs can post how-to's.

Another option would be to kill Community Server all together and slap up a semi-secured wiki (i.e. read-only for public, editable for email validated users). Much more free-form, but that also has it's uses.

Anyhow - let me know what you think the future of ArcDeveloper.net should be. If someone else wants to take over the site (and move it to their hosting account) I'd be up for that too.

I recently recieved an email from a reader asking about role based security & ArcGIS Server. Specifically they were interested in restricting access to columns, based on the user's credentials.

I'd been doing some thinking on similar issues related to row based edit permissions, and I thought I'd share my ideas on possible solutions to these problems.

Fine Grained Access Permissions

Essentially, you can't use the DBMS security capabilities to restrict user access to columns or rows because ArcSDE does not support security at this fine grained level (FeatureClass, ObjectClass or FeatureDataset only). There may be some hacks, but as far as I know nothing like this is supported directly.

Next, and possibly more important, regardless of how you actually create the connections (i.e. store the connection properties in the MXD file, or use pass-through authentication and grant the SOC process permission to connect into the database) all interactions with features are done as the SOC Service user. The identity of the front end user  is not passed back to the SOC. Thus, regardless of who is running the application, the SOC identity is always the same. The diagram below shows how far you can pass back the end user's identity (assuming you setup impersonation in your web app).

 

The SOC always runs as the SOC Service user - so any use of the end user's identity to control any sort of access is not going to work.

Possible Solutions - Restricting Access to Columns

There are a variety of ways to tackle this, but none are particularly flexible - you will need to define the roles up front, and then setup the application to respect those roles explicity. Adding additional roles will require code changes. Four options come to mind

1) Let the SOC return all the data and apply the permissions in the client application. This is not a bad solution if you control the web app. It basically means re-writing a lot of the out of the box tools so that they will respect this additional business logic.

2) Setup the spatial data so that it has the minimal set of field that all users can see (including ObjectId). When doing an inquiry, the web app would get the base set of fields from the featureclass, and then use the ObjectId  to run a second query against a multi-version view to retreive the other "secured"  attributes. There would be a multi-version view for each role in the application, and the columns would be restricted as needed. The application logic would need to handle the mapping of the users role to the correct view.

3) Use Spatial Views. Similar to #2, but the SOC would load layers based on the user's role, and the layers would actually be Spatial Views with only the columns that are visible for the particular role of the user. I suspect this would be slower than the previous two options, as the SOC would need to load layers all the time, but if the application is using a non-pooled SOC anyhow, it may not bemuch more of a performance hit - just start with a minimal map (just the layers which do not have any restrictions on them). The upside is that you can likely use most of the out of the box tools since the column restrictions are being applied by the view, at the ArcSDE level. Another upside of this model is that the restricted data is never available - either in the SOC, or in the web application tier.

4) Similar to #3, you could separate out the restricted fields into separate tables, and apply joins on the fly. Same benefits as #3, but you'd still want to work with non-pooled SOCs. Not sure how well this would play with the out of the box tools, but it may be ok.

I'm sure there are other options, but I suspect they are all variations on this same theme. If the application was super simple (like 1 ID tool), you could just limit the fields returned by setting the IQueryFilter.SubFields property.

Row Based Edit Permissions

For our applications, there are typically business rules which restrict the area in which a user can edit features, and related attribute tables. In the past we have implemented this logic in feature class extensions. In the extension, we simply get the current principal (the user's Windows Login identity) lookup their access permissions from our repository and allow/block the edit. We have logic in the extension which allows for editing of shared boundaries (which I believe is one of the issues with implementing this sort of logic at the DBMS level)

This works just fine for Desktop and ArcEngine applications, but is more problematic for web applications. As I noted at the start of this post, all connections into ArcSDE from ArcGIS Server run as the SOC user. Thus we have a problem - if we apply a class extensions to the geodatabase, it's logic always fires - regardless of the client appliction. But, if all the edits from everyone using the web app effectivly run in the same context, (agssoc or whatever you call it on your system), then this logic is both moot, and a waste of processor time. But we still want to use this logic for the desktop apps, so we had a bit of a problem.

Our solution is to create feature class extensions which are themselves extensible. Basically we use  the Provider Model to add in various security providers at run-time. When the class extension spins up, it uses the ConfigurationManager to check the configuration file (ArcMap.exe.config for desktop, web.config for web apps) for "SecurityProviders". In the desktop apps, there will be a provider configured, the class will be instantiated, and the logic applied to all the edits. In the web app, we do not configure a security provider, thus there is no logic applied. 

This still begs the question of "how does the row based edit permission get applied in the web app?". At this time, there is no (easy) way to pass the user identity back into the SOC, let alone get the SOC to impersonate a user such that the class extension would be running in the context of the front end user. So, for now, the answer is - the web application must handle this logic itself. While this is not idea, at least the class extensions can be used in the desktop apps - which is where the vast majority of the edits will occur anyhow.

I've been working on the System Architecture for a Citrix based enterprise GIS system, and in the process have dug up a bunch of ESRI whitepapers on the topic. If you run ArcGIS in a Citrix environment, these are some good reading...

 
System Design Strategies - ESRI Whitepaper (PDF Jan 2007)

This is the mothership for all ESRI system design topics. Check pages 8-8-3 for the WTS/Citrix server sizing models (partially shown below)

According to the model, you can run 45 ArcMap sessions on an Intel Xeon with 2 Dual Core 3Ghz (5160) CPUs and 16GB of RAM. In case your were wondering - you can pick up a Dell PowerEdge 1950 with this configuration for ~$10,000. And if you want to be really cool, you could go with the quad core CPU's for about ~$2,000 more. RAM starts to get pretty pricy when you go much above 16GB, so while you likely can't stuff 90 users onto a quad core system, for $2,000 more, your 45 users will have a lot more power at their disposal.

Citrix / WTS Support Overview - Whitepaper (HTML - Feb 2006)

This covers the main topics, including how to configure multiple products (ArcView/ArcEditor) to run on the same server.

Force ArcMap to close when logging off Citrix - ESRI Tech Article

Printing from ArcGIS on Citrix - ESRI Whitepaper (PDF - Oct 2006)

System Architecture to Support Printing on Citrix - Citrix Whitepaper (PDF - Nov 2001)

Metaframe Printing Optimization Techniques - Citrix Whitepaper (PDF - Feb 2004)

 

I'll update this post as I find more resources.

 

Just got the notice from ESRI that it's time to register for the Developer Summit!

Last year it was a great event - good technical info in the sessions and a chance to talk to the developmen teams, other bloggers and other developers. I'm planning on going, and brining my project leads along as well.

Last year there were "Birds of a Feather" talks which were informal meetings, setup by attendees, on various topics. Last year I informally led a session on unit testing with ArcObjects, which drew a much larger crowd than I anticipated. (see below - I expected nobody to show up!)

Assuming anyone is interested (and that they have these slots available again), I'll try to get another one setup for unit testing and a second one about code generation and the concept of an object based data access layer for the geodatabase, and using databinding with that.

If this sounds like a good idea (or if it sounds painfully boring) let me know in the comments.

Also - I think I've got the IE problems solved on this site, if I don't please let me know (or use FireFox!)

See you in Palm Springs!

Since we are hiring, we need to get some more workstations - thus it is time once again to see how much power a couple of grand will buy. If you have recently bought a kickin workstation for ArcGIS Development - I'd love to hear about your experiences...

Instead of simply specing out a top end box without much idea as to how it would perform, I decided to try and figure our what ESRI would recommend. I started by cracking open the 2006 System Design Strategies white paper from ESRI. I say cracking, but really it was quite a bit of waiting as Acrobat loaded the 289 page document.

Here's the important bit - on page 7-17, the document states that ESRI uses the "SPECInt Rate 2000" as their system comparison benchmark. Conveniently the rates for all kinds of systems are published so you can check things out.

After further digging in the system design document, it turns out there was little in there that I could directly apply to the developer experience i.e. they have not done benchmarking for how fast ArcMap starts, or how fast visual studio is while debugging. So, I decided to check the rating on our current systems, so I'd at least have a relative ranking.

Current Workstations:

Dell Precision 370, 3.8Ghz P4 w/ hyper threading, 3GB RAM, 256Mb dual head video card, ~100GB disk

While not specifically listed, a similarly equipped Acer system with this chip rates a 23.5

While not bad, these workstations certainly could be faster - particularly when starting ArcMap, and running Visual Studio 2005 with large solutions. Thus, I started looking at systems which had a higher Spec rating, and more specifically at the CPU's which had the high ratings.

AMD Athlon vs Intel Core Duo

Obviously there are other factors which make a PC fast, but at the heart of it all is the CPU, so may as well start there.

The top rated AMD chip is the Athlon 64 FX-62, with a 43.7

The top of the heap from Intel is the Core 2 Extreme X6800 at 63.5 with other Core 2 Duo's coming in at 58.7 (E6700) and 53.7 (E6600). Based on this, I figured that we should be able to at least double our workstation performance. The next step was figuring out what is was going to cost.

Looking for Systems

Traditionally we've been a Dell shop, so I started there.

The Dell Precision 390  can be configured with each of the Core 2 Duo chips. For the pricing below, all have 4GB of DDR2 RAM, and 2 80GB harddrives, 128Mb dual head video card, CD/DVD burner WinXp. (For those scratching their heads on the 80GB drives - we really don't need a lot of disk space, as 90% of our projects use ArcSDE as the data store.)

Here are the prices (as of 10/27/06)

1. Core 2 Extreme X6800: ~$2800 (Spec 63.5)
2. Core 2 Duo E6700: ~$2350 (Spec 58.7)
3. Core 2 Duo E6600: ~$2150 (Spec 53.9)

Not bad, but last winter I built a box for home and I've been really happy with it, so I thought I'd check out what I can get from there - GlobalComputer.com.

They have a Systemax system that's build to order - same basic config - 4GB DDR2 RAM, 2 80GB hard drives, 256Mb dual head video card, CD/DVD burner, Win Xp.

1. Systemax Core 2 Duo E6700: ~$1950 (Spec 58.7)
2. Systemax Core 2 Duo E6600: ~$1650 (Spec 53.9)

Verdict?

It's hard to say - Dell does have great support, and it's the exact system noted in the Spec ratings, so that's a bonus. That said, is the support really worth the $400-$500 difference? An even better question relates to the cost difference vs. performance difference between the top $2800 system (63.5) and the much cheaper $2150 system (53.9). Is that 10 spec points really worth $650? Or is there just a premium on the fastest chips?

We'll have to make the call next week, but right now, I'm leaning towards the Systemax boxes. We've had great luck with other off-brand boxes (1/2 our servers are SuperMicros) and I'm sure that I can find some other cool stuff to get with the remaining cash.

The Sanborn GIS Development Team is expanding!

At this time, we are looking for two full-time ArcObjects developers to help us build enterprise GIS systems. These are entry to mid-level positions, with plenty of opportunity for growth. You'll be working with a relaxed team of professional developers who strive to build the best damn GIS software solutions out there. These positions will be working on both desktop customizations and back-end ArcGIS Server component development. Current projects include an ecosystem managment toolset for the Marine Corps (ArcGIS 9.1 + .NET 2.0), and a state wide forest managment system for Pennsylvania (ArcGIS 9.2 Desktop & Server + .NET 2.0).


What is the job?
Developing enterprise scale applications with ArcGIS and .NET - these are end-to-end systems, typically involving multiple technologies - ArcMap, ArcGIS Server, ArcSDE, ArcIMS, smart client, winforms, web forms. 

When I look at positions, one thing that I always ask about is the tools, so here's what you'll be working with:

• Microsoft Team System - Source control  project managment
• ArcSDE on Microsoft SQL Server and Oracle
• Visual Studio 2005 - VB.NET or C#
• MSDN subscription
• ESRI EDN Subscription
• Dual flat-screen workstation (3+GB RAM, 3+Ghz CPU)
• Citrix farm for client test access
• Good desk and chair - sorry no aerons. (yet!)

Who will fill it?
Are you the go-to person on your team? Are you sick of the general 'hacking' nature of GIS development? Do you know ArcObjects like the back of your hand? Interested in a more 'agile' approach to development? Can you whip out winforms apps? Dig web services and smart client applications?  Are you applying Unit Testing to your GIS applications? We want to hear from you!

Regardless of the laundry lists of skills in the formal postings, we are really looking for quick learning, self directed team players who really give a rip about producing quality software, and staying on the leading edge of software development. And if you blog, all the better!

Where is it?
These positions are located in sunny Fort Collins, Colorado - I've been here 10 years, and love it, but don't just take my word for it -this year Money Magazine rated it the #1 place to live in the US.

When can I start?
We are looking to fill one position by early November, and the second by early January. Check out the detailed position descriptions and then email me (dbouwman AT sanborn DOT com) your resume.

Things are going well with our ArcDAL project (Code Generation + Geodatabase = Goodness), and now that our base class library is pretty solid, I'm adding in unit tests. We want these tests to be run on every build, on any development system, so I want as much separation between the tests and any underlying required resources - for example - ArcSDE instances. Conveniently Scott Hanselman's podcast from last week was all about using Mock objects in unit testing.

What is a Mock?
Basically, a mock object is a "fake" object that is created solely for the purpose of running a test. It's generated on the fly using some mocking framework, and you basically tell it what to expect in terms of requests (properties / methods) and what to return. One place where this is really handy is when you want to avoid binding your tests to a specific instance of ArcSDE, or a specific file location for a personal geodatabase. It can also be used to avoid having to make calls out to other ArcObjects if you don't actually need to.

I'll go through a simple example to show what I mean...

Example:
Before I go any further, let me say that I'm using Visual Studio 2005, and the MSTest framework, but this will be pretty much the same in VS2003, and/or NUnit.

We'll look at the tests for a Workspace Utility class...

This class just has 3 very simple shared methods which return information about an IWorkspace. The thing is that in order to test this with real instances IWorkspace, we'd need to actually connect to ArcSDE and a personal geodatabase. And this will make the tests brittle, as it's unlikely that we'll keep an instance up for a long time, and file paths will be different on different development machines.

Thus, we use Mock objects to "fake" the workspace. I've chosen to use the