Dev Summit 2008 is now over and done. Great to see so many old friends and meet tons of new ones. This conference is "the" event for developers working with the ESRI product line, so if you could not make it this year, start working on your managers now to get approval for next year.

One idea that ESRI threw out at the closing session was the idea of running another "track" that would be technical sessions presented by users. I think this would be awesome (and not just because I'd love to give a talk!). I briefly talked with them after the Q&A session was over and the idea would be that the developers would be able to submit some sort of a description of the presentation, and then the rest of the community would vote for the sessions they want to see. This would be fantastic. If this sounds like a good idea... um... yeah. I just looked for a link on EDN to send feedback (about something other than the site itself), but did not find anything that looked suitable. Just leave a comment, and I'll find out the best way to push that back to ESRI.

While decompressing on the lawn outside the conference center, it occurred to me that having some other smaller, focused, but really technical events at other times during the year would be a great addition to the Dev Summit. Sure they could not drag the dev teams all over the place (someone has to work on 9.4!) but maybe bring some folks from tech support / professional services? Just an idea...

A quick note on my posts from this week - they were just the notes, as I typed them during the sessions, and I hope that was useful for those who could not make it out here. If something was particularly cryptic, shoot me an email, and I'll see if I can figure out what I meant.

It's been a fun week, and thanks to everyone at ESRI for putting on a consistently excellent event. See you next year!

When I got in this morning I had an email from ESRI asking me to take the 2008 ESRI Developer Survey. I asked the rest of the team and they had not gotten it (yet?) so I thought I'd shoot the link out to everyone. The survey is a bunch of questions related to ESRI developer resources - basically what are you using, how often to you use it, what languages are involved. I'm not sure who all is "officially" invited to take this, but it seems that anyone can access the survey. So - if you want to be heard, this is a chance to let ESRI know what you think and what you want. Get over there and take the survey!

Ok, it's a little grainy, but if you look closely you can see the shiny new ArcGIS Server REST API behind all that fur.

Kudos to ESRI for actually showing some beta software. Typically this sort of stuff is locked down until the final release. Does this mean that people in the beta program will be able to stage up applications for the public to view? Will they be able to talk / blog about it? And most of all - does this mean that we'll see a release before the User Conference?

Just read an article on GISUser.com (also at GISCafe) about ESRI bringing on Dirk Gorter (formerly with Symantec) as new Director of Product Management. Here's a quick quote:

Gorter's focus is on implementing a product management process that is similar to the agile product development process commonly used by software development teams.

This is great news! Hopefully ESRI will be talking up agile as they go through their own changes. This level of exposure is sure to help raise awareness of the benefits of agile development within the broader geospatial industry - maybe we'll see some ESRI people at the Agile Development Practices conference next week!

Some quick items from the plenary that were interesting...

ArcGIS Server
More mentions of supporting ArcSDE on PostGIS. Mild applause from the crowd, but it seems like this is still on track to ship at 9.3
Support for ArcSDE on Oracle Express
Image Server will be integrated into Server. Not clear if it will be an extension or part of the core.
On the Mobile ADF side of things, there will be an out of the box Mobile app that ships with 9.3 - so it's no longer just a developer toolkit - I think it's now called ArcGIS Mobile...
Javascript API and REST services for the mashup crowd.

ArcGIS Explorer
Really good demo of ArcGIS Explorer and ArcWeb Services. The data was really really fast - not clear if this was running "live" or from local servers, but it was screaming fast (maybe it was all pre-cached?)
Also notable was the ESRI Content Sharing service - apparently they will host your data for you if you agree to share it. And I may have the name wrong, but I think it's called ArcDataServices which gives you the ability to request (buy? was not clear) data, which is re-projected/extracted etc for you. What was cool is that it's powered by i-cubed's DataDoors on the back end. Kudos to the whole team at i-cubed!

Support Issues
The spin on the current huge backlog of support issues was that it's due to the massive and rapid adoption of 9.2, which has overwhelmed support. That may be, but it's also pretty buggy. Anyhow - the plan to address these issues was laid out. Over the last year they have been re-structuring their whole lifecycle to include "support" as an active element. Support personnel have been physically co-located with the developers. Additionally they said they will open up the entire issue database. Total transparency. What they see, we will see. This got a lot of applause, but we will see...

There were also some cool demos, and the requisite Jack Dangermond "grand vision" segments, but overall a very good opening session. I'm actually excited to talk to the ESRI people about some of the new stuff and some of the 9.2 stuff that I had not known about.

Recently ESRI has been taking surveys as to what they should be focusing on @ support.esri.com. In their "early results" posting, they are apparenly taking notice of the limited search capabilities on the site. Just wanted to add that the search on EDN is also in need of some love.

In prepping another post, I went to the "Current Library" page, and typed in "IObjectClassExtension" - which returned a whole mess of things which make some reference to IObjectClassExtension, and it actually hilights the term in BOLD, but it did not have a link to the actual page that describes the interface.
I changed the search setup a little, and got all 32 references back on the same page - the interface was at #24. I'm not sure how much more explicit I can be with this. Please please please work on making this search better!

Incidentally, Google nails it (ok, it's the 9.1 Java doc, but at least it's the interface!). Instead of trying to tweak whatever search engine they have, maybe they could work with Google to ensure that the site is accurately and regularly indexed, and ensure the site:edn.esri.com <search> works (currently this does not return any results)

Where did April go? Anyhow - I'm finally getting my head above water, and was catching up on my blog reading when I noticed that James posted a link to the podcast interview I recorded at the 2007 ESRI Developer summit.

For those interested, here is a direct link to the MP3. And kudos to whoever edited it, since it sounds much more coherent than I recall!

I'm thinking the next few weeks will be some what sane, and I've got a few blog posts started, so things should be back to normal shortly.

ESRI has posted a PDF with summary stats of the Dev Summit pre-conference survey results. I think it's very cool that they have released this information, and there are a few interesting nuggets in there...

Importance of Blogs as a Resource

I find this pretty suprising, since I believe that if you asked the same question to the much broader .NET developer community, blogs would be in the top 5 - mainly because of the positive impact that the Microsoft developer blogs have had. Since 97% of respondents said that "Developer Resources" are important, I believe that this result simply reflects slow adoption within the GIS community.  Despite this result, I hope that ESRI increases blogging by the product teams and professional services.


Developer Technologies


Three things stand out on this one...

ASP.NET Supremacy
ASP.NET seems to be pretty important to almost 80% of the respondents. Since I'm pretty sure that there are still an awful lot of people writing code for desktop, it makes me think that there is a tight correlation between ArcGIS Server developers and a willingness to take on-line surveys.


Visual Basic 6 beats Python, Java....
This suprised me. I loved VB6 when it was the only tool we had to work with, but it's just so painful compared to using .NET - and the transition to VB.NET is about a smooth as can be - and you can get the Free Express Editions - why why why are you still writing VB6? (I can understand support & maintenance) The fact it beat out Python and Java makes me think that there are not many Python/Java/Perl etc. developers out there.

REST? What's a REST?
While this is still a fermenting concept, REST architecture is a great way to do simple "servicey" things. I'm glad that ESRI included it on the list, and I think it will move up as more people start grokking it.

I was catching up on my blog reading over lunch and happened onto Jason Birch's blog, where I found a posting about how to add EDN search directly into your browser (IE7 or FireFox 2.x).

For some reason his blog is not aggregated on PlanetGeospatial [UPDATE: It is listed under the blog name "Random Nodes" and not his name - thanks James!], and was not in by blog list, so I had not seen this. And since there were not comments on the post, I'm guessing that others have also not seen this. Check out the post and leave a comment for Jason here. Add EDN Search to your browser here. This is great. Thanks Jason!

Upon further reading, it looks like Rob @ the Spatial Miscellany blog posted the seed of this. Jason wove in the OpenSearch stuff so you don't have to dirty your clip board with a URL - it just pulls it from the page. All very cool.

The ArcExperts.net feed is up and running. It caches for 15 minutes, pulls it's own copy of the feeds, and is running through Feed Burner.

You can subscibe using this url: http://feeds.feedburner.com/arcexperts or directly at ArcExperts.net

Neither the site, nor the RSS feed are particularly optimized beyone caching. Specifically, they do not utilize Entity Tags or SkipHours in the source RSS feeds (read more here). Both of these allow the consumer of a feed to be more intelligent about when it pulls feed data - thus saving everyone bandwidth and improving performance. Also, the underlying JRN blog roller code does not specify how far back the posts should be pulled - it just says how many posts should be pulled.  So, for blogs where there are sparse postings - say one in May, one in June, 2 in September and one in November - the current code would fetch all of them because it's just pulling the last 5 posts in the feed (this assumes the person's rss feed is not date sensitive - many are).

If we actually get a reasonable number of subscribers / users of the site, then I'll look at re-tooling it - either with Dimitry Robsman's ASP.NET RSS Toolkit, or just try to re-work the JRN blog roller code to be a little smarter.

Anyhow - it's up and running, and we'll see how it goes. Contact me if you've got questions / ideas / want a blog added or dropped etc.

Just a quick note - ArcExperts.net does not have an RSS feed yet. Your browser or RSS reader may detect a Feed on the page, but it's just picking up the feeds on the Blog Roll.

I'm going to try and whip up an RSS feed this evening, but we'll see how that goes. I think it should be pretty easy to put together as I parse the feed items, but I'm a little unsure about ordering the xml fragments by date. It will also use a 15 minute cache.

Also - If you've asked to have your feed added,  they will be added this evening.

I will post the feed once I have it working.

Since PlanetGeospatial is running in "Lite" mode, and I had a few hours while my son was napping,  I decided to put up a blog roller that will aggregate ArcGIS / ESRI Developer blogs. I like PlanetGeospatial, but all the Google Earth / Virtual Earth and Island shoreline stuff was a little off-topic for me.

Anyhow - you can play with it over at ArcExperts.net. I grabbed this domain a year or so ago, and it's just been wasting space, so it seemed like a good home for something like this. As usual, this is a free service - any ads which show up are part of the originating RSS feed.

About the code...
The site is ASP.NET 2.0 code, based on the JRN Blog Roller project that's hosted over at CodePlex.com. This gave me the foundation of the roller itself - conveniently packaged as a UserControl that is easily embedded into any page.

At it's current 0.3 release, JRN's blog roll configuration is a semi-colon delimited list, stored in the web.config file. Since this was kinda kludgy, I grabbed an OPML parser from Bruno 'Shine' Figueiredo's site http://www.brunofigueiredo.com . By folding these two together, I'm able to manage the blog list as an OPML file (download here), and still leverage the bulk of JRN (which actually is pretty lean).

I also took this opportunity to play with XmlDataProvider and found out just how easy it is to work with Xml files in ASP.NET. The blog list is literally 5 lines of code... I'm going to do post about this since it was just so crazy easy to use.

Getting on the Roll...

If you want your blog added to this site, contact me through this site, or over at ArcExperts.net. If your blog is listed, and you want it removed, same thing. Right now it's working with RSS 2.0 feeds and stuff from Feedburner. I'm skeptical if it will parse Blogger.com feeds - if someone wants to have one added, I'll try.

Performance...
I've set things up to cache for 15 minutes, so as long as people hit it occasionally, it should be pretty speedy. I'm still having issues with my hosting, so we'll see how this works out.

Future Additions...
When I have some more time, I'm going to change the main page to background load the feeds via Ajax. From there I'll create a RSS feed from the site, and run that through feedburner.

I'm open to any other ideas, and would love to hear how this works for everyone.

[UPDATE: 1/31/07 I got a note from a member of the ArcMap team who indicated that when using the IActiveView interface on a MapDocument object, you need to call IActiveView.Activate. The sample has been updated to reflect this, and you can read more about this in the ArcObjects documentation for IActiveView.Activate]

Thought I’d cook up another simple example application that shows how you can streamline the creation of repetitive maps. In this sample I’m going to create either a PDF of the map layout, or a JPEG of the map. This sample also shows another use of the configuration section handlers I posted recently. It's written in VB.NET, and it’s based on ArcGIS 9.2.

Scenario

You work for a county GIS department and every day people come in, and ask for maps of their property. These are very simple maps – basically just the parcel outline with other base data. Since you are an industrious person, you always have ArcMap open doing some other work, and it’s a pain to close your current map, open the county’s standard Parcel Map mxd, zoom to the parcel and export the map. You want a simpler way to do this.

IMapDocument & MapDocument Class

This is the central point for getting things started. The MapDocument Class can be instantiated and used to load a map document. It’s worth noting that this happens much faster than opening ArcMap – the whole thing – open map, select feature, zoom to it, and export to PDF takes ~ 1 second. Time savings of not watching ArcMap spin up could more than pay for the effort to customize this sample!

Once you have the document, the ArcObjects code needed to zoom to a particular area is very simple. The export to an image is somewhat complex, but the sample wraps that up in the ExportMapToImage and ExportPDF methods.

Sample Solution

The sample solution has two projects – the MapMaker class library that contains the configuration handler, and the code that actually exports the maps, and a simple WinForms app that can drive the class library, Sample data and a MXD file is included in the zip file. All you need to change to get this to run is set the path to the mxd file in the app.config of the MapMakerForm project.

 

MapMaker Class Library

The MapMaker Class library really only has one class of signifigance - the MapMaker itself. Since I like to make things flexible, it also has a configuration section handler, which is used to define the various map files that the class can work with. These are referred to as “MapMakerServices”. The class model is below

A “service” has a name, an mxd file, the layer the query will be run on, and the field that the "where clause" in the query will be applied. The following shows the app.config section for the sample.

By having a config section, you can cook up a simple application that can access a wide array of map files, without having to change the actual implementation at all. Although there is only one service defined in the sample, the list of MapMakerServices is bound to a combo box.

Functionality

Basically, you pass in a whereclause, the height and width of the map you want, the type of output (JPEG or PDF) and an output file. The MapMaker will then locate the layer specified in the service, query it for a matching feature, zoom to it, and create the map.

 

The data that comes with the sample is of parcels. So the usage in the sample is that the user passed in a PIN, the MapMaker locates and zooms to the parcel, and then exports a map image. The layout in the sample is plain as can be, but I’ll leave the cartography up to you.

 

Anyhow – this is a super simple example of how to automate ArcMap map creation without actually firing up ArcMap.

Download the source code (2.25Mb includes sample data)

Have fun!

I was just getting the details for our trip out to the Dev Summit, and I noticed that they added a little something new...

What's cool is that the community gets to vote on the entries - so it will be interesting to see what get's submitted.

Avoiding IP Issues
Before everyone starts submitting code, I'd recommend clearing your submission with whoever you report to. Some organizations can be very concerned about intellectual property, and many times the people who are the most concerned are the least knowlegable about what can be shared vs what is an actual "trade secret". Best be safe than sorry.

Damn - I just noticed that James posted about this! There is no getting ahead of him!

Despite my sometimes pointed postings regarding ESRI, here's a little something from the DevSummit site...

This started as a comment over on James' blog, but it got a little long for just a comment.

"gonzo" had replied to the comment that “ESRI has no network infrastructure to compete with Google.” with this:

This point could be argued, considering that ESRI services are published from AT&T data centers in Virginia and New Mexico, also offering triple redundancy of power and geographic redundancy.

[UPDATE 12/7/06: upon re-reading this post today, I concluded that the section I have removed, did not add to the conversation.]

I'd just like to point out that while ESRI hosts servers in  "AT&T data centers", Google and Microsoft have multiple data centers, with 100's of thousands of servers. While this infrastructure is not dedicated to their mapping applications (I'd hesitate to call them GIS's), they can leverage these to provide a level of service that's almost unparalled.

Thus, while I commend ESRI for hosting public globe services, I think it's unrealistic for users expect the same level of data streaming performance from these services that you will get from Google Earth or Virtual Earth.

I just gave Steve's post about how ESRI's fleeting movement towards community based, open communication seems to have been quashed by a segment of the organization who are wed to 90's communication paradigms - slick marketing, throw sheets, and a tall shiny enterprise facade with nary a human in sight.

It is an interesting post, and although I've never been on the "inside", the people I know at ESRI are all fantastic and very dedicated to building great software and systems. This is what makes ESRI more than the sum of it's software. Yet this is sorely lacking in their communication strategy. So, being on the outside, the thing that comes to my mind is that there must be some layer in ESRI which is just not "getting" the whole open discussion = better control of the message. Since Jack seems to have

As Steve notes, open communications can also head off heated threads...

There was not one official word from ESRI anywhere clarifying this new stance on licensing. There was a heated thread going on and we all know that the ESRI folks read blog threads. We never see them officially answer anything there but from my logs and discussions with others, I know they read those kind of threads. Even if they don’t respond on the thread there should be somewhere they can address this issue in a formal manner. There was enough heat there to deserve a little bit of light.

While it's a small step, David Maguire's blog is just too "happy+shiny" to be taken as much more than repackaged marketing. For more on what it takes to have a good "CEO" blog (or any blog for that matter), check Seth Godin's free e-book 'Who's There?' 

Anyhow - to the point of the post - in the comments on Steve's post, Sean Gilles noted that someone posted to the OpenLayers mailing list re: an ArcGIS Server connector. A little digging on at OpenLayers.org, and here's the posting...

I'm working toward creating a layer class for the ArcGIS Server 9.1
MapServer Object web service.  So far I've created the AGS layer class by
extending OpenLayers.Layer.Grid and it seemed that I also needed to create
an AGS tile class by extending OpenLayers.Tile.Image. [continues...]

-- Brian Hatchl on OpenLayers "Dev" Mailing List Nov 29,2006
 

A Call to Action!
If everyone who does not like the ADF licensing model simply contributed 8 hours to this effort, we could have a free, robust, open, alternative that would address the needs of many many organizations. I mean - if I want to edit versioned vector data in a browser, then sure, I'll pay for the ADF because that's the powerful part of it's power. But if I just want to server up a simple map, with simple functions, then OpenLayers would be just fine. Just look at how many mashups were created wit GoogleMaps - clearly pushpins and seamless panning can cover a lot of use cases!  

Anyhow - looks like it's time to signup for the OpenLayers "Dev" list and dust off my Javascript! 

This week I'm attending the ESRI's System Architecture Design Strategies class, taught by Dave Peters. Last summer I posted about working with Dave at one of our client sites, and the class is a real transfer of his knowledge and experience in a very accessible form. If you are in charge of planning your GIS system design, be sure to get to this class.

Some interesting items from todays class:

I had never found the System Integration group's section of the ESRI web site, and they have a whole mess of performance test reports there - all kinds of intersting stuff like ArcGIS Server/ArcIMS performance on 64bit Windows Server, ArcIMS perfomance on Dual Core CPUs and a bunch of papers on high-availability solutions. Very cool if you are thinking about your technology platform.

We also discussed the 9.2 Server licensing situation.

First ArcSDE...
At 9.2 they have put a lot of work into addressing the "chatty" nature of the direct connect, so it's now supposed to be pretty close in performance to running ArcSDE on the server (giomgr.exe processes). So, that's nice. The upside of using direct connect is that you don't have to pay for an ArcSDE license for each socket on your DBMS box. So, this should be a cost savings since there is less load on your DBMS box (no giomgr.exe processes) - which means the same hardware can serve more users. And when you do scale up, there is no licensing hit to ESRI.

The downside is that for direct connect, everything must match in terms of version - so, your 9.2 clients must attach to a 9.2 database. Not bad, but service packs may also be linked - so if you patch the server, you'll need to patch all clients before they can connect again. So - while cheaper out of the gate, the admin complexity may make you think twice.

Which leads to a third option - running the ArcSDE service & giomgr.exe on a totally separate machine. [UPDATE 11/29: Apparently this has been an option for quite some time] Apparently this is a new option with 9.2, and although you have to pay for the sockets, it may be a better mix between SDE on the DBMS, or direct connect.

The ADF...
We also talked about the ADF, and as I noted, the current stance on the ADF is that the server it is deployed on must be licensed for all sockets.

I can't quote anyone on this, but apparently that there is quite a bit of room to negotiate with ESRI in regard to using the ADF to upgrade existing (particularly ArcIMS) applications.

That's it for today - tomorrow we're going to be talking about security & firewalls. This should be very interesting, as it will hilight some of the issues I see with having to have the web controls on the SOM/SOC box (or pay double)

[Update: Once James Fee linked to this blog, the comments started to pile up over at his site. Here's a link to them.]

[DISCLAIMER: I do not work for ESRI, not do I represent any official stance on licensing. This is how I understand things from talking with ESRI. I'm fully willing to ammend this posting if ESRI has substantially changed their policy from what is described here.]

Ok - I'm not the first one to post the "reported" price for ArcGIS Server - someone else did in comments on James Fee's blog. Anyhow, I've been looking around the ESRI ArcGIS Server pages for the list pricing, and have not found anything. So, for this post, lets use $XX,XXX instead of specifics. The actual dollar price is not my point here. Suffice to say that ArcGIS Server is not cheap, but the devil is in the details.

The other thing I've been looking for on-line is the actual licensing breakdown for the ArcGIS Server components. Since the server has multiple components (SDE, SOC, SOM and ADF), I want to know how those pieces are licensed, and how I should expect to configure a system.

In particular I want to know how the pricing changes if you run ArcSDE on a separate machine, and the ADF on a separate web server. I'm not sure about how you do things, but typically when I'm building a web application, I assume it's going to be running on a web server. The geodatabase will be on a separate DBMS box, and (for server) the SOM/SOC is on a dedicated server. This is what we've been instructed to do in the past, so I'd assume this is the model to move forward with.

First, ArcSDE. The deal is that while you "get" ArcSDE with ArcGIS Server, if you want to run the ArcSDE "service", the DBMS and the service must be on the ArcGIS Server box. If it's not on that server, then you must pay full per-socket pricing for the DBMS server. Ok - this is pretty much how the ArcSDE pricing model went. If you choose to use 'direct connect' then you leave the "licenses" on the ArcGIS Server machine, and simply connect to the DBMS server - at no additional cost.

Conclusion: Direct connect is gonna get a whole lot more common.

Next up - the Web ADF. Since the Web ADF has all kinds of cool functionality, I was really interested to see if there was a licensing fee for deploying it. At ArcGIS Server 9.1, you had the ADF Runtime that you could install on the target webserver - there was no separate licinsing for this component. So, I had high hopes. But, what I've come to learn (by talking with ESRI because it's not published anywhere I can find) is that the ADF is licensed at 9.2. And it's not just a little run-time cost. It's the full per-socket price of the server. That's right - if you want to run your web site on a different system from your SOC's, you need to fully licence that machine - at the same level as the SOC's it talks to.

Here are my concerns:
Security. Assuming you got with the all-on-one box model - where are you going put it? On the LAN? Then you need to open ports from the internet onto your LAN - not exacly something your security admin will like doing. If not the LAN, then in the DMZ? How many security admins are gonna let you put your enterprise dbms in the DMZ?? Thus, it would seem to me that for anything other than intranet apps, you're looking at a 2x price bump simply because you need to put the ADF on a separate web server.

In my discussions with ESRI on this, they said that because the ADF now handles the tile cache, it actually off-loads much of the map making work from the SOCs, which means it should carry the same licensing. While it's true that a tile cache saves a lot of work, is true it's not exactly a compelling argument. Creating a tile cache is not magic that can only be done by ESRI. Brian Flood has implemented this in the next version of Arc2Earth (pro edition costs $299). Slap OpenLayers (free) in front of that, and you've got the same thing running for a tiny fraction of the cost. So telling me that I should pony up twice the money for server because of a tile cache is not compelling at all.

So - where are we. If you want to design a system the way you have it running today - you're looking a 3 times the base pricing for server - this will get you ArcSDE running as a service on a separate DBMS (assuming 2 sockets) and the web ADF running on it's own server (also 2 sockets).

Anyhow, I just wanted to post the details as I understand them since ESRI (at this point) has not posted anything related to this, and I know there are a lot of people who are excited about the web ADF, and this is going to come as a big suprise.

 

Just got the notice from ESRI that it's time to register for the Developer Summit!

Last year it was a great event - good technical info in the sessions and a chance to talk to the developmen teams, other bloggers and other developers. I'm planning on going, and brining my project leads along as well.

Last year there were "Birds of a Feather" talks which were informal meetings, setup by attendees, on various topics. Last year I informally led a session on unit testing with ArcObjects, which drew a much larger crowd than I anticipated. (see below - I expected nobody to show up!)

Assuming anyone is interested (and that they have these slots available again), I'll try to get another one setup for unit testing and a second one about code generation and the concept of an object based data access layer for the geodatabase, and using databinding with that.

If this sounds like a good idea (or if it sounds painfully boring) let me know in the comments.

Also - I think I've got the IE problems solved on this site, if I don't please let me know (or use FireFox!)

See you in Palm Springs!

UPDATE: This post got caught up in the tidal wave of blogger spam on PlanetGeospatial, so I am re-posting it with todays date. I'm really interested in hearing what other people think about this. However, in re-posting, dasBlog seems to have misplaced the post - and along with it a comment - doh! For those who subscribe directly to by RSS feed, sorry about the repetition. -- Dave

---

With the upcoming launch of 9.2, I was thinking about what I'd really like to see. While the international marketing extravaganza, with canned demos of simplistic use cases will be interesting, it's not overly illuminating as to what users with "real" needs can do.

Rather I'd like to see some real live examples that people can play with. You would not buy a car without test driving it, so I'm not sure why we would expect any less from our software vendor.

Here are some ideas...

ArcGIS Server: Web ADF

Let's see a real application with large, complex data model - maybe based on the Local Government data model. Show us an editing environment customized for non-GIS pros who don't know or care about versioning. Throw in some imagery from Image Server and make sure access is managed via ASP.NET security. Sounds like a tall order, but consider that Local Governments are a prime target market, and it's not that unreasonable.

From a developer perspective, I'd love to read about how it was built, the best practices which lead to clean, maintainable, testable code. And a glimpse behind the scenes at the hardware that's powering it. This does not need to be totally public - make people login with their ESRI Global Account, or limit it further to EDN subscribers. Whatever - but let's see the beast in action so we know what we can expect.

ArcGIS Server: Geoprocessing

Although the "GP Solves all Problems" rehetoric of last years DevSummit has somewhat died down, it would be cool to see a real GP model - doing more that a trivial buffer operation or a hill shade on a limited area, running in ArcGIS Server, accessible for EDN developers. Get really daring and have it do some raster processing. 

Make the model itself downloadable so we can see how you've built it, talk about how you have optimized it, what hardware it's running on, and what it's really doing.

ArcGIS Server: OGC Services

Fire up some WMS/WFS services that have real data volumes behind them (say the state of California).  Then let the community hit it with whatever client they want to use and report requests per second, data volumes and CPU utilization. If that level of access is a little broad, show us how to limit access via a and restrict it to EDN subscribers.

ArcGIS Server: KML

If you've got the OGC Services up, add on the KML service. Let people see what can be expected with this service. Of course it's not going to be as fast as Google, but be transparent about the hardware, and load, and I think that the user base will follow along.

ArcGIS Explorer "Tasks"

With the launch of ArcGIS Explorer, it would be nice to see some real "tasks" that do something a little heavier than reverse geocoding or returning the Zip+4 for a point. How about a site analysis within an AOI? Or a network trace? Something that's actually using the analytical power of ArcGIS. Something burly because that's what people are going to expect/want when they drop cash for the backend ArcGIS Server software that will support it.

 
Overall, I'd really like to see demos that set realistic expectations. To date, the ArcGIS Server 9.2 hype has been reaching snake-oil levels:

• Publish maps by merely thinking about it!
• Shed pounds by sleeping!
• Code? Nah - use geoprocessing for everything!
• Cures hemhroids & rhumatism!
• Run ArcGIS Server, ArcSDE and your web site all on one box!

Some reality will help everyone manage expectations.

With the upcoming launch of 9.2, I was thinking about what I'd really like to see. While the international marketing extravaganza, with canned demos of simplistic use cases will be interesting, it's not overly illuminating as to what users with "real" needs can do.

Rather I'd like to see some real live examples that people can play with. You would not buy a car without test driving it, so I'm not sure why we would expect any less from our software vendor.

Here are some ideas...

ArcGIS Server: Web ADF

Let's see a real application with large, complex data model - maybe based on the Local Government data model. Show us an editing environment customized for non-GIS pros who don't know or care about versioning. Throw in some imagery from Image Server and make sure access is managed via ASP.NET security. Sounds like a tall order, but consider that Local Governments are a prime target market, and it's not that unreasonable.

From a developer perspective, I'd love to read about how it was built, the best practices which lead to clean, maintainable, testable code. And a glimpse behind the scenes at the hardware that's powering it. This does not need to be totally public - make people login with their ESRI Global Account, or limit it further to EDN subscribers. Whatever - but let's see the beast in action so we know what we can expect.

ArcGIS Server: Geoprocessing

Although the "GP Solves all Problems" rehetoric of last years DevSummit has somewhat died down, it would be cool to see a real GP model - doing more that a trivial buffer operation or a hill shade on a limited area, running in ArcGIS Server, accessible for EDN developers. Get really daring and have it do some raster processing. 

Make the model itself downloadable so we can see how you've built it, talk about how you have optimized it, what hardware it's running on, and what it's really doing.

ArcGIS Server: OGC Services

Fire up some WMS/WFS services that have real data volumes behind them (say the state of California).  Then let the community hit it with whatever client they want to use and report requests per second, data volumes and CPU utilization. If that level of access is a little broad, show us how to limit access via a and restrict it to EDN subscribers.

ArcGIS Server: KML

If you've got the OGC Services up, add on the KML service. Let people see what can be expected with this service. Of course it's not going to be as fast as Google, but be transparent about the hardware, and load, and I think that the user base will follow along.

ArcGIS Explorer "Tasks"

With the launch of ArcGIS Explorer, it would be nice to see some real "tasks" that do something a little heavier than reverse geocoding or returning the Zip+4 for a point. How about a site analysis within an AOI? Or a network trace? Something that's actually using the analytical power of ArcGIS. Something burly because that's what people are going to expect/want when they drop cash for the backend ArcGIS Server software that will support it.

 
Overall, I'd really like to see demos that set realistic expectations. To date, the ArcGIS Server 9.2 hype has been reaching snake-oil levels:

• Publish maps by merely thinking about it!
• Shed pounds by sleeping!
• Code? Nah - use geoprocessing for everything!
• Cures hemhroids & rhumatism!
• Run ArcGIS Server, ArcSDE and your web site all on one box!

Some reality will help everyone manage expectations.

Earlier this week I had the pleasure of working with Dave Peters - ESRI's System Architecture and Design guru. Many of you have likely read (or skimmed) the
System Design Strategies document which Dave authors, and been excited and/ or overwhelmed with the volume of information. We had Dave join me on site at  the Pennsylvania Bureau of Forestry to help plan for the implementation of an enterprise system we are building for the state.

System Design Planning
As part of these meetings, Dave showed us a preview of his new Capacity Planning tool. This tool takes all the logic and models from his testing and puts it into a very usable spreadsheet (which may become a .NET app). This tool allows you to quickly plug in the number of various types of users (local desktop, terminal server, ArcIMS, ArcGIS Server etc), and then calculates the expected load based on benchmark testing. It calculates both the server CPU load and the network load (megabits per second, per user). The network load is an aspect that I had not considered, but it makes sense: adding a faster ArcSDE server will not make a difference if you're network has no extra capacity!

Once the load is determined, it uses data imported from Spec.org (hardware performance testing group) which allows you to specify particular hardware configurations on which to run the system, and then it show the expected maximum load on that hardware. If you underspec the hardware - as in the max load is > 100% of the capacity - it lets you know. The simplification of this information, and the integration of the Spec data makes informed capacity planning a possibility for many more organizations (those without a full time hardware performance guru).

Upgrade Planning
What's really cool is that once the system is in place, and you are collecting actual metrics (as opposed to the model driven default values) you can adjust the parameters in the spreadsheet, making it more accurate. Then, as your user load or needs change, you can go back to this tool, load in new Spec.org infor, and make informed decisions re: the actual improvements you'll see by upgrading hardware or network capacity.

It was really great to have a one-on-one walk-through of all of this, followed up with his presentation to the whole group. Of course not everyone is going to get this opportunity. The good news is that Dave is going to present this tool and his latest performance benchmarking next week at the User Conference.

So - if you are in any way responsible for your GIS hardware or network, I highly recommend attending his talks.

ArcGIS System Administration: Choosing the Right Architecture
Wed Aug 9, 1:30 - 2:45pm - Cardiff Room, Marriott
Thu Aug 10, 3:15 - 4:30pm - Cardiff Room, Marriott

Enterprise GIS: Design & System Configuration Strategies
Wed Aug 9, 3:15 - 4:30pm - Room 15A

Jeff Germain and I have been doing some cool work with the geodatabase, and I'm going to summarize our work over a series of postings. This post introduces the key ideas...

Standard "Business Objects"
In keeping with object-oriented design, most applications are now utilizing classes (business objects) to represent the real-world entities of concern to the application. When it comes time to store these entities in a database, the data access operations are typically bundled into the business objects themselves, or in associated helper classes. The advantages of this model are many, including:

• cleaner code design
• easier maintenance
• encapsulation of data storage logic
• single point for database schema changes
• with .NET 2.0, object data binding is possible (very cool)
  

There are many design options which take this model to various levels of complexity. You can find all sorts of information about Object Relational Mapping at ORMappers.com

Geo "Business Objects"
Since this is such a nice design pattern, we wanted apply the same logic to the geodatabase. Afterall, you design classes when you design a geodatabase, but the geodatabase API is cumbersome as compared to working with standard business objects. 


To be clear - we're not talking about replacing IFeature and IRow, after all, you still need to talk to the geodatabase, and that's how ArcObjects works. But, in order to avoid working with the geodatabase API all over our code, we designed wrapper classes that hide the ArcObjects code behind a nice "business object" facade.

Some might ask - why would you go to all this trouble? The first part is that for large enterprise systems, where they may be 10 or more associated attribute tables related to one feature class, being able to work with a set of business objects greatly simplifies creation of attribute editing forms. And, if designed correctly, these objects could be used in Desktop, Engine and Server applications. The second part is that once you set it up, it's not much work. Which brings us to...

Code Generation
A nice thing about business objects is that they follow a pattern. And computers can do great things with patterns - like write the code for us. This is what code generation tools bring to the table. A typical code generation tool connects to the database, reads the schema, and creates classes based on code templates. This process takes a few seconds, and can be re-run at any time, and many teams add this as a step in their build process. Depending on the templates used, there are options where the developer can add custom logic which is not over-written when the code is re-generated (via inheritance or partial classes). As you may have guessed, this results in massive time savings when writing applications.


For example, I needed to create an application to edit the tables in a standard relational database (Process table shown above). Using RapTier, I was able to create a complete data access layer in a few minutes. Once that part was done, I was able to very rapidly wire the business objects into an interface, and the application was complete. The image below shows the classes created for the "Process" table. You'll notice that these objects do not have "Get", "Save" or "Update" methods. In the abstration model used by RapTier, all these functions for all the classes in the database are in a single helper class.


But this is a Geodatabase...
So, this works well for traditional RDBMS tables, but it's not going to fly for the geodatabase. First off, you can't infer all the details of a geodatabase model directly from database schema itself. Specifically, you'd miss thinks like relationships, domains and subtypes. Additionally, if you are working aginst ArcSDE you'll get all those pesky A, D, F, and S tables. Finally, the code that would be generated would use standard SQL (or stored procs) to insert and update the data. So, clearly we had some issues to work out.

Our Process
We've got two large projects on the go right now, and both are going to be subject to many geodatabase schema changes over their life span. To that end, we are working on a process where by we are able to generate business objects (and class extensions) directly from a geodatabase. The diagram below shows how this works, but the high-level steps are as follows:

1. Export the geodatabase schema as Xml
2. Transform that into a simplified Xml
3. Use this simple Xml to generate the code
   
4. Use object data binding in .NET 2.0 to simplify the form wiring
5. Repeat when the schema changes
   

In the next installment, I'll review the process, the business object templates we've created and tools we've used to so this.

Just catching up on my blog reading, and read some posts about Web 2.0 and "Labs" by Adena at AllPoints and James Fee. One thing I think they both miss is what makes a "Labs" effort successful: customer feedback.

It's all about the money...
The bottom line is that companies are trying to sell something - in some cases it's software (ESRI, Microsoft), in others it's the advertising that wraps around the software (Google, Yahoo). In order to have people want the software it must be usable. The best way to detemine the REAL usability of something is to have people use it, and provide feedback. (While this should be obvious, it took the industry a long time to learn this).

Release Early and Often (to the Lab)
This feeds directly into a major concept in both Agile development and Web 2.0: "release early and, release often." What's implied in that statement is that between releases, there is a learning period where the company changes the software to address usability issues, and incrementally move the software forward. This is what makes a "Labs" effort successful. The result is a collaboration between the company and the clients - which is a whole lot more "2.0" than a monolithic release followed by a few years of vaccum, possibly punctuated by a service pack or two.

What about Beta Programs?
Labs should be different from a beta program. A beta program is basically a way to expand your testing team. The beta participants do not get to help set the direction of the software, rather they try the software in existing workflows to locate bugs which may not have been found in the standard regression testing. Software released into the lab, while possibly incomplete, should not be "beta" quality. The idea is to have the working part "work" and to get input on improvments and additional functions. The goal is to have your customers help refine your vision of what they want. This is really helpful unless you have a team of psychics who already "know" what the users want. (note: regardless of what they say, the marketing team is not psychic.)

ESRI Labs (beyond ArcWebServices)
Granted they have the ArcWebServices Labs, but it's pretty small potatoes in the whole Arc spectrum. What would really turn things on their head would be a Labs area relating to the core software. This would be very interesting, and would involve a major shift in the level of transparency. Right now, it's pretty reasonable to describe the company as an "Ivory Tower". Once (or twice) a year there is the week-long love fest, where it's all about the customer and everything is on the table. And then it's back to Redlands and all we see if the "Wall of Marketing" for the rest of the year. Sure they have publications, and trade shows, and betas, but all this is either driven by marketing, or weighed down by a giant NDA. There are some brave souls who do blog, but those few voices in the wilderness are a far cry from transparency. The recent effort towards having a blog seems stilted - it's unclear if this is a group blog, a marketing effort, or (shudder) user-generated-content. what. And thus far, it's pretty fluffy for  the 800lb gorilla of the spatial industry. Back to the Labs thing...

Until ESRI can have a consistent on-going real conversation with their customers, I skeptical they can embrace a real "Lab", where rapid customer feedback cycles drive the direction of the software.

After my previous post, I got an email from a reader internal to ESRI (since it was an email and not a comment, I'll allow them to remain anonymous) pointing me to two new (April 10th) techincal articles which address lifetime managment.

Problem:  Microsoft .NET Framework applications explicitly manage references to remote COM objects

FAQ:  Do I need to explicitly manage references to remote COM objects in a Microsoft .NET Framework application? (Answer is now No.)

If you apply the referenced patches from Microsoft (they are included in XP Sp2), you no longer need to explicitly manage the lifetime of these objects.

For all my digging on the ESRI site, one thing that led me astray was reading the "Managing Server Resources in ArcGIS Server .NET Applications" white paper (October 2005), which desribes using the IDisposable pattern and WebObject.ManageLifetime. Since this was a "whitepaper", I assumed it was the last word on this, and did not look further. Hopefully they can get this new information into this document or at least it's description.

Anyhow - thanks for pointing me in the right direction. Since I am running XP Sp2, it does explain why adding or removing WebObj.ManageLifetime made no apparent difference.

I wanted to give a shout out to the entire proposal team at my company, as we have finally signed the contract to build the Pennsylvania Enterprise Forest Managment system. I'm very excited about this award, as I was very involved in the technical aspects of the proposal, and led the technical presentation to the state (a rather harrowing experience). I'll also be leading the system design and development from our Fort Collins office. 

Since this is a 3 year contract, we'll be building in .NET 2.0, using ArcGIS  9.2 and using a mix of Desktop/Engine (via Citrix) for complex geometry editing, and Server/IMS for most other functionality. We'll see if we can sneak the Mobile ADF into the mix as well

Like the previous enterprise system's we've built, we'll be implementing this as set of tools/applications which work against a "secured" geodatabase. By secure, I mean a set of column level editing permissions, which matrix into a set of workflow permissions, using Active Directory to manage identity and access and implemented with Feature Class extensions. This allows us to implement complex, multi-stakeholder, multi-step business processes, while still enforcing all the required business logic - and we do it at the geodatabase level so it can not be circumvented. To make things more interesting, this time we'll need to make this work in both a desktop and ArcGIS Server envionment.

Sound interesting? Send me your resume! - we're looking for developers and a business analyst.

We're just starting up the project, and will be doing some system level architecture up front, and then jumping into the functional modules. This will be my main focus for the next few years, so there will be plenty of postings on the technical aspects of how we are building this system.

Just a heads up to anyone else thinking of running IE7 Beta 2 - it will break ArcToolbox. I had not connected the dots on this, but ESRI Tech support mentioned this as a possible cause of my problem, and they were right.

The symptom is that when you try to open an GP Task by double clicking on in - either in a model, or in ToolBox, ArcMap will crash. No useful message, just a "Please report this to Microsoft" pop-up box. As soon as you un-install IE7, it works just fine.

This is a pain because having tabs in IE made working with our Team System portals much nicer than having a bunch IE sessions open. I have not tried running IE7 without installing it, as J noted on his blog next week. It would be cool if this fixes the problem.

This edition of the Denver Developer User Group was much better, and not just because I gave a breif chat about ArcDeveloper.net & ArcUnit!

Things started off with a quick round of introductions - name, affiliation, what you are doing with ArcGIS, and I was quite suprised at how many groups were seriously looking at switching a lot of GIS functionality over to ArcGIS Server. Very cool!

The ESRI Denver guys, (Chris, Tom, and Jeremiah) did a session on setting up and installing ArcGIS server. Now this seems like a somewhat dull topic until you actually try to install it. Then you realize that, yes, you do need to think about what's going on more than any other ESRI product. They even had a common "gotcha" occur.

Tom (running the demo) had added his domain account to the local "ArcGIS Server Admin" group, and then tried to connect to ArcGIS Server via Catalog, and it threw an error stating he did not have permission to access the resource. After checking that the service was running, someone in the audience reminded him that he needed to log out and log in before his account would "know" about the group change. This is exactly the sort of thing that can make installing and configuring AGS a confusing process. Unless you are installing on a development box, I'd highly suggest having a little Active Directory / Windows managment under your belt, or involve a system admin.

After than, Bryan Dickerson of Woolpert gave a review of a number of ArcGIS Server projects that they have done. Bryan is a system & software architect, so it was a good - non-marketing - talk about how all the parts work together. He concentrated a fair bit on how to use the right software in the right place - when to use ArcGIS Server vs ArcIMS, and how to blend them.

I believe his presentation, along with my few slides, and Tom/Jeremiah's Installing ArcGIS Server slide decks will be on the ESRI Denver site in the next day or so.

Also of note - if you're in the Denver area, and want to present at one of these, Chris Longo in the Denver office is the man to talk to.

Tomorrow (April 6,2006), ESRI is holding another Developer User Group meeting at the Denver office. Last time I was not overly impressed by the content presented, but I think things will be much better this time.

Skill Level: Beginner | Intermediate | Advanced This Developer

User Group will focus on ArcGIS Server in the .Net environment. We will start by taking a brief look at the ArcGIS Server architecture followed by user’s case studies. These will be highlighting specific projects and explore success and pitfalls in the development process. We will conclude the session with a Q & A along with various .Net and ArcGIS Server users.

Additionally, I'll be giving a short (5-10 minute) intro to ArcDeveloper.net, the ArcUnit project, and throw out the idea of a Northern Colorado ESRI Developer User Group to see what the level of interest is.

If you're in the Fort Collins area, and would like to see a user group up here, let us know - go to ArcDeveloper.net and post in the "ArcDeveloper.net forum".

I'll post tomorrow with a run-down of how it all went.