So my company is sending me out to Where 2.0 - I've never been and expect it will be quite different from many of the other GIS conferences I've been to - more MacBooks and soul patches I'm guessing.

I'm particularly excited about the pre-conference workshops, as it will be a chance to get the low down on some technologies I have not worked with in the past.  I'm hitting the Geo-ify your Web Site session in the morning (Mapstraction) and the The Open Geo-Stack: OpenLayers, GeoServer, PostGIS session in the afternoon.

In prep for the workshops I thought I'd get PostGIS (with ArcSDE 9.3 beta) installed on my notebook - ideally I'd like to roll into the session with a little background so I can ask relevant questions!

Installing Postgresql + PostGIS + ArcSDE 9.3 (beta)

Overall, not exactly the smoothest installation I've ever done. No step was particularly difficult, there were just points where I had to do some head scratching, Googling and then "just do it". I have to give ESRI credit as the ArcSDE installer was smooth as silk, and they included Postgresql 8.2 on their media. However, having not used PostGIS before, exactly how this fits into the mix was a little unclear. I will say that if you have used ArcSDE in the past, and just want to get rolling without paying for a database - the install is really pretty easy.

As noted, I installed PostgreSql 8.2 directly from the 9.3 beta media - this is super simple. I then downloaded the PostGIS Installer for PostgreSql 8.2. Again this was pretty easy, after you locate and download it. As per the ArcSDE instructions, I then ran the ESRI post installer, authorized ArcSDE and all looked good.

I could immediately connect to the sde database as the sde user, and used ArcCatalog to load a shapefile into PostGIS. Smooth.

But how do I know that this is actually using "PostGIS"? Would this data be accessible to non-ESRI clients?

Enter uDig. This was a quick and painless install, and uDig was installed.

When I browsed to my local PostGIS server, things were not looking good.

Where was my data? Clearly something is not lining up here. When I tried to connect to ArcSDE via uDig (after I located the jar files) it could not connect. Not sure if this is related to using the beta 9.3 jar files, or something else.

Anyhow, I decided I should try loading the same data via the command line post gis tools. Here I ran into a bit of a wall. The documentation for the PostGIS tools is not bad, but certainly aimed at the unix sys admin type of person. The basic process is to convert your shapefile into a Sql file, and then run that in Postgresql. Seems simple. A quick little batch file ought to work this right out. The sql file was created just fine, but it failed to load.

Sorting this out really meant it was time to do a little learning about Postgresql. After sorting out schemas, and learning that the most likely encoding of my shapefile was "LATIN1" I was able to load my data and "see" it in uDig.

Actual loading the layer into uDig never completed. The application just sort of hung stating that the "Project command" was executing. This was just a map of all the counties in the US, so not sure what the issue was, but I'm pretty sure it was my fault.

I then tried to connect via ArcCatalog to see if I could "see" this layer... and not too surprisingly I could not.

Summary

Thus far I've simply come to realize that I have a lot to learn. I need to grok a lot more about Postgresql and PostGIS to start, and then add ArcSDE into the mix. I'm sure that if I read more of the ESRI documentation I'll find some answers, but at this point it's unclear to me if they will be supporting direct access to PostGIS data, or if they will be supporting the PostGIS spatial format for data stored in traditional ArcSDE schemas (i.e. with all the GDB_ and SDE_ tables). I also learned that I've been sheltered from the reality of "data" for many years!

I'll continue this series as I make progress getting things sorted out.

...from Jeff Atwood author of the (wildly popular) Coding Horror blog. Hopefully you all subscribe to his blog, but for those who may not, it's full of goodness, and you should check it out on a regular basis.

As for MVC - the results of the Geospatial Developer Survey show only about 50% of respondents are using patterns, and about 46% were not using any sort of Model View * patterns. This article is a great entry point to a pattern that can be applied to a very wide range of scenarios (including easing the creation of unit tests!)

Check it out at CodingHorror.com

Just a quick reminder that the Geospatial Developer Survey will be closing this Friday, May 9th.

We're at 262 respondents who are describing themselves like this:

 

 

 

Hey this is all .NET Stuff!

I've gotten a few comments back along these lines, so I thought I'd try to clarify this a little. Yes, on questions where you get to select a particular implementation, most of the options will be .NET, with one exception - "Other". I did not do this to slight other platforms, or indicate that .NET is somehow superior.  I live in .NET land, and I simply do not know a lot about non-.NET technologies. If you are doing automated builds with ant and maven, just pick Other and write it in (15 of you already have). I'll add these into the mix when crunching the numbers.

 

When will we see the results?

Survey Monkey does a great job of crunching out some simple results, and I expect to post a few interesting things during the week of the 19th.

Seems that I have not been reading my own blog on the site, and apparently the CSS was more than a little jacked up. If your read this @ the site, you may notice that it's now a fluid layout, and hopefully the font sizing makes a little more sense.

Anyhow - I'm also planning on updating my actual website (It's on the long list of "stuff" to do while out of the office and not a) hanging with my family, b) cooking or c) riding).  I've gotten quite a bit of feedback on the fundamentals section, and I've got some ideas on other topics that could be covered there. We'll see if I can get to that over the next few months.

Recently I needed to parse a complex query string from a Url, and configure a map based on those settings. This parsing occurs in the server side Page_Load event, and the general process of parsing the Querystring NameValueCollection is straight forward.

The complex part comes from the fact that my application is heavily Ajax based - other than the initial markup, everything else is fetched from the server via HTTP GET's to ASP.NET JSON web services. Still sounds good. But - here's the trick. The query string is passed to the page, not the web service. I guess I could have tried to write all the parsing in javascript and handled it on the client side but there are some pretty complex options that involve database lookups, and I like me some MBUnit Row Tests, so server side was a better option.

Just use Session State...

The first though is - ok - I'll parse the query string, and stuff the resultant "map settings" object into session, and have the client fetch that from the web service. Nice, but no cigar, since the web service can not access the HttpContext that the page is running in. So much for simple solutions.

ASP.NET Page Methods To the Rescue (or not!)

Next contender was ASP.NET "page methods" (aka web methods). These are static methods They should work, as they can access the HttpContext of the page. After parsing and stuffing the "map settings" into the session state, the client side code could call back to the "Page Method", which would grab the map settings from the session and send back to the client. Sweet. After some head scratching and reviewing of sample code that does this, and a bunch of Googling, I found that Page Methods do not work on User Controls (ascx). Since I was creating a module for use in DotNetNuke, there was no way around using a user control. Time to get ugly...

 

Serialize to Json and Cram into the Page...

Yep - the old inject-it-as-it-renders trick. Ugly, but it works. So, in page load, I parse query string, create and populate the map settings class and then use the System.Web.Script.Serialization.JavascriptSerializer  to convert my object to JSON. Some people have reported issues with this serializer, but it's been working smoothly for us so far. If you have problems, you can roll your own or use an open source option like JRock or JSON.Net. From there, I just emit it into the page as part of a script block:

As alluded to the in the comments, I ran into all kinds of trouble with dates. Read the referenced forum article for the "why" behind the string replacement bit.

Then in the client side Page_Load (I'm using the MSAjax stuff, so we get a Page_Load that automatically linked into the client page lifecycle), we use the client side deserializer to convert our JSON string back into a handy dandy javascript object.

//Deserialize back into an object
mapSettings = Sys.Serialization.JavaScriptSerializer.deserialize(mapSettingsJson);

From here we just proceed with the rest of the page startup, and we get the correct map.

 

Note for .NET 3.5:

If you have .NET 3.5 installed, then JavaScriptSerializer is marked as deprecated, and suggests you use the DataContractJsonSerializer. And if you are using Complex types (like I am) you will need to add the [DataContract] and [DataMember] attributes to the classes being serialized. I tried both ways and for me they both worked.

Just wrapping up a project that's built on DotNetNuke, and was using the built in module packager to create a nice zip installation. Great. Only problem is that it packs up all my Subversion "_svn" folders as well as the actual files I want. Doh! Initially I just exported from my Subversion working copy to a new location, added that as a virtual directory in IIS, and used that instance to create the package. But this is a real pain when you need to release small patches and bug fixes, so this being open source and all, I dove into the code.

And there is a simple fix... assuming you have the entire source tree laying around ( you can download it from www.dotnetnuke.com if you don't have it)

The module packager is the PaWriter class, which is in the DotNetNuke.Library project, under the Components/Modules path.

All you need to do is add two snippets of code to the two ParseFolder private methods:

Change:

to:

Once you've made this change, re-build the DotNetNuke.Library, and copy the DLL into the bin folder of your DNN instance (if you are working directly in the full source tree, just build the website).

Given that Subversion is so very widely used, it would be nice if the DNN team would add this (and .svn) to the "core" code for the packager.

I was doing some work with a database driven version of the Virtual Earth Tile Service HTTPHandler, and needed to do some debugging. I ran into some odd behavior which took some time to track down, so hopefully this saves someone else some time.

Background

To use a HttpHandler, you need to register it in your web.config. Here's what I had :

<system.web>
      <httpHandlers>
        <add verb="*" path="*.ve" type="DBTileHandler" />
      </httpHandlers>
    </system.web>

This just simply routes requests for anyoldthing.ve to the DBTileHandler class, which must implement IHTTPHandler. Very simple.

Debugging Handlers

Typically I open websites "from file" in Visual Studio as this seems to work a little more consistently with VisualSVN (I think this is some issue with my box, as others on my team have no problems opening via IIS). As a result, when I debug the site, it spins up the ASP.NET Development Webserver (aka "Cassini"), and binds to a random port (e.g 2323). Then the site launches on http://localhost:2323/site. All is well and you can step through your aspx pages.

However, you can't debug http handlers by just starting up the site from Visual Studio. You need to change the Start Action of the project to "Don't open a page. Wait for a request from an external application" as shown below.

Then once you start the web site, and it builds, you need to use Debug --> Attache to Process, and select aspnet_wp.exe.

At this point you can make requests to your handler, and breakpoints will be hit - curiously you attach to the aspnet_wp.exe process even if you are running the site through Cassini.

So, I was happily debugging my tile handler, and got everything worked out, tile were showing up in my VE Map and all was well in the world. Then I went to try it through IIS. Doh! No tiles!

Firebug showed me that the requests were correct, but they were returning with 404's

I went back and fired it up in Cassini, and got tiles. IIS - no tiles. Cassini tiles. IIS no tiles.

This was quite odd - why would my handler work in one case and not the other? Clearly there is something different between the web servers but what?

To make a long dull story involving much Googling shorter, the answer is that Cassini automatically maps file extensions specified in the Http Handlers section of the web.config, but IIS requires that the file extension mapping be applied explicitly in IIS. Obviously.

Solution

There are two ways to solve this issue:

1) Add the .ve extension to IIS and map it to the aspnet_isapi.dll

or

2) change the extension in your configuration to a type already handled by ASP.NET - such as ashx or aspx.

I chose the latter because it will simplify the deployment of the site. All I did was tack ".aspx" on the end of the path attribute in my web.config and all was working again.

<system.web>
      <httpHandlers>
        <add verb="*" path="*.ve.aspx" type="DBTileHandler" />
      </httpHandlers>
    </system.web>

The last two days we've been setting up a demo integrating the National Geographic MetaLens service with Virtual Earth. It's now live and can be seen at: http://www.metalens.org/mlve/.

MetaLens is a geospatial content management and archival system that National Geographic uses to secure and manage it's content. "Out of the box" it has a Flash interface, but it also has a nice REST interface. We built some server side code that requests the public assets in the current view, and then applies clustering. At this time, the site is only fetching images, but we expect to extend the demo to include audio and video as well.

Map Tools

What's a map without some tools? The Zoom To tool allows you to hop over to a location where there is some data of interest.

For Lake Tahoe, we have added in a Parcels layer that comes from an ArcDeveloper.net Tile Service.

For Vermont, we have a map from National Geographic being tiled using the same code base.

You can toggle these layers on and off using the Map Data command on the toolbar. We have also added a Base Map drop down that allows you to select which of the VE base layers to view.

 

What's on the Server Side

The main app is a ASP.NET with a data provider class library that actually talks back to the Metalens service. I really like this pattern because it allows us to test the provide completely independently of the web site itself. Basically we just wire up some unit tests, and then via TestDriven.net they are automagically re-run when we build that class library. Sweetness.

MetaLens

MetaLens is built on a product called Clear Path Explorer (CPX) built by a local Fort Collins company called Clear Path Labs. Their web site talks more about CPX, and it's API. From out brief experience working with it, it's pretty nice.

Tile Services

Not surprisingly we are leveraging the ArcDeveloper.net Tile Service. At this time we are just pulling in 2 services, and both use the ArcGIS Server SOAP tile provider. The only wrinkle is that one service is hosted by DTS in our Fort Collins office, and the other is on a National Geographic server.

I have to give a big shout out to my colleague Mike Juniper who did an awesome job tracking down a bunch of CSS weirdness, and put the final polish on the site. We knocked this out in about 2 days start to live. Amazing what you can accomplish these days! There are still a few strange things in IE6, but that's pretty much how it is with that browser. Works great in IE7 and FireFox.

Over the years I've seen a few "developer" surveys put out by various GIS organizations, and they have all been pretty bland. After talking to a few people at the Developer Summit about this, I thought I'd see if we could do a little better - thus the launch of the 2008 Geospatial Developer Survey.

Why are you doing this?

The main reason is that I'm interested in knowing more about how other people doing GIS/Geospatial development work. How many people view themselves as software engineers vs. occasional coders who hack some stuff together as needed. I want to know if people are exploring and adopting the tools, patterns and practices that are common place in main stream software development. And if not, why.

And beyond my curiosity, I'm also looking for some insight as to what topics people want to know more about. Although I have some ideas, lets hear from you. I get a fair number of requests to write more "fundamentals" type posts, and at the same time get requests for more on patterns, Inversion of Control and Unit testing. Since I don't write this blog for a living, I need to focus my efforts, so I hope that this survey will help quantify which way I should lean in future postings.

Who should take the survey?

Anyone who writes any code that is geospatial in nature. From the Google Maps Mashup dudes, to those running some geoprocessing tasks with python. If you write code that has anything to do with a map, you're the perfect person to take the survey. Be sure to pass this url around to everyone you work with who is a geospatial developer - the more data points we get, the better. Who knows - we may even be able to influence the content at some conferences...

How?

It's simple - just follow this link to SurveyMonkey, and fill out the questions. Yes there are 30 questions, but they should be pretty quick - I'd guess no more than 10 minutes.

What Next?

I'll be leaving they survey open until May 9th 2008. Then I'll crunch the numbers and do a post or two about the results.

Link (for copy pasting to all your developer buddies):

http://www.surveymonkey.com/s.aspx?sm=cugzupf5rAG_2bIu5WJ_2f7rxQ_3d_3d

UPDATE 4/8 @ Noon: After recieveing some feedback on the questions, I have added more "Not Applicable" and "Other" options.

I did a podcast interview while out at the Developer Summit. It's focused on the ArcGIS Server Tile Server project that I submitted to the code challenge. I give a high-level overview of the project, and talk a little about my ideas on designing for ArcGIS Server - big ups for the SOAP API, and the 9.3 REST API.

You can check it out here.