While at Where 2.0, Chris Spagnuolo and I recorded a podcast with the Very Spatial crew.

In this podcast we talk (at a high-level) about internals of the Global Avian Influenza mapping application we built for the Wildlife Conservation Society, and some of the complexities and how we overcame them (think 1-M-M-M database relationships, with complex query criteria and 300ms response times).

For those that know us, you won't be suprised that we also talked about Agile - how it fits in with the Where 2.0 crowd, and how we work Agile into our contracting. All -in - all, a fun experience.

The podcast went live today. Thanks Jesse & Sue!

Back in November I was asked to write an article on Agile Practices for "GIS Development" magazine. For those who are not familiar this publication, it's based in India, and is quite similar to GeoWorld. I quickly recruited Chris Spagnuolo and we whipped up a 5 page article covering the high-points of agile development and some of the tools used to enable the process.

A week or so ago we got hard copies (always cool if you don't publish a lot of stuff!), and I was just notified that the article is now online.

Article: Agile Project Management for GIS

Do you have solid ASP.NET, Ajax, Javascript, and CSS skills? Are you a web development guru? If you are, this might be your workstation. Data Transfer Solutions is hiring in our Fort Collins Colorado office, and we need someone who can do some serious web development. We’re an agile development shop and we’re looking for a web developer that will fit well with our existing team. Experience with agile development practices and geographic information systems are a plus but are not a prerequisite. If you’re a web developer and you’re looking for a new place to sit, drop me an email using the Contact link above.

Regardless of how you manage your software development, at the end of the day it's really all about communication. Communications in a software development project are typically chain-like. The User tells an analyst what they want, analyst translates into analyst-speak and writes this down. This is passed to a system architect who re-interprets the document, and creates a design and a document in architect-speak. The requirements and design are then passed to the developers, who re-interpret both and then implement their understanding of what the user wants in code. This sounds well and good, but it only works if extreme care is taken at every link in the chain. Recall the game of "telephone"...

Although typically a game for grade school kids, "telephone" showed us early on that communication chains are fraught with peril. If the message started out "Today's forecast is calls for snow", at the end of the circle, it would be "Tookeys feet are slow". This was quite funny back then, but we seem to lose this valuable lession as we age. We see this same thing can happen with software requirements - and then it's not quite as funny. (Besides - how can I code slow feet?)

Agile seeks to reduce the bulk of the "traditional chain" by having the team (developers + designer + analyst) collaborate with the end users to create a workable shared vision of the product. The key thing here is ensuring that the collaboration actually occurs. Too often the users are busy doing their "real" jobs, and the team is "pretty sure" they know what to build. But if the collaboration is short changed, then one of the major benefits of agile falls out.

Recently our team finished just such a sprint. The client was very busy, and the team was pretty sure we knew what they needed. During the sprint review with the client, some of the functions we had coded missed the mark. When we went back and reviewed the requirements, what we had developed seemed a reasonable solution. The lack of collaboration had bit us. We did not have the shared vision - just a requirements document that was written by a different consultant.

Luckily we are doing two-week sprints, so we were only a little off course, and should have no trouble getting things back on track. To address this issue, we have decided to adapt our Sprint Planning meeting to include a more formal review of our understanding of what each user story entails. Prior to this meeting the team will review the top priority user stories, the associated requirements, and the existing application, and determine what they believe the "shared vision" should be. Then at the meeting we will review this with the users and they can correct or add anything we had missed.  I'll let you know how this works out.

In early January Chris Spagnuolo and I recorded a podcast with Adena Schutzberg of Directions Magazine. The topic was "An Introduction to Agile".

One of the fun things about recording a podcast is that it's very "off-the cuff". The title of this post comes from my comparison of "waterfall" with ordering of a hamburger at a fast food joint. Give it a listen and let me know if you think this analogy works. 

Around the same timeframe we did the podcast, Chris did an interview with Ron Exler at theGeoFactor.com - his article titled "CanGIS Be Agile?" went up on January 15th.

Just before leaving for Christmas, Chris Spagnuolo and I did an interview with TechTarget.com as part of an article titled "Agile Tools for Agile Development".  Jack Vaughn was interested in talking to us about our Agile practices and the tools we use for both development and management. Beyond our musings about agile and the tools we use, the article gives a good overview of the roots of agile, and some of the tools that are available.

Check it out at TechTarget.com

On January 17th, I'll be giving a talk at Denver University on using Agile methodologies to manage GIS projects. Dr Joseph K. Berry asked me to talk as part of the Special Topics in GIS series of lectures.  Although this is part of a course, the public is also welcome to attend. I will also be posting the slide deck.

The talk will focus on my team's implementation of Scrum on an enterprise GIS project. I'll cover the basics of Scrum, the benefits of agile management, as well as the available tools that help streamline the agile developer's life. I will also address the end-user benefits of using an agile process.

If you are in the Denver area and interested in attending, here's the course URL: http://www.innovativegis.com/basis/courses/GM_topics08

Many people jokingly refer to the "cult" of agile. But the reality is that to outsiders, many Agile teams, particularly those doing Scum, seem to have the messianic zeal of cult members. I think this is just fine, and really no different than the zeal you see from Ruby on Rails converts. These people are simply excited about having found something that really works for them, and they want to tell you about it. This becomes problematic when a dogma emerges - where you start to see semantic arguments which boil down to "I'm more agile than you because X."

What I've seen this week at the Agile Development Methodologies conference is an underlying theme which more or less boils down to:

Agile = good, Dogmatic Agile = bad

Although this seems obvious, it's the sort of thing that's worth repeating because we are starting to see more people engage in "Are you Agile^[tm] Enough" debates on blogs and forums. This is dogma rearing it's ugly head. If you happen across these threads, ignore them - they are irrelevant at best and detrimental at worst. Quite simply they are missing the point. The successful groups realize that silver bullets do not exist - agile or otherwise. There simply is no methodology which works for all teams in all situations. The key is to apply the agile concepts to your situation. Try some techniques - keep what works, drop what does not. Inspect and Adapt. But just be sure to leave the dogma out of it!

I'm super excited about attending the Agile Development Practices Conference in Orlando on December 5th & 6th.

In addition to showing a serious commitment to agile practices and software quality on the part of DTS, what I really think is cool about this event is that it will dig into some of the more technical aspects - testing strategies, automated builds, refactoring etc. Since these things are challenging when applied to regular .NET development, I'm especially interested to get some ideas on overcoming some of the extra challenges the ArcGIS stack adds into the mix (i.e. using mocks & fakes for a large interface based API like ArcObjects)

And getting a chance to hear people like Scott Ambler, Roy Osherove, Mike Cohn, Tom and Mary Poppendiek, and Andy Hunt talk about agile development should be great. I just picked up a new laptop today, so I expect to be taking copious notes and likely more than a few blog postings while at the conference. Hope to see some other GIS people out there as well!

Just read an article on GISUser.com (also at GISCafe) about ESRI bringing on Dirk Gorter (formerly with Symantec) as new Director of Product Management. Here's a quick quote:

Gorter's focus is on implementing a product management process that is similar to the agile product development process commonly used by software development teams.

This is great news! Hopefully ESRI will be talking up agile as they go through their own changes. This level of exposure is sure to help raise awareness of the benefits of agile development within the broader geospatial industry - maybe we'll see some ESRI people at the Agile Development Practices conference next week!

Yesterday we sent out a press release via Chris Spagnuolo's GeoScrum blog. Thus, I think it's now ok to say that my entire team and I are now with Data Transfer Solutions.

 

The team is super excited about the new opportunities this change brings (the blazing workstations, "green" servers and build out, IKEA workstations, great chairs, and a strong interest in adopting agile methods doesn't hurt either!).

On the technology front, we are still sticking with .NET and ESRI for the most part. We expect to be doing much more web work than in the past, and with the release of Visual Studio 2008 and .NET 3.5 this will be very cool. I expect we'll be working with ArcGIS Server / tile caches on the back end, as well as Virtual Earth and Open Layers for the client UI. Mix in some unit testing, and it should be lots of fun.

We're also going to be adding some new capabilities - specifically Agile Consulting and Coaching. This is still in the early stages so I'll be posting more about it as things develop, but I'm very excited about some things we have planned, and the opportunity to share our experiences, methodologies and practices with others.

I'll wrap this up by saying that I'm also very excited to work with the existing DTS development team. Although I've only met a few of them, and only very briefly, but their depth and breadth of skills are pretty amazing -- I heard that someone wrote some ArcGIS Desktop tools using Boo (scratching your head - here's a hint: Boo is a Python-like scripting language for the ECMA CLR implementations such as .NET and Mono.)  I think they will be keeping us on our toes!

I'll be posting more about starting up an office, and getting things rolling over the next few weeks. Some tasks on our current backlog - order & setup some servers, get a firewall (advice welcomed!), move out of Chris's living room into our "temp" space, finalize the build out plans for our "real" space... fun!

I had originally setup a SourceForge project for ArcDeveloper.net, but the whole SourceForge experience is clunky and web 1.0. So when I found out about Assembla, I immediately setup a project.

Assembla is a really cool, free Subversion + TRAC + wiki + Scrum development tool / service. It's really amazing. It's totally free for projects up to 200Mb in size (which is a MOUNTAIN of source code).

ArcDeveloper Projects

A repository and some tools is nice, but a bigger question is what projects will be there. Well, my goal for ArcDeveloper is to provide a set of code resources that will make unit testing on ArcGIS projects easier. Since software quality is something that all developers must address, and since it's the same regardless of your vertical market, I think that few organizations would object to using and contributing to this effort.

Getting Access

The repository is open for all users, just point your Subversion client to http://svn2.assembla.com/svn/arcdeveloper and you can get latest on everything. If you are not familiar with Subversion, Assembla has some handy starter info here, and you can Google for mountains of information.

Project Status (TRAC)

The project is managed in TRAC - since it's a web based tool, you can access it here: http://trac2.assembla.com/arcdeveloper/wiki. There's not a whole lot in there yet, but I'll be posting about the inital project shortly.

Bug Reporting / Feature Requests / Contributing

You'll need to be a member on the project to add a TRAC ticket. The easiest way to do this is to contact me, and I'll send you an invite directly from Assembla. If you are already an Assembla member (sigh-up here - takes 30 seconds and it's free), you should be able to contact me through the project's team page. Either case, as far as I can tell, I have to invite you into a project - I don't think you can "apply" to join.

Once you are a member of the project, you will also be able to contribute code.

So that's it for now - as I said, I'll be posting about the initial code base shortly...

No sooner do I mention Seth Godin in my "Changing the Game..." post, than he puts up a post called... wait for it... "Changing the Game". Beyond being a cool coincidence, the post is exactly the type of "Seth Godinsim" I was referring to in my post. Give it a read, then subscribe to his blog - a wealth of thought provoking business and marketing ideas come rolling out of there on a daily basis.

Over the weekend I was talking with a friend who's company had recently been invited to a finalist interview for a large state contract. As a company, they have been adopting agile with good results, despite the fact that most of their current contracts are fixed price - fixed deliverable.

For a while now they have been looking for agile contracts, but none have come up. So they decided to change the game. They went into the finalist interview, and in addition to presenting their qualifications, experience, and proposed solution, they also did a quick presentation on their project management methodology (Scrum + Selective XP Techniques). No discussion of why it's better than waterfall - just "this is how we do things".

Given that state agencies are not known for embracing change, it was a gutsy move, but it paid off and they won the contract. Now they have a great contract, and can run it in a sensible way.

I love this story - it's very "Seth Godin" - do things differently to get the results you want! They were sick of fixed price projects but were not seeing many requests for agile, so they changed the rules. Instead of trying to "win" a client over to using agile, they just stated "we use agile". Period. They made it the default position.

In addition to winning, it also sets them up for more success. By writing up a case-study about how the project worked, they can show both their success in the vertical market and how agile can be successful in one of the most challenging environments - state government. And that can be used to win over other, possibly skeptical clients.

This morning Vish sent me a link to a video of the Microsoft Patterns & Practices team space at Microsoft. I checked it out over lunch, and besides showing off some really sweet developer digs, Peter Provost & Ed Jezieski talk about how the whole Patterns & Practices team uses agile methodologies. They also mention that the Team System Database Professional team is using Scrum to build their project, and that more and more product teams are "dropping by" to see how agile is working for them. Given that this video is over a year old (posted on Oct 5th 2006), it would seem that Agile has been making serious inroads at Microsoft.

Organizations the size of Microsoft have a lot of inertia, and things don't change easily. But if Microsoft can see the value and adapt, what can possibly be holding other groups back? If you are not using Agile - why not?

Getting started is pretty simple - the hardest thing is deciding to start. As for what you need - lots of teams run almost entirely on 3 by 5 index cards, with a little Excel used to make burn-down charts. Of course once you get rolling, using some software to track everything simplifies things (in my opinion), but it's certainly not needed to get started. Once you show the benefit to the organization, I think you'll find that funding for Agile tools is pretty easy to come by.

I've been talking to a number of people around the GIS industry lately, and when I bring up our use of Agile methods and Scrum, most people have had a pretty blank look on their faces. Once I explain the idea and the methodology most seem genuinely interested, and the most common response is - "that makes sense".

Throw in the that that since adopting Scrum, our team has become more productive and focused that ever, and this seems to be a very common outcome. So I'm surprised that more people are not familiar with agile. 

In an attempt to see if anyone else in GIS is talking about agile and/or Scrum, I did some searching, and found this...

 

Apparently these guys have had 6 conferences on "AGILE" and GIS. Unfortunately this has nothing to do with the wider "Agile" concepts as applied to software development (see the Agile Manifesto for more on this).

Anyhow, I continued searching. With the exception of my occasional post, Chris Spagnuolo's blog, and some job postings from our company which note the use of Agile methodologies, I found only a few other things:

1. A post on the MapButcher blog re: Managing GIS Projects which talks about Scrum
2. A post on Human-Debugger (Shani Raba)'s blog about some Scrum experiences
3. Paolo Corti has some posts on Test Driven Development (TDD), but they are related to his excellent series on MonoRail and the Castle Project.
4. A description of an agile methodology used by Dynamic Ventures (off-shoring group) for all their project - including GIS.
5. Infinity Solutions (also an off-shoring shop) also came up, but I could not actually find anything GIS related on their site.

There may be more, but was 5 pages into the search results and things were getting pretty far off base.

Development Trends outside of GIS

This lack of discussion about GIS and Agile is very interesting because in the broader software industry we are seeing a much different story. Anecdotally, I can say that the Certified Scrum Master classes we've looked into for other team members have been pretty full. The Scrum classes I've attended have been full and had a wide range of people in them - from senior "PMI" project managers to video game designers. We hear that there are many teams in Yahoo!, Microsoft and Google which have adopted scrum and other agile processes with great success. But lets look for some numbers to back some of this up...

Last year Scott Ambler did a survey about agile adoption (here's Scott's PowerPoint with all the data and some charts etc). He used the Dr Dobbs Journal (a long-running software development magazine) mailing list, and got 4232 responses.

Two questions were particularly interesting:

Have you adopted any agile techniques? - 65% Yes

Have you adopted an agile methodology? 41% Yes

So - assuming that this is a somewhat representative sample, 65% of the software industry have adopted some agile techniques and 40% have adopted an agile methodology. 

So why are we not seeing more discussion of agile in the geospatial industry? Is the GIS industry still caught in the waterfall? Thoughts?