Friday, February 23, 2007
File Geodatabase Rasters: Fast!
Posted on Friday, February 23, 2007 5:06:17 PM (Mountain Standard Time, UTC-07:00)  Comments [2] | 
Categories: ArcMap | Geodatabase
I had heard that raster performance in a File Geodatabase was good, and thought I'd check into it. For source data I used 4 tiles of 6 inch orthophotography. The files are TIFFs, and come in at 715MB each.

Initially I tried loading all 4 into ArcMap, and it just took too long to display, so I killed ArcMap, and re-started by just loading one.

To load and draw the file (no pyramids) took ~60 seconds (based on my watch)



Next, I used ArcCatalog to created pyramids for another of the tiles (took a few minutes and created an 80Mb RRD file). This loaded in about 5 seconds.




Finally, I loaded all 4 TIFFs into a file geodatabase (again took a few minutes). The resulting File Geodatabase is 2.87 GB in size, so it's a little bigger than the source TIFFs, which were 2.7GB. When loading into ArcMap, the layer took about 1 second to draw (again using my watch - it maybe a half second - hard to tell, other than it was FAST).




The other thing to note is that in both the TIFFs, the resulting image looks pretty choppy at this scale (compare the images above). The File Geodatabase image looks great.

As far as zooming and panning, the File Geodatabase just rocked. It's screaming fast. This next shot is at full resolution - I zoomed in 3 times drawing a rectangle each time, and the screen re-draw just about as fast as I could draw the boxes. Amazing.




The TIFF with pyramids seemed to be about the same speed with panning and zooming, but the TIFF without was consistenly a much slower 2-5 seconds for a pan or a zoom.

So - if you've got imagery that's slow, try out the file geodatabase and see if you get the same boost in speed.

Note: All these tests were done on a local disk - connecting across a network will impact performance.
Friday, February 23, 2007 8:57:39 PM (Mountain Standard Time, UTC-07:00)
My suspicion is that this has nothing at all to do with format, and everything to do with the processing applied to the data.

Firstly, TIFFs without pyramids will always be slower than any format (including pyramided TIFF!) that has pyramids. So there is no point even comparing the two. Without pyramids, the software has to resample the image on the fly, which is a slow, expensive undertaking.

So, that leaves two cases. The zooming around test, which you found to perform essentially the same for both formats. No surprise there. And the fully zoomed out draw, which you found fast and pretty in FGDB, and slower and uglier in TIFF.

So the question is why, in the fully zoomed-out case, is TIFF slower and uglier. My guess is that (a) the TIFF pyramid is topping out sooner than the FGDB pyramid, so it is slower to re-sample from and (b) the TIFF pyramid is built with a stupider re-sampling algorithm (nearest neighbor?) than the FGDB pyramid.

But why stop at supposition!?! Let's test! (For this, Dave, you'll have to play along, I hope you don't mind.)

- Download FWTools from here: http://fwtools.maptools.org/
- Now use the 'gdaladdo.exe' to build a new (hopefully prettier and faster) pyramid on your TIFF: gdaladdo.exe -r average yourtiff.tif 2 4 8 16 32 64 128 256
- Now back to ArcMap, load up that newly processed TIFF. Is it faster? Is it prettier? I hope so!
Paul Ramsey
Saturday, February 24, 2007 10:01:34 AM (Mountain Standard Time, UTC-07:00)
I tried something similar using an ArcIMS 9.1 ArcMap (MXD) image server. I was comparing file GDB vs. MrSid files, using ArcIMS's logging to get exact processing times.

I processed a 155mb Sid into about a 2.6gb FGDB. 90+% of the time, the FGDB was mildly faster than the Sid in a single user setting. Drawing times were commonly in the 0-1.5s range on an old test box. Rarely was the FGDB more than 30% faster, but I did see some 1.5s vs 4s outliers. Trivial cases had trivial speed differences.

The interesting side-note is that the FGDB imagery extracted from a Sid looked crisper than the on-the-fly extracted Sids.

I'm not sure if ArcIMS uses a speed-optimized Sid extraction, or if the crispness in the FGDB is a false product of the resize filter. I'm leaning to the former, because I want to believe that sharper imagery is a good thing.
Bill Thorp
Comments are closed.