Configuration Files For Geoprocessing Script Tools

Coming from a staunch ArcObjects background I was long a detractor of geoprocessing. But several painful excursions with Server Object Extensions coupled with a new client deploying ArcGIS Server on Linux have me seeing the utility of using Python to develop script tools and deploy them to ArcGIS Server.

The Scenario:

A client has an OpenLayers application which allows users to select a list of features of interest from a tabular widget. An ID attribute of these features is used to formulate a definition expression and the features are extracted from a geodatabase into a shapefile and then packaged into a zip file available for download as a url resource. This is accomplished by creating a script tool with Python and publishing it in a Toolbox with ArcGIS Server. This publication creates a simple, accessible RESTFul ‘interface’ (hope all the RESTifarians out there can forgive my language misuse, I know proper semantics are important – Busting RESTFul Myths ).

The Problem:

In a word, ESRI documentation. I jest, but not entirely. I like the simplicity that using Python with the GP object provides. My desktop thesaurus gives both uncomplicated and dimwitted as synonyms for simple. Though we all strive for the first when working with software, the ESRI documentation for Python seems to think I am more the latter and overlooks many great features of a rich language. Most of this can probably be attributed to the code sample dilemma discussed in a great post here. The code samples in the documentation are focused on expressing some simple concepts, as they should be. But this simplification often results in overlooking some patterns and practices that should be followed. One such overlooked practice is applying the DRY principle to application configuration. The client described above deploys applications to three environments; Development, Test and Production. Each environment has its own geodatabase with its own credentials and naming conventions. How do we store this information in a configuration file accessible to the script tools and Toolboxes we publish using ArcGIS Server?

Out of the Box:

The ESRI documentation gives us almost all the necessary information needed to do accomplish this. We just need to take one step further to review a core Python module and we’re home free. The first piece the geoprocessing documentation provides is a recommendation for sharing toolboxes that I believe everyone should standardize on. The second piece is a slightly archaic Python technique for finding the executing script’s source location. A quick web search reveals several libraries for structuring and accessing configuration information in Python. For this example we’ll stick with the standard module ConfigParser though PyYAML provides a good alternative. We’ll follow ESRI’s toolbox sharing structure and create a configuration file named ‘Configuration.cfg’ in the ToolData directory. If we are configuring connection strings it might look something like this

[CONNECTIONSTRING]
sdeConnectionFile = \\SharedResources\ConnectionFiles\Production.sde

To access this connection information from our script tool we can then use

import sys, string, os, ConfigParser, arcgisscripting
gp = arcgisscripting.create(9.3)
ToolShareDirectory = os.path.dirname(sys.path[0])
ConfigFile = os.path.join(ToolShareDirectory, "ToolData", "Configuration.cfg")
Config = ConfigParser.ConfigParser()
Config.read(ConfigFile)
SDEConnectionFile = Config.get('CONNECTIONSTRING', 'sdeConnectionFile')

Note that the use of

os.path.join

is preferable to the string concatenation technique shown in the ESRI sample since the os module will handle the presence or absence of path separators in the path components seamlessly. Since I’m pointing out minutiae in someone else’s sample code, I should probably practice as preached and illustrate my sample as something slightly more defensive like

import sys, string, os, ConfigParser, arcgisscripting
gp = arcgisscripting.create(9.3)
ToolShareDirectory = os.path.dirname(sys.path[0])
ConfigFile = os.path.join(ToolShareDirectory, "ToolData", "Configuration.cfg")
if os.path.exists(ConfigFile):
    try:
        Config = ConfigParser.ConfigParser()
        Config.read(ConfigFile)
        SDEConnectionFile = Config.get('CONNECTIONSTRING', 'sdeConnectionFile')
    except:
        gp.AddError("Configuration setting unavailable")

Now we can simply edit the name of the .sde file in Configuration.cfg file as the Toolbox moves between deployment environments. The real value becomes apparent when there are multiple script tools in a single Toolbox that can all share the same configuration information.