Programming in ArcGIS with Python – A Beginners Guide
As a GIS analyst, you can raise your market value by learning to program. This is easier said than done: the possibilities can be overwhelming, especially if you don´t know where to begin. That´s why I put together this guide to get you started with Python programming for GIS.
GIS programming in my opinion starts with Python, but certainly won´t end there. From a beginner’s perspective, it´s not recommended to start learning as many languages right away, but instead to start easy and differentiate later on, depending on your needs. Python is a good choice to start for learning GIS programming, as it can be used as a scripting and programming language. In ArcGIS, Python scripting can be used for automating tasks (through running Python scripts), as well as writing applications, such as add-ins. This guide focuses on Python scripting as Python programming is more complex and takes more time to learn – for now, we´re good with learning Python scripting.
One: First things first – learning the basics
So, to begin with, I’d recommend learning the basics of Python –I´ll get to the GIS part later. In order to be able to use Python within a GIS, you need to learn about Python syntax, data types, commands, built-in functionalities, loops and the like. If you´re new to programming, this can be a little overwhelming at first as Python has a lot of built-in functionalities. It´s not necessary to know the language from a to z to use it within a GIS – with some basic knowledge you can get up to speed pretty fast and use it within ArcGIS. I also recommend learning how to request help files within the interpreter and list available methods, functionality and modules, rather than consulting the external help files. This gives you more control of the language itself.
For the Python beginner, there are a lot of free Python beginner tutorials available online and almost all of them cover the same basic topics: strings, functions, data types, syntax – all essential topics that will be used anywhere when working with Python – skip theory about object-oriented programming, as you don’t need to know that for Python scripting (however, you WILL need it when you decide to create Python add-ins in ArcGIS).
At this point, you might need a local version of Python installed on your computer (from www.python.org), or opt for an online IDE, such as Python Fiddle that lets you write and run Python through a web browser. Whether option you choose, you will notice there are different Python versions available – GIS users are recommended to learn Python 3, as this is the wave of the future. However, that doesn’t mean that Python 2 has no value, quite the contrary: ArcGIS Desktop for example uses Python 2 and is likely to keep doing that. To get an idea of the differences between both versions, do a Google search for “Python 2 vs 3” or “Key differences between Python 2 and 3” and read some documentation on the topic.
If you have ArcGIS Desktop installed on your computer, it comes with Python automatically installed so you don´t have to make the choice for a version. Just click at the ArcGIS sub-menu in the Windows task bar, click the IDLE (Python GUI) . A Python terminal opens with a command line (“>>>”) where you can start writing code right away. Execute the code by hitting enter and see the direct results printed at the screen.
Two: Mastering the basics through practice
We’re still learning the basics – and learning Python or any programming language means a lot of practice. Practice with online exercises from Code Academy or the free “Try Python” course from Code School. Once you’re familiar with the basics, it´s time to do some research on how Python scripting works and the differences between writing and running scripts and using an interactive Python interpreter. As this is not covered in many online courses, this may require some online searching but for GIS analysts this is essential knowledge. You also might want to practice with writing and running a few of your own scripts – no matter how simple they are. Also learn how to comment your code – no matter how simple the code is, you’ll be thankful later you did this when you first wrote it.
It is recommended to use an IDE (Integrated Development Environment) for writing and running Python scripts. This is nothing more than a piece of software that functions as a code-editor, enabling you to write, run, debug and save code. You can download and install a (free) IDE – such as PythonWin – on your computer and write, run and debug your scripts from there. IDE’s are a preferred place to write code as they help you by coloring your code as to separate different elements, making it easier to notice errors before and after running a script.
While we’re at it, you might want to check out some of the built-in functionality of Python, such as the math module that is part of the standard library that comes with a Python installation. Working with numbers is again an essential part of GIS and Python, and that math knowledge from college might need some refreshing. As Python allows you to work with geometries from objects in a GIS, knowing what´s available with this library might come in handy latter. The Python documentation (available both online and locally, as it comes with an installation) can be consulted to see what´s available. Other skills that will in handy later are learning to write data to a new file with Python, setting source data paths correctly, how to write SQL statements correctly in Python, use “try and except” blocks and learn about error messages.
Three: Using Python in ArcGIS
With some Python practice under your belt, you might want to move on and use it in ArcGIS. ArcGIS users (both ArcMap and ArcGIS Pro) might start using the Python window that is a great way to start learning the ArcPy site package. This is a collection of modules that allow access to all geoprocessing functions and more. Notice that ArcGIS Pro uses Python 3 while ArcMap uses version 2. The Python window has a great interactive help window that helps you writing code, by specifying the parameters for a tool, as well as autocompletion of code, saving you a lot of time. This is a great way for learning ArcPy and should be your first start. Once you´re confident using ArcPy, try writing some standalone scripts using local GIS data, run them and see what happens.
Common sense is your best friend here: it’s common to have filenames and -paths spelled wrong, which may cause errors. Even though a Python script has been checked correctly by the IDE for mistakes, it may contain errors, such as misspelled filenames, paths, commands and tools. Wrong use of capitals is also a common error and Python is very sensitive to this. You can control the process of your script by letting it print out messages to see how things are progressing, such as “print(“new geodatabase created correctly”). If a script terminates without errors, open ArcMap and visualize the results your script created. If the output is not as expected, there may be something wrong with your code.
To become acquainted with ArcPy and Python scripting in general, there are many online tutorials and books available that explain in detail how everything works. Again, start with the basics and then move on to more complex topics. The ArcGIS Help section is essential reading for learning ArcPy, but you might be served with extensively annotated example scripts and exercise books with example data to experiment with, created by experts.
Although a niche market, there are many helpful books available about GIS and Python scripting. What´s good to know is that many of these books are written for intermediate and advanced users, so make sure you know your basic Python skills and get to know ArcPy well before starting with these. A good introduction to the ArcPy site package is Paul Zandbergen´s book “Python Scripting for ArcGIS” – although it´s more a reference book than a hands-on book. Be sure to combine it with a hands-on book, such as “A Python Primer for ArcGIS”, the first book on Python scripting for ArcGIS, written by Nathan Jennings. It focuses on python scripting and has a lot of well-annotated scripts and step-by-step tutorials that I enjoyed a lot: even though it´s a little dated, all the scripts run just fine with later versions of ArcGIS. I´d rate this as an Python book for the intermediate Python student. In addition, David Allen´s “GIS Tutorial for Python scripting” offers the most comprehensive overview of what´s possible with Python in ArcGIS. To get the most value out of this book, ask for an answer sheet from Esri Press as the editing of the book is problematic at times and the example code contains errors (the instructor code on disk works fine though).
Four: Python scripting and beyond
With some Python experience, you might be interested in knowing where to go next. Before delving into other languages and/or frameworks, look for techniques to improve your current Python code, for example by refactoring it by using list comprehensions where possible. Look for new geospatial modules that make your coding work easier. Compare the work from professionals with each other and use good coding habits from others to improve your work.
The learning path described above is time-consuming as well as iterative – you´ll find yourself going back and forth between different stages and learn something new every day. But it is also fun and full of surprises. The more you learn and practice, the easier it becomes. The good thing is that Python extends GIS – think of many other geospatial libraries or using the language “R” with Python.
CIA declassified its map collection on Flickr and it is totally awesome
“An intelligence agency is a government agency responsible for the collection, analysis, and exploitation of information and intelligence in support of law enforcement, national security, military, and foreign policy objectives” we can read on Wikipedia. Looking at this definition it’s clear that there is no intelligence agency without maps.
CIA realized that quite early and launched its Cartography Center back in October 1941. Since the beginning the mission of the unit was to provide a full range of maps, geographic analysis, and research in support of the Agency, the White House, and policy makers.
Over the years The Cartography Center produced thousands of maps. As you might imagine almost all of them were classified. Recently in honor of the unit’s 75th anniversary, the agency has released an amazing collection of declassified maps. These collection of 130 maps and over 200 images of old cartographic tools possibly played an important role in many significant events in the world’s history.
The collection is divided into different decades from the history of the unit. It starts with 1940s where we can see for example a map of occupation zones of the Korean Peninsula.
Another decade is 1950s where we can see an amazing map showing the exploration and claims of Antarctica.
As well as a map showing the production and international vegetable trade around the world. In 1960s we can see a striking map of Burma climate that must have been used in the planning of the military operations.
1970s collection features a stunning hand rendered terrain map of the world.
In 1980s one of the most interesting map shows the oil & gas resources in Indonesia.
In 1990s we can see a map of coca cultivation and trafficking. Looks like the job of Pablo Escobar.
The new millennium comes with more freedom in choosing the cartographic visualization. The 2001 Afganistan’s Konar Valley has been analyzed in 3D before the military operation after 9/11.
Another example is a cartogram showing flags of Africa
The most recent examples from the current decade include 2013 ivory trafficking map.
2014 Lebanon Battle in 3D
as well as an interesting map showing the comparison of area in different map projects.
The overall collection is amazing!