There is so much raster data hosted on the web. We have access to nearly endless imagery and elevation datasets. While these data are exceptionally rich and useful, they can produce a problem of their own. The downsides of raster data Data requires storage. And storage gets messy. It’s inevitable that we’re going to download…
Python is a powerful language with many powerful packages. However, managing Python packages can often be difficult and tedious. This is especially the case when there are dependency conflicts between different packages and Python versions. This is why package managers were invented. To automate the process of creating functional, repeatable Python environments so that we…
leafmap_split Visualizing raster data is important for assessing the results of analysis and models. One of the best ways to visualize raster data is to plot it on a map where you can interact with the raster data an other relevant layers. When performing analysis in Python, it can be difficult to create an interactive…
GDAL, the Geospatial Data Abstraction Library, was designed to interact with raster and vector (geospatial) data. Over time it has evolved to include sophisticated data manipulation and analysis tools. The GDAL source code is C++, which makes it fast to run but difficult to install. Many software and applications use GDAL for handling and analyzing…
Relative elevation models (REMs) adjust a DEM so that the elevation is relative to another feature instead of sea level. One common way to create a REM is to make elevation relative to a river bed. Then, the resulting REM displays the height of each pixel above the nearest river location. River REMs produce some…
Having trouble finding an easy way to install Python that always works? You’re in the right place. After years of trying different Python installation methods, I’ve come up with a method that will get you installing packages and running code in no time! The hardest part of learning Python isn’t writing Python code. There are…
Clipping, extracting to masks, and subsetting rasters are common GIS operations. Using the gdal package for Python it is very easy to automate these processes. This tutorial will demonstrate how to use the Warp() function from the gdal Python API to clip rasters to a specified extent and to clip rasters to a polygon layer. These operations are also often referred to as mask extraction or extract to mask.
One of the key tasks for data scientists and computer programmers is to read, write, organize, and manipulate data. Perhaps the most intuitive format in which these data are stored is the tabular format. This format organizes data into tables, which are also commonly referred to as data frames or spreadsheets. Because of data, programs…
Over the last few years, I’ve identified 4 reliable methods to run Python code in VSC. If one of these methods doesn’t work or seems too complicated, don’t worry there are three others you can try!
GeoPandas is a high-level Python package for reading, writing, analyzing, and manipulating vector (points, lines, and polygons) geographic data. As the name ‘GeoPandas’ implies, it brings pandas data frame functionality to geographic data so that you interact with the data much as you would an attribute table in ArcMap or QGIS or a spreadsheet in…
If you’re looking for a way to automate your GIS workflows and products, the QGIS Python API (PyQGIS) may be able to save you a lot of time. With PyQGIS you can write Python code that automates QGIS tasks that range from loading data to creating and saving map layouts. I’ve used PyQGIS to write…
The Geospatial Data Abstraction Library (GDAL) is the standard for managing spatial data formats. GDAL has been incorporated into many different enterprise and open source GIS projects. Because GDAL is open source, it can be used by all. Learning to use GDAL with Python can help you automate workflows and implement custom raster processing solutions….
Python is a very versatile and popular cross-platform programming language that is used for many computational tasks. One thing that makes Python so versatile is that scripts can easily be run from both integrated development environments (IDEs) and the command prompt or terminal (check out this tutorial for instructions). Additionally, you can type and run…
Automating file downloads can save a lot of time. There are several ways for automating file downloads in Python. The easiest way to download files is using a simple Python loop to iterate through a list of URLs to download. This serial approach can work well with a few small files, but if you are…
Over the past couple of years, Visual Studio Code (VSC) has become my favorite Python IDE and all-around code editor. It works well for almost any programming language, has an intuitive layout, can be customized and extended with extensions, and (most importantly) it’s completely free. One awesome Visual Studio Code feature is that it integrates…
Random number generation is a common programming task that is required for many different programs and applications. In Python, the most common way to generate random numbers is arguably the NumPy module. NumPy is fast, reliable, easy to install, and relied on by many programs. As of 2022 (numpy version 1.22) the proper way to…
Visual Studio Code and Anaconda are powerful tools for Python developers. However, if you are trying to use Anaconda and Visual Studio Code together there is a good chance you have run into some problems. As common as these tools are for programming with Python, it can be difficult to figure out how to get…
In Python, arguments are passed to a script from the command line using the sys package. The argv member of sys (sys.argv) will store all the information in the command line entry and can be accessed inside the Python script.
In Python, matrix-like data structures are most commonly represented with numpy arrays. The numpy Python package has been well-developed for efficient computation of matrices. The first step to using numpy arrays is to initialize, or create, an array. In Python, there are many ways to create a numpy array. In this article, I’ll demonstrate how to create numpy arrays in ten different ways.
With Python, there is a multitude of methods to iterate through lists and tuples. Some of these methods can be complicated. If you’re just learning about iteration in Python, you’ll be best served by learning the easier, more common interaction methods that can be applied to almost any situation. Here, I present 5 easy methods to iterate through Python lists and tuples.
Recently, I’ve started using rioxarray to read NetCDF data into xarray format. If you’re not familiar with the xarray python package it’s basically a wrapper (for lack of a better term) around numpy arrays that allows metadata to be included with the arrays (more on this later with an example).
npwhere ¶ The where function from numpy is a powerful way to vectorize if/else statements across entire arrays. There are two primary ways to use numpy.where. First, numpy.where can be used to idenefity array indices where a condition is true (or false). Second, it can be used to index and change values where a condition…
The most used python package for reading and writing raster data is probably rasterio. The rasterio package wraps around the Geospatial Data Abstraction Library (GDAL, also available as a python package) to provide core functionality for accessing raster data.
The Point Data Abstraction Library (PDAL) is a powerful, open-source tool designed to work with dense point clouds. PDAL has applications that range from engineering to geography and is especially useful for working with lidar and structure from motion data. PDAL is written in C++ and is also available as a command-line executable and Python…
Anaconda is a Python distribution with additional tools to make it easy to manage Python environments and packages. If you’re the kind of person that wants to work on a project without worrying about compatibility between dependencies or building open-source packages from source, then Anaconda will probably be a helpful tool for you. The downside…
Creating Python scripts that can be run from the command line makes it much easier to abstract and share your code so that it can be reused and shared with others. Running scripts from the command line can also streamline your development and analysis workflows to make them more concise and make you more productive….
The numpy Python module is widely used for many different analyses and as a dependency for many other Python packages. In many instances, it is necessary to determine if numpy is installed and which numpy version is installed. This article will show you multiple ways to determine your numpy version and installation status. Check if…
Installing GDAL for Python with pip , the package installation package, is not as easy as it may seem. If you try to install GDAL to a python environment with a command like python -m pip install gdal it will initially seem like the installation is going to happen. However, you will receive a message…
The Geospatial Data Abstraction Library (GDAL) is a fundamental package for spatial analysis with Python. GDAL is a powerful package with a lot of functionality. However, it is notoriously difficult to install. There are two reliable ways to install the GDAL python package: from the conda-forge channel using the conda installer or using pip to…
Just as spatial joins can be performed in QGIS with the Join Attributes by Nearest tool, the QGIS Python API (PyQGIS) can be used to perform spatial joins in the same manner. This article will demonstrate, with code, how to perform a spatial join with PyQGIS. This example will join the attributes of a line…
Finding the shape of a nested list or tuple is much more complicated than finding the shape of a single element because each element in a list of lists or tuple of tuples can have a different dimension. Unlike NumPy arrays, lists and tuples aren’t required to have a set number of rows or columns….
NumPy is a fundamental module used in many Python programs and analyses because it conducts numerical computations very efficiently. However, for those new to NumPy, it can be difficult to grasp at first. Specifically, understanding array indexing and sorting quickly becomes complex. Fortunately, NumPy has some built-in functions that make performing basic sorting operations quite…
Using attribute fields in calculations is one of the most fundamental components of GIS analysis. Field calculations are what makes it possible to take data linked to spatial features and analyze features based on what they are and where they are. Manually performing field calculations in QGIS can be tedious, especially if you’re working with…
Python tuples and lists are objects that contain multiple values and it is a common task to determine the number of elements (or shape) in a tuple or list object. While this seems simple enough, there is one nuance that makes it a little more complicated. Shape of a One-dimensional Python Tuple or List The…
There’s a good chance you’ve done something today that used a sliding window (also known as a moving window) and you didn’t even know it. Have you done any photo editing? Many editing algorithms are based on moving windows. Do you do terrain analysis in GIS? Most topographic raster metrics (slope, aspect, hillshade, etc.) are…
Filling NumPy arrays with a specific value is a typical task in Python. It’s common to create an array, then initialize or change some values, and later reset the array to a starting value. It’s also common to initialize a NumPy array with a starting value, such as a no data value. These operations may…
When you work with spatial data it’s inevitable that you will need to implement information from both a vector and raster data source for the same location. This task can easily be accomplished manually, but it often becomes quite cumbersome when the process must be automated across a large number of features, time periods, and/or…
The QGIS Python API (PyQGIS) makes it really simple automate a feature dissolve. PyQGIS gives access to all the QGIS processing tools directly from the Python interface. This tutorial will demonstrate three different dissolve examples with Python in QGIS. Basic Dissolve with Python First, lets dissolve polygon features into a single polygon, this is the…
It is a common need to summarize information from a gridded dataset within an irregularly shaped area. While at first glance this may seem simple, reconciling differences between raster (gridded) and vector (polygon) datatypes can quickly become complicated. This article shows how to implement a zonal statistics algorithm in Python in 4 steps. Load raster…
Gridded, spatial data are commonly stored in NetCDF files. This is especially true for climate data. NetCDF files offer more flexibility and transparency than some traditional raster formats by supporting multiple variables and detailed metadata. Because of the metadata and file structure NetCDF files can be more difficult to access than traditional raster formats. This…
Aerial imagery is used for purposes ranging from military actions to checking out the backyard of a house you might buy. Our human brains can easily identify features in these photographs, but it’s not as simple for computers. Automated analysis of aerial imagery requires classification of each pixel into a land cover type. In other…
Did you know you can write parallel Python code that will run on your laptop and on a super computer? You can, and it’s not as difficult as you might expect. If you already write code for asynchronous parallelization then you won’t even have to do much restructuring. High Performance Computing (HPC) distributes pieces of jobs across…
A flexible method to speed up code on a personal computer Do you wish your Python scripts could run faster? Maybe they can. And you won’t (probably) have to buy a new computer, or use a super computer. Most modern computers contain multiple processing cores but, by default, python scripts only use a single core….
Important information describing the data contained in netCDF files is embedded directly in the netCDF file. Accessing and reading of this information (i.e. metadata) can help you automate tasks, debug code, and validate results easily. This tutorial covers how to access information describing the metadata, dimensions, and variables contained in a netCDF file. For the…
Many geographic data are store in netCDF format. This is especially true for climate data. Interacting with netCDF data files is an important skill for any geographer, physical scientist or data scientist to have. This tutorial will get you started using the netCDF4 Python package to read netCDF data. First, you’ll need to install netCDF4….
Use the random forests algorithm to classify image segments into land cover categories. This post is a continuation of Geographic Object-Based Image Analysis (GeOBIA). Herein, we use data describing land cover types to train and test the accuracy of a random forests classifier. Land cover data were created in the previous post. Step-be-step video instructions…
It is very simple to clip vector layers with PyQGIS. This tutorial will demonstrate how to use Python in QGIS to clip a line layer with a polygon layer. First, import the processing module and set the paths to the line, polygon and output layers. The next step is to run the clip tool. This…
This tutorial will walk you through segmenting and classifying high resolution imagery using Python. Part 1 of this tutorial teaches how to segment images with Python. After you have completed Part 1, Part 2 will teach how to use machine learning methods to classify segments into land cover types with Python. YouTube videos give step-by-step…
This tutorial will demonstrate how to measure the distance between two geometries in QGIS using Python. First we will measure the distance between points in the same layer. Then we will measure the distance between features in a points layer and a lines layer.
Creating a buffer around features is a common spatial analysis task. This blog post demonstrates how to customize and automate buffer operations with Python in QGIS (PyQGIS).