Automating Complex Workflows using Processing Modeler (QGIS3)¶
GIS Workflows typically involve many steps - with each step generating intermediate output that is used by the next step. If you change the input data or want to tweak a parameter, you will need to run through the entire process again manually. Fortunately, QGIS has a graphical modeler built-in that can help you define your workflow and run it with a single invocation. You can also run these workflows as a batch over a large number of inputs.
We will take a point layer of maritime piracy incidents and create a processing model to produce a density map by aggregating them over a global hexagonal grid.
In the QGIS Browser Panel, locate the directory where you saved your downloaded data. Expand the ne_10m_land.zip and select the ne_10m_land.shp layer. Drag the layer to the canvas. Next, locate the ASAM_shp.zip file. Expand it and select the ASAM11OCT18.shp layer and drag it on to the canvas.
The ASAM11OCT18.shp layer does not have projection information associated with it, so you will be prompted to select a CRS in the Coordinate Reference System Selector. Here, the points are in the Latitude and Longitude coordinates, so select the WGS84 CRS and click OK.
Once the layer is loaded, you can see the individual points representing incidents of piracy locations. Let’s start building our Processing model to process these layers. Go to Processing ‣ Graphical Modeler….
In the Processing Modeler dialog, locate the Model Properties panel. Enter piracyhexbin as the Name of the model and projects as the Groups. Click the Save button.
Save the model as piracy_hexbin.
Now we can start building a graphical model of our processing pipeline. The Processing modeler dialog contains a left-hand panel and a main canvas. On he left-hand panel, locate the Inputs panel listing various types of input data types. Scroll down and select the + Vector Layer input. Drag it to the canvas.
Enter InputPoints as the Parameter name and Point as the Geometry type. This input represents the piracy incidents point layer.
Next, drag another + Vector Layer input to the canvas. Enter BaseLayer as the Parameter name and Polygon as the Geometry type. This input represents the natural earth global land layer.
As we are generating a global hexagonal grid, we can ask the user to supply us the grid size as an input instead of hard-coding it as part of our model. This way, the user can quickly experiment with different grid sizes without changing the model at all. select a + Number input and drag it to the canvas. Enter GridSize as the Parameter name and click OK.
Now that we have our user inputs defined, we are ready to add processing steps. All of the processing algorithms are available to you under the Algorithms tab. The first step in our pipeline will be to reproject the base layer to the Project CRS. Search for Reprojectlayer algorithm and drag it to the canvas.
Note
The necessity of this reprojection step will become clear shortly. The grid generation algorithm requires us to specify the extent of the grid in the unit of the Project CRS. We can supply this reprojected layer to compute this extent.
In the Reproject layer dialog, select BaseLayer as the Input layer. Check the Use project CRS as the Target CRS. Click OK.
In the Processing Modeler canvas, you will notice a connection appear between the + Base Layer input and the Reproject layer algorithm. This connection indicates the flow of our processing pipeline. Next step is to create a hexagonal grid. Search for the Creategrid algorithm and drag it to the canvas.
In the Generate grid dialog, choose Hexagon(polygon) as the Grid type. Select Extentof'Reprojected'fromalgorithm'ReprojectLayer' as the Grid extent. Click the 123 button under the Horizonal spacing label and choose Model input.
Select GridSize input for Using model input. Repeat the same process for Vertical Spacing. Click OK.
At this point, we have a global hexagonal grid. The grid spans the full extent of the base layer, including land areas and places where there are no points. Let’s filter out those grid polygons where there are no input points. Search for Extractbylocation algorithm and drag it to the canvas.
For Extract features from, select 'Grid'fromalgorithm'GenerateGrid', Where the features (geometric predicate) as Intersect and By compating to the features from as Inputpoints. Click OK.
Now we have only those grid polygons that contain some input points. To aggregate these points, we will use Countpointsinpolygon algorithm. Search and drag it to the canvas.
Select 'Extracted(location)'fromalgorithm'Extractbylocation' as the value for Polygons. The Points layer would be InputPoints. At the bottom, name the Count output layer as Aggregated. Click OK.
The model is now complete. Click the Save button.
Switch to the main QGIS window. You can find your newly created model in the Processing Toolbox under Models ‣ projects ‣ piracy_hexbin. Now it is time to run and test the model. As our goal is to aggregate the input points over hexagonal grids, it is important that the grids are generated using a equal-area projection. This will ensure that regardless of the location of the grid, it will cover exactly the same area. Our model doesn’t explicitely ask for a CRS, but uses whatever CRS is set as the Project CRS. Let’s choose a global equal area projection as the Project CRS. Go to Project ‣ Properties.
In the Project Properties dialog, switch to the CRS tab. We will use a global Mollweide projection for this exercise which is a equal area projection. Search for Mollweide in the Filter box and select World_MollweideEPSG:54009 as the CRS. Click OK.
You will see the layers getting reprojected on-the-fly to the selected CRS. Locate the piracy_hexbin model in the Processing Toolbox and double-click it.
Our Base Layer is the ne_10m_land and the Input Points layer is ASAM11OCT18. The Grid Size needs to be specified in the units of the selected CRS. The World_Mollweide CRS unit is meters, so we specify 100000 m (100 Kms) as the Grid Size. Click Run to start the processing pipeline. Once the process finishes, click Close.
You will see a new layer Aggregated loaded as the result of the model. As you explore, you will notice the layer contains an attribute called NUMPOINTS containing the number of piracy incidents points contained within that grid feature. Let’s style this layer to display this information better. Right-click the Aggregated layer and select Properties.
Switch to the Symbology tab. Select Graduated symbology and NUMPOINTS as the Column. Click Change.. next to Symbol label.
Select Simple fill symbol and check the Transparent Stroke button under Stroke color. This is to make the hexagon edges transparent.
Click the dropdown next to Color ramp and select the Viridis ramp. Click the dropdown again and select Invert Color Ramp to reverse the order of color.
The Graduated symbology will divide the values in the selected column into distinct classes and assign a different color to each of the classes. Select NaturalBreaks(Jenks) as the Mode and click Classify and click OK.
Back in the main QGIS window, turn off the ASAM11OCT18 layer. You will see a nice visualization of piracy hotspots across the globe.
Now that you have encoded the full data pipeline in the model, it is easy to reproduce your results. A model also allows you to experiment quickly without manually repeating each intermediate step every time. If your inputs change over time, say an updated database of piracy is released after a few months, you can run your model on that input to generate a similar visualization without having to remember each step.
