.. _project-tutorial:

# RiskScape projects and bookmarks

## Before we start

This tutorial is aimed at new users who want to start creating their own projects.
Projects need to be setup first, before you can build and run your own risk models in RiskScape.
We expect that you:

- Have completed the :ref:`wizard-how-to` guide and are familiar with building and running a RiskScape model.
- Have a basic understanding of geospatial data and risk analysis.

The aim of this tutorial is to get you familiar with creating projects and bookmarks in RiskScape,
so that you can start building models on your own.

## Getting started

### Setup

Click [here](../project-tutorial.zip) to download the example project we will use in this guide.
Unzip the file into the :ref:`top_level_dir` where you keep your RiskScape projects.
For example, if your top-level projects directory is `C:\RiskScape_Projects\`,
then your unzipped directory will be `C:\RiskScape_Projects\project-tutorial`.

Open a command prompt and `cd` to the directory where you unzipped the files, e.g.

```none
cd project-tutorial
```

You will use this command prompt to run the RiskScape commands in this tutorial.

The unzipped project contains a few sub-directories:
- `project-tutorial\data` contains the input data files we will use in this tutorial.
  This data is similar to the :ref:`Upolu tsunami data <Upolu_data>` that we used in the previous tutorials.
- `project-tutorial\models` contains a pre-built model we will use to test our project as we go along.

.. note::
    This input data was provided by `Earth Sciences New Zealand <https://earthsciences.nz/>`_, as well as the `PCRAFI <http://pcrafi.spc.int/>`_
    (Pacific Risk Information System) website.
    The data files have been adapted slightly for this tutorial.

There is also an initial `project-tutorial\project.ini` file that we will modify.
Open this `project.ini` file in Notepad (or your preferred text editor).

## Background

### Project INI files

You may have noticed from previous tutorials that RiskScape gets all its configuration
information from a `project.ini` file.
This tells RiskScape things like what models can be run, and what input data should be used in the models.

The `project.ini` file is in the [INI format](https://en.wikipedia.org/wiki/INI_file)
and can be modified in any plain-text editor, such as Notepad or `gedit`.

INI files contain key-value pairs, which are organized into sections.
Square brackets are used to indicate the start of a section.
A simple INI section might look something like this:

```ini
[section my-id]
key-one = some value
key-two = 2.0
```

In the previous tutorials, we have used INI files to save our model's parameters.
For example:

```ini
[model basic-exposure]
description = Simple example of a RiskScape model
framework = wizard
input-exposures.layer = data/Buildings_SE_Upolu.shp
input-hazards.layer = data/MaxEnv_All_Scenarios_50m.tif
sample.hazards-by = CLOSEST
analysis.function = is_exposed
```

Here the section starts with `model`, indicating that we are defining a RiskScape model,
followed by the ID of the model (`basic-exposure`).
The lines that follow store the settings for the model's parameters as key-value pairs.

In addition to models, the `project.ini` also stores details about what goes into the models.
These are:
- The input data files to use, which RiskScape calls *bookmarks*.
- Risk *functions* that will determine the impact the hazard has on each element-at-risk
  (e.g. the probability of building damage).

This tutorial will look at how to configure bookmarks in more detail.
We will look at creating our own risk functions in the next tutorial.

.. tip::
    The idea behind the ``project.ini`` file is that it provides a way to organize your RiskScape models,
    much like a work-space, so that you can keep related models (i.e. ones that use similar data or functions) together.
    Completely unrelated models can go in a separate ``project.ini`` file in another directory.

## Bookmarks

A RiskScape bookmark identifies a file that can be used as an input layer in a model.
Imagine your file system is a book - your bookmarks tell RiskScape what to use and how to use it.

### A simple bookmark

Let's look at a simple example. Add the following to your `project.ini` file.

```ini
[bookmark Samoa_electoral_boundaries]
location = data/Samoa_constituencies.shp
```

Each RiskScape bookmark has an ID, which is the text that follows `[bookmark ...]`.
In this case, the bookmark ID is `Samoa_electoral_boundaries`.
All RiskScape bookmarks must also have a `location`, which specifies the input data to read.

.. note::
    Your bookmark's ID can contain spaces, e.g. ``[bookmark cool file]``.
    However, this makes some RiskScape commands slightly harder to use.
    You will need to enclose the bookmark ID in double-quotes when you use it on the command line, e.g.
    ``riskscape bookmark info "cool file"``

Save the `project.ini` file and enter the following command in your terminal to check that RiskScape now knows
about the new bookmark.

```none
riskscape bookmark list
```

You should see output similar to the following:

```none
+--------------------------+-----------+----------------------------------------------------------------------------+
|id                        |description|location                                                                    |
+--------------------------+-----------+----------------------------------------------------------------------------+
|Samoa_electoral_boundaries|           |file:///C:/RiskScape_Projects/project-tutorial/data/Samoa_constituencies.shp|
+--------------------------+-----------+----------------------------------------------------------------------------+
```

.. tip::
    You can add an optional ``description`` key for most things in the ``project.ini`` file.
    The description is purely to help you keep track of what each model/bookmark/function does.
    You can also add comments to the INI file by using ``#`` at the start of the line.

Enter the following command into your terminal to view more detailed information about the bookmark:

```none
riskscape bookmark info Samoa_electoral_boundaries
```

You should see output similar to the following:

```none
"Samoa_electoral_boundaries"
  Description :
  Location    : file:///C:/RiskScape_Projects/project-tutorial/data/Samoa_constituencies.shp
  Attributes  :
    the_geom[MultiPolygon[crs=EPSG:4326]]
    fid[Integer]
    NAME_1[Text]
    Region[Text]
  Axis-order  : long,lat / X,Y / Easting,Northing
  CRS code    : EPSG:4326
  CRS (full)  : GEOGCS["WGS 84",
  DATUM["World Geodetic System 1984",
    SPHEROID["WGS 84", 6378137.0, 298.257223563, AUTHORITY["EPSG","7030"]],
    AUTHORITY["EPSG","6326"]],
  PRIMEM["Greenwich", 0.0, AUTHORITY["EPSG","8901"]],
  UNIT["degree", 0.017453292519943295],
  AXIS["Geodetic longitude", EAST],
  AXIS["Geodetic latitude", NORTH],
  AUTHORITY["EPSG","4326"]]
  Summarizing...
  Row count   : 43
  Bounds      : EPSG:4326 [-172.8041 : -171.3977 East, -14.0772 : -13.4398 North] (original)
```

This output is quite technical-looking, but it tells us a few useful things:
- The attributes that are present in the input data, i.e. `the_geom`, `fid`, `NAME_1`, and `Region`.
  It also shows us what *type* of data each attribute holds, e.g. `fid` is an `Integer` whereas `NAME_1` is a `Text` string.
- The [Coordinate Reference System](https://en.wikipedia.org/wiki/Spatial_reference_system) (CRS) (i.e. EPSG:4326 or WGS 84) and axis-order (i.e. `long,lat`) of the geometry.
- The number of rows of data the file holds (i.e. `Row count`).
- The geographic bounds of the data.

.. note::
    The CRS is important part of the input data, which we will learn more about.
    Conveniently, the CRS information for shapefiles is already all defined in a ``.prj`` file (i.e. ``Samoa_constituencies.prj``),
    so we don't have to worry about specifying a CRS for the bookmark.

### Manipulating the input data

The main benefit of bookmarks is that they tell RiskScape *how* to load the data into the model.
This means you can change the data on the fly, as it gets loaded into the model, instead of modifying
the geospatial file directly.

When you are working with Shapefiles, GeoTIFFs, and [ESRI Grid](https://en.wikipedia.org/wiki/Esri_grid) (i.e. `.asc`) files,
most of what RiskScape needs to know is already encoded into the file format.
However, even with these file formats, bookmarks still allow you to manipulate the input data in useful ways.

Let's look at a example of this in action.
Your project comes with a `exposure-by-region` model, which is already defined in the `models/models_exposure-by-region.ini` file:

```ini
[model exposure-by-region]
framework = wizard
description = Produces a total count of buildings in each region exposed to tsunami inundation
input-exposures.layer = data/Buildings_SE_Upolu.shp
input-exposures.geoprocess = false
input-hazards.layer = data/MaxEnv_All_Scenarios_50m.tif
input-areas.layer = data/Samoa_constituencies.shp
input-areas.geoprocess = false
sample.hazards-by = CLOSEST
analysis.function = is_exposed
report-event-impact.filter = consequence = 1
report-event-impact.group-by[0] = area
report-event-impact.aggregate[0] = count(*) as Exposed_buildings
report-event-impact.select[0] = area.Region as Region
report-event-impact.select[1] = Exposed_buildings
```

This model counts the number of exposed buildings by region,
similar to models we have used in previous tutorials.

Run this model now, by entering the following command:

```none
riskscape model run exposure-by-region
```

It should produce a `output/exposure-by-region/TIMESTAMP/event-impact.csv` results file,
where `TIMESTAMP` is the current date/time, e.g. `2022-01-13T17_38_2`.
We can use the `more "FILENAME"` command to quickly look at a text file's contents from the terminal,
such as the `event-impact.csv` file produced here, e.g.

```none
more "output/exposure-by-region/TIMESTAMP/event-impact.csv"
```

.. tip::
    Forward slashes in file-paths generally work OK in the Windows Command Prompt, as long as
    you surround them in double-quotes, e.g. ``"output/some-file.csv"``.
    This means you can copy-paste the results filename from the URI that RiskScape displays.
    Simply select the text and use ``Ctrl`` + ``c`` and ``Ctrl`` + ``v`` to copy-paste in the Windows
    Command Prompt.

The `event-impact.csv` file should contain the following:

```none
Region,Exposed_buildings
,10
Aleipata Itupa i Lalo,526
Aleipata Itupa i Luga,340
Falealili,749
Lepa,288
Lotofaga,146
```

Now let's say we wanted a slightly different regional breakdown of the results.
By default, the model uses the `Samoa_constituencies.shp` file as the *area-layer*.
However, the area-layer is just a parameter to the model, so RiskScape will let us replace the parameter with a different file.

Try running the following command to use the `data/ws_districts.shp` file as our area-layer.

```none
riskscape model run exposure-by-region -p "input-areas.layer=data/ws_districts.shp"
```

This time, instead of running our model, RiskScape gives us an error:

```none
There was a problem with the parameters for wizard model
  - Failed to load the saved model. Some parameters specified may be invalid. If you have
    altered parameters manually, try going through the interactive wizard again
    - Could not apply 'report-event-impact.select' parameter as it results in an invalid pipeline
      - Failed to validate 'select({area.Region as Region, Exposed_buildings})' step ...
        - Failed to validate expression '{area.Region as Region, Exposed_buildings}' ...
          - Attribute 'Region' (at column 15) does not exist, available attributes are: [District, fid, the_geom]

```
#### Troubleshooting RiskScape errors

RiskScape errors are often nested like this.
The top problem describes the high-level operation that failed,
and the subsequent problems then drill-down into more and more specific context about what went wrong.

Let's look at these errors in more detail and try to work out what went wrong:
- The first error tell us there was a problem loading the saved model, possibly related to the model parameters that we used.
- The next error says the problem was specifically with the `report-event-impact.select` parameter.
  We didn't actually change that parameter at all. In our model, that parameter looks like this:

    ```ini
    report-event-impact.select[0] = area.Region as Region
    ```

- The next two errors specify the *pipeline step* and *expression* that failed.
  We will learn more about these concepts in subsequent tutorials.
- The final error tells us that the `Region` attribute does not exist.
  Only the `District`, `fid`, and `the_geom` attributes are present in the model.

So, what went wrong?
The attributes that are available in a RiskScape model depend on what input data the model uses.
In this case, it appears that our original area-layer has a `Region` attribute,
but our new area-layer does not.

Let's confirm this by taking a closer look at our new area-layer.
Enter the following command:

```none
riskscape bookmark info "data/ws_districts.shp"
```

You can see from the output that the file does not contain a `Region` attribute,
although it does have a `District` attribute instead, i.e.

```
Location : file:///C:/RiskScape_Projects/project-tutorial/data/data/ws_districts.shp
  Attributes  :
    the_geom[MultiPolygon[crs=EPSG:4326]]
    fid[Integer]
    District[Text]
...
```

.. tip::
    In many cases, bookmarks and file paths can be used interchangeably in RiskScape.
    For example, here we passed a file path directly to the ``riskscape bookmark info`` command.
    This means you can use file paths as model parameters without necessarily creating bookmarks.

#### Consistent input data

In order to reuse the same model with different input files,
some attributes in the input data (in this case, the `Region` attribute) will need to be consistent across the files.

The naive approach would be to manually rename the attribute in the input data, and re-save the shapefile.
However, this can be cumbersome and error-prone if you need to do it often.

RiskScape bookmarks can solve the problem for us.
Let's create a _new_ bookmark for this second area-layer shapefile.
Add the following to your `project.ini` file and save it.

```ini
[bookmark Samoa_districts]
location = data/ws_districts.shp
set-attribute.Region = District
```

The last line is setting a new attribute called `Region`, which will hold whatever value is in the `District` attribute.
Enter the following command to see what the bookmark data looks like now:

```none
riskscape bookmark info Samoa_districts
```

You should see that there is now a new `Region` attribute in the output.
The original `District` attribute is still also present.

```none
"Samoa_districts"
  Description :
  Location    : file:///C:/RiskScape_Projects/project-tutorial/data/data/ws_districts.shp
  Attributes  :
    the_geom[MultiPolygon[crs=EPSG:4326]]
    fid[Integer]
    District[Text]
    Region[Text]
...
```

Now enter the following command to use our new bookmark in the model.

```none
riskscape model run exposure-by-region -p "input-areas.layer=Samoa_districts"
```

This time the model runs successfully because all the attributes it needs are present in the input data.

.. note::
    In this case we simply copied an existing attribute in the input data,
    but you can manipulate the data in more complicated ways.
    For example, you could convert imperial units into the metric system using:
    ``set-attribute.metres = feet / 3.281``

.. _bookmark_filter:

### Filtering

Let's just take a quick look at the `event-impact.csv` results file that the last `riskscape model run` command produced.
Use `more "output/MODEL/TIMESTAMP/event-impact.csv"` to look at the results, e.g.

```none
more "output/exposure-by-region/2022-01-13T17_38_25/event-impact.csv"
Region,Exposed_buildings
Aleipata Itupa i Lalo,507
Aleipata Itupa i Luga,339
Falealili,749
Lepa,283
Lotofaga,146
Marine Area,35
```

If you look carefully, you will notice there is a 'Marine Area' region now present in the results.
Our model now thinks some buildings are located in the sea, which is not ideal.

Often area-layer shapefiles will contain polygons that denote bodies of water, however,
we generally want to ignore these areas in our model.

Bookmarks also let us *filter* the input data so that only certain rows of data are included in the model.
We can specify a true/false condition, and only input data that satisfies that condition will be used in the model.

In your `project.ini` file, add the following line to your `Samoa_districts` bookmark, and save the file.

```none
filter = Region != 'Marine Area'
```

Your bookmark should now look like this:

```ini
[bookmark Samoa_districts]
location = data/ws_districts.shp
set-attribute.Region = District
filter = Region != 'Marine Area'
```

.. note::
    We are using a ``!=`` condition here, because we want to *exclude* a specific row of data,
    i.e. include everything *except* the 'Marine Area' row of data.

Now try using the updated area-layer bookmark in your model by running the following command:

```none
riskscape model run exposure-by-region -p "input-areas.layer=Samoa_districts"
```

Take a look at the `event-impact.csv` file that the model produces. It should look like this:

```
Region,Exposed_buildings
,19
Aleipata Itupa i Lalo,518
Aleipata Itupa i Luga,341
Falealili,749
Lepa,286
Lotofaga,146
```

The 'Marine Area' is no longer present in the results, although we do have 19 buildings that were not matched to any region now.

If you look carefully, you will notice that 35 buildings were previously matched to the 'Marine Area',
but now only 19 buildings have no region. This is because some buildings (16) were straddling a regional boundary.

We use 'closest' spatially matching for the area-layer.
When a building intersects *two* regions, we assign it to the region that's closest to the building's centroid.
When we removed the 'Marine Area', it meant that 16 buildings now only intersected _one_ region instead of two.

We could potentially use the `sample.areas-buffer` model parameter here to assign *all* buildings to a region,
like we did in the previous tutorial.

.. tip::
    The bookmark ``filter`` parameter essentially works the same as the 'filter' *geoprocessing* option in the wizard.
    Using the wizard can make it easier to build filter expressions.

### Problematic input data

Dealing with real world data can sometimes be a little messy.
Let's look at some examples of how RiskScape deals with problematic data.

In the `data/` sub-directory, there is also a `problematic.shp` file.
Try run the following command to use it as the model's area-layer.

```none
riskscape model run exposure-by-region -p "input-areas.layer=data/problematic.shp"
```

You should see an error message like this:

```none
15:29:14.642 [main] WARN  n.o.r.e.d.r.FeatureSourceBookmarkResolver - No crs could be parsed
  for feature source from file:///C:/RiskScape_Projects/project-tutorial/data/problematic.shp,
  falling back to generic 2d
There was a problem with the parameters for wizard model
  - Failed to load the saved model. Some parameters specified may be invalid.
    If you have altered parameters manually, try going through the interactive wizard again
    - Could not apply 'input-areas.layer' parameter as it results in an invalid pipeline
      - Could not apply the answer to the 'input-areas.layer' parameter to your model
        - The given Geom type does not contain the required spatial meta-data (i.e. CRS). This
          could be because the input data comes from a CSV file and 'crs-name' needs to be set
          in the bookmark
```

The error tells us that RiskScape could not read the CRS information for this shapefile.
If you look closely at the `data/` sub-directory, you will see that the `.prj` file
that contains all the shapefile's CRS information is actually missing, i.e. there is no `problematic.prj` file.

.. tip::
    In Windows Command Prompt, you can use the ``dir`` command to get a list of any
    matching files in a directory, e.g. ``dir data\problematic.prj``

Let's try doing what the error suggests and create a bookmark with `crs-name` set.
We know the CRS for this file _should_ be [EPSG:4326](https://epsg.io/4326), or WGS 84,
so add the following to your `project.ini` file and save it.

```ini
[bookmark problematic]
location = data/problematic.shp
crs-name = EPSG:4326
```

Now, try running the following command to use the new bookmark in the model:

```none
riskscape model run exposure-by-region -p "input-areas.layer=problematic"
```

This time the model runs to completion.
However, we still see some warnings about invalid input data displayed:

```
WARNING: An invalid row of input data has been skipped
  - An invalid geometry which cannot be fixed automatically has been detected. Caused by:
    Invalid Coordinate at or near point (NaN, -172.03240134903). Refer to the Geometry
    reference in the RiskScape documentation for tips on how to avoid this. The row
    containing this geometry was: {fid=999, Region=Bad geo…}

WARNING: Problems found with 'problematic' bookmark in location
  file:///C:/RiskScape_Projects/project-tutorial/data/problematic.shp
  - Invalid geometry has been detected and fixed automatically. Refer to the Geometry
    reference in the RiskScape documentation for tips on how to avoid this. The record
    containing this geometry was: {fid=1, Region=Marine …}
```

These warnings tell us that RiskScape encountered invalid geometry in the input data.

The first message tells us that a row of input data was _skipped_ because it contained
invalid geometry. This means that this particular row of input data was omitted from our model.

The second message also deals with invalid geometry, but this time RiskScape _fixed_ the
geometry for us and continued to use it in the model.

.. note::
    Under the *Reference Guides* in RiskScape's documentation, there is a page on Geometry that
    contains more details about :ref:`invalid-geometry`.

If you wanted to, you can control what RiskScape does in these situations using bookmark parameters:
- The `skip-invalid` bookmark parameter determines what RiskScape should do when an invalid row of input data
  is detected.
  By default, the invalid row is simply skipped and RiskScape continues, but this can be changed so that
  the `riskscape model run` command stops with an error by using `skip-invalid = false`.
- `validate-geometry` controls whether or not RiskScape validates geometry and attempts to fix it.

.. tip::
    The default bookmark settings *should* be sufficient for most modelling, so you
    shouldn't need to worry too much about changing these bookmark parameters.

### Using CSV data

Let's try another bookmark example.
This time we will replace the model's *exposure-layer*.

We have a `data/Buildings_SE_Upolu_centroids.csv` Comma Separated Values (CSV) file that contains building centroid data for south-eastern Upolu.
If you use the `more` command to look at this file, it contains data that looks like the following:

```none
more "data/Buildings_SE_Upolu_centroids.csv"
ID,Use_Cat,Cons_Frame,lat,long
1610,Residential,Masonry,-14.01740536257795,-171.7205566517351
1609,Outbuilding,Masonry,-14.018577318447557,-171.72027568834153
1614,Residential,Masonry,-14.016838008974906,-171.72055898904915
1615,Residential,Masonry,-14.016665519919313,-171.72080582923437
1616,Residential,Masonry,-14.016369557887204,-171.7209374846784
...
```

Try using this CSV file in the model using the following command:

```none
riskscape model run exposure-by-region -p "input-exposures.layer=data/Buildings_SE_Upolu_centroids.csv"
```

Note that the model ran successfully, even though we are using a *CSV file* as input,
instead of geospatial data, like Shapefile.
This is because RiskScape supports automatic type-detection for CSV data.
RiskScape can detect that the `lat` and `long` columns in the CSV file are WGS84 coordinates,
and automatically turn the coordinates into geometry that can be spatially matched against the
other input layers.

RiskScape's type-detection has limitations - it currently only supports WGS84 lat, long coordinates.
Try running the following command, which uses a CSV file containing the exact same building data
*except* that the coordinates are in Well-Known-Text (WKT).

```
riskscape model run exposure-by-region -p "input-exposures.layer=data/Buildings_SE_Upolu_WKT.csv"
```

You should see an error message like the following, saying that RiskScape could not find
any geometry attribute in the given input data:

```
There was a problem with the parameters for wizard model
  - Failed to load the saved model. Some parameters specified may be invalid.
    If you have altered parameters manually, try going through the interactive wizard again
    - Could not apply 'input-exposures.layer' parameter as it results in an invalid pipeline
      - Could not apply the answer to the 'input-exposures.layer' parameter to your model
        - Geometry attribute required but none found in {WKT=>Text, ID=>Floating, Use_Cat=>Text, Cons_Frame=>Text}
```

.. note::
    Turning CSV data into geometry manually can be a bit fiddly to do in a bookmark,
    so we won't look at it in detail here.
    If you do need to load geometry data manually, the :ref:`csv` reference page will 
    walk you through the process.

### Testing bookmarks

RiskScape bookmarks manipulate your input data *on the fly*, as it gets loaded into the model.
However, it can sometimes be helpful to see what your input data actually looks like
*after* RiskScape has modified it.

Using the `riskscape bookmark evaluate BOOKMARK_NAME` command will produce a GeoPackage that
contains all the changes that your bookmark applies to the input data.
This GeoPackage can then be easily viewed in your preferred GIS application.

As we have covered earlier, bookmark IDs and files can often be used interchangeably in RiskScape commands.
So you can run the following command to turn the CSV building data into a shapefile that you
can view more easily in a GIS application:

```none
riskscape bookmark evaluate "data/Buildings_SE_Upolu_centroids.csv"
```

### Bookmark formats

How RiskScape loads input data depends on the file _format_ that the data is in.
In our bookmark examples so far, RiskScape has determined the file format based on the file extension.
However, we can use the `format` parameter to specify explicitly what file format the data is in.

Try adding the following bookmark to your `project.ini` file and save it.

```ini
[bookmark Te_Araroa]
description = An online map of the Te Araroa trail, NZ
location = https://opendata.arcgis.com/api/v3/datasets/330fe731ff444471a45d88d8b681e53d_0/downloads/data?format=geojson&spatialRefId=4326
format = geojson
```

This hyperlink points to a map of the [Te Araroa](https://www.teararoa.org.nz) walking trail, in GeoJSON format.
RiskScape can download remote data and use it in a model, however, we need to explicitly set the bookmark's `format` in this case.

Check that RiskScape can load the bookmark's data by running the following command:

```none
riskscape bookmark info Te_Araroa
```

It should display output similar to the following:

```none
"Te_Araroa"
  Description : An online map of the Te Araroa trail, NZ
  Location    : https://opendata.arcgis.com/api/v3/datasets/330fe731ff444471a45d88d8b681e53d_0/downloads/data?format=geojson&spatialRefId=4326
  Attributes  :
    geometry[Geom[crs=EPSG:4326]]
    OBJECTID[Integer]
    SEQUENCE[Integer]
    STATUS[Text]
    LENGTH[Floating]
    NAME[Text]
    ISLAND[Text]
    LEGALSTAT[Text]
    complete[Integer]
    Notes[Text]
    Fromkm[Floating]
    Tokm[Floating]
    category[Integer]
    Cycle[Integer]
    walkid[Integer]
    mapName[Text]
    link[Text]
    editor[Text]
    create_dt[Text]
    last_editor[Text]
    last_edit_dt[Text]
    SHAPE_Length[Floating]
  Axis-order  : long,lat / X,Y / Easting,Northing
  CRS code    : EPSG:4326
  CRS (full)  : GEOGCS["WGS84",
  DATUM["WGS84",
    SPHEROID["WGS84", 6378137.0, 298.257223563]],
  PRIMEM["Greenwich", 0.0],
  UNIT["degree", 0.017453292519943295],
  AXIS["Geodetic longitude", EAST],
  AXIS["Geodetic latitude", NORTH],
  AUTHORITY["Web Map Service CRS","84"]]
  Summarizing...
  Row count   : 482
  Bounds      : EPSG:4326 [167.8103 : 175.6674 East, -46.6253 : -34.4267 North] (original)
```

#### Supported formats

The file format can affect what bookmark parameters RiskScape will accept.
For example, a shapefile bookmark will support some parameters that cannot be used with a GeoTIFF bookmark.

To see a list of supported input formats, use the command:

```none
riskscape format list
```

To see what parameters a particular bookmark format supports, use the command:

```none
riskscape format info FORMAT_NAME
```

### Importing bookmarks

You can define your model components (i.e. bookmarks, functions, etc) across several
different INI files, and then "import" these INI files into your project.
This lets you easily reuse common bookmarks and functions across several different RiskScape projects.
For example, you will probably use the same area-layer data any time you need to do regional aggregation in a model.

To import RiskScape definitions from another INI file you simply use `import =` and
then the path to the INI file.
This needs to go in a `[project]` section at the top of your `project.ini` file, i.e.

```ini
[project]
import = /path/to/INI/file
```

The RiskScape Engine is 'data agnostic', in that it does not come with any
exposure, region, or hazard layers "baked in" to the software.
However, some layers that are commonly used for modelling in New Zealand are available
in the following GitHub repository:

- https://github.com/GNS-Science/riskscape/

You could either clone the git repository, download a copy of it, or just point
RiskScape directly at the GitHub repository via a HTTPS link.
We will try doing the latter.
There is a line at the top of your `project.ini` file that looks like this:

```ini
#import = https://raw.githubusercontent.com/GNS-Science/riskscape/refs/heads/main/data/transpower/project.ini
```


The `#` at the start of the line means that it is an INI file comment.
Remove the `#` character to uncomment the line and save your `project.ini` file.
This pulls in bookmarks for the [Transpower Open Data portal](https://data-transpower.opendata.arcgis.com),
which represent the electric power transmission infrastructure in New Zealand.

If you now run the `riskscape bookmark list` command you should now see some new Transpower bookmarks.
Try running the following command to examine one in more detail:

```
riskscape bookmark info Transpower_Substations
```

This bookmark contains the [sub-stations](https://data-transpower.opendata.arcgis.com/datasets/Transpower::sites/about)
in New Zealand's national grid.
The raw source data is in GeoJSON format, and is being accessed via HTTPS.
This data could now easily be used as an exposure-layer in any RiskScape model.

.. note::
    The description in the ``riskscape bookmark info`` contains licensing information 
    (i.e. `Creative Commons <https://creativecommons.org/licenses/by/4.0>`_) about this
    Transpower data.
    You can browse the various `GitHub README files <https://github.com/GNS-Science/riskscape/blob/main/README.md>`_
    to find out more about licensing details, as different open data sources will have different licensing implications.

### Working with remote bookmark data

When a bookmark's location is a HTTPS or WFS link, RiskScape will try to download the data *every* time
the bookmark gets used in a model. Accessing remote data like this makes it easy to share,
but it gets pretty inefficient to use it repeatedly in models, especially for large
datasets like [LINZ building outlines](https://data.linz.govt.nz/layer/101290-nz-building-outlines).

You may recall that the `riskscape bookmark evaluate` effectively 'exports' a RiskScape bookmark's
contents and saves it as a local file. You can use this command to avoid downloading the remote
bookmark data each time you run a model. For example, try running the following command:

```
riskscape bookmark evaluate Transpower_Substations
```

This will create a `output/bookmark-eval/Transpower_Substations.gpkg` file that could then be used
in models instead of the `Transpower_Substations` bookmark. Because `Transpower_Substations.gpkg`
is a local file, it will be more efficient to use in models than the original `Transpower_Substations`
bookmark, which accesses remote data.

.. note::
    Some RiskScape bookmarks, such as for `Koordinates <https://koordinates.com/>`_ WFS data,
    may require you to define a API key in your RiskScape :ref:`secrets_ini` in order to access the data.
    The `GitHub README <https://github.com/GNS-Science/riskscape/blob/main/README.md>`_
    for the relevant data source provide more details on how to do this, if needed.

## Recap

Let's review some of the key points we have covered so far:

- The `project.ini` file holds the bookmarks and functions that the model will use.
- Bookmarks configure the input data that RiskScape models can use.
- The attributes in a RiskScape model correspond to the attributes that are present in the input data.
- All the data in a RiskScape model has _type_ information associated with it.
- Bookmarks let you manipulate the input data *before* it gets used by the model.
- The input data for RiskScape models *always* needs a geometry-type attribute present and a CRS defined.
- File-paths and bookmarks can often be used interchangeably in RiskScape.
  In particular, shapefiles, GeoPackage, GeoTIFF, ESRI Grid, and GeoJSON files generally have
  all the information RiskScape needs, such as the CRS, saved as part of the file format.
- The `riskscape bookmark info` command is a useful way to find our more about
  a file or bookmark, such as the attributes the data contains or its CRS.
- You *always* need to define a bookmark in order to use CSV input data in a model.
  The bookmark will need to define `set-attribute.geom` and `crs-name` for the CSV data.
- RiskScape can do some error-checking on the input data, such as whether the geometry is valid.
- You can use the `riskscape format info` command to find out more about what parameters a bookmark supports.
- There is a [GitHub repository](https://github.com/GNS-Science/riskscape) with open-source examples
  of commonly used bookmarks and functions. A bookmark that accesses remote input data will fetch the
  data each time it's used, so it can be more efficient to store a local copy of the file.

Now that you are familiar with editing a `project.ini` file,
have a go at the :ref:`functions_tutorial` tutorial.

## Extra for experts

If you want to explore bookmarks a little further, you could try the following exercises out on your own.

- Practice adding a `description` to some of the bookmarks you created in the `project.ini` file.
  Try also using `#` to add a few INI file comments.
- Some buildings are not assigned to any region when you run the `exposure-by-region` model.
  Try specifying the `sample.areas-buffer` parameter when you run the model. See if you
  can work out the buffer distance needed to assign _all_ buildings to a region.
  Start off with 100m, 1250m, 500m, 1000m, and so on.
- Read the :ref:`csv` page and see if you can create a bookmark that loads the WKT geometry in
  `data/Buildings_SE_Upolu_WKT.csv`.
