.. _bookmarks:

# Bookmarks for data sources

A bookmark identifies a resource that can be used as an input layer in a RiskScape model.
Imagine your file system is a book - your bookmarks tell RiskScape what to use and how to use it.

Bookmarks let you specify additional details about the input data, such as its CRS,
and can instruct RiskScape to transform the input data to better suit your model.

.. tip::
    For a hands-on walk-through on creating your own bookmarks, see the :ref:`project-tutorial` tutorial. 

.. _bookmark_examples:

### Documented examples

Some input data sources can be quite specific to configure.
The RiskScape documentation contains examples of using some of these more complicated input data sources, such as:

- :ref:`arcgis`
- :ref:`coverage_ref`, i.e. GeoTIFF and ASC files
- :ref:`combining_layers`
- :ref:`csv`
- :ref:`NetCDF <netcdf>`
- :ref:`OpenQuake <openquake>`
- :ref:`PostGIS <postgis>`
- :ref:`shakemaps`
- :ref:`WFS <wfs>`, including a Koordinates example

## Supported formats

The `bookmark format` commands command can be used to find out more about the formats that are supported by RiskScape,
as well as the bookmark parameters supported by those formats.

```none
riskscape format list
riskscape format info FORMAT_NAME
```

For example, to find all the parameters you can use with a `.shp` file bookmark,
you would use `riskscape format info shapefile`.

.. note::
    These commands will list the formats that your current installation of RiskScape will support.
    Sometimes additional formats, such as NetCDF or HDF, can be supported by enabling RiskScape plugins.
    See :ref:`plugins` for more on what RiskScape plugins exist and how to enable them.

## Defining bookmarks

Bookmarks are specified using a INI file format that looks like this:

```ini
[bookmark id1]
description = description1
location = bookmark1.shp

[bookmark id2]
description = description2
location = bookmark2.shp
```

### Common bookmark parameters

description:
    _Optional_. Allows for a description of the bookmarked resource.

location:
    _Required_. The location of the bookmarked resource.
    Often this will be a file that can be specified either as a _relative_ path from the `project.ini` file (e.g. `Data\cool.shp`),
    or as an _absolute_ path (e.g. `C:\Users\JDoe\Proj101\Data\cool.shp`).
    Remote locations, such as `https://` links are also supported.

format:
    _Optional_. Specifies the format of the bookmarked resource.
    For common formats, RiskScape can infer the format based on the file extension.

`crs-name`:
    _Optional_. Can be used to set the projection of the bookmarked resource. E.g. `crs-name = EPSG:2193`.
    Note that this does **not** reproject the data.
    It specifies the CRS that the geometry coordinates are currently in, not what CRS you would _like_ them to be in.

`crs-longitude-first`:
    _Optional_. See :ref:`crs-lat-long` for more details of when to use this setting.

`requires-secret`:
    _Optional_. Indicates that a secret of the given framework will be required to use this bookmark.

type:
    _Optional_. This *coerces* the data in the bookmark resource to match the specified type.
    This can be useful when you want the input data to match a common data format, or schema.

.. _bookmarks-mapping-attributes:

### Mapping attributes

RiskScape can manipulate the attributes in the underlying data as it reads it.
This can include changing the attribute name, transforming the value (e.g. converting from feet to metres),
or changing the type that the data has (such as `Integer` instead of `Text` string).

Consider the following example bookmark: 

```ini
[boomark example]
location = example.csv
crs-name = EPSG:4326
set-attribute.the_geom = geom_from_wkt(WKT)
set-attribute.another_geom = create_point(lat, long)
set-attribute.construction_type = constr_1
```

This reads data from an `example.csv` and does the following:
* Creates a `the_geom` geometry attribute based on the `WKT` CSV column, which stores geometry in the Well Known Text (WKT) format.
* Creates a `another_geom` geometry attribute based on the `lat` and `long` columns in the CSV file.
* Renames the `constr_1` CSV column to be a `construction_type` attribute.

.. note::
  The ordinate order in ``create_point()`` and ``geom_from_wkt()`` is determined by
  the axis order of the EPSG projection set by ``crs-name``.
  The axis order can be flipped from *lat,long* to *long,lat*
  using the ``crs-longitude-first`` setting.

.. tip::
    The ``map-attribute`` option works the same as ``set-attribute``, but it must only be used when a ``type`` is set for the bookmark.

### Types

A :ref:`RiskScape type <types>` can be used to define the underlying shape of the data.
For common situations, like dealing with input data from a shapefile, you would not normally need to define your own types.
Defining a `type` for the bookmark may be useful in the following cases:

- You want to 'remove' attributes from the input data so certain attributes are not included in your model results.
- You want to make sure that the underlying data conforms to specific types. For example, a certain attribute may always need to be an integer (whole number).
- The underlying shapefile contains null values for certain attributes.
  The model may handle these null values better if they are defined as :ref:`nullable <type-nullable>`, i.e. `nullable(integer)` instead of just `integer`.

.. tip::
    You can also use RiskScape to define highly-specific types.
    You can use ``range`` to ensure that numbers fall within a desired range, or use ``set`` to ensure that data values are coded correctly.
    For example, defining a ``set('Timber', 'Concrete', 'Steel')`` type will throw out a warning or error if it encounters a typo like ``'Cnocrete'`` or ``'Timbre'``.

## OGR bookmarks

.. note::
  OGR bookmarks rely on GDAL being installed and configured correctly.

  Because of this support is provided in an optional plugin which can
  be enabled by copying `${RISKSCAPE_INSTALL_DIR}/plugins-optional/ogr.jar`
  to `${RISKSCAPE_HOME}/plugins/ogr.jar`

OGR is a suite of applications created by [GDAL](https://gdal.org) that may
be used to access a variety of [Vector data formats](https://gdal.org/ogr_formats.html)
including:

* [PostgreSQL](https://gdal.org/drv_pg.html)
* [ESRI File Geodatabase](https://gdal.org/drv_openfilegdb.html)

.. tip::
    It may be simpler to use RiskScape's built-in :ref:`PostGIS <postgis>` support,
    rather than using OGR to read from PostGIS databases.

The actual formats available will depend on the GDAL installation on the target system.

.. note::
  The ``ogrinfo`` command may be used to list available formats with
  ``ogrinfo --formats``.

  ``ogrinfo`` may also be used to inspect the OGR source.

### Additional bookmark parameters

#### `ogr-format` (required)

Specifies which OGR driver is to be used. This is the *Format Name* from
[Vector data formats](https://gdal.org/ogr_formats.html).

Example:
```ini
ogr-format = PostgreSQL
```

#### `ogr-source` (optional)

Many OGR formats require a connection string that is not a file, such as the PostgreSQL
example below.

In these cases `ogr-source` is required to contain the required connection string
and the bookmark *location* is ignored. In these cases it is recommended to set
`location = ogr:source`.

Refer to the OGR Driver documentation for the required `ogr-source` format.

Example:
```ini
ogr-source = PG:dbname='riskscape' host='riskscape-postgis' port=5432 user='riskscape' password='riskscape'
```

#### layer (optional)

Specifies the specific layer that is to be used.

Required when the bookmarked resource contains multiple layers, such as a database.

Example:
```ini
layer = tablename
```

### Windows

On Windows OGR can be installed from [OSGeo4w](https://trac.osgeo.org/osgeo4w/) following the
*Quick Start for OSGeo4W Users* instructions.

This will install the OGR tools to `C:\OSGEO4W64\bin` which must be added to the PATH
environment variable to make it available to RiskScape.

RiskScape will also require the GDAL library name to to set with JAVA_OPTS. This can be done
in PowerShell using:

```ini
$Env:JAVA_OPTS = "-DGDAL_LIBRARY_NAME=gdal204"
```

The actual name should reflect name of the `gdalxyz.dll` file in `C:\OSGEO4W64\bin`.

### Linux

On Debian Linux systems OGR can be installed with:

```none
sudo apt install gdal-bin libgdal-java
```
