.. _python-functions:

Python functions
======================

RiskScape allows **user defined** Python functions to be used for risk modelling.
This allows RiskScape to be extended by scientists/risk modellers without
needing software developers.

## Basic function

Let's start off with the simplest possible example Python function: a 'Hello World' example:

```python
def function(name):
  return 'Hello, ' + name
```

Python functions can be defined in a `.py` file (e.g. `hello.py`).
The function in this file that RiskScape will execute should always be called `function`.

You then need to add a section to your `project.ini` file that tells
RiskScape how your function can be used.
For example, the following sections loads the `hello.py` function and names it `hello`. 

```ini
[function hello]
description = Simple 'Hello world' function example
location = hello.py
argument-types = [ text ]
return-type = text
```

.. _inline-functions:

### Inline functions

If a function is reasonably short, it can make sense to "inline" it. Instead of 
giving a `location` you can provide a `source`. You will need to 
:ref:`triple quote <multiline-values>` it so that RiskScape recognizes it 
as a multi-line string.   

```
[function hello]
description = Simple 'Hello world' function example
argument-types = [ text ]
return-type = text
framework = jython
source = '''
def function(name):
    return "Hello, " + name
'''
```

.. note:: Though this will work for any size function, we recommend that larger
    functions are kept in their own `.py` files. This will make them easier to 
    organize and test. When using ``source`` to define a function, you will also
    need to specify the ``framework`` as either :ref:`Jython or CPython <jython_vs_cpython>`.

Once you have defined your function, you should see it successfully displayed by the `riskscape function list` command.
For example:

```none
riskscape function list
+-----+-------------------------------------+---------+-----------+----------+
|id   |description                          |arguments|return-type|category  |
+-----+-------------------------------------+---------+-----------+----------+
|hello|Simple 'Hello world' function example|[Text]   |Text       |UNASSIGNED|
+-----+-------------------------------------+---------+-----------+----------+
```

You can then use (or 'call') your function from any RiskScape expression. For example:

```none
riskscape expression evaluate "hello('world')"
Hello, world
riskscape expression evaluate "hello('Ronnie')"
Hello, Ronnie
```

## Types

All the input data that RiskScape processes has :ref:`type information <types>` associated with it,
such as `text`, `integer`, `floating`, `geometry`, etc.
The same also applies to a function's arguments and return value - you need to define what data types
your function accepts before you can use it.

The INI file definition tells RiskScape what types the function expects.
For example, our `hello` function expects a single `text` string argument.
This means we will get an error if we call it with different arguments, such as an integer or multiple text strings.

```none
riskscape expression evaluate "hello(1)"
 Failed to evaluate expression
   - Could not find function hello with arguments [Integer] - function 'hello' only supports arguments [Text]
riskscape expression evaluate "hello('Ronnie', 'Janice')"
 Failed to evaluate expression
   - Could not find function hello with arguments [Text, Text] - function 'hello' only supports arguments [Text]
```

### Anything type

RiskScape functions support an `anything` argument-type.
This basically acts like a 'type wildcard' and means your function will accept _any_ data type as an argument.

.. tip::
    Using the ``anything`` type as a function argument can make life easier if you are getting
    RiskScape errors complaining about '*Could not find function with arguments...*'.

Let's try using the `anything` type with our `hello` function.
Update your `project.ini` file so that it looks like this:

```ini
[function hello]
description = Simple 'Hello world' function example
location = hello.py
argument-types = [ anything ]
return-type = text
```

We can still pass our function a `text` argument and it behaves the same way.

```none
riskscape expr eval "hello('Ronnie')"
Hello, Ronnie
```

Let's check what error we get when we pass the function an `integer` type, like we did in the previous section.

```none
riskscape expr eval "hello(1)"
 Failed to evaluate `hello(1)`
   - Problems found with 'hello' function (from source file:/C:/RiskScape_Projects/function-demo/hello.py)
     - Your CPython function raised an error: TypeError: can only concatenate str (not "int") to str

  File "/C:/RiskScape_Projects/function-demo/hello.py", line 2, in function
    return 'Hello, ' + name
```

Notice that the error is now coming directly from our Python code, rather than RiskScape complaining that the types do not match.

This is the main drawback of using the `anything` type.
RiskScape no longer provides type-safety checks for you, nor guarantees that the attributes in the input data will match what your Python function expects.

.. note::
  Using the ``anything`` type may reduce performance slightly, as it adds some additional processing overhead for RiskScape.
  This would be most noticeable when your input data contains complex line or polygon geometry.

.. _hello_world:

### Multiple arguments

The `argument-types` is a comma-separated list of types, surrounded by `[ ]`s.

Let's expand our example so that we can pass our function a name _and_ a greeting.
Our Python code will look like: 

```python
def function(greeting, name):
  return greeting + ', ' + name
```

We then need to update the `argument-types` in our INI file definition so that it matches.
Our function now takes _two_ `text` string arguments.

```ini
[function hello]
description = Simple 'Hello world' function example that takes multiple arguments
location = hello.py
argument-types = [ text, text ]
return-type = text
```

We can then call our function, passing it _two_ arguments this time.

```none
riskscape expression evaluate "hello('Kia ora', 'Ronnie')"
Kia ora, Ronnie
```

.. note::
    RiskScape does not inspect your Python code for you.
    Any errors in your Python code will result in an error when you try to call your function.
    If the number of arguments in your Python code *differs* to your INI file definition,
    then this will result in an error.

Here's an example of what an error looks like when our Python code has _two_ function arguments,
but our INI definition only declares _one_ argument-type.

```none
riskscape expr eval "hello('foo')"
 Failed to evaluate `hello('foo')`
   - A problem occurred while executing the function 'hello'. Please check your Python code carefully for the likely cause.
     - TypeError: function() takes exactly 2 arguments (1 given) (no traceback available)
```

### Keyword arguments

You can optionally specify keyword names for each of your argument-types, to make things clearer.
These keywords get used on the _RiskScape_ side of things, rather than on the Python side.

For example, update the `argument-types` in your INI file definition to the following:

```ini
[function hello]
description = 'Hello world' function example that takes keyword arguments
location = hello.py
argument-types = [ greeting: text, name: text ]
return-type = text
```

The new keyword arguments will be displayed in `riskscape function list`.
The keywords can also _optionally_ be used in any RiskScape expression, e.g.

```none
riskscape expression evaluate "hello(greeting: 'Kia ora', name: 'Ronnie')"
Kia ora, Ronnie
riskscape expression evaluate "hello('Greetings', name: 'Janice')"
Greetings, Janice
```

.. tip::
    Keywords help add clarity and avoid confusion when your function takes several arguments of the same type.
    For example, ``do_some_maths(mean: 0.22, stddev: 0.74)`` is easier to read than ``do_some_maths(0.22, 0.74)``.

### Structs

So far, our example function's argument and return types have been _simple_ types, like `text` or `floating`.
They can also be a complex type, called a _Struct_.
A `struct` can represent a set of several different attributes from your input data bundled together.

We can define a struct as a _type definition_ using the following format:

```none
struct(attribute-name: simple-type, ...)
```

As an example, let's change our `hello` function to take a basic struct containing two attributes: a greeting and a name.

```ini
[function hello]
description = 'Hello world' function example that takes a struct
location = hello.py
argument-types = [ struct(greeting: text, name: text) ]
return-type = text
```

This looks very similar to our previous example, but instead of having _two_ keyword `text` arguments,
we now have a _single_ `struct` argument.

In our Python code, we can access the `struct` in a similar way to a Python dictionary:
`struct.get('attribute-name')`. For example:

```python
def function(details):
  return details.get('greeting') + ', ' + details.get('name')
```

We can now try calling our function and pass it a struct.
We can define a struct in a RiskScape _expression_ using `{ }`s, and specify the actual _value_ rather than the _type_.
For example:

```none
riskscape expression evaluate "hello({greeting: 'Kia ora', name: 'Ronnie'})"
Kia ora, Ronnie
riskscape expr eval "hello({greeting: 'Greetings', name: 'Janice'})"
Greetings, Janice
```

### Struct coercion

When the data supplied to a function does not match the expected argument types exactly,
RiskScape can often _coerce_ the argument types, or 'make do with what it has got'.

This means you can pass a function a struct with _more_ attributes than the function expects,
as long as the attributes are a _subset_ of the expected struct.

For example, we can call our function with an extra `time` attribute that our function will just ignore. 

```none
riskscape expression evaluate "hello({greeting: 'Kia ora', name: 'Ronnie', time: '11:00am'})"
Kia ora, Ronnie
```

.. tip::
    Try to limit your function's ``argument-types`` to just the bare essentials,
    i.e. only include the attributes that the function actually needs to use.
    Your function's ``argument-types`` do not have to match your input data *exactly*.
    This will lead to functions you can more easily reuse with different input data files.

### User-defined types

The `struct()` type expressions can get complicated when you have many different attributes you are interested in.
To make life easier, you can also define your own types in the INI file, and then _reference_ these in your function definition.

For example, we can define our own `greet` type and then use that in our function.

```ini
[type greet]
type.greeting = text
type.name = text

[function hello]
description = 'Hello world' function example that takes a user-defined type
location = hello.py
argument-types = [ greet ]
return-type = text
```

This behaves exactly the same as our previous example.
However, it can make it simpler to follow what your function does when dealing with complicated types,
or the same types that you end up using a lot.

.. note::
    If you want to use a user-defined type *within* another ``struct`` definition, then you will have to use ``lookup()``
    to resolve the type reference.
    For example, to use our ``greet`` type as an attribute in another struct, we have to do: ``struct(the_greeting: lookup('greet'))``.

### Optional arguments

You can define a function that takes an optional argument.
This means the argument value may or may not be specified.
In your Python code, the argument will be `None` if it is omitted. E.g.

```python
def function(details):
  if details is None:
      return 'Hello, world'
  return details.get('greeting') + ', ' + details.get('name')
```

In order for this to work you have to also tell RiskScape that the argument
may be `None` or 'nothing'. In RiskScape terminology, the type is called :ref:`nullable <type-nullable>`.
Your function definition in your `project.ini` file should look like this:

```ini
[type greet]
type.greeting = text
type.name = text

[function hello]
description = 'Hello world' function example with optional argument
location = hello.py
# note that we now surround the argument type in 'nullable' (this means we need 'lookup()' as well)
argument-types = [ nullable(lookup('greet')) ]
return-type = text
```

Our function will now work if we completely omit the arguments.

```none
riskscape expr eval "hello()"
Hello, world
riskscape expr eval "hello({greeting: 'Kia ora', name: 'Ronnie'})"
Kia ora, Ronnie
```

.. note::
    The hazard intensity measure in your model will always be nullable,
    i.e. if the hazard-layer did not intersect with an asset-at-risk, then the hazard value will be null.
    If you do not define the hazard argument as nullable, then RiskScape will simply not call
    your function when the hazard value is null.

### Return types

Defining the function's `return-type` in the INI file is similar to defining the `argument-types`.

Let's make our function's return value a little more complicated.
As well as returning a greeting, let's say we also want to return a 'diary entry' of when we met the person.

```ini
[function hello]
description = 'Hello world' function example with struct return-value
location = hello.py
argument-types = [ text, text ]
return-type = struct(greeting: text, diary_entry: text)
```

To return a struct in our Python code, we just define it as a Python dictionary.
For example:

```python
import time

def function(greeting, name):
  return {
    'greeting': greeting + ', ' + name,
    'diary_entry': 'Met with ' + name + ' at ' + time.asctime()
    }
```

When we call our function it now looks like this:

```none
riskscape expr eval "hello('Kia ora', 'Ronnie')"
{greeting=Kia ora, Ronnie, diary_entry=Met with Ronnie at Fri Aug 20 16:06:24 2021}
```

### Multiple functions in the same file

You may have noticed this example contains _two_ Python functions in the same file.
Your Python file can contain several other smaller functions that help your primary Python function to do its job.

Remember that the primary Python function that RiskScape executes should always be called `function`.

.. tip::
    Splitting the work up into several functions can help to make your Python code easier to follow and to test.
    
### Reusing functions for different input files

Ideally, you should be able to reuse the same function across multiple different input data files,
as long as the input data is essentially the same and the function is doing the same job.

RiskScape will try to _coerce_ the input data as best it can to make it fit your function.
However, one common problem is if the same attribute is called different things in different files.
This is particularly a problem with shapefiles, which often have truncated attribute names.

For example, a common problem when trying to build or run your model is getting an error like:

```none
- Could not find function quake with arguments [{REPLACE_1=>Integer, CONSTRUCT_1=>Integer}, Floating] -
  function 'quake' only supports arguments [building, Nullable[Floating]]
```

In this case the attribute names in the input data (`REPLACE_1`, `CONSTRUCT_1`) don't match what our function expects (`replace`, `construct`).

The best way to fix the problem is to simply change the attribute names in the input data to match
what our function expects.
We can do this on the fly in the :ref:`bookmark <bookmarks>` definition without needing to modify our input data file at all.
For example:

```ini
[bookmark buildings]
location = buildings.shp
set-attribute.replace = REPLACE_1
set-attribute.construct = CONSTRUCT_1
```

## Function categories

All built-in RiskScape functions are assigned to a category.
Categories are mostly useful for grouping functions in `riskscape function list --category`.

Using categories be helpful in organizing your functions if you start to add a lot of them.
You can assign categories to your own functions in the INI definition, e.g.

```ini
[function hello]
description = 'Hello world' function example with category
location = hello.py
argument-types = [text, text]
return-type = text
# assign our function to the 'language' category
category = language
```

.. _jython_vs_cpython:

## Jython vs CPython

RiskScape will use _Jython_ to execute your Python function by default.
Jython support is included as part of the RiskScape software image.
Refer to :ref:`jython-impl` for more specifics on using Jython functions in RiskScape.

However, Jython is a little different to the Python you would normally use outside of RiskScape,
which is called _CPython_.
Experienced Python users may prefer to use CPython (i.e. regular Python) instead of Jython.

However, CPython is _not_ distributed as part of the RiskScape software.
So you have to manually configure RiskScape to use same Python executable that you use locally.

Refer to :ref:`cpython-impl` for more specifics on using CPython functions in RiskScape.

.. tip::
    If you are not sure what Python implementation you should use, then the Jython default is
    probably good enough for most simple Python functions.
    If you need to import other Python packages, such as for maths operations,
    then we recommend using CPython.

