Alert Basics
------------

Into ntopng, alerts are divided into two groups:
  - *Stored alerts* (also known as *Flow alerts*)
  - *Engaged alerts*

A stored alert represents a single alert event which happens at a given time.
On the other end, engaged alerts report events with defined start and end times.
An engaged alert remains engaged as long as the alert is in progress. Eventually,
it will be released when the problem is solved.

An alert carries the following information:
  - *ifid*: The numeric interface identifier that triggered this alert. You can
  find the interface Id in the user interface at `/lua/if_stats.lua`.

  - *entity_type*: Identifies the id of the entity that triggered the alert.
    The defined entity types are listed in `alert_consts.alert_entity_keys`.
    The entity types have a corresponding C enumeration `AlertEntity`.

  - *entity_value*: A string that, together with the *entity_type*, uniquely
    identifies the entity subject of the alert. For example, the tuple an
    host alert on the host "192.168.1.1" would have `entity_type="host"`
    and `entity_value="192.168.1.1"`.

  - *alert_type*: identifies the type of the alert. See `alert_consts.alert_type_keys`
    for a list of defined alert types.

  - *alert_severity*: defines the severity of the alert. See `alert_consts.alert_severity_keys`
    for a list of defined alert severities.

  - *message*: An informative message for the user describing what happened. The
    message is localized and can contain html.

  - *timestamp*: The epoch of the alert triggering.

  - *alert_engine*: only for engaged alerts, an identifier for the process which
    generated the alert. See `alert_consts.alert_engine_keys` for a list of defined alert engines.

  - *duration*: only for released alerts, the duration of the alert (how long it has been
    engaged until being released).

Custom Alert Endpoint
---------------------

Alert endpoints are lua script executed whenever an alert event occurs. An example
of alert endpoint is the email export endpoint, which sends email with alert information.

Users can define a custom alert endpoint to be called on alert events. The endpoint
could trigger, for example, an external bash script with custom logic.

The custom alert endpoint can be enabled as follows:

  1. Run `cp /usr/share/ntopng/scripts/lua/modules/alert_endpoints/{sample.lua,custom.lua}`
  2. Enable "Alerts to Third-Party Endpoints" from the "Alert Endpoints" settings
  3. Run `redis-cli set ntopng.prefs.alerts.custom_notifications_enabled 1`
  4. Restart ntopng

The file `/usr/share/ntopng/scripts/lua/modules/alert_endpoints/custom.lua` can then be
modified with custom logic.

Stored Alerts
-------------

Stored alerts can be easily generated by lua code by simply calling `interface.storeAlert` .
ere is how to generate an alert for IP/mac association change:
```
  interface.storeAlert(alertEntity("mac"), "00:11:22:33:44:55",
    alertType("mac_ip_association_change"), alertSeverity("warning"),
    i18n("alert_messages.mac_ip_association_change", {device=name, ip=elems.ip}))
```

Threshold Cross Alerts
----------------------

If the alert is a threshold_cross alert on an a new metric, the process is pretty simple.
For example, in order to get an alert when the total youtube traffic crosses a specified threshold:

- Extend `alert_consts.alert_functions_info`, `alert_consts.alarmable_metrics` end
  `alert_consts.alert_functions_description` by adding a new metric `youtube`.
- In `alert_utils.lua`, create the following function:
  ```
  function youtube(old, new, interval)
    return proto_bytes(old, new, "Youtube")
  end
  ```

The new metric should be now available into the ntopng gui, where it will be possible
to set the threshold on an host/network/interface and select to be checked every
minute/5 minutes/hour/day.

Engaged Alerts
--------------

Engaged alerts are more difficult to deal with because they have an associated status.
Examples of such alerts are the `threshold_cross` alerts. Engaged alerts are associated
to an *alert_engine*, which defines the process which generated the alert and which is
in charge to maintain its status until it's released. Such a status is stored into an
engine specific *alert cache*, which is synchronized with the internal alerts database.
The alerts cache essentially provides fast access to engaged alert status without the need to
manually query the (slower) alerts database.

In order to generate a new engaged alert, the following modifications should be applied:
  - choose an appropriate alert engine for the alert or create a new one.
  - in `alert_utils.scanAlerts`, add the appropriate check function (e.g. `check_snmp_alerts`)
    Remember that scanAlerts is called multiple times (e.g. every minute, every 5 minutes, ..) so
    you can check the `granularity` parameter to avoid doing the same task multiple times.
  - into the check function, iterate the entity to process (e.g. the SNMP devices), and
    retrieve the current status of the entity (either from a `checkpoint` or from a redis table)
    and pass it to the `check_entity_alerts` function
  - extend `check_entity_alerts` to deal with the new kind of information (e.g. the port status of
    the SNMP device) and call `addAlertInfo(..., port_info)` when an alerted status is detected.
  - Extend `formatAlertMessage` to give a localized message of the alert (possibly using the
    port_info field specified before, now available as the *alert_info* parameter).

External Programs
-----------------

It is possible to easily generate *stored alerts* from external programs, integrating
them with the ntopng alerts framework.
This can be accomplished by putting a valid JSON into the redis queue
`ntopng.alert_process_queue`. Here is an example of how to do this from a bash
script:

```
redis-cli "RPUSH" "ntopng.alert_process_queue" "{ \"ifid\": 0, \"entity_type\": 5, \"entity_value\": \"4a:00:06:a0:7c:51\", \"type\": 17, \"severity\": 1, \"message\": \"IP 1.2.3.4 changed association from <a href=\\\"\\/lua\\/mac_details.lua?host=4a:00:06:a0:7c:50\\\">4a:00:06:a0:7c:50<\\/a> to <a href=\\\"\\/lua\\/mac_details.lua?host=4a:00:06:a0:7c:51\\\">4a:00:06:a0:7c:51<\\/a>\", \"tstamp\": 1523616620, \"action\": \"store\" }"
```

and here is the formatted JSON:

```
{
   "ifid":  0,
   "entity_type":  5,
   "entity_value":  "4A:00:06:A0:7C:51",
   "type":  17,
   "severity":  1,
   "message":  "IP 1.2.3.4 changed association from <a href="/lua/mac_details.lua?host=4a:00:06:a0:7c:50">4a:00:06:a0:7c:50</a> to <a href="/lua/mac_details.lua?host=4a:00:06:a0:7c:51">4a:00:06:a0:7c:51</a>",
   "tstamp":  1523616620,
   "action":  "store"
}
```

The format of the JSON message is very similar to the format described above. Here numeric
fields are used in place of the string representation, see `alert_consts` for mappings
between string and numeric representation.
