Introduction

In this tutorial, we will delve into the process of data processing for subsequent use in notifications and alerts on the map within the BellaDati report. We will also explore options for layer filtering on the map for specific scenarios, configure notifications, and link them to map layers. This guide will provide detailed insights into these technical aspects, enabling effective setup and utilization of notifications and map layers in BellaDati.

System Overview

In our case, the data to be processed is stored in a special data set, serving as a repository for raw information that requires processing through an ML script. Once processed, this data is then placed into another data set in its final form. When all the data is prepared, we can use it for display on the map while applying the necessary filters. Additionally, we will explore customization options for these filters using extensions.

In our scenario, the data set containing raw data is automatically populated through an HTTP connection to an external resource. You can find instructions on how to connect to this and other data sources here. This automated data retrieval and integration is a fundamental part of the system's functionality and ensures a continuous flow of fresh information for processing and utilization.

It's worth noting that all data reading and storage actions are performed with system data sets, and all references to them start with "SYSTEM.", such as "SYSTEM.DATASET_CODE". For guidance on how to create and retrieve data from a system data set, please refer to the detailed instructions available here.

Key Considerations in the ML Script

Data Format Transformation

The raw data is initially provided in JSON format. To work with this data, we can use the JSON object with its "fromJSONString" function.

def data = JSON.fromJSONString(jsonString);

An alternative method we used here is the Groovy class "JsonSlurper", which allows us to transform the data into a hash table for further processing.

import groovy.json.JsonSlurper;

JSON_SLURPER = new JsonSlurper();
def data = JSON_SLURPER.parseText(jsonString);

Date and Time Handling

For effective date and time manipulation you can use:

import org.joda.time;

def localDateTime = currentLocalDateTime() // Creates a time object in the zeroth time zone.
def japaneseDateTime = localDateTime.plusHours(9) // Creates a new instance of date and time with an addition of 9 hours.
def inRowJapaneseDateTime = currentLocalDateTime().plusHours(9) // Short chained definition
def currentDateTime = new LocalDateTime(0) // Creates a date-time object on "1 January 1970 00.00.00.000". Usage without any input returns the same as "currentLocalDateTime"
def predefinedDateTime = new LocalDateTime("2023-10-24T00:09:21.566") // Creates a new date-time object from the string

Additionally, to work with Japanese characters and ensure correct sorting of arrays in alphabetical order, use the Collator with Locale:

import java.text.Collator;
import java.util.Locale;

COLLATOR = Collator.getInstance(Locale.JAPANESE);

def allAreas = eventList.sort{ a, b -> COLLATOR.compare(a.areaname, b.areaname) }.collect{ "${it.areaname} - ${it.height_desc}" }

Mathematical functions are handled through the import of Math.

import java.lang.Math

Math.abs(x)
Math.sin(x)
Math.cos(x)
Math.asin(x)
Math.atan2(x)
// ...

Data Filtering

The last set of data from the raw data set is obtained by filtering based on the date. This ensures that only records added after the last successful ML script execution are processed. This can be achieved through filtering the date column using the "lastSuccessExecutionDT" variable, which contains a date object of the "LocalDateTime" class or "null" if there have been no successful executions. Pay attention to time zone differences when working with dates from different time zones.

def searchFromDT = lastSuccessExecutionDT ?: new LocalDateTime(0);
def filter = isGreaterOrEqualFilter('L_DATETIME', searchFromDT);

Geo Mappers

In addition, it's important to mention the usage of geomappers. Geomappers are a special dataset type in BellaDati that, under specific geographic coordinate or other values, return corresponding codes or names of regions, for example. You can utilize them in ML Studio by using the "inShape" function. This function takes two parameters: the value you want to map and the name of the geomapper itself.

def prefName = inShape(coords, 'Prefecture-Name');

The whole script code you can find here.

Notifications

Once our data is processed and ready for use, we can configure notifications. You can find detailed instructions for configuration here. It's worth mentioning that for dynamic date filtering, you can utilize the "APPLYING VARIABLE AS RELATIVE DATE" section at this link.

Configuration of map layers

To place layers on the map that show additional information on hover, you can configure "Map layers." To do this, open the map settings:

Then select "Map layers," and click "Add layer":

In the window that opens, you can configure the following parameters:

  • Layer name – the name that will be displayed in the list of notifications
  • Basic color – the color used to highlight the area on the map
  • Drill down by – the column with unique values that will be matched with the geomapper
  • Places definition – the geomapper used to calculate the areas displayed on the map
  • Formula – a formula that should return "null" if this area should not be displayed

Additional features include:

  • Enable mouseover - when this option is enabled, you can create a notification that appears on hover over the highlighted area (which can be plain text or HTML markup)
  • Enable label – a formula that determines the label on the highlighted area even if the cursor is not hovering over it
  • Canvas layer - when this feature is enabled, you can manually draw the required area on the map using JavaScript. Inside the Canvas layer, you can use Groovy code by wrapping it in "{formula}" and have access to the canvas object, which is an HTMLCanvasElement.

You can notice the comment in the code with a format like "//importFromExt(Weather Data Receiving,WeatherDataReceiving)". This is a special feature for importing code from an extension. The first parameter specifies the extension's name, and the second parameter is the name of the file with a ".groovy" extension (often matching the class described within the file). You can find more detailed information by visiting this link: Reusing pieces of Groovy formulas.

//importFromExt(Weather Data Receiving,WeatherDataReceiving)
def prefCode = memberValue('L_PREF_CODE');

if (!WeatherDataReceiving.hasAccess(prefCode)) {
  return null;
}

In the example above, "Weather Data Receiving" is the extension's name, and "WeatherDataReceiving" is the name of the ".groovy" file.

Linking layers to the system notifications

Let's consider a scenario where we need to link a system notification to map layers. When you hover over the notification, the corresponding map layer will be highlighted.

Again, go to the menu with the list of layers. Here, to the right of the layer name, click on the bell icon:

Here, you can fill in the following fields:

  • Notifications - A dropdown list of available system notifications.
  • Timeout (min) -  The number of minutes after which the notification will be visible to the user again.
  • Location attribute - The code of the column by which values will be matched with map zones (these values should match the "Drill down by" field of the layer).
  • Header attribute - The code of the column from which the value will be taken for displaying the notification header.
  • Datetime attribute - The code of the column from which the value will be taken for displaying the notification time.
  • Icon - The icon in the notification header.

If everything is set up correctly, you should achieve a result similar to this:

  • No labels