V - Introduction to Grafana

a - Presentation

Grafana is an open source platform for monitoring, analyzing and visualizing system data in real time. The aim of this solution is to easily and intuitively present a large amount of data from different sources. After receiving the data, Grafana analyzes it and presents it on intuitive dashboards, which are easy to read and largely customizable.

Grafana lets us query, visualize, alert and explore our metrics and logs wherever they're stored. It gives us the tools we need to monitor and analyze the data source, enabling us to detect and resolve.

Organizations use Grafana to monitor their logs and infrastructure analytics, largely to improve functional efficiency. Dashboards track events and users, as they automate data management, display and collection. Security analysts, developers and product managers use this data to make their decisions.

Grafana is one of the most popular dashboard and visualization tools for metrics. Grafana dashboards are a very important part of monitoring the infrastructure and our various applications. We can create Grafana dashboards for the most important metrics of a virtual machine, create advanced dashboards with filters for several instance metrics, import and export dashboards, refresh at specific intervals the dashboards.

We'll need a metrics source from which to add metrics to Grafana for visualization. We'll use Prometheus in this course as the data source and node exporter to export metrics from a virtual machine, Docker containers or even a Kubernetes cluster to Grafana.

b - Grafana Dashboard

Grafana is organized into Dashboards and Panels. A Dashboard represents a view of a system's performance, and each Dashboard consists of one or more Panels, which represent information on a specific metric related to that system. A dashboard is a group of widgets, but it also provides many more features such as folders, variables (to modify visualizations in widgets), time ranges and automatic widget refresh. Dashboards can be created in a number of ways:

• With a form to choose the elements to be added to the dashboard.

• With Json.

• With the ID of a dashboard created by the community or by the Grafana team on the site [https://grafana.com/grafana/dashboards/].

• With the Grafana API.

c - Dashboard JSON template

A dashboard in Grafana is represented by a JSON object, which stores the metadata of its dashboard. Dashboard metadata includes dashboard properties, panel metadata, template variables, panel queries, etc.

JSON fields

When a user creates a new dashboard, a new dashboard JSON object is initialized with the following fields:

{
  "id": null,
  "uid": "cLV5GDCkz",
  "title": "New dashboard", // Name of the Dashboard
  "tags": [],
  "style": "dark",
  "timezone": "browser",
  "editable": true, // Editable
  "graphTooltip": 1,
  "panels": [],
  "time": {
    "from": "now-6h", // Time Interval
    "to": "now"
  },
  "timepicker": {
    //Element Type
    "time_options": [],
    "refresh_intervals": []
  },
  "templating": {
    "list": []
  },
  "annotations": {
    "list": []
  },
  "refresh": "5s",
  "schemaVersion": 17,
  "version": 0,
  "links": []
}

Each JSON dashboard field is explained below with its usage:

d - Panels (Panel)

Panels are the building blocks of a dashboard. It consists of data source queries, chart types, aliases, etc.... A JSON panel consists of an array of JSON objects, each representing a different panel. Most fields are common to all panels, but some fields depend on the panel type. Here's an example of a JSON panel from a text panel.

"panels": [
  {
    "type": "text",
    "title": "Panel Title",
    "gridPos": {
      "x": 0,
      "y": 0,
      "w": 12,
      "h": 9
    },
    "id": 4,
    "mode": "markdown",
    "content": "# title"
  }

Panel size and position

The gridPos property describes the size and position of the panel in grid coordinates.

• w : 1-24 (the width of the panel is divided into 24 columns)
• h : In grid height units, each represents 30 pixels.
• x : The x position, in the same unit as w.
• y : The y position, in the same unit as h.

The grid has a negative gravity that moves panels upwards if there is an empty space above a panel.

e - Time selector

"timepicker": {
    "collapse": false,
    "enable": true,
    "notice": false,
    "now": true,
    "refresh_intervals": [
      "5s",
      "10s",
      "30s",
      "1m",
      "5m",
      "15m",
      "30m",
      "1h",
      "2h",
      "1d"
    ],
    "status": "Stable",
    "type": "timepicker"
  }

The use of fields is explained below:

f - templates

The templating field contains an array of template variables with their stored values as well as other metadata, for example:

 "templating": {
    "enable": true,
    "list": [
      {
        "allFormat": "wildcard",
        "current": {
          "tags": [],
          "text": "prod",
          "value": "prod"
        },
        "datasource": null,
        "includeAll": true,
        "name": "env",
        "options": [
          {
            "selected": false,
            "text": "All",
            "value": "*"
          },
          {
            "selected": false,
            "text": "stage",
            "value": "stage"
          },
          {
            "selected": false,
            "text": "test",
            "value": "test"
          }
        ],
        "query": "tag_values(cpu.utilization.average,env)", // request
        "refresh": false,
        "type": "query"
      },
      {
        "allFormat": "wildcard",
        "current": {
          "text": "apache",
          "value": "apache"
        },
        "datasource": null,
        "includeAll": false,
        "multi": false,
        "multiFormat": "glob",
        "name": "app",
        "options": [
          {
            "selected": true,
            "text": "tomcat",
            "value": "tomcat"
          },
          {
            "selected": false,
            "text": "cassandra",
            "value": "cassandra"
          }
        ],
        "query": "tag_values(cpu.utilization.average,app)",
        "refresh": false,
        "regex": "",
        "type": "query"
      }
    ]
  }

The use of the above-mentioned fields in the templates section is explained below: