How to install and configure Loki on centOS 7

We need to install three  tools, one is loki , second is promtail and third is grafana.

Loki is a server application specifically designed to aggregate log data with a focus on high availability and scalability. To be unlike its competitors really easy to install and enormously resource efficient.

Promtail is a log data supplier. As an agent, it is installed exactly where the log data is purchased. Promtail then reads out the desired log files and sends the content to Loki.

Installation Loki

Loki is a software written in Go and therefore very easy to install. We must download the binaries, unzip them and start them.

First, we download the binary.

mkdir /opt/loki/ && cd /opt/loki/
wget https://github.com/grafana/loki/releases/download/v1.0.0/loki-linux-amd64.gz
gunzip loki-linux-amd64.gz
chmod a+x loki-linux-amd64

Now we add the basic configuration to the /opt/loki/config.yaml.

auth_enabled: false

server:
http_listen_port: 3100

ingester:
lifecycler:
address: 127.0.0.1
ring:
kvstore:
store: inmemory
replication_factor: 1
final_sleep: 0s
chunk_idle_period: 5m
chunk_retain_period: 30s

schema_config:
configs:
– from: 2018-04-15
store: boltdb
object_store: filesystem
schema: v9
index:
prefix: index_
period: 168h

storage_config:
boltdb:
directory: /tmp/loki/index

filesystem:
directory: /tmp/loki/chunks

limits_config:
enforce_metric_name: false
reject_old_samples: true
reject_old_samples_max_age: 168h

chunk_store_config:
max_look_back_period: 0

table_manager:
chunk_tables_provisioning:
inactive_read_throughput: 0
inactive_write_throughput: 0
provisioned_read_throughput: 0
provisioned_write_throughput: 0
index_tables_provisioning:
inactive_read_throughput: 0
inactive_write_throughput: 0
provisioned_read_throughput: 0
provisioned_write_throughput: 0
retention_deletes_enabled: false
retention_period: 0

Loki is now ready for use. With the following command the server can be started easily:

/opt/loki/loki-linux-amd64 -config.file /opt/loki/config.yaml

An example of a system unit can be found here:

create file /etc/systemd/system/loki.service (Copy the data in the file)

[Unit]
Description=Like Prometheus, but for logs.
Documentation=https://github.com/grafana/loki
[Service] ExecStart=/opt/loki/loki-linux-amd64 -config.file /opt/loki/config.yaml
[Install]
WantedBy=multi-user.target

Promtail is an agent that provides the log data for Loki and sends it to Loki. The contents of the log files can be labelled, manipulated and restructured by Promtail.

Promtail is as easy to install as Loki.

mkdir /opt/promtail/ && cd /opt/promtail/
wget https://github.com/grafana/loki/releases/download/v1.0.0/promtail-linux-amd64.gz
gunzip promtail-linux-amd64.gz
chmod a+x promtail-linux-amd64

Also Promtail wants to be configured a little bit. Therefore we add a /opt/promtail/config.yaml.

# Promtail Server Config
server:
http_listen_port: 9080
  grpc_listen_port: 0  
# Positions
positions:
  filename: /tmp/positions.yaml  
# Loki Server URL
clients:
  – url: http://localhost:3100/loki/api/v1/push
 scrape_configs:
  – job_name: syslog   
  static_configs:  
   – targets:  
       – localhost      
labels:   
      job: syslog
        host: localhost
        __path__: /var/log/syslog

The most important configuration here is the scrape_configs section. Here we define which log files should be read and considered. The manipulation of log files can be realized in the pipeline_stages.

In our example we simply read the syslog and send it unchanged to Loki.

Promtail can now be started via console with

/opt/promtail/promtail-linux-amd64 -config.file /opt/promtail/config.yaml

or with Systemd

create file /etc/systemd/system/promtail.service (copy the data in the file)

[Unit] Description=Promtail
Documentation=https://github.com/grafana/loki
[Service] ExecStart=/opt/promtail/promtail-linux-amd64 -config.file /opt/promtail/config.yaml
[Install]
WantedBy=multi-user.target

Now validate the service is running or not

Loki Url :- http://your-ip:3100

From this moment on, log data should already be running into Loki. In Grafana we now add Loki as Datasource and can observe, aggregate and filter the logfiles in Explorer.

Also checkout This Setup:-  How can we know type of element in Python!