Task:

Connect a Linux and one Windows EC2 instance with Grafana and monitor the different components of the server.

Setup Grafana
Install Docker:

Updates the package lists on the system, ensuring that the latest package information is available for installation.
```
 sudo apt-get update
 sudo apt install docker.io
 sudo usermod -aG docker $USER
 sudo reboot
```
3. Download Loki Config:

Use the following command to download the Loki configuration file:
```
 mkdir grafana_configs
 cd grafana_configs
 wget 
 https://raw.githubusercontent.com/grafana/loki/v2.8.0/cmd/loki/loki-local-config.yaml -O loki-config.yaml
```
- This command uses wget to download the Loki configuration file from the specified URL.
- The configuration file contains settings for Loki, such as storage configurations, retention policies, and remote storage options.

Download Promtail Config

Download the Promtail configuration file using the command below in grafana_configs directory:

wget 
https://raw.githubusercontent.com/grafana/loki/v2.8.0/clients/cmd/promtail/promtail-docker-confi

Run Loki Docker container:

Execute the following command to run the Loki Docker container, providing the downloaded configuration file:

sudo docker run -d 
--name loki -v $(pwd):/mnt/config -p 3100:3100
grafana/loki:2.8.0 --config.file=/mnt/config/loki-config.yaml

Edit the inbound rule in the security group of the ec2 instance to allow port 3100.

Copy public-ip of instance and paste in browser on https:<public-ip>:3100/ready and check loki is ready..?

In Loki what kind of logs or metrics are there we can get those like this use /metrics`

Run Promtail Docker container:

Run the Promtail Docker container with the downloaded configuration file, mounting the host’s log directory:

sudo docker run -d --name promtail -v $(pwd):/mnt/config -v /var/log:/var/log 
--link loki grafana/promtail:2.8.0 
--config.file=/mnt/config/promtail-config.yaml

- Add Data source in Grafana

Now, navigate to the Grafana web app and on the homepage choose the add data source option.
- Provide the HTTP URL as below to connect the loki data source to Grafana so that loki will send the logs to grafana.
- localhost:3100
- Click on save & test to connect the data source.

Checking logs in Loki:

Click on explore in the below screenshot after adding the data source.

Run Query will display below output

Create Dashboard:

Let’s add the log to the dashboard by choosing the option from the above screenshot location.

In Label filters choose job and varlogs and line contains to error to show all the lines Run Query.
On right hand side choose Logs > Click to change visualization> select row format as shown in below.

let’s check the error lines in grafana log that is placed in /var/log/grafana/grafana.log

To display the Grafana log, we must specify the Grafana log path in the promtail config YAML file within the target section, as illustrated below.

After editing promtail_config.yaml file we have to restart our promtail docker container

Use the proper label filters to show an aggregate sum of words repeating nginx while installing. This can be achieved by setting the varlogs as label filters.

Happy Learning!

Day 74 - Connecting EC2 with Grafana .

Setup Grafana

Install Docker: