System Monitoring

Using Filebeat to ship logs to Logstash

I have already written different posts on ELK stack ( Elasticsearch, Logstash and Kibana), the super-heroic application log monitoring setup. If you are not familiar with them, please check my posts here. We were using SCP to copy files from the application server to the log server and the logs were then read by the Logstash component. As the application moved to microservices-based architecture, we started having more and more servers running small services and setting up the SCP from each server to ELK server became a pain.

Time for Filebeat

This is where Filebeat came to the rescue. To quote about Filebeat from the official website.

Continue reading

Advertisements

Setting up ELK Stack for near real-time log monitoring in AWS

ELK is a software stack that can help us combine all the logs from different systems and then analyze , monitor and evaluate in a single dashboard. The ‘ELK’ is an acronym made with the first letters of the components in it. Those are ‘Elasticsearch’,’Logstash’ and Kibana.

Keep Reading

Analyzing & monitoring application logs using ELK stack

One of major issues faced by modern day enterprise applications are the management and monitoring of application logs. During the phase of development, the application logging is overlooked as a rather unimportant part. But once the application is live in production, we realize that the first level of understanding to  what is happening in the system is generated logs. With the modern day architecture , the components are independent and they all have their own logging. It becomes very cumbersome to have all these checked and analyzed when the system is very huge. This is where the ‘ELK’ stack comes to the rescue.

What is ELK ??

Keep Reading