Using Filebeat to ship logs to Logstash

I have already written different posts on ELK stack ( Elasticsearch, Logstash and Kibana), the super-heroic application log monitoring setup. If you are not familiar with them, please check my posts here. We were using SCP to copy files from the application server to the log server and the logs were then read by the Logstash component. As the application moved to microservices-based architecture, we started having more and more servers running small services and setting up the SCP from each server to ELK server became a pain.

Time for Filebeat

This is where Filebeat came to the rescue. To quote about Filebeat from the official website.

Continue reading