Back to top

A practical guide how to setup ELK and use log driven development to improve your Drupal dev workflow

Session type

Sooner or later the development or life cycle of every website reaches a state when something has to be debugged. In such cases many people waste precious time in order to get to the source of a problem and eventually resolve it. Very often people also don’t have the information they need in order to solve a complex problem. In such situations websites go down, developers and support teams get angry and something which is far worse - when this happens enterprises could lose money and possible clients.

That is why Drupal agencies and website/business owners should learn more about LDD - Log Driven Development which provides valuable information when you need it and is also the foundation for good events monitoring.

In this lecture the LDD idea will be presented to the audience and how log driven development helps teams to quickly solve issues, get products to market and close the feedback loop. The target audience of this talk consists of people who want to learn more about LDD but do not have the time to read long tutorials. Previous experience is not a requirement and newbies are most welcome. Still this talk will include examples of service configurations and attendees should be comfortable working with SSH.

Except for the theoretical part during the lecture we will also talk about a very popular LDD setup for Drupal - the ELK stack (ElasticSearch + LogStash + Kibana). The theory together with the practical guide how to setup ELK for Drupal will allow devs, team leads and system administrators to start from scratch with LDD and they will see examples why LDD is a super efficient way to track issues, solve them and focus on the important things in your life :)

We will also share some real life examples from our own experience of problems solved with the help of the Log Driven Development.

At the end of the session attendees will walk away with answers to the following questions:

- What is LDD and how it can help you?
- What do I need to do in order to start with LDD right now?
- What is the ELK stack and how is it related to LDD?
- What does the ELK stack give me and why it is so awesome?

The presentation itself will be devided into the following sections:

Section 1 - What is the theory behind Log driven development and ELK
a) what is LDD - summary of what is this methodology
b) what is ELK - explanation of what Elastic, Kibana, Logstash are and how it all fits together

Section 2 - Practical guide to setup and use ELK
- which are the logs that will be included
- separate server for ELK
- installation of filebeat on the source and transfer logs to the ELK /how filebeat works/
- how logstash works and what plugins does it provide
- what’s happening with JSON format logs
- show filters and how to create such
- issues with upgrading Elasticsearch
- how does Kibana work - graphics, dashboards, filters

This section will also include information about Filebeat, Logstash, Elasticsearch, Kibana requirements, installation and configuration

Section 3 - Case study

Section will be a general overview of why using LDD+ELK is awesome, plus a case study how ELK helped us to solve complex issues, optimize performance and dev workflow

At the end - Conclusion + Questions & Answers