Centralization logger with spring boot , elk stack 


→ spring boot use log4j as a default option is you want to local log you application 

Log stack i

Log monitoring 
→ actively log , montorie a set of logs to 



—> Socket : ( Inputs , Filters , Outputs  ) → Logstash pipeline 
—-> Elasticsearch : (NoSql ) data base oky 
—> Kibana : ui visualization 


today we are how one can integrate elk stack with spring boot application for log aggregation and monitoring spring boot provides a default logging framework spring boot starter logging which uses log back as a implementation instead of default logging we will be using log 4 J2 the main difference between spring boot logging and the log for J2 is the underlying logging framework log for J2 is part of log4j2 module one of the advantages of using log4j2 module as it uses an implementation of log4j2 and it is known for its performance
00:50
improvements also we will be introducing Elk elk is a collection of three open source applications one is elasticsearch and another one is log stack and third one is kibana elasticsearch mainly used for stored storing the data and indexing the data it has no SQL database based on lesion open source search engine since elasticsearch is developed using Java therefore it can run on different platforms one particular aspect where it excels is indexing streaming of data such as locks the other module that we have is
01:36
logstash log stash is a tool that integrates with variety of deployments it is used to collect Parts transform buffer data from variety of sources the data collected by logstash can be shipped to one of the other targets like elasticsearch the third component in Elk stack is kibana kibana act as an analytics and visualization layer on top of elasticsearch kibana can be used to search View and interpret the data stored in elasticsearch log monitoring what is log monitoring application log monitoring refers to a
02:17
process of actively monitoring and analyzing log files generated by application it involves collecting parsing analyzing of log data to gain insights in the application Behavior performance and potential issues why is it important it is important because of four Prime reasons the first reason is troubleshooting log monitoring helps to identify and diagnose issues such as errors exceptions performance bottlenecks and helps in assisting in troubleshooting and resolving them quickly second Point performance optimization by analyzing locks one can
02:57
identify the areas of performance Improvement optimize resource usage and enhance overall efficiency of an application that point most important point is security log monitoring can helps to detect security breaches or suspicious activity by analyzing a pattern or anomalies in the logging data fourth one is compliance many Industries have a regular requirement for log management and monitoring making it crucial for organizations to implement log monitoring to implement compliance standards there are various tools
03:33
available in Market such as dynatrace gray log and a logging let's take a deep look at our architecture we will be using uh spring boot application which runs on ah jdk 17 uses Maven 3.8.3 and as I said earlier it will be using uh instead of logging module it will be using log for J2 and hence we use it various appenders like socket appender log file appender and console appender in this example uh the socket appender which runs on TCP Port will be connected to the log stash log stash is a pipeline which will have
04:21
like inputs filters and outputs so the output of a log stash will be connected to the elastic search module and the data is saved in the elastic search and in order to visualize the data we'll be using the kibana so all the installation of log stash elasticsearch and kibana we are not going to install it instead we are going to use a Docker Docker file where we are going to host all these modules in the docker and the version of the docker that I will be using is 24.000.
04:59
2 as part of this video I am not going to cover the installation of Docker desktop or install relation of Docker plugin in an IDE as this were already done in the previous videos I am skipping the installation of Docker desktop and Docker ID without any delay let's get started let's open the browser and let's look for the starter Dot spring dot IO so the configuration that that I will be interested is mavin project mavin and language as Java and springboot is a 3.1.
05:41
3 at the time of recording this is the latest and instead of demo I'll be mentioning the group ID to be com.face and instead of demo I'll be saying as a template and then I'll be using the packaging as a jar file and Java 17 because that is a LTS version and the dependencies that I'll be interested is a spring web and the other dependencies is the lambook these are the two dependencies that that I'll be interested in and now I'll click on generate file uh the the this on generation it creates a folder in my desktop and I'll import that in my
06:24
IntelliJ IDE I have imported the folder in IntelliJ and the pawn.xml of that is like this because we we have only two dependencies one is the starter web and another is a lamboc so inside this starter web if we go inside this one then we will be seeing like various dependencies one among them is the spring boot starter if we inside go into the spring boot starter then we will be seeing uh various dependencies and one of the dependencies that that we come across is the uh is basically spring boot starter login
07:10
so instead of using the starter login I am interested to use a different one how do I do that is basically I copy paste the module uh and what I'll be saying is I I want to exclude this particular starter login from a starter the module that that I have copied from is basically from the spring boot starter so I will copy this I still need the spring boot starter but I don't uh want the following dependencies so what what I will be saying saying is all I I want to exclude it so we we have a tab called as an uh exclusion so
08:01
so uh the the moment I include this particular tag uh all I need to mention what is the dependency that that uh or sorry what is the uh group ID and artifact ID that that I need to exclude it so this this is what I want to exclude it from the list of exclusions so so I have my list of exclusions uh with my exclusion tag and uh then I'm trying to exclude this particular thing from it so and what I am interested to include is a log 4J module the way how I am going to use the log4j plugin to be included uh is basically adding its dependency again
09:02
lock for J it comes with the spring boot starter module so instead of login I'll be using as a log 4J so I'll copy one of the dependencies and and I'll I'll be mentioning that instead of a spring boot starter logging logging all I I want to include is the log 4J dependencies block 432 so the the this is the module that that I am I I'm interested to uh include and there is one more uh thing uh as part of testing that that I I want to include uh basically when whenever uh we are working with the test modules in
09:48
this exercise I'm not going to write any test cases but in I ideally whenever you are trying to include it we uh we we use this annotation uh run with uh annotation uh where this annotation uh ideally comes from the uh uh J unit which needs to be added to this uh spring dependency so if I add it the other way of adding uh to pawn.
10:17
xml is by searching the dependencies and adding it so the the the moment it acts it uh all we want to you know import the unit so if I go back to the pawn.xml again you can see the dependency has now been added I have noticed that there is a compilation issue because of this and we will be fixing that in a minute by importing the class and it has been fixed now what we will be doing is we'll be creating a class named as an user for this and we create that as a user class and then inside this class what what I will be
11:03
interested is to Annette it that class are you using the Lang book expressions and because I'll be using the Builder pattern uh um I will also be using the no arcs Constructor and and as and as well as the other Constructor which is all arcs Constructor so and I'll be in interested to import all the classes Associated to it all the input statements Associated to it so so now in terms of the fields what I'll be interested is I'll be interested in creating like uh a field called as an ID and another field of string type and
11:58
I'll be making it as a name and another one which will be local date time date time and I'll be taking us date of birth with the time date of so birth details probably I'll say as a birth birth details so and I'll be importing the class for this and I will also be yeah I can move that to the package different package where I can create as a package and I can say as a name of the that package I can make it as a domain and all I'll be moving this to that domain where I can do the refract but likewise I'll be creating
12:56
one more package where I'll be seeing the package name to be a controller and I'll be creating a class called a user controller so and what I'll be doing is in terms of this controller because it is a rest controller I am just going to say as a rest controller and import import it and um because we we are interested in uh going for the locks so SFL 4G is the one that that I will I'll be interested in importing it um and I'll be writing a small code where where it is basically the git mapping of it sorry caps is on get
14:00
mapping of it and I'll be importing this as well uh and the URI is basically he it will use users slash and I'll be interested to pass the ID um so whenever we access it it is a local host uh Port colon 8080 slash user slash ID and the method that I am interested is basically to how it to be as a response entity uh basically I I can make it as a class type but for now I'll be interested to make it as uh enter where I'm uh trying to find a user by ID uh so and I I can again use like uh string params or I can directly say as a path


Q- given this transcript given step by step tutor how can do it oky