Welcome back to Tech with Vuresh in the continuation to our series on Hadoop interview questions. Today we look a very interesting problem and that problem is called split brain problem. So this is the problem in the Hadoop 2 and in this particular video we will see what exactly is this problem, what are the reasons for this problem and what are the solutions provided by Hadoop to tackle a brain split kind of scenarios. So guys let's start and before I move forward I would like to request to all my subscribers if you like my videos do subscribe to the channel and also like and leave your comments so that we can improve on the quality of videos. So guys let's start with split brain problem. So split brain problem is a scenario which may happen in Hadoop 2 architecture where both the name nodes, the active and the standby became active at the same time. So in the typical Hadoop 2 architecture there is a facilitation for an active name node and the passive name node which is the backup name node which will take over in the case of failovers but sometime it may happen that both the nodes become active at the same time and this leads to a lot of problems especially to the corruption of the metadata regarding the namespace information of the name nodes is one of the biggest problems. So guys this whole scenario wherein both the nodes become active at the same time is called split brain issue and the problem this particular issue will bring in is this will lead to the corruption of the metadata the name space information kept to handle the underlying data nodes information regarding the blocks for the allocation of tasks and the storage of files etc. Another issue that will arise is the whole cluster will get divided into two smaller clusters each led by individual name nodes and some of the data nodes would associate themselves with one data node and others would be with the name node 2 and both these groups will believe that it is the only active cluster that should be the case in idle scenario but they will behave in a split manner the whole architecture the whole resource set will be divided into two. So this entire scenario is called as split brain issue or split brain scenario. So guys look forward what are the different reasons for this scenario or issue to happen and what are the probable solutions provided. If we quickly look a little bit of history in the context into the Hadoop architecture before Hadoop 2 name node happens to be the single point of failure. So if in case name node goes down for some reason the entire cluster is unavailable and there was no mechanism for any sort of automatic failover before Hadoop 2. With the advent of the variable architecture they brought in the concept of two name nodes one would be the active name node another one would be the passive name node act as the standby and it will take over in the cases of failover scenarios. Now for these two name nodes to work in tandem and be in sync from the namespace information perspective they've also brought in the concept of general nodes. So they maintain a quorum of general nodes which help in sharing the name space and the metadata information between the two name nodes active and passive. So as you've already discussed the Hadoop 2 highly available architecture has a concept of active and passive name nodes which comes is that if we have to keep the two name nodes in sync from the underlying data node perspective we'll have to make sure that the entire information regarding the blocks and the storage of different blocks of data is supplemented to both these name nodes. So if in case there happens to be a scenario of switch over from active to passive name that should be smooth and all the information should be available with the new node. So for this what they facilitate is now the data node is configured in such a way that it will send the block information to both the name nodes at the same time. Another facilitation that they make so that both the name nodes would have their active and you're at one point in time they make sure that all the commands read right are handled by one node at a time and that name node would be the active one. But as we have seen there may arise a scenario where during a failover while the one name node is still shutting down it may handle some request which are already supplemented to it and is foil or corrupt the entire metadata. You put a solution for such kind of scenarios Hadoop 2 gives a solution in the form of fencing. So in your HDFS site XML configuration files you can configure different fencing mechanisms. Fencing mechanism means that it'll make sure that at any single moment of time in the entire lifecycle of Hadoop cluster there can only be one name node active. There would not be a scenario of overlap and for this we can configure different fencing methods provided. If we talk about different fencing methods these methods in itself could be different custom Java objects where you've written a logic to kill the shutting down name node explicitly or some sort of scripts that kind of go ahead and kill that process of you know dying name node. So the fencing mechanism provided in the Hadoop 2 is one is SSH fence which can be configured in this form where you set up a property DFS dot H a dot fencing dot methods and never the method in this case would be SSH H fence. What it does primarily is it will do a SSH on that particular failing name node supplemented with the information from the configuration file regarding the IP address in the port and will explicitly kill that process. So that while the new node name node takes over the previous one is killed explicitly and there would not be any overlap scenario. Another method provided for fencing is the shell wherein we can path the you know relative path of the script that script may be carrying some sort of custom logic to either to explicitly kill that process or to make sure that that process is not accessed anymore from the command perspective. So these are some of the fencing mechanism provided by the Hadoop 2. So guys that this particular video where we have discussed this split brain issue and the probable solution provided by Hadoop 2. Keep watching. Happy learning. Have a great day. Bye bye.