How to configure HDFS cluster using Ansible?



Lets perform a task of how to use hadoop with the ansible playbook and also configuration of hadoop cluster using the ansible playbbok. Lets first see the task:-


🔰 Configure Hadoop and start cluster
services using Ansible Playbook.


Prequisites for the following task:-

  • You must have the ansible configured in the base os.
  • Also in the inventory file of the ansible must have configured the target node.

Follow the bellow mentioned steps to solve the above task,

The below mentioned steps can be used for hadoop cluster configuration on the namenode, targetnode and clientnode.

Step 1: Copying the hadoop and jdk software :-

Step 2: Installing the java and hadoop software :-

Step 3: Create the Directory:-

For client node we dont need to create a directory.

Step 4:Configuration the core-site.xml:-

“Blockinfile” module of ansible is used to write the block of code in the hdfs-site and in core-site file. We have use the insertafter property of the blockinfile to insert the block of code after the <configuration> line inside the hdfs and core site file.

Step 5: Cofiguration the hdfs-site.xml:-

Step 6: Start datanode or namenode:-

We have use the “command” module instead of the “service” module to start the hadoop namenode or targetnode service because in the service module we can’t give options.

Step 7:-Format the namenode:-

Steps to be followed for namenode only:-

Code for namenode :-

Output of the namenode:-

Code for targetnode:-

Output for targetnode:-

Thanks For Reading….



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store