> For the complete documentation index, see [llms.txt](https://docs.invariant.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.invariant.io/platform/platform/multi-node-cluster-install.md).

# Multi-Node Cluster Install

Setting up a multi-node Hadoop cluster involves configuring several nodes (master and worker nodes) to work together. In this guide, we'll focus on a **three-node cluster**, where one node will act as the **NameNode** (master) and the other nodes will act as **DataNodes** (workers). We'll use a **Linux-based OS** (Ubuntu/CentOS) for this installation.

**Pre-requisites**

1. **Java 8 or higher** installed on all nodes.
2. **SSH** passwordless setup between all nodes.
3. **All nodes should have static IP addresses**.
4. **Network configuration**:
   * Ensure that all nodes can communicate with each other via their IP addresses.
   * All nodes should be able to resolve hostnames (`hostname` should resolve to an IP).

***

#### **Step 1: Configure SSH on All Nodes**

1. **Generate SSH key** on the master node:

   ```bash
   ssh-keygen -t rsa
   ```
2. **Copy the public key** to all nodes (master and workers):

   ```bash
   ssh-copy-id user@worker-node-ip
   ssh-copy-id user@master-node-ip
   ```

   This will enable passwordless SSH access between the nodes.
3. **Test SSH** between the nodes:

   ```bash
   ssh user@worker-node-ip
   ssh user@master-node-ip
   ```
4. **Set up a hostname for each node** in `/etc/hosts` (on all nodes):

   Open `/etc/hosts` and add entries like:

   ```
   192.168.1.1    master-node
   192.168.1.2    worker-node-1
   192.168.1.3    worker-node-2
   ```

   This allows each node to resolve the names of the others.

***

#### **Step 2: Install Java on All Nodes**

1. Install Java 8 or higher (if not already installed) on all nodes:

   On **Ubuntu**:

   ```bash
   sudo apt update
   sudo apt install openjdk-8-jdk
   ```

   On **CentOS**:

   ```bash
   sudo yum install java-1.8.0-openjdk-devel
   ```
2. **Set JAVA\_HOME** on all nodes:
   * Add the following to `~/.bashrc` or `~/.profile`:

     ```bash
     export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
     export PATH=$PATH:$JAVA_HOME/bin
     ```
   * Reload the profile:

     ```bash
     source ~/.bashrc
     ```

***

#### **Step 3: Install Hadoop on All Nodes**

1. Download Hadoop and extract it on all nodes:
   * On the **master node**:

     ```bash
     wget https://downloads.apache.org/hadoop/common/hadoop-3.3.0/hadoop-3.3.0.tar.gz
     tar -xzvf hadoop-3.3.0.tar.gz
     sudo mv hadoop-3.3.0 /usr/local/hadoop
     ```
   * On each **worker node**: Download Hadoop and extract it in the same way as on the master node:

     ```bash
     wget https://downloads.apache.org/hadoop/common/hadoop-3.3.0/hadoop-3.3.0.tar.gz
     tar -xzvf hadoop-3.3.0.tar.gz
     sudo mv hadoop-3.3.0 /usr/local/hadoop
     ```
2. **Set Hadoop environment variables** on all nodes: Add the following to the end of `~/.bashrc`:

   ```bash
   export HADOOP_HOME=/usr/local/hadoop
   export HADOOP_INSTALL=$HADOOP_HOME
   export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
   export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
   export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
   ```
3. Reload `~/.bashrc`:

   ```bash
   source ~/.bashrc
   ```

***

#### **Step 4: Configure Hadoop on All Nodes**

**1. Configure `hadoop-env.sh`**

1. Edit `hadoop-env.sh` on all nodes (`/usr/local/hadoop/etc/hadoop/hadoop-env.sh`):

   ```bash
   nano /usr/local/hadoop/etc/hadoop/hadoop-env.sh
   ```

   Add the following line to set `JAVA_HOME`:

   ```bash
   export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
   ```

**2. Configure `core-site.xml`**

1. Edit `core-site.xml` on all nodes (`/usr/local/hadoop/etc/hadoop/core-site.xml`):

   ```bash
   nano /usr/local/hadoop/etc/hadoop/core-site.xml
   ```

   Add the following configuration:

   ```xml
   <configuration>
     <property>
       <name>fs.defaultFS</name>
       <value>hdfs://master-node:9000</value>
     </property>
   </configuration>
   ```

**3. Configure `hdfs-site.xml`**

1. Edit `hdfs-site.xml` on all nodes (`/usr/local/hadoop/etc/hadoop/hdfs-site.xml`):

   ```bash
   nano /usr/local/hadoop/etc/hadoop/hdfs-site.xml
   ```

   Add the following configuration:

   ```xml
   <configuration>
     <property>
       <name>dfs.replication</name>
       <value>3</value>
     </property>
     <property>
       <name>dfs.namenode.name.dir</name>
       <value>file:///usr/local/hadoop/hadoop_data/hdfs/namenode</value>
     </property>
     <property>
       <name>dfs.datanode.data.dir</name>
       <value>file:///usr/local/hadoop/hadoop_data/hdfs/datanode</value>
     </property>
   </configuration>
   ```

**4. Configure `mapred-site.xml`**

1. Edit `mapred-site.xml` on all nodes (`/usr/local/hadoop/etc/hadoop/mapred-site.xml`):

   ```bash
   cp /usr/local/hadoop/etc/hadoop/mapred-site.xml.template /usr/local/hadoop/etc/hadoop/mapred-site.xml
   nano /usr/local/hadoop/etc/hadoop/mapred-site.xml
   ```

   Add the following configuration:

   ```xml
   <configuration>
     <property>
       <name>mapreduce.framework.name</name>
       <value>yarn</value>
     </property>
   </configuration>
   ```

**5. Configure `yarn-site.xml`**

1. Edit `yarn-site.xml` on all nodes (`/usr/local/hadoop/etc/hadoop/yarn-site.xml`):

   ```bash
   nano /usr/local/hadoop/etc/hadoop/yarn-site.xml
   ```

   Add the following configuration:

   ```xml
   <configuration>
     <property>
       <name>yarn.nodemanager.aux-services</name>
       <value>mapreduce_shuffle</value>
     </property>
     <property>
       <name>yarn.resourcemanager.address</name>
       <value>master-node:8032</value>
     </property>
     <property>
       <name>yarn.resourcemanager.scheduler.address</name>
       <value>master-node:8030</value>
     </property>
   </configuration>
   ```

***

#### **Step 5: Set Up Directories for HDFS**

1. **On the master node**: Create the directories for HDFS and give appropriate permissions:

   ```bash
   sudo mkdir -p /usr/local/hadoop/hadoop_data/hdfs/namenode
   sudo mkdir -p /usr/local/hadoop/hadoop_data/hdfs/datanode
   sudo chown -R user:user /usr/local/hadoop/hadoop_data
   ```
2. **On each worker node**: Create the directories for HDFS:

   ```bash
   sudo mkdir -p /usr/local/hadoop/hadoop_data/hdfs/datanode
   sudo chown -R user:user /usr/local/hadoop/hadoop_data
   ```

***

#### **Step 6: Format the Hadoop Filesystem (Only on Master Node)**

Run the following command on the **master node** to format the NameNode:

```bash
hdfs namenode -format
```

***

#### **Step 7: Start the Hadoop Cluster**

1. **Start HDFS**: On the **master node**:

   ```bash
   start-dfs.sh
   ```
2. **Start YARN**: On the **master node**:

   ```bash
   start-yarn.sh
   ```
3. **Check the status of your Hadoop cluster**:
   * NameNode: `http://master-node:9870`
   * ResourceManager: `http://master-node:8088`

***

#### **Step 8: Stop the Hadoop Cluster**

To stop Hadoop, use the following commands:

1. **Stop HDFS**:

   ```bash
   stop-dfs.sh
   ```
2. **Stop YARN**:

   ```bash
   stop-yarn.sh
   ```

***

#### **Troubleshooting Tips**

* **SSH issues**: Make sure passwordless SSH works correctly between nodes.
* **Permissions**: Ensure that the Hadoop user has appropriate permissions for directories.
* **Firewall settings**: Check firewall settings if the nodes cannot communicate.
* **Logs**: Look into Hadoop log files (located in `$HADOOP_HOME/logs`) for detailed error messages if something goes wrong.

This multi-node setup allows Hadoop to operate efficiently across multiple machines, enabling scalable and distributed data processing.