my net house

WAHEGURU….!

Rocks cluster for virtual containers

First of all I would like to thanks Rocks community for saving our lots of money, at initial we were thinking about buying very expensive hardware and use it as dedicated server on which we would be able to run multiple docker containers as well as multiple virtual machines. Such systems are quite  expensive: Following examples of such systems are considerable when you are really serious about some kind of computing power either for research or for server business kind of thing.

  1. http://www.ebay.in/itm/191944430915 (A base class example, price range is 50 K)
  2. http://stores.ebay.com/Cypress-Technology-Inc/HP-9000-Servers-HP-UX-/ (Other possible high availability options price range is more thank 100K)

 

But now we had to do setup with solution which should not be costlier more than 20-30K and we want at least 8 cores and 16 GB of RAM. Presently our requirement was not too high so rather than spending much amount on SSDs(Solid state drives) we just zeroed to normal mechanical HDs.

We used used core2duo and dual-core CPUs as slave nodes for Rocks cluster, Presently we are having i3 second generation home PC for Front-Node that we might upgrade in near future but it is really efficient and working pretty much fine on CentOS. ❤ ❤

 

Now when we talk about Cluster-computing only one thing comes in mind a set of connected CPUs to perform heavy operations and using all core together to run some kind of simulation and feel like a scientist at NASA. 😀

Thanks to ‘Dr. H.S. Rai’(http://hs.raiandrai.com/) that he introduced me Rock’s Cluster many months ago that really changed my perception about super-computers,parallel-processing and most of the stuff which I am still not able to remember. 😛

So back to clusters! There are many types of clusters it just really depends on your problem, like what kind of problem you want to  solve using such systems.

Problem: User/Client wanted a simple Machine having multiple cores and GBs of memory so he/she will be able to create new virtual container for any new user as per the requirement

This tutorial assumes that you have installed Rocks Front-node in one of the system and have lots of other hardware available to you to connect with your Front-node.

Something like this:

 

 

This:

OR EVEN THIS:

If you are still not getting what I am trying to say you better be first go to Rocks cluster website and look what they really are doing!Adding compute Nodes:(It is one form of cluster)

 

For adding virtual containers Rocks comes with XEN(http://www.rocksclusters.org/roll-documentation/xen/5.0) roll. There are so many Rock’s roles those come as per the requirement. For example there is HPC roll that comes with OpenMPI(Open message protocol interface) that can be used if you want to execute your code more than two or three nodes using computing cores of most systems together. A generic way is something like this:

# execute_progarm compute-node-0 compute-node-1 compute-node-3

Such type of systems are used when you have lot of data to analyse or handle but even for that present industry rely on expensive stuff rather than using Rock’s implementation. 😦 ;D let’s save this for another day and concentrate only on visualization stuff.

 

So for implementation of vitalized containers we have to install XEN roll in Front-node, that can be installed while normal installation of Front-node if you are using Jumbo DVD(comes with all rolls ~ size of 3.SOMETHING GBs) or Rocks also provide all rolls(http://www.rocksclusters.org/roll-documentation/) as different ISOs.

After successful installation of XEN roll one need to connect slave node either via direct to Ethernet card or use network-switch. (Make sure while doing all this stuff you are logged-in as root user)

Execute following command.(That’s my favourite command in the whole world, I FEEL like GOD 😀 )

#  insert-ethers

You will get screen like following or it could be different if you are using other version of Rocks but you only have to concentrate on VM-Containers. Mkae sure at this time your slave node is having PXE-boot enabled. To enable PXE boot you have to look for slvave-node BIOS.

 

 

Hit enter after choosing required option and you will look installation on slave node will be started. It could take some time so have patience. 😛

While your VM container is being installed please have a look at the stuff we are doing so you will be able to understand the architecture or our cluster.

 

 

 

 

 

Or you can also see our creativity as well. 😛

20160919_184952

 

 

 

 

 

 

 

 

After successful installation of VM container you can see it will be available in your System, save your node and quit. To analyse all this process or to get idea what I am really talking about you can look for this link as well but let me clear that first we are using VM container here not compute node.(http://www.rocksclusters.org/rocks-documentation/4.3/install-compute-nodes.html)

 

To assign IP to your VM container run following command but make sure you have your Static IP so one will be able to access your container from public internet.

# rocks add cluster ip="your_static_IP" num-computes=<1,2,or 3>

above commands look simple just mention your static IP and number of compute nodes
you want to use. It could be 1 or 2 depending on how many nodes you have and
how many VM containers you want to create.

Now clear one thing first yet we have only created virtual cluster not Virtual machines

# rocks list cluster

Above command will give you available VM clusters present in your system.

Now before creating Virtual machines we need to create RSA key pairs so we will be able to do login from Front-node to

Virtual cluster and do required operations:

# rocks create keys key=private.key passphrase=no
setting option: passphrase=no will not ask you for password but you can skip
that if you want to use password with your security key.

Add that key to your newly created virtual cluster:

# rocks add host key frontend-0-0-0 key=public.key
Following command will start installation of VM on your Virtual cluster:

# rocks open host console frontend-0-0-0 key=private.key

After installation of VM on frontend now it depends on you how you want to add virtual nodes to your system. This time these nodes will be real virtual and you can associate your static IP with those.

Again we are back to insrt-ethers but this time we are logged-in to our Virtual frontend node that we created by combining one and more node connecting together. (this text is written in bold format because it is mind blowing concept and I have blown my mind many times while understanding this step but I really don’t want you to blow yours. :D)

AGAIN: I am shouting that we have to login to VM Front-node not real Front-node 😀

# insert-ethers

Select “Compute” as the appliance type. (This time select compute and you don’t have to worry about booting and setting up slave node because we are working Virtually!!! yeah man!  I am high I think at this point and songs are being played in my mind:D)

##########################################################################################

In another terminal session on vi-1.rocksclusters.org, we’ll need to set up the environment to send commands to the Airboss on the physical frontend. We’ll do this by putting the RSA private key that we created in section Creating an RSA Key Pair (e.g., private.key) on vi-1.rocksclusters.org.

Prior to sending commands to the Airboss, we need to establish a ssh tunnel between the virtual frontend (e.g., vi-1) and the physical frontend (e.g., espresso, where the Airboss runs). This tunnel is used to securely pass Airboss messages. On the virtual frontend (e.g., vi-1), execute:

# ssh -f -N -L 8677:localhost:8677 espresso.rocksclusters.org

###########################################################################################

Now we can securely send messages to the Airboss.

Did I tell you what is Airboss?

 

 

 

 

Now make sure you know mac address of your systems so you would be able to power them on/off using following command:

# rocks set host power <mac-address> key=private.key action=install

How to get MAC address?

# rocks list host macs <your-cluster-name> key=private.key

<your-cluster-name> is the real-font-node that you named while installing the Rocks on your system first time!

Above command will be give output like this: (yours output will be definitely different according to your compute nodes)

MACS IN CLUSTER  
36:77:6e:c0:00:02
36:77:6e:c0:00:00
36:77:6e:c0:00:03

when you will power on your real node in VM container you will see that it is
detected by VM containers as follows:

 

To turn your VM off following command should be executed:

# rocks set host power compute-0-0 key=private.key action=off

It was all about setting up virtual cluster and turning on/off your nodes in-between the VM containers. Let me clear one thing that VM container is one that contains various physical nodes those we can use in combined form to create virtual machines as big or as small we want. (:P  that looks like easy definition :P)

 

OK now that was most difficult part and if you have reached here just give yourself a BIG SABAASH!

 

If you are aware of  Virt-manager provided by the RED hat you are good to have smooth ride from here otherwise defiantly take a look at  (https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Administration_Guide/chap-Virtualization_Administration_Guide-Managing_guests_with_the_Virtual_Machine_Manager_virt_manager.html)

Now keep yourself in Real-Front-node as root user and make sure the required Virtual-clusters are running as required otherwise you will not be able to create virtual machines on Virtual containers. (I am using ‘virtual’ word so many times and  I am really not sure either I am in real world or virtual?)

As root user in Front-node run:

# virt-manager

You should be able to see your VM containers there and virtual-machines:

 

Voilaa!!!

Now Don’t ask What to do with your Virtual Machines. 😛
Here is our setup:

 

4377333377526847638account_id1-jpg

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: