SlapOS Home SlapOS

    SlapOS HowTo Format SlapOS Node

    FINAL - Document showing how to (re-)format a SlapOS node.
    • Last Update:2018-06-26
    • Version:001
    • Language:en

    Format SlapOS Node

    This document will cover formatting a SlapOS node. Formatting may be necessary on a newly installed SlapOS node in case the recommended single line installer (includes formatting) was not used.

    Formatting may also be necessary when a node needs to be repurposed for example to increase the number of computer partitions or adding more storage capacity.

    For this how-to command line access to the node is required. Please refer to the SlapOS core documentation for more information on the command line commmands available.

    Table of Content

    • Initial Formatting
    • Repurpose Existing Node
    • Reconfiguring Node for Virtual Machines
    • Reformat after adding SSD Storage

    Intial Formatting

    Initial formatting of a node is only required when installing a node from source or packages. The recommended single line installer already handles default formatting during the installation process.

    Formatting always includes registering a node with a master. This is not mandatory but recommend as a node is supposed to be run from a master. In order to register, a token (key and X509 certificate) are required. Follow the steps outlined in installing a Slave node to request a token before formatting a node.

    Register SlapOS node

    $ sudo su
    # slapos node register --interface-name lo --partition-number 20 [COMPUTER_NAME]

    To create the configuration files required to format a SlapOS node, choose a recognizeable computer name to identify the node in a network and run:

    sudo su
    # slapos node register --interface-name lo --partition-number 20 [COMPUTER_NAME]

    This will generate several files:

    • /etc/opt/slapos/slapos.cfg: The configuration of your SlapOS Node
    • /etc/opt/slapos/ssl/certificate: Your server SSL Cetificate
    • /etc/opt/slapos/ssl/key: Your server SSL Private Key

    Next the default slapos.cfg has to be configured for IPv6.

    IPv6 Support

    A node must have IPv6 installed. As described in the SlapOS architecture, IPv4 and VPN work in theory - provided availability of a large enough address space to add 100+ IPv4 adresses on every node.

    When using the single line installer, adding IPv6 is outlined in how to add IPv6 to a SlapOS node. The process for an unconfigured node is the same but some presets have to be made before adding IPv6. In the configuration file /etc/opt/slapos/slapos.cfg (consider eth0 to be the internet interface):

    # if using re6st add:
    interface_name = eth0
    ipv6_interface = lo
    
    # if eth0 already has IPv6, only add:
    interface_name = eth0
    
    # for tapVPN add:
    ipv6_interface = tapVPN
    interface_name = eth0

    Slapformat

    $ sudo su
    # slapos node format --now

    To format or reformat a SlapOS node the slapformat command is used (see all command line options). This will configure a node based on the configuration in /etc/opt/slapos/slapos.cfg. Running slapformat without creating or changing the configuration will only "ping" the slapos master

    $ sudo su
    # slapos node format --now
    2018-03-28 15:32:58 slapos[18922] INFO Updating computer
    2018-03-28 15:33:30 slapos[18922] INFO Posting information to 'https://slap.vifib.com/'
    2018-03-28 15:33:31 slapos[18922] INFO slapos successfully prepared the computer.

    If a node has not been registered or formatted before, slapformat will create 20 computer partitions along with users, their directories, tap interfaces and IPv6 addresses for every service of each computer partition (see SlapOS architecture for more info on what a node contains).

    Note, for initial formatting after registering a node, the alter_user option has to be set:

    $ sudo su
    # slapos node format --alter_user=True --now

    Repurpose Existing Node

    $ sudo su
    # slapos node format --now

    slapformat will look for the slapos.cfg file and update a node according to the configuration file. In case a node needs to be repurposed, the slapos.cfg has to be updated before slapformat is called. Then simply call slapos node format --now again.

    Reconfiguring Node for Virtual Machines

    example missing

    Hosting virtual machines and serving VM instances requires modifying the configuration of a SlapOS node - unless the node was installed from a dedicated disk image like used by Vifib master.

    Network Configuration

    example missing

    To host virtual machines, a bridge needs to be configured to support multiple tap interfaces used in virtual machines like a KVM before calling slapformat to acknowledge and modify the configuration. As using bridges on a computer/server can be tricky they are not enabled by default on SlapOS (unless using the Vifib master disk image).

    Note: KVM instances will have no internet access without a bridge being configured (no problem if only used for planning/testing). And note, that setting network configuration will not work over wifi.

    Debian Bridge

    # /etc/network/interfaces
    iface eth0 inet manual
    auto br0
    iface br0 inet dhcp
    bridge_ports eth0
    
    # /etc/rc.local
    /opt/slapos/bin/slapformat -c /etc/opt/slapos/slapos.cfg
    /opt/slapos/bin/bang /etc/opt/slapos/slapos.cfg

    On a Debian distribution (Debian, Ubuntu, Mint, et al.), setting up a bridge is done by changing the file /etc/network/interfaces and restarting the network. Make sure you add relevant physical interfaces to the bridge. Also ensure that if you add a bridge to a remote server, you still have a way to contact the server in case of failure of the configuration (this happens frequently).

    Generic Bridge

    # /etc/rc.local
    brctl addbr br0
    brctl addif br0 eth0
    ip l s dev br0 up

    On a generic GNU/Linux distribution, a bridge setup can be setup entirely from /etc/rc.local, together with slapformat. This is not the most beautiful way to achieve the expected result but it works. Note, that any interface added to a bridge should no longer be configured by the system, meaning the configuration file for that interface should be removed.

    Update SlapOS Configuration

    nano /etc/opt/slapos/slapos.cfg
    
    # under [slapformat] change parameters to:
    interface_name = br0
    create_tap = true
    
    # then call
    slapos node format --now

    Start by updating slapos.cfg to use the newly created br0 bridge as interface. Then call slapformat (which also creates partitions and allocates IPs) to also create the "TAP" virtual interfaces for each partition and connect them to the bridge. Note to make sure and use the lastest SlapOS version as the create_tap options was not available on older releases.

    KVM Configuration

    # modprobe kvm
    # modprobe kvm_intel
    # modprobe kvm_amd
    # chmod 777 /dev/kvm

    Load the required kernel modules and change the rights of /dev/kvm to 777.

    and under [slapformat] change the parameters to:

    interface_name = br0
    use_tap = true

    Note, for this to work, virtualization must be activated in the BIOS of your computer.

    You should be set.

    Reformat after adding SSD Storage

    This section will explain how to directly use newly mounted SSD storage disks from running services in SlapOS computer partitions.

    Additinal SSD may be required because of growing backup/log files or to attach a multi-storage, for example for reducing the I/O load on Cloudera data nodes with multiple virtual disks (one system disk, many data disks) by adding storage to eventually have a phyiscal disk per virtual disk.

    Mount and Configure Disks

    $ mkdir /data
    $ mkdir /data/data1 /data/data2
    $ mount -t ext2 /dev/sdb1 /data/data1
    $ mount -t ext2 /dev/sbc1 /data/data2
    
    # vim /etc/fstab
    # Retrieve STORAGE_SDB1_UUID and STORAGE_SDC1_UUID

    After plugging in the new disk(s) they need to be mounted. Assume you have two disks, /dev/sdb1 and /dev/sdc1 which should be used to extend partitions. The disks will be formatted like a normal SlapOS partition to allow deployed instances to access the disks directly from their partition folder.

    Start by creating a base storage folder /data where disks will be mounted and add data[x] folder for each storage to mount into our storage. Then mount the disks.

    In case of server reboot, we want to auto remount storage.

    # vim /etc/fstab
    UUID=STORAGE_SDB1_UUID              /data/data1                ext2    default    0 2
    UUID=STORAGE_SDC1_UUID              /data/data2                ext2    default    0 2

    where STORAGE_SDB1_UUID and STORAGE_SDC1_UUID can be retrieved using the command blkid.

    Enable Support for Attached Storage

    # vim /etc/opt/slapos/slapos.cfg
    [slapos]
    ...
    instance_storage_home = /data
    
    # then
    slapos node format --now -v

    After mounting SlapOS needs to be informed where to find the attached storage disk. This is done by editing the configuration in the SlapOS config file at /etc/opt/slapos/slapos.cfg and then setting the instance_storage_home parameter to the base storage folder created earlier.

    Finish by running slapos format to reconfigre the attached storage. When formatting finishes, the storage /data/data[x] will contain new folders created and managed by SlapOS to be used by slap partitions when new instances will be deployed.

    # ls /data/data2/
    total 120
    drwxr-x--- 2 slapuser0  slapuser0  4096 Mar 23  2015 slappart0
    drwxr-x--- 2 slapuser1  slapuser1  4096 Mar 23  2015 slappart1
    drwxr-x--- 2 slapuser10 slapuser10 4096 Nov  7 02:29 slappart10
    drwxr-x--- 2 slapuser11 slapuser11 4096 Nov  7 02:29 slappart11
    ...

    Each partition on the other hand will have one additional new folder called DATA which contains all mounted disks (data1 and data2). Instance can use them as part of their partition file system.

    # ls -l /srv/slapgrid/slappart19/
    total 96
    drwxr-x--- 2 slapuser19 slapuser19  4096 Jun 15 14:10 bin
    -rw-r----- 1 root       slapuser19  9376 Nov  8 12:24 buildout.cfg
    -rw-r----- 1 slapuser19 slapuser19  8282 Nov  8 12:24 buildout-switch-softwaretype.cfg
    drwx------ 2 slapuser19 slapuser19  4096 Jun 15 14:10 DATA

    The folder DATA also contains all mounted data folders:

    # ls -l /srv/slapgrid/slappart19/DATA/*/
    /srv/slapgrid/slappart19/DATA/data1/:
    total 0
    /srv/slapgrid/slappart19/DATA/data2/:
    total 0

    Note, that if the instance is destroyed, all attached storage content will be removed when SlapOS will clean up the partition.

    Thank You

    Image Nexedi Office
    • Nexedi SA
    • 147 Rue du Ballon
    • 59110 La Madeleine
    • France