Format SlapOS Node
This document will cover formatting a SlapOS node. Formatting may be necessary on a newly installed SlapOS node in case the recommended single line installer (includes formatting) was not used.
Formatting may also be necessary when a node needs to be repurposed for example to increase the number of computer partitions or adding more storage capacity.
For this how-to command line access to the node is required. Please refer to the SlapOS core documentation for more information on the command line commmands available.
Table of Content
- Initial Formatting
- Repurpose Existing Node
- Reconfiguring Node for Virtual Machines
- Reformat after adding SSD Storage
Intial Formatting
Initial formatting of a node is only required when installing a node from source or packages. The recommended single line installer already handles default formatting during the installation process.
Formatting always includes registering a node with a master. This is not mandatory but recommend as a node is supposed to be run from a master. In order to register, a token (key and X509 certificate) are required. Follow the steps outlined in installing a Slave node to request a token before formatting a node.
Register SlapOS node
$ sudo su
# slapos node register --interface-name lo --partition-number 20 [COMPUTER_NAME]
To create the configuration files required to format a SlapOS node, choose a recognizeable computer name to identify the node in a network and run:
sudo su
# slapos node register --interface-name lo --partition-number 20 [COMPUTER_NAME]
This will generate several files:
/etc/opt/slapos/slapos.cfg
: The configuration of your SlapOS Node
/etc/opt/slapos/ssl/certificate
: Your server SSL Cetificate
/etc/opt/slapos/ssl/key
: Your server SSL Private Key
Next the default slapos.cfg
has to be configured for IPv6.
IPv6 Support
A node must have IPv6 installed. As described in the SlapOS architecture, IPv4 and VPN work in theory - provided availability of a large enough address space to add 100+ IPv4 adresses on every node.
When using the single line installer, adding IPv6 is outlined in how to add IPv6 to a SlapOS node. The process for an unconfigured node is the same but some presets have to be made before adding IPv6. In the configuration file /etc/opt/slapos/slapos.cfg
(consider eth0
to be the internet interface):
# if using re6st add:
interface_name = eth0
ipv6_interface = lo
# if eth0 already has IPv6, only add:
interface_name = eth0
# for tapVPN add:
ipv6_interface = tapVPN
interface_name = eth0
Slapformat
$ sudo su
# slapos node format --now
To format or reformat a SlapOS node the slapformat
command is used (see all command line options). This will configure a node based on the configuration in /etc/opt/slapos/slapos.cfg
. Running slapformat
without creating or changing the configuration will only "ping" the slapos master
$ sudo su
# slapos node format --now
2018-03-28 15:32:58 slapos[18922] INFO Updating computer
2018-03-28 15:33:30 slapos[18922] INFO Posting information to 'https://slap.vifib.com/'
2018-03-28 15:33:31 slapos[18922] INFO slapos successfully prepared the computer.
If a node has not been registered or formatted before, slapformat
will create 20 computer partitions along with users, their directories, tap interfaces and IPv6 addresses for every service of each computer partition (see SlapOS architecture for more info on what a node contains).
Note, for initial formatting after registering a node, the alter_user
option has to be set:
$ sudo su
# slapos node format --alter_user=True --now
Repurpose Existing Node
$ sudo su
# slapos node format --now
slapformat
will look for the slapos.cfg
file and update a node according to the configuration file. In case a node needs to be repurposed, the slapos.cfg
has to be updated before slapformat
is called. Then simply call slapos node format --now
again.
Reconfiguring Node for Virtual Machines
example missing
Hosting virtual machines and serving VM instances requires modifying the configuration of a SlapOS node - unless the node was installed from a dedicated disk image like used by Vifib master.
Network Configuration
example missing
To host virtual machines, a bridge needs to be configured to support multiple tap interfaces used in virtual machines like a KVM before calling slapformat to acknowledge and modify the configuration. As using bridges on a computer/server can be tricky they are not enabled by default on SlapOS (unless using the Vifib master disk image).
Note: KVM instances will have no internet access without a bridge being configured (no problem if only used for planning/testing). And note, that setting network configuration will not work over wifi.
Debian Bridge
# /etc/network/interfaces
iface eth0 inet manual
auto br0
iface br0 inet dhcp
bridge_ports eth0
# /etc/rc.local
/opt/slapos/bin/slapformat -c /etc/opt/slapos/slapos.cfg
/opt/slapos/bin/bang /etc/opt/slapos/slapos.cfg
On a Debian distribution (Debian, Ubuntu, Mint, et al.), setting up a bridge is done by changing the file /etc/network/interfaces
and restarting the network. Make sure you add relevant physical interfaces to the bridge. Also ensure that if you add a bridge to a remote server, you still have a way to contact the server in case of failure of the configuration (this happens frequently).
Generic Bridge
# /etc/rc.local
brctl addbr br0
brctl addif br0 eth0
ip l s dev br0 up
On a generic GNU/Linux distribution, a bridge setup can be setup entirely from /etc/rc.local
, together with slapformat. This is not the most beautiful way to achieve the expected result but it works. Note, that any interface added to a bridge should no longer be configured by the system, meaning the configuration file for that interface should be removed.
Update SlapOS Configuration
nano /etc/opt/slapos/slapos.cfg
# under [slapformat] change parameters to:
interface_name = br0
create_tap = true
# then call
slapos node format --now
Start by updating slapos.cfg
to use the newly created br0 bridge as interface. Then call slapformat (which also creates partitions and allocates IPs) to also create the "TAP" virtual interfaces for each partition and connect them to the bridge. Note to make sure and use the lastest SlapOS version as the create_tap options was not available on older releases.
KVM Configuration
# modprobe kvm
# modprobe kvm_intel
# modprobe kvm_amd
# chmod 777 /dev/kvm
Load the required kernel modules and change the rights of /dev/kvm to 777
.
and under [slapformat]
change the parameters to:
interface_name = br0
use_tap = true
Note, for this to work, virtualization must be activated in the BIOS of your computer.
You should be set.
Reformat after adding SSD Storage
This section will explain how to directly use newly mounted SSD storage disks from running services in SlapOS computer partitions.
Additinal SSD may be required because of growing backup/log files or to attach a multi-storage, for example for reducing the I/O load on Cloudera data nodes with multiple virtual disks (one system disk, many data disks) by adding storage to eventually have a phyiscal disk per virtual disk.
Mount and Configure Disks
$ mkdir /data
$ mkdir /data/data1 /data/data2
$ mount -t ext2 /dev/sdb1 /data/data1
$ mount -t ext2 /dev/sbc1 /data/data2
# vim /etc/fstab
# Retrieve STORAGE_SDB1_UUID and STORAGE_SDC1_UUID
After plugging in the new disk(s) they need to be mounted. Assume you have two disks, /dev/sdb1
and /dev/sdc1
which should be used to extend partitions. The disks will be formatted like a normal SlapOS partition to allow deployed instances to access the disks directly from their partition folder.
Start by creating a base storage folder /data
where disks will be mounted and add data[x]
folder for each storage to mount into our storage. Then mount the disks.
In case of server reboot, we want to auto remount storage.
# vim /etc/fstab
UUID=STORAGE_SDB1_UUID /data/data1 ext2 default 0 2
UUID=STORAGE_SDC1_UUID /data/data2 ext2 default 0 2
where STORAGE_SDB1_UUID and STORAGE_SDC1_UUID can be retrieved using the command blkid.
Enable Support for Attached Storage
# vim /etc/opt/slapos/slapos.cfg
[slapos]
...
instance_storage_home = /data
# then
slapos node format --now -v
After mounting SlapOS needs to be informed where to find the attached storage disk. This is done by editing the configuration in the SlapOS config file at /etc/opt/slapos/slapos.cfg
and then setting the instance_storage_home parameter to the base storage folder created earlier.
Finish by running slapos format to reconfigre the attached storage. When formatting finishes, the storage /data/data[x]
will contain new folders created and managed by SlapOS to be used by slap partitions when new instances will be deployed.
# ls /data/data2/
total 120
drwxr-x--- 2 slapuser0 slapuser0 4096 Mar 23 2015 slappart0
drwxr-x--- 2 slapuser1 slapuser1 4096 Mar 23 2015 slappart1
drwxr-x--- 2 slapuser10 slapuser10 4096 Nov 7 02:29 slappart10
drwxr-x--- 2 slapuser11 slapuser11 4096 Nov 7 02:29 slappart11
...
Each partition on the other hand will have one additional new folder called DATA which contains all mounted disks (data1 and data2). Instance can use them as part of their partition file system.
# ls -l /srv/slapgrid/slappart19/
total 96
drwxr-x--- 2 slapuser19 slapuser19 4096 Jun 15 14:10 bin
-rw-r----- 1 root slapuser19 9376 Nov 8 12:24 buildout.cfg
-rw-r----- 1 slapuser19 slapuser19 8282 Nov 8 12:24 buildout-switch-softwaretype.cfg
drwx------ 2 slapuser19 slapuser19 4096 Jun 15 14:10 DATA
The folder DATA also contains all mounted data folders:
# ls -l /srv/slapgrid/slappart19/DATA/*/
/srv/slapgrid/slappart19/DATA/data1/:
total 0
/srv/slapgrid/slappart19/DATA/data2/:
total 0
Note, that if the instance is destroyed, all attached storage content will be removed when SlapOS will clean up the partition.
Thank You
- Nexedi SA
- 147 Rue du Ballon
- 59110 La Madeleine
- France