The goal of this design document is to initiate the discussion regarding the implementation of the attach/detach operations for Synnefo Volumes. We will first document what is the current state of Synnefo (0.16.1 as of writing this) regarding Volumes, explain some Ganeti issues that make the implementation a bit tricky and then provide a design draft.
Since 0.16, Synnefo has provided an API for Volumes that is compatible to the Openstack Cinder API (see Block Storage API Guide). Using this API, the user can create Volumes, update and list them. There is just a minor nitpick, that the user cannot create a Volume without specifying a VM that it will be attached to.
Moreover, the user currently has no way to detach a volume from a VM and attach it to another.
Ganeti 2.14 has added support for the attach/detach volume operations. However, these operations still have some limitations and caveats.
The volume name that is generated by Ganeti and used by Synnefo follows roughly this form: UUID.disk_template.diskX. This means that during the disk’s lifetime, its name can change. Note that Ganeti can provide two names to the ExtStorage scripts:
For more info, you can read the related Ganeti doc. Archipelago currently uses VOL_NAME as the name for its volumes.
Ganeti does not support multiple disk templates for VMs.
Ganeti does not pass user-provided parameters for Ext templates during the removal of a disk.
In this section, we will propose a way to decouple the Create operation from the Attach operation and add a Detach operation. Also, we will suggest an altered Delete operation from the existing one.
Before showing the design behind the above operations, we will list a few compromises that are needed to make the attach/detach operations work:
Since the user is not aware of Ganeti nodes and clusters, we must provide an Attach operation that will work Synnefo-wide. This is achievable only with Archipelago volumes, which are visible from all Ganeti clusters.
In order to be able to reference the Archipelago volumes from different clusters, we have to resort in using the VOL_CNAME instead of VOL_NAME, since the first will remain the same throughout the Volume’s lifetime.
For more info regarding this issue, you can consult the Backend names of existing volumes section.
We must alter Ganeti in order to pass an option that will keep the disk data after the removal of a disk from the VM. The snf-ganeti package already has this option (--keep-disks), so we only need to fix a minor bug.
We will now proceed with the design of each action:
The Create operation currently is tightly coupled with the Attach operation, and requires the presence of a server_id argument in order to create and attach a Volume to a VM at once. In order to decouple these two operations and maintain backwards compatibility, we can make this argument optional.
Therefore, a Create operation with no server_id provided will simply do the checks that it does now and stop at the step where it stores a Volume entry in the database. The actual volume will be created once the user attempts to attach it to an instance.
The Attach operation will have two targets, a VM id and a Volume id. This operation will continue from where Create left off, i.e. it will send a disk creation job (gnt-instance modify --disk -1:add,name=<VOL_CNAME>,reuse_data=True) to Ganeti. The reuse_data Ext parameter should inform Archipelago to not create a new volume but reuse an existing one. Note that there are two cases here:
The Detach operation will not use the Ganeti detach operation as one would expect, but the remove operation (gnt-instance modify --keep-disks --disk <VOL_CNAME>:remove). As mentioned above, the --keep-disks will keep the disk data intact. In Archipelago terms, the detach ExtStorage script will be called but not the remove script.
The rationale behind this choice is to avoid having duplicate references to the same volume from different clusters, since the remove operation must operate only in one. Also, a detached Ganeti volume cannot be destroyed from Ganeti (see limitation 1), therefore it must be done from Archipelago. In this case, we do not want any reference of this volume to exist in any Ganeti cluster.
The Delete operation will have two cases:
The above design has some practical issues which must be tackled in order to have a functional Synnefo installation with detachable volumes.
The attach/detach feature cannot work out-of-the-box for existing Synnefo installations which have live Archipelago volumes. The reason is that the name of these volumes is the VOL_NAME ExtStorage parameter which cannot be used as it is not consistent across Ganeti clusters.
Preferably, we would like to change the name of the Archipelago volumes to match the one that is stored in the DB (VOL_CNAME). However, this is not easy to do, especially for live volumes. Thus, we suggest to do the opposite, i.e. read the Ganeti config of each Ganeti cluster, find all Archipelago volumes and store their Ganeti name in the Cyclades DB.
In order to do the above, we need to add a new field in the Volume model, since the backend_volume_uuid field is not an actual column in the DB, but a Python class property which has the following definition:
@property
def backend_volume_uuid(self):
return u"%svol-%d" % (settings.BACKEND_PREFIX_ID, self.id)
We propose the following change in the Volume class:
legacy_backend_volume_uuid = models.CharField("Legacy volume UUID in backend",
max_length=128, null=True)
@property
def backend_volume_uuid(self):
return (self.legacy_backend_volume_uuid or
u"%svol-%d" % (settings.BACKEND_PREFIX_ID, self.id))
With this change, we can:
Note
The above change needs a migration script to run before the new Archipelago version is installed in Synnefo. This migration script will be similar to the add_unique_name_to_disks script.
In order to be able to delete a detached volume, there has to be a helper server instance in an accessible Ganeti cluster. This means that the administrator must create some helper servers, preferably one for every Ganeti cluster, using the following command:
snf-manage server-create ... --helper --backend-id <id>
Also, for security reasons, the helper servers should be in stopped state, which means that the administrator must use the following command for each server:
snf-manage server-modify ... --action=stop
To make the administrator’s life easier, the above can be wrapped in an snf-manage command.
Synnefo’s current API implementation regarding Volumes is almost fully compatible with the OpenStack Cinder and Nova (os-volume_attachments) API. The only change that it needs to be marked as fully compatible is to lift the requirement of a server id during the creation of a Volume. The user will still be able to provide a server id, in order to retain the backwards compatibility, however it should no longer be necessary.