snf-burnin¶

snf-burnin is an integration testing tool for a running Synnefo deployment. Using the Synnefo REST APIs, it simulates a real user and tries to identify any bugs or performance issues by running a series of a tests. The tests are devided into the following categories:

Authentication Tests
Image Tests
Flavor Tests
Server Tests
Network Tests
Storage Tests

Usage¶

snf-burnin is a command line tool written in python. It supports a number of command line options though which the user can change the behaviour of the tests.

A typical usage of snf-burnin is:

snf-burnin --token=USERS_SECRET_TOKEN \
           --auth-url="https://accounts.synnefo.org/identity/v2.0" \
           --system-images-user=SYSTEM_IMAGES_USER_ID \
           --image-id=IMAGE_ID \
           --log-folder=LOG_FOLDER

The above options are the minimal ones (mandatory) that one has to speficy in order for snf-burnin to properly function. The first two are the credentials needed to access Synnefo’s REST API and can be found in the user’s dashboard. The third is needed by some Image Tests as we will see later. The forth tells snf-burnin which image to use for creating our test servers and the last one specifies the log folder where any results should be saves.

For more information about snf-burnin and it’s command line options, run snf-burnin with help.

Usage: snf-burnin [options]

snf-burnin runs a number of test scenarios on a Synnefo deployment.

Options:
  -h, --help            show this help message and exit
  --auth-url=AUTH_URL   The AUTH URI to use to reach the Synnefo API
  --system-images-user=SYSTEM_IMAGES_USER
                        Owner of system images
  --token=TOKEN         The token to use for authentication to the API
  --nofailfast          Do not fail immediately if one of the tests fails
                        (EXPERIMENTAL)
  --no-ipv6             Disables ipv6 related tests
  --action-timeout=TIMEOUT
                        Wait SECONDS seconds for a server action to complete,
                        then the test is considered failed
  --build-warning=TIMEOUT
                        Warn if TIMEOUT seconds have passed and a build
                        operation is still pending
  --build-fail=BUILD_TIMEOUT
                        Fail the test if TIMEOUT seconds have passed and a
                        build operation is still incomplete
  --query-interval=INTERVAL
                        Query server status when requests are pending every
                        INTERVAL seconds
  --fanout=COUNT        Spawn up to COUNT child processes to execute in
                        parallel, essentially have up to COUNT server build
                        requests outstanding (EXPERIMENTAL)
  --force-flavor=FLAVOR ID
                        Force all server creations to use the specified FLAVOR
                        ID instead of a randomly chosen one, useful if disk
                        space is scarce
  --image-id=IMAGE ID   Test the specified image id, use 'all' to test all
                        available images (mandatory argument)
  --show-stale          Show stale servers from previous runs, whose name
                        starts with `snf-test-'
  --delete-stale        Delete stale servers from previous runs, whose name
                        starts with `snf-test-'
  --force-personality=PERSONALITY_PATH
                        Force a personality file injection.
                        File path required.
  --log-folder=LOG_FOLDER
                        Define the absolute path where the output
                        log is stored.
  -V, --verbose         Print detailed output about multiple processes
                        spawning
  --set-tests=TESTS     Set comma seperated tests for this run.
                        Available tests: auth, images, flavors,
                        servers, server_spawn,
                        network_spawn, pithos.
                        Default = all

Log files¶

In each run, snf-burnin stores log files under the folder defined in the –log-folder parameter. For every run, it creates a new subfolder using a timestamp and the image-id as unique names. The name prefixes of the log files are:

details: Showing the complete log of snf-burnin run.
error: Showing the testcases that encountered a runtime error.
failed: Showing the testcases that encountered a failure.

Detailed description of testcases¶

Here we have a complete list of all the tests snf-burnin performs, each listed under the category in which it belongs. The user can choose to run some or all of the categories listed below using the “–set-tests” command line flag.

UnauthorizedTestCase¶

Use a random token and try to authenticate to Astakos service. The expected responce should be “401 Unauthorized”.

ImagesTestCase¶

Request from Cyclades the list of all registered images and check that its length is greater than 0 (ie test that there are registered images for the users to use).
Request from Cyclades the list of all registered images with details and check that is length is greater than 0.
Test that the two lists retrieved earlier contain exactly the same images.
Using the SYSTEM_IMAGES_USER_ID choose only the images that belong to the system user and check that their names are unique. This test can not be applied for all images as the users can name their images whatever they want.
Again for the images that belong to the system user check that the “osfamily” and the “root_partition” metadata values have been defined. These metadata values are mandatory for an image to be used.
Download from Pithos+ the image specified with the “–image-id” parameter and save it locally.
Create a new container to Pithos+ named “images”.
Upload the download image to Pithos+ under the “images” container.
Use Plankton service to register the above image. Set the “osfamily” and “root_partition” metadata values which are mandatory.
Request from Cyclades the list of all registered images and check that our newly registered image is among them.
Delete image from Pithos+ and also the local copy on our disk.

FlavorsTestCase¶

Request from Cyclades the list of all flavors and check that its length is greater than 0 (ie test that there are flavors for the users to use).
Request from Cyclades the list of all flavors with details and check that its length is greater than 0.
Test that the two lists retrived earlier contain exactly the same flavors.
Test that all flavors have unique names.
Test that all flavors have a name of the form CxxRyyDzz where xx is the vCPU count, yy is the RAM in MiB, and zz is the Disk in GiB.

ServersTestCase¶

Request from Cyclades the list of all servers with and without details and check that the two lists have the same length.
Test that simple and detailed servers lists have the same names.

SpawnServerTestCase¶

Submit a create server request to Cyclades service. Use the IMAGE_ID specified from the command line. If FLAVOR_ID was specified as well use that one, else choose one randomly. The name of the new server will start with “snf-test-” followed by a timestamp so we can know which servers have been created from snf-burnin and when. Also check that the response from Cyclades service contains the correct server_name, server_flavor_id, server_image_id and the status of the server is currenlty “BUILD”. Finally from the above response, extract the server’s id and password.
Request from Cyclades the list of all servers with details and check that our newly created server has correct server_name, server_flavor_id, server_image_id and the status is “BUILD”.
Request from Cyclades the details from the image we used to build our server. Extract the “os” and “users” metadata values. Using the first one update the server’s metadata and setup the “os” metadata value to be the same with the one from the image’s metadata. Using the second one determine the username to use for future connections to this host.
Retrieve the server’s metadata from Cyclades and verify that server’s metadata “os” key is set based on image’s metadata.
Wait until server changes state to ACTIVE. This is done by querying the service for the server’s state every QUERY_INTERVAL period of time until BUILD_TIMEOUT has been reached. Both QUERY_INTERVAL and BUILD_TIMEOUT values can be changed from the command line.
Request from Cyclades service a VNC console to our server. In order to verify that the returned connection is indeed a VNC one, snf-burnin implements the first basic steps of the RFB protocol:
- Step 1. Send the ProtocolVersion message (par. 6.1.1)
- Step 2. Check that only VNC Authentication is supported (par 6.1.2)
- Step 3. Request VNC Authentication (par 6.1.2)
- Step 4. Receive Challenge (par 6.2.2)
- Step 5. DES-Encrypt challenge, using the password as key (par 6.2.2)
- Step 6. Check that the SecurityResult is correct (par 6.1.3)
Request from Cyclades the server’s details and check that our server’s has been assigned with an IPv4 address.
Check that our server has been assigned with an IPv6 address. This test can be skipped if for some reason the targeted Synnefo deployment doesn’t support IPv6.
Test that our server responds to ping requests on IPv4 address.
Test that our server responds to ping requests on IPv6 address. This test can also be skipped.
Submit a shutdown request for our server.
Wait and verify that the status of our server became “STOPPED”.
Submit a start request for our server.
Wait and verify that the status of our server became “ACTIVE” again.
Test if server responds to ping on IPv4 address (verify up and running).
Test if server responds to ping on IPv6 address (verify up and running).
If the server is a Linux machine, SSH to it using its IPv4 address and verify that it has a valid hostname.
If the server is a Linux machine, SSH to it using its IPv6 address and verify that it has a valid hostname.
If the server is a Windows machine, try to connect to its RDP port using both its IPv4 and IPv6 addresses.
If during the creation of the server, the user chose a personality file to be used, check that this file is been presented in the server and its contents are correct.
Submit server delete request.
Wait and verify that the status of our server became “DELETED”.
Request from Cyclades the list of all servers and verify that our newly deleted server is not in the list.

NetworkTestCase¶

Submit create server A request.
Wait and verify that the status of our A server became “ACTIVE”.
Submit create server B request.
Wait and verify that the status of our B server became “ACTIVE”.
Submit create private network request. Wait and verify that the status of the network became “ACTIVE”.
Connect the two servers (A and B) into the newly created network. Wait and verify that both machines got an extra nic, hence have been connected to the network.
Reboot server A.
Test if server A responds to ping on IPv4 address (verify up and running)
Reboot server B.
Test if server B responds to ping on IPv4 address (verify up and running)
Connect via SSH and setup the new network interface in server A.
Connect via SSH and setup the new network interface in server B.
Connect via SSH to server A and test if server B responds to ping via their new interface.
Disconnect both servers from network. Check network details and verify that both servers have been successfully disconnected.
Send delete network request. Verify that the network has been actually deleted.
Send request to delete servers and wait until they are actually deleted.

PithosTestCase¶

Request from Pithos+ the list of containers and check that its length is greater than 0 (ie test that there are containers).
Test that the containers have unique names.
Create a new container. Choose a random name for our container and then check that it has been successfully created.
Upload a file to Pithos+ under our newly created container.
Download the file from Pithos+ and test it is the same with the one uploaded.
Remove created file and container from Pithos+ and verify that they have been successfully deleted.

Burnin as alert tool¶

Burnin can be used to verify that a Synnefo deployment is working as expected and verify the admins in case of an error. For this there is a script under the /snf-tools/conf directory named snf-burnin-run.sh which is intended to be used from cron to periodically run burnin. It runs simultaneous many instances of burnin for a number of different users and report errors though email.