Synnefo Developer’s Guide

The suggested method of setting up an environment for development purposes is first to install Synnefo on a Debian Jessie system and then to use Python’s development mode to run Synnefo from a cloned repo and see your changes instantly.

Synnefo installation

There are two main ways of installing Synnefo, each of which will be described below:

  • Existing Debian Jessie system (snf-deploy): This method requires that you have setup a Debian Jessie system. On this setup, you can install Synnefo using snf-deploy.
  • Synception (snf-ci): This method builds Synnefo within Synnefo (you read right), which means that you need to have an account in an existing Synnefo installation. Then, using snf-ci in conjunction with kamaki, you can create a new Debian Jessie VM in that installation. The rest is handled automatically by snf-ci, which uses snf-deploy to install Synnefo in that VM.

Note

The first method will build Synnefo from our stable branch. If your development takes place in develop or in a feature branch, then you may need to update your installation manually, e.g. run database migrations or install new packages.

Existing Debian Jessie system (snf-deploy)

1. Install Synnefo

In order to create a one-node Synnefo installation, you can follow the instructions here.

2. Install Devflow

Once you’re done, you will need to install devflow. Devflow is a tool that helps you to manage package versions, implement the git flow development process and build Python and Debian packages. You will need it to create your code’s version.

# pip install devflow

3. Clone Synnefo repo

First, you need to install git:

# apt-get install git

and clone the Synnefo repo, preferably as non-privileged user:

$ git clone https://github.com/grnet/synnefo.git

4. Configure the code’s version

Enter the directory where you have cloned the Synnefo repository and issue the following command:

$ devflow-update-version

Synception (snf-ci)

1. Install Kamaki

Kamaki is a cli tool for managing clouds. It implements the Openstack APIs and can be used to create VMs, networks and generally allow the user to do what he/she would normally do from the cloud’s UI.

If you haven’t installed kamaki yet, you can follow these instructions.

Once you have kamaki installed, you need to connect it to your account on an existing Synnefo installation (called cloud in kamaki lingo). To do so, you can consult the Setup section. Alternative, you can visit the API access page of your account in that cloud and download your personalized .kamakirc file.

2. Clone Synnefo repo

First you need to install git and fabric (the latter is used internally by the snf-ci script):

# apt-get install git fabric

and clone the Synnefo repo, preferably as non-privileged user:

$ git clone https://github.com/grnet/synnefo.git

3. Install Synnefo remotely on a VM

Enter the directory where you have cloned the Synnefo repository and then enter the ci folder. In this folder, you will find the snf-ci script. A common usage of snf-ci is the following:

$ ./ci/snf-ci create,build,deploy --cloud <cloud>

The above command will use your kamaki cloud that you have setup in Step 1. In this cloud, snf-ci will create a Debian Jessie VM, checkout the develop branch from the official Synnefo repo, build the Synnefo packages from source and install them using snf-deploy. Of course, all the previous actions can be tweaked with command-line arguments or configuration files. To see a list of possible command-line arguments, you can use snf-ci -h. Also, you can edit the ci_jessie.conf configuration file for more permanent changes.

Tip

You can use the –local-repo argument to instruct snf-ci to use the current branch. This means that you can install Synnefo from any branch, even your own.

Tip

You can view details for the created VM, such as IP, username, password etc., by doing cat /tmp/ci_temp_conf.

Development mode

At this point you should have a working Synnefo installation. The rest of the instructions will take place in that Synnefo installation.

1. Use Python’s development mode

From the top directory of your Synnefo repo, you can use the following script:

$ ./ci/install.sh

This means that every installed snf-* package will be overridden (python setup.py develop -N) with the code that exists in the currently checked-out branch of the cloned Synnefo repo. If you wish to leave the development mode, you can use another script:

$ ./ci/uninstall.sh

2. Change Gunicorn permissions

If you have cloned the repository as root, then Gunicorn will not be able to read your source files, since by default its user/group permissions are www-data. However, you can change the Gunicorn permissions by editing the /etc/gunicorn.d/synnefo configuration file and replacing every www-data instance with root.

Warning

Gunicorn’s should never run as root in production environments.

Accessing the Synnefo UI

If you want to access the Synnefo UI through your browser, you can take a look at the Accessing the Synnefo installation section.

Caveats regarding code changes

At this point, you would expect that every change you’ve made in your branch will be instantly visible to Synnefo. Most of the times this will be the case, but in certain circumstances you may need to restart a service or move files around:

  • If you have changed a Synnefo setting or Django view/url, you may need to restart Gunicorn with:

    # service gunicorn restart

  • If your changes affected snf-dispatcher (from package snf-cyclades-app) or snf-ganeti-eventd (from snf-cyclades-gtools) you will need to restart these daemons, too.

    # service snf-dispatcher restart
# service snf-ganeti-eventd restart

  • If you have edited a static file of a Synnefo component, you will need to copy it to the respective folder under /usr/share/synnefo/static/.

Logs

You will find useful information for debugging in the following files:

  • /var/log/gunicorn/synnefo.log, for most Synnefo components.
  • /var/log/apache2/other_vhosts_access.log, for issues with site configuration.
  • /var/log/postgresql/postgresql-9.1-main.log, for database issues.

Testing

Synnefo has two main testing endpoints:

  • The snf-ci script can create a VM from a custom branch and run all available tests on it.

  • The ./ci/tests.sh script can be used in an existing Synnefo installation to run all or specific tests. You can use it as following:

    $ ./ci/tests.sh [--dry-run] component1[.test] component2[.test] ...

Developer Guidelines

  • If you want to use transactions in your code, you must NOT use the default Django transactions (see the Synnefo transactions design doc). Depending on the models you want to edit, you must import the corresponding transaction implementation. For Astakos models do:

    from astakos.im import transaction

    while for Cyclades models do:

    from synnefo.db import transaction