Log all stdout and stderr output of every invocation of snf-manage, on unique filenames under a given directory.
All Synnefo’s management commands are written as custom django-admin commands. This means that every management command is in fact a class that extends Django’s BaseCommand class.
Django’s BaseCommand provides the attributes self.stdout and self.stderr and Django’s documentation encourages the users to use these attributes if they wish to write to the console. Django doesn’t provide an option to write the output to files and the user has to implement this explicitly when implementing the handle method.
We would like to extend the above mechanism to allow every snf-manage command to log all stdout and stderr output on a unique filename under a given directory. The implementation should change nothing in the way that users write management commands (only acceptable change is that the new commands may have to inherit a new class and not the BaseCommand one). This means that existing management commands should play out of the box and also that the logging mechanism will globally apply to all of them.
A new Synnefo setting named LOGGER_EXCLUDE_COMMANDS has been added that specifies which commands will not be logged. By default, commands that do not alter the state of the server (i.e. *-list and *-show commands) will be excluded from the logging mechanism. One can disable this logging mechanism all together by setting the above variable to ”.*”.
In this section we will try to explain the way that the new logging mechanism will be implemented as well as the reasons behind these decisions.
As we previously saw, we want the logging mechanism to be global and to work for all the snf-manage commands without extra effort. This means that the management commands will continue to use the self.stdout and self.stderr attributes from BaseCommand class to provide console output. Therefor we have to provide our own self.stdout and self.stderr objects that will preserve the previous functionality and log to files at the same time. There are two ways to achieve that:
Patch the Django’s BaseCommand class and replace self.stdout and self.stderr attributes.
This solution requires the minimum amount of changes to the management commands’ code as they will use our patched version of BaseCommand. The downside is that we have to patch a library provided class. We are not encouraging these type of patches because it obfuscates the code (the programmer is expecting to use Django’s BaseCommand class, not ours) and does not preserve compatibility with other Django versions (if the implementation of Django’s BaseCommand changes our patch will not work).
Create a new class that extends Django’s BaseCommand.
The downside of this solution is that we have to change the existing code so all management commands will inherit our new class and not Django’s BaseCommand. But we find this solution to be cleaner.
For the above reasons we decided to go with the second option.
Django’s self.stdout and self.stderr are implemented as OutputWrapper objects. We will create our own class (SynnefoOutputWrapper) which will use python’s logging library to handle the file part of the logging and the original OutputWrapper object to handle the console part (since we want to preserve the functionality of Django’s OutputWrapper and the style functions it uses to pretty print the messages).
Our new class has to be a descriptor. This is because BaseCommand doesn’t initialize the stdout and stderr attributes at __init__ but sets them only when it needs to (meaning inside the execute method).
The above classes will be written in snf-django-lib package meaning that all the other packages will have a dependency in snf-django-lib.
We will combine timestamp, command name and PID to form unique names, e.g.: 20140120113432-server-modify-4564, where “4564” was the PID. The timestamp will be first so that files will be chronologically sorted.
The implementation will follow the following steps: