One of the great things that I enjoy about Python, are the decorators. I try to use them whenever possible in my Python projects, and sorely miss them at times in my Perl projects.
Last week I was thinking of profiling my Perl applications remotely and was pining for decorators. With Perl 5.10 lacking decorators, I created a Perl filter package to create “decorator like” syntax-sugar to support the emission of certain metrics like:
– Label assigned to function
– Time elapsed
– Return val
This triplet is then sent via UDP to some listener that can do with it as it may. Aggregate it, store it in a noSQL store, whatever.
At work I deal with lists of IP addresses all of the time, and I find it handy to be able to strip out all IP addresses from a region of a text file. It’s easy enough to pass a file through a filter to accomplish this, but I like to do it in buffer via vim so that I don’t have to resort to creating a scratch file.
The first function (mapped to CTRL-i) will extract all ips in the visual region, then the second function (mapped to CTRL-u) will remove repeat addresses.
Consider the following example: A class decodes a JSON string and inspects it for certain authentication tokens. These token can be anything representing an authenticated session, I’m using the simple boolean check for my example.
By using a decorator like the following:
You can use the decorator above any method that handles JSON requests for authentication.
At work, we’re looking into using GIT to replace SVN for our team SCM. I’m looking into “hosting” a repository on my workstation, and have outlined a few steps to make this as painless as possible.
Step 1: Create a local user “git”
This should be as easy as issuing “useradd git” as root, and setting all of the required permissions. I’m going to assume that the home dir for git is “/home/git”, and all of my scripts will make the same assumption.
Step 2: Create script to restrict SSH access to just git
We’re going to handle permissions by sharing SSH keys. But, in order to stop users from just SSHing into the machine, and messing with things, we’ll use this wonderful tiny script to handle this. Install the script as described in the page, set the permissions, and we’re done.
Step 3: Import a test SSH key
From another computer, generate a public SSH key, and paste the public key into git’s .ssh directory, within the file “authorized_keys”. Take note that we’ll restrict certain SSH actions:
Step 4: Create a utility script for creating new bare projects
We’re only concerned with handling new bare project at the moment, and not importing existing project. So, we can use this little shell script to do the needful:
Now, on the server, create a new project:
git@slowpoke:~$ bin/create_new_project.sh foo
Initialized empty Git repository in /home/git/tmp/foo/.git/
[master (root-commit) 988e2da] initial project
0 files changed, 0 insertions(+), 0 deletions(-)
create mode 100644 README
Cloning into bare repository /home/git/foo.git...
done.
And our project is now available for consumption.
Step 5: Clone from remote
On a remote machine (my laptop in this case) I’ll clone the new git project. Note, I’ve already had my public ssh key for my laptop installed under my workstation’s “git” user’s .ssh/authorized_keys file, so no extra authentication should be required.
The Flowchart outlines a simple path in which a developer committing to branch breaks the regressions tests, triggering the CI server to lock the repository from subsequent commits. The expectation is that the developer will be notified via build report that the branch is effectively broken, and they must ensure that their subsequent commit will pass all regression tests (note, the developer should be running regression tests against their branch before merging into the master, but not everyone will remember this). The SCM will be unlocked by some mechanism, be it a particular message indicating that the subsequent commit will fix it, re-opening the SCM to commits.
A successful regression test will trigger the SCM to be locked until the current build it successfully deployed. The staging server will create a new chroot image based on a meta file containing all software packages, creating said software packages from the most recent commit to the SCM.
Upon successful stage, the CI server will request from the Deployment server that this new chroot environment be deployed to whatever environment (QA, Testing, Production, etc) for any subsequent User Acceptance testing, or for the truly brave, live production.
Once deployed, the CI server will unlock the SCM, allowing developers to commit more merges into the master branch, thus triggering this process over again.
Looking at how to handle a continuous deployment system in conjuncture with a continuous integration system. My thoughts are to use a CI system like Jenkins to automate the testing of all recently committed git repositories, and to kick off a check list of dependencies.
The general idea would be:
1. Developer commits to git/master
2. CI pulls from git/master
2.1 CI runs all regression tests against git/master
2.1.1 CI returns report to developer and halts if any fail
2.2 CI Looks up the package manifest of a given project, and pushes project to staging server
3. Staging server determines which OS project belongs under (Debian, etc) and generates debootstrap image if none exist
3.1 Staging server creates clone of debootstrap image, and apply all .deb packages from manifest to image
3.2 Staging server stores image to shared file space
4. CI calls deployment server to deploy the image to a given environment
5. Deployment server checks manifest, and determines which class of server image is to be deployed, and attempts to deploy the image to appropriate Dom0 for the given environment, as the appropriate DomU