Buildouts and Eggs

I have been resisting buildout ever since I looked at it last spring. At that time, there was only one document that I could find on the thing, and it was very confusing to me. Now, though, we have Martin Aspelli’s fabulous new Plone book, and I know several people that have made buildouts. Time to jump back in and see what all the fuss is about.

Buildout for Python is a way of building packages that brings in dependencies on it’s own. I use Plone, and buildouts seem to be the new preferred way of creating packages (or whatever they are calling them now) on the file system for development.

Unfortunately, using buildout didn’t come on it’s own for me. I knew how to use subversion before for checkouts and such, but I had to delve further into it and some project management theory before all was said and done. Learning about buildout also means learning about eggs, which I had also not yet had time to delve into.

I will break this into sections so that you can skip over any parts that you already know about.

Subversion

Organization

There are at least two different ways to handle the organization of subversion repositories (that I know of).

1. One repository, all projects inside it.
2. One repository for each project.

I have always been a fan of option 2, because it annoys me when someone checks in a change in a different project and the revision number for my project changes.

Handling packages the Zope 3/Plone 3 way opens the possibility of having subpackages. You have a major namespace like p4a, then sub packages under that namespace like pra.audio and p4a.video.

I started out creating subpackages as their own repositories in subversion, because they seemed like separate entities to me. I later found out that the standard way of handling this for my group is to have a repository for the main project, then the subprojects inside. Each of the subprojects has it’s own trunk, tags, branches tags.

package1 (Single repository)
|--package1.skin
 |--trunk
 |--tags
 |--branches
|--package1.buildout
 |--trunk
    |--bootstrap.py
    |--buildout.cfg
    |--products/
    |--README.txt
    |--src/
 |--tags
 |--branches

To be honest, I haven’t used this method enough yet to speak much to the merits. It doesn’t seem much more organized to me than the way I was handling things, but we will see how it turns out. This is the way that plone projects are being organized now, so maybe there is more to it than I realize.

Propedit

Propedit is a way of editing properties in subversion. I had never before realized that subversion had properites, so this was an interesting thing to find out. You can set a property on a file or directory.

svn:ignore

svn:ignore can be used to ignore all files and directories that fit within a specific pattern. Buildout creates a bin directory each time that you run it. You wouldn’t want to save this in subversion, because it will be different on different machines. We will use svn:ignore to have subversion ignore the bin directory, and .pyc files, for good measure.

Start with a directory that is checked out. For this example, a checked out buildout directory would be ideal.

$ svn propedit svn:ignore . #Set svn:ignore for the current directory

A file opens, in vim on my machine. Enter the following lines and save them.

bin
*.pyc

Now when you check in files or do an svn stat on this directory, all bin and .pyc files will be ignored.

svn:externals

svn:externals does additional checkouts into the directory that you set it on. Lets use the example subversion project layout I mentioned above, and see how you might use this.

$ svn co file:///path/to/package1/package1.buildout/trunk package1
$ svn propedit svn:externals src

A file opens again. Enter the skin package information here.

package1.skin file:///path/to/package1/package1.skin/trunk

Add additional lines to add additional packages. When you do an svn up on the main package directory, it will now check out package1.skin into the src subdirectory. This keeps the buildout directory lean, and the skin directory at the top level, where it belongs.

An interesting thing to note here is how to handle tags. Let’s say that you have done some work, and want to share your buildout with a friend. You also want to continue working on the skins package, but want your friend to work with the whole package as it is now. You create a tag for buildout, package1/package1.buildout/tags/1.0. But what about the changes to the skins package? Make a tag for skins as well, package1/package1.skin/tags/1.0. Then, update your svn:externals to reflect that tag. That particular package1.buildout tag will now only grab the 1.0 tag of package1.skin.

Eggs

Eggs are nice. I have been using them to install Python packages in classes, but had never before created one myself. It is much easier than I had thought it would be.

To get started, you need to install setuptools, if you don’t already have it. On Ubuntu, that’s apt-get install python-setuptools.

Here’s how to do it manually:

$ wget http://peak.telecommunity.com/dist/ez_setup.py
$ python2.4 ez_setup.py #Make SURE you use python 2.4, or you will get the wrong packages
$ easy_install ZopeSkel

Ok, now that that’s taken care of, let’s create an egg!

You can see the types of eggs available to build using paster create --list-templates. Let’s build a Plone theme.
$ paster create -t plone3_theme package1.theme

Add the results to the appropriate location in your checkout.
$ cp -r package1.theme/* package.buildout/src/package1.theme

We can also use this process to create the buildout files.
$ paster create -t plone3_buildout package1.buildout
The files are added to your existing package1.buildout directory, if you have one.

Check in your new files, and you now have a couple of working packages 🙂

Buildout

Finally, we get to buildout. We’ve already created a buildout using paster, but we probably need to make some changes to it.

Go into your buildout directory and open up buildout.cfg.

Look for the line that says “eggs =”. elementtree is already listed there, but we should also add our own egg. On the next line, add package1.skin. You will also want to add it under the next heading, “develop = “. In this location, however, you want to put the actual location, src/package1.skin.

Now move on to [productdistros]. You will see a line for “urls =”. Add links to any third party packages you want to use here. Also add the security hotfix http://plone.org/products/plone-hotfix/releases/20071106/PloneHotfix20071106.tar.gz

Look down to [instance], and notice that you can turn debugging on and off here. You will also notice the login you created with paster in plain text. This is not meant to be secure, you would change it in Plone for a live installation.

Notice another “eggs =” line, and list the eggs that you want installed for your instance, package1.theme again.

That should be everything you need for your buildout. Now would be a good time to do a subversion check in.

Now, let’s build this thing!

$ python2.4 bootstrap.py
$ ./bin/buildout

If you forget and use Python 2.5 to run bootstrap, it will screw up the files and you will get weird errors. Just do another checkout to fix them.

The ./bin/buildout step sometimes takes a while. It is downloading and installing everything that you told it to, and Zope is rather large.

When the buildout finishes, run ./bin/instance fg. Open up your browser, and go to http://localhost:8080 (or whatever port you chose). You now have a self-contained Zope installation! No more managing multiple versions of Zope and Plone, and trying to remember which one is installed where, or which instances go with which versions.

Advertisements

~ by Liz on November 9, 2007.

2 Responses to “Buildouts and Eggs”

  1. Thank you so much for the article! This is exactly the information I was seeking regarding the organization of my Plone development infra-structure. You saved my day!

    Regards,

    Marcelo.

  2. Yes indeed, nice write up! Thx

    Marcel

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: