SourceForge.net Logo

BuilDog - Tool for building software

Buildog was inspired by my need to track the steps involved in software builds. Particularly the software packages in the Linux From Scratch book.

It is not a package manager in the traditional sense, it will not track dependences. It tries to stay out of your way as much as possible.

If you grow weary of RPM spec files, or this months "package manager" (there are about a dozen or so of them out there, with new distributions adding their own every month) Buildog may be for you.

What it will do for you is log the information required to build it, files it created, URL's it needs (it will fetch them for you too if you want, with support for mirrors and caching) a log of the build stages, notes and descriptions, it's home page, and any other information you may wish to add.

It's pretty simple to use, if you've ever installed packages from source, you'll get the hang if it quickly.

Design inspiration - UNIX philosophy

I must admit, when I started the project I experimented with variations of spec files, including plain-text, XML, weird "shell-like" formats.. then it hit me..

What makes UNIX such a wonderful environment is it's toolbox approach - do one thing and use plain text so other tools can process it.

we already have the perfect "spec file format", the same one we've been using for years! The humble and infinitely flexible shell script! there is no need to learn a new set of configuration directives.

Typical session
$ buildog --template >SomePackage-1.0.7.bds

$ emacs SomePackage-1.0.7.bds

$ buildog --script SomePackage-1.0.7.bds

$ buildog --import

The above creates a template for you. (you could also invoke it from within your editor, it's just sent to standard output) You then edit this template with the commands required to retrieve and build the software. The buildog --script SomePackage-1.0.7.bds actually runs the build. The buildog --import imports the files into the tar file. (it inspects a MANIFEST.txt document, adds md5 checksums and various file attributes, creates a CATALOG, removes any temp files, creates a tar.gz file, copies it to it's database area and reindexes everything if CDB_File is available.)

The reason for using CDB_File is that the index is "read-mostly" data, if you don't have CDB_File installed, don't worry too much. Some of the functionality will be missing but it will still work. (this was important, since when building linux from scratch you probably won't want to deviate with things like installing CPAN modules)

Different stages.

Another problem area is what to do if a stage fails? Do you have to start all over again if make install fails? maybe it failed because you forgot to specify an option or set a permission or something simple like that. For compiling large projects (like glibc) the prospect of rebuilding an entire package simply because 'mkdir /tmp/foo' failed is not very appealing.

Typical Build failure
[yn] Would you like to run: [CONFIGURE]?>
Running: /home/joe/sandbox/Buildog/src/scripts/runner.sh Swish-e-2.4.0
Logging to: /usr/local/src/bUiLdOgScRaTcHdIr/CONFIGURE.log  /home/joe/
($func $*)                                                            
+ CONFIGURE                                                           
+ cd swish-build                                                      
/home/joe/sandbox/Buildog/src/scripts/runner.sh: cd: swish-build: No s
rc=$?                                                                 
+ rc=1                                                                
set +exv                                                              
+ set +exv                                                            
 Run: Swish-e-2.4.0.bds (Time: 0:0)
Completed (1) 
Swish-e-2.4.0.bds CONFIGURE Error(1)
        (e) .. Edit Swish-e-2.4.0.bds
        (v) .. View log
        (i) .. Ignore error
        (r) .. run again
        (!) .. shell
[!veirq] What would you like to do?>_

For the above reasons, buildog uses a unpack, ./configure, make, make check, make install approach. If any step fails, you are given the opportunity to edit the build script and continue from that point. (Note that you are not forced to use this pattern, in some cases it just doesn't apply)

The above example shows a failure in the CONFIGURE stage, (it didn't create a required directory) You can figure out what went wrong by hitting 'v' and looking at the log. After you find out what happened, hit 'e' to edit the Swish-e-2.4.0.bds file and insert the required mkdir command.

You don't need to start the process all over again simply because a directory did not exist. (in fact, you could simply hit '!', and create the directory, but then it wouldn't be reflected in the steps required to build it.)

Buildog features online help

The 'help system' is fairly simple but effective. (particularly from within an editor, emacs users just hit M-! buildog -qh)

Buildog help
bash-2.05$ buildog --qh
about - About buildog
build - The build process.
environment - Environment variables
import - Importing the build
install - How to install this thing.
interactive - When compile / install requires user interaction.
options - The command line options
qhelp - Using the documentation
script - Describes the format of the build script
url - Fetching URL's.
util - Utilities that can be loaded in to your build script.
wfiles - Working files of a package.
--
Use --qh "topic" for a specific topic
bash-2.05$ 

You just hit buildog -qh topic where the topic is one of those listed above. If there is no such topic it will search it's online help and provide a list of matching topics. (if there is only one topic, it simply dumps that topic out to you)

Accessing your data

Remember, we're talking tar here. :-) accessing your data can be as simple as inspecting the tar file and extracting the information you want. A simple 'awk' script could create a file for the md5sum program to use:

Checking the status of a package

bash-2.05$ tar zxf Gettext-0.12.1.tar.gz -O ./CATALOG.txt | awk '$1 ~/f/ {print $8" "$2}' | egrep -v "N/A" >/tmp/md5sum.txt

bash-2.05$ md5sum -c /tmp/md5sum.txt | egrep -v "OK"

bash-2.05$ rm /tmp/md5sum.txt


Obviously if you do this a lot, you may choose to put the above commands into a shell script.

If perl is your style, there are a couple libraries that allow you 'tied access' to your information. (Buildog::TiePackDB and Buildog::Pack) Using this, you can access your packages in the perlish way:

  while( ($name,$pack) = each(%mydb) ){
	# Do what you want with $pack here, 
        # it's an object that lets you 
        # access the packages contents. 
  }

If this type of access appeals to you (or you want to do your own thing with the packages, IE: create a custom query interface) The "public API" packages are documented in Pod, and are Buildog::TiePackDB and Buildog::Pack.

It also supports a --showpackage option to display some quick info about a given package (or list of packages)

Installation

Quite a bit of work has gone into making this install nicely. Perl programs are kind of notorious for being difficult to install.

It uses the autoconf to guess your location of perl, supports the usual --prefix --bindir --sysconfdir and --localstatedir options. Type 'make install' to install it, (a plain 'make' shows what will be installed where)