Testing libpqxx

If you're looking at this page, chances are that you'll just want to run the libpqxx regression test to make sure things work. A lot more testing is done as part of development, however, and this page will also describe these procedures in increasing order of intensity.

At the end of this page you'll find a recipe for building, installing, starting, and using a local postgres server (and cleaning up afterwards) without superuser privileges—no su or sudo or logging in as root required. You can run a DBMS right in your own home directory. This can be useful for testing other software against multiple PostgreSQL versions like we do with libpqxx.

Simple test: "make check"

If you just want to make sure libpqxx works properly in your environment, or you're building a libpqxx package for your system environment that is tied to your PostgreSQL version of choice, you can run the built-in regression test: make check.

To prepare for this, set things up such that clients can log into a default database without specifying a host, database, user name, or password. The README gives a brief description of the environment variables you can use to set defaults for these parameters (e.g. PGHOST, PGDATABASE, PGUSER, PGPASSWORD). The postgres manual goes into more detail.

You can speed up the build on multiprocessing or multithreading systems by adding an option like "-j 4"; this tells make to run up to 4 processes (or whatever number you specify) simultaneously.

Pre-release test: "preprelease"

Before every release, we run tools/preprelease. This performs a plethora of checks:

  • It checks the latest entries in NEWS and debian/changelog against the VERSION file which defines the current libpqxx version.
  • It creates a libpqxx source tarball. All subsequent builds and runs are done from this distribution tarball to ensure that it is complete.
  • It builds libpqxx against a list of available compilers. The list defaults to a hard-coded list of g++ versions, but you can override this by setting the environment variable COMPILERS to a space-separated list of compiler executables. The script will skip any compilers it can't find. You may also set additional arguments for the configure script in CONFIG_ARGS.
  • It runs "make check" for each of the compiler-specific builds.

If you have multiple processors at your disposal, or at least a multithreaded one, you may want to set CONCURRENCY_LEVEL to the number of concurrent processes you want to run while building libpqxx. This can save you some serious time.

Extensive test: "pqxx-fulltest"

The tools/pqxx-fulltest script (Unix-like systems only, and only tested on GNU/Linux so far) consecutively starts different postgres backends entirely locally—on the local server, under the current user identity, in the current directory, without claiming a TCP/IP port etc. It then tests libpqxx against that backend using various libpq versions (using the preprelease script in the process).

An optional argument tells pqxx-fulltest where to find the libpqxx source you wish to test. If none is given, it will check out the latest development source tree. The script will also FTP into a PostgreSQL mirror site—host and postgres source directory can be overridden by setting FTPSERVER and FTPDIR respectively—and download all current postgres versions for testing. Everything is built from source, so things should work on any processor architecture and every Unix-like operating system.

The environment variables that customize preprelease will also work for pqxx-fulltest. On GNU/Linux, CONCURRENCY_LEVEL will default to the number of processors/threads on the system. The script will also make a very basic attempt to detect available compilers.

How to set up a private PostgreSQL cluster

It seems many people want to know how to set up and use a private postgres instance like pqxx-fulltest does, so here's an overview:

  1. Make postgres install and run in your home directory (or some other place you own) under your own uid. I'll use "$MYPG" as the example directory; it must be an absolute path, starting with "/" (forward slash). Run postgres' ./configure with the option "--prefix=$MYPG". Then "make && make install" postgres—no superuser rights required!
  2. Your private instance is going to have very lenient authorization settings by default. To make it truly private without having to configure postgres security, just make the thing inaccessible to other users: "chmod go-rwx $MYPG". This ensures that nobody else will have access to your private server socket.
  3. Add $MYPG/lib to the front of your LD_LIBRARY_PATH, $MYPG/bin to the front of your PATH, set PGDATA to $MYPG/data, and PGHOST to $MYPG.
  4. Having set up your environment, initialize your private database cluster by running initdb. No arguments are needed; the PGHOST and PGDATA variables carry all the information initdb needs.
  5. Still with those environment variables set, run "postgres -F -k $MYPG -h '' >"$MYPG/backend.log" 2>&1 &". The "&" (ampersand) makes it a background process, so you can continue working in the same shell. The -F option tells postgres not to bother flushing every change to disk—use it only if you're not keeping any serious data in this postgres instance, and it'll help performance. The -h option with an empty argument suppresses the hostname to serve from; this stops postgres from claiming a TCP port. We'll use the local socket only. And the -k option specifies where the instance should create its local socket; by keeping that in the instance's own directory we can avoid clashes with any other running postgres instances. The example command line dumps any output from the server to a file backend.log.
  6. If you ran postgres as a background process, wait for a bit to give it the chance to get up and running. A quick-and-dirty way to hack this up: just "sleep 10" to give it an ample ten seconds.
  7. There is no need to set up a user identity; you'll automatically be the superuser for this cluster. But you do need to set up a database. The easiest is to create one with your own username, since this is the one you'll log into by default: "createdb "$USER"". Make sure you still have your PATH and LD_LIBRARY_PATH set up correctly while you do this. Now if you run a postgres client such as psql, you should be logged into this database by default.
  8. Use your local database server! If you set up your environment as described above, this should just work. Try typing the "psql" command, and you should get the version belonging to the postgres you just started. If you want to use client programs and libraries belonging to a different postgres version, installed in some location $MYPQ, just add $MYPQ/lib to the front of your LD_LIBRARY_PATH and $MYPQ/bin to the front of your PATH. PGHOST should still be set to what it was before: $MYPG. If you wish to build libpqxx against your local libpq version, just make sure MYPG/bin is at the front of your PATH before you run its configure script.
  9. When you're done, stop your backend by sending it the INT signal. If you know its process id $PID, do this using "kill -INT $PID". Once the server has had a few seconds to die off, you can uninstall your custom postgres instance simply by deleting the entire $MYPG directory.