Tech:Installing

From Cyclopath

Warning: Please note that these instructions are incomplete and poorly tested, as we have thus far only used them internally.

Please see also The Cyclopath Open Source Install Guide which contains instructions for both Open Source and Core installs on a virtual machine.

Contents 1 Software Prerequisites 1.1 Debian packages 1.2 Postgres 1.2.1 Postgres itself 1.2.2 GEOS 1.2.3 PostGIS 1.2.4 Apache 1.3 Python packages that come in .egg files 1.4 Flex & Friends 1.5 MapServer 1.6 TileCache 1.6.1 Test TileCache 2 Basic installation instructions 3 tilecache_update.py cron job 4 Building Cyclopath 5 Configuration 5.1 Pyserver 5.2 Flashclient 5.3 Within the Database 5.3.1 Table draw_param

[edit] Software Prerequisites

These directions assume you are installing in $CPROOT Set this variable to the path of the location you wish to install the software in.

[edit] Debian packages

The following packages should be installed (the MySQL stuff is so cron jobs can access the Bugzilla database):

apache2
apache2-mpm-worker
apache2-modules
gdal-bin
libapache2-mod-python
libagg-dev
libboost-iostreams-dev
libboost-python-dev
libboost-program-options-dev
libboost-serialization-dev
libgeos-dev
libgdal-1.4.0
libgdal1-dev
libgd-dev
libltdl3-dev
libpam0g-dev
libpng12-dev
libpq-dev
libxslt1-dev
logcheck
postgresql-8.2
postgresql-client-8.2
postgresql-filedump-8.2
postgresql-server-dev-8.2
proj
pylint
python-beautifulsoup
python-gdal
python-lxml
python-mysqldb
python-profiler
python-psycopg
socket

The following packages should be removed (the versions in Ubuntu Hardy are stale):

dia-common
dia-gnome
dia-libs

[edit] Postgres

[edit] Postgres itself

Installed using deb package. Configuration:

1. Run scripts/permissions-cleanup.sh (also fixes Apache permissions).
2. Edit config files.
3. Move data directory (/var/lib/postgresql/8.2/main) somewhere that can accumulate a lot of space, and make a symlink from the old location to the new.
4. Edit /etc/logrotate.d/postgresql-common and add "create 640 root www-data" so logcheck can analyze it
5. Add to /etc/sysctl.conf (adjusting size appropriately):

   # Make postgresql happy by increasing the shared memory limits
   # 
   # segment size 256M -- units are bytes
   kernel.shmmax = 268435456
   

   Say sudo sysctl -p to make it take effect.

6. Create the cycling user within the database:

   createuser -U postgres -SDR cycling

[edit] GEOS

1. ./configure --prefix=$CPROOT/geos
2. make install
3. add $CPROOT/geos/lib to /etc/ld.so.conf
4. sudo ldconfig
5. fixperms --public $CPROOT/geos

[edit] PostGIS

Must be installed in standard location to make database dump/reload work.

1. ./configure --prefix=$CPROOT/postgis-1.3 --with-geos=$CPROOT/geos/bin/geos-config --with-pgsql=/usr/lib/postgresql/8.2/bin/pg_config
2. fixperms --public $CPROOT/postgis-1.3

[edit] Apache

Installed using deb package; config notes:

• Change "sleep 10" to "sleep 1" in /etc/init.d/apache2 (not on production box).
• sudo a2enmod deflate
• sudo a2enmod rewrite
• Tech:Sample Apache Config

[edit] Python packages that come in .egg files

(None for Hardy, but some info left in source comments in case it becomes useful again.)

[edit] Flex & Friends

• Flex SDK
• fcsh
• Flash Player 10, debug version: http://www.adobe.com/support/flashplayer/downloads.html. To get the trace output under Linux:
  1. Install the debug player.
  2. Create a file mm.cfg in your home directory with the following content:

     TraceOutputFileEnable=1
     ErrorReportingEnable=1
     MaxWarnings=0
     

  3. Open a terminal and watch the trace output with

     tail -F ~/.macromedia/Flash_Player/Logs/flashlog.txt
     

• The Firefox extension "FlashBlock" seems causes a slow application load, so be patient.

[edit] MapServer

These directions written against version 5.6.1.

If you are an internal developer, you can probably just copy the mapserv binary from another machine with the same architecture and Ubuntu version.

1. Download from [1] and untar.
2. Patch Makefile.in as follows (this is a workaround for a bug in MapServer; see [2]), making sure to replace $CPROOT first:

   --- Makefile.in.orig    2010-02-23 15:42:26.000000000 -0600
   +++ Makefile.in 2010-02-23 16:04:36.000000000 -0600
   @@ -254,7 +254,8 @@
    CXXFLAGS = @CXXFLAGS@ $(FLAGS)
    
    # Link flags and shared libs only
   -SUP_LIBS =  $(GD_LIB) $(AGG_LIB) $(OGL_LIB) $(FTGL_LIB) $(PDF_LIB) $(TIFF_LIB) $(PROJ_LIBS) \
   +SUP_LIBS = -L$CPROOT/geos/lib \
   +            $(GD_LIB) $(AGG_LIB) $(OGL_LIB) $(FTGL_LIB) $(PDF_LIB) $(TIFF_LIB) $(PROJ_LIBS) \
              $(JPEG_LIB) $(PNG_LIB) $(SDE_LIB) $(GDAL_LIB) $(MING_LIB) $(POSTGIS_LIB) \
             $(MYGIS_LIB) $(CURL_LIB) $(ORACLESPATIAL_LIB) $(GEOS_LIB) \
             $(THREAD_LIB) $(ICONV_LIB) $(FASTCGI_LIB) \
   

3. Unload the soft/qgis module (it has some libraries we don't want).
   • This is important! You will get subtle problems if you don't do this. If you have problems with libgdal, and ldd says "libgdal.so.1 => not found", unload soft/qgis and rebuild.
4. Build and install with:

   ./configure --with-wmsclient \
               --with-proj \
               --with-agg \
               --with-ogr  \
               --with-gdal \
               --with-freetype \
               --with-postgis=/usr/lib/postgresql/8.2/bin/pg_config \
               --with-geos=$CPROOT/geos/bin/geos-config
   make
   mkdir $CPROOT/mapserver
   cp mapserv $CPROOT/mapserver
   fixperms --public $CPROOT/mapserver
   

5. For each working directory, you'll need to either create a symlink from database.map to database.map.template or copy and edit.

[edit] TileCache

These directions written against 2.10.

1. Download from [3] and untar.
2. Patch as follows (prevents verbose, poorly buffered log spew):

   --- TileCache/Caches/Disk.py.orig       2010-03-19 17:50:28.000000000 -0500
   +++ TileCache/Caches/Disk.py    2010-03-19 17:52:16.000000000 -0500
   @@ -120,4 +120,5 @@
            try:
                os.rmdir(name)
            except OSError, E:
   -            print >>sys.stderr, "unlock %s failed: %s" % (name, str(E))
   +            if (E.errno != 2):
   +                print >>sys.stderr, "unlock %s failed: %s" % (name, str(E))
   

3. Install with (no build needed):

   mkdir $CPROOT/tilecache-cache
   chgrp www-data $CPROOT/tilecache-cache
   fixperms --public $CPROOT/tilecache-cache
   mv tilecache-2.10 $CPROOT/tilecache
   fixperms --public $CPROOT/tilecache
   cd $CPROOT/tilecache
   mv tilecache.cfg tilecache.cfg.orig
   ln -s $CPROOT/cp/mapserver/tilecache.cfg .
   

   (Note: this assumes that your Cyclopath working directory is at $CPROOT/cp.)

[edit] Test TileCache

Set an environment variable for the script and then run the script as user www-data.

export PYSERVER_HOME=$CPROOT/cp/pyserver/
cd $PYSERVER_HOME
sudo -u www-data -E $PYSERVER_HOME/tilecache_update.py -a

The script doesn't output to the terminal, but rather to a logfile. You can watch the progress of the operation by tailing the logfile.

tail -F /tmp/pyserver.tilecache.log

If the script runs for less than a minute, it most likely failed. To debug, see pyserver.tilecache.log and /var/log/apache2/error.log.

On some systems (like Ubuntu 9.04, which install libgdal 1.5 instead of 1.4), mapserv complains that it can't find libgdal.so.1, so make a link if this happens.

sudo ln -s /usr/lib/libgdal1.5.0.so.1.12.2 /usr/lib/libgdal.so.1

If the script runs for a number of minutes, it should have worked. Verify the tilecache cache directory now contains a bunch of graphics files.

find $CPROOT/tilecache-cache -name "*.*"

[edit] Basic installation instructions

(FIXME: This section is incomplete.)

To install Cyclopath:

• Install all prerequisite software.
• Arrange backups of the following
  • Database
  • /etc
  • $CPROOT
  • ... (anything else needed)
• Create tile directories & copy a base tileset from somewhere (e.g. tiles.tar.gz).
• Check out and build code.
• Fix permissions.
• Configure.
• Create /etc/init.d script to run routed

[edit] tilecache_update.py cron job

tilecache_update.py needs to be run periodically as user www-data in order to update TileCache. This can be a relatively heavyweight operation (a few minutes if there are changes that need to be processed), so tile currency needs to be balanced against resource usage. The script will complain if runs overlap.

Do this using cron (e.g., "sudo -u www-data crontab -e"). An interval of 15-30 minutes is probably reasonable; make sure you set the e-mail to go to an address you read. Sample cron for www-data:

MAILTO=$YOUR_EMAIL_ADDRESS

# m h  dom mon dow   command
0-59/30 * * * *  (cd $CPROOT/pyserver && ./tilecache_update.py --new)

[edit] Building Cyclopath

The Flashclient is built with make; the file Makefile contains build instructions.

The Pyserver does not need to compiled, but you do need to do a few things to make your changes available:

1. Fix permissions.
2. Restart Apache.
3. Restart routed.

Note: building the Flashclient creates the file which tells Pyserver what version it is.

[edit] Configuration

[edit] Pyserver

A template for the Pyserver configuration file is stored in the Subversion repository, pyserver/CONFIG.template. Copy this to pyserver/CONFIG and edit appropriately; it should be mostly self-documenting. The config file itself must not be checked into Subversion to prevent local options from inappropriately finding their way into the general codebase. You must watch carefully when you svn update for changes to the template, as new options may appear and the config file may need to be updated. It is reasonable when actively developing the configuration file to symlink the real file to the template rather than copying, but be very careful that you break this link when it is no longer needed.

[edit] Flashclient

The Flashclient has no per-installation configuration, so its configuration, Conf.as, is stored in Subversion. Be careful not to commit this file with debugging variables enabled. (Splitting this into template and real files like the Pyserver would preclude linking the two together because mxmlc, as of version 2.0.1, does not follow symlinks, and hard links don’t stay linked for reasons I haven’t figured out.)

[edit] Within the Database

Some configuration data is stored within the database itself. This is data needed by both the client and server, and frequently it is used directly in SQL queries.

[edit] Table draw_param

This table defines how the different classes of byways are drawn. It specifies, for each draw class and zoom level, with what width to draw the byways and whether to label them. Widths do not include the 1-pixel shadow. A missing draw_class, zoom pair indicates that byways of that draw class are not drawn at that zoom level.