Tech:Open Source/Install Guide
From Cyclopath
These are detailed installation notes for setting up Cyclopath Open Source on a Ubuntu VirtualBox. They also include notes for CP team core developers. NOTE: Parts of these instructions are still poorly tested and may be incorrect and/or incomplete.
Contents |
[edit] Install OS
If you have not done so, install VirtualBox on your machine.
Unless you have a machine dedicated to Cyclopath development, it's recommended that you install Cyclopath in a fresh Linux distribution using a virtual machine. This minimize the risk that you'll mess up your computer.
It is highly recommended to review the VirtualBox documentation for your version of VB and host OS. The instructions below were written for VirtualBox 3.1.2 installed on a Mac OSX. so instructions may be different. Always refer to current VB documentation.
also available:
[edit] Download Ubuntu
Download an Ubuntu iso. This is file that is designed to be burned onto a CD for installation, but it is not necessary to do so; VirtualBox can use the iso file as downloaded.
These instructions originally written for Ubantu 9.04 Desktop version. If using a different version, instructions may vary.
Go to the ubuntu site, go to the complete list of download mirrors: http://www.ubuntu.com/getubuntu/downloadmirrors (I found that downloading from the Ubuntu site extremely slow, especially during the day.) The UofM site seems like a good choice, unless another site is closer to you.
[edit] Setup/Install Ubuntu VirtualBox guest
Consult the User Manual and other resources at http://www.virtualbox.org for complete instructions.
Click the New button, and follow the instructions in the wizard. (I picked 25 gigabytes for the dynamic hard drive, since I didn't think 8 gigabytes would be enough.)
Click the Start button to install the OS.
In the First Run Wizard, under Select Media Mode; Click the folder icon next to the Media Source. In the Virtual Media Manager dialog box, click Add, browse to the previously downloaded iso file, select it and click Open. Then click Select.
After the wizard finishes, you will install the Ubuntu OS. Ubuntu is designed to be simple as possible to install. Keep in mind the Ubuntu is installing in a virtual machine environment provided by VirtualBox, but Ubuntu thinks it is installing on a real machine. The partition Ubuntu finds will be the virtual one supplied by VB (according to your instructions).
After installation, reboot as instructed by Ubuntu. You will be ask to remove the CD. Press the mouse pointer release button, to get back on your real system. On the VB menu, Click on Devices->CD/DVD and either select the host CD, or select unmount CD. Then click on the virtual guest window and press enter to finish booting.
As with any new OS installation, run the update manager to get the OS up to date.
[edit] Guest Additions
It is not necessary but adding guest additions is useful for resizing the guest window, and allowing to seamless moves between the guest window and your host windows. To install click on the devices menu, and click add guest additions. Then in the guest window, click on the VBOX additions “CD” icon, and run the autorun.sh. When finished reboot the guest. See VB documentation for more details.
[edit] Basic Linux Customization (Not Cyclopath-necessary)
[edit] Create Cyclopath Bash Script
These instructions assume a single user, so for simplicity everything is installed from the users home directory. If it's a multiple user installation, then $CPROOT should be set elsewhere, and appropriate permission commands added. CP Core team uses /export/scratch
For single users, substitute your Ubuntu login name whereever !USER! appears in the instructions.
In your home directory (type 'cd' and enter to get there)
Edit a new file with gedit or use your favorite editor:
gedit .bashrc-cyclopath
Add the following contents to the file:
## Default mask # Set to r/w/x for owner and group, and just execute for everyone # else, which is good for group-based development. #umask 0006 # Set $CPROOT (remember to replace !USER! CPROOT='/home/!USER!' ## Cyclopath paths # Local Flex compiler export PATH=$CPROOT/flex/bin:$PATH # Remote GroupLens scripts #export PATH=/project/Grouplens/bin:$PATH ## Cyclopath shortcuts # Restart Apache alias re='sudo /etc/init.d/apache2 restart' # Watch the Server logs alias logs='sudo tail -F /var/log/apache2/access.log \ /var/log/apache2/error.log \ /var/log/postgresql/postgresql-8.3-main.log \ /tmp/pyserver.apache.log \ /tmp/pyserver.routed.log \ /tmp/pyserver.tilecache.log' # Watch the Client logs alias logc='tail -F ~/.macromedia/Flash_Player/Logs/flashlog.txt' # Login to the Database alias cpdb='psql -U cycling cycling' # Run routed (from $cp/pyserver) alias rd='sudo -u www-data ./routedctl'
Note: CP development team core members; uncomment the umake and Remote Grouplens export PATH statements, and change /home/!USER! to /export/scratch (or maybe /export/scratch/!USER!).
Now edit your .bashrc file and add the following to the end of the file:
source ~/.bashrc-cyclopath
Close Terminal and open a new one to see if these settings take effect. (For example, type: echo $PATH).
[edit] Install & Configure Packages
For a complete list of packages to install, see:
[edit] Postgres
Type the following to install Postgres:
sudo apt-get install \ postgresql \ postgresql-8.3 \ postgresql-client \ postgresql-client-8.3 \ postgresql-client-common \ postgresql-common \ postgresql-filedump-8.3 \ postgresql-server-dev-8.3
Edit /etc/logrotate.d/postgresql-common and add "create 640 root www-data" so logcheck can analyze it
CP team core developers: Run scripts/permissions-cleanup.sh (also fixes Apache permissions).
CP team core developers may need to move data directory
Edit /etc/sysctl.conf
gksudo gedit /etc/sysctl.conf
Add these lines near the beginning
#Make postgresql happy by increasing the shared memory limits # # segment size 1G kernel.shmmax = 1073741824
Create user cyclopath:
sudo -u postgres createuser -U postgres -SDR cycling
[edit] Ident Authentication Maps
Edit Ident Authentication Maps:
gksudo gedit /etc/postgresql/8.3/main/pg_ident.conf
Add these lines (to end of file) that are not already there (replace landonb with your login for the OS):
# MAPNAME IDENT-USERNAME PG-USERNAME local postgres postgres local !USER! postgres local www-data cycling
Note: The preceding instructions may not be sufficient to avoid user authorization errors when the server is run. If you are a sole user for this machine, you can set the following to avoid these errors:
Edit:
gksudo gedit /etc/postgresql/8.3/mainpg_hba.conf
At the bottom of the file there is a table. Change the METHOD on two of the lines so it looks like this:
# Database administrative login by UNIX sockets local all postgres trust # TYPE DATABASE USER CIDR-ADDRESS METHOD # "local" is for Unix domain socket connections only local all all trust
Restart postgres:
sudo /etc/init.d/postgresql-8.3 restart
[edit] GEOS
First need to install C++ stuff or it won't build:
sudo apt-get install build-essential
cd $CPROOT wget http://download.osgeo.org/geos/geos-3.2.0.tar.bz2 tar xvf geos-3.2.0.tar.bz2 cd geos-3.2.0 ./configure --prefix=$CPROOT/geos make make check make install
Next, edit ld.so.conf
gksudo gedit /etc/ld.so.conf
It should have just one line. Add the location of the GEOS lib (replace path with value of $CPROOT):
include /etc/ld.so.conf.d/*.conf /home/!USER!/geos/lib
Reload:
sudo ldconfig
Finally, fix the permissions on the geos directory
chmod 2755 $CPROOT/geos
CP Core team only: fixperms --public /export/scratch/geos
[edit] PostGIS
Get prerequiste tool:
sudo aptitude install flex
wget http://postgis.refractions.net/download/postgis-1.3.6.tar.gz tar xvf postgis-1.3.6.tar.gz cd postgis-1.3.6
Configure the source to install locally, to use geos, and to use Psql, and then build and install it.
./configure --prefix=$CPROOT/postgis-1.3 \ --with-geos=$CPROOT/geos/bin/geos-config \ --with-pgsql=/usr/lib/postgresql/8.3/bin/pg_config make make install
Fix the permissions of the installation
chmod 2775 $CPROOT/postgis-1.3
CP Core team only: fixperms --public $CPROOT/postgis-1.3
[edit] Python
Ubuntu 9.04's package manager installs 2.6.2, and Cyclopath currently runs on Python 2.5.2. Cyclopath is known to run on Python 2.6, but there's no guarantee everything will work perfectly. Open Source developers can choose to skip downloading and compiling 2.5.2. CP team core developers click here
Install the following additional Python 2.6 packages:
sudo aptitude install python2.6-lxml python2.6-psycopg python2.6-psycopg2 python2.6-gdal
Note: Possibly need also to install: python-psycopgda
[edit] Tile Generation and Cache
Instructions are not yet available. For now, please see the old installation instructions.
[edit] Apache
sudo aptitude install apache apache2-mpm-worker apache2-utils apache2.2-common libapache2-mod-python
Note: I found the following already "1" after install, but you can check to make sure:
Change "sleep 10" to "sleep 1" in /etc/init.d/apache2 (not on production box).
This is only for CP dev core members: Make the logs group-readable.
sudo gvim /etc/logrotate.d/apache2
In the block, find 'create 640 root adm' and change to:
create 640 root grplens
(Everyone) Do these:
sudo a2enmod deflate sudo a2enmod rewrite sudo a2enmod mod_python sudo a2enmod headers
In order to use localhost, edit httpd.conf. If you have an actual domain name, use that; there may be more things you need to do which I don't know about. These instructions will let you test CP on the same (virtual) machine.
gksudo gedit /etc/apache2/httpd.conf
Add the following at the end of file:
ServerName localhost
Finally, restart Apache:
sudo /etc/init.d/apache2 restart
[edit] Configure sites-available
Rather than use httpd.conf, Ubuntu uses sites-available to configure Apache.
gksudo gedit /etc/apache2/sites-available/cyclopath
Get a copy of someone's existing file or click here, paste to new file, change the paths and usernames in the file to reflect your machine paths and your username.
These are CP Core team only instructions for getting sites-available/cyclopath:
sudo rsync -t -a -v landonb@foobar:/etc/apache2/sites-available/cyclopath /etc/apache2/sites-available/cyclopath sudo chown !USER! /etc/apache2/sites-available/cyclopath chgrp grplens /etc/apache2/sites-available/cyclopath
(Everyone) enable cyclopath in Apache:
sudo a2ensite cyclopath sudo a2disite default
This is equivalent to:
sudo ln -s /etc/apache2/sites-available/cyclopath /etc/apache2/sites-enabled/cyclopath sudo rm /etc/apache2/sites-enabled/000-default
Then, either reload or restart:
sudo /etc/init.d/apache2 reload sudo /etc/init.d/apache2 restart
[edit] Flex SDK
Download Adobe Flex 3.4 SDK; currently it's at:
http://opensource.adobe.com/wiki/display/flexsdk/download?build=3.4.0.9271&pkgtype=1
Create $CPROOT/flex directory, extract zip to there.
[edit] Flash Plugin & Browsers
Firefox browser comes preinstalled on Ubuntu. To install other versions, see: Install Firefox and Flash, or other browsers, see Notes on Other Browsers.
To install the debug flash 10 plugin (32 bit):
Download flash 10 debug plugin; currently at:
http://download.macromedia.com/pub/flashplayer/updaters/10/flash_player_10_linux_dev.tar.gz
Extract downloaded file.
Open the README file for instructions on installing the plugin.
To get the trace output under Linux:
Create a file mm.cfg in your home directory with the following content:
TraceOutputFileEnable=1 ErrorReportingEnable=1 MaxWarnings=0
Open a terminal and watch the trace output with
tail -F ~/.macromedia/Flash_Player/Logs/flashlog.txt
The Firefox extension "FlashBlock" seems causes a slow application load, so be patient.
[edit] 64-bit Flash
The Flash debug plugin is 32-bit. If you're running 64-bit Linux, refer to Installing the 64-bit Flash Debug Plugin.
[edit] Mapserver & Tilecache
These instructions may be incomplete Prerequite packages:
cd wget http://mirror.its.uidaho.edu/pub/savannah/freetype/freetype-2.3.10.tar.bz2 tar xvf freetype-2.4.10.tar.bz2 cd freetype-2.3.10 ./configure make make install
cd wget http://www.libgd.org/releases/gd-2.0.35.tar.bz2 tar xvf gd-2.0.35.tar.bz2 cd gd-2.0.35 ./configure make make check make install
sudo aptitude install libgdal-dev gdal-bin sudo aptitude install libexpat1-dev libxslt1-dev libagg-dev sudo aptitude install libboost-iostreams-dev libboost-python-dev libboost-program-options-dev libboost-serialization-dev sudo aptitude install libpam0g-dev libpng12-dev libpq-dev libxslt1-dev logchec sudo aptitude install proj pylint socket sudo aptitude install python-beautifulsoup python-mysqldb python-profiler sudo aptitude install libreadline-dev
Mapserver:
cd wget http://download.osgeo.org/mapserver/mapserver-5.6.1.tar.gz tar xvf mapserver-5.6.1.tar.gz cd mapserver-5.6.1 ./configure --with-proj --with-agg --with-ogr --with-gdal --with-freetype --with-postgis=/usr/lib/postgresql/8.3/bin/pg_config --with-geos=$CPROOT/geos/bin/geos-config make mkdir $CPROOT/mapserver cp mapserv $CPROOT/mapserver fixperms --public $CPROOT/mapserver
Tilecache:
cd wget http://tilecache.org/tilecache-2.10.tar.gz tar xvf tilecache-2.10.tar.gz mkdir $CPROOT/tilecache-cache chgrp www-data $CPROOT/tilecache-cache fixperms --public $CPROOT/tilecache-cache mv tilecache-2.10 $CPROOT/tilecache fixperms --public $CPROOT/tilecache cd $CPROOT/tilecache mv tilecache.cfg tilecache.cfg.orig ln -s $CPROOT/cp/mapserver/tilecache.cfg .
[edit] Setup for Group Development
It is especially important for core developers to get setup properly for group development. Independent Open Source users can skip this step.
[edit] Building Cyclopath
Cyclopath is open source software under a slightly modified version of the Apache license; see the README file details.
Get source code from svn. Refer to section in Source_Code document:
[edit] Flashclient & Pyserver
The Flashclient is built with make; the file Makefile contains build instructions.
cd $CPROOT/cp/flashclient make
If the build fails with the message java not found, then get java (note: I don't know if the plugin & fonts are actually needed):
sudo apt-get install sun-java6-jre sun-java6-plugin sun-java6-fonts
If you get m4 errors, then get:
sudo apt-get install m4
Prepare the Pyserver configuration file:
cd $CPROOT/cp/pyserver cp CONFIG.template CONFIG gedit CONFIG
Change all occurrences of /export/scratch/reid to the value in $CPROOT. Also change the email stuff if appropriate. Change other settings in CONFIG as appropriate.
The Pyserver does not need to compiled, but you do need to do a few things to make your changes available:
Fix permissions: Download the following into cp: http://magic.cyclopath.org/source/fixperms
Make file executable. Run fixperms.sh:
./fixperms.sh
Restart Apache.
sudo /etc/init.d/apache2 reload sudo /etc/init.d/apache2 restart
Restart routed. [only if routed has been activated (not necessary)]
Note: building the Flashclient creates the file which tells Pyserver what version it is.
[edit] Initialize database
Download the following:
http://magic.cyclopath.org/source/data.sql http://magic.cyclopath.org/source/schema.sql
This is the part that is likely to be especially rocky for early adopters.
Do a search and replace /export/scratch/reid/postgis-1.3 in schema.sql with the path where you installed PostGIS.
Create database and Run schema.sq & data.sql
sudo -u postgres createdb cycling sudo -u postgres psql cycling < schema.sql sudo -u postgres psql cycling < data.sql
You now have a valid, but empty database.
The following will 'seed' the db tables. Open a SQL shell with:
psql -U cycling
then run:
insert into revision (id, timestamp, host) values (rid_inf(), now(), '_DUMMY');
and check the results (type the first line), should look like:
select * from revision where id = rid_inf(); id | timestamp | host | username | comment | bbox | geosummary | geometry | skip_geometry ------------+----------------------------+--------+----------+---------+------+------------+----------+--------------- 2000000000 | 2008-05-28 17:19:57.370999 | _DUMMY | | | | | | f (1 row)
Fill in some tables:
INSERT INTO visualization VALUES (1, 'Normal'); INSERT INTO visualization VALUES (2, 'Rating'); INSERT INTO visualization VALUES (3, 'Byway Type'); INSERT INTO visualization VALUES (4, 'Slope'); INSERT INTO visualization VALUES (5, 'Bonus/Penalty'); INSERT INTO tag_preference_type VALUES (0, 'ignore'); INSERT INTO tag_preference_type VALUES (1, 'bonus'); INSERT INTO tag_preference_type VALUES (2, 'penalty'); INSERT INTO tag_preference_type VALUES (3, 'avoid');
Exit when finished:
\q
These instructions currently do not work:
Download some map data from MNDOT: http://www.dot.state.mn.us/maps/gisbase/html/datafiles.html (you want one or more of the 7 metro counties) Use the import scripts in scripts/ to put it in your database. Run scripts/nodes_build.py, or import_bmpolygon.py
In theory, you should now have a working Cyclopath database. Let us know how it goes in practice. :)
[edit] Testing & Running Cyclopath
To run, go to: http://localhost or http://yourmachinename
See Debugging tips below on getting error logs.
To start routed (route finder): "sudo -u www-data ./routedctl start" in the pyserver directory
Do not add watch regions unless you switch off all the email notifications; the server can hang trying to send email when email servers aren't configured. Same probably applies to entering a feedback of a revision, and the like.
[edit] Hack for Saving Byways Problem
Using the existing instructions above, I had a problem saving any byways (road segments). In the database, the necessary entry for 'byway_rating' could not be added because the default user in TABLE user_ did not exist. The following hack is for manually (forcing) the creation of this user, which is probably okay for open source early adopters, but probably NOT okay for CP team core developers.
When the proper instructions for creating this user, and requisite TABLES and conditions are properly documented above, please remove this section.
HACK INSTRUCTIONS--use at own risk. (note: don't enter the 'cycling=>' prompts)
psql -U cycling
cycling=> ALTER TABLE user_ DISABLE TRIGGER user_alias_i;
cycling=> INSERT INTO user_ (username, email, login_permitted, alias,route_visualization) VALUES ('_r_generic', 'whoever@email.com','t','johnny cyclopath', 1);
You should be able to re ENABLE the TRIGGER again to remove the hack.
[edit] Adding a login user
- Do an insert command like above, and replace '_r_generic' with a user name, and use another alias name.
- Run:
cycling-> select password_set('user','password');
Restart apache (not sure it's necessary), then hit log in button on the client. Do not try to create a new user from the client.
[edit] Development Tips
See debugging hints for tips on debugging Python and Flex, as well as diagnosing problems with Apache, Postgres, etc.
[edit] Online Resources
Adobe has exhaustive documents on developing Flash (a/k/a ActionScript, a/k/a Flex)... if you know where to look.
FIXME I'm not sure these match the versions Cyclopath uses. Also, I usually just Google "flex blah blah" when looking for help.
Curious how to format documents for the Cyclopath Wiki? Check out the official reference.
FIXME Need links to Python docs, PostGIS, MapServer, etc., etc.
