Continuous Integration with phpUnderControl and Git

September 8, 2009 · in Development, How-Tos, Server

I was looking for a decent continuous integration solution for my PHP projects for some time now, but always had the problem that most of the described solutions used SVN instead of Git as VCS system. Yesterday I found an article which describes the setup exactly as I needed it: phpUnderControl with Git on a Debian/Ubuntu system. Using the article, I managed to set up a working system quickly, which basically works as expected: CruiseControl checks the repository for modifications and starts the build process if there are any new commits. The build process includes generating API documentation (phpdocumentor), running static code analysis (php-codesniffer) and executing unit tests (phpunit). If the build succeeds, the results are published and can be accessed through a nice webinterface powered by phpUnderControl (see screenshot above which I stole from the phpUnderControl site).

However, the described setup has a few issues which bugged me:

CruiseControl runs from the shellscript as root, posts all output to the console and is not automatically started at boot time.
CruiseControl runs on port 8080, but I wanted to manage access to the webinterface through the apache which is already running on the box
There’s no authentication – everybody can access my CI server, see the build details and start new builds through the webinterface.

I solved these issues with 2 steps.

Init script for CruiseControl

I wrote a simple init script which allows me to control CC. I’m not really into shellscripting and just hacked around on the apache init script until I got a working solution, so if you have any suggestions how to improve this script please let me know. The script implements the following functions: start, stop, restart, status. Save it to /etc/init.d/cruisecontrol and make it executable (chmod +x /etc/init.d/cruisecontrol).

#!/bin/sh
### BEGIN INIT INFO
# Provides:          cruisecontrol
# Required-Start:    $local_fs $remote_fs $network $syslog
# Required-Stop:     $local_fs $remote_fs $network $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start/stop cruisecontrol CI server
### END INIT INFO
#
# cruisecontrol         This init.d script is used to start cruisecontrol.
#                       It basically just calls cruisecontrol.sh.

# ENV="env -i LANG=C PATH=/usr/local/bin:/usr/bin:/bin"

CCON_PATH=/opt/cruisecontrol
CCON_USER=www-data
PIDFILE=$CCON_PATH/cc.pid
LOGFILE=/var/log/cruisecontrol.log

. /lib/lsb/init-functions

ccon_is_running() {
    if [ -f $PIDFILE ]; then
       PID=`cat $PIDFILE`
       PID_COUNT=`ps aux | grep -E $PID | grep -v grep | wc -l`
       if [ $PID_COUNT = 1 ]; then
         return 1
       fi
    fi

    return 0
}

ccon_start() {
        ccon_is_running
        rc=$?
        if [ $rc -eq 1 ]; then
                log_failure_msg "CruiseControl is already running"
        else
                log_daemon_msg "Starting CI server" "CruiseControl"
                cd $CCON_PATH
                sudo -E -H -u $CCON_USER ./cruisecontrol.sh >> $LOGFILE 2>&1
                log_end_msg $?
        fi
}

ccon_stop() {
        ccon_is_running
        rc=$?
        if [ $rc -eq 1 ]; then
                log_daemon_msg "Stopping CI server" "CruiseControl"
                PID=`cat $PIDFILE`

                retval=0
                i=0
                while $(kill "$PID" 2> /dev/null);  do
                        if [ $i = '60' ]; then
                                echo ""
                                log_failure_msg "CruiseControl is taking too long to shutdown"
                                retval=1
                                break
                        else
                                if [ $i = '0' ]; then
                                        echo -n " ... waiting "
                                else
                                        echo -n "."
                                fi
                                i=$(($i+1))
                                sleep 1
                      fi
                 done

                log_end_msg $retval
        else
                log_failure_msg "CruiseControl is not running"
        fi
}

ccon_status() {
        ccon_is_running
        rc=$?
        if [ $rc -eq 1 ]; then
                PID=`cat $PIDFILE`
                log_success_msg "CruiseControl is running (pid $PID)."
        else
                log_failure_msg "CruiseControl is not running."
        fi

}

case $1 in
        start)
                ccon_start
        ;;
        stop)
                ccon_stop
        ;;
        restart)
                ccon_stop
                ccon_start
        ;;
        status)
                ccon_status
        ;;
        *)
                log_success_msg "Usage: /etc/init.d/cruisecontrol {start|stop|restart|status}"
                exit 1
        ;;
esac

Then change the ownership of the cruisecontrol installation to the user you specified in the init script (in my case www-data):

$ chown -R www-data.www-data /opt/cruisecontrol-bin-2.8.2/

You can now control CC just by using the init script.

$ /etc/init.d/cruisecontrol status
 * CruiseControl is not running.
$ /etc/init.d/cruisecontrol start
 * Starting CI server CruiseControl                        [ OK ]
$ /etc/init.d/cruisecontrol status
 * CruiseControl is running (pid 13721).
$ /etc/init.d/cruisecontrol stop
 * Stopping CI server CruiseControl
 ... waiting ..                                            [ OK ]
$ /etc/init.d/cruisecontrol status
 * CruiseControl is not running.

Additionally you can add it to the default runlevels to start it automatically on system boot.

$ update-rc.d cruisecontrol defaults

Access the webinterface via Apache

I wanted to use my existing apache installation to serve the webinterface. Doing this you can make the webinterface accessible through port 80, control authentication through apache’s various auth modules and eventually even use SSL to encrypt the traffic. The first thing I did was to bind the CruiseControl’s webserver (Jetty) to the local IP. Open /opt/cruisecontrol/etc/jetty.xml and find the following line:

<Set name="host"><SystemProperty name="jetty.host" /></Set>

Add the default attribute to that line and afterwards restart CC.

<Set name="host"><SystemProperty name="jetty.host" default="127.0.0.1" /></Set>

Now try to access CruiseControl through the external IP, you should not get a connection. But if you try to open a connection from the same host, you should get a response:

$ curl http://127.0.0.1:8080/cruisecontrol/

To serve this instance via apache you need 3 apache modules (install them via apt if you don’t have them installed):

$ a2enmod proxy proxy_http ext_filter

Create a new virtual host config file in /etc/apache2/sites-available/. Modify ServerName and ExtFilterDefine to match your environment.

<VirtualHost *:80>
        ServerName cruisecontrol.example.org

        ProxyRequests Off

        <Proxy *>
                Order deny,allow
                Allow from all
        </Proxy>

        ProxyPass / http://127.0.0.1:8080/
        ProxyPassReverse / http://127.0.0.1:8080/

        ExtFilterDefine fixurls mode=output intype=text/html cmd="/bin/sed s%http://127.0.0.1:8080%http://cruisecontrol.example.org%g"
        SetOutputFilter fixurls
</VirtualHost>

As a last step, enable the virtual host and restart apache (I named the config file cruisecontrol):

$ a2ensite cruisecontrol
$ /etc/init.d/apache2 restart

Now you should have access to your phpUnderControl installation via http://cruisecontrol.example.org/cruisecontrol. As a last step, we add basic apache authentication (replace with your preferred apache auth method). Edit the vhost config file and modify as follows:

        <Proxy *>
                Order deny,allow
                Allow from all

                AuthName        "phpUnderControl"
                AuthType        Basic
                AuthUserFile    /etc/apache2/cruisecontrol.htpasswd
                require         valid-user
        </Proxy>

As a last step, you have to create the .htpasswd file, which contains the user map.

$ htpasswd -c /etc/apache2/cruisecontrol.htpasswd myuser

Add additional users with the following command (same command but without the -c switch):

$ htpasswd /etc/apache2/cruisecontrol.htpasswd otheruser

Finally, restart apache and you are done:

$ /etc/init.d/apache2 restart

Sources

Tags: agile, apache, continuous integration, cruisecontrol, git, php, phpundercontrol, unit testing

http://uxdriven.com Eric Clemmons

Just letting you know that this post was critical in getting my environment up & going.

Thanks so much for taking the time on this post!
- http://maff.ailoo.net/ Mathias
  
  Thanks for the feedback :) One thing I noticed some time after writing this post was that the JMX console which handles requests like invoking a new build manually and lets you do some other stuff is still not secured with this setup. The console is running on port 8000 by default and the above setup just handles proxying and securing of the frontend while the console is still insecure. I wanted to update the post but was busy and never found time to write that down (as usual…). So, I’ll try to explain it quickly in this comment.
  
  What I first did was restricting access to port 8000 just to the local IP via iptables. This should look somehow like this:
  
  iptables -A INPUT -s 127.0.0.1/32 -p tcp -m tcp –dport 8000 -j ACCEPT
  iptables -A INPUT -p tcp -m tcp –dport 8000 -j REJECT –reject-with icmp-port-unreachable
  
  Then, I created an extra DNS entry for a subdomain which handles the JMX console and modified my Apache config. I’m using GnuTLS for SSL-based virtual hosts here, but just letting it run without SSL should work fine too. With this config, the frontend is reachable via cruisecontrol.example.org and the console via jmx.cruisecontrol.example.org, both using the same authentication backend. The frontend vhost takes care of rewriting all requests to the JMX domain with some more output filter magic. Should more or less work copy&paste, just take care of the URLs in the output filters. My final config looks something like this:
  
  <VirtualHost *:80>
  ServerName cruisecontrol.example.org
  RewriteEngine on
  RewriteRule ^(.*)$ https://cruisecontrol.example.org$1 [R=301,L]
  </VirtualHost>
  <VirtualHost *:443>
  ServerName cruisecontrol.example.org
  
  [...SSL Stuff...]
  
  ProxyRequests Off
  
  <Proxy *>
  Order deny,allow
  Allow from all
  
  AuthName “CruiseControl”
  AuthType Basic
  AuthUserFile /etc/apache2/cruisecontrol.htpasswd
  require valid-user
  </Proxy>
  
  ProxyPass / http://127.0.0.1:8080/cruisecontrol/
  ProxyPassReverse / http://127.0.0.1:8080/cruisecontrol/
  
  ExtFilterDefine proxyurl mode=output intype=text/html cmd=”/bin/sed s%http://127.0.0.1:8080%https://cruisecontrol.example.org%g”
  ExtFilterDefine proxypathfix mode=output intype=text/html cmd=”/bin/sed s%/cruisecontrol/%/%g”
  ExtFilterDefine jmxurl mode=output intype=text/html cmd=”/bin/sed s%http://cruisecontrol.example.org:8000%https://jmx.cruisecontrol.example.org%g”
  ExtFilterDefine ipfix mode=output intype=text/html cmd=”/bin/sed s%127.0.0.1%cruisecontrol.example.org%g”
  
  SetOutputFilter proxyurl;proxypathfix;jmxurl;ipfix
  </VirtualHost>
  
  <VirtualHost *:80>
  ServerName jmx.cruisecontrol.example.org
  RewriteEngine on
  RewriteRule ^(.*)$ https://jmx.cruisecontrol.example.org$1 [R=301,L]
  </VirtualHost>
  <VirtualHost *:443>
  ServerName jmx.cruisecontrol.example.org
  
  [...SSL Stuff...]
  
  ProxyRequests Off
  
  <Proxy *>
  Order deny,allow
  Allow from all
  
  AuthName “CruiseControl”
  AuthType Basic
  AuthUserFile /etc/apache2/cruisecontrol.htpasswd
  require valid-user
  </Proxy>
  
  ProxyPass / http://127.0.0.1:8000/
  ProxyPassReverse / http://127.0.0.1:8000/
  </VirtualHost>
  
  Regards,
  Mathias
- antimbe
  
  Hello,
  
  very hapy to see that a big part of my choices of continuous integration server around php was been solved by this blog (Ubuntu, git, phpundecontrol/cruisecontrol).
  
  Very Good Job.
  
  Thanks a lot.

http://uxdriven.com Eric Clemmons

Just letting you know that this post was critical in getting my environment up & going.

Thanks so much for taking the time on this post!

http://maff.ailoo.net/ Mathias

Thanks for the feedback :) One thing I noticed some time after writing this post was that the JMX console which handles requests like invoking a new build manually and lets you do some other stuff is still not secured with this setup. The console is running on port 8000 by default and the above setup just handles proxying and securing of the frontend while the console is still insecure. I wanted to update the post but was busy and never found time to write that down (as usual…). So, I’ll try to explain it quickly in this comment.

What I first did was restricting access to port 8000 just to the local IP via iptables. This should look somehow like this:

iptables -A INPUT -s 127.0.0.1/32 -p tcp -m tcp --dport 8000 -j ACCEPT
iptables -A INPUT -p tcp -m tcp --dport 8000 -j REJECT --reject-with icmp-port-unreachable

Then, I created an extra DNS entry for a subdomain which handles the JMX console and modified my Apache config. I’m using GnuTLS for SSL-based virtual hosts here, but just letting it run without SSL should work fine too. With this config, the frontend is reachable via cruisecontrol.example.org and the console via jmx.cruisecontrol.example.org, both using the same authentication backend. The frontend vhost takes care of rewriting all requests to the JMX domain with some more output filter magic. Should more or less work copy&paste, just take care of the URLs in the output filters. My final config looks something like this:

<VirtualHost *:80>
        ServerName cruisecontrol.example.org
        RewriteEngine on
        RewriteRule ^(.*)$ https://cruisecontrol.example.org$1 [R=301,L]
</VirtualHost>
<VirtualHost *:443>
    ServerName cruisecontrol.example.org

    [...SSL Stuff...]

    ProxyRequests Off  
   
    <Proxy *>  
        Order deny,allow  
        Allow from all

        AuthName        "CruiseControl"
        AuthType        Basic
        AuthUserFile    /etc/apache2/cruisecontrol.htpasswd
        require         valid-user
    </Proxy>  
   
    ProxyPass / http://127.0.0.1:8080/cruisecontrol/  
    ProxyPassReverse / http://127.0.0.1:8080/cruisecontrol/

    ExtFilterDefine proxyurl mode=output intype=text/html cmd="/bin/sed s%http://127.0.0.1:8080%https://cruisecontrol.example.org%g"
    ExtFilterDefine proxypathfix mode=output intype=text/html cmd="/bin/sed s%/cruisecontrol/%/%g"
    ExtFilterDefine jmxurl mode=output intype=text/html cmd="/bin/sed s%http://cruisecontrol.example.org:8000%https://jmx.cruisecontrol.example.org%g"
    ExtFilterDefine ipfix mode=output intype=text/html cmd="/bin/sed s%127.0.0.1%cruisecontrol.example.org%g"

    SetOutputFilter proxyurl;proxypathfix;jmxurl;ipfix
</VirtualHost>

<VirtualHost *:80>
        ServerName jmx.cruisecontrol.example.org
        RewriteEngine on
        RewriteRule ^(.*)$ https://jmx.cruisecontrol.example.org$1 [R=301,L]
</VirtualHost>
<VirtualHost *:443>
    ServerName jmx.cruisecontrol.example.org

    [...SSL Stuff...]

    ProxyRequests Off  
   
    <Proxy *>  
        Order deny,allow  
        Allow from all

        AuthName        "CruiseControl"
        AuthType        Basic
        AuthUserFile    /etc/apache2/cruisecontrol.htpasswd
        require         valid-user
    </Proxy>  
   
    ProxyPass / http://127.0.0.1:8000/  
    ProxyPassReverse / http://127.0.0.1:8000/
</VirtualHost>

Regards,
Mathias

antimbe

Hello,

very hapy to see that a big part of my choices of continuous integration server around php was been solved by this blog (Ubuntu, git, phpundecontrol/cruisecontrol).

Very Good Job.

Thanks a lot.

Esky

Thanks for this great guide it helped me install it on FreeBSD :)
Esky

Thanks for this great guide it helped me install it on FreeBSD :)
Pingback: Analysing Ruby on Rails? Is it too late? « GC | Seeing how long he can blog for without getting bored…
http://rtconcept.de/ Nightfly

Extremely useful to get this running on Gentoo. Thanks!
Petro Slyvko

Due to lack of the authentication in phpUnderControl & CC, you article is just what I needed! Thanks for your effort. Got it working on CentOS 5 with some modifications!

← Windows 7 RC auf älterem PC installieren: Code Error 5

Windows 2000 48-Bit-LBA →

Continuous Integration with phpUnderControl and Git

Init script for CruiseControl

Access the webinterface via Apache

Sources

Similar Posts

Search

Categories

Subscribe via E-Mail