Files
sailing-analytics/doc/ADMIN
T
Armin Zamani b501d1ca6f Merge branch 'racecommittee_integrated' of
ssh://armin@sapsailing.com/home/trac/git into racecommittee_integrated

Conflicts:
	java/com.sap.sailing.domain/src/com/sap/sailing/domain/leaderboard/impl/AbstractSimpleLeaderboardImpl.java
2013-04-11 00:28:12 +02:00

220 lines
7.7 KiB
Plaintext
Executable File

This file explains the typical administrative tasks to be performed on
the SAP Sailing server. For an explanation of what this server does,
please refer to the README file stored next to this ADMIN file.
Emergency Startup After Reboot
------------------------------
When for some reason the server re-booted, a number of manual steps
need to be performed to get back up and running. Proceed as follows:
- "ssh -A trac@sapsailing.com"
- "screen"
Open 10 screen sessions, using the "Ctrl-a c" keyboard shortcut in the screen session.
Name them, using "Ctrl-a A", according to the following scheme:
0$ dev 1$ test 2$ prod1 3$ prod2 4$ wind 5$ mongo 6$ ssh tunnel 7-$ build 8$ logs 9*$ storeandforward
Additional sessions may be used for the race committee app server.
The first four sessions are used to run the four server instances. Session 4
is used to run the udpmirror script that receives wind data through UDP and
forwards to the four server instances. Session 5 hosts a test MongoDB instance.
Session 6 hosts an ssh tunnel making sure port 12347 forwards to local port 22 to
create homogeneous SSH access to the machine for build and deploy scripts.
Session 7 is used for build processes. Session 8 can be used to monitor logs
and server load. Session 9 is used to record and forward inbound SwissTiming data.
In each screen session it is necessary to run the /home/trac/.bash_profile script by
entering
". .bash_profile"
This, in particular, adjusts the JAVA_HOME variable as needed.
Perform the steps "Launch MongoDB" and "Launch Python Server"
described below. Launch the four server instances by cd'ing to
/home/trac/servers/<instance> and invoke "./start" there. If you
additionally want to upgrade from a fresh build, call "./install"
before calling "./start".
Follow the step "Launch UDP Mirror" described below. Follow the step
"Launch SwissTiming forwarder" described below.
Launching Readily-built Java Server Instances
---------------------------------------------
We usually launch, manage and observe the Java server instances using
a screen session. See below for how to update and
restart an already running instance.
- ssh -A trac@sapsailing.com
- screen -rd
- Connect to the screen session for your server, e.g., session 0 for dev:
Ctrl-a 0
- "cd servers/<instance>" where <instance> is one of
dev/test/prod1/prod2
Then launch the server:
- ./start
This should start an OSGi console with the latest version of the
bundles deployed to ~trac/servers/<instance>/plugins.
Launch MongoDB
--------------
The MongoDB is used to store wind information and general
leaderboard-related information required by the Python web
application.
- log in as user "root"
- "cd /opt/mongodb/bin"
- "./supervisord"
This will launch the MongoDB servers. If not, check for old mongod.lock
files in the server directories and remove them, then try again.
Launch UDP Mirror
-----------------
The two production instances shall share the same wind
information. Therefore, it is necessary to launch a "UDP mirror" that
distributes UDP packets received on the single inbound UDP port for
the production instances to the two Java server instances.
In screen session 4 (Ctrl-a 4), do the following:
- "cd ~/servers/prod1"
- "./udpmirror -v 2012 localhost 2013 localhost 2014 localhost 2010 localhost 2011"
The "-v" argument ensures that UDP packets received will be dumped to
the console. The process will listen for inbound UDP packets on port
2012 and forward anything received on that port to localhost:2013 as
well as localhost:2014. See also the list of ports as documented in
the README file.
Launch SwissTiming forwarder
----------------------------
Go to screen session 9 (Ctrl-a 9), then do the following:
- "cd ~/servers/prod1"
- "./swisstiminglistener 3500 3501"
Apache HTTP Server configuration
--------------------------------
To define the different URL's to access the java servers you need to configure
the corresponding virtual hosts of the Apache Http server. The configuration file
can be found at /etc/httpd/conf.d/kiel.conf
Re-Building the Java Server
---------------------------
If, e.g., after a bug fix, it is necessary to re-build and re-deploy
the Java server, you should consider doing this first on the "dev"
branch of the git versioning system. The master branch contains the
sources for the "dev" server instance. The git branch to server
instance mappings are as follows:
git branch server instance
---------- ---------------
master <none> (internal development inside SAP network)
dev dev
test test
prod1 prod1
prod2 prod2
First, check out the branch you want to re-build:
- log in as user "trac"
- ensure that your SSH key is loaded / unlocked, either by logging in
using SSH with the -A argument, or by invoking the alias "sa" and
entering the password for the trac user's SSH key
- "cd git/java"
- "git checkout -f <branchname>" where <branchname> is either one of
dev, test, prod1 or prod2
- ensure the "ulimit -n" output is at least 4096 to enable the GWT compiler to assemble
the resource sets which consist of many files that all need to be opened concurrently.
Currently, the maximum value for "ulimit -n" is configured in /etc/security/limits.conf
and is set to 16384. This specified the maximum amount to which a user's shell can set
this value. The ~trac/.bash_profile contains a "ulimit -n 4096" command, but when
running "screen" the shells usually are no login shells. You need to make sure you
run ~trac/.bash_profile in the build shell to set the limit of open files to at least
4096.
- rm -rf com.sap.sailing.gwt.ui/com.sap.sailing.gwt.ui.*; mvn -fae -s /home/trac/git/settings.xml -P debug.without-proxy clean install 2>&1 | tee log
This will compile the sources, bundle them up in to OSGi bundle JAR
files, run all tests which required online access to the TracTrac
servers, and, if successful, creates an eclipse product in the directory
/home/trac/git/java/com.sap.sailing.feature.p2build/bin/products/raceanalysis.product.id/linux/gtk/x86_64
which contains all files required to install and run the build result.
Note that the running server(s) are not updated automatically. See next topic.
OSGi Server Instances
---------------------
We usually have four OSGi consoles running with a terminal window
each, visible at the VNC server running on por 5902. The title bar
shows the current working directory which should be something like
"~/servers/<instancename>" where <instancename> is one of dev, test,
prod1 and prod2.
At the "osgi>" prompt of the OSGi console you can type
- "exit"
to terminate the particular Java server instance.
You can either install a complete new server instance or just update an existing one.
Install a new OSGi Server Instance
----------------------------------
To install a complete new server instance you can issue
- "./install"
This command will delete the old installation and copy all required files
from the build result to the right directories of the server instance.
To start the server instance again, type
- "./start"
and the server should welcome you again with the "osgi>" prompt after
a successful start.
Update an existing OSGi Server Instance
-----------------------------------------
To update the server you can issue
- "./update"
This command will NOT delete the old installation but copy the updated plugins
from the build result to the right directories of the server instance.
To start the server instance again, type
- "./start"
and the server should welcome you again with the "osgi>" prompt after
a successful start.