eclipse-sailing-analytics/sailing-analytics
1
0 Fork 0
mirror of https://github.com/eclipse-sailing-analytics/sailing-analytics.git synced 2026-10-06 12:23:55 +00:00
Code Activity
Files
26fc5aeebbce35fa366435476c00859b3585da1f
sailing-analytics/wiki/extreme-sailing-series.md
T

23 KiB
Raw Blame History

Specifica for the Extreme Sailing Series

[[TOC]]

The Extreme Sailing Series is a hospitality event that aims to make invitees participate as close as possible at a sailing race. To achieve this goal the races take place in front of a tribune placed directly at the water (called Stadium Racing). Invitees can be assigned to a boat crew and that way be part of a race like it is not possible in any other series.

Each race has a course that is set that way that spectators can see as much as possible regardless of the wind direction (upwind start not to be implied). The event is set to span a week, racing happens on 4 or 5 days. Every day up to 10 races take place depending on the wind conditions. Race course is defined such that a race lasts (under normal conditions) not longer than 20 minutes.

Usually not more than 8 competitors race against each other. One of the competitors is always an invitational team from the local spot that is allowed to race.

External Links

Equipment Needed

  • From WDF
    • 4x SIM Cards (2x WindBot, Spares)
    • 1x World Adapter
  • In Container
  • 2x Samsung Tablet
  • 2x WindBots
  • 1x Satellite Modem
  • 1x Plug Strip
  • Provided by OC
  • 1x Rack with servers and switches
  • 2x radio

Official information

The current notice of race and other information can be accessed on the following page: https://octpftp.egnyte.com/. The username is ess_main and the password is x402014ess.

Boats

To achieve the goal of providing invitees an exciting event a new type of boats have been designed. Capable of reaching speeds usually reserved to motorboats even in medium wind conditions, the Extreme 40 has been designed by Olympic champions Yves Loday and Mitch Booth, with the aim to provide the international sailing arena with a visually stunning and 100% performance-focused multihull.

Flying a hull in as little as 8 knots of breeze (15 kph), the 40-foot (12m) long carbon speed machine requires coordination, finesse but also sheer muscular power from the crews who battle it out. The generous sail area allows the Extreme 40s to sail faster than the wind, which might seem puzzling at first - in just 15 knots of wind, an Extreme 40 is capable of traveling at over 25 knots

Scoring

Each race is scored using a high point system where the winner gets 10 points, the second gets 9 and so on (going not further than 3 points). At the end of an event the last race points get doubled (20 for the winner). If there is a tie break between two competitors then the last race sets the winner (breaks tie break). Points from each race are accumulated and result in the overall score for an event.

In addition to the overall leaderboard specific to an event a global leaderboard is being maintained that denotes positions for all events during a year. This scoring scheme used for the global leaderboard has the same rules as an event specific leaderboard. The winner of an event gets 10 points and so on.

Scoring can be altered by the race committee using standard rules like DNS (DidNotStart), DNF (DidNotFinish), DNC (DidNotCount) or RDG (RedressGiven). These rules (except the last one) usually lead to a score of 0 for the race.

Group Racing

There is a special format where either a group of 6 or 8 boats race against each other. This format is applied whenever the conditions do not allow 10 or more boats to race on the selected race area.

  • Group of 6: 2 groups of 6 boats each will race agains each other. The winner will get 8 points, the second 2 points and so on.
  • Group of 8: Three groups A,B and C with 4 competitors each will race against each other. There will always be two groups racing together so that the fleet size is 8.

If group racing is announced then make sure to select the right scoring scheme that is called HighPoint, winner gets 10 points (or 8). In order to inform the Sailing Analytics about a race being a group race, there is an activator in the race committee app that allows the race committee to mark a race as a group race. In the background the server will add a new event to the RaceLog that is called AdditionalScoringInformation.

Event Setup

The setup for such an event usually consists of the following departments:

  • Race Committee
    • Boat on the water
    • On the beach (Shore Control)
  • Commentator
    • For online streaming and video (beach)
    • Live (beach)
  • Video and Television
    • Cameras (water)
    • Camera (beach)
    • Tech team (Controller, Cutter, Streamer)
  • Visualization
    • Provider for GPS fixes, course layout and competitor names (TracTrac)
    • 3D Visualization provider (BeTomorrow)
    • Wind information (SAP)
    • Live and official result provider (SAP Sailing Analytics)

Roles

  • OC Sport (Event Organizer)
    • Clementine d'Oiron (Senior SAP Account Manager)
    • Jonathan Meadowcroft (CTO, in charge of IT and network infrastructure)
    • Liam Lavers (IT Support)
  • GMR (SAP Lounge)
    • Mark Angell (Senior SAP Account Manager)
    • Nick Houchin (SAP Account Manager, Host Manager)
  • TracTrac
    • Jakob Oedum (Senior Event Manager)
    • Jorge Llodra (IT Specialist)
  • Sunset & Vine APP (TV)
    • Sarah Greene (Manager)

Technical Architecture

The technical infrastructure can be divided into two major parts.

ON-PREMISE

This part physically consists of a rack that holds all servers needed to run the event in terms of tracking and network setup. All data (gps, wind, ...) is gathered here and then replicated to the cloud.

  • The DNS-Server serves not only as a Gateway to the internet but also has a Nameserver installed that redirects some urls to the local setup. In addition to that a caching proxy makes sure that redundant bits of information are cached.
  • The Sailing Analytics server holds an Apache webserver that answers to *.sapsailing.com requests and of course the Java application running the analytics itself.
  • A TracTrac server that receives tracking data and provides the API to the analytics server to get GPS fixes in realtime.
  • Last but not least a Hot-Spare server that can be used in case of failure of another server.

In addition to the servers some switches make sure to have enough space for connecting additional networks and machines. To power the whole thing in the not so unlikely event of a power failure two mid-size UPS give enough power to at least shut down the servers cleanly.

CLOUD

The cloud consists of one or more Sailing Analytics servers that present the data to the general public not being on premise. The on-premise servers replicate their data constantly to the cloud.

Detailed Setup

The following image depicts the setup that is currently implemented. It features a local setup where the dependency on a reliable and fast internet connection is minimized as much as possible.

The core of this setup is a server that not only hosts a SAP Sailing Analytics but also the TracTrac server. This way the distribution of analytical information is not dependent on the speed and bandwidth of the local internet connection. By adding a DNS server in front of this analytics server local requests can be directed to the local server even when guests use a public internet address (e.g. www.sapsailing.com).

In case of a problem with the local server requests can be redirected to the external analytics server. This server is constantly fed with data by a replication channel that gets information bits from the local analytics server.

  • The SAP Sailing Analytics server that is authoritative for computing the results is no longer in the cloud but installed on premise. That way it is no longer dependent on a replicated TracTrac server but gets data directly from local TracTrac server.
  • TracTrac Server is integrated with SAP Sailing Analytics on two physical appliances. That eases maintenance and data exchange between SAP and TracTrac services.
  • Every leaderboard related information is gathered by accessing the local server. That way the dependency from the internet is drastically mitigated. Everyone on site always gets the right information without delay.
  • A routing server manages the DNS resolution and in case of a local failure is able to transparently redirect data to the cloud.
  • Score corrections also are not longer dependent on the internet connection but get fed directly into the local server.

To not be dependent on a shaky power source that can be restored by "wiggling pieces a little to fix the generator" it has been decided to introduce UPS that can fed important hardware with power up to half an hour. The following picture depicts a quick shot on how this looks:

DNS Server Setup

The DNS Server acts as a gateway/router, caching proxy and nameserver.

Gateway

All traffic to and from the internet is going through this server. To achieve that goal it routes all internal connections by default to an interface called em2. This interface is in most cases connected to an external router and must be configured according to the specifications sent over by the ISP. The routing is achieved by first having the right configuration for the interface (example from Cardiff ISP):

[root@SAPDNS ~]# more /etc/sysconfig/network-scripts/ifcfg-em2
DEVICE="em2"
BOOTPROTO="static"
HWADDR="D4:AE:52:C9:F4:78"
NM_CONTROLLED="no"
TYPE="Ethernet"
UUID="e72740f5-ce09-4d3f-becf-a58f605211db"
ONBOOT="yes"
IPADDR=188.164.228.178
NETMASK=255.255.255.240
GATEWAY=188.164.228.177
DNS1=192.168.1.202
NAMESERVER=194.150.200.22

Then the firewall needs to be configured to act as a router. There is a script at /root/start_proxy.sh that takes care of activating the right settings for the firewall but also for the proxy:

#!/bin/sh
# Squid server IP
SQUID_SERVER="192.168.1.202"
# Interface connected to Internet
INTERNET="em2"
NETWORK="rename4"
MOBILE="p1p2"
# Address connected to LAN
LOCAL="192.168.1.0/24"
LOCAL2="192.168.4.0/24"
# Squid port
SQUID_PORT="3128"
# Clean old firewall
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
# Enable Forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward
echo 1024 32768 > /proc/sys/net/ipv4/ip_local_port_range
echo 8192 > /proc/sys/net/ipv4/tcp_max_syn_backlog
ulimit -HSd unlimited
ulimit -HSn 16384
# Setting default filter policy
iptables -P INPUT DROP
iptables -P OUTPUT ACCEPT
# Unlimited access to loop back
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT
# Allow UDP, DNS and Passive FTP
iptables -A INPUT -i $INTERNET -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -i $NETWORK -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -i $MOBILE -m state --state ESTABLISHED,RELATED -j ACCEPT
# set this system as a router for Rest of LAN
iptables -t nat -A POSTROUTING -o $INTERNET -j MASQUERADE
iptables -t nat -A POSTROUTING -o $NETWORK -j MASQUERADE
iptables -t nat -A POSTROUTING -o $MOBILE -j MASQUERADE
iptables -A FORWARD -s $LOCAL -j ACCEPT
iptables -A FORWARD -s $LOCAL2 -j ACCEPT
# unlimited access to LAN
iptables -A INPUT -s $LOCAL -j ACCEPT
iptables -A OUTPUT -s $LOCAL -j ACCEPT
iptables -A INPUT -s $LOCAL2 -j ACCEPT
iptables -A OUTPUT -s $LOCAL2 -j ACCEPT
# DNAT port 80 request comming from LAN systems to squid 3128 ($SQUID_PORT) aka transparent proxy
iptables -t nat -A PREROUTING -s $LOCAL -p tcp --dport 80 -j DNAT --to $SQUID_SERVER:$SQUID_PORT
# if it is same system
iptables -t nat -A PREROUTING -i $INTERNET -p tcp --dport 80 -j REDIRECT --to-port $SQUID_PORT
#open everything
iptables -A INPUT -i $INTERNET -j ACCEPT
iptables -A OUTPUT -o $INTERNET  -j ACCEPT
iptables -A INPUT -i $NETWORK -j ACCEPT
iptables -A OUTPUT -o $NETWORK  -j ACCEPT
iptables -A INPUT -i $MOBILE -j ACCEPT
iptables -A OUTPUT -o $MOBILE  -j ACCEPT
# udp DNS
iptables -A OUTPUT -p udp --dport 53 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -p udp --sport 53 -m state --state NEW,ESTABLISHED -j ACCEPT
# tcp DNS
iptables -A OUTPUT -p tcp --dport 53 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -p tcp --sport 53 -m state --state NEW,ESTABLISHED -j ACCEPT
# DROP everything and Log it
iptables -A INPUT -j LOG
iptables -A INPUT -j DROP
echo "Started firewall"
rndc querylog
echo "Started query logging for named"

On interface p1p2 in most cases there is a 4G router connected that provides an internet connection in case the main internet on em2 is failing. In case of a failure on em2 it does make sense to only provide internet to selected clients because the bandwidth on the backup line is very limited. In order to achieve this there is a script at /root/activate_4gmodem.sh that can be used to reroute some clients to the 4g line. This script needs the iproute package installed.

[root@SAPDNS ~]# more configuration 
# Hosts that are routed via 4G modem
MODEM_4G_HOSTS="192.168.1.184 192.168.1.201"

[root@SAPDNS ~]# more activate_4gmodem.sh 
source /root/configuration
echo "Activating redirection for hosts $MODEM_4G_HOSTS to 4G modem (192.168.200.1)..."
ip route add default via 192.168.200.1 dev p1p2 table 4gmodem
for host in $MODEM_4G_HOSTS; do
    echo "  $host"
    ip rule add from $host lookup 4gmodem prio 1000
done
ip route flush table 4gmodem
ip route flush table main
sh /root/restart_firewall.sh

[root@SAPDNS ~]# more deactivate_4gmodem.sh 
source /root/configuration
echo "Removing all configured connections to 4G modem line..."
for host in $MODEM_4G_HOSTS; do
    echo "  $host"
    ip rule del from $host
done
ip route flush table 4gmodem
ip route flush table main
sh /root/restart_firewall.sh

Nameserver

The nameserver makes sure to redirect some urls to the local server. In the current configuration it will redirect all ess40-2014.sapsailing.com requests to local servers unless disabled. The basic configuration can be found in /etc/named.conf

options {
	listen-on port 53 { 192.168.1.202; 192.168.4.202; };
	listen-on-v6 port 53 { ::1; };
	directory 	"/var/named";
	dump-file 	"/var/named/data/cache_dump.db";
        statistics-file "/var/named/data/named_stats.txt";
        memstatistics-file "/var/named/data/named_mem_stats.txt";
	allow-query     { any; };
	recursion yes;

	dnssec-enable yes;
	dnssec-validation yes;
	dnssec-lookaside auto;

	/* Path to ISC DLV key */
	bindkeys-file "/etc/named.iscdlv.key";

	managed-keys-directory "/var/named/dynamic";
};
logging {
        channel default_debug {
                file "data/named.run";
                severity dynamic;
        };
};
zone "." IN {
	type hint;
	file "named.ca";
};
zone "0.0.127.in-addr.arpa" IN {
	type master;
	allow-query { any; };
	allow-update { none; };
	file "localhost.db";
};
zone "1.168.192.in-addr.arpa" IN {
	type master;
	allow-query { any; };
	allow-update { none; };
	file "1.168.192.in-addr.arpa.db";
};
zone "4.168.192.in-addr.arpa" IN {
	type master;
	allow-query { any; };
	allow-update { none; };
	file "4.168.192.in-addr.arpa.db";
};
zone "portal.extremesailingseries.com" IN {
	type master;
	allow-query { any; };
	allow-update { none; };
	file "portal.extremesailingseries.com.db";
};

zone "www.sapsailing.com" IN {
	type master;
	allow-query { any; };
	allow-update { none; };
	file "sapsailing.com.db";
};
zone "ess40-2014.sapsailing.com" IN {
	type master;
	allow-query { any; };
	allow-update { none; };
	file "sapsailing.com.db";
};

zone "live.ess40-2014.sapsailing.com" IN {
	type master;
	allow-query { any; };
	allow-update { none; };
	file "sapsailing.com.db";
};
zone "ess40-mobile.sapsailing.com" IN {
	type master;
	allow-query { any; };
	allow-update { none; };
	file "sapsailing.com.db";
};
zone "ess40.local.sapsailing.com" IN {
	type master;
	allow-query { any; };
	allow-update { none; };
	file "sapsailing.com.db";
};
zone "live1.local.sapsailing.com" IN {
	type master;
	allow-query { any; };
	allow-update { none; };
	file "sapsailing.com.db";
};
zone "live2.local.sapsailing.com" IN {
	type master;
	allow-query { any; };
	allow-update { none; };
	file "sapsailing.com.db";
};
zone "ocsportmobile" IN {
	type master;
	allow-query { any; };
	allow-update { none; };
	file "portal.extremesailingseries.com.db";
};
include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";

That basic configuration only specifies the urls the nameserver will answer requests. For each url you then need to configure the answer at /var/named/sapsailing.com.db. Make absolutely sure to end each url entry with a dot!

$TTL 86400
@ 	IN 	SOA ns.ocsportmobile. admin.ocsportmobile. (
			201304131
			1H
			7200
			1D
			1H)
		NS 	ns.ocsportmobile.
		TXT	"ESS DNS"

ess40-2014.sapsailing.com.	A	192.168.1.201
live.ess40-2014.sapsailing.com.	A	192.168.1.201
ess40-2014.ru.sapsailing.com.	A	192.168.1.201
live.ru.ess40-2014.sapsailing.com.	A	192.168.1.201
ess40-mobile.sapsailing.com.	A	192.168.1.201
ess40-local.sapsailing.com.	A	192.168.1.201
ess40.local.sapsailing.com.	A	192.168.1.201
live1.local.sapsailing.com.	A	192.168.1.201
live2.local.sapsailing.com.	A	192.168.1.201
www.sapsailing.com.	        A	192.168.1.201

The tool to show configuration and manage entries at runtime is called rndc. If you want to see stats then you have to dump them using rndc stats. They will then get dumped to /var/named/data/named_stats.txt.

Caching Proxy

The transparent caching proxy (Squid) intercepts all connections on port 80 (configured by the firewall script that forwards all connections on port 80 to port 3128 where the proxy runs) and caches some information bits like images or stylesheets. This helps reducing the bandwith. The configuration can be found at /etc/squid/squid.conf. The configuration is optimized for high traffic - make sure you understand what you're doing before changing anything.

http_port 3128 transparent
cache_mem 2048 MB
cache_store_log none
cache_swap_high 100%
cache_swap_low 80%
half_closed_clients off
maximum_object_size 512 KB
memory_pools on
memory_pools_limit 2048 MB
maximum_object_size_in_memory 512 KB
ipcache_size 16000
ipcache_low 90
ipcache_high 97
debug_options ALL,2
dns_nameservers 192.168.1.202
positive_dns_ttl 8 hours
negative_dns_ttl 120 seconds
fqdncache_size 10000
#hierarchy_stoplist cgi-bin ?
#acl QUERY urlpath_regex cgi-bin \?
#cache deny QUERY
acl ads dstdomain "/etc/squid/ads.txt"
http_access deny ads
acl apache rep_header Server ^Apache
access_log /var/log/squid/access.log squid
hosts_file /etc/hosts
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?.+) 0 0% 0
refresh_pattern .*?\?$ 14400 50% 18000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
refresh_pattern -i \.jpg$ 14400 50% 18000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
refresh_pattern -i \.gif$ 14400 50% 18000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
refresh_pattern -i \.png$ 14400 50% 18000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
refresh_pattern -i \.css$ 14400 50% 18000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
refresh_pattern -i \.js$ 14400 50% 18000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
refresh_pattern -i \.woff$ 14400 50% 18000 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private ignore-auth
refresh_pattern -i . 0 20% 4320
acl all src all
#0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8
acl lan src 192.168.1.0/24
acl lan src 192.168.4.0/24
acl doNotCacheDestinationIP dst 192.168.1.201 192.168.1.203
acl doNotCacheDomains dstdomain .sapsailing.com
acl SSL_ports port 443 563 # https, snews
acl SSL_ports port 873 # rsync
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 563 # https, snews
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 631 # cups
acl Safe_ports port 873 # rsync
acl Safe_ports port 901 # SWAT
acl purge method PURGE
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access allow localhost
http_access allow lan
http_access deny all
http_reply_access allow all
cache deny doNotCacheDestinationIP
cache deny doNotCacheDomains
icp_access allow all
visible_hostname ocsportmobileproxy
always_direct allow all
coredump_dir /var/spool/squid

There is a web interface that is configued in /etc/httpd/conf.d/squid.conf that enables you to access some stats from http://192.168.1.202/Squid/cgi-bin/cachemgr.cgi.

Additional information bits

Useful tools to get information about system

Competitor colors for 2014

#33CC33 Groupama
#FF0000 Alinghi
#2AFFFF GAC
#000010 ETNZ
#FFFFFF Gazprom
#999999 JP Morgan
#FFC61E SAP
#B07A00 Oman Air
#000099 Realteam
#990099 Red Bull
#16A6ED The Wave

Opening replication channel for Sailing Analytics server

The local server acts as a master and replicates data through a RabbitMQ running locally to an external server. The external server needs to connect to localhost on 9087.

autossh -M 20000 -R 5672:localhost:5672 -R 9087:localhost:8887 -L 2012:localhost:2012 sailing@54.194.10.242

Replicate Race Logs

#/bin/bash
EVENT_NAME="ESS 2014 Qingdao (Extreme40)"
REMOTE_USER=mongodb
REMOTE_SERVER=54.246.250.138
PARAM=$EVENT_NAME

echo "Exporting RACE_LOGS for $PARAM"
mongoexport --port 10201 -d winddb -c RACE_LOGS -q "{\"RACE_LOG_IDENTIFIER.a\" : \"$PARAM\"}" > race_logs.json
mongoexport --port 10201 -d winddb -c LEADERBOARDS -q "{\"REGATTA_NAME\" : \"$PARAM\"}" > leaderboards.json
ssh $REMOTE_USER@$REMOTE_SERVER 'rm /tmp/*.json'
scp race_logs.json $REMOTE_USER@$REMOTE_SERVER:/tmp 
scp leaderboards.json $REMOTE_USER@$REMOTE_SERVER:/tmp 
echo "Saved race logs and leaderboard"
ssh $REMOTE_USER@$REMOTE_SERVER '/opt/mongodb-linux-x86_64-1.8.1/bin/mongoimport --port 10201 --upsert -d winddb -c RACE_LOGS < /tmp/race_logs.json' 
ssh $REMOTE_USER@$REMOTE_SERVER '/opt/mongodb-linux-x86_64-1.8.1/bin/mongoimport --port 10201 --upsert -d winddb -c LEADERBOARDS < /tmp/leaderboards.json' 
echo "Finished"
View Git Blame Copy Permalink