In this article I will guide you through all the necessary steps to implement such sytem.

Introduction

I’ve already mentioned that the two important pieces of software for this project are DRBD and heartbeat. DRBD (Distributed Replicated Block Device) is comparable to RAID 1 in that it replicates write operations on a block device to another block device, thereby introducing redundancy. In contrast to a RAID array however, DRBD works over a TCP/IP network so your drives don’t have to be attached to the same machine.
But that’s only for the storage part. There’s not much use in having a highly available storage device when all running services are down in case of a failure. This is where heartbeat comes into play.
Heartbeat is a high-availability solution that detects server outages and can automatically restart the affected services on another node of the cluster. In fact heartbeat has out of the box support for DRBD devices, so using the two together is a popular choice. Heartbeat uses a so called heartbeat-network to send small special messages (“hearbeats”) to the other cluster nodes which basically is interpreted as “I’m still up and running, so there’s no need for you to take over any of my services”.

For this project we need two physical seperate computers with two NICs each and one free partition that should be about the same size on each of the nodes. Throughout the article I will use the following conventions:

• master-server – (host)name of the master node, that is the server that should normally run our samba and DRBD services
• slave-server – (host)name of the slave node that takes over if the other one goes down
• eth0 – NIC connected to the normal intranet
• eth1 – connection to the heartbeat network
• 10.0.0.1 – fixed IP of the master-node in the company intranet (eth0)
• 10.0.0.2 – same for the slave-node
• 10.0.0.5 – this is a highly available IP that can switch between the nodes; usual clients (such as SMB clients) should only use this IP to contact the services
• 10.0.1.1 – master-servers IP in the heartbeat network
• 10.0.1.1 – same for slave-server

Of course you’ll have to adapt these values in your specific environment.

Basic setup

The following steps are nearly identical for both nodes; there’s only one tiny difference in the network interface configuration.
First of all let’s install the two packages we use as a base – DRBD and Heartbeat.

$ apt-get install heartbeat drbd8-utils

We don’t have to install the DRBD driver as it is already part of the standard Ubuntu 8.04 Server installation. Now we have to check that both nodes have their heartbeat network interfaces configured correctly. As stated earlier, on my setup I use eth1 for the heartbeat interfaces.

As we want a static ip address automatically assigned to the interface on each startup, we have to add the following lines to /etc/network/interfaces

auto eth1
iface eth1 inet static
  address 10.1.1.1
  netmask 255.255.255.0

If there already was a section for eth1, you’ll have to replace that. Note that 10.1.1.1 is the ip address for the master node. Change that value to 10.1.1.2 on the slave. The next step is to add the nodes in /etc/hosts so they can be properly resolved.

10.1.1.1    master-server.domain.tld  master-server
10.1.1.2    slave-server.domain.tld  slave-server

DRBD

Both nodes

/etc/drbd.conf is where all DRBD configuration is done. It’s important to keep this file identical on both nodes, in our example we will use the following content:

resource r0 {
  # protocol to use; C is the the safest variant
  protocol C;
 
  startup {
    # timeout (in seconds) for the connection on startup
    wfc-timeout       120;
    # timeout (in seconds) for the connection on startup
    # after detection of data inconsistencies ("degraded mode")
    degr-wfc-timeout  120;
  }
 
  syncer {
    # maximum bandwidth to use for this resource
    rate 100M;
  }
 
  on master-server {
    ### options for master-server ###
    # name of the allocated blockdevice
    device     /dev/drbd0;
    # underlying blockdevice
    disk       /dev/sdc1;
    # address and port to use for the synchronisation
    # here we use the heartbeat network
    address    10.1.1.1:7788;
    # where to store DRBD metadata; here it's on the underlying device itself
    meta-disk  internal;                 
  }
 
  on slave-server {
    ### options for slave-server ###
    device     /dev/drbd0;
    disk       /dev/sdc1;
    address    10.1.1.2:7788;
    meta-disk  internal;
  }
}

Replace master-server and slave-server with the appropriate hostnames of your setup.
Now we’ll load the kernel module and add DRBD to the default runlevels:

modprobe drbd
echo 'drbd' >> /etc/modules
update-rc.d drbd defaults

Setting up a new DRBD resource requires explicit ceation of it, so let’s do that now.

/etc/init.d/drbd restart
drbdadm create-md r0
drbdadm up r0

Ignore errors and warnigs, as we will check the correct operation with

cat /proc/drbd

If everything is up and running, that command should print out something like this:

version: 8.0.11 (api:86/proto:86)
GIT-hash: b3fe2bdfd3b9f7c2f923186883eb9e2a0d3a5b1b build by phil@mescal, 2008-02-12 11:56:43
 0: cs:Connected st:Secondary/Secondary ds:Inconsistent/Inconsistent C r---
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0
        resync: used:0/31 hits:0 misses:0 starving:0 dirty:0 changed:0
        act_log: used:0/127 hits:0 misses:0 starving:0 dirty:0 changed:0

Master node

Initially DRBD doesn’t know which of the nodes should be primary, that’s why they are marked as “Inconsistent”. We have to force one of the nodes to be primary – in our case that will be master-server.

drbdadm -- -o primary r0

The synchronization should then start immediately, you can check its progress with cat /proc/drbd. After that we are now ready to format the new DRBD block device:

mkfs.ext3 /dev/drbd0

Mounting functions ust like with any other device, do a mount /dev/drbd0 /data if you want to test the setup, but do not add an entry into /etc/fstab, because heartbeat will mount the device later on.

Heartbeat

Both nodes

First of all put the following in /etc/ha.d/authkeys (this file must be chmod 600):

auth 1
1 crc

In /etc/ha.d/haresources we will list all the resources that we want to be highly available (amongst others those are the DRBD device and the Samba service).

master-server 10.0.0.10/24/eth0 drbddisk::r0 Filesystem::/dev/drbd0::/data::ext3 samba

This line says, that master-server is the preferred node for all of the listed resources

• the IP address 10.0.0.10 with netask /24 on interface eth0 (that interface has to be brought up by other means, typically by listing it in /etc/network/interfaces)
• the DRBD device r0
• a filesystem-mount; namely /dev/dbd0 will be mounted to /data
• the Samba service

All ressources on one line will be started one after another as it’s assumed that each one depends on the ressources listed before it. When the master node fails, the slave will take over all of those ressources and if we choose to do an automatic failback, master-server will then again take over when it’s recovered later on.

Master node

Into /etc/ha.d/ha.cf put the following content:

auto_failback   on
ucast           eth1 10.1.1.2
node            master-server slave-server
keepalive       2
deadtime        8

• auto_failback – whether the master node will become master after it becomes available again
• ucast – on which interface and ip to reach the partner node
• node – simply lists all nodes of the cluster
• keepalive – how often to send a heartbeat signal (every 2 seconds in this case)
• deadtime – time after which the other node will be assumed to be dead

Slave node

/etc/ha.d/ha.cf is quite similar we only have to change the ucast address.

auto_failback   on
ucast           eth1 10.1.1.1
node            master-server slave-server
keepalive       2
deadtime        8

Samba

The following steps apply to both nodes.
To make Samba highly available, we have to recall, that heartbeat should be in charge of starting / stopping the service. Therefor we have to remove Samba from the normal, runlevel-based, state management. Let’s do that first:

$ update-rc.d -f mysql remove

Now we have to put heartbeat in charge of controlling Samba. Open up /etc/ha.d/haresources again and replace the line we put there in the DRBD part with the following one:

master-server 10.0.0.10/24/eth0 drbddisk::r0 Filesystem::/dev/drbd0::/data::ext3 samba

See that little “samba” at the end? That tells heartbeat to take care of Samba in case of a resource takeover. You could actually add other things like mysql there, too. Heartbeat just takes any command and invokes /etc/init.d/ANY_COMMAND start|stop.
Now we have to make sure that both nodes use the same netbios name so that shares stay available after a resource takeover. As a bonus I like to restrict Sambda to only listen on the highly available ip address. Here is what we have to add to /etc/samba/smb.conf in the [global] section:

[global]
# ...
  netbios name = hv-samba
  interfaces = 10.0.0.10/24
  bind interfaces only = true
# ...

The final step is just an easy example for a Samba share that uses the DRBD device, which is mounted at /data. Add the following lines to /etc/samba/smb.conf:

[myshare]
  path = /data
  guest ok = yes
  read only = no
  browsable = yes

A few notes

It’s certainly useful to be notified somehow if a resource takeover takes place. I will demonstate here how that can be achieved via email.

Email notification

To get an email notification if master-server fails (and slave-server takes over the resources), we will use the simple smtp client “msmtp” which provides a sendmail interface. The version provided in the standard hardy repositories has a bug when used with TLS certificates, thus I installed it from the backport repository.
We’ll also need mailx, a non-interactive command line mailclient.

$ wget http://de.archive.ubuntu.com/ubuntu/pool/universe/m/msmtp/msmtp_1.4.16-1~hardy2_i386.deb
$ wget http://de.archive.ubuntu.com/ubuntu/pool/universe/m/msmtp/msmtp-mta_1.4.16-1~hardy2_all.deb
$ dpkg -i msmtp_1.4.16-1~hardy2_i386.deb msmtp-mta_1.4.16-1~hardy2_all.deb
$ apt-get -f update
$ apt-get install mailx

Now we need to edit /etc/msmtprc. You’ll have to replace all example values with some real smtp account credentials.

defaults
tls on
tls_certcheck off
tls_min_dh_prime_bits 512
 
account myaccount
host smtp.myprovider.tld
from myaddress@myprovider.tld
auth on
user SMTP_USER
password SMTP_PASSWORD
 
account default : myaccount

See the msmtp manpage for more details.

At last, append the following line to /etc/ha.d/haresources. If you don’t want to be emailed on a takeover by master-server, do this step only on slave-server.

master-server MailTo::someadmin@domain.tld::Subject

1. Download the LiveCD
2. Mount the CD Image and copy its contents to an unused partition (this *should* work with a in-use partition actually… but don’t blame me for lost data) like so:

   $ mkdir /tmp/fedora-cd
   $ mount -o loop F11-Alpha-i686-Live.iso /tmp/fedora-cd  # as root
   $ mkdir /tmp/installer
   $ mount /dev/sda4 /tmp/installer  # as root
   $ cp -R /tmp/fedora-cd/* /tmp/installer

3. Now you have to edit your boot loader configuration. For grub this will typically be done in a file called /boot/grub/menu.lst or something like that. Add the following lines:

   # adjust these settings for the device you're actually using
   title Fedora 11 Alpha LiveCD
   root (hd0,3)
   kernel /isolinux/vmlinuz0 root=/dev/sda4 rootfstype=auto ro liveimg quiet rhgb
   initrd /isolinux/initrd0.img

Now you can boot into the LiveCD from the according Grub menu entry.

Just wondering. Is this project still going on?
Hospital could do with an open source (enhanced) version.

I know some people are busy on it at http://www.rounddonut.co.uk/ but this project is going very very slow.

F

Now here is my answer:

Unfortunaltely I haven’t had much time for this lately. But indeed, there was some progress just a week ago – I made a few additions and cleanup-work in the wiki and also worked a bit on the viewer-program.
When there was some good progress about a year ago, I initially expected other interested people would build upon my findigs. This has not been the case yet.
Well…. I think there will be more stuff going on, but it may take some time – at least for the reverse engineering, as I kinda ditched the open-source-clone thing for now.

The latest THViewer brings improvements in file-handling and a completely Java based RNC decompressor.
If you’re interested in the sourcecode, you can use bazaar to branch/pull from my published branch:
bzr branch http://connection-endpoint.de/bazaar/THViewer

If you have already seen the video on content aware image resizing at seamcarving.com then you probably know what seam carving is. If not, I recommend you to do so.

Seam carving is a method to resize an image that takes the images content into account – and can therefore produce really great results. This is my attempt to provide the reader with an easy to understand explanation of how seam carving works.

A few notes before we start. The images I used for this were released under the terms of CC-BY. They are:

• Skyline Boston by ReneS
• BOLOGNA GRAFFITI by il CAMA
• P7122395 by :::Rui Ornelas:::

The Sourcecode that I wrote is available for download. You may use it for anything you like, but be warned that this is highly unoptimized and quickly thrown together code. To compile and run you will also need the DevIL library.
Also note that I will mostly write about horizontal resizing and that this is also the only thing my code can do. However there is no real difference between horizontal and vertical resizing, besides some variable-swapping here and there.

Basic algorithm

The algorithm itself is actually really easy to understand. The paper is easy to read and there is only one or two things I had to figure out by myself.
Here is an outline of the seam carving algorithm and its idea:

Step 1: calculate the energy for each pixel of the image
Step 2: build a map of the minimum cumulative energies
Step 3: do a backtrace on this data to get the path (“seam”) with the lowest energy
Step 4: delete the pixels which belong to the seam

A seam is an 8-connected path of pixels, where a vertical seam runs from top to bottom, containing exactly one pixel in each row. 8-connected means that for each pixel we only consider its 8 adjacent neighbors.

Calculating the energy

When we want to remove a seam with the lowest energy we need to decide how we calculate the energy of a single pixel. There are many ways to do that – the original paper lists a few. I chose to implement a gradient measure by using the so called Sobel operator.
Have a look at the following images to see what results the sobel operator yields:

Original	Sobel Operator

This is also commonly referred to as “edge detection”. Here is an image that illustrates the gradient calculation:

To calculate the y-gradient of a pixel, you would add up the values of its neighbors, scaled following the sobel matrix. Lets call the result Gy.
Now we usually want to be able to work with RGB images. To do this, I just evaluated the gradient for each color-channel (R, G, B) independently – say GyR, GyG and GyB – and took the average for Gy. Thus Gy = (GyR + GyG + GyB) / 3.
Then I did the analogous part for the horizontal gradient Gx and finally stored the (x, y)-gradient magnitude at this pixels position in the energy map. The images you see above are directly rendered from that energy map.
Note that my code sets the energy for border pixels (that is pixels which are at the very top / bottom / left / right) always to maximum energy.

Cumulative Energy Map

The next step with our seam carving is to prepare a map of the cumulative energies on that we can do the backtracing later on. This will be the dynamic programming part and it can be easily summarized by the following formula:
M(x, y) = energy(x, y) + min( M(x-1, y-1), M(x, y-1), M(x+1, y-1) )
To compute the cumulative minimum energy M(x, y) for a pixel (x, y), add its energy to the minimum cumulative energy of one of its predecessors (remember the “8-connected” thing?).
I again rendered the cumulative energy map:

Original	Energy	Min. Cumul. Energy

Backtracing

This is the part where we need to use the cumulative enrgy map (CE/M) to find the vertical seam with the lowest energy. Backtracing is a common method for doing such kind of things, so let me give you a brief explanation of how it works.
We start by searching the pixel with the lowest CE in the very last row. After adding that pixel to the seam, we look at its 3 neighbors to the top and agian choose the one with the lowest CE. Have a look at the following image:

The algorithm starts at the bottom row and finds that 11 is the minimal CE. We add that pixel to our seam and look at the two adjacent pixels above. 8 is the minimal CE, so this is the next pixel to be added to the seam. Again in the next step we consider the thre pixels with CE values 7, 2 and 6 respectively. Can you tell how this goes on?
Now let’s think about this for a few seconds. In each step we choose the pixel with the minimal CE, therefore when the algorithm arrives at the top of the image, we know the seam with the overall minimal CE. Remove all these pixels and you’ll have seem carving!
Here is a real example with the seam rendered as a red line and a zoomed version:

Results

Seam carving results depend almost entirely on the chosen energy function. I used a gradient based method because I thought that it was easy to implement and produces quite good results. Of course this will not work as well on some images as it does on others. Quality of the outcome can also be improved by providing the user with tools to manually add energy to parts of the image (therefore “protecting” those parts) or remove energy (force the removal of parts).
Now for the results. I have used my program to resize three different images and these are the results.

Original
Energy
CEM
First Seam
Result

Kleiner Screenshot:

Sehr einfaches Script, vor allem dank Glade und Hpricot.

The few things that were done since the last post:

• figured out how shadows and litter is represented in a savegame
• I worked out how the animation layers are stored; this is why the people have heads and different clothes on the new screenshot
• soundfiles; yeah right! I messed with the sound files format and I now know how this works… well at least to a large extent

And as always I also updated the wiki and implemented the new stuff in my TH Viewer. Get the latest version here.

Outlook

With the last bits waiting to be uncovered comes the time to think about the next steps.

Finish the savegame format.
The next coupe of days/weeks are still mainly for finishing up the reverse engineering and getting the wiki in the right shape. Savegames are crucial in getting a feeling for “how Theme Hospital does stuff” without having to go through many painful debugging sessions. So that is something I’d like to get done.

Finish other reverse engineering
Not too much left and not really that importent – unless of course we want a clone that is fully compatible with the original game.

Toolchain evaluation
I will probably stick with java for a programming language, however I’m still not quite sure what graphics/media libraries to use. SDL may be a good choice but I also consider choosing a LWJGL based engine.

Spread the word
I already received an email from Ted Tycoon recently, who expressed his interest in the project. I would also like to see what other people have to say and if there are more players and developers interested.

Still, there are some unknown pieces to the whole specification. But my guess would be that we are now at about 75% done and nearly all files classified by their format. By now it should be already possible to begin a theme hospital clone – if not even complete it.

So without further ado, I present to you:

- the format specification
- THViewer (132 kB, 7zip)

The Sourcecode of THViewer is included in the archive. I’ve also included a Win32 Binary and the sourcecode of dernc. Please read the readme.
A word of warning: the source code of THViewer is quite a hack and the whole thing is more like a proof of concept / test-tool. It is by no means “user friendly”.

Kleiner Überblick

Gegeben war eine Beispielexceltabelle, die dem Format der anderen verwendeten Exceltabellen entsprach. Desweiteren hatte ich ein etwa zwanzigseitiges Dokument, in dem das Zielformat spezifiziert war.
Das Problem mit Exceltabellen: Prinzipiell sind sie nicht strukturiert. Es gibt für die Zeilen und Spalten kein festgelegtes Format, und ein Benutzer kann Daten in eine Zelle schreiben, die – bezogen auf das Zieldateiformat – nicht viel Sinn machen. Aufgrund der Tatsache, dass es keine eindeutige Abbildung von Quell- auf Zieldaten gibt, musste eine grafische Benutzeroberfläche her. Der Anwender soll die Möglichekeit bekommen, die konvertierten Daten nachträglich nocheinmal zu bearbeiten und, wo nötig, zu korrigieren.

Kleine Helferlein

Zum Einlesen der aus den Excelfiles erzeugten csv-Daten, diente mir der Excel-csv-Parser von http://ostermiller.org/utils/CSV.html.
Entwickelt habe ich das ganze unter Ubuntu Linux, der Einsatzbereich würde jedoch eine Win2K Umgebung sein. Hier kann Java natürlich seine cross-platform Stärke ausspielen; die JGoodies Look & Feel’s sorgen für ein plattformunabhägiges, ansprechendes Auftreten.
Beim Bauen der Oberfläche hat mit das Visual Editor Project von Eclipse sehr gute Dienste geleistet.

Wichtige Kleinigkeiten

Während des Projektes sind ein paar Sachen aufgetaucht, deren Lösung zunächst schwierig schien. Dabei waren dies keine größeren Probleme, allerdings doch Unbequemlichkeiten für den Nutzer der GUI.

JComboBox ist zu eng

Auch eine JComboBox wird, wie alle anderen GUI-Elemente, nur so breit gezeichnet wie die Fenstergröße und der LayoutManager zulässt. An sich kein Problem – wäre es nicht so, dass auch das Comboboxpopup in nur eben dieser Breite gezeichnet wird.

Im ersten Bild ist schnell ersichtllich: So lässt sich nicht arbeiten. Dieses Problem ist schon lange bekannt. So gibt es auf bugs.sun.com, Reports die auf Ende 1998 datieren. Umso verwunderlicher, dass es bis heute noch keine Standardmöglichkeit gibt das gewünschte Verhalten zu erzeugen.
Allerdings finden sich dort auch mehrere Workarounds.

Scrollbarposition soll fest bleiben

Sobald sich die Cursorposition innerhalb einer JTextArea ändert, wird automatisch der zu sehende Ausschnitt innerhalb einer JScrollPane verschoben. Meistens ist dies sinnvoll und erwünscht.

Kurzer Exkurs zur oben erwähnten Import-Funktion. Eine bestimmte Zeile pro Probe, sollte aus der csv-Datei in das Protokoll übernommen werden. Der jeweilige Text sieht für jede Probe ziemlich gleich aus – beim Durchblättern der Proben ist es jedoch wichtig, dass ein ganz bestimmter Teil des Protokolls direkt lesbar ist.

Nun wird beim Durchblättern der Beprobungen, immer der Text im Protokollfeld (man klicke oben auf den kleinen Screenshot) neu gesetzt. Damit wurde zunächst der Textausschnitt immer bis ans Ende gescrollt. Es ist zwar relativ einfach möglich, einfach den Cursor im Textfeld neu zu setzten (z.B. ganz an den Anfang). Die gewünschte Funktionalität war jedoch, dass der Scrollbalken immer an der gleichen Position bleibt.

Folgender Codeschnipsel garantiert dies:

prevRect = textArea.getVisibleRect();
textArea.setText( someStringHere );
 
SwingUtilities.invokeLater( new Runnable() {
  public void run() {
    textArea.scrollRectToVisible( prevRect );
  }
});

Der wichtige Teil ist hier das invokeLater. Normalerweise ist es so, dass die TextArea ihr Model mit dem neuen Text updatet, und dann irgendein Event feuert. Dieses Event führt dann bei der Parent-Scrollpane zum verschieben des Bildausschnittes.
Nun ist es so, dass die AWT-Eventbehandlung in einem extra Thread abläuft. Wenn man also direkt nach dem Setzen des Textes zum gewünschten Bereich scrollt, ist es sehr wahrscheinlich dass das Swinginterne Scrollevent erst danach behandelt wird. Schließlich erfolgt das in diesem separaten Thread.
Die Sachen die nun in invokeLater stehen, werden innerhalb des AWT-Eventthreads ausgeführt – und zwar erst nach allen anderen Swinginternen Events.
Diese Geschichte geht sogar soweit, dass es heute wohl Best Practice ist, direkt am Anfang der main-Methode, alles Weitere in den Eventthread zu legen.