Using git-svn for cross-platform development

I do a lot of cross-platform development, mostly Windows and Linux. Being able to build and test on multiple platforms before pushing changes to other people is crucial in order to avoid build and test failures. Git is almost the ideal tool to do this, but since we are using Subversion at work (that won't change in the forseeable future), I've been trying out git-svn to see if it is up to the task.

A typical work-flow goes like this:

1. Update local source tree with new changes from central repository
2. ... hack hack ...
3. Build and test on Linux
4. Transfer local changes to a Windows machine
5. Build and test on Windows machine
6. Fix problem which only shows up on Windows
7. Transfer changes back to Linux machine
8. Commit all changes to central repository, confident that everything builds and runs on both Linux and Windows.

The tricky parts are the "transfer changes from machine A to machine B" without committing to the central repository. I've tried several solutions to this (source code on a network share, "manually" copying the sources (for various values of "manually")), but none of them really flew. I always ended up committing to the central repo anyway. Fortunately we have a fairly low-traffic repo, and I can usually fix broken things before breaking anyone elses build.

(For the rest of this post, I'm ignoring Subversion branches, even if git-svn does handle Subversion branches as well as could be expected.)

Now, using git-svn isn't completely unproblematic either. In an ideal world, I would like to be able to both push and pull to/from the central repo from both machines A and B as well as push/pull between them, but git-svn does not support that. The translation of git commits and their Subversion counterparts has to be done in a single place, or havoc will ensue. In practise, this means that we have a single repo where we run git-svn. This repo can be pulled from, but should not be pushed to (or git-svn will get confused when trying to pull new changes from the central Subversion repo), nor should we pull from any other repo into it. The exact reasons for this is the topic for another post.

How do we then get our changes back into the git-svn repo if we cannot push or pull into it? The solution is to use the command "git format-patch". This command creates a patch-set containing a set of changes. "git format-patch origin" will create a patch for all changes present in the local repo, which is not present in the "origin" remote, so if I on the Windows box pull from the Linux machine, I can do "git format-patch origin" to get all the local commits I've made which I need to apply to the repo on the Linux machine (the git-svn repo) and from there committed to the central Subversion repo.

Once the patches have been transferred to the Linux machine (this can be done in many ways, I just copy the .patch files over a network share), they can be applied using "git am *.patch". This will apply the patches and commit the change. Conflicts are resolved just as normal git conflicts.

The workflow then becomes:

1. Bring in new changes from the central Subversion repo: git svn rebase
2. ... hack hack ...
3. Pull changes to Windows machine: git pull
4. Build and test on Windows
5. Fix eventual problems on Windows
6. Generate patch set to bring back changes to Linux: git format-patch origin
7. Copy patch files back to Linux
8. Apply changes: git am *.patch
9. Commit things back to Subversion: git svn dcommit

It is still a bit cumbersome, but the benefits are several:

• I can produce commits which I have tested on both Windows and Linux.
• I can use "git rebase -i" to edit/reorder the commits before finally pushing them to Subversion. This makes it easy to produce clear and concise commits.
• I can use "git gui" to commit partial files. I can include a single added line from a file with hundreds of other changes in it. This also helps in produce commits which are easy to understand.

In short, git and git-svn allows me to write better code.

Resistance is futile. You will be simulated.

Resistance is futile. You will be simulated. was the text printed (using a Star Trek font) on a limited number of t-shirts we had printed when I worked at Virtutech. The simulator is called Simics, and is a full-system simulator capable of simulator entire computer systems with sufficient fidelity that the simulated software is not aware that it is being simulated.

Raymond Chen recently posted an entry on his excellent The Old New Thing blog, titled If you can detect the difference between an emulator and the real thing, then the emulator has failed. This made remember a fun challenge once held at a party at Virtutech after I had left the company (I don't remember the occasion). There were two black screens displaying a Linux command prompt. One of the screens was connected to a physical PowerPC board, and the other was connected to a simulated PowerPC board. The task was then to decide, using the only the command lines of the two machines, which machine was simulated and which one was not.

Doing "cat /proc/cpuinfo" only revealed that both systems where running on PowerPC hardware (the output was different, but then only the people setting up the challenge new exactly which PowerPC hardware was being used), and the simulation was fast enough to not reveal anything performance wise. Now, I happened to have some insider info allowing me to solve the problem: Simics does not (or did not then) have any USB-support, which meant that the "lsusb" command should provide some crucial hints. Quite correctly, the simulated machine did not have any USB-devices, while the real hardware had a number of USB-related devices. I'm not sure if I won the challenge or not, but I remember a bunch of people standing around the terminal discussing different ways of beating the system.

Egit

I have a hobby programming project I work on from time to time. The goal is to implement a JVM (or something resembling one). It will most certainly never be completed, but it is fun to work on it.

One thing I've used it for is to learn and experiment with git in a way were I don't risk breaking things for other people.

Git's command line interface is as impressive as they come, but since I've used Eclipse CDT for writing the code, I wanted to try Egit. Also, since git is on it's way to replace cvs at Eclipse, I thought it would good to know.

The general impression about Egit is that it is pretty solid, but a few areas needs polishing, for example, there's no easy equivalent to "git pull". The "fetch from" command gives me a big dialog with no sensible defaults. A lot of thought has been put into the command line interface, and creating a GUI with the same expressiveness is not done overnight.


- Posted using BlogPress from my iPhone

Error recovery features in Clang

Came across this blogpost about the cool clang error recovery features:

http://blog.llvm.org/2010/04/amazing-feats-of-clang-error-recovery.html

Clang really goes "above and beyond the call of duty" to deliver good error messages. For example, it has a built-in spell-checker, so that if you type "int64" instead of "int64_t", it will actually point that out. It also detects source code conflict markers, and only parse one side of it (failing of course, but without attempting to parse the conflict markers).

This is probably very true:

It's sad but true that being an experienced C++ programmer really means that you're adept at decyphering the error messages that your compiler spews at you.

New keyboard

While at EclipseCon, I visited the Apple Store at the Valley Fair Mall, and bought an Apple Keyboard. Very nice and clean keyboard which frees up quite a bit of space on my desk.

Setting it up wasn't really that difficult, but in order to be able to access keys like forward delete and page-up

/page-down, I had to install the boot-camp drivers. This question on Superuser.com proved useful in order to find the drivers. Now I just need to train my fingers a little to avoid pressing Fn instead of Control.

Highly recommended keyboard. If you can't live without the numerical keyboard, I recommend that you buy this one instead.

EclipseCon 2010

I'm leaving for EclipseCon 2010 tomorrow morning. Looking forward to an interesting week, and hopefully some SF sightseeing as well.