Great Crisis inspires Great Leadership

Have you ever participated in a great crisis? Probably the answer is no. Most of the people in control of the Western World as we know it, have never seen a great crisis. Or as Tyler Durden would say:

We have no Great War. No Great Depression. Our Great War’s a spiritual war… our Great Depression is our lives.

We and everyone else around is enjoys a comfortable sedentary lifestyle. Even if there is individual crisis and the associated trauma, it has been a long time that there has been a collective crisis. And why is this a challenge for us, and for the western civilization? Because great crisis is a good teacher, it requires and therefore develops great leadership.

Have you been to the Moon recently? No? And neither was anyone else. The last manned Moon landing was in 1972! About ten years before I was even born. At that time a publicly accessible Internet wasn’t even available yet. Still, this incredible feat of engineering was accomplished.

What was driving those people at those times, how did they get the capable leadership and vision to pull through such an unimaginable project? One of the driving forces was the cold war, the result of World War II, which inspired a sense of competition between superpowers. But World War II also allowed produced a vast population of veterans, pilots and officers. People that were very capable precisely because the had gone through such a great crisis. They had a sense of what was possible through conducted cooperation, and they learned to release their knowledge and energy for the good of humanity, and made the moon landing possible.

And now Civilization is at a crossroad again, we are slowly losing that practical knowledge, the world lacks people and understanding of the forces that drives it. Beneath the surface is a vast network of machines, structures and organizations, that are for a large part on autopilot operated be the ants and bees on its levers and buttons, but without any oversight or insight.

I like to compare this to the scene in Matrix 2, where Neo goes to see Councilor Harmann in the underbelly of the secret underground city. The look at large machinery, that keeps the city running, and the Councilor laments how he does not understand what exactly it is that they do, but that he knows they keep the system up.

So now we have this challenge, on the one hand, there is the desire to avoid such a gruesome experience like a worldwar again, but there is on the other hand a lack of human beings, that carry the experience of overcoming such a crisis. So how do you learn what can only be learned through crisis, and at the same time work to avoid such a crisis?

Well, that is a good question, but it is a question for another time, as I have to leave know.

But maybe, I will feel inspired to write a follow up, or a part 2.


C++ Challenge: SimpleServer API

This week I’ve prepared a little challenge in C++ programming. The example is not extrodinarily complex, but not completely simple either. The purpose of this example is, to challenge myself a little to get to know boost::asio networking, and to contemplate about how simple APIs for prototyping could look.

So the challenge is to create a class, that allows prototyping simple network servers, by implementing a convenient base class. The interfaces and base classes are already layed out, including an example that send the reply “Hello” for any incoming connection.

The specification is a bit open, and is not so much concerned with corner cases. So using the simplest solution possible is probably the right thing to do.

Ok, here we go for the boilerplate: read on, I will add a few explanations below.

#include <string>
#include <functional>
#include <iostream>

class IConnection
    typedef std::function<void(const std::string& str)> send_function;
    virtual void receive(const std::string& str) = 0;
    virtual ~IConnection() {}
    IConnection(send_function send_) :
        send(send_) {}
    send_function send;

class IConnectionFactory
    virtual IConnection* create(IConnection::send_function send_) = 0;
    virtual ~IConnectionFactory() {}

class HelloConnection : public IConnection
        HelloConnection(IConnection::send_function send_) :
            IConnection(send_) {}
        void receive(const std::string &str) {
            std::cout << "Received: " << str << std::endl;
            std::cout << "Sending Reply: \"Hello\"" << std::endl;

class HelloConnectionFactory : public IConnectionFactory
        IConnection* create(IConnection::send_function send_) {
            return new HelloConnection(send_);

class SimpleServer // <-- IMPLEMENT THIS CLASS!
        SimpleServer(int portno, IConnectionFactory *connection_factory) {}
        void run() {}

int main(int argc, const char *argv[])
    SimpleServer server(1234, new HelloConnectionFactory);;
    return 0;

So, it goes like this: IConnection is the virtual interface for implementing a connection. That is, for every new connection, a new instance of an IConnection implementation is spawned, that is bound to that connection. When the connection is closed, the instance is deleted.

The IConnection receives a function pointer as argument, so that it can send replies. You can see how this is supposed to work in the example: “HelloConnection”. That example simple prints the received string on stdout, and sends “Hello” back.

SimpleServer is the class that you need to implement, here you can have all the bookkeeping, and you are probably going to need some internal connection class, that handles  each socket created, etc.

Anyways, I will probably be implementing the SimpleServer myself this week, so I might be able to post a follow up with a solution.

But anyways, this task is simple enough, that a veteran C++ programmer should be able to do it. And it is definitely intended as a practice to work one ones programming skill, so go ahead and try it.

Post your solution in the comments, or send them via email:

If you send via mail, I will add your solution to a follow-up post. 🙂

Have fun,

Managing C++ libs with CMake ExternalProject and Git

Writing modular software, you might sooner or later come into the situation, that you have some functionality, that you want to share between different projects. Let’s say, to have built a couple of convenience classes and function that you want to reuse.

There are different ways to skin the cat, one of them might be using a dedicated C++ dependency manager like Biicode. Or a general package manager for your platform of choice, like apt-get on Debian and Ubuntu, or Homebrew on OSX.

Usually you will start with putting your module or library into a git repositry, if you haven’t done that already. For Code that I want publish publicly, I usually choose my github account.

Now for creating a homebrew package, or a debian package, you either need to create some project description, a bit of boilerplate, and need some platform like launchpad to publish packages. Or in the case of biicode, you need an additional account at another platform, and you have to install another tool.

However, there is another approach, that uses software that you probably already have, and that is CMake and Git.

CMake has a functionality called ExternalProject_add, which can use, to automatically check out a git repository, and build that as part of the normal build process.

Of course, the same functionality would be possible with git submodules, but honestly, I always found this part of git rather cumbersome, and unnecessarily complex, for what I want to do.

So, I’ve created two small examples, that show how to create a small library, and link to it through CMake ExternalProject_add:

The first one is the library: cmake_simple_lib

And the second one is the project that uses the library: cmake_simple_demo

I hope you find that useful.


The decline of the hierarchical Organization in Popular Culture

Being a fan of the popular Space Saga “Star Trek”, and especially of the “Next Generation” Enterprise with Captain Jean Luc Piccard, I cannot help but notice the stark discrepancy in tendency to currently popular Television Series.
Star Trek and the Star Fleet is largely based around a huge hierarchical military organization. The chain of command is clearly defined, the captain receives orders from Star Fleet, the first officer respects orders by the captain, and everyone else respects orders from the first officer and the captain and so on. But it does not stop there, there are further roles within the organization, that supplement the strict hierarchical order. There is the doctor and the counselor, both of which are below the captain, in the chain of command, but never the less can override the will of the captain as experts, should they decide that the captain is incapable of fulfilling his obligations because of a medical or emotional condition.

In a way, the structure is laid out almost like a game of chess, where different roles have different advantages and disadvantages that can override each other in complex scenarios, making strategic games possible, and interesting.
I regard Star Trek as a historical artifact, that allows us today to take a peak into a system of cultural thinking an expectations that has changed not so long ago. A TV-Series is in a way a reflection of the society that created it, and a reflection of the society that were its primary consumers.

And in comparing the series to todays popular television shows, I cannot help but notice quite a stark contrast. So I keep asking myself, what has changed. What has changed in the perception of society in the perception of writers and producers, as well as consumers that made such a change inevitable.

Of course, there are series that contain plots that are concerned with hierarchical structure, and military organizations, Game of Thrones being the most prominent one that comes to my mind. But the topic of this series is never to indulge in the intricacies of a hierarchical organization, but rather to wreck havoc in every possible way of backstabbing and killing leaders and moving hierarchies around.

This is in stark contrast to the Star Trek series, where the concept of the structure of Star Fleet is never questioned as such. It is accepted as a de facto institution. The game is about how to work within the confines of this structure, and seldom and only in far and in between edge cases to question Star Fleet structures as such. So that does happen, but it is the exception, rather than the rule.

And if a series is not about wrecking havoc and creating intense drama and violence, it goes of into the other end of the spectrum.

Take all of the popular sitcoms, like Friends, How I met your mother, Big Bang Theory, The office and probably some more. These Series feel more like a kindergarten of childish innocence. The characters trapped in a Hell of Douglas Copelandian Nihilism. Nothing is serious, everything is wrapped in layers of self-depreciating sarcasm and cynicism. The Brave new World, everything is fun and games, and nothing is serious or important. The topic is no longer about creating great things, and exploring unknown territories, but rather it is about cuddling up in a warm and safe nest, and exchange pleasantries with ones fellow inmates. Every screwup can be shrugged of with a joke, and every tragedy with an outbreak of over acting dramatization, often about not much more than spilled milk.

Contrast this with seriousness of uncomfortable conversations between the leading officers of the Enterprise. Where they spend time pondering how their next move will be perceived, and whether it is moral and ethical to proceed in a certain path. Whether it is a viable tradeoff to insult the feelings of an adversary, or even if stall or a bluff is an appropriate response. None of those Strategic meetings are taking place in modern TV.

Is it that people have changed, that desires and goals have changed, or maybe it is our perception. The idea that we are no longer seeing us as serious and important in the things we do. That in fact the seriousness in which Jean Luc Piccard and his mates acted were overacting devoid of any resemblance in the real world, where all is actually fun and games.

Well, that is probably for everyone to decide for him- or herself.

Group Ritual Design

What makes a good climbing group? In a way it is the process of working together as a group, that actually creates the group, or that contributes to group learning.

But the same is true for any good work group. People work together on a challenge and through the challenge they learn how to work together.

Designing these kind of experiences is usually difficult. It is easier and can happen in situations that are arranged in a play. Like for example a climbing garden, there is a predetermined Course, with constructed obstacles, that are clearly designed to be challenging, but not too difficult or dangerous. There is a guide, and there are safety lines installed, that are controlled and solid.

The experience of going through that obstacle course is similar of a challenging group experience, but because it is prepared and limited in time, it is safe for the participants.

Developing an unlimited challenge is more difficult, because first of all it needs to be decided whether it will take place within the professional part of the participants time, or within his leisure time. If it takes part in his professional time, some argument of economic viability needs to be constructed. And for the leisure time project, the activity needs to be at least attractive and stimulating enough, to be considered of value to the participants.

What makes the process attractive could be rewards in the form of respect and validation within the social group.

What is needed to get at least some group coherence is a form of interaction. Those interactions could happen during regular meetings, while practicing together, a play or an act. The interactions could also happen through phone or email, when the topic is more of an abstract matter.

There is also the question of how much time should be spend in interaction. For some kind of sports on an amateur level, one or two hours each day can be enough. In the workplace 8 hours is considered adequate, however, not all of this time is usually spent in interaction.

What is also important, is the have the right intensity and variation of interaction. When the goal of the group interaction is to learn or improve a skill, then intense practice is needed, but also eventually alone time for the participants, where they can reflect upon their progress, and then again collective gatherings of reflection might be helpful as well.

The meetings and gatherings are conducted is a matter of ritual, and ritual can either be accidental and informal, or structured and preplanned. Both are valid approaches, and both can fail or work in different ways.

Usually when it is a physical meeting, it happens in a room, and there is a date and time, when the gathering happens.

The room already provides a setting, there might be chairs, suggesting the meeting will be conducted sitting, the chairs might be arranged in a circle, or facing a conductor in the front. This already sets a mood. Or the chairs me be stacked in a corner, suggesting that the participants themselves should choose an arrangement. There might be a carpet and sitting-cushions, inviting the participants to take a seat on the floor.

There are very different modes of acquiring status within a group, in some groups the publishing of a written paper is an adequate form of expression. The paper has to conform to a certain writing style, and it has to fit into a template, that is set up for the occasion.

In other groups the mode of expression means taking part in a competition, pitching one member of a sports-team against a member of another team in a duel.

And all those group interactions and modes of operation are codified into some set of ritual, and as the popularity of groups or rituals spreads, rituals get copied and adopted. Like the Gene-Pool, they become part of the Meme-Pool.

Popular Memes multiplying, mutating, and unsuccessful ones disappearing.

How the Fossil DVCS solves the privacy problem of cloud based applications

If you do not know what a DVCS (Distributed Version Control System) is, just think of Git, the popular source code management system used by the Linux kernel.

But it is quite different from Git, and seems to solve a couple of problems, that always left me a bit unsatisfied when dealing with Git. In fact, it seems to give me the same nice and warm feeling I got from using the Darcs DVCS back in the days.

I will start out with a couple of issues I had when using git that I experienced recently.

1) For many developers today, when they say they are using Git, they are in fact using a service that has Git in its core, but provides a number of additional services. A nice webinterface, a ticket tracker, a wiki and probably more that I am not yet aware of. However, while the product and concept is well executed, it is still a hosted solution, and while you are using it, you are dependent on an external company to provide this service.

2) I’ve been trying to replicate a similar service that provides on my own machines, and experimented with software like indefero and phabricator, but those solutions leave quite a lot to be desired. InDefero for example provides both a wiki and a ticketing system, but now I have the following problems: I have to provide a MySQL Database for the wiki content, and the ticket system, that needs to be maintained, updated and backuped. Which is kind of a headache, especially considering the nice solution that Fossil is providing.

3) There are various variants of git based wikis that I’ve tried, but installing them and keeping them running is almost always a nontrivial issue.

4) Overall, using git without relying on 3rd parties to manage various aspects of the deployment, is almost impossible, or creates quite a lot of busy-work. I only quite realised that when I came across the ingenious solution that Fossil came up with.

So lets see what Fossil does that makes it so nice. It is quite simple:

Fossil stores Tickets and Wiki data within the repository. And not only that, it comes with an integrated web interface, to conveniently use this ticket system and wiki. And said web interface can be easily made available on a public website.

These choices enable a couple of convenient side effects. Whenever you have a checkout you automatically can easily use wiki and ticket system offline, and once you push to the shared repository everything is update again. Great for using it on the road without internet access. Also, makes keeping a backup of the complete project way easier.

Makes for a much more resilient and independent tool, way better suited for the lone hacker and his small crew. Hosting is trivial, you can setup a private server, or use any kind of virtual server hosting.

In fact, I like that concept so much, I am already thinking about what kind of webapps could be “unclouded” with such an approach as well. Using a distributed versioned database that keeps a complete local backup and only needs to resync periodically to the master host.

Maybe it would be possible to extract some of the Fossil infrastructure and bake it into a general web framework. Or build something similar for different kinds of webapplications.

Have fun,

A new way of thinking and risk assessment

If you haven’t started reading the ribbonfarm blog I strongly urge you to consider reading it, because I find it to be on of the most pleasurable blogs right now. Venkats writing is very precise, and he manages to explain even very difficult concepts clearly and easily understandable.

But he is not the only lonely writer to be working on these ideas. There seems to be quite a trend of going into systems thinking and finding ways to incorporate risk and probability into every day decision making.

"The Black Swan" by Nassim Nicholas Taleb started that discussion in popular literature, followed by "Antifragile", by the same author.

But also "The World Until Yesterday" by Jared Diamond devotes a chapter to discuss the topic of risk assesment. And then most recently, Scott Adams published his book: "How to Fail at Almost Everything and Still Win Big", which goes on to discuss the idea of systems thinking on a more practical level.

And to be frank, I feel that I can apply this kind of thinking at a lot of situations. It is as if everywhere I look systems thinking could improve something or indeed does improve things for me already.

Have fun,

Engineering Managers Should Code

Hi Everyone, in this post I just want to direct your attention to the following article at Dr.Dobb’s:

Engineering Managers Should Code 30% of Their Time

Of course I really like the article, and I agree to that statement. And I want to keep this as a reminder for myself on this blog. 😉 And I also think it fits really well with some of the articles that I have written recently.

Have fun,

Building a virtual Software Company

Roles and Infrastructure
So let’s build a virtual software company. That is, a company that creates and sells software. While this kind of effort might be possible to do alone, to make it a bit more interesting it will be designed as a collaborative effort.

One idea of collaboration is to pool resources, and exploit the removal of redundant effort. This will be one motivation of that idea, but the other part will be that of a learning/teaching opportunity. Knowledge transfer if you want to call it that.

So in a highly structured effort which software development is, lets define a few roles, that might be worth considering.

Sets up Servers, installs operatingsystems, configures the routers. Eventually DNS, Firewalls, etc.
However, as cloud services, vservers, etc. are somehow a commodity, I’d prefer to keep these tasks to a minimum or outsource them entierly by using some service.

Mail Master
That role will be neccessery, even if it is decided that the company will use an external service for email. But someone has to add new teammembers, so this role will be admin for Email. The role could include setting up an email server on some virtual machine. Eventually the company will be sending out some kind of newsletter to customers, so somebody needs to take care that mail gets out. If a Newsletter Service is used, that role will include configuring that service.

Additionally, that role will likely include setting up and maintaining test-email accounts at various freemail providers, to make sure that mails go through to them unharmed, if custom email systems are deployed.

Web Master
That role will essentially do the same thing for web as the mail-master does for email. It will include setting up webspace, provide ftp-accounts, eventually installing and setting up apache, etc. If some kind of external Hosting Provider is used, and that should be done, this role will take care of manning the web hosting control panel.

Eventually the Web Master will also make sure that there is webspace available for having some kind of testing stage, when web application changes are rolled out.

And if the kind of software that the company is providing is web based, then there will be close collaboration with the next role.

Development operations is kind of the glue between web master and programmers. Devops basically provides and maintains tools for programmers. This role will be doing nightly builds, will make sure test suites are run, will make sure git repositories are available. They will be packaging applications, upload them to the file area, etc. They will also install the applications that the programmers create. They will maintain the tools to continously collect code metrics.

They will install development tools on virtual machines, and set up images for build servers.

This role will be doing the actual programming, and that is for web applications. This means both, public facing widgets that are on the web site, signup-forms, databases, cms-tools, crm-tools. The application that is being sold, and internal tools that will support other parts of the business, may it be a company adressbook, wiki, or whatever.

The job of that role is to commit stuff to a git repository. Whatever is being used as development environment is up to the individuals, as long as coding conventions are followed, unit-tests are maintained, etc.

While related, the advent and wide spread adoption of smart phone app markets has probably made it necessary to have some of those as well, so this group will be working on that. Devops will be creating packages and submitting them to app-markets. App-Programmers will submit code and tests to git.

This role will make sure that programmers do not lose focus. Here Application workflows will be created, and layouts will be sketched out, and later refined. This role will probably work closely with programming. Much of this role will also include making sure that the programmers go all the way through and implement clean validations for forms, that everything is pixel perfect and not thrown together.

Last words
And finally, don’t get me wrong, every role will be programming, mail-masters, web-masters and devops won’t be doing their tasks by hand, they will be creating scripts and control panels to streamline whatever they find themselves doing. May it be shell-scripting or automating a service via an API.

The idea here is to have some “easier” tasks, that youngsters can get their hands dirty, while creating some automation that will allow them to move up from their roles and graduate into more interesting fields.