Open source is good for IT

I gave a presentation at OSBC (open source business conference) last week. I don't often speak at conferences, but I have been working on an open source strategy for 4 years and believed that presenting that plan might help other CTOs/CIOs in their open source plans. Two bloggers (Matt Asay and Zack Urlocker) reviewed my presentation at http://www.cnet.com/8301-13505_1-9903582-16.html?tag=head and http://weblog.infoworld.com/openresource/archives/2008/03/kaplan_guiding.html?source=rss .

First, Kaplan was not the only company describing their use of open source. Other included CBS interactive, Paypal, Weather Channel interactive, NY times, Electronic Arts, LA times, Christian Science Monitor. In addition, a full 40% of attendees were IT workers, not open source vendors.

In my presentation, I laid out the case for open source, my 3 step plan deploying open source, my experience with vendors and challenges with open source (good summaries on two blogs linked above). I got a myriad of questions from the audience at the end. Our chief architect, Gautam Guliani, was there to answer as well when we were approached privately after general Q&A.

The most difficult question during general Q&A was from Matt Asay, OSBC organizer and also an open source vendor. He asked me the following:- "If an open source platform is stable and my team experienced with it, would I continue to pay annual support fees?". Unfortunately for the vendors, the answer was "no". It truly is a contradiction because if customers don't pay a vendor, they'll go out of business and their products will not be further developed. Frankly, I don't have a better answer. But that said, IMO, open source is a model that is here to stay. Vendors are diving into it and making money, although nowhere near Microsoft's current profit margins.

Open source is good for IT (the customer), period. I believe it is us, IT customers, who are driving open source adoption. In a way, it reminds me of the music industry, and how digital music was driven not by the big music companies, but by consumers.

My question now is:- What are large software companies like Oracle and Microsoft going to do about it?

The wisdom of herds

In evaluating technologies, it can often be a struggle to compare competing products. When done diligently, you will probably use a product comparison matrix with weighted scoring. By the way, allow yourself many weeks to collect and evaluate all of the data. And if other people are involved, then also expect lots of meeting to argue pros and cons. And if you also want to do a test drive, there's more time gone.  And you haven't built a darned thing yet!

There is another way, but unfortunately it is not always available to us. This other way is:- "What do other people choose?". Why spend time evaluating and/or testing a product when others have done it? This was, in fact, a strong arm technique used by IBM sales in the 80s. The argument was "No one ever got fired for choosing IBM". It actually worked, for a while.

Now, I am not suggesting that this is the hard and fast way to pick a technology, what if the technology is too new?. Rather, use the herd test as a primer before you start doing your own analysis. The results will at least tell you something. I recently heard that 99% of non-profit CIOs chose Windows XP over Vista. Now, that tells me a lot!

Finding out what others are doing before you get started on a lengthy evaluation reminds of a technique I learned in Fire fighting school (I am a volunteer fire fighter). When arriving at a fire scene, after getting hoses charged with water and ready, is to gain entry to the premises, often through the front door. We carry heavy tools to do this, usually an axe and a halogan. But before using these powerful tools on the door, try the handle first to see if it opens :-)

Search as a utility

We were discussing search at a recent NY CTO club meeting, and a thought occurred to me (as it frequently does in these meetings):- Is search a utility? Meaning, is search a plug-in function and not something to be developed by the tech team.

Every system we build has a search function built into it, usually hand-crafted (proprietary). Why? When I programmed years ago, every system had a screen-writer, which updated the characters and pixels on the screen. But no more, this is now a utility provided by the operating system. It would be crazy to do otherwise. I have plenty of other examples (showing my age!).

So, why isn't search just "available", like google desktop? Why aren't searches in different systems more effective? Why doesn't every search return results in "relevance" order? Why don't some systems have a decent search? Why is search completely different in every system (ever tried MS Outlook search)? Why can't searches be combined across systems?

Search on the internet, whether it be google, youtube, facebook, amazon, ebay, or linkedin, is solved for me, I always find what I need. And I believe the same is true for most consumers. But why not in the enterprise? Seems like a solution waiting to happen....

Effective post-mortems

One of my tasks as a CTO is running post-mortem meetings after we have an incident or outage. This is an extremely important step toward making progress in system stability and performance. Teams that don't do post-mortems miss the opportunity to get ahead of system issues.

Post-mortem meetings take some finessing, they can easily turn into a blame-game, or he-said she-said. So let me layout how I run post-mortem meetings, and how that makes them most effective.

First, important rules for post-mortems:-
1) Timely to issue (next day is best)
2) All relevant members present (no meeting if someone is missing)
3) Impartial moderator
4) Empty whiteboard to describe incident/issue
5) There is no blame

It is critically important to have "No blame", as you will make no progress otherwise. You need to acknowledge that everyone is working hard, systems can fail and its nobody's fault, and that openly discussing the issues together is the best road to ultimate resolution and system growth.

Now, even if your team knows "no blame", they will probably still be on edge at the start of the meeting. Its natural, failing systems create pressure. The team may also be avoiding dealing with the issue, hoping it will go away, and you may have to bring them back into it. What I find in post-mortems is that teams try too quickly to get to a solution. Don't let them, instead have them focus on a timeline of events.

I've found that starting the meeting with a chronology of events is extremely effective. I ask the team "what happened", and "when", and make them be exact about the when (i.e. 5:14pm), and I transcribe it all to the whiteboard. We include communications and hand-offs in the timeline, and any other information we collected at the time or gleaned later from system logs. Something about the focus on an exact time-line gets everyone to focus as a team. Maybe its because it turns us into detectives examining someone elses problem, not ours (if anyone has a better theory as to why, let me know).

After an hour, we usually have a list of immediate, medium, and long-term actions to take to remedy the issue. From that, the candidate cause/solution usually stands out, and we make sure we have alternate solutions should our candidate be wrong. These are captured on the whiteboard, and we make sure have both mitigations and solutions (making a problem go away is sometimes as good as fixing it).

Email wrong-number

Sometimes technology can have significant and unintended cultural impacts. Email in its infancy was a minefield, in that we never realized it was an inappropriate (and one-sided) tool for expressing emotions. It took us a while, but we finally taught ourselves to reread and pause before hitting the send button. In Japanese, there is a word for "unsay" to take something back, but alas not in English, nor is there a reliable unsend in email.

I've been using a Blackberry for phone and email for a while, and I've noticed an interesting phenomenon. I will call it the "Friendly wrong number". I meant to call Wendy A, but instead called Wendy B, since I dialed from my email address book and they were adjacent. "Hi, Wendy?" "Yes, who's this?" "Its Jon" "Jon? Oh, we haven't spoken for a few years, You still working at...?". Damn, wrong Wendy. But here's the kicker. I know Wendy and can't just say sorry, wrong number. This could get dicey if your boss is "Jamie B", and your best buddy is "Jamie C".

A variation is the call-back from someone you just spoke to, but by mistake. Some new phone feature must be causing that, I am guessing. Another variation is in email, the dreaded auto-fill feature in Outlook.

This makes me wonder what unintended future impacts technology will have. Many of my LinkedIn contacts have photos, and it won't be long before they also show on phone and email messages on my Blackberry. Will the prevalence of GPS maps make us lose our orientation, and we'll need to carry compasses with us where ever we go (Compass, what's a compass?). I guess we'll find out.

Worthless processes

When reviewing a write-up or new process/project proposal, a filter that I will apply is "Does this make a difference?".

Let's say one of your team was asked to write up a plan for "managing execution risk", and they wrote a document describing a process for doing this. After you read the document, you decide that while it all makes logical sense, it really will not make a difference. For example, tracking how many lines of code are written each day makes no difference. Who cares? Its business outcomes that matter.

As a CTO, it is your job recognize tasks that don't make a difference, and tell your team that they can stop doing. If something is worthless, say so. Or go one step further, and ask your team to tell you tasks they believe are not useful or optimal. Think of it as spring cleaning. Some tasks are not negiotable and are in fact important for compliance reasons (ie Sarbanes Oxley). But if its internal driven, reexamine it. Our teams are so busy, they'll appreciate the time savings.

So, what processes/tasks are your team doing that you can throw away?

Birds of a Feather

One of my most rewarding experiences is spending time with other CTOs. Who else knows what its like to do what you do? Nothing like hanging out over chinese food and a few beers with like-minded colleagues that I trust.

Earlier in my career, I felt competitive with my peers, both at work and outside. I felt that it was an "either/or" proposition, either I succeeded/got promoted, or they did. I was comparing myself to them. Eventually I realized that my success (and others) was independent of anyone else. If I was successful, I would be rewarded, regardless of anyone else.

Now I am realizing that younger technology managers I've watched over the years are joining our peer network, and its great to see them advancing as I did. It's such a pleasure to have them as peers, and to see them grow into their leadership roles.

One thing I always experience from peers is "have you thought about...". Its great because (1) I trust and respect their views and, (2) it challenges my thinking on something I hadn't considered. Reminds me of my high school days with a great teacher when I would have a break-through epiphany on a complex concept. There is always something to be learned from your peers.