Mitch's Blog

In the last post, I mentioned Conway’s Law:

organizations which design systems […] are constrained to produce designs which are copies of the communication structures of these organizations.

Dr. Conway was referring to people in his article–but what if we substitute “organization” with your product’s source tree and “the communication structures” to how functions and data structures interact? Let’s talk more about Conway’s Law in the context of source tree layout.

Many products of moderate complexity involve multiple moving parts. Maybe a company has a cloud service (back end) and a Web UI (front end). Or a back end and a mobile front end. Or a daemon, some instrumentation scripts, and a CLI. Or a firmware and a cloud service.

I’ve had my hands in a number of companies at various depths of “hands in.” Those who lay out a source tree that fully acknowledges the complexity of the product as early as possible tend to be the ones who win. Often, a company is started to build a core product–such as an ability to move data–and the user interface, the start-up scripts, the “stuff” that makes the algorithm no longer a student project but a product worth many millions of dollars–is an afterthought. That’s fine until someone creates a source tree that looks like this:

trunk/
	cool_product/
		main.c
		error.c
		util.c
		network.c
	stuff/
		boot.c
		cli.c
		gui.c

What’s wrong here? Presumably, some of the code in util.c could be used in other places. Maybe some of the functions in error.c would be handy to abstract out as well. An arrangement like this in which the cool_product is a large monolithic app likely means it’s going to be difficult to test any of the parts inside of it; likely modules and layering are not respected in a large monolithic app. (Note that I am not saying it’s impossible to get this right, but I am saying it’s unlikely that tired programmers will keep the philosophy in mind, day in and day out.)

A slightly different organization that introduces a library might look as follows:

trunk/
	lib/
		util.c
		error.c
		network.c
		tests/
			Unit tests for the lib/ stuff
	prod/
		cool_product/
			main.c
		gui/
		cli/
	tools/
		Build scripts or related stuff required,
		code generation, etc.
	platform/
		boot.c

As a side effect, we can also improve testing of the core code, thus improving reliability and regression detection. Ideally, the cool_product is a small amount of code outside of libraries that can be unit tested independently.

More than once I’ve heard the excuse, “We don’t have time to do this right with the current schedule.”

“I don’t have time for this” means “This isn’t important to me.” When you say, “I don’t have time to clean up the garage,” we all know what you really mean.

I was incredibly frustrated working with a group who “didn’t have time” to do anything right. Years later, that company continues to ship buggy products that could have been significantly less buggy. A few weeks of investment at the beginning could have avoided millions of dollars of expense and numerous poor reviews from customers due to the shoddy product quality. And it all comes back to how hard (or easy) it is to use the existing code, i.e., the communication structure of the code.

If you don’t have time to get it right now, when will you have time to go back and do it right later?

Getting it right takes some time. But getting it wrong always takes longer.

Teams with poor source tree layout often end up copying and pasting code. Sometimes a LOT of code. Whole files. Dozens of files. And as soon as you do that, you’re done. Someone fixes a bug in one place and forgets to fix it in another–over time, the files diverge.

If you’re taking money from investors and have a screwed up the source tree layout, there are two ethical options:

1. Fix it. A week or two now will be significantly cheaper than months of pain and glaring customer issues when you ship.
2. Give the money back to the investors.

If you’re reading this and shaking your head because you can’t believe people paint themselves into a corner with their source tree layouts, I envy you! But if you’re reading this and trying to pretend you don’t face a similar position with your product, it might be time to stop hacking and start engineering by opening up the communication paths where they should be open and locking down the isolation and encapsulation where they should not. This holds true for any language and for any type of product.

In 1968, Dr. Melvin E. Conway published an article called, “How Do Committees Invent?”

In this paper, buried towards the end, is the following insight:

organizations which design systems […] are constrained to produce designs which are copies of the communication structures of these organizations.

Thinking back on my product experiences, this has been the case every time. The cracks in products show up where teams didn’t talk to each other, where two people didn’t get along, or where someone wasn’t willing to pick up the phone and call someone else. Features or modules that integrated well and worked smoothly reflect where two or more people worked well together. In cases where one person went off by himself and re-invented the wheel, sometimes even large core parts of a product, led to internal difficulties and those internal difficulties turned into product difficulties when the product shipped.

As an engineer, every time you don’t pick up the phone to call a colleague about an integration point, you’re making life harder on your customer. As a manager, every time you don’t deal with someone not communicating, you’re making life harder on your customer. Meanwhile your competition who play well together are building beautiful products that flow.

The communication successes and failures of an organization are independent of the organization size. It’s fashionable to say that small teams work better than large organizations (37signals vs Microsoft), but in fact, a small team can be incredibly dysfunctional, just as a large organization can work well (many start-ups vs Apple).

Of course, the scope of “systems” goes beyond products. IT deployments–if your VPN guy and your Exchange guy don’t like each other, how many times do you have to login to different computers? Marketing strategies–700 folks clicked on an emailed link, but did those people have a good experience on the landing page? Sales operations–much time was invested in segmenting and building custom collateral but were those materials used or ad hoc assembled in the field? Manufacturing–sure, everyone signed off on the Micron chips, but “someone” decided to build half the boards with Hynix and didn’t tell anyone? Support–Is your support experience congruent with the product, or is it outsourced with its own login, and the support folks have their own culture?

A team that doesn’t communicate openly, frequently, and freely is expensive to operate and builds lower quality products, end-to-end.