The Open/Closed/Open Principle

First published June 2009. I still see folks making this mistake—”design good APIs”. Sure, fine, but what about when the APIs turn out not to be good? Or they were good but now they’re not? In addition to drawing boundaries between elements, we need to also be prepared to periodically re-draw those boundaries. I think this is difficult because re-drawing often crosses team boundaries, creating mis-aligned incentives (calling team needs the change but the called team has other priorities). I address cross-team design in the followup to Tidy First?, soon to be written with draft chapters available to paying subscribers.

The Open/Closed Principle always bothered me. I agree with it philosophically—good designs make it possible to add functionality without disturbing existing features—but in my experience there are no permanently closed abstractions. Superclasses or APIs might be stable for a (relatively) long time, but eventually even the most fundamental classes and interfaces need updating to meet emerging needs. I got to thinking about Kuhn‘s The Structure of Scientific Revolutions and its relationship to software design and found, much to the surprise of my jetlagged mind, a resolution to this dilemma.

Kuhn divides science into ordinary and revolutionary. Ordinary science is the kind that goes on all the time–gathering data, extending ideas a little at a time, publishing papers. Once in a while, though, some of the facts stubbornly refuse to fit the theory. Eventually the misfit is large enough to trigger revolutionary science. Crackpots propose wild ideas. Eventually one of them catches on, triggering another round of ordinary science.

Software Design

I got to wondering if software design goes through ordinary and revolutionary phases, and if it does, so what?

Ordinary design is the kind we do every day–extract a method, extract an object, move a bit of logic or state closer to where it belongs. The open/closed principle pretty much works. Superclasses sit. APIs sit. New features fit the design without much change.

Then comes a feature that really doesn’t fit the design. The fundamental elements and relationships have to be twisted to implement it. The fact (feature) just doesn’t fit the theory (design).

Kuhn describes revolutionary science as a time of chaos, with various odd ideas competing for attention. Each theory fits the new fact, but at the cost of not explaining the existing facts. In software, though, there is a preliminary phase. When the need for design change becomes apparent, software designers can isolate the part of the system that is to change from the part that will remain stable. Assumptions about a wire protocol, for example, may be scattered about the code, but before changing the protocol the designer can gather these assumptions in a single element. Then the parts of the system that don’t care about the wire protocol are unlikely to be accidentally affected. Isolation reduces the cost of change by reducing risk.

Revolutionizing designs without first isolating change puts a bigger burden on the designer. The challenges of revolutionizing a design while working in safe steps is generally enough for me without adding the challenge of keeping track of changes all over the code base. Isolating change is low-risk and fairly mechanical, while giving me an overview of areas of the system that are about to be overhauled.

An alternative (or sometimes companion) strategy is to start from scratch. If you have no idea how to support a new wire protocol, you can build a system that just has the wire protocol and nothing else. You will be missing many of the feature you expect, but you will be able to thoroughly explore designs for the protocol without distractions. What comes out of a de novo design is generally an understanding of the needs of the new design to be folded back into the existing system. Occasionally, though, you discover that you really don’t need all that other stuff and you have a tidy, and much smaller, system to work with.

For example, we started from scratch when we implemented JUnit 4. First we made sure we could run tests marked with annotations. When we understood that thoroughly we made sure the new tests worked alongside older tests. For the recent introduction of interceptors, though, we carefully isolated the code that ran tests and made it modular before finding a new way of modifying test running.

Revolutionary design violates the open/closed principle, almost by definition. The feature you want to add needs new elements and relationships that don’t fit with the existing design. The basic abstractions need to be reopened to modification. Once the feature is added, they can close again. Further development can use the new elements and relationships as vocabulary for further extension. This extension takes place against a background of ordinary responsive design.

Example

I’ve been interested for some time in better support for remote pair programming. The assymmetrical response time of screen sharing makes equal contribution impossible. What is needed is multi-local editing, where every user sees immediate feedback from their keystrokes and later processing handles remote updates and conflict resolution.

The Eclipse editor, not surprisingly, is not designed to handle such a feature. There is no clean separation between user events and changes to the internal model of the source file. This is not intended as a criticism of the Eclipse design. Design in advance of need is waste. The fact of multi-local editing simply doesn’t fit in the theory of the current design.

Following the outline above, the first step would be to modify the editor to clearly separate event processing from model updating. With that in place it should be possible to revolutionize updating to include the possibility of several sources for updates and for trying various architectures, peer-to-peer and master-slave, for detecting and resolving conflicts.

Conclusion

As I said in the opening paragraph, so what? Being aware of when I’m doing ordinary design and when I’m doing revolutionary design helps me cut down on the design space. When I’m doing ordinary design (which is my default), I try to support a feature with small incremental changes at the fringe of the design. Only if limited changes don’t work do I change hats and look for more fundamental and far-reaching changes. Ordinary design changes are easy to communicate and can be created at full speed.

Revolutionary changes requires much more care, both technically and socially. The change may require succession so the audience can absorb it. Revolutionary changes may also require many technical steps to achieve, both in the isolation phase, the experimentation phase, and the execution phase. Some experiments will fail–yes this design supports the new feature but it will never support the old feature. Sigh… back to the sketch pad.

UPDATE–Revolutionary design also requires different values from ordinary design. In ordinary design, I ignore design changes that make the code worse. Deliberately introducing duplication, for example, is unthinkable. In revolutionary design, though, I will happily duplicate code as long as I suspect I can eliminate it later. Sometimes an effective revolutionary strategy is just to inline absolutely everything in a class and helpers into a single method and begin re-extracting from there. I visualize this with the Design is an Island metaphor. Ordinary design is uphill, revolutionary design is under water.

Ordinary and revolutionary design are both necessary for responsive design. While most features can be supported with no design changes or only ordinary changes, new classes of features require revolutionary design. Staying in ordinary design as long as possible but shifting to revolutionary design as necessary and only as long as necessary helps keep the design lean, pliable, and a good platform for the needs of stories as they emerge.