In Part 1 of this series we looked at the principles of module organization. It’s time to apply them in a real world scenario.
A concrete modularization example
At HomeAway, we grow a diverse ecosystem of applications organized around a highly scalable service platform, and at the same time maintain a number of acquired or legacy systems that are gradually integrated with, or consolidated into, the platform.
As part of this consolidation effort, we identified the need to decouple the service layer from the web (REST) layer for our Order Management component. Our goal was to extract the domain, data and service in a new Maven module to allow it to evolve independently. We realized this change was not straightforward due to cyclic dependencies between the REST and the service layers. Cycles had to be removed first.
We used a dependency management tool called JDepend to transform our component from a monolithic state to a pattern commonly used in HomeAway:
What is JDepend? JDepend analyzes Java bytecode and computes design quality metrics. From Mike Clark, the author:
JDepend allows you to automatically measure the quality of a design in terms of its extensibility, reusability, and maintainability to manage package dependencies effectively.
At its core, JDepend is a lightweight Java library capable of managing a simple model of classes, packages, and dependencies. JDepend’s own UI interface and the JDepend4Eclipse plugin depend on this library.
More importantly, build system plugins for Maven and Gradle are using JDepend to produce design quality reports as part of the build cycle. Based on this library, scripts or unit tests can be written to validate various dependency rules for a system implemented in Java.
An IntelliJ IDEA plugin for JDepend is not available, but IDEA users benefit from built in support for module and package dependency analysis via the Module Dependencies Tool Window. For more complex projects, there is an advanced tool: Dependency Structure Matrix (DSM Tool Window).
In Eclipse, the plugin can be launched via “Run JDepend Analysis” in the context menu of a source folder (e.g. src/main/java) that belongs to your Eclipse project. Upon completion, JDepend will automatically switch to its perspective, and display the analysis results in three views: packages, dependencies, and metrics. The screenshots below are showing the JDepend Eclipse plugin in action.
The state of our component before modularization shows a crowded ‘package as a directory’ structure that contained several packages involved in cycles:
Why having zero package dependency cycles is important
As we saw in Part 1, packages involved in a dependency cycle cannot evolve independently: they are difficult to separate, reuse and repackage. It is much easier for different teams to maintain a system if it has no package cycles.
With a focus on removing package cycles, we repeatedly applied the refactoring actions described in Part 1 and ended up with a simplified ‘package as a module’ structure with no cycles:
This iterative process included test updates and frequent test runs to ensure no regressions were introduced. Along the way, we identified overlapping units and took the opportunity to consolidate, leading to a ~10% drop in code. The effort required to complete was one week for one developer.
Removing cyclic dependencies was an important milestone because it allowed us to move the service code into a Maven module that is now built as a jar and depended upon by the REST module. We stopped refactoring here, but further improvements are certainly possible with help from the additional metrics described below.
These metrics have exploratory value and help formulate the right questions about a system. They are very good at tracking relative improvement from one change to the next:
- Number of concrete and abstract classes
- Afferent Couplings (Ca): count of packages dependent upon this package, is an indicator of package responsibility
- Efferent couplings (Ce): count of packages this package depends upon, is an indicator or package independence
- Abstractness (A): ratio of number of interfaces/abstract classes to total number of classes; metric range is [0,1]; A=0 for a concrete package, A=1 for an abstract package
- Instability (I): defined as I = Ce / (Ca + Ce), a metric for package resilience to change; metric range is [0,1], I=0 for highly stable, I=1 for highly unstable
- Packages involved in cycles
- Distance from the Main Sequence (D): defined as D = abs(1 – (A + I)), ranging in [0,1], measures balance between abstractness and instability. Ideally, packages are either abstract and stable or concrete and unstable. D measures how far a package is from this idealized state.
You should not rush into design decisions based on the above values seen in isolation. What we learned is the “before versus after” variation is more relevant than concrete values. After refactoring, it is possible to assess if the change has increased/decreased the metrics relevant to your goal while not regressing on the rest. In our case, these are the relevant variations:
|Metric||Before -||-> After|
|Package count||17 -||-> 6|
|LoC||29184 -||-> 26696|
|Class count||276 -||-> 242|
|Packages involved in cycles||13 -||-> 0|
An IDE plugin like JDepend4Eclipse or IDEA’s built-in support for code analysis offers the developer quick feedback during refactoring of a specific project. But looking at the big picture, design quality is just one of the code quality aspects that must be continuously monitored at the enterprise level.
Coding rules, unit tests, complexity, potential bugs, and comments are important chapters of code quality and at HomeAway we take quality seriously. We use Sonar to continuously examine quality metrics and trends and establish thresholds. This allows teams to focus on agility and speed. Teams prioritize and schedule refactoring in every sprint to counter quality erosion, deliver business value on time, and keep quality metrics above thresholds.
Design has a magnifying effect on software quality. As engineers we must have a deep understanding of our stakeholders’ long-term vision and be able to visualize and grow scalable solutions. As systems grow, we need discipline, methods and tools to develop healthy software.
We’ve mentioned concepts like responsibility, module, dependency, high cohesion and low coupling quite a few times, and with good reason: these should be part of our everyday design efforts. The last two are in fact included in a set of nine responsibility assignment patterns called GRASP. I’ll leave the other seven for you to discover.