Since the emergence of software architecture as a discipline in the early 1990s, much attention has been given to its use in evaluating competing solution options. The IEEE recommended practice puts “analysis of alternative architectures” as the first of thirteen uses of architectural descriptions.
Dr. Simon Field, Enterprise Architect, Admiral Group plc; simon.field@admiralgroup.co.uk
Simon will be speaking at the Enterprise Architecture & Business Process Management Conference Europe 22-25 October in London on the subject, “Evaluating Architectures: A Hands-on Introduction to the Solution Architecture Review Method“
Scenario-Based Architecture Analysis Method (SAAM) was the first widely promulgated architecture analysis method, originally created to analyse an architecture for modifiability. The authors contended that “software architectures are neither intrinsically good nor intrinsically bad; they can only be evaluated with respect to the needs and goals of the organisations which use them”. Their approach, of putting competing architectures into the context of the requirements for a given system, has formed the basis on which almost all subsequent architecture review methods have been built. This article won’t attempt to document the history of architecture review methods, but one successor to SAAM, the Architecture Trade-off Analysis Method (ATAM), has become the most widely used, and most mature, of the many evaluation methods that have been proposed since SAAM. ATAM was first published in 2000 by Rick Kazman, Mark Klein and Paul Clements of the Software Engineering Institute at Carnegie Mellon University.
ATAM involves an intensive evaluation of competing architectures by a team of stakeholders, which might include developers, a system’s owner, some users and those who will have responsibility for running, operating and maintaining the system. The team create and evaluate a set of scenarios against key quality characteristics (such as Reliability, Usability, Maintainability) by creating a “utility tree”, where each leaf node is a scenario, and the branch nodes are the quality characteristics. The tree is then analysed for each architectural approach that is under consideration, uncovering risks, sensitivity points and trade-off points in the tree and deriving a view of the overall utility of each approach. A major benefit of this method is that the assessment is directly related to those quality characteristics that are most important to stakeholders. It reflects the “multi-attribute” nature of a system’s behaviour and exposes the inevitable trade-offs between the different characteristics.
In 2005, when I was CTO at the Office for National Statistics, we had some success in applying ATAM to evaluate solution options for several major systems. However, we also found that ATAM’s complexities could inhibit achieving a common understanding amongst a wide-ranging group of stakeholders. The following problems were identified:
- The language of quality varied with each project, and seemed distant from the reality of business concerns;
- Utility trees were difficult to create and discuss, with debates too often becoming technical among the IT specialists, with the role of other stakeholders being reduced to that of mere spectators;
- The notion of utility seemed too abstract, and team members found it hard to agree on specific values of utility;
- ATAM focuses on software architecture, and we could see the need to evaluate competing solutions for wider systems and services.
This led us to further develop our approach to architecture reviews to address these concerns. The desire to evaluate service designs as well as IT systems also led me to consider in more detail the perspectives of a solution’s varied stakeholders. Although ATAM encourages the involvement of stakeholders in the process, different stakeholder views are not considered in detail in its analysis of alternatives.
Further refinement in partnership with the UK Border Agency, Emirates Group and some other global corporations has resulted in the Solution Architecture Review Method (SARM). It adopts a standard quality model, such as ISO 25010, which allows an organisation to get used to a common language with which to describe and classify high-level requirements. SARM replaces the use of utility with the widely understood business language of risk, considering the impact and likelihood of failure to satisfactorily deliver desired scenarios for different solution options. Experience with SARM has shown that business participants are much happier discussing and agreeing levels of risk than they are utility. And by using a spreadsheet tool to support architecture reviews, we can estimate levels of risk from different perspectives, including quality characteristics and stakeholders. SARM’s use of a stakeholder model allows an evaluation team to understand the trade-offs inherent in competing architectures from the perspectives of each stakeholder group.
The tool exists to support a team exploration, analysis and discussion of the risk trade-offs in competing solution options. It does not tell the team which solution to select – that remains in the hands of the evaluation team. But experience with ATAM, SARM and other architecture review methods has shown that a structured approach to evaluating competing architectures leads to the selection of systems and services that are more effective and sustainable, and can avoid costly problems further into an implementation project.
Simon Field is an Enterprise Architect at Admiral Group, and will be leading a hands-on ½ day introduction to SARM at the IRM Enterprise Architecture Conference Europe on 22nd October. His involvement in Enterprise Architecture began when he was appointed Chief Technology Officer at the Office for National Statistics in 2004 and continued when he took up the role of Chief Architect at Emirates Group in 2011. Prior to returning to the UK in 2017, he spent five years as Senior Executive Partner for Gartner, coaching and advising business and IT executives across the Gulf. His interest in evaluating competing solution architectures dates back almost 20 years to a project he led as head of e-business research at IBM’s Zurich Research Laboratory, and the resulting Solution Architecture Review Method has been further developed and deployed in a number of major corporations in Europe and the Middle East. Further information about SARM can be found at www.sarm.org.uk.
Copyright Simon Field, Enterprise Architect, Admiral Group plc