Traditional requirements happen after a multi-system architecture has been defined.

But what about the requirements that feed into that architecture? The requirements that drive the enterprise architecture decisions in the first place? We haven’t talked about those before.

Setting The Stage

The typical enterprise software requirements project ecosystem looks something like the following:

1. An existing application exists within a network of (logically) interconnected applications that collectively comprise an enterprise architecture.
2. Business processes exist, each of which utilizes one or more of these applications to achieve tangible business objectives.
3. Some of these applications have user interfaces, and therefore have users who are trying to achieve the goals associated with the business processes.

Within that environment, the typical enterprise software requirements project looks very familiar:

1. There are new processes, or capabilities that are desired by the business.
2. An enterprise architect has decided that these capabilities will be embodied into one or more of the existing applications. The end result is that each application has a “defined scope”, and a set of goals.
3. The requirements project for each application under development is initiated with these objectives and boundaries.

What if you are tasked with defining the ecosystem, instead of defining a product within a pre-defined ecosystem? How would you go about it?

• The architect makes design decisions about what each system in the architecture does.
• The developer makes design decisions about what each class in the application does.

Isn’t this just design, writ large? Why should business analysts or product managers be involved? This is just an implementation activity on steroids.

What We Are NOT Talking About

There is a very good article at IBM that defines architectural requirements as application requirements that have significant impact on the architecture of a single application. That is not what we are talking about. Here’s a link, since that is a good article – if that’s what you were hoping to read today.

The Lifecycle of Creating Software

In One Man’s Trash, we looked at the life cycle of developing a software product from the perspective of the contributors to that product.

Our analysis was of a single product – not an application that has to live in an ecosystem with other products. That analysis showed four layers of solving the problem, but only one meta-layer of solving the problem.

In the uselessly big picture view, there are multiple layers of “requirements” and “design” throughout the life cycle. When defining requirements within the “single software product” scope we wrote about before, most of the following is actually a distraction.

1. Companies and individuals have money to invest, and they have goals associated with those investments.
2. They will choose a mix of investments, where that mix is designed to achieve a set of objectives.
   1. Manage volatility
   2. Generate ROI (where the return is utility, but is easiest to simplify as a financial return)
   3. Mitigate risk
3. Within the constraints imposed by that mix strategy, they design a portfolio. That portfolio delegates roles to individual investments. Investment 1 is responsible for generating a high return at a high risk. Investment 2 is responsible is designed to hedge investment 1, mitigating risk while reducing overall return. Investment 3 may be responsible for generating a smaller, but highly reliable return. Each investment thus has a scope and a set of goals.
4. If one of those investments is in a company, it now has a scope and a set of goals – and a large company will develop its own portfolio of investments – choices in how it allocates funds internally. One of those investments may be a choice to operate in a particular industry, where those operations are defined by a set of goals and constraints. [Note - conglomerates may have additional, recursive sets of layers 1 through 4, as they manage a portfolio of companies.]
5. Those operations may manifest as a set of business processes, initiated by people using software, and supported by other systems internally. The choice to use people, or software, and how to organize the systems is one of design.
6. Once that design choice is made, each individual piece of software has a defined scope – a set of responsibilities within the larger architecture. And those responsibilities become goals and constraints within the context of a set of contracts between the systems. The entire chart above lives within this one step in the bigger picture.

When managing software requirements in the typical enterprise software project, this scope and these goals is already defined. Here are a couple examples of an application being defined within an enterprise architecture:

• A company needs to employ direct sales reps to sell their products. The company has a software application that the reps use to sell the products, and that application is integrated with another system that manufacturing uses to fulfill the orders. An accounting system also exists that is responsible for collecting funds due for each of the orders. Sales reps have to be compensated for their work, and a compensation system is identified to calculate sales rep compensation. It must be integrated with the existing systems and is responsible for determining how much the reps should be paid. The compensation system must integrate with the payroll system that is responsible for paying the sales reps.

Each of the applications (sales, fulfillment, payroll, collections, compensation calculation) has a defined role to play in the architecture. A typical requirements project would be to define the requirements for the compensation system. It helps to understand the ecosystem – the context in which this product has defined responsibilities – but that is generally a constant, not a variable. So, most requirements books, tools, and techniques are designed to operate with a presumption of a fixed set of goals and scope. The goals and scope of the product usually need to be discovered and validated, but they are perceived as immutable – at least within the timeframe of the project.

“Standalone software product” product management operates under essentially the same principles – the company is assumed to have a strategy, a target market, goals and a scope for the product. Requirements work for a standalone product is similar to the requirements work for an enterprise application that “knows its place” within an enterprise architecture.

But what about the requirements that drove the enterprise architect to choose an approach that involves separate applications for sales, fulfillment, collections, payroll, and compensation? These are architecture requirements

Defining Architecture Requirements

We know how to define and manage requirements within a predefined environment. Start with goals, and work our way down to “design.” Other people on the team will start at design, and continue through development, test, deployment, and adoption. This is a pretty comfortable environment. It is also an effective environment. When a problem involves the interaction of multiple systems, an efficient way to tackle that problem is through decomposition. Just as an investor decomposes a portfolio into individual investments, a company will decompose enterprise software into multiple discrete systems. And a developer will decompose a single system into multiple classes (or multiple modules, each made of

And we know the importance of requirements – the most successful teams don’t go immediately from goals and constraints to development. Multiple studies demonstrate the cost-effectiveness of using requirements effectively to prevent mistakes in implementation. It stands to reason that we are missing an extremely large opportunity if we don’t apply equivalent rigor to the architectural decisions that lead to the definition of those goals and constraints. So how do we go about defining the requirements that feed into the architecture?

Here’s a really good article by Scott Ambler that provides a more detailed perspective, and practical guidance for enterprise architects. The rest of this article is geared towards business analysts who find themselves supporting enterprise architects pursuing a path like the one Scott describes.

The first step is to identify the business processes – all of the business processes that are relevant. Using our example, these are the compensation-management processes, sales and fulfillment processes, payroll processes, and collection processes.

Part of defining these processes is identifying the stakeholders. We’re dealing with a much larger scope here – so valuation of the different stakeholder goals becomes much harder. It is easier to prioritize “sell bundled products” versus “sell customizable products” than it is to prioritize “pay sales reps” versus “fulfill orders.”

A very high level view of the processes is enough to do an first iteration of an architectural design. This is also sometimes called a straw man architecture, or candidate architecture. Notice that we said “first iteration.”

The next step – and arguably, it is a concurrent step – is to define the boundaries between those systems. Where / when does the order placement system (first system) transfer responsibility to the fulfillment system (second system)? And what does the first system have to provide to the second system for fulfillment (and possibly collections and compensation) systems to do their jobs? And how must those systems respond? What information must they provide to meet user requirements? What information must they provide to satisfy audit requirements or regulatory rules?

While this looks like design, I would argue that this is also feasibility analysis. The architects are designing the enterprise architecture – or the system under development. But that system is also an input into the requirements processes for each of the individual applications in the architecture. The scope of each individual application is defined by the system boundaries. The goals for each individual application are a direct result of the responsibilities that are identified by mapping the business processes to the individual applications.

Now we have a candidate architecture, a set of identified business processes, and a mapping of those processes onto that architecture – defining the responsibilities of each system. This yields a feasibility analysis by the architects. They will take this assessment (and any identified gaps), and modify the architecture – adding or removing systems, changing responsibilities, and otherwise iterating on the candidate architecture. Then they will review and iterate again – until there is a feasible design – just as developers will invest in the design of an application until there is a feasible solution to meet the application requirements. And this process should involve collaboration with stakeholders, and revisiting (or even redesign) of business processes with the candidate architectures in mind.

Conclusion

Developing architecture requirements (and architectures) is an iterative layering exercise. As much as it isn’t “our job”, our contributions to defining the architecture requirements will not only serve to verify the feasibility of the architectural approach, but they will provide clarity about the role of each application in the architecture. And that clarity of purpose sets the stage for a more effective “traditional” requirements process for that application.

This will also cause a significant reduction in requirements churn later – as flaws at the inter-system level will be identified and resolved before application requirements get underway in earnest.

We’ve written before about the importance of separating rules from requirements, particularly in use cases. We wrote that with the goal in mind of reducing the costs of system maintenance. Low-level rules like decision, calculation and inference rules tend to change frequently – and independently of other requirements. So a documentation approach that separates these rules from requirements can both reduce implementation costs (by encouraging separated implementation) and reduce the time required to manage and approve changes.

There are also benefits to abstracting high-level, or procedural rules, when dealing with global business requirements.

Global Businesses

When companies get large enough that they start operating globally, they face an additional level of complexity in managing their processes. And with that management burden comes the additional complexity of managing the software that implements those processes.

Global businesses usually don’t have the luxury of using the exact same processes, procedures, and rules in every company in which they operate. They also have to deal with nasty problems like selling a product from one country to a customer in a second country, with a delivery address in a third country. And the product may be shipped from a fourth country. Now imagine that the order is split to deliver portions of it to each of multiple destinations. And imagine it is further (orthogonally!) split because the products are being sourced from multiple locations. That’s a lot of complexity. How would you approach managing the requirements for something like that?

Let’s keep it simple and just imagine dealing with different processes and procedures in different countries. Global businesses usually organize around regions, like the Americas, Europe, Asia, etc. There are no formal rules to this organization, but the regional definitions are mostly consistent. Within some of those regions, there are often variations in how the business operates within countries in those regions. US Companies usually treat domestic US business differently than they would Canadian or Mexican operations – even on the same continent. And Europe, Africa and the Middle East have even more variation and distinct differences between countries.

Regional Growth

When companies grow into multi-national organizations, they don’t initially start in “every country” and then gradually grow all of the divisions in lock-step. Companies rightly focus on a single country until they are driven to grow into other countries. Then they establish footprints in those countries, grow those businesses, and eventually reach a size where they need to organize regionally.

The businesses in those countries and regions are usually given both guidance and latitude in the development of their businesses. As a result, the company will both develop new processes and extend existing processes into the new countries. Imagine a company like Caterpillar or John Deere that sells heavy construction equipment around the world. The company initially, likely, both manufactures and sells all of the vehicles in the same country. Then the company begins exporting vehicles to the other countries. Eventually, the business grows until the company sets up shop in the foreign country.

The company is now doing the same things in multiple countries (at a high level). Selling vehicles, leasing vehicles from a maintained fleet, financing purchases, etc. Consider the business of leasing vehicles – different countries have different cultures that approach borrowing differently. They also have different regulations about leasing versus sales. Our company may even decide to partner with a local third-party company that manages a fleet of lease vehicles for them in one country, while maintaining their own fleet in another country.

The people running these country-specific businesses have to react to local issues, regulations, and market dynamics. Success in those countries may require different operations than success in the home country. And the people responsible for these businesses are usually able to “do what it takes” to be successful in the different countries.

Global Processes

Now the company is faced with the situation where the same process (leasing a vehicle) exists in multiple countries or regions. At the same time, the process is different in each of the countries, because of the differences in how the business operates in those countries.

The company now has the same process defined multiple times. And that process is defined not only based on local influences, but also corporate policies, the policies that are in effect in the country where the company is based (and therefore reports income, etc), and those things that are common across the regions.

This means there is both variation and duplication in the process. And that creates opportunities for improvement.

Consider a use case that is part of a global process. For the vehicle leasing process, a use case (or sub-process) may be “Approve credit of lessee.” The company requires that all leases be made only to people who have been approved as credit-worthy. But determining credit-worthiness can vary by country. And the process required to determine credit worthiness will vary by country.

Variation by Country

Consider the definition of credit-worthiness. This is defined for a single country with a set of rules that represent the results of an actuarial analysis that does a risk-scoring process designed to weed out unprofitable loans. The credit-worthiness of a lessee in a third-world country would necessarily be different than the analysis in a major industrialized nation. And that means a different set of rules.

What about country regulations? There are different privacy laws in place in different countries. You may be allowed to contact character-references in one country, but not in another. You may have access to three years of records in one country, and seven years in another. These variations in regulation affect what you are allowed to do in the different regions.

And the businesses in different countries have evolved their processes independently over time. One business may choose to pre-approve a lease for repeat customers, while another country may require credit checks for all new leases.

These differences make the goal of globalizing processes and requirements seem insurmountable.

Why Globalize Requirements

There are many reasons to understand processes at a global level. Let’s look at just one. Reporting. Management accounting is the science (or art, if you ask a financial accountant) of inspecting measurable elements of your business so that you can make informed decisions about how you run your company. Being able to understand, analyze, and measure the same process in every country of the world provides benefits in strategic reporting and decision making. This consolidation can also be valuable for financial accounting and reporting of business performance for public companies. Making good management decisions is enough of a reason, so we’ll ignore the added financial accounting and external reporting benefits of a consolidation.

OK, so your construction equipment company operates in multiple countries. And you lease equipment in multiple countries. You want to know how profitable your leasing business is. Each country has developed different processes for leasing, with different rules for determining details like who to approve for a lease, and different calculations of profitability. For example, one country may monitor gross margins, another may track contribution margin. What about the rules for revenue recognition? One country’s business may recognize revenue at the time of signing a lease, while another may recognize revenue only when the payments are due (or when payments are made).

To get better control on your businesses, or more specifically, more consistent reporting, you need every country to follow the same process. Even though the “business” in each country needs to be handled slightly differently.

Consolidate Requirements

One approach is to take all 50 countries, analyze all 50 process variations, document 50 different use cases, and determine the proper reporting for each of the 50 situations. That’s a lot of analysis work. And a lot of risk to the reporting effort. You avoid disrupting the individual businesses, but you may make mistakes in any of the many analyses. And whenever any of the countries has a change in regulation, or when any of the businesses in those countries has a change in process or policy, you have to review and update your process, use case, and analysis for that country.

Another approach is to mandate that everyone follow an identical process – come hell or high water. This is completely impractical. For some countries, this will work. For others, this will force the businesses to operate at a competitive disadvantage (remember, there was a locally rational decision to deviate in the first place). And in the worst case, you may be “requiring” the business to operate illegally (or prevent it from operating at all) in some countries.

A third, more reasonable approach is to look for a common abstraction. Develop a description of the process and use case that applies to all of the countries, while allowing for a representation of the variations. This might encourage individual countries to make changes to their processes (in order to appear more consistent with the rest of the business), but it allows them to continue to deviate.

How is this possible – we’ve identified that each country has a different process, so how can the same process documentation be correct for the different countries?

Business Rules Provide the Abstraction

Most of the variations in process identified in the example above are really variations in rules, happening within similar or identical processes. Think of it from a use case perspective – here’s a snippet from the normal flow of the “Approve Credit” use case:

1. Lessee submits request for financing.
2. System approves lease per BR01, Credit Approval Rules.
3. Lessee confirms intent to lease.
4. System reserves vehicle.
5. …

The key is the abstraction of Credit Approval Rules. Even though those rules vary by country, all countries will follow the flow defined above. Note – this does not address the case when the processes are structurally different.

Then, within your business rules repository (or spreadsheet, or wherever you maintain the rules), you will have a ruleset (BR01) that defines when credit is approved and when it is rejected. What you can do is define different rulesets for different countries (or regions). BR01-USA, BR01-Canada, BR01-Mexico, etc.

When the business within a particular country changes its credit approval criteria, you only update those rules – and not the process or use case documentation. The use case remains unchanged.

In our calculate profitability example, you will need to either require the businesses to calculate the same way, or, because the data will be consistent, you can aggregate and report on the information at a global level – taking into account which transactions happen in what countries.

When Processes Are Structurally Different

What about when the process is significantly different in some countries? Perhaps the system reserves the vehicle in advance of qualifying the lessee (pre-approval of the lease)? We have a different process. Technically, it is a variation on the same process – the process would share the same name and have the same high level description. Only in the details would it be different.

We get to “make up an answer” here, I think – I don’t know of anyone who has standardized on a good solution for this problem. There are two reasonable obvious approaches that come to mind.
The first approach is to define multiple use cases, and organize them hierarchically beneath the original (global) version of the use case. If the use case is “UC01 Approve Credit”, then you would add “UC01A” or “UC01-USA” to reflect that this use case is a variation on the first one. Then you would trace the special case use case to the global use case.

This works, and minimizes redundancy, but it could be confusing to deal with this added complexity, in light of subordinate use cases, use case versions, and tracing of requirements. Would the requirement for “User submits application” be traced to UC01, or UC01, UC01A, etc.? You would trace to all variations for which the requirement is required. Each country-specific implementation team can then look at each use case and say “Do we use the global one, or do we have a special variation?” The team would then know precisely which requirements they had to implement.

Another approach would be to define the global use case – “UC01 Approve Credit”, and then for each country that has an alternative process, define an alternate flow through the use case. This takes care of the confusion with subordinate use cases and versioning. And it reduces the need for redundant tracing. But it makes the individual use case harder to read. And since most software today only traces requirements to use cases (and not individual steps in the use cases), it makes it harder for the implementation teams to know which requirements they have to implement for their countries variation.
Ultimately, we document requirements to foster improved communication. An approach that makes a use case harder to read (and approve), or harder to implement (or trace dependencies) is not as good as one that might make repository-maintenance more difficult, while improving the ability of the teams to understand both the contents and the dependencies of use cases.

Summary

Global businesses have variations in processes and use cases in the different countries where they do business. Documenting the individual processes leads to redundant documentation of requirements, makes it more difficult to analyze the processes (including reporting and accounting), and does not help in standardization of practices.

Abstracting rules, and finding commonalities in process patterns allows for consolidation of process documentation – which improves communication and analysis of requirements.

Utilizing alternate use cases to represent when individual companies or regions deviate from the global standard allows businesses to deviate their processes when needed, while minimizing the impact on making and managing changes to the processes across the organization. It also provides a framework for analysis, tracking, and encouragement of standardization of processes across a multi-national company.

Each elicitation technique we have in our toolbox is a tool. But not every elicitation job is the same. If we have a hammer, we might be working with nails, or screws, or even an egg. In our analysis, we have to develop a deep understanding of our customer’s business(es). And that means we need to understand not only the goals and ROI, but the processes, rules, and requirements. Which is the right tool for each job?

Elicitation Techniques

The BABoK (Business Analyst Body of Knowledge) identifies ten different methods of gathering information. That list is a good one for describing the complete tool set that business analysts should have for elicitation. The same techniques are valuable for product managers too.

We wrote about those techniques almost a year ago, from a requirements gathering perspective. Check out that article for more details, but here’s the short list:

1. Brainstorming
2. Document Analysis
3. Focus Group
4. Interface Analysis
5. Interview
6. Observation
7. Prototyping
8. Requirements Workshop
9. Reverse Engineering
10. Survey

Different Views of the Business

Processes, requirements, and rules all represent different elements of how the business works, or how software must work to support the business. The most obvious distinction is one of perspective.
To oversimplify a little, some rules define the regulations and policies that constrain how a business operates. Some rules represent calculations, inferences, or enable actions that provide some precision about the business operates at a greater level of precision.

Processes represent a business design decision by the company. Given a set of goals and constraints (both “rule constraints” and practical ones – resources, costs, market forces, etc), processes are designed to achieve business objectives. They generally manifest as procedural instructions for the business. Note that they should be modeled at a higher level of detail than “open this file, walk down to purchasing…”

Requirements are an articulation of what a tool (in our case, a software product) must do when it is used in one of the business processes. The design of those processes has an impact on the requirements of the software. The rules that constrain the business (and by extension, the processes) must be enforced within the software.

Ultimately, we have to understand all three to create successful software.

Different Drivers of Change

A more subtle difference is in recognizing that processes, rules, and requirements change differently. Each class of artifacts changes for different reasons, on a different time scale.

Policy rules generally change as top-down mandates, or external forces. State or federal regulation changes can affect how a company does business. The Sarbanes-Oxley act has driven massive changes in how companies operate. These are relatively infrequent, but sweeping changes. Their effects cascade through everything the business does (and therefore everything that is required of the tools used to operate the business).

Decision rules generally change as part of feedback loops in processes. Consider the set of rules that an insurance provider uses for determining what premium to charge for a given policy. These rules are so extensive that they are maintained in rate tables, or other complex rules-calculation systems. An insurance company will have a department who’s responsibility is to maximize the profits that come from setting premiums. That means that they will tweak the rules in their rate table on a regular basis – like an ongoing science experiment. Predict, modify, inspect, repeat.

Process changes can happen as a result of policy changes, but more likely, they happen because of business re-engineering. This is the same feedback loop that affects low level rules, but on a grander scale. The big business consulting companies make very good money providing business consulting services. Part of that is strategic guidance, and part of it is process re-engineering. They look for inefficiencies and missed opportunities in processes. The predict the ROI of changing the processes, propose changes (maybe even help manage those changes), and repeat. These larger changes take more time than changing low-level rules.

Requirements articulate how the software must support the given processes, while implementing or enforcing the company’s rules. Requirements can change because of process changes, and policy rules changes. As long as you separate rules from requirements, then changes to low-level rules will not force changes in requirements. And ideally, for volatile rules, will not require you to modify (and test and re-release) your software. Requirements also change because they represent our understanding of the business’ understanding of what they wanted when we gathered them. And the business can change it’s mind, as it gets new data. One of the arguments for iterative development is that people don’t know what they want until you’ve given them what they don’t want. There’s some truth in that.

Effectiveness of Different Techniques

We’ve mentioned before that the same elicitation activities, and people, generally gather process data, rules, and requirements. And that presents one of the challenges in managing them independently. Here’s a table that summarizes the effectiveness of different elicitation techniques for uncovering and defining a business’ processes, rules, and requirements.

Each technique has a row, and is graded Consumer Reports style. Each type of artifact has a column. A full green circle means that the technique is well suited to eliciting that information. A half-filled circle means that you can use that technique to gather the data, but it is not as effective as the full-green-circle techniques. An empty circle means it isn’t impossible, but you should avoid it if you can.

As an example, reverse engineering is ineffective at defining any of the above. Reverse engineering can tell you what you have, but in no way can you assume that what you have is what you need. In fact, if it were, then your customer wouldn’t need to do the new project at all. Reverse engineering is a last resort that we sometimes have to rely on, but you should always treat it as a “starting point” – a baseline to be corrected and validated. It is certainly going to paint an incomplete picture – and often an inaccurate one.

And You?

These effectiveness assessments are based on experiences with dozens of clients in the enterprise software space, over a decade of elicitation. That means they are anecdotal. Thousands of people will read this article [Really. How cool is that?!]. So if you can confirm the above, or even better – correct it and share your experiences, we’ll be able to paint a much better picture. So – join in the discussion and let us all know.

We created a series of Visio 2003 stencils last September to support our series of articles on BPMN. Anthony Britton has created a Visio 2002 version of the stencils for people who do not have Visio 2003 (or Visio 2007).

Thanks Anthony!

The Visio 2002 stencils for BPMN can be downloaded here, or from the original article. Any future updates will be recorded in the original article (this post will not be updated).

Should you use use cases or process flow diagrams to document business requirements? At some level, they both document the same thing, they just document it differently. The best requirements will come from doing both – but what if you are forced to choose one? What are the tradeoffs between use cases and process flows? In this article we look at the documentation of failure handling.

Documenting Failure Modes or Error Handling

Failure handling is something that software has to do. In every process or use case, there are going to be non-trivial failures or other unanticipated events. The exercise of identifying those failure modes is relatively straightforward. Deciding what to do in each situation is harder. Documenting and communicating those requirements can be harder still.

A Simple Use Case Example of Failure
Alistair Cockburn uses a simple example in Writing Effective Use Cases, to describe how to handle this situation with a use case extension. We’re paraphrasing his example here:

1. User requests to update her billing information.

2. System requests authentication of user identity.

3. User enters authentication information.

4. System authenticates user.

5. System presents billing information for editing.

6. User edits her billing information.

7. User indicates that edits are complete.

8. System updates billing information.

9. System acknowledges update of billing information.

What happens if the user authentication fails?

Cockburn suggests using a use case extension to deal with this situation. His suggestion applied to our example would read:

4a. Failed authentication

4a1. System notifies user of failed authentication and re-requests authentication.

4a2. User re-enters authentication information.

4a3. System authenticates user.

One thing you will notice is that this is a linear representation of the use case. It includes the anticipated failure condition – failed authentication.

Process Flow Example of Failure

You can create the equivalent example of the Edit Billing Information use case as a process flow.

[larger image]

The process flow diagram provides two immediate benefits relative to the sample use case. First, by using swim-lanes, there is a visual reinforcement of the roles of the user and the system. The use case identifies the actor for each step as well – as text. Second, the decision box highlights the possibility of failure, and makes it obvious that there is a desired way to address failed user authentication. The use case shows this information, but only at the end of the prose, because of the linear structure of the text.

These benefits come from the spatial representation of the process. By laying out a diagram in two dimensions, you increase the information density of the artifact. The need for, and execution of authentication failure handling is more clear in this example.

More Complex Examples

Reviewing this simple example could lead you to conclude that there is near parity between the two documentation approaches. Consider a process with more failure modes, or even failures within failures. What if the system is required to lock out the user account after three failed authentication attempts? This is a straightforward extension to either example – but the use case form begins to get “clunky” with nesting extensions. The process flow diagram scales better to complexity.

Generalized Conditional Branching

This example can be generalized for any if…then situation.

When writing a use case, there is the often overlooked step of replacing “if…then” with “if…then…else.” It can be easy to overlook a missing branch to a decision tree. One of the goals of writing requirements is to write complete requirements. You want to avoid anything that makes it easy to write an incomplete requirement.

As a justification for making sure that you identify failure conditions and failure handling when writing use cases, Cockburn points out the completeness benefits. He says that you may even identify new actors, processes, and business rules while defining the failure handling extension of a use case step. We feel this is a weakness of the use case format, not a benefit of failure handling.

Other Factors

As with almost every element of requirements management, there are many factors to consider. Process flows are not universally better than use cases, or vice versa. This article focuses only on the need to deal with conditional branching.

For communicating the requirements associated with conditional branching in a process, a process flow diagram is better than a use case.

Outsourcing is gaining momentum not only as a way to reduce costs, but as a way to create global teams. This trend is driving an increase in demand for business analysts. The change in perspective is driving companies to think about how they manage their business in new ways, and driving interest in new tools for business analysts to achieve these goals.

Hat tip to Ryan for his article on the year of the BA. The source that Ryan found is this yahoo article on 7 trends for 2007.

Trends

The role of the business analyst has emerged as a focal point for enterprises trying to wring more out of their automation-technology investments.[...] Ironically, outsourcing is feeding a frenzy to bring aboard more analysts. The experience of outsourcing IT has taught firms their technology-project specifications were in much worse shape than they believed. When companies would deliver specs to outside firms and individuals who didn’t have deep experience with their existing automation solutions, the knowledge gaps became painfully apparent.

Neal McWhorter, in the Yahoo article (emphasis ours)

We’ve all talked about outsourcing, and many of us have designed teams with different collaboration models to address the communication challenges.

This Computerworld article suggests that business analysts are one of 5 top areas for hiring, and business process management is one of 5 top technologies being explored.

How Business Analysts Adapt

Neal suggests that BAs will evolve their role and responsibilities to be something analogous to a product manager. We’ve been treating the roles similarly here – with the primary distinction being a single-customer focus for business analysts.

Neal mentions the IIBA and their new certification program as a likely means of setting expectations of and for business analysts. Barbara writes about the upcoming certification schedule for 2007.

Tools

We can’t overemphasize the need to be good at writing requirements first, and then using tools to become more efficient second. With that said, the increasing importance of business analysts drives the increasing importance of helping BAs be more effective.

Two areas where there are huge opportunities for improvement are business process modeling and requirements gathering/management.

BPMN

There are tools and vendors for BPMN solutions, and it looks like training is finally going to be more available in 2007. Bruce Silver is developing training that uses ITP Commerce as a tool provider. We’re currently designing a training course as well, built by extending the tutorial series we wrote last year, using visio stencils and a vendor-agnostic approach.

Requirements Management Software

Today, the space can be characterized as having the following solutions available:

• Spend a lot on an enterprise package, and mandate that all your projects use it, in order to recover the costs.
• Build your own solution, using spreadsheets, documents, email and other tools.

We expect this to change in 2007 too. Both of the approaches above can be cost ineffective for single-projects within a large company, SMBs looking to become more effective, and small consulting shops providing services to other firms. There’s a lot of opportunity here for the right solution.

Conclusion

Is it the year of the business analyst? The need is indisputable. Will the C-level execs act on it?

What is the difference between a business rule and a business requirement? Does the difference matter? A business requirement is something that is multi-customer, and a business rule represents a single customer’s approach to meeting that requirement. Product managers and analysts care about both, but product managers emphasize requirements, and analysts focus more on rules.

A Simple Example

We’ll set the stage for discussion by showing a requirement, a rule, and a design and implementation.

• Requirement – The building will provide a measure of protection against fire damage.
• Rule – Firefighters will be assured access to the building to enable extinguishing of any fires.
• Design – The street near the fire hydrant and main entry of the building will be reserved for exclusive use of firefighters.
• Implementation – A sign will be posted in a visible location, instructing people not to park in the reserved location.

Background

The debate about requirements versus design, and the level of detail of specification has been an active one over the past year. We’ve written about how you can imagine software as an onion, where implementation is wrapped in design, which is wrapped in requirements. When we include awareness of business processes as part of what we’re doing, we can add another layer – business rules.

One point of confusion may be that many contracts are described as “gather requirements” when in reality, the work is “define rules appropriate to a set of predefined requirements.” This inconsistency leads to many debates that include arguments like “that isn’t a requirement, therefore it must be design” with the response of “that isn’t design.”

Ideation

Moving from each layer to the next involves ideation, and exhibits concreteness. For a given requirement, an approach can be defined for meeting that goal, and with that approach comes rules. For each rule, a design can be selected, and for each design, an implementation is created.

Each step involves ideation – making inventive choices about how to achieve that next step. Consider our example. We have the requirement of providing protection against fire damage. We could choose to achieve this requirement by mandating that only non-flammable materials be used in the construction of the building, or by integrating fire suppression equipment in the building, or by optimizing the access for firefighters to the building. Once we make that choice – an ideation step – we define the rules that express how we will achieve our goal with the selected approach. Our rule is assured access in this example. The design approach is to reserve the street near the main entry to the building. The implementation of this “reservation” is a sign. It could be special barricades that block access to anyone who isn’t a firefighter.

The point is that there are choices at each step of the way.

Each step becomes more concrete, more focused, and more narrowly applicable.

Product Managers Are Market-Centric

As a product manager, when interacting with customers to better define the needs of the market, we need to recognize that the rules are not the requirements. In the example, access for firefighters isn’t the requirement, protection against fire damage is the requirement. This is the level of abstraction that can be used to describe market needs.

Business Analysts Are Customer-Centric

A business analyst needs to understand the requirement (protection), but only in so much as it affects her company. This allows her to explore alternative rules (access, suppression, prevention, etc.) that will achieve the requirement.

Requirements Elicitation Can Fall Short
When we interview people to gather requirements, they will usually talk about implementation, design, and rules. Rarely will they describe requirements. Users and subject matter experts (SMEs) generally talk about implementation and design (“We have a sign, because the street needs to be reserved.”). Our elicitation techniques and listening methods help us identify rules by asking “Why?“ (“To assure access to the building for the firefighters.”). Without those techniques, we will only uncover implementation and design.

Getting to Requirements From Rules

To get to the requirements, we need to keep asking why. (“To provide protection against fire damage.”) If we keep asking “Why?” we will eventually get to a useless “primary goal.” And by useless, we mean that it is a goal that is too nebulous to drive anything actionable. Consider these possible chains of why questions and answers for our example:

• Why provide protection against fire damage? To allow us to meet city codes. Why does that matter? So that we can legally rent out space in the building. Why does that matter? Because we need to rent the space to make a profit. Why does that matter? Because our shareholders demand it.
• Why provide protection against fire damage? To allow us to charge higher rent to tenants. Why does that matter? Because we need to maximize our profits. Why? Shareholders.

Part of the art of product management is knowing when to stop. “Protection” is actionable. “Satisfying shareholders” and “Profit”, while laudable goals, do not provide the needed direction and detail to be actionable.

Summary

The line between requirements and design is broad, and is more of a region than a line. Rules live in that region – from one perspective they embody a design approach to achieving a requirement. From the other perspective, they represent the business requirement, in more detail. Use elicitation techniques and ask why to uncover the rules and requirements.

In support of our series of BPMN Tutorial posts, we’ve created a series of Visio 2003 stencils (*.vss) and a template (BPMN_Template.vst) of BPMN symbols. Anthony Britton has created a Visio 2002 version of the free stencils – thanks Anthony!

Download this free resource today courtesy of Tyner Blain!

BPMN

We wrote an introduction to BPMN as a process diagramming approach on July 18th. We followed that with a series of 24 articles providing an introduction to diagramming with BPMN. We created stencils to support our BPMN diagram creation, and a template, to make it easy to create diagrams in Visio 2003. With Anthony’s contribution, you can also use these stencils with Visio 2002.

Stencils

The 1.0 version of the stencils includes the following elements:

• Start, End, and Intermediate Events
• Tasks and Subprocesses
• Gateways
• Flow Elements
• Artifacts

Download

Download the 1.0 Version of BPMN Stencils and Template For Visio 2003 today.

http://tynerblain.com/downloads/BPMN_1_0_Stencils.Visio2003.v1_0.zip

Download the 1.0 Version of BPMN Stencils and Template For Visio 2002 today.

http://tynerblain.com/downloads/BPMN_1_0_Stencils.Visio2002.v1_0.zip

Usage

The contents of this zip file may be freely used for personal or commercial purposes. Modification or resale of this work is expressly prohibited without the consent of the author.

Redistribution/republication of these files is not permissable. Please direct people to this page for downloads. Any future updates or corrections to the files will be made available at that location.

Feedback

Please add comments with any feedback or suggestions for improvement. Otherwise, enjoy the free stencils!

All process improvements are not created equal. How should we select which processes (or process steps) to improve? How do we approach this for a really large migration project? Start with understanding the potential for improvement and then narrow it down from there.

The Basic Concept

When re-engineering a business process, ultimately we want to maximize the ROI of our improvements. That means understanding the costs of today’s process, the costs of tomorrow’s process, and the costs of creating and transitioning to the new process. On a large project, this can be a herculean task. We don’t have enough time to do all the math. We don’t want to get paralyzed with analysis activities. Where should we start?

We should start with the processes that have the highest potential for improvement. Since profit can be simplified to a simple equation of savings minus deployment costs, we should start by finding the processes (or process steps) with the highest initial costs (and therefore the highest potential savings).

A Simplification

To simplify the presentation of this idea, we will ignore probabilistic costs (risks, errors, modeling) – not because they aren’t relevant, but because the cloud the issue. Imagine for the rest of this article that the only costs in a process are operating costs. The same approach can be extended and refined to include other very real costs.

We will also use an example that depicts process steps – it could easily depict processes. The only difference is the level of detail. On really large projects, use this technique to identify high potential processes, then drill down into them to identify high potential process steps.

Potential

To calculate potential, we calculate the costs of existing process steps. Consider the following simple (existing) process:

The process has five steps, A through E. Step B is a decision, meaning that some times we execute step C, and other times we execute steps D and E.

We want to know which step of the process to focus on improving – so we have to identify the step with the greatest potential for savings.

There is a simple formula for defining the cost of any step in the process.

Frequency x Effort x Burden x Period

• Frequency (number of occurences per unit of time)
• Effort (units of time spent in the task)
• Burden (money per unit of time)
• Period (unit of time for the analysis)

We can represent this with a simple template of a spreadsheet.

Sample Data

By creating sample data for each input, and calculating the cost, we can compare the potential for each step in the process.

If we were to ignore frequency information, then step C would appear to have the most potential, because it has the highest “per occurence-cost (with an effort of 5 hrs at $20 / hr). However, by recognizing that step C is only executed 10% of the time, we see that the cost of step D is actually the highest (at $36,000 per year).

Interpretation

Our interpretation is that step D has the greatest potential for savings, because it has the highest cost. Steps A and E come are next, followed by step C. Step B is so cheap that it isn’t likely to be worth evaluating.

Next Step

The next step is to propose a replacement for the highest potential step (D). Alternately, we may look for a way to combine steps D and E with a single replacement (as mentioned in this article about process improvement). After exploring this solution approach, we would look at steps A and E as candidates for replacement. Estimating the costs of replacing these steps is the next thing we do. The costs of performing the steps today, minus the costs of performing the new steps in the future (plus the development and transition costs) determine if we should consider this as a step for replacement.

We will only consider those steps where the profitability of change exceeds our hurdle rate for investment.

Once we have profit data for each proposed step change, we can prioritize these changes, and then schedule them in conjunction with the skills of our development team.

Strategic Change Loophole

Executives like loopholes. Perhaps because their accountants find so many for them. We have a loophole here too – a process step may need to get replaced because of internal office politics, or for “strategic” reasons. We need to make sure our stakeholders have a way to include financially excluded choices.

Summary

Identify the most expensive processes to run. Determine ways to replace them. Calculate ROI and use it to drive prioritization.

Sandy Kemsley, of Column 2 fame, is blogging the Process 2006 convention “live” as it goes. Subscribe to her blog to stay on top of things. For now, here are the articles she’s posted from day 1.

• Peter Fingar Keynote
• Martin Ould
• Keith Harrison-Broninski
• David Deeks, Sunderland University
• Networking
• Peter Blatter, Citibank Germany
• Wrapup

Some choice quotes from Sandy’s posts:

“One thing that I really liked was his analogy about BPM: buying MS-Word doesn’t make you a novelist, and buying a BPMS doesn’t make you a process-oriented company.The technology is an important enabler, but there’s much more to it than that.”

“[OMG, we just hit a slide with 249 words -- the print is getting smaller and smaller. Keith, you're killing me!]”

“This is definitely the first presentation that I have ever seen, anywhere, that quotes Winnie the Pooh, and gives an example of strategic objectives as illustrated by Christopher Robin and Edward Bear. By the time he got to his main case study, which was the hospital administration process around having a vasectomy, I was wishing that he had been my prof at university.”

“Surprisingly, he then went on to talk about the emotional aspect of decision-making as it relates to customer centricity: how some decisions are almost purely emotional (like buying a convertible or having plastic surgery) and the financial practicalities of those decisions become less important. Not the argument that I expected from a reason-driven German banking COO, but he posits that being a customer-centric organization is understanding the balance between reason and emotion.”

I won’t tell you which quotes are from what articles – you’ll just have to check them out to find out more about the stuff you like. :)

Thanks Sandy for the blow-by-blow.

One danger of using a precise language like BPMN to describe business processes is that you can precisely get yourself into trouble. Deadlock (in BPMN) is a condition used to describe a process that can’t be completed. By designing (or describing) the wrong business process, you can create a process that never finishes.

Rewarding the Pedants

At the end of our earlier article about compensation events, we hinted that there was another problem with the diagram we debugged. That problem was deadlock. We diagrammed a process that could never complete. Congrats to the people who caught it!

A Simple Deadlock Example

Take a look at the following process diagram and see if you can identify the deadlock condition. Here’s a hint – you might want to review how gateways work in BPMN.

The process splits at an inclusive OR gateway, and then joins together with a parallel gateway. The deadlock case (a process that can’t complete) comes from the precise nature of the definitions.

• Flow from an inclusive OR gateway will follow at least one, and could follow some or all of the paths that leave it.
• Flow into a parallel gateway requires that all incoming paths be resolved before continuing

When the salesman sells more than twenty cars, both paths are taken from the OR gateway, and then both paths are combined in the parallel gateway. There is no deadlock condition. We have a problem when the salesman sells twenty or fewer cars.

In the problem case, one of the paths (“More than 20″) is not taken by the process (the salesman does not get a bonus). The parallel gateway still requires both paths before completing – it has no way of knowing that the bonus-path was never started (or rather, the parallel gateway doesn’t care). So the parallel gateway will “hold up” the process until both input paths are completed. Since the “bonus path” is never started, the parallel gateway will never be satisfied and the process will never finish. This is what BPMN folks call deadlock.

Simple Solution To A Simple Problem

The solution for this illustrative example is as simple as the problem.

The parallel gateway that joins the flows together before “Pay Bills” has been replaced with an inclusive OR gateway. This gateway specifically waits for every path that was activated. When only one path is initiated, only one path is required. When both paths are started, then both paths must be completed before proceding.

A More Complex Example

This example is more along the lines of the mistake we referenced in our earlier article.

In this example, our actor attempts to walk and chew gum at the same time. If he trips, he will reach one of the end events, but he won’t achive the multitasking award. At first glance this seems acceptable – the process starts and ends, and the actor doesn’t get a multitasking award.

However, this process is also deadlocked, because of the join gateway. The join gateway must complete both of the incoming sequence flows – but if the actor trips, the flow from the XOR gateway (trip?) will not be activated and the gateway will never complete.

A More Complex Solution

The key to solving this deadlock problem is to make sure that all of the paths into the parallel gateway are completed.

The XOR gateway (trip?) is now processed after the two sequence flows are reconnected. If the actor tripped, he will not receive the multitasking award. If the actor didn’t trip, he would.

Summary

When using parallel gateways to join paths, we must make sure that all incoming paths are handled. We can avoid the deadlock problem by replacing a parallel gateway with an inclusive OR gateway. We can also resequence our process steps to assure that all incoming sequence flows to a parallel gateway are activated.

Nils proposes his rule of three boxes as a consideration when developing software or software features to improve business processes. In short, make sure that you can actually execute the new process. It isn’t enough to create a good “replacement process” – you have to be able to transition to the new process and then back out of it. The new process is plugged into a business ecosystem, and it must coexist with the existing processes.

From Nils…

Improving the process is all well and good. But this rule addresses the fact that processes don’t live on their own – there are processes that come before and that come after.

Nils Davis

Nils presents an easy to understand diagram of the original process (A->B->C) and the end state after replacing “B” (A->B’->C).

Nils also presents four common causes of friction between the new improved process and the existing processes with which it works.

• Platform Mismatch
• Technology Change
• Unfamiliar Look and Feel
• Legacy Data Integration

From Tyner Blain

Nils uses photoshop as a great example – photoshop is easily an order of magnitude better at editing images than previous manual solutions. However, getting the image from a film negative to photoshop, and then onto paper (for use by the printer) was still hard. This slowed the growth of photoshop (or drove growth in the digital imaging and digital printing industries).

In addition to considering how best to integrate B’ with A and C, we should also consider A and C as “fair game” for re-engineering. We should consider the possibility of going from {A->B->C} to {A->D} or {D->C} or just {D}.

By keeping our requirements at the proper level of abstraction (not design and value focused), we can question the importance of replacing each element of the previous process. We can also develop an understanding of why the original process was put in place, and how best to modify it. This is a great approach when dealing with migration projects.

Another consideration is change management – what does our customer have to do when migrating from their old process to our new one? Perhaps the change management should break that transition down into stages (e.g. initial dual systems, then mixed systems, then finally a new system).

Conclusion

Make sure you’ve targeted the highest priority problems to fix. Then make sure the solution can be integrated into the existing process. Then determine how best to manage the transition of actually running the business from the old system to the new.

One of our readers (thank you!) pointed out that another blogger was critiquing one of our earlier business process modeling notation (BPMN) diagrams. Turns out we made a couple mistakes. Here’s a more detailed look at the compensation end event.

The Web-Publishing Process

One of the great things about writing and publishing on the Tyner Blain blog is the feedback loop. In BPMN, it looks a little like this:

Usually, when a critic finds a mistake he will point it out to me directly, or traffic will come from the critic’s site, and we’ll find out via our analytics.

It turns out that we did make a couple mistakes, so thanks Bruce for pointing them out.

First, The Mistakes

In one of our articles on end events, we created an example to demonstrate compensation end events, and we had a couple mistakes in the diagram – not too good for a tutorial. Here’s the offending diagram, with a couple annotations in red showing the problems.

These two mistakes (circled in red) indicate a mis-application of the compensation end event.

The compensating activity, “Record Possible CC Fraud” is a mis-application of the compensation intermediate event, because that task doesn’t actually undo, or compensate for the task to which it is associated (Process Credit Card).

The second mistake is that there are two gateways (a fork and an exclusive OR) that can be reached before the compensation end event, but no other activities. The compensation events, when triggering compensation, are designed to allow you to “go backwards” in the process, and as such, expect that at least one activity is processed before deciding to compensate. The presumption is that if compensation can be initiated without performing any other tasks, then it could happen immediately from within the task that has a compensation intermediate event on its boundary. The gateways don’t count as activities, because they only analyze existing data.

The Corrections

Here’s a new example diagram, without those two errors.

The first correction shows that the compensating task, “Record Transaction Recision”, is truly something that is done to compensate for recording the transaction. The second change is that there are two distinct tasks between the activity being compensated and the compensation end event. The first task, “Request Credit Card Charge”, sends and receives messages from the bank. This is the task that could cause a compensation (controlled by the gateway’s analysis of the results of that request). The second task, “Notify Customer of Failure” will also occur, but only if the bank denied the charge to the credit card.

Summary

Compensation events require that tasks be processed between the task being compensated and the triggering of that compensation. Compensating tasks must compensate for the task to which they are associated.

For the Truly Pedantic

There is also a semantic concern with the original diagram – the original diagram shows that the product would be delivered, even if the credit card charge failed to go through. This could indeed happen – when a vendor is unable to contact the bank, but unwilling to lose a sale. Think about the old “paper charge” handheld machines – they would create a slip that records the transaction in exchange for goods or services – and only later process that transaction, which could fail. But this is a concern, since it is very uncommon these days (only us old folks remember).

There is also another problem, although it may be more of a problem with the BPMN spec than the original diagram. We’ll talk about that in a future article. [Update - it is definitely a problem with the diagram, and not the spec. - Discussion to appear in 20 Sep article]

BPMN Diagrams don’t use the term control flow to describe processes. They use the terms sequence flow and message flow. Within sequence flow, there are four classifications of flow.

• Normal Flow
• Exception Flow
• Link Events
• Ad Hoc

Normal flow is exactly what you would expect – flow starts at a start event, travels through activities (both tasks and subprocesses), and then ends at an end event.

Exception flow is the type of flow that happens when a process deviates from the normal flow. A timer intermediate event, for example, can initiate exception flow at a specified time. After triggering at least one activity, his exception flow may lead to a stop event or may rejoin the normal flow.

Link events connect processes and subprocesses together. The “flow” that happens between two links is not represented in the diagram by the solid-arrow, but is inferred from the link events.

Ad Hoc flow can be considered a lack of flow, or an undefined flow. Ad hoc flows simply indicate that the activities within a subprocess that uses ad hoc flow can be processed in any order – there are no rules associated with the sequencing of the activities.

Normal Vs. Exception Flow

In this business process modeling example, a worker has a daily routine. The worker either goes to the office or works from home. She will work all day, except to take a break for lunch, and then will enjoy her evening. Each sequence flow in the diagram is identified as being either a normal flow or an exception flow.

The BPMN specification does not clarify exactly which flows qualify as exception flows. It is possible that only the flow immediately attached to an intermediate boundary event is considered to be exception flow, or it could be that all flow reached via an intermediate boundary event is exception flow. A third possibility would be that all flow that can be reached only via an intermediate boundary event should be considered to be exception flow.

Exception Flow occurs outside the Normal Flow of the Process and is based upon an event (an Intermediate Event) that
occurs during the performance of the Process. While Intermediate Events can be included in the Normal Flow to set
delays or breaks to wait for a message, when they are attached to the boundary of an activity, either a Task or a Sub-
Process (see Figure 10.53), they create Exception Flow.

We propose that all flow that can only result from an exception to the normal process should be considered exception flow.

Link Events

Link events connect processes and subprocesses. A link event can be an end event, or it can be an intermediate event. Both of those articles provide detail on how link events work. The “flow” between the first link event (leaving the current process) and the second link event (entering the new process) is considered to be a type of sequence flow, but no one talks about it that way. It is included in the list of sequence flows for completeness, not clarity.

Ad Hoc Flow

Ad hoc flow, precisely, is a set of activities with no pre-defined sequence requirements. Further, there is no specification about how many times any activity is performed.

The following example shows the unstructured day of our worker if she were to call in sick. The sequence flow that enters and exits the expanded subprocess in the diagram is normal flow, and all possible paths within the subprocess are considered to be ad hoc flow.

The worker can choose to perform the tasks within the subprocess at any time, in any order, and repeating any number of times.

Summary

The most common sequence flow is normal flow. Exception flow happens outside the course of the normal flow. When flow transfers from one link event to another, it is considered to be link flow. Ad hoc flow is the undefined flow within an ad hoc subprocess.

We can simplify BPMN Diagrams with intermediate multiple events. These events are combinations of different intermediate events, much like complex gateways combine different gateways.

Background

We presented an introduction to BPMN diagrams in July. Business analysts are often required to document as-is processes and to-be processes. These diagrams help identify the scope of a software project. The diagrams can also help uncover requirements that might be overlooked without diagramming the processes.

The BPMN specification is designed to establish a common language and convention for creating process diagrams. This common convention allows people who are familiar with modeling, but new to a project to avoid learning a new diagramming language on each project or for each client. We have a link to the official version of the spec in our introductory post (we will update that link if and when the spec changes).

Intermediate events are one of the more unfamiliar items in the BPMN language. They also allow for some of the cleanest, unambiguous expressions of steps in business processes.

Multiple Different Intermediate Events

In our previous example of the business process model for a day trader, we had two different intermediate events that could create exception flow from the second “Watch Stock” task. An intermediate rule event was used to prevent excessive losses or lock in a target gain on the stock transaction. An intermediate timer event was used to force the day trader to sell the stock at the end of the day, if he had not already done so.

Simpify With An Intermediate Multiple Event

We can combine the two intermediate events from the previous diagram (the rule intermediate event and the timer intermediate event) into a single multiple intermediate event.

In this bpmn example, we’ve also added the possibility of receiving a message from the boss requesting that we sell the stock. We have replaced three intermediate events (rule, timer, and message) with a single intermediate event (multiple).

Summary

The multiple intermediate event can be used to consolidate more than one intermediate events into a single entity in the diagram.

Drawing business process diagrams can be tricky. Just getting the layout on the page to look good can be tricky. Intermediate link events can be used to clean up diagrams. They can also be used to jump from a specified point in one process to a specific point in another process.

Background

We presented an introduction to BPMN diagrams in July. Business analysts are often required to document as-is processes and to-be processes. These diagrams help identify the scope of a software project. The diagrams can also help uncover requirements that might be overlooked without diagramming the processes.

The BPMN specification is designed to establish a common language and convention for creating process diagrams. This common convention allows people who are familiar with modeling, but new to a project to avoid learning a new diagramming language on each project or for each client. We have a link to the official version of the spec in our introductory post (we will update that link if and when the spec changes).

Intermediate events are one of the more unfamiliar items in the BPMN language. They also allow for some of the cleanest, unambiguous expressions of steps in business processes.

Business Process Model of a Requirements Analyst

One of the many jobs of a requirements analyst is to document requirements. Ideally writing good requirements, but regardless of quality there is a process to follow. First we review a classical flowchart of the requirements documentation writing process.

The requirements analyst begins by writing a requirements document. If she gets the document approved, she will baseline it and be done. If the document is not approved, it may only need editing, or it may need to be started over from scratch. If the document is edited, the approval process is repeated. If the document is approved after editing, it will be baselined. Otherwise, it will either be edited again or the requirements analyst will start over from scratch.

The “go to A” symbol and the “A” symbol work together to jump around on the same page without cluttering the diagram with too many lines (or line crossings). This is a handy tool in classical flowcharting, and BPMN has an analogous symbol.

BPMN Example

In this BPMN example of the same process, we use a link intermediate event to control the jumping around.

The requirements analyst writes a requirements document, then the flow proceeds to a complex gateway. If the document needs to be rewritten, flow returns to the original task. If it only needs to be edited, flow continues to the “Edit Requirements Document” task. If the document has been approved, it will be baselined and then the process will end.

A link intermediate event follows the “Edit Requirements Document” activity in the diagram. There is a single normal flow branch feeding into one of two link events. When control reaches this first half of the pair of link events, it jumps to the second link intermediate event at the top of the diagram. Flow proceeds from this link event to the complex decision gateway.

Summary

Using link intermediate events is just as powerful as using the “go to” symbols in a classical flowchart.

Business process modeling is rarely applied to simplistic processes. Real world business processes often embody complex decision making. Complex decisions imply choices of action. Rule intermediate events, in BPMN, are designed to express these hard ideas with easy to read diagrams.

Background

We presented an introduction to BPMN diagrams in July. Business analysts are often required to document as-is processes and to-be processes. These diagrams help identify the scope of a software project. The diagrams can also help uncover requirements that might be overlooked without diagramming the processes.

The BPMN specification is designed to establish a common language and convention for creating process diagrams. This common convention allows people who are familiar with modeling, but new to a project to avoid learning a new diagramming language on each project or for each client. We have a link to the official version of the spec in our introductory post (we will update that link if and when the spec changes).

Intermediate events are one of the more unfamiliar items in the BPMN language. They also allow for some of the cleanest, unambiguous expressions of steps in business processes.

Business Process Model of a Day Trader

We can use classical flowcharting to describe a process that a day trader (someone who buys and sells stocks during the same day, to attempt to capitalize on short-term market movements) would follow for purchasing a stock that he intends to sell by the end of the day for a quick profit.

The daytrader will inspect a stock, and if a buy signal is received (based on some undefined criteria), he will purchase the stock. If a buy signal is not received, he will continue to inspect the stock until one is received. Once the daytrader has purchased the stock, he enters into another loop of inspecting the stock until one of three conditions occurs. If the stock price drops by 2% or goes up by 5% he will sell the stock. If the daytrader still owns the stock at the end of the trading day he will sell the stock.

BPMN Example

We can use a much simpler BPMN diagram to represent the same process. In this diagram we will use two intermediate rule events and an intermediate timer event.

The day trader starts by watching a stock. There is an intermediate rule event attached to the boundary of the “Watch Stock” activity. Unlike the looping construct in the classical flowchart, repeatedly inspecting the stock, the BPMN activity represents a continuous activity of watching the stock. The intermediate rule event is triggered if the buy signal is received. This creates an exception flow that causes the day trader to buy the stock, and then immediately start watching it again.

The intermediate timer event will trigger an exception flow to the “Sell Stock” task if the day trader is still watching the stock at the end of the trading day.

The intermediate rule event will trigger an exception flow to the same “Sell Stock” task if either the 2% stop loss or the 5% profit price target is reached.

As an aside, we drew the exception flows both going directly into the “Sell Stock” task. We could have drawn an inclusive (OR) gateway showing the flows reconnecting into the same task. This is an optional element, but it should be used consistently within and across diagrams used by the same people.
Summary

Intermediate rule events in BPMN can be used to represent complex decision making in a concise diagram. When writing the rules for an event, keep in mind that if the rule evaluates to true, then the event is triggered and the flow diverts from the normal flow (aka sequence flow) to the exception flow coming out of the event.

Sometimes we can’t undo our actions. Water under the bridge. But we can make it right by doing something else to compensate. BPMN allows us to use intermediate events to compensate for mistakes in the past. A classic example is cancellation of a purchase. Our example is a little more fun.

Background

We presented an introduction to BPMN diagrams in July. Business analysts are often required to document as-is processes and to-be processes. These diagrams help identify the scope of a software project. The diagrams can also help uncover requirements that might be overlooked without diagramming the processes.

The BPMN specification is designed to establish a common language and convention for creating process diagrams. This common convention allows people who are familiar with modeling, but new to a project to avoid learning a new diagramming language on each project or for each client. We have a link to the official version of the spec in our introductory post (we will update that link if and when the spec changes).

Intermediate events are one of the more unfamiliar items in the BPMN language. They also allow for some of the cleanest, unambiguous expressions of steps in business processes.

This Sounds Familiar

In our article on end compensation events, we demonstrated how intermediate compensation events attached to the boundary of an activity can be used to compensate for an activity or set of activites. The intermediate compensation event is triggered by a business process reaching an end compensation event. Here’s the diagram from that article for a quick review.

In the business process modeling example above, when an invalid card is discovered within the subprocess flow, it triggers the compensation of the “Process Credit Card” task, with the “Record Possible CC Fraud” task. Since this end event terminates the flow within the subprocess, the parent process then continues to the “Record Transaction” step.

Intermediate compensation events are handy because they allow us to compensate for transactions that can not (or should not) be undone. We can use cancel events to truly undo steps within transactional subprocesses. Compensation events are used when we don’t want to undo a step, but do want to record both that step and a future activity (task or subprocess) that compensates for the original activity.

[Update 20060918]

The example above shows a process with two misinterpretations of compensation event semantics. A detailed explanation of the mistakes has been published. First, the compensating task above does not really compensate for (undo) the associated activity. Second, there are no additional activities between the task being compensated and the compensation end event.

The following example also suffers from the second of those two mistakes.

[End Update]

In the following example, our morning diner eats a breakfast burritto. This is a step that can not be undone. If our diner gets heartburn, he will take an antacid, which will make him feel better. He finishes up the process by enjoying the day – either because he didn’t get heartburn, or because he did, but taking the antacid compensated for his folly.

Summary

The key point to take from this is that compensation is used when we can’t undo a step in a process, but we can perform another action designed to neutralize the effect of the former step. [Update 20060918: Please review this correction to avoid the mistakes in this article]

Business processes often need to be cancelled. An error condition can cause a process to terminate, or an incoming message can cause a process to be terminated. Error conditions occur within transactional subprocesses and a cancel intermediate event is used to describe any special cancellation steps.

Background

We presented an introduction to BPMN diagrams in July. Business analysts are often required to document as-is processes and to-be processes. These diagrams help identify the scope of a software project. The diagrams can also help uncover requirements that might be overlooked without diagramming the processes.

The BPMN specification is designed to establish a common language and convention for creating process diagrams. This common convention allows people who are familiar with modeling, but new to a project to avoid learning a new diagramming language on each project or for each client. We have a link to the official version of the spec in our introductory post (we will update that link if and when the spec changes).

Intermediate events are one of the more unfamiliar items in the BPMN language. They also allow for some of the cleanest, unambiguous expressions of steps in business processes.

Before Intermediate Error Events

With classical flowcharting describing an interruptable process involves creating a couple decision boxes and a loop. One decision box tests to see if the process is done, cycling back until it is. The other decision box looks for an interruption, so that the flow can be diverted from the normal looping behavior. It would look like the following example.

In this flow, the publisher collects stories, organizes the daily edition and then begins the printing process by printing copies. If the editor calls to interrupt the process, the publisher will stop the presses and recycle the paper that was used. If the editor does not call, the publisher keeps printing until all the copies are complete, and then distributes the paper.

The BPMN Way

With BPMN we can handle the possibility of interruption a little more gracefully. Consider the following business process modeling example of the same process using BPMN.

The editor will again collect stories and organize the daily edition. He will then begin the “Publish Paper” subprocess. When that subprocess is complete, he will distribute copies of the paper – unless the printing was cancelled. If the subprocess is cancelled, the intermediate cancel event diverts flow to the “Recycle Paper” task.

The expanded subprocess is drawn with double lines, indicating that it is a transaction. Transactional subprocesses can be cancelled, and in our example, if an intermediate message is received from the editor during the “Print Copies” task, flow is diverted to stop the presses and cancel the subprocess. The intermediate cancel event in the main diagram shows how the cancel end event in the subprocess is handled. There is another example of this behavior in our article on BPMN cancel end events.

Also note that there are no awkward gateways or looping flows required to represent this process.  The BPMN representation shows that the process is either completed or diverted – without requiring an artificial looping construct.
Summary

The BPMN method provides for a cleaner diagram, as well as better encapsulation. It is clear in the BPMN diagram that the “Publish Paper” subprocess represents a distinct set of activities, where that isn’t as obvious in the classical flow chart. This clarity could be very handy when managing or optimizing a business process – for example, outsourcing of the printing process becomes an obvious topic of discussion when using the BPMN representation.

Business Processes might start out as easy-to-diagram simple processes. Over time, these processes get more complex, as they have to deal with real-world considerations and unanticipated situations. Things can go wrong. Classical flow diagramming gets complex when dealing with errors or exceptions in a process, while BPMN modeling keeps things simple.

Background

We presented an introduction to BPMN diagrams in July. Business analysts are often required to document as-is processes and to-be processes. These diagrams help identify the scope of a software project. The diagrams can also help uncover requirements that might be overlooked without diagramming the processes.

The BPMN specification is designed to establish a common language and convention for creating process diagrams. This common convention allows people who are familiar with modeling, but new to a project to avoid learning a new diagramming language on each project or for each client. We have a link to the official version of the spec in our introductory post (we will update that link if and when the spec changes).

Intermediate events are one of the more unfamiliar items in the BPMN language. They also allow for some of the cleanest, unambiguous expressions of steps in business processes.

Before Intermediate Error Events

Before we look at intermediate error events, we will take a look at how the same situation would be handled with classical flow-charting.

Original Plan

Consider a classical flow chart model of a bank robber’s original plan for robbing a bank.

The bank robber announces that he is robbing the bank, steals the money from the vault, and runs away. The only problem is that the process is too simplistic. If it were this easy, everyone would be robbing banks. The robber needs to improve his plan.  Business process modeling is trivial with simplistic processes, and interesting with interesting processes.
Improved Plan

We can redesign the bank-robbery process to account for contingencies and mistakes. The bank robber realizes that he might accidentally set of an alarm. Further, the robber realizes that it takes time to steal the cash from the vault. If he trips the alarm, he can still get away, if he’s already done stealing the cash. If he’s still inside the vault when the alarm goes off, he knows he will be arrested. And if he trips the alarm before he even gets into the vault, he can still escape, but without any money.

This improved plan is more representative of an actual bank heist. It also looks a lot more like a real-world business process than the original plan. Unfortunately, the classical flow chart diagram of this process is pretty convoluted. We have to create two parallel flows, one for possibly triggering the alarm and one for robbing the bank. Since triggering the alarm may or may not happen, we need to represent it with two entities – a decision entity and a task. We also have to use a combination of three entities to represent that stealing cash from the vault takes time, and if he’s still in the vault when the alarm goes off he will get arrested.

This business process model of a heist is more complex than it needs to be, but as simple as it can be using classical flowcharting techniques.

Using Intermediate Error Events

We can use BPMN intermediate error events to simplify the diagram, while describing the intent of the robber more explicitly. For people who were following BPMN when it was still a draft proposal, the error intermediate event is the updated name for the exception intermediate event. When you see references to ‘exception‘ in older documents, replace them with ‘error‘.

Interpreting the Intermediate Error Events

BPMN intermediate error events can be used in two ways – either as elements in the sequence flow of the process, or as boundary events attached to an activity. We have an example of each in this example.

An intermediate error event in the sequence flow (or normal flow) throws an error. An intermediate error event attached to the boundary of an activity catches an error. If the robber triggers the alarm (an optional path coming out of the gateway), it throws an error. The error may or may not get caught.

An intermediate error event attached to the boundary of an activity (either a task or a subprocess) will catch an error if that activity is being processed when the error is thrown. If the error is thrown after the activity has been completed, then it will not be caught.

Interpreting the BPMN Diagram of a Bank Robbery

Our robber will announce that he is robbing the bank, and by default will begin stealing cash from the vault (although he may not). He may or may not also follow the path of triggering the alarm. To put it another way, there are three possibilities for what happens in the gateway when the robber reaches it (after announcing that he is robbing the bank):

1. The robber starts stealing cash from the vault without triggering the alarm.
2. The robber triggers the alarm but does not start stealing cash from the vault (he panics and runs away).
3. The robber starts both triggers the alarm and starts stealing cash from the vault.

The interesting thing about the parallel processes is that the relative timing of tasks on each thread is not specified. The robber could trigger the alarm before or after stealing the cash, or could trigger the alarm from inside the vault. This is where the intermediate events help us determine what happens.

If the robber triggers the alarm, an error is thrown by the intermediate error event in the sequence flow following the “Trigger Alarm” task. The only activity in the diagram that can catch the error is the “Steal Cash From Vault” task. Technically, the intermediate error event attached to the boundary of the activity catches the error (not the activity itself).

If the robber is currently in the vault when the alarm is triggered and the error is thrown, the boundary intermediate error event with catch the error and divert the flow. The current task (“Steal Cash From Vault”) will be interrupted, and the process will move to the “Get Arrested” task.

If the robber has completed the “Steal Cash From Vault” task when the error is thrown, he will not change his plans – he will still escape with the money.

If the robber triggers the alarm before starting the “Steal Cash From Vault” task, he would presumably elect to not try and steal the cash, but just run away. It isn’t clear (to me) what the BPMN standard would specify in this circumstance – if the error isn’t immediately caught, does it continue to exist until it is caught, or does it dissappear? Using this example, if the alarm is tripped first, and the robber still chooses to steal the cash from the bank, would the boundary intermediate error event catch the error (because it is still out there, waiting) and get the robber arrested? Or would the robber run away with the money?

In the absence of an answer to this question, we could insert a gateway immediately prior to the “Steal Cash From Vault” task, that makes the robber’s choice explicit – if the alarm has been triggered already, he runs away.

Benefits of the BPMN Approach

This business process modeling diagram has several benefits over the classical flowcharting approach.

• Using intermediate error events simplifies the representation of a task that takes a while but might be interrupted.
• The flow of the process (as drawn) is easier to visualize and understand.
• The “get arrested” branch of the process is clearly identified as an error and not just an alternative branch. This makes the desired process easier to distinguish from the contingency paths.

Summary

Using intermediate error events to throw and catch errors greatly simplifies the expression of real-world processes that have contingency plans. And most real-world processes do have backups, error-handling, and other forms of alternative flow.