- •Contents
- •Preface
- •Acknowledgments
- •About the author
- •About the cover illustration
- •Higher product quality
- •Less rework
- •Better work alignment
- •Deriving scope from goals
- •Specifying collaboratively
- •Remember
- •Illustrating using examples
- •Validating frequently
- •Evolving a documentation system
- •A practical example
- •Business goal
- •An example of a good business goal
- •Scope
- •User stories for a basic loyalty system
- •Key Examples
- •Key examples: Free delivery
- •Free delivery
- •Examples
- •Living documentation
- •Remember
- •Tests can be good documentation
- •Remember
- •How to begin changing the process
- •Focus on improving quality
- •Start with functional test automation
- •When: Testers own test automation
- •Use test-driven development as a stepping stone
- •When: Developers have a good understanding of TDD
- •How to begin changing the team culture
- •Avoid “agile” terminology
- •When: Working in an environment that’s resistant to change
- •Ensure you have management support
- •Don’t make test automation the end goal
- •Don’t focus on a tool
- •Keep one person on legacy scripts during migration
- •When: Introducing functional automation to legacy systems
- •Track who is running—and not running—automated checks
- •When: Developers are reluctant to participate
- •Global talent management team at ultimate software
- •Sky Network services
- •Dealing with sign-off and traceability
- •Get sign-off on exported living documentation
- •When: Signing off iteration by iteration
- •When: Signing off longer milestones
- •Get sign-off on “slimmed down use cases”
- •When: Regulatory sign-off requires details
- •Introduce use case realizations
- •When: All details are required for sign-off
- •Warning signs
- •Watch out for tests that change frequently
- •Watch out for boomerangs
- •Watch out for organizational misalignment
- •Watch out for just-in-case code
- •Watch out for shotgun surgery
- •Remember
- •Building the right scope
- •Understand the “why” and “who”
- •Understand where the value is coming from
- •Understand what outputs the business users expect
- •Have developers provide the “I want” part of user stories
- •When: Business users trust the development team
- •Collaborating on scope without high-level control
- •Ask how something would be useful
- •Ask for an alternative solution
- •Make sure teams deliver complete features
- •When: Large multisite projects
- •Further information
- •Remember
- •Why do we need to collaborate on specifications?
- •The most popular collaborative models
- •Try big, all-team workshops
- •Try smaller workshops (“Three Amigos”)
- •Pair-writing
- •When: Mature products
- •Have developers frequently review tests before an iteration
- •When: Analysts writing tests
- •Try informal conversations
- •When: Business stakeholders are readily available
- •Preparing for collaboration
- •Hold introductory meetings
- •When: Project has many stakeholders
- •Involve stakeholders
- •Undertake detailed preparation and review up front
- •When: Remote Stakeholders
- •Prepare only initial examples
- •Don’t hinder discussion by overpreparing
- •Choosing a collaboration model
- •Remember
- •Illustrating using examples: an example
- •Examples should be precise
- •Don’t have yes/no answers in your examples
- •Avoid using abstract classes of equivalence
- •Ask for an alternative way to check the functionality
- •When: Complex/legacy infrastructures
- •Examples should be realistic
- •Avoid making up your own data
- •When: Data-driven projects
- •Get basic examples directly from customers
- •When: Working with enterprise customers
- •Examples should be easy to understand
- •Avoid the temptation to explore every combinatorial possibility
- •Look for implied concepts
- •Illustrating nonfunctional requirements
- •Get precise performance requirements
- •When: Performance is a key feature
- •Try the QUPER model
- •When: Sliding scale requirements
- •Use a checklist for discussions
- •When: Cross-cutting concerns
- •Build a reference example
- •When: Requirements are impossible to quantify
- •Remember
- •Free delivery
- •Examples should be precise and testable
- •When: Working on a legacy system
- •Don’t get trapped in user interface details
- •When: Web projects
- •Use a descriptive title and explain the goal using a short paragraph
- •Show and keep quiet
- •Don’t overspecify examples
- •Start with basic examples; then expand through exploring
- •When: Describing rules with many parameter combinations
- •In order to: Make the test easier to understand
- •When: Dealing with complex dependencies/referential integrity
- •Apply defaults in the automation layer
- •Don’t always rely on defaults
- •When: Working with objects with many attributes
- •Remember
- •Is automation required at all?
- •Starting with automation
- •When: Working on a legacy system
- •Plan for automation upfront
- •Don’t postpone or delegate automation
- •Avoid automating existing manual test scripts
- •Gain trust with user interface tests
- •Don’t treat automation code as second-grade code
- •Describe validation processes in the automation layer
- •Don’t replicate business logic in the test automation layer
- •Automate along system boundaries
- •When: Complex integrations
- •Don’t check business logic through the user interface
- •Automate below the skin of the application
- •Automating user interfaces
- •Specify user interface functionality at a higher level of abstraction
- •When: User interface contains complex logic
- •Avoid recorded UI tests
- •Set up context in a database
- •Test data management
- •Avoid using prepopulated data
- •When: Specifying logic that’s not data driven
- •Try using prepopulated reference data
- •When: Data-driven systems
- •Pull prototypes from the database
- •When: Legacy data-driven systems
- •Remember
- •Reducing unreliability
- •When: Working on a system with bad automated test support
- •Identify unstable tests using CI test history
- •Set up a dedicated continuous validation environment
- •Employ fully automated deployment
- •Create simpler test doubles for external systems
- •When: Working with external reference data sources
- •Selectively isolate external systems
- •When: External systems participate in work
- •Try multistage validation
- •When: Large/multisite groups
- •Execute tests in transactions
- •Run quick checks for reference data
- •When: Data-driven systems
- •Wait for events, not for elapsed time
- •Make asynchronous processing optional
- •Getting feedback faster
- •Introduce business time
- •When: Working with temporal constraints
- •Break long test packs into smaller modules
- •Avoid using in-memory databases for testing
- •When: Data-driven systems
- •Separate quick and slow tests
- •When: A small number of tests take most of the time to execute
- •Keep overnight packs stable
- •When: Slow tests run only overnight
- •Create a current iteration pack
- •Parallelize test runs
- •When: You can get more than one test Environment
- •Try disabling less risky tests
- •When: Test feedback is very slow
- •Managing failing tests
- •Create a known regression failures pack
- •Automatically check which tests are turned off
- •When: Failing tests are disabled, not moved to a separate pack
- •Remember
- •Living documentation should be easy to understand
- •Look for higher-level concepts
- •Avoid using technical automation concepts in tests
- •When: Stakeholders aren’t technical
- •Living documentation should be consistent
- •When: Web projects
- •Document your building blocks
- •Living documentation should be organized for easy access
- •Organize current work by stories
- •Reorganize stories by functional areas
- •Organize along UI navigation routes
- •When: Documenting user interfaces
- •Organize along business processes
- •When: End-to-end use case traceability required
- •Listen to your living documentation
- •Remember
- •Starting to change the process
- •Optimizing the process
- •The current process
- •The result
- •Key lessons
- •Changing the process
- •The current process
- •Key lessons
- •Changing the process
- •Optimizing the process
- •Living documentation as competitive advantage
- •Key lessons
- •Changing the process
- •Improving collaboration
- •The result
- •Key lessons
- •Changing the process
- •Living documentation
- •Current process
- •Key lessons
- •Changing the process
- •Current process
- •Key lessons
- •Collaboration requires preparation
- •There are many different ways to collaborate
- •Looking at the end goal as business process documentation is a useful model
- •Long-term value comes from living documentation
- •Index
186 |
Speciication by |
Example |
|
|
|
|
|
|
|
|
|
|
|
Speciications described |
in a technical language are |
inefective as a communica- |
|||||||||
|
tion tool. If business |
users |
care |
about an |
executable |
speciication, |
then |
it should |
||||
|
be described in a language |
they |
understand. If |
the |
business |
users |
don’t care |
|||||
|
about a |
speciication, |
it should |
be |
captured |
with |
a |
technical |
test |
tool. |
||
A living documentation system that contains technical automation concepts, such as the command to wait until a given time for a process to complete, is a signal for a team to revisit the design of the underlying software system. The need to use technical concepts in living documentation often points to problems with system design, such as reliability of asynchronous processes (see “Listen to your living documentation” at the end of this chapter).
Using technical language is acceptable only when the stakeholders are technical as well and can understand what’s going on in the technical language (such as SQL queries, DOM identiiers, and so on). Note that even if the stakeholders are technical, using such technical language tends to describe how something is tested rather than what the functionality is. Although such tests might initially be quicker to write, they might cause maintenance problems in the long term. See “Scripts aren’t speciications” in chapter 8 for more information.
Mike Vogel used DbFit, a database test script extension for FitNesse that I wrote, to describe acceptance tests in a project with technical stakeholders who could understand scripts. In hindsight, he thinks this was a mistake:
In the beginning they were happy because they could quickly use DbFit and not write custom ixtures, so they had automated tests from day one. Later on, as the complexity of the solution increased, there was no time in the release plan to go back and create test ixtures to make the tests simpler and more understandable and to make the system more testable. We ended up with too brittle, too complicated tests.
Living documentation should be consistent
Living documentation is probably the longest living artifact of a project. Technologies will come and go, code will be replaced with other code, but the living documentation system describes how the business works. We’ll add content to it over several months or years and we need to be able to understand it later. One of the biggest challenges for many teams was keeping the structure and the language of their living documentation consistent. Stuart Taylor explains it nicely:
188 Speciication by Example
For the most part, the steps were very similar, so we found that having a single step ile for the all of the stories that belonged to that one piece of business functionality was really better. Otherwise, even though they were one-liners, we had a lot of duplication.
Evolving |
a |
language helps reduce |
the cost of |
maintaining the |
automation |
layer |
because |
reusing existing phrases |
to describe |
new speciications |
leads to |
con- |
|
sistency |
of |
speciications. |
|
|
|
|
The fact that a living documentation system is automated and connected directly to software ensures that the software model aligns with the business model. Because of that, evolving a language for the living documentation system is a great way to create and maintain the ubiquitous language (as discussed in chapter 8).
Base the speciication language on personas
When: Web projects
Some teams described user stories through personas, especially when developing websites. In those cases, the speciication language can come from the activities that different personas can perform.
Personas can |
help |
simplify executable speciications |
make them |
easier |
to maintain. |
The team at Iowa Student Loan aligned the language used in their speciications with personas. Tim Andersen said:
Instead of users as an amorphous blob, we talked about different people and what their motivations to use the system are, how they use the system, and what they want to get out of it. We put names on different people. Boris was a borrower. Carrie was a co-signer.
Personas helped us because they made us think about how the system needs to behave from the perspective of a user. There are a bunch of positive side effects of using personas that we didn’t anticipate—for example, personas were test helpers [automation components] that were able to interact with our system at a more appropriate entry point.
Chapter 11 Evolving a documentation system |
189 |
Personas don’t make as much sense for projects that have little user interaction. Based on the success of using personas on a previous project, Andersen tried to apply the same concept to a technical data processing system. He eventually gave up and changed the language to a process low model.
Data comes from multiple sources and is loaded into a phone system so people can make phone calls. The phone data is updated, and we send it back to the entities who sent it to us. It is a batch process. Nobody actually uses it; it just runs. Personas weren’t a good it. We tried to get tests deined with personas and we got blank stares from the businesspeople. So I deleted all my persona code, and we changed it to be process based using the Given-When-Then keywords. That made it a lot clearer, and it made more sense to everyone.
Evolving the ubiquitous language around the activities of user personas allows us to ensure that our understanding of what individual personas need is aligned with how they use the system. This drives the structure and language used in speciications and helps us make the documentation system consistent.
Collaborate on deining the language
When: Choosing not to run speciication workshops
If you decide not to |
run |
big workshops |
and |
instead |
use one of the alternati |
approaches, make sure |
to |
collaborate on |
deining |
the |
language. |
Christian Hassa says that collaborating on a language was one of the biggest challenges for his team:
Building a domain language that was consistent and bound well was completely impossible without guidance. Testers wrote things that developers had to rephrase. Sometimes this was because the way testers were writing them down was unclear or not easy to bind [automate]. When the tester had written already a lot of things, we had to rephrase a lot of things. If we tried to bind [automate] the irst example immediately, we would notice that it was not easy to do.
It’s like doing pair programming compared to doing code reviews afterwards. If you do pair programming, the pair will tell you immediately if he thinks you are doing something wrong. If you do reviews, you say: Yes, next time I’ll do it differently, but this time let’s leave it like this.
