Thursday, 6 December 2012

File asserts and manipulations with SoapUI and Groovy

Very quick one, re adventures with SoapUI and Groovy.

Have recently been building tests which need to run batch jobs and then assert against their output within the file structure. Below is a simplified implementation of what I've been up to:

//Build the file name to look for

def date = new Date();
fileName = "OutputFile" + date.format('ddMMyyyyHHmmmss') + ".csv";
folderName = testRunner.testCase.testSuite.project.getPropertyValue("sftpPath");
filePath = folderName + fileName;;

//Read in the data
def inputFile = new File (folderName + fileName);

//Create a new list for asserting against
assertList = [];

//Populate with Contents of Text File
addContentstoList = {assertList.add(it)};

//Assert the number of records in the array, plus for the existence of a specific object
assert assertList.size() == 5;
assertCustID = assertList.any {3};
assert assertCustID;

Found this simple and effective. As an added bonus, this creates and deletes directories and builds CSV files from lists:

//Create New Directories
def inputDir = new File("C:/temp/in").mkdir()
def outputDir = new File("C:/temp/out").mkdir()

//Create an Array for Inputs
def inputFile = [

//Create a csv file
def BuildFile = new File("C:/temp/in/team.csv")
inputFile.each {
    def row = [, it.description]
    BuildFile.append row.join(',')
    BuildFile.append '\n'

//Copy to output folder
def sourceFile = new File ("C:/temp/in/team.csv")
def destinationFile = new File ("C:/temp/out/team1.csv")

sourceFile.withInputStream { is -> 
  destinationFile << is 

//Finally delete to tidy up
def delInput = new File("C:/temp/in").deleteDir()
def delOutput = new File("C:/temp/out").deleteDir()

Wednesday, 31 October 2012

I believe in Sprint Zero....

You want to do what?

‘There’s no time.’ That’s what I always hear. ‘There’s a date to hit.’ That’s another good one. When I raise an eyebrow and say ‘there’s always a date to hit’ it’s not just a consultancy mantra. I know a secret about project dates. They move. Slippery little blighters they are. Reality storms in and changes the game. I don’t know anyone who hasn’t experienced this, yet it remains a secret. So, there’s no time to think, let’s build.

Now that we’ve established there is no time for a Sprint Zero, I’ll start by communicating what I believe is a reasonable explanation of what a Timebox or Sprint Zero actually is. I generally go with ‘a protected time for the team decide how to work together, find out why the product needs to exist and a basic idea of what needs to be done in the near future’. If I’m not feeling particularly verbose then I’ll just call it ‘a team based starting point.’

The order in which I say these statements is important (to me) as the success of your endeavour has its basis in first the how (communicate and iteration), the why (the real business value) and finally the what (how do we go forward initially). Note the word ‘initially’, this can, will and probably should change. Funny how when someone says risk they actually mean fear of change and vice versa, that, is probably a whole other topic.

Its all about the community....

The foundation of a Sprint Zero for me is the challenge of beginning to build a team. Some organisations are finally having the hallelujah moment where soft and hard skills are equally valued so I’d like to stick with that happy thought. So you have a team of multi-skilled ‘developers’ (used in the holistic fashion), they are going to be together for a while so they need to rub along in an efficient manner which promotes business value driven, robust development. 

The first thing to consider is buy in. How can we convince the sceptical among us that two weeks of ‘team building’ (as they may see it) is a worthwhile exercise? On an experienced based note the ‘sell’ will be infinitely harder in organisations which are currently transforming to agile methodologies. This is always juxtaposed with previous behaviour, where an organisation will spend months preparing for a waterfall project but two weeks for a Sprint Zero is too much to ask. 

Understanding who needs to buy in is the key. You will need to consider both the team and other stakeholders. Primarily if the team doesn’t understand why it wants to embark on a two journey of practical and conceptual discovery then it will be difficult to explain it to anyone else.

The team should need a Sprint Zero and demand it of their organisation. It is their chance to understand, explore, ask big questions and implement before the project begins in earnest. 

Some teams will want the practical benefits of a Sprint Zero. For example the team do not want to implement their Backlog storage tool during the first/second/third Sprint. The team should not find out that they don’t have database permissions when they begin Sprint One and wait two days for Service Desk to sort it out. Some teams will be sold on the conceptual benefits. How do we interact? What are expectations/promises to each other? How do we define our relationship with the business?

Every team is different, its a blending thing......

Each team is usually somewhere in the middle and desire a mix of the practical and conceptual. I would recommend a 50/50 split, too practical and the team will struggle to engage with each other and the wider stakeholders once the project begins. Too conceptual and the basic building blocks are not there to begin with, leading to a stuttering first Sprint.

Other stakeholders bring their own layer of complexity plus their own agendas. Business or project management stakeholders might see agile development as a means of getting a project started quickly and seek to circumvent the Sprint Zero process. This can be as a result of an agenda or just a fundamental misunderstanding of the methodology in use, either way if your team is bought in to the idea then you have a much better and (more united) case to take forwards. 

Don't fool yourselves........

The main selling point is that, eventually we will need to do most of these things along the way anyway. You will need to invest this time, whether you like it or not. As with most project and/or team based decisions, you can either choose to do this in a controlled fashion when it gives the most value, or in a piecemeal fashion over the course of many Sprints, directly impacting your velocity.

How long?

In conjunction with the buy in will be the decision on how long you will spend on your Sprint Zero. I would recommend no longer than the length of a normal build Sprint. Teams can wallow in a Sprint Zero, especially those whose organisation who is in agile transformation, the desire to know more and more before beginning can be overwhelming. I have seen recent examples of teams in this state for 6 weeks, iterating over their Backlog again and again and again. Iterating while not moving forward is tantamount to procrastination, as no new information exists to change your train of thought. We should ask ourselves as a team, what, to us, is the most valuable content for a Sprint Zero in the context of this project?  

There are three main threads to consider when determining your Sprint Zero content, let’s stick with our How, Why and What theme:


I further split the How into four areas, to be noted that this is not an exhaustive list:

Experience tells me that when a team faced with a certain type of situation displays hesitant behaviour, they are unsure of their role, its boundaries and where primary responsibility lies. Primary responsibility is often neglected, statements like ‘we all own the Product Backlog’ generally translate as ‘no one owns the Product Backlog’. Avoid this trap. Spend time exploring, educating and setting expectations.

Whether you are embarking on a software project or working on a fishing boat (as a random example) communication will always be the greatest challenge you will face. In this context consider how your team will manage the ten percent of the Sprint dedicated to the Product Backlog and its refinement and understanding. An effective strategy here will serve as an enabler for the team to collaborate with each other and the wider business.

The team should consider how it iterates through the work it will commit to. What are the daily, Sprint and release cycles? How does work flow between analysis, programming, testing and release and back again.  Considering how you size and estimate will affect how you iterate. There is no right or wrong here. I choose story points as I consider ideal day’s too literal, therefore lacking subtlety and flexibility.

Different methodologies demand different artefacts. Scrum allows you to scale up the number and range of artefacts that you use whereas other methodologies demand more. Prior to embarking on the journey decide which artefacts you desire and who has primary responsibility for them. A Product and a Sprint Backlog are a given but you may include a Risk Register, Impediments Log and so on.


Without an understanding of why a product needs to exist a team will struggle to engage over a significant period of time. The main reason for building may well be that the organisation is being paid to do so. Is this enough to instil a sense of belief in the team that what they are doing is valuable? Ask the question. Once you have had a high level project briefing with the team and stakeholders ask if they know why and what value the product will provide. Mere money may not be enough.

Trust the team with the commercial value
Along the same lines is letting the team into the inner sanctum of the project. What are the terms under which they will be working? What is the Time vs. Cost vs. Functionality vs. Quality model? Many projects are conceived based on a flawed model to be utilised in an agile project where time, cost and functionality are fixed and quality is the variable. Knowing these terms and being trusted with them is enlightening and empowering for a team.


As with the How, I again split the What into 4 areas:

Building a simple prototype can drive out an early spiky area of development. Have tricky architecture you’ve never worked with before that you need to push data through? Build something that pushes a random number through. Have a User Interface you aren’t too sure about and need feedback on? Brainstorm, mock up and demonstrate to your users.

This is as much commitment based as methodology based. If you want to use Test Driven Development, backed by automated builds, with exploratory User Interface testing and automated API testing, now is the time to consider. This commitment will also feed into your sizing and strategy for iterating over slices of the product. It can and should change over time.   

This is one of my true favourites as the simplicity of it makes me smile. A project (that had no Sprint Zero) I worked on decided to use a particular environment which suited our needs. However after launching headfirst into the build phase we actually found out that it didn’t work. 10 days of heartache and a failed Sprint later, we had to abandon it. A simple Sprint Zero smoke test would have found the fundamental problem and saved an entire Sprint of development.

Product Direction
This aspect is entirely project and context dependent. However, let’s aim for a few things in terms of direction. Firstly let’s have a starting point, a first Story to tackle which displays real business value. Secondly have the first set of stories and features prioritised and the first release nominally planned, primed and ready for change when the real velocity comes out during build. Thirdly, have the bigger future epics in the Backlog and sized, even if it is hundreds of points. Once they are in they become real to the team.

You might need to battle for it....

As with most truly valuable project and team activities a Sprint Zero is worth fighting for and it is a belief that I carry with me to the organisations with which I work. That is not to say my advice is always heeded, sometimes the urge and pressure to build is too much for the team and project to bear.

Truly successful agile teams and organisations see that a Sprint Zero is not a nice to have but essential to the success of the project. Those who receive this time in a protected state and make good use of it give themselves a firm foundation to evolve and change from.

Tuesday, 2 October 2012

The prickly parts of project inception in an agile world....

One of the key themes at many organisations on their agile journey is how to start a project. One of the questions I often hear is how much information is enough to get started? What is required is a flexible method of defining enough information to start your project journey and at times redefine what is required.

I came across a technique in the interestingly titled The Agile Samurai by Jonathan Rasmussen (a man who takes his craft very seriously). The Inception Deck is a toolbox of exercises, designed to tease out information of the project stakeholders and the development team. I've used these in a multitude of different ways, the whole deck to kick off a project in its entirety and selected exercises to tease out information for individuals features. This has proved extremely affective for the latter, where consensus regarding individual features/epics has been harder to build.

So, my first experience was using it for a high level project kick off. I wanted to achieve a real sense of team buy in, as the organisation I was in really struggled to engage the development team with the business. The main reason I was drawn to the Inception Deck was that the exercises are relatively light at heart, or at least my approach to them was. Project kick off can be a painfully serious time and I wanted to get off to a bright and breezy start.

So I gathered the major stakeholders (Business, Sponsor, Product Owner, Project Management, Architecture, Development Team (in no particular order)) into a room and embarked on the Deck. Its probably worth noting here that I had set expectations before hand, including the content and the duration. It is a forgotten art to pay people the courtesy of letting them know what to expect from an agile 'ceremony', it is often assumed that people know why they have been asked into a room.

As we moved through the Deck, the Elevator Pitch stands out as an area which triggered a great deal of conversation. This really helped to expose the commercial reality of the project we were about to embark upon. As a Consultant I am painfully aware of the commercial realities of major projects but often a development team is shielded from that. I think it is important that a team has enough context to know what is at stake, but not so much that they are paralysed with worry for the duration of the project.

Post the execution of the Deck I asked for feedback from those present. The main feedback from the development team was the level of engagement with the sponsors and main stakeholders was unique for the projects they have worked on. In addition we had generated a baseline plan for the project, including a sketch of the solution, plus every stakeholder had a basic level of common knowledge.

Juxtaposed with the high level application of the Deck was applying it to individual features. My main purpose was to 'unblock' thinking regarding a particular feature. The team in question had knowledge and ideas for the feature in question but struggled to express them in a real and meaningful way. As the ScrumMaster I felt that the information was bubbling below the surface but needed teasing out.

Armed with my previous high level usage experience I used a cut down version of the Deck and added a few brainstorming exercises interspersed within, to try and make ideas flow a little better. I tried to focus on the 'Why' for the team as I often find the 'How' and 'What' comes a little easier for a development focussed team. To be honest, as is my retro style, I allowed the discussion to flow, as I was keen to be unrestrictive on the conversation, while still sticking loosely to the agenda. I find that to be a key facilitation challenge, to allow conversation to flow without going too far off topic. Often answers are found at the loose ends of those conversations.

To be honest, I dropped certain bits of the Deck based on the flow of conversation, and drilled into areas which seemed to pique the interest of the team. Again the Elevator Pitch really shone through, which I think asks questions which everyone assumes we know the answers to, but maybe aren't quite so straight forward. While exploring a feature I would timebox to about an hour, as you need to keep the time spent (after all its lots of expensive people in a room) in proportion with the value of the feature itself.

In summary, the things I really love about it:
  • You can spend as much or as little time as you like exploring with it. So many sessions to tease out information progress for far too long, far outweighing the benefit, consider what you are trying to acheive, who needs to be there and for how long.
  • It is a toolkit. As part of your larger toolkit. Your toolkit cannot be large enough. The more tools you have, the more flexible and effective you are.
  • It really talks about the big stuff. Releases, budgets, architecture. Its a fantastic lead in for teams who have traditionally not been involved in such decisions, as is the case for most teams in organisations who are in agile transformation.
  • Its basically a self documenting baseline plan. Bonus. Documenting while doing it.
A few things that I would like to improve on:
  • Teams can get a little self conscious with exercises such as the Product Box. Speak to your audience before, see how open they are.
  • Can be a lead in to a solution too early. If you want to leave your options a little more open then perhaps omit these sections until a later date.
  • I believe the way of working a team commits to is more important than the actual subject matter that they are working on. The Deck is more project focussed than team focussed.
And finally, it is only a tool, it needs good, honest clear and open communication. If employed effectively and the honesty is present, it can form a great start to your Sprint Zero, and as a solid foundation into the project beyond.


Saturday, 15 September 2012

Avoiding the self imposed proxy trap in SoapUI....

Very quick one to get me in the blogging mood.
So, you know when you work with API's across multiple environments using SoapUI. Depending on how your environments are structured you may need to switch proxies on or off.
After quite a while watching my tests fall over through no fault of their own I decided to solve this annoying problem. Use the following as either a setup script or a groovy step, combined with a project level property containing the URL you want to address:
import com.eviware.soapui.settings.*
import com.eviware.soapui.SoapUI
// set Proxy Dependent on Env
def aPI_URL = context.expand( '${#Project#API_URL}' )
if (aPI_URL.contains("dev")){
} else {
Really simple implementation shown here but it has changed the repeatibility of my tests completely. I have used similar implementations to switch within tests, dependant on which service I'm calling. You can use standard else if and switch statements with this approach too, which really changes how you can manage environments.
I find SoapUI is great for implementing really simple but flexible solutions to common environmental challenges, give it a whirl and see how manipulating your high level settings can really make your automation stand on its own two feet!
Right I'm off to see what else I can mess around with in SoapUI land.....

Monday, 25 June 2012

Scrum Master Mind Map

Just remembered I built this mind map a while ago on a Scrum Master contract. There are a few things that I have learn't since and would probably change. Its always interesting to look back though and see how you felt about something at the time.

I guess the main thing that leaps out at me is that I saw it as a leadership role, without the 'servant' bit in front of it. I may well have 'frustrated' other team members with my domineering attitude.

You'll be delighted to know that I haven't changed a bit.......

Testers are creative beings...or should be

One of the most common misconceptions around the world of software development is that testing is a structured, rigorous and linear process with no room for imagination or creativity. Over the last couple of years of my testing journey I've tried to embrace testing problems with free spirit and an open mind. This was mainly inspired by a distinctly uninspiring project where the lack of quality was only matched by the lack of imagination in the testing effort.

As usual it’s the hard projects which teach you the most and certainly taught me that investing a little time in exploring options can certainly pay dividends further on in the project. There is always a massive amount of pressure to get on with things but it is perfectly acceptable (in my view) to expend a bit of effort on looking for imaginative testing angles. With that in mind, I thought I would spend a little time exploring three of the options that I've used to add a little bit more edge to more recent projects....

1. Speak to People You Wouldn't Normally Ask About Testing

How do you normally approach building a test approach? Who are your 'go to guys'? Usually the tester who's been there for 20 years? Check. That dev guy who seems to know the entire configuration and is always involved in the really complicated stuff? Check. The Business Analyst, Product Owner, Project Manager, the list of usual suspects goes on and on. I do advocate speaking to all these people as their opinions are infinitely valuable but do they give you a different angle? 

So my advice is to try someone different. Perhaps not the Security guy (well depends on the type of testing I guess) but here's a few you might want to check with:
  • The Little Guy - this is probably my number one piece of advice. Usually your 'business contact' is a supervisor or a manager; it seems to be a fact of life in a lot of organisations. Again, it’s a valuable viewpoint but what about the little guy, the person who needs to use it day in and day out? As both a tester and a Scrum Master I've found this to be a really useful tool for getting feedback and guidance. Just walking up to the operations floor and asking for an opinion is also quite a powerful show of communication, if you have the disposition to attempt it!
  • The Middle Men - Your resident Change Control department are useful people to know and these guys see all the randomness which occurs within an organisation, have a great overall view and can really help when creating build validation test pack as they can see the danger before it occurs. Also, it never hurts to have the gatekeepers to your release on your side....
  • The Big Cheese - I definitely read somewhere that the bigger 'bears' in an organisation have an entirely different view on life. In my experience this also would seem to be true. If you have a project which has prioritisation issues in terms of test coverage then I would recommend this approach. You will soon learn what is important and what is not.

2. Pick an Aspect of the Functionality and Explore

Not so long ago I discovered (not claiming to be the first, I mean personal discovery) a technique called mind mapping. Generally on my travels I have seen this used a high level tool to build a picture of a project, however when I really want to understand a key requirement or aspect of the functionality I use it as a drill down tool to really understand a key area of the project and how it can be tested.

I generally look for a few things when identifying my target:
  • Confusion - where there is confusion over what something is and how it works the tester can add major value. Build into your approach tests to prove what it is and how it behaves and cascade that information to the relevant parties.
  • Assumptions - speak to the development, project and business team and try and drive out any assumptions that have been made and drill into them. You may get lucky and hear the phrase 'this part of the project is a black box.' Now, if you don't know what a black box is then now is the time to ask questions. Project teams often assume that bits of software that are already built or pre-tested 3rd party software do not need to be tested as part of the new project. I firmly believe that anything built by humans to be used by humans cannot be deemed a black box and, at the risk of sounding smug, I've generally been right.
  • No Subject Matter Expert - So, you're testing something and you ask around and get the feeling that the only guys that know anything about this piece of functionality are long gone. This is your chance to step in and create a lasting piece of analysis which will genuinely fill a major gap in knowledge and become the subject matter expert yourself.

So, when building your mind map for your specific aspect of the functionality a good heuristic generally works well. A personal favourite is FIBLOTS:
  • Frequent: Common application usage.
  • Intensive: i.e. Resource hogging activities.
  • Business Critical: Even if these activities are both rare and not risky
  • Legal: Stuff that will get you sued or not paid.
  • Obvious: Stuff that is likely to earn you bad press
  • Technically Risky: New technologies, old technologies, places where it’s failed before, previously under-tested areas
  • Stakeholder Mandated: Don’t argue with the boss (too much).

There really are lots of others but FIBLOTS has always given me a level of coverage I’m comfortable with. Its personal preference, I’m generally happy not knowing every little thing before I start testing but I know others prefer more rigor.

3. The Joy of Mocking

Last but by no means least, is the art of mocking. Again I'm relatively new to this but in our brave new world of API's and SOAP services this is an essential tool in the armoury of the creative tester.

Ask yourself this question. How many projects have you tested that require a call to multiple services, with a plethora of data dependencies in each? If you are me then the answer is lots. The key is to be able to test the actual functionality, not just wrestle with data combinations. If you are in control of the data being provided (and received) then your testing can be incredibly precise.

So when you are presented with a new project to test some of your creative thoughts might be:
  • Which bits reach out to external/3rd party systems?
  • Which bits need to be ‘Live’ and which bits can I mock?
  • Are there any areas I can effectively stub out? Any values which could affectively be controlled by setting an expected positive (or negative) response?

This spirit of creativity and technical flair is part of what’s really setting apart the good solid testers from the great ones who want to try new things. As the API becomes king, we as testers will be challenged to broaden our technical knowledge, and our creative selves will need to come to the fore.

Tuesday, 22 May 2012

A practical take on Exploratory/Session Based Testing

General Overview and Definition:
As companies look for more cost effective and agile testing methods Exploratory (or Ad Hoc testing) is coming to the fore. I like James Bach’s definition best: ‘Exploratory testing is essentially simultaneous learning, test design and test execution.’ The approach relies on your instinct as a tester and requires a high level view of the functionality.
However, planning an exploratory testing session is key. Planning should be high level and based on the key aspects of the functionality and your testing toolset.

Planning - A Practical Example:
A Software Development House needs to release a piece of functionality to a client. The client insists that it is tested by the end of the day, bugs notwithstanding.
This gives you, the tester, and one working day to test it. For the example this is 6 hours of test time, with half an hour for planning, and a further hour to tidy up and ask questions/raise defects.
The functionality is available for test and has passed initial deployment checks.
The piece of functionality regards creating an online account with an e-Commerce site has 7 requirements, split over three functional areas. This document assumes you have one or the other!
It is recommended that you add a priority and a size to each requirement or functional area. The priority may or may not be provided but you can apply your own common sense to the issue:

Functional Area
Account Details
User can add a Username

User can set a Password

User can set Address Details

User can add a Nickname
Add Payment Methods
User can add a Credit Card payment method

User can add an e-Wallet payment method
Post Account Creation
User is logged in after creating an Account

The key is the priority and the size; we can use this to work out a strategy for allocating time. This is done subjectively:

Percentage of TimeAllocated
Time Allocated in Hours
User can add a Username
User can set a Password
User can add a Credit Card payment method
User is logged in after creating an Account
User can set Address Details
User can add an e-Wallet payment method
User can add a Nickname

You now have a high level plan which communicates the priority order of testing and how much time this will take. Communicate this to the stakeholders (development team, project team etc) and you are ready to get started.

Exploratory Testing – Executing the Plan:
So we have decided our plan and testing can begin.
The first requirement is ‘User can add a Username.’ There is an hour allocated for this.
Obviously all the requirements for Account Details are linked (you may not be able to submit an Account Creation Request without some of the other items but our focus for the session is on the username. ‘Stub’ out the other fields with values you think should work.
Structure as follows:
User Interface – How is the UI presented? Is there any validation? Any help prompts? Can I interrogate the CSS,JavaScript, and Flash (if web based)?
Middleware – How does my username get to the DB? Is it a messaging system? Web service? Does it validate on its way through?
Database/Storage – Where/what is the DB? What are primary keys? Does my username need to unique? How do the database rules translate to the UI?
As you generate and execute these high level test ideas, note a sentence that describes what you have tested (try in a text file it easily communicable and you don’t need to rewrite them afterwards). Note any defects/questions which occur under the Test Idea. Do this incrementally as they occur, rather than storing them up.
Leave yourself sometime at the end to communicate your coverage of each requirement outwards, clarify any questions,and decide which of your potential defects need to be logged.

Basic Definition

Key Aspects of Exploratory Testing

Exploratory vs Experience Based Testing

Testing without a Map – when there are no requirements!