VSTS

VSTS for the Lone Wolf Developer – Stakeholders and Testing

Let’s go over the concept of Stakeholders and discuss testing in Visual Studio Team Services (VSTS). This is where a lone wolf might have to call in some help!

Stakeholders

Recall how, in the first post, we mentioned you get 5 free basic users? You also have unlimited free Stakeholders. This is awesome.

Stakeholders can’t access code but they can access the work items, which is really handy for logging bugs. Check this page for more detail on what they can/can’t do.

Testing

There are numerous features related to testing in VSTS.

Test Cases

When you create a user story you can assign tasks and bugs. You can also define test cases.

To create test cases for a user story you MUST first navigate to the Stories section. You can’t do enter test cases via your current iteration. I always find this confusing.

Once you are at the board click the … button in the top right of the user story and select Add Test.

You will be able to quickly enter test cases, just like tasks. But unlike tasks, make sure you open each one and enter the step-by-step instructions.

Remember: you could be assigning these tests to your stakeholders so be explicit and clear.

VSTS tests for a user story
A user story with some test cases
VSTS test case example
Sample test case

Running Test Cases

This is where it gets fun. Wait a second… I’m a dev who said testing is fun?

If you hover over to the right of the test you will get the … for the menu. Click it, then click Run test.

VSTS run test
The menu to run a test

VSTS will try to open a pop-up window. Make sure you have enabled pop-ups. Chrome likes to block it by default.

VSTS test runner.png
The Test Runner, with the test case created earlier

Using the Test Runner

Before you use the Test Runner have a separate window with the application open, ready for testing.

Caveat: some of the advanced features here are only useful for web-based applications. You can still use the test runner for other apps. I’m going to assume you have a web app.

Recall this is the test we created earlier. Go through everything step-by-step. Press the tick if the step has passed. Otherwise press the X.

If it fails, you have a wealth of options available. You can:

  • Enter a comment
  • Take a screenshot and attach it
    • The screenshot tool is powerful. You can take a clip of the area of interest, add arrows, circles/squares to identify areas, add test, blur content and more
  • Record a video and attach it
  • Capture user actions as an image action log
    • This is so awesome. It will record every precise click along with images. In my opinion, this is better than video.
  • Add more comments
  • Attach files
  • You can create a bug. I highly recommend you do this.
    • If you click the test again you can see where it’s left off but you can lose that info if someone restarts the test. Best save that info in a bug
VSTS test runner - completed test
Completed test plan with a screenshot and more attached

The bug report it generates is fantastic. Notice how it automatically captures system information?  You don’t have to ask basic questions such as “what browser did you use? what OS?”. It also links to the user story.

VSTS bug generated by test runner
The bug we created. Note that we have to provide a title in order to save. Look at that glorious detail!

Make sure to check out the attachments. I chose to include a screenshot and an action log

VSTS error files
The files attached to this bug report

The item below is a screenshot I took using the Test Runner.

VSTS error screenshot
The screenshot I created using the Test Runner

Below is a brief video demonstrating the Action Log. This is fantastic

Exploratory Testing

Exploratory Testing is another way of saying “just use the app, try to break it and/or spot anything that doesn’t make sense”. They’re not following a pre-defined test plan.

Caveat: this is only useful for web-based applications.

To do Exploratory Testing VSTS provides a free add-on to Chrome. It works a lot like test runner in that it will allow you to:

  • Automatically record device information
  • Automatically keep an action log
  • Allow you to take pictures, videos, add comments
  • Based on your actions you can create use it to create bugs
    • You can also use it to create tasks and test cases. The former doesn’t work for me – says it can’t find it – and the latter isn’t that useful IMO

This page from VSTS provides more information. Just remember that anyone using it must have a Stakeholder account (at a minimum) and the tool has to be setup to point to your VSTS account.

I really like this tool. It’s great for users as well. I was very happy when Microsoft announced this tool was going to be free.

Load Testing

Load testing is a great concept – simulate lots of users suddenly showing up. Afterwards you can see how quickly your app responded and, using your other monitoring tools, see when / where your app would fails.

Your VSTS account provides you with 20,000 VUMs (virtual user minutes) at no cost. If you were to get 25 virtual users to hit a URL for 2 minutes, that would consume 250 VUMs.

First off, click on the Test menu item at the top and select Load test. You have a few options for generating load tests.

HTTP Archive based test

There are four options but, as a lone wolf, this is likely the easiest. I believe the Visual Studio Test option requires an advanced version of Visual Studio (Enterprise?).

The basic concept is you get your browser to record what you’re doing. Once you’re done it spits out a .HAR file. VSTS can use this .HAR file to do the load testing.

Checkout this article for more information.

Example Results

VSTS load test resultVSTS load test charts


Next: Builds and Releases

Previous: Work Item Management

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s