Unit Testing

Testing shows the presence of bugs not the absence of bugs – Anonymous

What are Unit Tests?

Unit tests validate that individual components in a system function as designed. This leads to the question “What is a unit?” In object-oriented programming, a unit is typically a class. In functional programming, a unit would be a function.

The Three Steps of a Unit Test

Let’s look at the anatomy of a unit test. There are three steps in every unit test:

  • Arrange.
  • Act.
  • Assert.

In the Arrange step, the object under test is set up. This involves defining variables and setting the state for objects. This may be as simple as calling the constructor for a class whose method you will be testing, or may involve calling the constructor and a number of the class methods to place the object into the required state for the test.

In the Act step, you perform the action under test.

In the Assert step, the result of the action is tested against the expected result.

For example, let’s assume you have created the following class in C# 7.0:

public class AClass
{
    public AClass(String name)
    {
        Name = name;
    }
    public String Name { get; set; }
    public int NameLength => Name.Length;
}

Now let’s test the NameLength get accessor.

// Arrange
AClass a = new AClass("A string");

// Act
int length = a.NameLength;

// Assert
assert(length == 8);

You may write additional tests, such as testing the length of an empty string, and so forth.

Later in this post we will discuss what can and should be unit tested, and when to write unit tests; but first, let’s look at why you should write unit tests.

Why Write Unit Tests

Unit tests provide the following advantages:

  1. You can be certain that the code you have written the tests for functions as required, or at least as the tests say the code should function.
  2. You can be certain that any changes you make to existing code (bug fixes, refactoring, or adding new functionality), does not introduce bugs into the existing code or regress any changes that you previously made.
  3. You are more likely to refactor code that has unit tests, so your code can become simpler, and therefore easier to understand. You can also experiment with different algorithms.
  4. Provided you write simple tests, the tests will show others how to interact with the unit under test. This can reduce the need for sample applications if your deliverable is a library.
  5. The unit tests validate your design. The arrange and act steps in each unit test should be short and easy to understand. If this is not the case, then it is likely that your design does not exhibit high cohesion and loose coupling.

But, you say, I will have to write more code (the tests) and therefore it will take longer to deliver my project. My answer to that is maybe, but consider the following:

  1. Your code will contain fewer bugs. This reduces the time you will spend searching for elusive bugs, thereby possibly reducing the time to delivery. The sooner a bug is found, the lower the cost to fix it.
  2. Once delivered, your project will have to be maintained. Unit tests can greatly reduce the time and cost required to maintain the project because:
    • You have found more bugs during development, or never introduced them in the first place.
    • If you use Test Driven Development (discussed below), then each unit contains only the functionality it requires and nothing more.
    • Because unit tests encourage refactoring, your code is likely to be simpler and easier to understand, and therefore contain fewer bugs.
    • Future changes are validated to not change the functionality of anything that is not intended to change.

Now I know that I repeated some of what I included in the advantages, but I really want to get across the idea that in the long run, and sometimes even in the short run, unit tests save time and money. The cost of maintaining software usually greatly exceeds the cost of developing the software in the first place. Any slight increase in the cost of developing the software is more than paid back during maintenance.

What Code To Unit Test

The simple answer is to unit test as much of your code as you can. Aim for 100% coverage, but realize that you will never get there. Even 20% coverage is better than nothing at all. Over time you will learn from experience what can and should be tested. See the two items in Additional Reading, below, related to test coverage.

Let’s start by looking at what you do not need to test.

What Not to Test

  1. Any prototype that you code to work out how to solve a programming problem.
    I had a colleague who always insisted that programs should be written twice: the first time to figure out how to do it, and the second time to do it right. Prototypes are used to figure out how to make it work. Since you will be throwing away the code, though not necessarily the design, you do not need unit tests.
  2. Any small throwaway project. Just make sure that it gets thrown away!
  3. The visible parts of user interfaces, such as the buttons and other widgets on a window, dialog, or web page. But if you separate the visible portions of the project from the processing of the events generated by the widgets, you can still test the event handling.
  4. Database access, internet access, and any other potentially time consuming actions. The time to test those is during integration or system testing, not during unit testing. You can test the code that calls those accesses by writing mocks or stubs to replace those accesses. Again this requires designing your project for testing.
  5. Simple class constructors. That is, constructors that simply initialize the state of the object being constructed. You don’t need to test these because you will effectively be doing this when you create the objects during other tests.
  6. Simple accessors. That is, accessors that only get or set values in an object. Again, you will almost certainly be testing these by calling them in other tests. If you are not calling them in your tests, then they probably do not need to exist.
  7. Private class and struct methods, and private functions. These methods and functions are there to support methods and functions with greater access (protected, public, internal), so you will be testing the private methods and functions indirectly anyway.

What to Test

The simple answer is: everything else.

  1. Every public, protected, or internal method or function.
  2. If a class constructor contains more than a single path that stores values in the object, then you should write unit tests for each of the other paths. For example, if you validate the parameters passed into the constructor and throw an exception for invalid values, then write tests that call the constructor with invalid parameter values.
  3. If a class has an accessor that does more than just get or set a property in an object, then write unit tests that exercise every path in the accessor.
  4. If the input to a method or function is validated before being used, be sure to test boundary conditions. For example, if a function takes a parameter that is expected to be between 1 and 10, then write tests that input -1, 0, 1, 5, 10, and 11.

When to Run Unit Tests

Unit tests should be run every time you make a change to your code. They should also be run every time your changes are integrated into the code base. This latter run of testing should be automated into your continuous integration system.

The unit tests must execute quickly. If running unit tests adds a second or two to your typical compile and link, then you will likely run the tests. If your compile and link take a few seconds and the tests take a few minutes, then you will not run them very often because they take up too much time. If the tests take an hour to run, you will probably never run them.

Therefore, you must write tests that execute quickly. There are two main ways to accomplish this:

  1. Design your code to be tested.
  2. Create mock objects or stubs for testing to replace any objects that perform functionality that takes time to set up or to run. For example, write a mock to replace an object that does database or internet access. You can and should test with the real objects during integration or system testing. See 1, above.

When to Write Unit Tests

Assuming you want to write unit tests, when do you write them? There are three possible times to write the tests:

  • After you have coded a method or class (code first).
  • Before you code a class (test first).
  • Only when a bug is discovered or reported (bug driven testing).

Code First

In code first testing, you code a small portion of a class or you code a method of a class and then write unit tests to verify that the code you added to the class functions as expected.

You may also write tests for legacy code that you will be modifying. This tends to be much more difficult because the code was not designed and written to be tested.

Advantages

  1. You have code to test.
  2. You can see the code under test, so you can potentially test every path.
  3. You can write tests for any code to ensure that any changes you make do not inadvertently change its functionality. This applies to changing legacy code, and also to refactoring existing code that you wrote.

Disadvantages

  1. By writing the code first, you may write code that was not necessary to satisfy all of the system’s or class’s requirements.

Test First

Test first is also called Test Driven Development or Test Driven Design (TDD). You start by writing a simple test. The test program fails to link because you have not yet written the code you are testing. You then write just enough code to satisfy the one test. Next, you add another test which fails. You modify the code to satisfy both tests. This may result in throwing away the code you wrote to satisfy the first test. When both tests pass, add another test, and so on. After each test passes, you may refactor the code and run the tests again. If they pass, you have refactored the code correctly. This is known as red-green-refactor.

Let’s look at an example. Say you want to write a function that takes the cube of an integer. Let’s call the function Cube. Start by writing a test:

void TestCube2()
{
    int result = Cube(2);
   
    Debug.Assert(result == 8);
}

Now we write the simplest Cube function that passes this test:

int Cube(int num)
{
    return 8;
}

Build this and the test passes. Now add another test.

void TestCube3()
{
    int result = cube(3);

    Debug.Assert(result == 27);
}

This test will fail because Cube returns 8. Now modify Cube to return 8 when given the value 2 and 27 when given the value 3. One such solution is:

int Cube(int num)
{
    return num * num * num;
}

Now both tests will pass. Add more tests such as inputting 0 and negative numbers. If the tests do not pass, then modify Cube as needed. What about input values that cause integer overflow or underflow conditions? Do they behave as you expect or as needed by the requirements?

Advantages

  1. You only write code that satisfies the tests. In that way, if you write tests to cover all required functionality, you will end up with methods that contain the minimum functionality required.
  2. Refactoring simply becomes a step in the coding process. You can end up with better, “tighter” code.

Disadvantages

  1. If you follow the process religiously, you start by writing code that you will be throwing away as you write more tests. This could be seen as wasted effort.
  2. You may get into the habit of doing no up-front design work, and just let TDD dictate the design for you.

Bug Driven Testing

With bug driven testing, you only write tests when you or your users discover a bug in your code. You then write a test that reproduces the bug, and finally modify the code to eliminate the bug. You may also write other tests for the unit, but only for that unit.

Advantages

  1. Because you have few tests, there is less code to build and maintain.
  2. The time required to run the unit tests is very small.

Disadvantages

  1. Your code contains more bugs that you haven’t discovered yet.
  2. The code may contain more than the required functionality.
  3. You are less likely to refactor your code because you cannot be certain that the refactored code does exactly the same thing as the previous code and nothing else.

Additional Reading

Review

This post has discussed what unit tests are, why you should write unit tests, what code you should write unit tests for, and when to run the unit tests. Finally, the post discussed when you should write unit tests.

The next post will investigate how to write unit tests. It will also discuss how to select a unit testing framework.

Advertisements
Tagged with: , ,
Posted in Testing
2 comments on “Unit Testing
  1. […] post, Unit Testing, covered a number of aspects of unit testing. Hopefully that post has convinced you that unit […]

    Like

  2. […] simple tests in the arrange, act, assert format. See Unit Testing for more information on […]

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories
%d bloggers like this: