• Tweet

• Tweet

Automation testing overview

Proper testing has always been a primary concern in every industry when it comes to the quality of a product, and the IT industry is no exception. As developers, we are responsible for the development of high-quality software, building a stable architecture, flexible enough to accommodate future changes, which is a common case in today's business world. In this article, I'll share with you few tips related to unit testing. 

There are two general types of automation tests – integration and unit tests.
I'll be focusing on the latter type, starting with the general idea of writing unit tests, covering the different types of mocking – manual and with the help of a framework. I'll also show you a few common test scenarios along with the common pitfalls, associated with them.

The basics of unit testing

        public class EmployeesModel
        {
            public IEmployeeRepository EmployeeRepo { get; set; }
 
            public List<EmployeeDTO> GetActiveEmployees(bool markAsChecked)
            {
                EmployeeFilter filter = this.GenerateActiveEmployeeFilter();
 
                List<EmployeeDTO> activeEmployees = this.EmployeeRepo.GetEmployees(filter);
 
                if (markAsChecked)
                {
                    activeEmployees.ForEach(x => x.Checked = true);
                }
 
                return activeEmployees;
            }
 
            private EmployeeFilter GenerateActiveEmployeeFilter()
            {
                EmployeeFilter filter = new EmployeeFilter() { 
                                            Status = (int)EmployeeStatus.Active 
                                        };
 
                return filter;
            }
        }

        public class EmployeeRepository : IEmployeeRepository
        {
            public virtual List<EmployeeDTO> GetEmployees(EmployeeFilter filter)
            {
                // something we don't want to test here
 
                return new List<EmployeeDTO>();
            }
        }

It's a very simplified repository, no base classes, no LINQ queries, only the stuff we need. Notice that the class implements the IEmployeeRepository.

Using a unit testing framework

Unit testing frameworks provide us with an environment to write, execute and review unit tests. The one I personally use is MSTest – the default unit testing framework in Visual Studio. When you right-click on your project in VS, you will find an option to create new unit test project. The generated result will be something like this :

        [TestClass]
        public class EmployeesModelTest
        {
            [AssemblyInitialize()]
            public static void AssemblyInit(TestContext context)
            {
                // assembly init
            }
 
            [ClassInitialize()]
            public static void ClassInit(TestContext context)
            {
                // class init
            }
 
            [TestInitialize()]
            public void Initialize()
            {
                // test method init
            }
 
            [TestMethod]
            public void GetActiveEmployees_MarkAsChecked()
            {
                // setup
                // act
                // assert
            }
 
            [TestCleanup()]
            public void Cleanup()
            {
                // test method cleanup
            }
 
            [ClassCleanup()]
            public static void ClassCleanup()
            {
                // class setup
            }
 
            [AssemblyCleanup()]
            public static void AssemblyCleanup()
            {
                // Assembly Cleanup
            }
 
        }

Notice the attributes that these methods have. When you start the test, all these methods will execute consecutively, starting with the first one. The assembly one will execute only once per assembly load. The class related – only once per test class load. The TestInitialize and TestCleanup methods will execute only once per test method, respectively.

Pay attention to the test method in the middle – GetActiveEmployees_MarkAsChecked. The suffix "MarkAsChecked" means that the method will be invoked with the "markAsChecked" parameter set to true. Inside, the test method is usually divided in 3 parts – setup, act and assert, which are responsible for the setup of the environment for the method, the actual mocked invocation, and the check if the proper results were returned, respectively.

Check out the Assert and CollectionAssert classes for more information.

Creating a mock manually using interfaces

You should remember one base thing about unit tests : the idea of a unit test is to specify the behavior  of one module (in this case – a class or a method), independently from any other module. Be aware that we don't want to test any classes down the hierarchy. This, in our case, means that we don't want to test our repository.  Here I’ve used a simplified version of the data mapper repository pattern, in some other case we could have used some other pattern and technique – it doesn't matter. But how can we actually replace the default behavior of the method?

The answer is through Inversion of control and Dependency Injection. Or more specifically – through property injection.

So let's create the following mocked class :

        public class EmployeeRepositoryMock : IEmployeeRepository
        {
            public List<EmployeeDTO> FixedEmployees { get; set; }
 
            public List<EmployeeDTO> GetEmployees(EmployeeFilter filter)
            {
                return this.FixedEmployees;
            }
 
            // more code here
        }

Using that interface we specify the return value of the method in the setup phase, and then check if the returned value was actually what we expected (i.e. the method was called). In case you are working on an unmanaged C++ project, you can use pure virtual functions to achieve the same behavior. Now let’s create the actual test.

        public static class Fixtures
        {
            public static List<EmployeeDTO> fixedEmployees = new List<EmployeeDTO>() { 
                                                           new EmployeeDTO(),
                                                           new EmployeeDTO(),
                                                           new EmployeeDTO() };
        }
 
        [TestClass]
        public class EmployeesModelTest
        {
            private EmployeeRepositoryMock _employeeRepoMock;
            private EmployeesModel _employeesModel;
 
            [TestInitialize()]
            public void Initialize()
            {
                this._employeeRepoMock = new EmployeeRepositoryMock();
                this._employeesModel = new EmployeesModel();
            }
 
            [TestMethod()]
            public void GetActiveEmployees_MarkAsChecked()
            {
                // setup
                this._employeeRepoMock.FixedEmployees = Fixtures.fixedEmployees;
                this._employeesModel.EmployeeRepo = this._employeeRepoMock;
 
                bool isMarkedAsChecked = true;
 
                // act
                List<EmployeeDTO> employees = this._employeesModel.
                                            GetActiveEmployees(isMarkedAsChecked);
 
                // assert
                CollectionAssert.AreEqual(Fixtures.fixedEmployees, employees);
                Assert.IsTrue(employees.TrueForAll(x => x.Checked));
            }
 
        }

In the Initialize method we set up the main objects that we'll need throughout all the tests in the test class. The name of the test method must be descriptive, it represents the intention of the method with the specified parameters. In the Setup part we prepare the test, execute the method we're testing and assert if the returned result is what we actually expect.

Another test method for this test class would have the following name : GetActiveEmployees_DoNotMarkAsChecked. This is the convention I usually use to represent different parameter permutations. Also note that I’m using a standard convention for private members – underscore as a prefix. Another convention I tend to use, is placing “this” or “base” in front of a member or method invocation, depending if it’s a derived member or not.

Creating a mock manually using virtual methods

Creating a mock that implements a given interface is one possible way to mock a class. Another way is to override the required method in a derived class and use it in your test. In other words, use inheritance. In that case, the mocked repository would look like this :

        public class EmployeeRepositoryMock : EmployeeRepository
        {
            public List<EmployeeDTO> FixedEmployees { get; set; }
 
            public override List<EmployeeDTO> GetEmployees(EmployeeFilter filter)
            {
                return this.FixedEmployees;
            }
 
            // more code here
        }

In order for this to work, the repo property in the model should now be of type EmployeeRepository and the GetEmployees method in the original repository should be marked as virtual. Nothing changes in our test.

Creating a mock using a mocking framework

You can always mock all the methods you want manually. Some developers, however, consider this time-consuming, so one alternative to manual mocking is the use of a mocking framework. The one I usually use is Moq. So, let’s directly check how our unit test would look with it :

        [TestClass]
        public class EmployeesModelTest
        {
            private Mock<EmployeeRepository> _employeeRepoMock;
            private EmployeesModel _employeesModel;
 
            [TestInitialize()]
            public void Initialize()
            {
                this._employeeRepoMock = new Mock<EmployeeRepository>();
                this._employeesModel = new EmployeesModel();
            }
 
            [TestMethod()]
            public void GetActiveEmployees_MarkAsChecked()
            {
                // setup
                this._employeeRepoMock.Setup(x => x.GetEmployees(It.IsAny<EmployeeFilter>()))
                                      .Returns(Fixtures.fixedEmployees);
                this._employeesModel.EmployeeRepo = this._employeeRepoMock.Object;
                bool isMarkedAsChecked = true;
 
                // act
                List<EmployeeDTO> employees = this._employeesModel.
                                                GetActiveEmployees(isMarkedAsChecked);
 
                // assert
                CollectionAssert.AreEqual(Fixtures.fixedEmployees, employees);
                Assert.IsTrue(employees.TrueForAll(x => x.Checked));
            }
        }

As you can see, the EmployeeRepositoryMock class we created is no longer needed. The mocking framework has automatically created the necessary class for us. In this case, we assert if the GetActiveEmployees method is called (which is the main responsibility of the method), and if the employees are checked (which is specified in the input parameter). Nothing more, nothing less. Don’t forget that this is a design and documentation technique, we don’t need to “test” more than that.

Note that in order for this to work, the class we are mocking should be overridable, non-sealed, and the concrete method we mock should be marked as virtual. This is due to the fact that the actual mock that Moq creates is basically a subclass of the original class. Its interface is heavily based on lambdas, which are in fact anonymous methods (if you don’t know, the lambda functions are translated by the CLR into delegates, which on their part are translated into simple wrapper classes with 3 methods inside – Invoke, BeginInvoke and EndInvoke). The use of functional programming paradigms gives us great flexibility, but that’s another topic. You can get a concrete instance of the class you have mocked using the Object class method.

Another important feature is that we can explicitly check if a given method/property was called or set using the Verify() method. For example :

this._employeeRepoMock.Verify(x => x.GetEmployees(It.IsAny<EmployeeFilter>()));

This will check if the GetEmployees method was called with any instance of EmployeeFilter passed as a parameter. In our case, the method returns a result and that's why we keep the original assertions, but that's not always the case. Another variant would be to call VerifyAll, which automatically checks if all setups were used.

You can check the whole Moq interface here.

Common questions and issues

How to test private methods ?

Don’t. Only test the public interface (methods) of the class. But if you insist, there is a way. Check the “How to generate a private accessor” article. 

What if I want to mock a method inside the class under test ?

Sometimes we might need to mock a method which is in the same class like the one we are testing. In order to do that, we create the so-called partial mock. This is done in the same way as mocking the repository class, but  we mock the actual class under test instead.

this._employeesModel = new Mock<EmployeesModel>();

My base class methods return null, even if I haven’t mocked them. What’s the problem?

Remember that Moq (and actually almost all mocking frameworks) use derived classes in order to function.  There is a small catch, though. When you mock a given class, every property/method that returns a reference (not a structure), returns null by default if not set up properly. In order to disable this behavior and cause the derived proxy class to actually call the base class members, set the CallBase property of the mock to true.

Can I test a static class?

You can’t. Static classes are not unit testable. There is a workaround, though, which consists of the replacement of the static class with a singleton (which some consider an anti-pattern). When you replace the static class with a singleton, you can implement an interface and use the same technique we used with the manual mock.

Can I mock a non-virtual method with Moq?

  No. But there are some other frameworks which can achieve that (TypeMock isolator, for example).

What's the difference between a stub and a mock ?

Simply said, the general difference between these terms is :

A stub is a fake object, which replaces a given portion of the unit's logic while not defining any expectations. It's just a piece predefined to a certain output.

A mock is also a fake object, but it defines some expectations. For example, expected parameters, call count, verified output.

If you need a more detailed view of the differences between these terms, maybe this article will be a good place to start.

That is from me, I tried to keep the examples simple and focus on the bigger picture in unit testing. Hope you learned something new today. :)

Have any questions or recommendations ? Don't hesitate to leave a comment below ! ;)