Selenium Automated Testing using C# and TeamCity

Introduction

Our QA team have started creating Selenium scripts to automate browser functional testing. These allow us to automatically run through a set of actions on a website and flag up any issues.

Running Scripts Manually

You will need:

To create/open/run a script:

  1. Open Firefox
  2. Open Selenium (should be button on toolbar otherwise ctrl+alt+s or alt, Tools, Selenium IDE)
  3. Follow selenium documentation to record/playback scripts

Automating Scripts via Unit Tests

We can convert the scripts into C# unit tests and run them via TeamCity. This explains how to set this up. You will need:

  • First follow the above to create the test suite you wish to convert and ensure you can run scripts manually
  • Download Selenium Server from http://www.seleniumhq.org/download/ and store the .jar somewhere you can run it from. 
  • Run Selenium Server using Java (put this in a .bat file next to the .jar file and double click it:
    • “C:\Program Files\Java\jdk1.8.0_20\bin\java” -jar selenium-server-standalone-2.43.1.jar
  • Visual Studio 2013
  • Install NUnit from http://www.nunit.org/
  • NUnit Test Adapter (extension for VS 2013)
  • A .NET solution open to add the unit tests into

First prepare the solution for selenium unit tests:

  1. Add a new project (C# Class Library) suggest something like SiteName.SeleniumTests
  2. Right click the new project, Manage Nuget Packages, install the following:
    1. NUnit
    2. Selenium Remote Control (RC)
  3. Create a folder in which to place the unit tests (e.g. grouped like NewUserTests or just Tests)
  4. In Selenium IDE, choose File, Batch convert test cases, C#/Nunit/Remote Control
  5. Select your Tests folder as the destination folder
  6. Select all the test cases you wish to convert (html files) and they should be saved as C# files in the Tests folder
  7. In Visual Studio, show existing files, and Include in project all the generated .cs files
  8. Build (fix any compilation errors)
  9. Open Test > Windows > Test Explorer, this should show your tests (if not, rebuild)
  10. Close all Firefox windows (they can interfere with Selenium Server)
  11. Click Run All. Selenium Server should log the commands and launch Firefox to run the test.

To capture screenshots at the end of each test:

  1. Create a Screenshots folder in your unit test project
  2. In each .cs file, in Teardown method add the following line of code before selenium.Stop():
    1. selenium.CaptureEntirePageScreenshot(System.IO.Path.Combine(System.Environment.CurrentDirectory, @”..\..\Screenshots\” + this.GetType().Name + “.png”), “”);
  3. Run the test, screenshot should be saved into the Screenshots folder with same name as the test class.

Integration with TeamCity

To integrate into TeamCity, set up a build configuration to run the NUnit test with the following build steps:

  1. Compile selenium tests – Msbuild runner, point at Selenium.csproj file
  2. Run selenium tests – Nunit runner, run tests from (path to compiled DLL)

Set the Artifact paths setting on the build settings to point to the Screenshots folder to save the screenshots after each run.

To run via TeamCity the Selenium server must be running. You could start this manually same as above, hopefully this can be made to run automatically as a service but I’m not sure how.

Gotchas

Each test must be independent of any other as they may be run in any order, or perhaps only failed tests might be run. Also the server will restart the browser between each test so they can’t depend on the final state of the previous script (like they can in the IDE).

We’re going to try using New Relic Synthetics which has a feature for test automation – apparently you can upload a Selenium script and it can automatically run as and when, we have yet to try this out though!

4 thoughts on “Selenium Automated Testing using C# and TeamCity

    1. Hi Jan

      I think you can just export as Web Driver instead of Remote Control , and the rest works. Possibly easier as you just need Firefox installed I think, rather than running the selenium server. However in my small testing I found converting to web driver the tests were more fragile and failed quite a lot compared to when running them through the Selenium plugin. With RC it seems a lot more stable.

      Since I wrote this we have discovered Ghost Inspector which lets you record selenium style tests and run them in the cloud, triggered by a build server, it even compares screenshots (1 per test) and lets you know if something changes. I plan to write a post about it sometime..

      Hope that helps
      Andy

  1. Hi there, I did successfully run selenium tests with phantomjs driver as vstest in TeamCity but did notice one pretty bad thing, my next step is octopus promote deploy and it should run only if all previous steps are not failed but in fact it did run always

    1. Hi, we are using Ghost Inspector now (triggered by TeamCity) instead of selenium as you can import/export selenium scripts. It’s much easier to set up and run tests. Will blog about that one day. Not sure on your issue – but you should be able to set a unit test failure to be treated as a Fail on the Selenium step, then it won’t run the subsequent step.

Leave a Reply to Jan Crisologo Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.