Selenium Testing - Getting Started with NuGridium

Welcome! This documentation will guide you through the process of setting up the Nugridium Server and Nugridium Agent as well as executing an automated browser test run.

Introduction

The Nugridium server is a browser automation testing server, which you can install on premises and which allows you to easily execute your Selenium WebDriver tests in a parallelized fashion. The web api makes it easy to automate your browser test executions and hook them up to your existing CI-CD pipeline.

Nugridium consists of a server component and one or more agents. The server component runs as a lightweight windows service, while the agent run as command line app. It only takes a couple of minutes from downloading the installers to getting the server and agent(s) set up and executing an automated browser test run.

Nugridium server allows you to upload your Selenium WebDriver test binaries and configure your test environments and test runs. These test run configurations can then be scheduled or triggered from 3rd party apps such as Octopus Deploy or TeamCity. With enough agents it is possible to have all of your browser-specific tests execute fully in parallel every time.

Steps to get started

The following are the steps you need to follow to set up your server and agent components and get started executing tests.

1. Install the Nugridium server
2. Install the Nugridium agent
3. Connect server and agent
4. Create a Nugridium project (for the website to test)
5. Create a Nugridium test environment (with url for website to test)
6. Add the agent to the test environment
7. Upload some Selenium test binaries to the project
8. Create a test run configuration (a grouping of test methods)
9. Execute the test run configuration against the test environment with the available browsers on the agent

View video tutorial on server and agent installation and setup

Server Setup

Follow the instructions below to get the Nugridium Server up and running.

Install the Nugridium Server

Download the latest Nugridium server MSI installer from here.

Open the Server Manager

The Nugridium Server Manager is an administrator-only interface designed for configuring your Nugridium server.

When the MSI installer completes, you should have a fully set up and running server instance. To view and/or edit the server settings you can open the Nugridium Server Manager from your Start Menu.

The server manager is organized into 8 sections: Windows Service, Endpoints, Security, User Store, License, Azure Connectivity, Vmss Connectivity (Azure Virtual Machine Scale Sets) and Email Settings.

Windows Service section

This section allows you to start/stop the server windows service (Nugridium.Server) which hosts the server instance.

Endpoints section

This section allows you to configure the server endpoints.

The server has 2 default endpoints. One port is used by a TcpListener and the other by the server web portal (Web Api and AngularJs app). The web portal can have more than one endpoint configured, however it requires that at least one of them is secure (for server/agent communication). The agents communicate with the server via both of those endpoints using X509 server and client certificates using public key cryptography to secure both channels.

When editing or creating a new Ssl web binding, the x509 certificate chosen to secure the binding's port is the certificate identifying the server instance, which can be selected in the 'Security' section of the server manager.

The TcpListener endpoint is used to securely (using SslStream and TTL) transfer the Selenium test binaries to the agents executing the tests.

Security section

This section allows you to select the server ssl certificate that will identify the server instance.

When a new certificate is selected, any existing Ssl web bindings will be reconfigured to use the selected ssl certificate.

User Store section

This section allows you to choose how your web users are authenticated. With the 'Windows users' option the users on the domain can enter their NT passwords when logging in and a new user will be automatically registered for them. While with the 'Custom users' option the users need to register a new account before logging in. The first user to login will become the administrator.

License section

Both the agent and server installations come with a trial license. If you wish to extend the trial period you can email us at support@nugridium.com.

Azure Connectivity section (NEW)

Enable this feature in order to connect agent machines that are outside of the network on which the server is. With this setting on you can have many machines on different geographical locations running your tests. This connectivity relies on Azure Service Bus and Azure Storage.

You will need to have a Service Bus and a Storage account on your Azure subscription and provide their credentials here. Simply create a (or use an existing one) Service Bus namespace and a Storage account in the Azure Portal (or using PowerShell or any other means).

The Service Bus is used to queue the tests and the storage account is used to facilitate the exchange of test binaries and results artifacts between the agents and the server. You will need to use the same service bus namespace and storage account when setting up Agents that will connect to this server via this Azure feature.

Once this feature is enabled, you will see the corresponding option during the creation of a new test environment (in the web protal). See the image below which shows the 'Cloud Queues' option under the 'Queue Provider Type' section.

Vmss Connectivity section (NEW)

Enable this feature in order to have the ability to spin up to a hundred virtual machines for each of your test runs. With this setting on you will have (on each of your test environments) up to a hundred machines executing your tests. The machines will automatically scale in and out depending on whether you have tests in the queue or not.

This connectivity relies on Azure Virtual Machine Scale Sets, Azure Service Bus and Azure Storage. You will need to provide your Azure subscription information, a client id and shared secret that has the 'Contributor' role (see here for details on how to obtain these values) as well as provide a new or existing service bus namespace and storage account name and keys.

Once this feature is enabled, you will see the corresponding option during the creation of a new test environment (in the web protal). See the image below which shows the 'VM Scale Sets' option selected under the 'Queue Provider Type' section.

When you create a test environment in NuGridium, and choose the 'VM Scale Sets' option, a new Azure vm scale set (vmss) will be created under the subscription you provided. You can specify the max number of vms in that scale set, specify the custom vm to use as the source for the vm scale set (see more details on how to create a source vm for this purpose and how to use it in NuGridium).

Creating a custom VM to use as the source of a vmss

In order to enable the VM Scale Set option, you will need to have at least one custom vm to be used as the source for the vms in the vm scale set. This should be an azure vm that has the NuGridium agent installed, optionally, the Turbo.net vm engine installed.

This vm should have had the sysprep tool with the Generalize option invoked inside of it. It should also have been deallocated and generalized in Azure (see below for powershell command to use for this). Once generalized, you can use the Azure id of this custom vm in the NuGridium Server Manager Vmss Connectivity section.

The following steps are necessary in the creation of a custom vm to be used as the source of a vmss.

• On the Azure Portal create a new (or use existing) Windows Server vm). Remote desktop into this vm.
• Once inside the vm, install the NuGridium agent from here. Once installed, there is no need to configure the agent.
• Optionally, install Turbo.net by going to the turbo.net website. Open a free account. If you choose to install and use Turbo.net you do not need to install any browsers on the vm. You will be able to use a wide variety of different versions of IE, Firefox or Chrome browsers from the Turbo.net Hub. You will also not need to create a new custom vm when you wish to upgrade to a newer locally installed browser version.
• If not using Turbo.net, you need to install at least one browser on which you wish to execute your tests. You can install all three (IE, Firefox and Chrome). If you wish to use a locally installed IE instance, remember to change the necessary settings in that IE instance before generalizing the vm. (For Turbo.net IE instances customizations, there are steps to create customized IE containers in the Agent Manager section of this user guide)
• Generalize the virtual machine by doing the following steps:
  • Open a Command Prompt window as an administrator.
  • Change the directory to %windir%\system32\sysprep, and then run sysprep.exe
  • The System Preparation Tool dialog box appears. Do the following:
    
• Once the vm is stopped (can take few minutes), open a PowerShell command prompt and run the following command:
  Login-AzureRmAccount
Get-AzureRmSubscription -SubscriptionId [placeholder] | Select-AzureRmSubscription
Stop-AzureRmVM -ResourceGroupName "[placeholder]" -Name "[placeholder]"
Set-AzureRmVm -ResourceGroupName "[placeholder]" -name "[placeholder]" -Generalized
  Replace [placeholder] with the corresponding values.
• Your custom vm should now be ready to be used by the vm scale sets. Normally you will only need one vm such as this especially if you're using the Turbo.net containerized browsers feature. However if you're using locally installed browsers on these custom vms then you can create several custom vms with different browser versions.
• Using the NuGridium Agent Manager under the Vmss Connectivity section add this custom vm to the list of available vms to be used during the vmss environment creation.

Email Section (NEW)

You will then be able to choose to send the result emails during test run configuration creation or when you trigger a new test run. See more on the web portal documentation here.

You can customize the html of the email by editing the Razor file located at: C:\ProgramData\nugridium\NuGridium Server\razorTemplate_TestRunEmail.html

You can use tokens inside the email subject field, as well as inside the Razor template. Here is the list of allowed tokens:

Tokens for the test run object:

• @Model.Id
• @Model.ConfigurationName
• @Model.EnvironmentName
• @Model.DurationString
• @Model.Result
• @Model.TotalSucceeded
• @Model.TotalFailed
• @Model.Total
• @Model.TriggeredBy
• @Model.TestRunUrl
• @Model.ConfigurationUrl
• @Model.EnvironmentUrl
• @Model.UrlTested
• @Model.Tests

The test run object contains a list of test objects 'Test'. You can use that field to iterate over all tests executed and retrieve their information (see default template).

Tokens for the test objects:

• @test.Name
• @test.ResultState
• @test.WebPortalUrl
• @test.BrowserType
• @test.BrowserVersion
• @test.StartedTime.GetValueOrDefault().ToLocalTime().ToShortTimeString()
• @test.CompletedTime.GetValueOrDefault().ToLocalTime().ToShortTimeString()
• @test.DurationDisplay

Agent Setup

Follow the instructions below to get the Nugridium Agent up and running.

Install the Nugridium Agent

Download the latest Nugridium agent MSI installer from here.

Agent Manager

The Nugridium Server Agent is an administrator-only interface that helps you configure your Nugridium agent.

When the agent MSI installer completes you can open the Nugridium Agent Manager from your Start Menu.

The agent manager is organized into the following sections:

• Agent Process
• Security
• Server
• Connection
• Browsers
• Turbo.net
• Selenium Drivers
• License

If you have at least one browser installed on the agent machine, you will only have to set up the Security and the Server Connection sections before you can start executing tests on that machine.

If you enable the 'Use Azure to connect with server' feature, you will only need to provide a service bus access key and a storage key.

The Agent can be configured in one of two modes:

• Default (local) mode, which uses Msmq and Web API
• Azure (remote) mode, which uses Azure Service Bus and Azure Storage

Agent Process section

This section allows you to start/stop the agent process.

Security section (non-Azure mode)

This section allows you to enter the certificate thumbprint of the server that this agent will trust. It also displays the client certificate identifying this agent. This x509 client certificate was generated during installation.

This section is only required when the Azure setting is off.

Once you enter the server's thumbprint (that can be found on the server manager's Security section), it will be shown at the bottom of the section and the section will become valid.

The agent's thumbprint is provided to the server by going to the Agent section on the server website and adding/editing the corresponding agent. For more details see the Web Portal guide.

Server Connection section (non-Azure mode)

Once the agent's thumbprint has been entered on the server and the server's thumbprint has been entered into the Security section of the agent manager, we can then enter the hostname and the two ports to establish a connection with the server.

The setup endpoints dialog allows you to enter the host name and the two ports for the server.

To avoid the security error shown above, you must provide the agent's thumbprint to the server by going to the Agent section on the server website. For more details see the Web Portal guide.

Once the server is set up to trust this agent, the connection section will appear as shown below. The agent is now fully connected to the server and both parties established x509 certificate-based trust.

Server Connection section (Azure mode)

When the Azure mode is enabled for this agent, the security section goes away and a new panel appears which will allow you to enter the Azure service bus namespace and Azure storage keys that the agent will use to communicate with the NuGridium server.

Browsers section

The browsers section allows you to manage browser-related configurations. It also allows you turn on/off the virtual browsers powered by Turbo.net.

The 'View browser configurations' option will open the Available Browsers dialog that shows the configured browsers (installed and virtual) and the status of their Selenium drivers. Only browsers that are enabled in this list will be available for testing. If you no longer wish to support a browser you can remove it from this dialog.

The agent comes with a few default virtual browser configurations as shown above. You can remove these configurations and add your own. The virtual browsers are containers from the Turbo.net Hub. You will need to install the Turbo.net engine as well as set up a free account with them.

As new browser versions are released, you will be responsible for adding new browser configurations as well as installing the corresponing drivers for the new browser versions that you wish to use for testing. To install a new browser configuration, click on the 'Add browser configuration' option.

By adding a new browser configuration you're telling the Nugridium agent that you wish to use the configured browser for testing. If you're adding a new Chrome or IE browser configuration that is not supported by your existing Selenium drivers, you will need to install the corresponding Selenium driver (see Selenium Drivers section).

When entering a new browser configuration, you only need to provide the browser type and the browser version (only IE, Firefox and Chrome are currently supported). The Selenium driver and version as well as the container name will self populate. The driver version will be auto selected depending on the configured and installed drivers (see the Selenium Drivers section for details on how to add a driver).

The browsers section also has an option for enabling/disabling the virtual browsers feature. You must first have Turbo.net installed and have your user name and password configured in the Turbo.net section.

In this section you can also change the max number of parallel browsers setting. This number will depend on how many simultaneous browsers your agent machine can handle without running out of memory.

Turbo.net section

The Turbo.net section allows you to configure your user name and password for your Turbo.net account. You must first install the Turbo.net engine by going to the Turbo.net website and creating a free account. With a Turbo.net account you will be able to run any browser that is available on their Turbo.net Hub.

The setup credentials link will open a dialog that will allow you to enter your credentials as well as to create the test runner container that will be used to drive the virtual browsers. Creating this test runner container is as easy as clicking on the Create Container button.

If you wish to run the IE containers you will first need to customize those containers with the required settings that the Selenium IE driver requires. The 'Customize IE containers' option will let you customize the virtual IE browsers you wish to use during testing.

The customize operation will pull a clean container from the Turbo.net Hub of the IE version specified, and once the virtual browser opens (after a quick download), you can proceed to change the necessary settings. Once done, close the browser and click on the 'Finish' button, which will commit and push the newly created container to your Turbo.net account. This customized container will then be reused by all of your agents using that account.

Selenium Drivers section

In this section you can install and configure a new selenium driver to support newer versions of Chrome and IE. (Firefox does not require a Selenium driver).

To add a new driver you need to specify the driver type, version and the range of browser versions that the driver supports. You can find this information on each of the corresponding driver's websites. This is more relevant for the ChromeDriver than for the InternetExplorerDriver. For the IE driver you will only need one version (the latest), since it should support all IE versions up to IE 11 as of this writing.

Once you enter and save the Selenium driver type, version and supported browsers range, you will be able to copy and paste the driver executable file to the corresponding folder by clicking on the folder button that appears for each driver. When you have a 64 bit system with a 64 bit IE version installed, you will need two versions of the latest IE driver: the 32 and 64 bit versions.

As of this writing, the default Selenium drivers that come with the Agent installation are: ChromeDriver 2.21, 2.22 and 2.25, InternetExplorerDriver 2.48.0. The default Chrome drivers support the following Chrome versions: 46-55. The IE driver supports IE versions 6-11. If you wish to support newer versions as they are released, you will need to download the driver executables from the Selenium site and install them via the Add New Driver dialog.

Agent license section

Both the agent and server installations come with a trial license. If you wish to extend the trial period you can email us at support@nugridium.com.