Load testing with C # and Nbomber

By Oleg Zarevych, DevOps/Test Automation Engineer at InterLogic

There are many tools for load testing. The leader among them is the Apache Jmeter. This is a good tool but certain features make it not always easy to use.

Namely:

• Jmeter is a UI-based tool. That means that the test is created using UI. Many people can’t be bothered to look into this UI and the test creation process. But to create tests in your favorite IDE would be very cool.
• Tests in Jmeter are in XML format. It means that for multiple people to work on the same test and store it in git can be problematic.

These features of the Jmeter make it a convenient tool for beginners or engineers unfamiliar with programming. But in general, it would be convenient to write tests in any programming language. There are very popular tools such as Locust (Python), Gatling (Scala), k6 (JavaScript).

In the world of .NET it was possible to write tests in MS Visual Studio but only in the paid version. There we got a symbiosis of UI-base approach and Code-base. But since VS 2020, these tests have become deprecated and are no longer supported.

Nbomber

Nbomber is a library that positions itself as protocol-independent and is developed in F #. Tests can be written in both F # and C #. So far I have had experience only with C #, so my examples will be using it.

Nbomber is developing very actively and its API may vary slightly but it is mainly based on the following abstractions:

• Step
• Scenario
• Runner

Step

Step is the foundation of any test. Our test in particular will consist of several Steps.

Step is an action that we want to perform. For example, an HTTP request.

In fact, the idea of ​​being protocol agnostic is implemented through these steps. The steps can be different: HTTP, WebSocket, mongo, etc. There is also a Step class in which you can write anything and it will also run simultaneously.

The data between steps can be transferred. Nbomber contains a mechanism for data transfer in key-value format. You can also do some checks.

I will add an example of HTTP step:

Scenario

And here we have a few steps that can already be organized into a script. The script contains the steps it will follow sequentially and the load strategy.

Load strategy is an approach that determines how many parallel threads there will be in our test and for how long they will run.

There are currently 5 strategies:

RampConstant is a linear increase in the number of threads over a set time. That is, if you set the number of threads to 10 and the time to 10 seconds, a new thread will be created every second.

KeepConstant is a set number of threads that will work for a set time. That is, if you set the number of threads to 10 and the time to 10 seconds, then 10 threads will be created at once and they will run for 10 seconds.

An important point for the strategies above is that the threads will iterate. When a thread finishes, a new one will be created immediately which will run the script from the beginning. And so will happen until the time runs up.

RampPerSec is a linear addition of the number of threads every second to the set number of threads. For example, if you set the number of threads to 10 and the time to 60 seconds, then one new thread will start every second for the first 6 seconds, and from the 7th second 2 threads will start every second, and from the 13th second – 3 threads every second.

InjectPerSec means that the set number of threads will run every second for the set time.

InjectPerSecRandom is the same as in InjectPerSec but the number of new threads will be selected randomly

The important point is that for these strategies, the threads are not looped and finish after all the steps are done

An example of a Scenario is as follows:

Runner

In most cases, a test contains one script but you can create several of them. In this case, they will run simultaneously.

The scripts we want to run need to be passed to Runner. You can also set up launching, reporting, and other configurations.

The Run () method returns statistics on the test completion and you can perform some checks accordingly

How to run tests

There are two ways to run a test with Nbomber:

• as xUnit \ Nunit test: in this case, your load tests are no different from other tests. They are easy to run and structure
• as a console application: in this case you will run a graphical window where you will see the progress of the test. The only inconvenience here is that you will only have one test

I prefer the first approach. Here you can look at my examples but they may be a bit outdated

Summary

I really liked Nbomber. It has convenient API, sufficient functionality, and is actively developing. With C # you can implement any complexity of your flow and not structure the code as you like. I highly recommend trying it. Here is their github and official website.

Oleg Zarevych