How to create and run adversarial tests

🛠

️ Reach out

Currently, adversarial tests are only available for tabular data. Let us know if you'd like it to be available for other data types as well!

Creating the test

If you are new to adversarial tests, we recommend checking out our tutorial on it. However, this how-to guide is self-contained, so feel free to follow it if you prefer.

To create an adversarial test, first, go to the corresponding project page.

🚧

Don't have a project yet?

All tests live inside a project. If you haven’t created a project yet, make sure you create it first. In case you missed it, here is a tutorial about it.

On the project page, you will find the Tests block. That’s where all the tests you create will appear.

On the upper right corner of the Tests block, you will find a Create tests button. Click on Create tests to create a test.

You will be redirected to the test creation page. The first thing you’ll see, at the top of the page is the test category to select from.

🛠

️ Reach out

If you would like to use other testing frameworks, feel free to reach out so that we can accommodate your needs!

To create a confidence test, select Adversarial. After selecting it, the parameters control panel should appear below it.

Understanding the parameters:

  • Target label: the label we will try to make or model output;
  • Feature: the feature that will be perturbed as we try to flip the model’s predictions. This feature will be varied while keeping the other features constant and see if the model changes its prediction to the target label;
  • Sample size: controls the number of tests that will be generated.
    • number of tests: there are two options for this parameter. One possibility is defining an integer, which corresponds to the number of samples that will be drawn from the dataset at random to apply the perturbations. Alternatively, one can select a tag at the bottom of the page, directly specifying which samples will be used;
    • number of tests per row: the number of perturbations for each row selected.
  • Sample from tags: specify exactly the data samples in which will be used for testing.

Once you’ve defined the test parameters, you can click on Create on the right-hand part of the page.

Running the test

After creating a test, on the test panel on the test page, the newly created test is added to the table. Click on Run to run the test.

After the test finished running, you will be able to see information such as the number of passes and the number of failures.

Click on Open to look at the results in detail.

That's it! You've generated and run an adversarial test!


Did this page help you?