Test Configuration

Control Flags

Control flags are global variables that configure how the test will behave.

The mandatory control flags:

NS_MGMT_REQUIRED

The NS_MGMT_REQUIRED flag specifies whether the test will make of namespace management functionality. If namespace management is required for the test to run successfully, this flag should be set to 1.

Note

If the test requires namespace management but it is not supported by the controller, the test will be skipped.

Example:

NS_MGMT_REQUIRED = 1

PRESERVE_PREEXISTING_NS

The PRESERVE_PREEXISTING_NS flag is used to prevent preexisting namespaces from being removed before starting the test.

0 : Preexisting namespaces will be removed
1 : Preexisting namespaces will not be removed

Warning

If the current namespaces fill the entire drive, this may prevent the test from being run.

Note

If the device does not support namespace management, this flag will have no effect.

Example:

PRESERVE_PREEXISTING_NS = 0

RESTORE_NS_CONFIG

The RESTORE_NS_CONFIG flag is used to control whether the namespace configuration on the drive will be restored to its original state after the test is complete. Any namespaces created during the test will be removed.

Example:

RESTORE_NS_CONFIG = 1

Warning

NSIDs of the restored namespaces cannot be guaranteed as they are automatically assigned by the controller.

Tip

If you would like preexisting namespaces restored, but they don’t need to be preserved, use the following settings:

PRESERVE_PREEXISTING_NS = 0
RESTORE_NS_CONFIG = 1

PORTS

The PORTS flag defines the number of device ports utilized during the test.

The following options are available:

1: Single port test
2: Dual port test

Warning

The number of ports will change the script template slightly.

PORTS = 1

def main(test_config):
    @setup(test_config)
    def test(dut):          # <- dut (this is a single XML_API object)
        .
        .

PORTS = 2

def main(test_config):
    @setup(test_config)
    def test(duts):         # <- duts (this is a list of two XML_API objects)
        .
        .

For further information about duts, see Devices Under Test (DUTs).

PORT_LOCKING

The PORT_LOCKING flag allows users to block other tests from running on the controller. The following are available options:

0: Use no port locking
1: Lock this port
2: Lock other port - For use only with dual port tests
3: Lock both ports - For use only with dual port tests

Example: Lock this port

PORT_LOCKING = 1

ZONETYPE

The ZONETYPE flag must match the namespace type of the device being tested. It specifies a bit mask of the following options:

1 or 0b001 : Conventional
2 or 0b010 : Key/Value
4 or 0b100 : ZNS
7 or 0b111 : All types

Examples:

ZONETYPE = 1
ZONETYPE = 0b011
ZONETYPE = 7

DEBUG

The DEBUG flag allows the user to set which message levels will appear in the test logs. The following options are available:

0 : Error, Warning, Note, Skipped, Action, Detail
1 : Error, Warning, Note, Skipped, Action, Detail, Info
2 : Error, Warning, Note, Skipped, Action, Detail, Info, Debug

Example: Sample information displayed when a IO test is run with DEBUG = 0

DEBUG = 0

>> Thu Sep 02 16:09:10 2021 DETAIL: Starting Test: Read_1_0_100_1_11

Example: Sample information displayed when a IO test is run with DEBUG = 1

DEBUG = 1

>> Thu Sep 02 16:11:58 2021 DETAIL: Starting Test: Read_1_0_100_1_11
>> Thu Sep 02 16:11:58 2021   INFO: OnError=StopTest LBAInPatterns=1 VerifyWritesMode=1 NoPathTimeout=180000 VerifyAtomicWrites=1,0
>> Thu Sep 02 16:11:58 2021   INFO: iotype : Read
>> Thu Sep 02 16:11:58 2021   INFO: threads : 1
>> Thu Sep 02 16:11:58 2021   INFO: blocks : 1
>> Thu Sep 02 16:11:58 2021   INFO: iosperthread : 0
>> Thu Sep 02 16:11:58 2021   INFO: seektype : 0
>> Thu Sep 02 16:11:58 2021   INFO: startpaused : 0
>> Thu Sep 02 16:11:58 2021   INFO: pattern : 0

See Logging for more details on how to output custom log messages.

Informational Variables

VERSION

Version is a required parameter, but it can be any string.

Example:

VERSION = 'V1.0'

Control Files

Test behavior is managed by files in the script’s directory in the REST tree. These files can be controlled by script commands during program execution or before the test via the command line.

Setting Control Files

Using the GUI

On the SBExpress Manager page, many of these can be set prior to starting a test.

Using set_var

Control files can be set during a test with the set_var function.

The command set_var takes two parameters. The first is the control file to be changed, and the second is the value to be written.

Example:

set_var('passtimer', -1)

Using the Command Line

Control files are located in the ref:REST tree, in the same directory as the script.

For example, test 1 on port 0, target 100, lun 1 can be found at the following location on your system:

/rest/sanblazes/1/ports/0/targets/100/luns/1/tests/1

To navigate to this directory on your system, you can use the following CLI command

$ cd /rest/sanblazes/1/ports/0/targets/100/luns/1/tests/1

Once inside this folder, you can view the REST tree files to get information about the latest test run.

$ ls

To read from the REST tree files, you can use the cat command.

For example, to find the length of the last test pass you can use the following command:

$ cat passtimer_estimated

Control files can be changed from the command line before or after program execution. This can be done by issuing the following type of command:

$ echo '-1' > passtimer

Warning

Changing the control flags via the command line while a test is running is possible, but is strongly discouraged. This can cause problematic behavior that is difficult to troubleshoot.

allowederrors

Sets the maximum number of errors allowed before a test will be automatically failed. This can be set to any positive integer. For example, you may want to run a test for many passes and allow testing to continue after a single error.

Example: Set allowederrors to 2

set_var('allowederrors', 2)

allowedwarnings

Sets the maximum number of warnings allowed before a test will be automatically stopped with state ‘Warning’. This can be set to any positive integer. For example, you may want to run a test for many passes and allow testing to continue after a single warning.

Example: Set allowedwarnings to 2

set_var('allowedwarnings', 2)

passes

Controls how many times the test inside the script is run. In this case, test refers to all actions that occur inside the while keep_running() loop. This can be set to any positive integer.

Example: Set passes to 5

set_var('passes', 5)

passtimer

If set to a positive integer (or zero) this determines the minimum duration of each pass in seconds.

Note

When all actions in the pass are complete and the minimum duration is reached, the pass will end regardless of the state of any IO tests. Any outstanding IO tests will be stopped.

If set to -1, the pass will end when all actions in the pass are complete and there are no outstanding IO tests.

Set the passtimer to 60 seconds:

set_var('passtimer', 60)

passtimer_estimated

Sets the value of Sec/Pass in GUI if passtimer is not a positive integer. If the value of passtimer is a positive integer, setting this has no effect.

Example:

set_var('estimated_passtimer', 40)