CMSC330

Test: routine that checks the operation of your code

Why Test

• Save Time
• Allows for Expansion
• Increases Trust
• Helps Collaboration

Unit test: testing units of a program

int square(int x){
  return x * x;
}
          

We can test in with print statements:

printf("%d squared is %d",2,sqaure(2));
printf("%d squared is %d",0,sqaure(0));
printf("%d squared is %d",-2,sqaure(-2));
          

We can test in with print statements:

printf("%d squared is %d",2,square(2));
printf("%d squared is %d",0,square(0));
printf("%d squared is %d",-2,square(-2));
          

Or in a repl:

> square(2)
4
> square(0)
0
> square(-2)
4
          

Or in a repl:

> square(2)
4
> square(0)
0
> square(-2)
4
          

Or use a Testing Framework (OUnit, JUnit, assert.h)

void test_square(){
  assert(square(2) == 4);
  assert(square(0) == 0);
  assert(square(-2) == 4);
}
          

There are some limitations

• Need to think of things to check
• Writing becomes O(n)
• Edge Cases hard to think of

Enter Property Based Testing

Enter Property Based Testing

Property Based Testing (PBT): testing if code satisfies properties

Property: Something we expect to be true

Property: Something we expect to be true

Some examples

• Squaring a number should not change its parity
• Reversing a list twice should have no effect
• Adding an element to a list, then looking up that element should succeed
• Deleting an element in a list, then looking up that element should fail
• A sorted list should have smallest value at index 0

• Squaring a number should not change its parity

Once we have a property, we need to express our property in terms of code

int square_parity(int x){
  return x%2 == square(x)%2;
}
          

• Squaring a number should not change its parity

Once we have a property, we need to express our property in terms of code

int square_property(int x){
  return x%2 == square(x)%2;
}
          

Then generate random inputs

for(int i=0; i < 1000; i++){
  int x = rand();
  assert(square_property(x));
}
          

Then generate random inputs

for(int i=0; i < 1000; i++){
  int x = rand();
  assert(square_property(x));
}
          

Or use a PBT framework

#include <theft.h>
...
theft_config config = { 
.alloc = generate_int_array,
.prop1 = square_property,
... 
}
theft_run(&config);
          

Limitations on PBT

• Not everything can be a property
• Property may be invalid
• Conversion of property to code can be incorrect
• May just get lucky

Limitations on PBT

• Not everything can be a property
• User input is very hard to make properties for
• Property may be invalid
• Conversion of property to code can be incorrect
• May just get lucky

Limitations on PBT

• Not everything can be a property
• Property may be invalid
• square(x) > x
• Conversion of property to code can be incorrect
• May just get lucky

Limitations on PBT

• Not everything can be a property
• Property may be invalid
• Conversion of property to code can be incorrect
• square(x) <= x
• May just get lucky

Limitations on PBT

• Not everything can be a property
• Property may be invalid
• Conversion of property to code can be incorrect
• May just get lucky
• generator randomly makes small values, no int overflow