CS23 : Focus on Testing

Testing as an exhaustive method of understanding the operation of algorithms and of testing methodologies. In this section we look at what testing is and the methods which can be used to implement it.

We are learning ...
  • About testing strategies
So that we can ...
  • Differentiate between syntax (grammar), runtime (trappable) and semantic (logic) errors in code
  • Appreciate that testing is not a summative operation but occurs at all stages of software development
    - iterative (during development)
    - summative (at the end)
  • Black box, white box testing
  • Select and justify the choice of suitable testing data
    - normal (typical)
    - boundary (right on the edge)
    - erroneous (just plain wrong)
  • Design suitable testing plans (including testing table layout)
  • Generate testing data using random number generators
  • Use trace tables / dry run testing as a technique to isolate semantic errors

CGP The Revision GuidePage 59, 60
CGP Exam Practice WorkbookPage 68, 69

# Get Ready.png

Activity 1
Errors in code 

Click to enlarge

In general, syntax errors are avoidable if you type your code correctly! There really is no excuse. Prevention of runtime errors usually requires the use of validation or verification techniques which we've already met. Semantic errors (logic errors), are hardest to spot and can only really be picked up by tracing operation of the code manually.

Task 1.1 Syntax Errors
Where we learn how to spot and correct syntax errors

In a script

A student has been asked to create a script which asks the user for their name and creates a lovely message by picking a random greeting from a list. The desired output should look like this ...

Look carefully at the following script that the student wrote. Not a bad attempt, but there is
no way that it will run.

Either download yourself a copy and correct it or type it in correctly. The process of correcting the errors is called debugging - you've been doing this all the way through the course so it's not new!

Demonstrate your understanding

Hopefully, you have a set of whiteboards in your learning space. With a whiteboard pen, add a definition and a code example to the whiteboard to help to explain what syntax errors are to our friend Magdalene.

Task 1.2 Runtime Errors
Where we learn that we have already learned how to trap runtime errors!

Yep! It's called 'revision'!

Refresh your memory

You've already covered ways to prevent these in Validate and Verify - refresh your memory.

Write some notes

Write yourself some notes on the five types of validation checks that you learnt. For each validation check ...
  • explain what it does;
  • give an example of where you could use it.

Task 1.3 Semantic Errors
Where we learn that the hardest errors to correct are logic errors

Wow - nice little script to convert temperature from degrees Fahrenheit to degrees Celsius! That's what I wanted!

Hang on! When I run the script, it seems to work but something seems wrong. I know that it *should* give the following results when I test it ...

... but it actually gives me these results ...

In a Python script

Enter the program exactly as written into a new Python script. if you are having trouble (or are running out of time), you can download yourself a copy instead.

In a word processed document

Take a screenshot of the script and paste this into a new word processed document. Remember to include a suitable header and footer in your document (so your teacher knows who you are!)

In the script / in the word processed document

Run the script - it *seems* to work fine, but it doesn't work in the way in which it is intended. There are 6 semantic (logic) errors in this script that prevent it working as a Fahrenheit to Celsius converter ('cause that's what I asked for). Correct the errors. In your word processed document, write about the changes you have made underneath the first screenshot.

In the word processed document

Take a screenshot of the new script and paste this into the word processed document. Print this out for your notes.

Demonstrate your understanding

Again, using your whiteboard pen, add either a definition or an example (preferably code based) of semantic (logic errors) to help to explain what this means to our friend, Magdalene.

Mix and match exercise - Python errors and scenarios

Activity 2
Practical testing 

Click to enlarge

Testing can (and should) happen at any point during software development - after writing your first line right through to testing your complete program with your end users.

Task 2.1 Testing strategies
Where we learn that there are four different approaches we can take to software testing

There are four standard testing strategies ...
  • Unit testing
  • Integration testing
  • System testing
  • Acceptance testing

Watch the animation

Watch the animation carefully and read out the definitions to your shoulder partner.

This task is meant to challenge your resilience ... and you'll need some!

Complete the worksheet

Download the worksheet Testing strategies and complete the definitions in the table by hand. If you are struggling to copy them down, get your shoulder partner to read out the definitions from the animation.

Reading from the World Wide Web

There are some really useful hints on testing on the aptly named Software Testing Fundamentals website. Visit the page and read about the four different software testing methodologies. You have to click on the links in the table.

Demonstrate your understanding

Mackenzie is unsure of which testing strategy to use in the following circumstances. Write your suggestions in full sentences in your notebooks / on paper so that you can show him (or your teacher).
  1. One of Mackenzie's clients want a copy of the software to test herself.
  2. Mackenzie's co-worker has written a subroutine which uses the output from one written by Mackenzie.
  3. Before the software is shipped out to the client, Mackenzie's boss wants to see it all working.
  4. Mackenzie has written a single subroutine that he wants test before he carries on developing.

Task 2.2 Test data
Where we learn about the three categories of testing data

Test data can be carefully chosen to stress subroutines that we have written to check how robust they are. Test data falls into three common categories ...
  • Boundary data : Values which are used to check the inner and outer edges of validation ranges.
  • Normal data : Values which lie within the edges of validation ranges.
  • Erroneous data : Values which should be rejected outright, for instance, outside range or wrong data types.
More often than not, testing data is numerical in nature. We can use a number line to represent valid ranges. 

Click to engage

Remember that erroneous data can also be values with the wrong data type - for instance, in this case, a letter or symbol would also class as erroneous data because they should be rejected immediately.

In your notebooks / on paper

Copy down the definitions of the three categories of testing data for revision. Learn them (recall, easy!)

On the worksheet

Download and complete the worksheet Ranges.

Demonstrate your understanding

Write a simple Python script (preferably as a subroutine) which will only accept numerical values in the range 6 to 9. Use this script to write an explanation (in your notebooks or on paper), for Mackenzie, of the meaning of boundary, normal and erroneous data.

Task 2.3 White box and Black box Testing
Where we learn that testing can be done with or without knowledge of the underlying code

The terms white box and black box refer to the 'transparency' of the code for the parts of the system you are testing.

  • A white box test involves the tester (normally the programmer) testing the units, combinations of units or the entire system with complete knowledge of the underlying code.

  • A black box test involves the tester (often not a programmer) testing combinations of units or the whole system without any awareness of the underlying code.

In your notebooks / on paper

Write yourself a definition of white box and black box testing. Can you think of any examples?

Activity 3
Designing testing plans

There is a standard way of documenting your testing plan and results. You will need to make sure you know this structure when you are designing testing plans for your project work.

  • Test reference : The number of the test you are carrying out
  • Scope of test : The unit / subroutine that the test is focused on
  • Type (BNE) : Whether the testing data is boundary, normal or erroneous
  • Input values : The actual test values.
  • Expected result (Reference) : What you expect to happen. You could reference further manual work.
  • Actual result (Reference) : What actually happens, normally referenced to a screenshot.
  • Corrective action (Reference) : Whether any corrective action is necessary and whether it was carried out.

Task 3.1 Designing testing plans
Where we practice drawing a simple testing plan table

In your notebooks / on paper

Make a note of the column headings for the standard testing table including the descriptions of each column. You will learn what sort of information you put in each column during later topics.

Assessment Task (Homework)

Look back through all the scripts you have written or used over the course of the past few / many lessons. Produce a formal test plan and testing results for this script. Make sure you test boundary, normal and erroneous data and that you present your testing results in a suitable format (using a table).

Grading rubric

MASTER : You have chosen a suitably complex script and produced a standard testing plan and results.
APPRENTICE You have chosen a simple script and given some evidence that you have tested it.
NOVICE : You have chosen a simple script, run it with some simple test data and given a screenshot.

# Flash cards.png
Click to load key word list to help you make your own flash cards 

Hungry for more?

Epoch Fail
Epoch Fail


Q : What is an 'epoch fail'?
A : This is a play on the words 'epic fail' but, since an epoch is a 'period of time', and the seated programmers code keeps crashing for pre-1970 dates (an epoch), this is funny.

Q : 'Ainsley Grohl'
A : This is not a question - stop it.