Usually, both sides of constants can be generated from some common format, or the HW side have to create a interface header containing the constants needed for the SW. Either way when a constant is changed it needs to tested properly.
But what happens when the HW side changes and the new FPGA code is released? Mayhem, since the HW doesn't have the complete SW source to test against, as their FPGA project could be in a different system, in a different source control etc.
In this case the HW should at least create a simple program that verifies their naming conventions, addresses and masks are the same as the last release. This is IMHO the smallest possible release test of a new FPGA image that can be run before releasing the HW. I'm not stating this is enough to actually test the HW, but it is enough to ensure that the SW does not break due to new names and wrong constants.
For this I have written a small C program. This program uses the MACROS defined by the HW and tests these. Firstly against C compilation, this step is important because changing a name would break compilation on the SW side, and changes like these should not be possible. Secondly, changing masks and address values should also be verified, at least against the previous version.
This will ensure that the HW side will know exactly what they break on the SW side if they change anything. And they'll clearly have a visual indication of what fails, enabling them to report any changes.
/***
name: macro dumper
version: 0.1
author: john
description:
This program is a simple interface test for the defined memory map.
The program dumps the macro's name and value to stdout.
The program purpose is to test all generated macros against their generated
name and value to ensure an interface test of the generated HW memory map
The program will fail to compile any of the macros change their name according
to the first version of the HW interface.
The test program below is supposed to fail in the deliberate fail macro! This
line may be commented in a release version of the simple test program
Remember that new test cases must be added when new HW is introduced
When you're introducing new macros in the hardware interface the following
regular expressions will assist you in getting the macros from the interface
header and into valuable C code that you can paste into the source's test
sections below.
The comment format change for a reason, namely the regular expression characters
contain both * and / which is comment end ;)
***/
// Regular expressions to:-
// List all macros in the interface with all other things stripped:
//
// egrep "#define [a-zA-Z0-9_]+ +" macro_dumper.c | perl -p -e "s/#define +(.+) \((.*)\)/NAME: \$1 VALUE: \$2/g"
//
// Replace all macros with a corresponding test call for inserting into the name test:
//
// egrep "#define [a-zA-Z0-9_]+ +" macro_dumper.c | perl -p -e "s/#define +(.+) \((.*)\) ?.*/TEST_NAME(\$1);/"
//
// Replace all macros with a corresponding test call for inserting into the value test:
//
// egrep "#define [a-zA-Z0-9_]+ +" macro_dumper.c | perl -p -e "s/#define +(.+) \((.*)\) ?.*/TEST_VALUE(\$1,\$2);/"
#include <stdio.h>
/*
This test result construction is meant for the test to produce an output for any other command line tool.
The purpose is that the test_result returns >0 in case of errors and report these to the OS.
A return value of zero indicates a failed test
*/
static int override_return_value = 0;
static int test_result = 0;
static int test_case = 0;
int assign_test_result(int value,int varname)
{
if(value != varname) {
test_result++;
return 0;
}
return 1;
}
#define TEST_NAME(varname) fprintf(stdout, "%s = %x\n", #varname, (varname));
#define TEST_VALUE(varname,value) fprintf(stdout, "%d - %s = %s\n",++test_case, #varname, (assign_test_result(varname,value))?"ok":"error");
/* #include "memorymap.h" */
/* The macros below are ment to be placed in the memorymap file provided by HW, here they are simply a test of the concept */
#define TEST1_MACRO (0x010000000) /* emulating some start address */
#define TEST2_MACRO (0x2000) /* emulating some offset */
#define TEST3_MACRO (TEST1_MACRO + TEST2_MACRO) /* Check to see that the macro names are reported correctly when combined */
#define TEST_MACRO_MASK (0x0010) /* Test for some kind of mask */
#define DELIBERATE_ERROR_MACRO (0xff) /* The check for this macro should not be 0xff as we deliberately want this check to fail */
int main(int argc, char** argv)
{
override_return_value = argc-1;
fprintf(stdout,"%s","This program checks the name and address values of memory map macros\n");
if(override_return_value) {
fprintf(stdout,"%s","Running in override return value mode, errors are only reported not breaking!\n");
}
/*
This is the name test part of the program, any non complying marcos will cause a compilation error
on the exact line of the non existing macro name. The test is valuable for finding non compatible
name changes in the interface
*/
TEST_NAME(TEST1_MACRO);
TEST_NAME(TEST2_MACRO);
TEST_NAME(TEST3_MACRO);
TEST_NAME(TEST_MACRO_MASK);
TEST_NAME(DELIBERATE_ERROR_MACRO);
/*
This part tests against the original macro values to ensure that we know when memory addresses actually change
This test is to ensure that we know when a macro value changes. This is not a test to break the interface
It's simply an indication that an address or mask has changed.
*/
TEST_VALUE(TEST1_MACRO,0x010000000);
TEST_VALUE(TEST2_MACRO,0x2000);
TEST_VALUE(TEST3_MACRO,0x010000000 + 0x2000);
TEST_VALUE(TEST_MACRO_MASK,0x0010);
TEST_VALUE(DELIBERATE_ERROR_MACRO,4); /* This is a deliberate error to test the test macro ;) */
/*
Reporting test case statistics
*/
fprintf(stdout,"Tests run: %d\n",test_case);
fprintf(stdout,"Failures: %d\n",test_result);
return (override_return_value)?test_result:0;
}
The program is compiled using gcc or equivalent by issuing:
john@BlackWidow gcc macro_dumper.c -o macro_dumper
In your terminal. You run the program by issuing:
john@BlackWidow ~/john/src/c $ ./macro_dumper
This program checks the name and address values of memory map macros
TEST1_MACRO = 10000000
TEST2_MACRO = 2000
TEST3_MACRO = 10002000
TEST_MACRO_MASK = 10
DELIBERATE_ERROR_MACRO = ff
1 - TEST1_MACRO = ok
2 - TEST2_MACRO = ok
3 - TEST3_MACRO = ok
4 - TEST_MACRO_MASK = ok
5 - DELIBERATE_ERROR_MACRO = error
Tests run: 5
Failures: 1
john@BlackWidow ~/john/src/c $ echo $?
0
Above you can see the output from the program. Keep in mind that the program should be compiled at every HW release and run to check against the interface.
The program is constructed in such a way that giving 1 or more arguments of any type will cause the program to report test case failure to the OS meaning that it could be used to break a Jenkins build bot on failures.
Yes it can be expanded. Yes it can be re factored. Yes there's lots of potential for improvement. But for now it's the smallest possible thing that will safe many hours of non working interfaces if something went wrong.
No comments:
Post a Comment