Wednesday, 18 May 2016

Question : Unit testing C code?


Question posted on stackexchange:
http://programmers.stackexchange.com/questions/317326/unit-testing-c-code/318686#318686


I learnt about the check testing framework today that seems good. This far I've scripted tests that uses valgrind so that the tests both display output from the tests and from valgrind. Is there a disadvantage in using many test frameworks like valgrind testing for internal memory management, check testing for unit tests and scripts for integration and regression tests? I didn't begin yet writing tests with check and if you know a good/"better" framework for testing C code then please let me know.
This is how my test looks like so far, a script that tests my command-line program including output from valgrind.
If I already can do this with a script, will I even need the check testing framework?
#!/bin/shecho "-- Testing our implementation of OpenShell --"echo ""echo "- If you have any problem in passing a test read the corresponding"echo "- source file to understand what the test is checking"echo ""printf "********************* PRESS ENTER TO RUN TESTS ... "read _ printf "********************* TEST WILDCARDS by listing all files in your home directory... "read _valgrind ./shell << EOF ls
ls -al *.*EOFprintf "********************* TEST ALGORITHMS ... "read _echo "top -b -n1|head -8|tail -1" | ./shell printf
printf "********************* TEST ALGORITHMS Part II. ... "read _valgrind ./shell << EOF whowho|awk '{print \$4 ; print \$3}'|sort -n|wc -l EOF printf
EOF printf "********************* TEST CHECKENV. ... "read _valgrind ./shell << EOF checkenv EOF printf
checkenvEOFprintf "********************* TEST DONE. YOU SHOULD SEE OUTPUT FROM TEST ABOVE ... "read _
Test Output 
$ ./RUN_TESTS
-- Testing our implementation of OpenShell -- - If you have any problem in passing a test read the corresponding
- source file to understand what the test is checking
********************* PRESS ENTER TO RUN TESTS ... ********************* TEST WILDCARDS by listing all files in your home directory... ==20256== Memcheck, a memory error detector
==20256== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.==20256== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==20256== Command: ./shell
==20256== 'PATH' is set to /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin.stdin is a file or a pipe p[0][0] ls pp[0][1] -al pp[0][2] .git pp[0][3] .gitignore pp[0][4] .idea pp[0][5] CMakeLists.txt pp[0][6] CommandEntry.h pp[0][7] README.md pp[0][8] a.out pp[0][9] code.txt pp[0][10] code.txt.tar.gz pp[0][11] do.h pp[0][12] errors.c pp[0][13] errors.h pp[0][14] foo.txt pp[0][15] git.version pp[0][16] main.c pp[0][17] main.o pp[0][18] openshell-0.16430.tar.gz pp[0][19] openshell.h pp[0][20] username.txt pp[0][21] util.c
-rwxrwxr-x 1 dac dac 8584 maj 2 02:02 a.out
-rw-rw-r-- 1 dac dac 465 apr 30 04:58 CMakeLists.txt
-rw-rw-r-- 1 dac dac 50798 maj 1 15:03 code.txt
-rw-rw-r-- 1 dac dac 13206 maj 1 15:03 code.txt.tar.gz
-rw-rw-r-- 1 dac dac 339 apr 30 05:21 CommandEntry.h
-rw-rw-r-- 1 dac dac 362 apr 30 05:21 do.h
-rw-rw-r-- 1 dac dac 1160 maj 2 10:41 errors.c
-rw-rw-r-- 1 dac dac 372 maj 2 10:42 errors.h
-rw-rw-r-- 1 dac dac 0 apr 27 07:44 foo.txt
-rw-rw-r-- 1 dac dac 244 apr 21 09:09 .gitignore
-rw-rw-r-- 1 dac dac 14 apr 23 19:15 git.version
-rw-rw-r-- 1 dac dac 17407 maj 2 11:15 main.c
-rw-rw-r-- 1 dac dac 89472 maj 2 11:16 main.o
-rw-rw-r-- 1 dac dac 12869 apr 30 11:37 openshell-0.16430.tar.gz
-rw-rw-r-- 1 dac dac 1765 maj 2 11:16 openshell.h
-rw-rw-r-- 1 dac dac 1167 maj 1 12:04 README.md
-rw-rw-r-- 1 dac dac 1976 maj 2 11:16 username.txt
-rw-rw-r-- 1 dac dac 32117 maj 2 09:43 util.c
.git:total 64drwxrwxr-x 8 dac dac 4096 maj 2 11:37 .drwxrwxr-x 5 dac dac 4096 maj 2 11:37 ..drwxrwxr-x 2 dac dac 4096 apr 21 09:09 branches
-rw-rw-r-- 1 dac dac 6 maj 2 10:42 COMMIT_EDITMSG
-rw-rw-r-- 1 dac dac 264 apr 21 09:09 config
-rw-rw-r-- 1 dac dac 73 apr 21 09:09 description
-rw-rw-r-- 1 dac dac 97 maj 1 12:04 FETCH_HEAD
-rw-rw-r-- 1 dac dac 23 apr 21 09:09 HEAD drwxrwxrdrwxrwxr-x 2 dac dac 4096 apr 21 09:09 hooks
-rw-rw-r-- 1 dac dac 1034 maj 2 10:42 index drwxrwxrdrwxrwxr-x 2 dac dac 4096 apr 21 09:09 info drwxrwxrdrwxrwxr-x 3 dac dac 4096 apr 21 09:09 logs drwxrwxrdrwxrwxr-x 235 dac dac 4096 maj 2 10:42 objects
-rw-rw-r-- 1 dac dac 41 maj 1 12:04 ORIG_HEAD
-rw-rw-r-- 1 dac dac 107 apr 21 09:09 packed-refs drwxrwxrdrwxrwxr-x 5 dac dac 4096 apr 21 14:49 refs
.idea:total 84drwxrwxr-x 2 dac dac 4096 maj 2 11:37 .drwxrwxr-x 5 dac dac 4096 maj 2 11:37 ..-rw-rw-r-- 1 dac dac 732 maj 1 22:50 misc.xml
-rw-rw-r-- 1 dac dac 270 maj 1 22:50 modules.xml
-rw-rw-r-- 1 dac dac 1335 maj 1 22:50 openshell.iml
-rw-rw-r-- 1 dac dac 180 apr 21 09:10 vcs.xml
-rw-rw-r-- 1 dac dac 58649 maj 2 11:37 workspace.xml
==20256== ==20256== HEAP SUMMARY:==20256== in use at exit: 140,264 bytes in 275 blocks
==20256== total heap usage: 346 allocs, 71 frees, 191,821 bytes allocated
==20256== ==20256== LEAK SUMMARY:==20256== definitely lost: 242 bytes in 24 blocks
==20256== indirectly lost: 11 bytes in 1 blocks
==20256== possibly lost: 11 bytes in 1 blocks
==20256== still reachable: 140,000 bytes in 249 blocks
==20256== suppressed: 0 bytes in 0 blocks
==20256== Rerun with --leak-check=full to see details of leaked memory
==20256== ==20256== For counts of detected and suppressed errors, rerun with: -v
==20256== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)********************* TEST ALGORITHMS ... 'PATH' is set to /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin.stdin is a file or a pipe {top} {-b} {-n1} {|} {head} {-8} {|} {tail} {-1} {|}p[0][0] top pp[0][1] -b pp[0][2] -n1 pp[1][0] head pp[1][1] -8p[2][0] tail pp[2][1] -1[20261] [20262] 3093 dac 20 0 6241068 1,491g 38248 S 13,3 9,6 244:10.93 java
********************* TEST ALGORITHMS Part II. ... ==20274== Memcheck, a memory error detector
==20274== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.==20274== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info ==20274== Command: ./shell
==20274== 'PATH' is set to /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin.stdin is a file or a pipe {who} {|} {awk} {{print $4 ; print $3}} {|} {sort} {-n} {|} {wc} {-l} {|}p[0][0] who pp[1][0] awk pp[1][1] {print $4 ; print $3}p[2][0] sort pp[2][1] -n pp[3][0] wc pp[3][1] -l
[20276] [20277][20278]2==20274== ==20274== HEAP SUMMARY:==20274== in use at exit: 131,709 bytes in 274 blocks
==20274== total heap usage: 349 allocs, 75 frees, 150,591 bytes allocated
==20274== ==20274== LEAK SUMMARY:==20274== definitely lost: 490 bytes in 46 blocks
==20274== indirectly lost: 0 bytes in 0 blocks
==20274== possibly lost: 0 bytes in 0 blocks
==20274== still reachable: 131,219 bytes in 228 blocks
==20274== suppressed: 0 bytes in 0 blocks
==20274== Rerun with --leak-check=full to see details of leaked memory
==20274== ==20274== For counts of detected and suppressed errors, rerun with: -v
==20274== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)********************* TEST CHECKENV. ... ==20282== Memcheck, a memory error detector
==20282== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.==20282== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info ==20282== Command: ./shell
==20282== 'PATH' is set to /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin.stdin is a file or a pipe [20284][20285]==20282== ==20282== HEAP SUMMARY:==20282== in use at exit: 131,301 bytes in 231 blocks
==20282== total heap usage: 305 allocs, 74 frees, 150,103 bytes allocated
==20282== ==20282== LEAK SUMMARY:==20282== definitely lost: 11 bytes in 2 blocks
==20282== indirectly lost: 0 bytes in 0 blocks
==20282== possibly lost: 99 bytes in 1 blocks
==20282== still reachable: 131,191 bytes in 228 blocks
==20282== suppressed: 0 bytes in 0 blocks
==20282== Rerun with --leak-check=full to see details of leaked memory
==20282== ==20282== For counts of detected and suppressed errors, rerun with: -v
==20282== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)********************* TEST DONE. YOU SHOULD SEE OUTPUT FROM TEST ABOVE ...

Answer:

Your certainly on the right track. For my projects I use:
"check" to unit test all method (including as many code paths as I can - have time for). This runs super fast and gives me confidence that the parts of my application are doing what I expect.
"Valgrind" to check the memory usage of the final application while running system/regression tests. This is really really slow, but give me confidence that my application has the desired functionality and the interactions between the methods that make up the application are not causing any unforeseen memory management issues.
So, my advice - do both. Good luck.

No comments:

Post a Comment