Difference between revisions of "Test"
(→See also) |
|||
(13 intermediate revisions by the same user not shown) | |||
Line 2: | Line 2: | ||
==Running tests== | ==Running tests== | ||
− | You will need to have TCL installed ( | + | You will need to have [http://tcl.tk/ TCL] installed (free download available from [http://www.activestate.com/store/activetcl/download/ ActiveState]) |
===Under Windows=== | ===Under Windows=== | ||
− | Open a command window (Start, Run: cmd). Navigate to your | + | |
+ | You must build odamex as a console application. This is because if you run odamex from the command window (Start, Run: cmd) and it detaches from the command console, the tests will not work. | ||
+ | |||
+ | Open a command window (Start, Run: cmd). Navigate to your binary directory (the one which contains client and server binaries). Run one of the tests: | ||
<pre> | <pre> | ||
− | cd trunk | + | cd trunk\bin |
− | tests\commands\cmdSay.tcl | + | tclsh85 ..\tests\commands\cmdSay.tcl |
</pre> | </pre> | ||
− | ===Under | + | ===Under Unix=== |
You can run all tests with ''make test'', or a specific test by navigating to the working directory and invoking one of the tcl test scripts from there: | You can run all tests with ''make test'', or a specific test by navigating to the working directory and invoking one of the tcl test scripts from there: | ||
<pre> | <pre> | ||
Line 19: | Line 22: | ||
==Writing tests== | ==Writing tests== | ||
− | Tests are written in TCL, as this is a portable scripting language. It is best to start by copying an existing test. Tests should output lines containing the words "PASS" or "FAIL". | + | Tests are written in [http://tcl.tk TCL] (pronounced ''tickle''), as this is a portable easy to learn scripting language. It is best to start by copying an existing test. Tests should output lines containing the words "PASS" or "FAIL". |
− | Tests can start/stop clients and servers, | + | Tests can start/stop clients and servers, send them commands and monitor the output. Complexity of the test increases with the complexity of the feature, this encourages smaller changes. |
==Example== | ==Example== | ||
Line 30: | Line 33: | ||
exec tclsh "$0" "$@" | exec tclsh "$0" "$@" | ||
− | # Create a list of demos to test | + | # Create a list of demos to test (a list of lists of 3 items) |
lappend demos "DOOM2.WAD DEMO1 {15eb4720 3ccc7a1 3fc7e27 800000}" | lappend demos "DOOM2.WAD DEMO1 {15eb4720 3ccc7a1 3fc7e27 800000}" | ||
lappend demos "DOOM2.WAD DEMO2 {cea29400 289b9c2 fece4356 600000}" | lappend demos "DOOM2.WAD DEMO2 {cea29400 289b9c2 fece4356 600000}" | ||
Line 38: | Line 41: | ||
# Run this demo | # Run this demo | ||
set stdout [exec ./odamex -nosound -novideo \ | set stdout [exec ./odamex -nosound -novideo \ | ||
− | -iwad [lindex $demo | + | -iwad [lindex $demo 0] \ |
+demotest [lindex $demo 1]] | +demotest [lindex $demo 1]] | ||
Line 55: | Line 58: | ||
} | } | ||
</pre> | </pre> | ||
+ | |||
+ | ==Advanced TCL== | ||
+ | All relevant documentation is easily available on the internet; this section briefly talks about things you are likely to encounter | ||
+ | |||
+ | ===Pipes=== | ||
+ | ''exec'' is a synchronous command, it will wait for execution to finish before returning control to your script. This is not good when you want to control both a client and a server. What you need is a pipe: | ||
+ | |||
+ | <pre> | ||
+ | set control [open "|./odamex" "w"] | ||
+ | </pre> | ||
+ | |||
+ | Problem is, pipes are one-way (either read only or write only), so you'd need to redirect the output and read it a different manner (after a short delay to allow the output file to be created): | ||
+ | |||
+ | <pre> | ||
+ | set control [open "|./odamex > odamex.log" "w"] | ||
+ | exec sleep 1 | ||
+ | set output [open "odamex.log" "r"] | ||
+ | </pre> | ||
+ | |||
+ | And you must remember to close both pipes, but before you do, you must ensure that the application has quit (otherwise ''close'' will never complete). To do this, use ''puts'' to write the quit command to the client (as if it were typed). Applications do not like multiple puts without having the stream flushed, so a ''flush'' follows the ''puts''. | ||
+ | |||
+ | <pre> | ||
+ | set control [open "|./odamex > odamex.log" w] | ||
+ | exec sleep 1 | ||
+ | set output [open "odamex.log" r] | ||
+ | |||
+ | puts $control quit | ||
+ | flush $control | ||
+ | |||
+ | close $control | ||
+ | close $output | ||
+ | </pre> | ||
+ | |||
+ | Most of this functionality should be put into separate procs so that it does not hurt your head. A lot of it is already available as functions within existing tests, which you can copy. | ||
==See also== | ==See also== | ||
* [[Coding standard]] | * [[Coding standard]] | ||
+ | * [http://odamex.org/regression.html Latest regression test report] |
Latest revision as of 00:12, 3 June 2008
Odamex has a lot of working features. The doom engine is sensitive to change, so to preserve functionality we have a set of tests to run on every submission. They are located in the tests directory.
Contents
Running tests
You will need to have TCL installed (free download available from ActiveState)
Under Windows
You must build odamex as a console application. This is because if you run odamex from the command window (Start, Run: cmd) and it detaches from the command console, the tests will not work.
Open a command window (Start, Run: cmd). Navigate to your binary directory (the one which contains client and server binaries). Run one of the tests:
cd trunk\bin tclsh85 ..\tests\commands\cmdSay.tcl
Under Unix
You can run all tests with make test, or a specific test by navigating to the working directory and invoking one of the tcl test scripts from there:
cd trunk tests/commands/cmdSay.tcl
Writing tests
Tests are written in TCL (pronounced tickle), as this is a portable easy to learn scripting language. It is best to start by copying an existing test. Tests should output lines containing the words "PASS" or "FAIL".
Tests can start/stop clients and servers, send them commands and monitor the output. Complexity of the test increases with the complexity of the feature, this encourages smaller changes.
Example
#!/bin/bash # Do not change these first three lines \ exec tclsh "$0" "$@" # Create a list of demos to test (a list of lists of 3 items) lappend demos "DOOM2.WAD DEMO1 {15eb4720 3ccc7a1 3fc7e27 800000}" lappend demos "DOOM2.WAD DEMO2 {cea29400 289b9c2 fece4356 600000}" lappend demos "DOOM2.WAD DEMO3 {dca00040 fd6a4b9c ff7bee0a ff000000}" foreach demo $demos { # Run this demo set stdout [exec ./odamex -nosound -novideo \ -iwad [lindex $demo 0] \ +demotest [lindex $demo 1]] # Take the last line of output set result [lindex [split $stdout "\n"] end] # Take the last item in this demo line (see top of test) set expected [lindex $demo 2] # Compare them if { $result != $expected} { puts "FAIL $demo | $result" } else { puts "PASS $demo | $result" } }
Advanced TCL
All relevant documentation is easily available on the internet; this section briefly talks about things you are likely to encounter
Pipes
exec is a synchronous command, it will wait for execution to finish before returning control to your script. This is not good when you want to control both a client and a server. What you need is a pipe:
set control [open "|./odamex" "w"]
Problem is, pipes are one-way (either read only or write only), so you'd need to redirect the output and read it a different manner (after a short delay to allow the output file to be created):
set control [open "|./odamex > odamex.log" "w"] exec sleep 1 set output [open "odamex.log" "r"]
And you must remember to close both pipes, but before you do, you must ensure that the application has quit (otherwise close will never complete). To do this, use puts to write the quit command to the client (as if it were typed). Applications do not like multiple puts without having the stream flushed, so a flush follows the puts.
set control [open "|./odamex > odamex.log" w] exec sleep 1 set output [open "odamex.log" r] puts $control quit flush $control close $control close $output
Most of this functionality should be put into separate procs so that it does not hurt your head. A lot of it is already available as functions within existing tests, which you can copy.