tcltest man page on Mageia

Printed from http://www.polarhome.com/service/man/?qf=tcltest&af=0&tf=2&of=Mageia

tcltest(n)		     Tcl Bundled Packages		    tcltest(n)

______________________________________________________________________________

NAME
       tcltest - Test harness support code and utilities

SYNOPSIS
       package require tcltest ?2.3?

       tcltest::test name description ?option value ...?
       tcltest::test name description ?constraints? body result

       tcltest::loadTestedCommands
       tcltest::makeDirectory name ?directory?
       tcltest::removeDirectory name ?directory?
       tcltest::makeFile contents name ?directory?
       tcltest::removeFile name ?directory?
       tcltest::viewFile name ?directory?
       tcltest::cleanupTests ?runningMultipleTests?
       tcltest::runAllTests

       tcltest::configure
       tcltest::configure option
       tcltest::configure option value ?option value ...?
       tcltest::customMatch mode command
       tcltest::testConstraint constraint ?value?
       tcltest::outputChannel ?channelID?
       tcltest::errorChannel ?channelID?
       tcltest::interpreter ?interp?

       tcltest::debug ?level?
       tcltest::errorFile ?filename?
       tcltest::limitConstraints ?boolean?
       tcltest::loadFile ?filename?
       tcltest::loadScript ?script?
       tcltest::match ?patternList?
       tcltest::matchDirectories ?patternList?
       tcltest::matchFiles ?patternList?
       tcltest::outputFile ?filename?
       tcltest::preserveCore ?level?
       tcltest::singleProcess ?boolean?
       tcltest::skip ?patternList?
       tcltest::skipDirectories ?patternList?
       tcltest::skipFiles ?patternList?
       tcltest::temporaryDirectory ?directory?
       tcltest::testsDirectory ?directory?
       tcltest::verbose ?level?

       tcltest::test name description optionList
       tcltest::bytestring string
       tcltest::normalizeMsg msg
       tcltest::normalizePath pathVar
       tcltest::workingDirectory ?dir?
_________________________________________________________________

DESCRIPTION
       The  tcltest  package  provides	several utility commands useful in the
       construction of test suites for code instrumented to be run by  evalua‐
       tion of Tcl commands.  Notably the built-in commands of the Tcl library
       itself are tested by a test suite using the tcltest package.

       All the commands provided by the tcltest package	 are  defined  in  and
       exported	 from  the  ::tcltest  namespace, as indicated in the SYNOPSIS
       above.  In the following sections, all commands will  be	 described  by
       their simple names, in the interest of brevity.

       The  central  command  of tcltest is test that defines and runs a test.
       Testing with test involves evaluation of a Tcl script and comparing the
       result  to an expected result, as configured and controlled by a number
       of options.  Several other commands provided by tcltest govern the con‐
       figuration  of  test and the collection of many test commands into test
       suites.

       See CREATING TEST SUITES WITH TCLTEST below for an extended example  of
       how to use the commands of tcltest to produce test suites for your Tcl-
       enabled code.

COMMANDS
       test name description ?option value ...?
	      Defines and possibly runs a test with the name name and descrip‐
	      tion  description.   The name and description of a test are used
	      in messages reported by test during the test, as	configured  by
	      the options of tcltest.  The remaining option value arguments to
	      test define the test, including the scripts to run,  the	condi‐
	      tions  under  which  to  run  them, the expected result, and the
	      means by which the expected and actual results  should  be  com‐
	      pared.   See TESTS below for a complete description of the valid
	      options and how they define a test.  The test command returns an
	      empty string.

       test name description ?constraints? body result
	      This form of test is provided to support test suites written for
	      version 1 of the tcltest package, and also a  simpler  interface
	      for  a  common  usage.  It is the same as “test name description
	      -constraints constraints -body body -result result”.  All	 other
	      options  to test take their default values.  When constraints is
	      omitted, this form of test can be distinguished from  the	 first
	      because all options begin with “-”.

       loadTestedCommands
	      Evaluates	 in  the caller's context the script specified by con‐
	      figure -load or configure -loadfile.  Returns the result of that
	      script  evaluation,  including  any  error raised by the script.
	      Use this command and the related configuration options  to  pro‐
	      vide  the	 commands  to be tested to the interpreter running the
	      test suite.

       makeFile contents name ?directory?
	      Creates a file named name relative to  directory	directory  and
	      write  contents to that file using the encoding encoding system.
	      If contents does not end with  a	newline,  a  newline  will  be
	      appended	so  that  the file named name does end with a newline.
	      Because the system encoding is used, this command is only	 suit‐
	      able  for	 making	 text  files.  The file will be removed by the
	      next evaluation of cleanupTests, unless it is removed by remove‐
	      File  first.   The  default  value of directory is the directory
	      configure -tmpdir.  Returns the full path of the	file  created.
	      Use this command to create any text file required by a test with
	      contents as needed.

       removeFile name ?directory?
	      Forces the file referenced by name to  be	 removed.   This  file
	      name  should  be	relative  to directory.	  The default value of
	      directory is the directory configure -tmpdir.  Returns an	 empty
	      string.  Use this command to delete files created by makeFile.

       makeDirectory name ?directory?
	      Creates  a directory named name relative to directory directory.
	      The  directory  will  be	removed	 by  the  next	evaluation  of
	      cleanupTests,  unless  it	 is  removed by removeDirectory first.
	      The default  value  of  directory	 is  the  directory  configure
	      -tmpdir.	 Returns  the full path of the directory created.  Use
	      this command to create any  directories  that  are  required  to
	      exist by a test.

       removeDirectory name ?directory?
	      Forces  the  directory  referenced  by  name to be removed. This
	      directory should be relative to directory.  The default value of
	      directory	 is the directory configure -tmpdir.  Returns an empty
	      string.  Use this command to delete any directories  created  by
	      makeDirectory.

       viewFile file ?directory?
	      Returns the contents of file, except for any final newline, just
	      as read -nonewline would return.	This file name should be rela‐
	      tive to directory.  The default value of directory is the direc‐
	      tory configure -tmpdir.  Use this command as a convenient way to
	      turn  the contents of a file generated by a test into the result
	      of that test for matching against an expected result.  The  con‐
	      tents  of	 the  file  are read using the system encoding, so its
	      usefulness is limited to text files.

       cleanupTests
	      Intended to clean up and summarize after several tests have been
	      run.   Typically	called	once  per test file, at the end of the
	      file after all tests have been completed.	 For  best  effective‐
	      ness,  be	 sure  that  the  cleanupTests is evaluated even if an
	      error occurs earlier in the test file evaluation.

	      Prints statistics about the tests run  and  removes  files  that
	      were  created  by	 makeDirectory	and  makeFile  since  the last
	      cleanupTests.  Names of files and directories in	the  directory
	      configure	 -tmpdir  created since the last cleanupTests, but not
	      created by makeFile or makeDirectory are printed to  outputChan‐
	      nel.  This command also restores the original shell environment,
	      as described by the ::env array. Returns an empty string.

       runAllTests
	      This is a master command meant to run an entire suite of	tests,
	      spanning	multiple  files and/or directories, as governed by the
	      configurable options of tcltest.	See RUNNING  ALL  TESTS	 below
	      for  a complete description of the many variations possible with
	      runAllTests.

CONFIGURATION COMMANDS
       configure
	      Returns the list of configurable options supported  by  tcltest.
	      See  CONFIGURABLE	 OPTIONS  below	 for the full list of options,
	      their valid values, and their effect on tcltest operations.

       configure option
	      Returns the current value of the supported  configurable	option
	      option.	Raises	an  error if option is not a supported config‐
	      urable option.

       configure option value ?option value ...?
	      Sets the value of each configurable option option to the	corre‐
	      sponding value value, in order.  Raises an error if an option is
	      not a supported configurable option, or if value is not a	 valid
	      value  for  the  corresponding option, or if a value is not pro‐
	      vided.  When an error is raised, the operation of	 configure  is
	      halted, and subsequent option value arguments are not processed.

	      If  the  environment variable ::env(TCLTEST_OPTIONS) exists when
	      the tcltest package is loaded (by package require tcltest)  then
	      its  value is taken as a list of arguments to pass to configure.
	      This allows the default values of the configuration  options  to
	      be set by the environment.

       customMatch mode script
	      Registers	 mode  as  a  new  legal value of the -match option to
	      test.  When the -match mode option is passed to test, the script
	      script  will be evaluated to compare the actual result of evalu‐
	      ating the body of the test to the expected result.   To  perform
	      the  match,  the	script is completed with two additional words,
	      the expected result, and the actual result,  and	the  completed
	      script  is  evaluated  in	 the  global namespace.	 The completed
	      script is expected to return a boolean value indicating  whether
	      or  not  the results match.  The built-in matching modes of test
	      are exact, glob, and regexp.

       testConstraint constraint ?boolean?
	      Sets or returns the boolean value associated with the named con‐
	      straint.	See TEST CONSTRAINTS below for more information.

       interpreter ?executableName?
	      Sets  or	returns	 the  name  of	the executable to be execed by
	      runAllTests to run each test file when configure -singleproc  is
	      false.   The  default  value  for interpreter is the name of the
	      currently running program as returned by info nameofexecutable.

       outputChannel ?channelID?
	      Sets or returns the output channel ID.  This defaults to stdout.
	      Any test that prints test related output should send that output
	      to outputChannel rather than letting that output default to std‐
	      out.

       errorChannel ?channelID?
	      Sets  or returns the error channel ID.  This defaults to stderr.
	      Any test that prints error messages should send that  output  to
	      errorChannel rather than printing directly to stderr.

SHORTCUT COMMANDS
       debug ?level?
	      Same as configure -debug ?level?.

       errorFile ?filename?
	      Same as configure -errfile ?filename?.

       limitConstraints ?boolean?
	      Same as configure -limitconstraints ?boolean?.

       loadFile ?filename?
	      Same as configure -loadfile ?filename?.

       loadScript ?script?
	      Same as configure -load ?script?.

       match ?patternList?
	      Same as configure -match ?patternList?.

       matchDirectories ?patternList?
	      Same as configure -relateddir ?patternList?.

       matchFiles ?patternList?
	      Same as configure -file ?patternList?.

       outputFile ?filename?
	      Same as configure -outfile ?filename?.

       preserveCore ?level?
	      Same as configure -preservecore ?level?.

       singleProcess ?boolean?
	      Same as configure -singleproc ?boolean?.

       skip ?patternList?
	      Same as configure -skip ?patternList?.

       skipDirectories ?patternList?
	      Same as configure -asidefromdir ?patternList?.

       skipFiles ?patternList?
	      Same as configure -notfile ?patternList?.

       temporaryDirectory ?directory?
	      Same as configure -tmpdir ?directory?.

       testsDirectory ?directory?
	      Same as configure -testdir ?directory?.

       verbose ?level?
	      Same as configure -verbose ?level?.

OTHER COMMANDS
       The  remaining  commands	 provided  by tcltest have better alternatives
       provided by tcltest or Tcl itself.  They are retained to support exist‐
       ing test suites, but should be avoided in new code.

       test name description optionList
	      This  form  of  test was provided to enable passing many options
	      spanning several lines to test as a single  argument  quoted  by
	      braces,  rather  than  needing  to  backslash quote the newlines
	      between arguments to test.  The optionList argument is  expected
	      to be a list with an even number of elements representing option
	      and value arguments to pass to test.  However, these values  are
	      not  passed  directly,  as  in  the  alternate  forms of switch.
	      Instead, this form makes an  unfortunate	attempt	 to  overthrow
	      Tcl's  substitution rules by performing substitutions on some of
	      the list elements as an attempt to implement a “do what I	 mean”
	      interpretation  of  a  brace-enclosed  “block”.	The  result is
	      nearly impossible to document clearly, and for that reason  this
	      form  is	not  recommended.   See	 the examples in CREATING TEST
	      SUITES WITH TCLTEST below to see that this form  is  really  not
	      necessary	 to avoid backslash-quoted newlines.  If you insist on
	      using this form, examine the source code of tcltest if you  want
	      to  know	the  substitution  details,  or just enclose the third
	      through last argument to test in braces and hope for the best.

       workingDirectory ?directoryName?
	      Sets or returns the current  working  directory  when  the  test
	      suite is running.	 The default value for workingDirectory is the
	      directory in which the test suite was launched.	The  Tcl  com‐
	      mands cd and pwd are sufficient replacements.

       normalizeMsg msg
	      Returns  the  result  of removing the “extra” newlines from msg,
	      where “extra” is rather imprecise.  Tcl offers plenty of	string
	      processing  commands  to modify strings as you wish, and custom‐
	      Match allows flexible matching of actual and expected results.

       normalizePath pathVar
	      Resolves symlinks in a path, thus creating a path without inter‐
	      nal redirection.	It is assumed that pathVar is absolute.	 path‐
	      Var is modified in place.	 The Tcl command file normalize	 is  a
	      sufficient replacement.

       bytestring string
	      Construct	 a  string  that consists of the requested sequence of
	      bytes, as opposed to a string of properly formed	UTF-8  charac‐
	      ters using the value supplied in string.	This allows the tester
	      to create denormalized or improperly formed strings to pass to C
	      procedures  that	are  supposed  to accept strings with embedded
	      NULL types and confirm that a string result has a	 certain  pat‐
	      tern  of	bytes.	 This is exactly equivalent to the Tcl command
	      encoding convertfrom identity.

TESTS
       The test command is the heart of the tcltest  package.	Its  essential
       function	 is  to	 evaluate  a Tcl script and compare the result with an
       expected result.	 The options of test define the test script, the envi‐
       ronment	in which to evaluate it, the expected result, and how the com‐
       pare the actual result to  the  expected	 result.   Some	 configuration
       options of tcltest also influence how test operates.

       The valid options for test are summarized:

	      test name description
		      ?-constraints keywordList|expression?
		      ?-setup setupScript?
		      ?-body testScript?
		      ?-cleanup cleanupScript?
		      ?-result expectedAnswer?
		      ?-output expectedOutput?
		      ?-errorOutput expectedError?
		      ?-returnCodes codeList?
		      ?-match mode?

       The  name  may  be  any	string.	  It  is conventional to choose a name
       according to the pattern:

	      target-majorNum.minorNum

       For white-box (regression) tests, the target should be the name of  the
       C  function  or	Tcl  procedure being tested.  For black-box tests, the
       target should be the name of the feature being  tested.	 Some  conven‐
       tions  call  for	 the  names of black-box tests to have the suffix _bb.
       Related tests should share a major number.  As a test suite evolves, it
       is  best	 to have the same test name continue to correspond to the same
       test, so that it remains meaningful to say things  like	“Test  foo-1.3
       passed in all releases up to 3.4, but began failing in release 3.5.”

       During  evaluation  of  test, the name will be compared to the lists of
       string matching patterns returned by configure  -match,	and  configure
       -skip.	The  test will be run only if name matches any of the patterns
       from configure -match and matches none of the patterns  from  configure
       -skip.

       The description should be a short textual description of the test.  The
       description is included in output produced by the test, typically  test
       failure	messages.   Good description values should briefly explain the
       purpose of the test to users of a test suite.  The name of a Tcl	 or  C
       function being tested should be included in the description for regres‐
       sion tests.  If the test case exists to reproduce a  bug,  include  the
       bug ID in the description.

       Valid attributes and associated values are:

       -constraints keywordList|expression
	      The  optional  -constraints attribute can be list of one or more
	      keywords or an expression.  If the -constraints value is a  list
	      of keywords, each of these keywords should be the name of a con‐
	      straint defined by a call to  testConstraint.   If  any  of  the
	      listed  constraints  is  false  or  does	not exist, the test is
	      skipped.	If the	-constraints  value  is	 an  expression,  that
	      expression  is  evaluated.  If the expression evaluates to true,
	      then the test is run.  Note that the expression  form  of	 -con‐
	      straints	may  interfere	with  the operation of configure -con‐
	      straints and configure  -limitconstraints,  and  is  not	recom‐
	      mended.	Appropriate  constraints  should be added to any tests
	      that should not always be run.  That is, conditional  evaluation
	      of a test should be accomplished by the -constraints option, not
	      by conditional evaluation of test.  In that way, the same number
	      of  tests are always reported by the test suite, though the num‐
	      ber skipped may change based on the  testing  environment.   The
	      default  value is an empty list.	See TEST CONSTRAINTS below for
	      a list of built-in constraints and information  on  how  to  add
	      your own constraints.

       -setup script
	      The  optional  -setup  attribute indicates a script that will be
	      run before the script indicated  by  the	-body  attribute.   If
	      evaluation  of  script raises an error, the test will fail.  The
	      default value is an empty script.

       -body script
	      The -body attribute indicates the script to run to carry out the
	      test.   It must return a result that can be checked for correct‐
	      ness.  If evaluation of script raises an error,  the  test  will
	      fail.  The default value is an empty script.

       -cleanup script
	      The  optional -cleanup attribute indicates a script that will be
	      run after the script indicated by the -body attribute.  If eval‐
	      uation  of  script  raises  an  error,  the test will fail.  The
	      default value is an empty script.

       -match mode
	      The -match attribute determines how expected answers supplied by
	      -result,	-output,  and -errorOutput are compared.  Valid values
	      for mode are regexp, glob, exact, and any value registered by  a
	      prior call to customMatch.  The default value is exact.

       -result expectedValue
	      The  -result  attribute supplies the expectedValue against which
	      the return value from script will be compared. The default value
	      is an empty string.

       -output expectedValue
	      The  -output  attribute supplies the expectedValue against which
	      any output sent to stdout or outputChannel during evaluation  of
	      the  script(s)  will be compared.	 Note that only output printed
	      using ::puts is used for comparison.  If -output is  not	speci‐
	      fied,  output  sent to stdout and outputChannel is not processed
	      for comparison.

       -errorOutput expectedValue
	      The -errorOutput attribute supplies  the	expectedValue  against
	      which  any  output sent to stderr or errorChannel during evalua‐
	      tion of the script(s) will be compared. Note  that  only	output
	      printed using ::puts is used for comparison.  If -errorOutput is
	      not specified, output sent to stderr  and	 errorChannel  is  not
	      processed for comparison.

       -returnCodes expectedCodeList
	      The optional -returnCodes attribute supplies expectedCodeList, a
	      list of return codes that may be accepted from evaluation of the
	      -body  script.  If evaluation of the -body script returns a code
	      not in the expectedCodeList, the test fails.  All	 return	 codes
	      known  to	 return,  in both numeric and symbolic form, including
	      extended return codes, are acceptable elements in the  expected‐
	      CodeList.	 Default value is “ok”return.

       To  pass,  a  test  must	 successfully  evaluate its -setup, -body, and
       -cleanup scripts.  The return code of the -body script and  its	result
       must  match  expected  values,  and if specified, output and error data
       from the test must match expected -output and -errorOutput values.   If
       any  of	these  conditions are not met, then the test fails.  Note that
       all scripts are evaluated in the context of the caller of test.

       As long as test is called with valid syntax and legal  values  for  all
       attributes,  it	will  not  raise  an error.  Test failures are instead
       reported as output written to outputChannel.  In default	 operation,  a
       successful  test	 produces  no output.  The output messages produced by
       test are controlled by the configure -verbose option  as	 described  in
       CONFIGURABLE  OPTIONS  below.   Any output produced by the test scripts
       themselves should be produced using ::puts to outputChannel  or	error‐
       Channel, so that users of the test suite may easily capture output with
       the configure -outfile and configure -errfile options, and so that  the
       -output and -errorOutput attributes work properly.

TEST CONSTRAINTS
       Constraints  are	 used  to  determine  whether  or not a test should be
       skipped.	 Each constraint has a name, which may be any  string,	and  a
       boolean	value.	 Each test has a -constraints value which is a list of
       constraint names.  There are two modes  of  constraint  control.	  Most
       frequently, the default mode is used, indicated by a setting of config‐
       ure -limitconstraints to false.	The test will run  only	 if  all  con‐
       straints in the list are true-valued.  Thus, the -constraints option of
       test is a convenient, symbolic way to define  any  conditions  required
       for  the	 test  to be possible or meaningful.  For example, a test with
       -constraints unix will only be run if  the  constraint  unix  is	 true,
       which indicates the test suite is being run on a Unix platform.

       Each  test  should  include  whatever -constraints are required to con‐
       strain it to run only where appropriate.	 Several constraints are  pre-
       defined	in  the	 tcltest  package,  listed below.  The registration of
       user-defined constraints is performed by	 the  testConstraint  command.
       User-defined  constraints  may appear within a test file, or within the
       script specified by the configure -load or configure -loadfile options.

       The following is a list of constraints pre-defined by the tcltest pack‐
       age itself:

       singleTestInterp
	      test can only be run if all test files are sourced into a single
	      interpreter

       unix   test can only be run on any Unix platform

       win    test can only be run on any Windows platform

       nt     test can only be run on any Windows NT platform

       95     test can only be run on any Windows 95 platform

       98     test can only be run on any Windows 98 platform

       mac    test can only be run on any Mac platform

       unixOrWin
	      test can only be run on a Unix or Windows platform

       macOrWin
	      test can only be run on a Mac or Windows platform

       macOrUnix
	      test can only be run on a Mac or Unix platform

       tempNotWin
	      test can not be run on Windows.  This flag is used to  temporar‐
	      ily disable a test.

       tempNotMac
	      test  can not be run on a Mac.  This flag is used to temporarily
	      disable a test.

       unixCrash
	      test crashes if it is run on Unix.  This flag is used to	tempo‐
	      rarily disable a test.

       winCrash
	      test crashes if it is run on Windows.  This flag is used to tem‐
	      porarily disable a test.

       macCrash
	      test crashes if it is run on a Mac.  This flag is used to tempo‐
	      rarily disable a test.

       emptyTest
	      test  is	empty,	and  so not worth running, but it remains as a
	      place-holder for a test to be written in the future.  This  con‐
	      straint  has value false to cause tests to be skipped unless the
	      user specifies otherwise.

       knownBug
	      test is known to fail and the bug is not yet fixed.   This  con‐
	      straint  has value false to cause tests to be skipped unless the
	      user specifies otherwise.

       nonPortable
	      test can only be run  in	some  known  development  environment.
	      Some  tests  are	inherently non-portable because they depend on
	      things like word length, file system configuration, window  man‐
	      ager, etc.  This constraint has value false to cause tests to be
	      skipped unless the user specifies otherwise.

       userInteraction
	      test requires interaction from the user.	 This  constraint  has
	      value false to causes tests to be skipped unless the user speci‐
	      fies otherwise.

       interactive
	      test can only be run in if the  interpreter  is  in  interactive
	      mode (when the global tcl_interactive variable is set to 1).

       nonBlockFiles
	      test  can	 only  be  run if platform supports setting files into
	      nonblocking mode

       asyncPipeClose
	      test can only be run if platform supports async flush and	 async
	      close on a pipe

       unixExecs
	      test  can	 only  be  run if this machine has Unix-style commands
	      cat, echo, sh, wc, rm, sleep, fgrep, ps, chmod, and mkdir avail‐
	      able

       hasIsoLocale
	      test can only be run if can switch to an ISO locale

       root   test can only run if Unix user is root

       notRoot
	      test can only run if Unix user is not root

       eformat
	      test  can	 only run if app has a working version of sprintf with
	      respect to the “e” format of floating-point numbers.

       stdio  test can only be run if interpreter can be opened as a pipe.

       The alternative mode of constraint control is enabled by	 setting  con‐
       figure -limitconstraints to true.  With that configuration setting, all
       existing constraints other than those in the constraint	list  returned
       by  configure -constraints are set to false.  When the value of config‐
       ure -constraints is set, all those constraints are set  to  true.   The
       effect  is  that when both options configure -constraints and configure
       -limitconstraints are in use, only  those  tests	 including  only  con‐
       straints	 from  the configure -constraints list are run; all others are
       skipped.	 For example, one might set up a configuration with

	      configure -constraints knownBug \
			-limitconstraints true \
			-verbose pass

       to run exactly those tests  that	 exercise  known  bugs,	 and  discover
       whether any of them pass, indicating the bug had been fixed.

RUNNING ALL TESTS
       The  single  command  runAllTests  is  evaluated	 to run an entire test
       suite, spanning many files and directories.  The configuration  options
       of  tcltest  control  the  precise operations.  The runAllTests command
       begins by printing a summary of its configuration to outputChannel.

       Test files to be evaluated are sought in the directory configure -test‐
       dir.   The  list	 of files in that directory that match any of the pat‐
       terns in configure -file and match none of the  patterns	 in  configure
       -notfile	 is generated and sorted.  Then each file will be evaluated in
       turn.  If configure -singleproc is true, then each file will be sourced
       in  the	caller's  context.  If it is false, then a copy of interpreter
       will be exec'd to evaluate each file.  The multi-process	 operation  is
       useful  when  testing  can cause errors so severe that a process termi‐
       nates.  Although such an error may terminate a child process evaluating
       one  file,  the	master	process can continue with the rest of the test
       suite.  In multi-process operation, the configuration of tcltest in the
       master  process	is passed to the child processes as command line argu‐
       ments, with the exception of configure -outfile.	 The runAllTests  com‐
       mand in the master process collects all output from the child processes
       and collates their results into one  master  report.   Any  reports  of
       individual test failures, or messages requested by a configure -verbose
       setting are passed directly on to outputChannel by the master process.

       After evaluating all selected test files, a summary of the  results  is
       printed	to  outputChannel.   The  summary includes the total number of
       tests evaluated, broken down into  those	 skipped,  those  passed,  and
       those  failed.	The  summary also notes the number of files evaluated,
       and the names of any files with failing tests or errors.	 A list of the
       constraints  that  caused  tests to be skipped, and the number of tests
       skipped for each is also printed.  Also, messages  are  printed	if  it
       appears	that  evaluation of a test file has caused any temporary files
       to be left behind in configure -tmpdir.

       Having completed and summarized all selected  test  files,  runAllTests
       then  recursively  acts	on  subdirectories of configure -testdir.  All
       subdirectories that match any of the patterns in configure  -relateddir
       and  do	not  match  any of the patterns in configure -asidefromdir are
       examined.  If a file named all.tcl is found in  such  a	directory,  it
       will  be	 sourced  in the caller's context.  Whether or not an examined
       directory contains an all.tcl file, its subdirectories are also scanned
       against the configure -relateddir and configure -asidefromdir patterns.
       In this way, many directories in a directory tree can  have  all	 their
       test files evaluated by a single runAllTests command.

CONFIGURABLE OPTIONS
       The configure command is used to set and query the configurable options
       of tcltest.  The valid options are:

       -singleproc boolean
	      Controls whether or not runAllTests spawns a child  process  for
	      each  test  file.	  No  spawning	when boolean is true.  Default
	      value is false.

       -debug level
	      Sets the debug level to level, an integer value  indicating  how
	      much  debugging  information  should be printed to stdout.  Note
	      that debug messages always go  to	 stdout,  independent  of  the
	      value  of	 configure  -outfile.  Default value is 0.  Levels are
	      defined as:

	      0	     Do not display any debug information.

	      1	     Display information regarding whether a test  is  skipped
		     because  it  does	not  match  any of the tests that were
		     specified using by	 configure  -match  (userSpecifiedNon‐
		     Match) or matches any of the tests specified by configure
		     -skip (userSpecifiedSkip).	  Also	print  warnings	 about
		     possible  lack of cleanup or balance in test files.  Also
		     print warnings about any re-use of test names.

	      2	     Display the flag array parsed by the command line proces‐
		     sor,  the	contents  of  the  ::env  array, and all user-
		     defined variables that exist in the current namespace  as
		     they are used.

	      3	     Display  information  regarding  what individual procs in
		     the test harness are doing.

       -verbose level
	      Sets the type of output verbosity desired to level,  a  list  of
	      zero  or more of the elements body, pass, skip, start, error and
	      line.  Default value is {body error}.  Levels are defined as:

	      body (b)
		     Display the body of failed tests

	      pass (p)
		     Print output when a test passes

	      skip (s)
		     Print output when a test is skipped

	      start (t)
		     Print output whenever a test starts

	      error (e)
		     Print errorInfo and errorCode, if they exist, when a test
		     return code does not match its expected return code

	      line (l)
		     Print source file line information of failed tests
       The single letter abbreviations noted above are also recognized so that
       “configure -verbose pt”	is  the	 same  as  “configure  -verbose	 {pass
       start}”.

       -preservecore level
	      Sets  the	 core  preservation level to level.  This level deter‐
	      mines how stringent checks for core files are.  Default value is
	      0.  Levels are defined as:

	      0	     No	 checking  — do not check for core files at the end of
		     each test command, but do check for them  in  runAllTests
		     after all test files have been evaluated.

	      1	     Also  check  for  core files at the end of each test com‐
		     mand.

	      2	     Check for core files at all times	described  above,  and
		     save  a  copy  of	each  core  file produced in configure
		     -tmpdir.

       -limitconstraints boolean
	      Sets the mode by which test honors constraints as	 described  in
	      TESTS above.  Default value is false.

       -constraints list
	      Sets all the constraints in list to true.	 Also used in combina‐
	      tion with configure -limitconstraints true to control an	alter‐
	      native  constraint  mode	as  described in TESTS above.  Default
	      value is an empty list.

       -tmpdir directory
	      Sets the temporary directory to be used by makeFile,  makeDirec‐
	      tory,  viewFile,	removeFile, and removeDirectory as the default
	      directory where temporary files and directories created by  test
	      files should be created.	Default value is workingDirectory.

       -testdir directory
	      Sets  the	 directory  searched by runAllTests for test files and
	      subdirectories.  Default value is workingDirectory.

       -file patternList
	      Sets the list of patterns used by runAllTests to determine  what
	      test files to evaluate.  Default value is “*.test”.

       -notfile patternList
	      Sets  the list of patterns used by runAllTests to determine what
	      test files to skip.  Default value is “l.*.test”,	 so  that  any
	      SCCS lock files are skipped.

       -relateddir patternList
	      Sets  the list of patterns used by runAllTests to determine what
	      subdirectories to search for an all.tcl file.  Default value  is
	      “*”.

       -asidefromdir patternList
	      Sets  the list of patterns used by runAllTests to determine what
	      subdirectories to skip  when  searching  for  an	all.tcl	 file.
	      Default value is an empty list.

       -match patternList
	      Set  the	list  of  patterns used by test to determine whether a
	      test should be run.  Default value is “*”.

       -skip patternList
	      Set the list of patterns used by test  to	 determine  whether  a
	      test should be skipped.  Default value is an empty list.

       -load script
	      Sets  a  script  to be evaluated by loadTestedCommands.  Default
	      value is an empty script.

       -loadfile filename
	      Sets the filename from which to read a script to be evaluated by
	      loadTestedCommands.  This is an alternative to -load.  They can‐
	      not be used together.

       -outfile filename
	      Sets the file to which all output produced by tcltest should  be
	      written.	 A file named filename will be opened for writing, and
	      the resulting channel will be set as the value of outputChannel.

       -errfile filename
	      Sets the file to which all  error	 output	 produced  by  tcltest
	      should  be  written.   A	file named filename will be opened for
	      writing, and the resulting channel will be set as the  value  of
	      errorChannel.

CREATING TEST SUITES WITH TCLTEST
       The fundamental element of a test suite is the individual test command.
       We begin with several examples.

       [1]    Test of a script that returns normally.

		     test example-1.0 {normal return} {
			 format %s value
		     } value

       [2]    Test of a script that requires context setup and cleanup.	  Note
	      the  bracing  and	 indenting style that avoids any need for line
	      continuation.

		     test example-1.1 {test file existence} -setup {
			 set file [makeFile {} test]
		     } -body {
			 file exists $file
		     } -cleanup {
			 removeFile test
		     } -result 1

       [3]    Test of a script that raises an error.

		     test example-1.2 {error return} -body {
			 error message
		     } -returnCodes error -result message

       [4]    Test with a constraint.

		     test example-1.3 {user owns created files} -constraints {
			 unix
		     } -setup {
			 set file [makeFile {} test]
		     } -body {
			 file attributes $file -owner
		     } -cleanup {
			 removeFile test
		     } -result $::tcl_platform(user)

       At the next higher layer of organization,  several  test	 commands  are
       gathered	 together  into	 a  single  test file.	Test files should have
       names with the .test extension, because that  is	 the  default  pattern
       used  by runAllTests to find test files.	 It is a good rule of thumb to
       have one test file for each source code file of your  project.	It  is
       good  practice to edit the test file and the source code file together,
       keeping tests synchronized with code changes.

       Most of the code in the test file should be  the	 test  commands.   Use
       constraints to skip tests, rather than conditional evaluation of test.

       [5]    Recommended  system  for	writing	 conditional tests, using con‐
	      straints to guard:

		     testConstraint X [expr $myRequirement]
		     test goodConditionalTest {} X {
			 # body
		     } result

       [6]    Discouraged system for writing conditional tests,	 using	if  to
	      guard:

		     if $myRequirement {
			 test badConditionalTest {} {
			     #body
			 } result
		     }

       Use  the	 -setup and -cleanup options to establish and release all con‐
       text requirements of the test body.  Do not make tests depend on	 prior
       tests  in  the  file.   Those prior tests might be skipped.  If several
       consecutive tests require the same context, the appropriate  setup  and
       cleanup	scripts	 may  be  stored in variable for passing to each tests
       -setup and -cleanup options.  This is a better solution than performing
       setup  outside of test commands, because the setup will only be done if
       necessary, and any errors during setup will be reported, and not	 cause
       the test file to abort.

       A test file should be able to be combined with other test files and not
       interfere with them, even when configure -singleproc 1 causes all files
       to  be evaluated in a common interpreter.  A simple way to achieve this
       is to have your tests define all their  commands	 and  variables	 in  a
       namespace that is deleted when the test file evaluation is complete.  A
       good namespace to use is a child namespace test of the namespace of the
       module you are testing.

       A  test	file should also be able to be evaluated directly as a script,
       not depending on being called by a master runAllTests.  This means that
       each test file should process command line arguments to give the tester
       all the configuration control that tcltest provides.

       After all tests in a test file,	the  command  cleanupTests  should  be
       called.

       [7]    Here  is	a  sketch  of  a  sample  test file illustrating those
	      points:

		     package require tcltest 2.2
		     eval ::tcltest::configure $argv
		     package require example
		     namespace eval ::example::test {
			 namespace import ::tcltest::*
			 testConstraint X [expr {...}]
			 variable SETUP {#common setup code}
			 variable CLEANUP {#common cleanup code}
			 test example-1 {} -setup $SETUP -body {
			     # First test
			 } -cleanup $CLEANUP -result {...}
			 test example-2 {} -constraints X -setup $SETUP -body {
			     # Second test; constrained
			 } -cleanup $CLEANUP -result {...}
			 test example-3 {} {
			     # Third test; no context required
			 } {...}
			 cleanupTests
		     }
		     namespace delete ::example::test

       The next level of organization is a full test suite, made up of several
       test files.  One script is used to control the entire suite.  The basic
       function of this script is to call runAllTests after doing  any	neces‐
       sary  setup.   This script is usually named all.tcl because that is the
       default name used by runAllTests when combining	multiple  test	suites
       into one testing run.

       [8]    Here is a sketch of a sample test suite master script:

		     package require Tcl 8.4
		     package require tcltest 2.2
		     package require example
		     ::tcltest::configure -testdir \
			     [file dirname [file normalize [info script]]]
		     eval ::tcltest::configure $argv
		     ::tcltest::runAllTests

COMPATIBILITY
       A  number of commands and variables in the ::tcltest namespace provided
       by earlier releases of tcltest have not been documented here.  They are
       no  longer part of the supported public interface of tcltest and should
       not be used in new test suites.	However, to continue to support exist‐
       ing  test suites written to the older interface specifications, many of
       those deprecated commands and variables	still  work  as	 before.   For
       example,	 in many circumstances, configure will be automatically called
       shortly after package require tcltest 2.1 succeeds with arguments  from
       the variable ::argv.  This is to support test suites that depend on the
       old behavior that tcltest was  automatically  configured	 from  command
       line  arguments.	  New test files should not depend on this, but should
       explicitly include

	      eval ::tcltest::configure $::argv

       to establish a configuration from command line arguments.

KNOWN ISSUES
       There are two known issues related to nested evaluations of test.   The
       first  issue  relates to the stack level in which test scripts are exe‐
       cuted.  Tests nested within other tests may be  executed	 at  the  same
       stack level as the outermost test.  For example, in the following code:

	      test level-1.1 {level 1} {
		  -body {
		      test level-2.1 {level 2} {
		      }
		  }
	      }

       any  script  executed  in  level-2.1  may be executed at the same stack
       level as the script defined for level-1.1.

       In addition, while two tests  have  been	 run,  results	will  only  be
       reported by cleanupTests for tests at the same level as test level-1.1.
       However, test results for all tests run	prior  to  level-1.1  will  be
       available when test level-2.1 runs.  What this means is that if you try
       to access the test results for test level-2.1, it will may say that “m”
       tests  have run, “n” tests have been skipped, “o” tests have passed and
       “p” tests have failed, where “m”, “n”, “o”, and “p” refer to tests that
       were run at the same test level as test level-1.1.

       Implementation  of  output  and	error  comparison  in the test command
       depends on usage of ::puts in your application code.  Output is	inter‐
       cepted  by  redefining the ::puts command while the defined test script
       is being run.  Errors thrown by C procedures or printed directly from C
       applications  will not be caught by the test command.  Therefore, usage
       of the -output and -errorOutput options to test is useful only for pure
       Tcl applications that use ::puts to produce output.

KEYWORDS
       test, test harness, test suite

tcltest				      2.3			    tcltest(n)
[top]

List of man pages available for Mageia

Copyright (c) for man pages and the logo by the respective OS vendor.

For those who want to learn more, the polarhome community provides shell access and support.

[legal] [privacy] [GNU] [policy] [cookies] [netiquette] [sponsors] [FAQ]
Tweet
Polarhome, production since 1999.
Member of Polarhome portal.
Based on Fawad Halim's script.
....................................................................
Vote for polarhome
Free Shell Accounts :: the biggest list on the net