aboutsummaryrefslogtreecommitdiffstats
path: root/doc/kyua.1.in
diff options
context:
space:
mode:
Diffstat (limited to 'doc/kyua.1.in')
-rw-r--r--doc/kyua.1.in400
1 files changed, 400 insertions, 0 deletions
diff --git a/doc/kyua.1.in b/doc/kyua.1.in
new file mode 100644
index 000000000000..2fca5eb09f9f
--- /dev/null
+++ b/doc/kyua.1.in
@@ -0,0 +1,400 @@
+.\" Copyright 2011 The Kyua Authors.
+.\" All rights reserved.
+.\"
+.\" Redistribution and use in source and binary forms, with or without
+.\" modification, are permitted provided that the following conditions are
+.\" met:
+.\"
+.\" * Redistributions of source code must retain the above copyright
+.\" notice, this list of conditions and the following disclaimer.
+.\" * Redistributions in binary form must reproduce the above copyright
+.\" notice, this list of conditions and the following disclaimer in the
+.\" documentation and/or other materials provided with the distribution.
+.\" * Neither the name of Google Inc. nor the names of its contributors
+.\" may be used to endorse or promote products derived from this software
+.\" without specific prior written permission.
+.\"
+.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+.Dd May 12, 2015
+.Dt KYUA 1
+.Os
+.Sh NAME
+.Nm kyua
+.Nd Testing framework for infrastructure software
+.Sh SYNOPSIS
+.Nm
+.Op Fl -config Ar file
+.Op Fl -logfile Ar file
+.Op Fl -loglevel Ar level
+.Op Fl -variable Ar name=value
+.Ar command
+.Op Ar command_options
+.Op Ar command_arguments
+.Sh DESCRIPTION
+.Em If you are here looking for details on how to run the test suite in
+.Pa /usr/tests
+.Em ( or
+.Pa __TESTSDIR__ ) ,
+.Em please start by reading the
+.Xr tests 7
+.Em manual page that should be supplied by your system .
+.Pp
+Kyua is a testing framework for infrastructure software, originally
+designed to equip BSD-based operating systems with a test suite.
+This means that Kyua is lightweight and simple, and that Kyua integrates well
+with various build systems and continuous integration frameworks.
+.Pp
+Kyua features an expressive test suite definition language, a safe
+runtime engine for test suites and a powerful report generation engine.
+.Pp
+Kyua is for both developers and users, from the developer applying a
+simple fix to a library to the system administrator deploying a new
+release on a production machine.
+.Pp
+Kyua is able to execute test programs written with a plethora of testing
+libraries and languages.
+The test program library of choice is ATF, which
+.Nm Ns 's
+design originated from.
+However, framework-less test programs and TAP-compliant test programs can also
+be executed through
+.Nm
+.Ss Overview
+As can be observed in the synopsis, the interface of
+.Nm
+implements a common subcommand-based interface.
+The arguments to the tool specify, in this order: a set of common options
+that all the commands accept, a required
+.Ar command
+name that specifies what
+.Nm
+should do, and
+a set of possibly-optional
+.Ar command_options
+and
+.Ar command_arguments
+that are specific to the chosen command.
+.Pp
+The following options are recognized by all the commands.
+Keep in mind that these must always be specified before the command name.
+.Bl -tag -width XX
+.It Fl -config Ar path , Fl c Ar path
+Specifies the configuration file to process, which must be in the format
+described in
+.Xr kyua.conf 5 .
+The special value
+.Sq none
+explicitly disables the loading of any configuration file.
+.Pp
+Defaults to
+.Pa ~/.kyua/kyua.conf
+if it exists, otherwise to
+.Pa __CONFDIR__/kyua.conf
+if it exists,
+or else to
+.Sq none .
+.It Fl -logfile Ar path
+Specifies the location of the file to which
+.Nm
+will log run time events useful for postmortem debugging.
+.Pp
+The default depends on different environment variables as described in
+.Sx Logging ,
+but typically the file will be stored within the user's home directory.
+.It Fl -loglevel Ar level
+Specifies the maximum logging level to record in the log file.
+See
+.Sx Logging
+for more details.
+.Pp
+The default is
+.Sq info .
+.It Fl -variable Ar name=value , Fl v Ar name=value
+Sets the
+.Ar name
+configuration variable to
+.Ar value .
+The values set through this option have preference over the values set in the
+configuration file.
+.Pp
+The specified variable can either be a builtin variable or a test-suite
+specific variable.
+See
+.Xr kyua.conf 5
+for more details.
+.El
+.Pp
+The following commands are generic and do not have any relation to the execution
+of tests or the inspection of their results:
+.Bl -tag -width reportXjunitXX -offset indent
+.It Ar about
+Shows general program information.
+See
+.Xr kyua-about 1 .
+.It Ar config
+Inspects the values of the configuration variables.
+See
+.Xr kyua-config 1 .
+.It Ar db-exec
+Executes an arbitrary SQL statement on a results file and prints the
+resulting table.
+See
+.Xr kyua-db-exec 1 .
+.It Ar help
+Shows usage information.
+See
+.Xr kyua-help 1 .
+.El
+.Pp
+The following commands are used to generate reports based on the data previously
+recorded in a results file:
+.Bl -tag -width reportXjunitXX -offset indent
+.It Ar report
+Generates a plaintext report.
+Combined with its
+.Fl -verbose
+flag and the ability to only display specific test cases, this command can also
+be used to debug test failures post-facto on the console.
+See
+.Xr kyua-report 1 .
+.It Ar report-html
+Generates an HTML report.
+See
+.Xr kyua-report-html 1 .
+.It Ar report-junit
+Generates a JUnit report.
+See
+.Xr kyua-report-junit 1 .
+.El
+.Pp
+The following commands are used to interact with a test suite:
+.Bl -tag -width reportXjunitXX -offset indent
+.It Ar debug
+Executes a single test case in a controlled environment for debugging purposes.
+See
+.Xr kyua-debug 1 .
+.It Ar list
+Lists test cases defined in a test suite by a
+.Xr kyuafile 5
+and, optionally, displays their metadata.
+See
+.Xr kyua-list 1 .
+.It Ar test
+Runs tests defined in a test suite by a
+.Xr kyuafile 5 .
+See
+.Xr kyua-test 1 .
+.El
+.Ss Logging
+.Nm
+has a logging facility that collects all kinds of events at run time.
+These events are always logged to a file so that the log is available when
+it is most needed: right after a non-reproducible problem happens.
+The only way to disable logging is by sending the log to
+.Pa /dev/null .
+.Pp
+The location of the log file can be manually specified with the
+.Fl -logfile
+option, which applies to all commands.
+If no file is explicitly specified, the location of the log files is chosen in
+this order:
+.Bl -enum -offset indent
+.It
+.Pa ${HOME}/.kyua/logs/
+if
+.Va HOME
+is defined.
+.It
+.Pa ${TMPDIR}/
+if
+.Va TMPDIR
+is defined.
+.It
+.Pa /tmp/ .
+.El
+.Pp
+And the default naming scheme of the log files is:
+.Sq <progname>.<timestamp>.log .
+.Pp
+The messages stored in the log file have a level (or severity) attached to
+them.
+These are:
+.Bl -tag -width warningXX -offset indent
+.It error
+Fatal error messages.
+The program generally terminates after these, either in a clean manner or by
+crashing.
+.It warning
+Non-fatal error messages.
+These generally report a condition that must be addressed but the application
+can continue to run.
+.It info
+Informational messages.
+These tell the user what the program was doing at a general level of
+operation.
+.It debug
+Detailed informational messages.
+These are often useful when debugging problems in the application, as they
+contain lots of internal details.
+.El
+.Pp
+The default log level is
+.Sq info
+unless explicitly overridden with
+.Fl -loglevel .
+.Pp
+The log file is a plain text file containing one line per log record.
+The format of each line is as follows:
+.Bd -literal -offset indent
+timestamp entry_type pid file:line: message
+.Ed
+.Pp
+.Ar entry_type
+can be one of:
+.Sq E
+for an error,
+.Sq W
+for a warning,
+.Sq I
+for an informational message and
+.Sq D
+for a debug message.
+.Ss Bug reporting
+If you think you have encountered a bug in
+.Nm ,
+please take the time to let the developers know about it.
+This will ensure that the bug is addressed and potentially fixed in the next
+Kyua release.
+.Pp
+The first step in reporting a bug is to check if there already is a similar
+bug in the database.
+You can check what issues are currently in the database by going to:
+.Bd -literal -offset indent
+https://github.com/jmmv/kyua/issues/
+.Ed
+.Pp
+If there is no existing issue that describes an issue similar to the
+one you are experiencing, you can open a new one by visiting:
+.Bd -literal -offset indent
+https://github.com/jmmv/kyua/issues/new/
+.Ed
+.Pp
+When doing so, please include as much detail as possible.
+Among other things, explain what operating system and platform you are running
+.Nm
+on, what were you trying to do, what exact messages you saw on the screen,
+how did you expect the program to behave, and any other details that you
+may find relevant.
+.Pp
+Also, please include a copy of the log file corresponding to the problem
+you are experiencing.
+Unless you have changed the location of the log files, you can most likely
+find them in
+.Pa ~/.kyua/logs/ .
+If the problem is reproducible, it is good idea to regenerate the log file
+with an increased log level so as to provide more information.
+For example:
+.Bd -literal -offset indent
+$ kyua --logfile=problem.log --loglevel=debug \\
+ [rest of the command line]
+.Ed
+.Sh ENVIRONMENT
+The following variables are recognized and can be freely tuned by the end user:
+.Bl -tag -width COLUMNSXX
+.It Va COLUMNS
+The width of the screen, in number of characters.
+.Nm
+uses this to wrap long lines.
+If not present, the width of the screen is determined from the terminal
+stdout is connected to, and, if the guessing fails, this defaults to infinity.
+.It Va HOME
+Path to the user's home directory.
+.Nm
+uses this location to determine paths to configuration files and default log
+files.
+.It Va TMPDIR
+Path to the system-wide temporary directory.
+.Nm
+uses this location to place the work directory of test cases, among other
+things.
+.Pp
+The default value of this variable depends on the operating system.
+In general, it is
+.Pa /tmp .
+.El
+.Pp
+The following variables are also recognized, but you should not need to set them
+during normal operation.
+They are only provided to override the value of built-in values, which is useful
+when testing
+.Nm
+itself:
+.Bl -tag -width KYUAXCONFDIRXX
+.It Va KYUA_CONFDIR
+Path to the system-wide configuration files for
+.Nm .
+.Pp
+Defaults to
+.Pa __CONFDIR__ .
+.It Va KYUA_DOCDIR
+Path to the location of installed documentation.
+.Pp
+Defaults to
+.Pa __DOCDIR__ .
+.It Va KYUA_MISCDIR
+Path to the location of the installed miscellaneous scripts and data
+files provided by
+.Nm .
+.Pp
+Defaults to
+.Pa __MISCDIR__ .
+.It Va KYUA_STOREDIR
+Path to the location of the installed store support files; e.g., the
+directory containing the SQL database schema.
+.Pp
+Defaults to
+.Pa __STOREDIR__ .
+.El
+.Sh FILES
+.Bl -tag -width XXXX
+.It Pa ~/.kyua/store/
+Default location for the results files.
+.It Pa ~/.kyua/kyua.conf
+User-specific configuration file.
+.It Pa ~/.kyua/logs/
+Default location for the collected log files.
+.It Pa __CONFDIR__/kyua.conf
+System-wide configuration file.
+.El
+.Sh EXIT STATUS
+.Nm
+returns 0 on success, 1 on a controlled error condition in the given
+subcommand, 2 on a general unexpected error and 3 on a usage error.
+.Pp
+The documentation of the subcommands in the corresponding manual pages only
+details the difference between a successful exit (0) and the detection of a
+controlled error (1).
+Even though when those manual pages do not describe any other exit statuses,
+codes above 1 can be returned.
+.Sh SEE ALSO
+.Xr kyua.conf 5 ,
+.Xr kyuafile 5 ,
+.Xr atf 7 ,
+.Xr tests 7
+.Sh AUTHORS
+For more details on the people that made
+.Nm
+possible and the license terms, run:
+.Bd -literal -offset indent
+$ kyua about
+.Ed