This document describes the general steps on getting started with the LSB Distribution Testkit (LSB DTK).

1. Download the LSB Distribution Testkit package (tar.gz) for your architecture from the Downloads page.
2. Unpack the package using the following command:
   tar xzf lsb-dtk-<version>.<architecture>.tar.gz
3. Navigate to the subdirectory ./lsb-dtk created when unpacking the tarball.
4. Run the installation script:
   ./install.sh
   and follow the instructions. You may need to enter the root password.

Now you will need to run the LSB DTK Manager, by typing in the following command:
/opt/lsb/test/manager/bin/lsb-dtk-start.sh [port-number]

The port-number argument is optional (by default 8888 is taken). This script will start a simple web server embedded in the LSB DTK Manager and will try to locate and start your web browser with the proper address to this server.

Note: If the script could not find your web browser and you wish to use the web interface, please start the browser manually and specify the address http://localhost:8888/ (the port number should be the same as specified above).

Note: You may also connect to the LSB DTK Manager from any remote computer by specifying the URL http://<test-machine-name>:8888/ in your web-browser.

Using the Web Interface

In the browser you will see the main page of the LSB DTK Manager. It allows you to choose whether to perform certification tests or custom set of tests. Depending on your choice, either the Get Certified or Custom Tests page will be opened.

1. On the Get Certified page, you can see the following options:
   
   You should specify your name, organization, and (if you wish) e-mail address for sending the report. You may also change the architecture, LSB version, and other settings for the tests.
2. Click the Run the Tests button.
3. The Custom Tests page contains the extended set of options:
   
   The first time this page appears, the list contains the most essential test suites only. Click the Refresh List button to update the list from the LF's FTP server.
4. Specify necessary information in the edit fields
5. Check the boxes for the test suites you wish to execute.
6. Specify the source for every test suite selected. Localhost, FTP or FTP (beta) stand for the packages already downloaded and/or installed, located on the LF's FTP server or in the beta subdirectory of the LF's FTP server, respectively. See the detailed description on the Help page of the LSB DTK Manager.

Note: There are some prerequisites for the test suites to be able to run (listed in the /opt/lsb/test/manager/README file). If they are not met, an error message will be shown at the top of the page when you try to run the tests. In this case you will have to eliminate the problems and run the tests again.

7. Click the Run Selected Tests button at the bottom of the page. You will be redirected to the Execution page, where you will see the general status of the execution and the terminal emulation that shows on-the-fly output of the tests.
   
8. When the automated tests are finished, the Summary Report page will open. It contains just the general information about the status of the automatic and manual tests:
   
   
   • If manual tests need to be performed, a link is provided to the Manual Tests page. Depending on the tests selected and their status some additional information is shown on this page: what to do with failing tests, how to apply for certification, where to find the journal files and how to return back to this page after closing it. Also you can open the detailed report by clicking the corresponding link.
   • For a certification test run, this page contains the Apply for certification link which redirects you to the Certification System, where you will be able to use the test results for certifying your distribution.
   • For custom test run, you will see the Upload the test results link instead. You cannot start certification with these results, but you can use them as additional data for the product already being certified.

Other report pages are available to assist you:

• The Detailed Report page contains the exhaustive information about every test suite. Any tests you have failed that have already been waived will be marked as such in the report.
  
• On the Manual Tests page you can perform the tests which cannot be automated.
  
  You need to go through each of the tests shown in the left table (that is for each test the result should become either Passed or Failed). Then just press the Finish button and you will be redirected to the updated Summary Report page with new status of the manual test.
  Note: You do not need to perform all the manual tests at once. You can go through just some of them (or even none), close the browser and return to these tests later. The link for continuing the manual tests can be found on the Results page (see below).
• The list of all the reports generated for every test run is shown on the Results page:
  
  To view any particular summary report, just click the corresponding link in the Date/Time column. Additional icons in the right bottom corner allow you to quickly open the Summary Report page, Detailed Report page or, if you have not yet finished with the manual tests, to continue with them. To remove some of the reports, just select them with the check boxes and click the Remove Selected Entries button.

Using the Command Line Interface

Instead of using the web interface, you may use the command line interface provided by the LSB DTK extensions for lsb-autotest.

1. Navigate to the directory /opt/lsb/test/manager/autotest-ext.
2. To run the certification tests, execute the command (with root privileges):
   # ./lsb-tef.pl --cert
3. For saving the whole console log the `tee' utility can be used:
   # ./lsb-tef.pl --cert | tee /tmp/tests_output.8888
   If /tmp/tests_output.8888 file is used for saving, then this log file will be automatically copied to the result directory after the tests are finished.
4. To run all installed tests, execute the command (with root privileges):
   # ./lsb-tef.pl all
5. To run the custom set of tests, execute the command:
   # ./lsb-tef.pl test1 test2 ... 
   test1, test2 etc. are the specific test suites you wish to run. To obtain the list of supported test suites, use ./lsb-tef.pl --help.

In addition, the following options may be useful:

• -D: Automatically download the test packages needed.
• --lsb=<LSB-version>: Allows to explicitly specify the LSB version to test against.
• -T<profile>: Allows to specify the LSB profile to test against. Possible values are: core,c++ and core,c++,desktop (default).
• -r <results-dir>: Specify the directory for the results.
• -v[0-2]: Specify verbose mode.

You can use the Command Line interface for nightly test runs. There are detailed instructions on this topic.

Please see
 ./lsb-tef.pl --help 
for the full list of the options available in the current version. All the options are also described here: LSB DTK Manager Command Line Options.

Managing the Test Results Repository

The results for test runs are stored in the directory /var/opt/lsb/test/manager/results. All the files from each test run are stored in separate subdirectory named <arch>-<machine>-<date>-<time>. For the result to be visible on the Results page of the web interface it should be listed in the file /var/opt/lsb/test/manager/results/HISTORY.

If you are using the web or command line interface for running the tests, the HISTORY file is managed automatically. If you copy the test results from another machine or another directory by hand, you will need to write the directory names into this file manually.

Administering LSB DTK

• To stop the web-server, you may either run the script
   /opt/lsb/test/manager/bin/lsb-dtk-stop.sh 
  or go to the Administration page in the web interface and press the Stop server button.
• You can uninstall all packages installed by LSB DTK Manager using the command
   /opt/lsb/test/manager/autotest-ext/lsb-tef.pl --cleanup all 
• If you wish to uninstall the LSB DTK Manager as well, use your package manager (rpm or dpkg) to remove the package lsb-dtk-manager. After that you may also wish to delete the package cache, old results and other data from the /var/opt/lsb/test/manager directory.