git / moonbite-core / test/README.md · raw

This directory contains integration tests that test litecoind and itsutilities in their entirety. It does not contain unit tests, whichcan be found in [/src/test](/src/test), [/src/wallet/test](/src/wallet/test),etc.This directory contains the following sets of tests:- [functional](/test/functional) which test the functionality oflitecoind and litecoin-qt by interacting with them through the RPC and P2Pinterfaces.- [util](/test/util) which tests the litecoin utilities, currently onlylitecoin-tx.- [lint](/test/lint/) which perform various static analysis checks.The util tests are run as part of `make check` target. The functionaltests and lint scripts can be run as explained in the sections below.# Running tests locallyBefore tests can be run locally, Litecoin Core must be built.  See the [building instructions](/doc#building) for help.### Functional tests#### DependenciesThe ZMQ functional test requires a python ZMQ library. To install it:- on Unix, run `sudo apt-get install python3-zmq`- on mac OS, run `pip3 install pyzmq`#### Running the testsIndividual tests can be run by directly calling the test script, e.g.:```test/functional/feature_rbf.py```or can be run through the test_runner harness, eg:```test/functional/test_runner.py feature_rbf.py```You can run any combination (incl. duplicates) of tests by calling:```test/functional/test_runner.py <testname1> <testname2> <testname3> ...```Wildcard test names can be passed, if the paths are coherent and the test runneris called from a `bash` shell or similar that does the globbing. For example,to run all the wallet tests:```test/functional/test_runner.py test/functional/wallet*functional/test_runner.py functional/wallet* (called from the test/ directory)test_runner.py wallet* (called from the test/functional/ directory)```but not```test/functional/test_runner.py wallet*```Combinations of wildcards can be passed:```test/functional/test_runner.py ./test/functional/tool* test/functional/mempool*test_runner.py tool* mempool*```Run the regression test suite with:```test/functional/test_runner.py```Run all possible tests with```test/functional/test_runner.py --extended```By default, up to 4 tests will be run in parallel by test_runner. To specifyhow many jobs to run, append `--jobs=n`The individual tests and the test_runner harness have many command-lineoptions. Run `test/functional/test_runner.py -h` to see them all.#### Troubleshooting and debugging test failures##### Resource contentionThe P2P and RPC ports used by the litecoind nodes-under-test are chosen to makeconflicts with other processes unlikely. However, if there is another litecoindprocess running on the system (perhaps from a previous test which hasn't successfullykilled all its litecoind nodes), then there may be a port conflict which willcause the test to fail. It is recommended that you run the tests on a systemwhere no other litecoind processes are running.On linux, the test framework will warn if there is anotherlitecoind process running when the tests are started.If there are zombie litecoind processes after test failure, you can kill themby running the following commands. **Note that these commands will kill alllitecoind processes running on the system, so should not be used if any non-testlitecoind processes are being run.**```bashkillall litecoind```or```bashpkill -9 litecoind```##### Data directory cacheA pre-mined blockchain with 200 blocks is generated the first time afunctional test is run and is stored in test/cache. This speeds uptest startup times since new blockchains don't need to be generated foreach test. However, the cache may get into a bad state, in which casetests will fail. If this happens, remove the cache directory (and makesure litecoind processes are stopped as above):```bashrm -rf test/cachekillall litecoind```##### Test loggingThe tests contain logging at five different levels (DEBUG, INFO, WARNING, ERRORand CRITICAL). From within your functional tests you can log to these differentlevels using the logger included in the test_framework, e.g.`self.log.debug(object)`. By default:- when run through the test_runner harness, *all* logs are written to  `test_framework.log` and no logs are output to the console.- when run directly, *all* logs are written to `test_framework.log` and INFO  level and above are output to the console.- when run by [our CI (Continuous Integration)](/ci/README.md), no logs are output to the console. However, if a test  fails, the `test_framework.log` and litecoind `debug.log`s will all be dumped  to the console to help troubleshooting.These log files can be located under the test data directory (which is alwaysprinted in the first line of test output):  - `<test data directory>/test_framework.log`  - `<test data directory>/node<node number>/regtest/debug.log`.The node number identifies the relevant test node, starting from `node0`, whichcorresponds to its position in the nodes list of the specific test,e.g. `self.nodes[0]`.To change the level of logs output to the console, use the `-l` command lineargument.`test_framework.log` and litecoind `debug.log`s can be combined into a singleaggregate log by running the `combine_logs.py` script. The output can be plaintext, colorized text or html. For example:```test/functional/combine_logs.py -c <test data directory> | less -r```will pipe the colorized logs from the test into less.Use `--tracerpc` to trace out all the RPC calls and responses to the console. Forsome tests (eg any that use `submitblock` to submit a full block over RPC),this can result in a lot of screen output.By default, the test data directory will be deleted after a successful run.Use `--nocleanup` to leave the test data directory intact. The test datadirectory is never deleted after a failed test.##### Attaching a debuggerA python debugger can be attached to tests at any point. Just add the line:```pyimport pdb; pdb.set_trace()```anywhere in the test. You will then be able to inspect variables, as well ascall methods that interact with the litecoind nodes-under-test.If further introspection of the litecoind instances themselves becomesnecessary, this can be accomplished by first setting a pdb breakpointat an appropriate location, running the test to that point, then using`gdb` (or `lldb` on macOS) to attach to the process and debug.For instance, to attach to `self.node[1]` during a run you can getthe pid of the node within `pdb`.```(pdb) self.node[1].process.pid```Alternatively, you can find the pid by inspecting the temp folder for the specific testyou are running. The path to that folder is printed at the beginning of everytest run:```bash2017-06-27 14:13:56.686000 TestFramework (INFO): Initializing test directory /tmp/user/1000/testo9vsdjo3```Use the path to find the pid file in the temp folder:```bashcat /tmp/user/1000/testo9vsdjo3/node1/regtest/litecoind.pid```Then you can use the pid to start `gdb`:```bashgdb /home/example/litecoind <pid>```Note: gdb attach step may require ptrace_scope to be modified, or `sudo` preceding the `gdb`.See this link for considerations: https://www.kernel.org/doc/Documentation/security/Yama.txtOften while debugging rpc calls from functional tests, the test might reach timeout beforeprocess can return a response. Use `--timeout-factor 0` to disable all rpc timeouts for that partcularfunctional test. Ex: `test/functional/wallet_hd.py --timeout-factor 0`.##### ProfilingAn easy way to profile node performance during functional tests is providedfor Linux platforms using `perf`.Perf will sample the running node and will generate profile data in the node'sdatadir. The profile data can then be presented using `perf report` or a graphicaltool like [hotspot](https://github.com/KDAB/hotspot).To generate a profile during test suite runs, use the `--perf` flag.To see render the output to text, run```shperf report -i /path/to/datadir/send-big-msgs.perf.data.xxxx --stdio | c++filt | less```For ways to generate more granular profiles, see the README in[test/functional](/test/functional).### Util testsUtil tests can be run locally by running `test/util/bitcoin-util-test.py`.Use the `-v` option for verbose output.### Lint tests#### Dependencies| Lint test | Dependency | Version [used by CI](../ci/lint/04_install.sh) | Installation|-----------|:----------:|:-------------------------------------------:|--------------| [`lint-python.sh`](lint/lint-python.sh) | [flake8](https://gitlab.com/pycqa/flake8) | [3.8.3](https://github.com/bitcoin/bitcoin/pull/19348) | `pip3 install flake8==3.8.3`| [`lint-python.sh`](lint/lint-python.sh) | [mypy](https://github.com/python/mypy) | [0.781](https://github.com/bitcoin/bitcoin/pull/19348) | `pip3 install mypy==0.781`| [`lint-shell.sh`](lint/lint-shell.sh) | [ShellCheck](https://github.com/koalaman/shellcheck) | [0.7.1](https://github.com/bitcoin/bitcoin/pull/19348) | [details...](https://github.com/koalaman/shellcheck#installing)| [`lint-shell.sh`](lint/lint-shell.sh) | [yq](https://github.com/kislyuk/yq) | default | `pip3 install yq`| [`lint-spelling.sh`](lint/lint-spelling.sh) | [codespell](https://github.com/codespell-project/codespell) | [1.17.1](https://github.com/bitcoin/bitcoin/pull/19348) | `pip3 install codespell==1.17.1`Please be aware that on Linux distributions all dependencies are usually available as packages, but could be outdated.#### Running the testsIndividual tests can be run by directly calling the test script, e.g.:```test/lint/lint-filenames.sh```You can run all the shell-based lint tests by running:```test/lint/lint-all.sh```# Writing functional testsYou are encouraged to write functional tests for new or existing features.Further information about the functional test framework and individualtests is found in [test/functional](/test/functional).