Skip to content

Getting started

Install

Install the library into the Python environment of the program that uses it, as the home page shows.

The project under test keeps its own runtime and tools. A Python project needs an interpreter with pytest, plus coverage.py for coverage and Cosmic Ray for mutation, such as python -m pip install pytest coverage cosmic-ray in the project's environment, each pinned as name==version to the version Tested environments names. A Java project needs a JDK and the project's Maven or Gradle build (its wrapper or the tool on the PATH). A C project needs what C projects lists. On Ubuntu 24.04 on x86_64, also in WSL, langadapter toolchains python java c installs these tools, as Install the toolchains describes. The library downloads the JUnit launcher, JaCoCo and PIT into its cache on first use, unless LANGADAPTER_JAVA_TOOLS_OFFLINE is set to 1, as Java projects describes.

Run a suite

The examples are in the examples directory of the repository. A pip installation also puts them in share/doc/langadapter/examples inside the environment, the directory that python -c "import sys; print(sys.prefix)" prints. Run the commands below from the directory that holds examples, or give the examples' full path. Besides python_suite.py, which this section runs, they are python_mutation.py (Mutation sessions), metric_provider.py (A metric provider) and toy_adapter.py (The conformance check).

examples/python_suite.py runs a project's pytest suite and prints what the adapter observed:

python examples/python_suite.py path/to/project tests/test_example.py src/example.py

The arguments are the project directory, the test file and, optionally, the source file whose line coverage the example prints. The test file and the source file are paths inside the project directory. The example runs the project with the Python that runs the example, so install pytest, coverage.py and Cosmic Ray into the environment that holds LangAdapter before you run the examples. The tests must import the code the way pytest runs them from the project directory. With the code in src, as here, that takes pytest's own setting (pythonpath = ["src"] under [tool.pytest.ini_options] in pyproject.toml) or the project installed into that Python. Without either, the run reports collection_failed.

In outline, where project, test_file and target are the three arguments and interpreter is the path of the project's Python:

import langadapter as la
from langadapter.runs import SuiteRunRequest

context = la.ProjectContext(root=project, interpreter=la.Interpreter(path=interpreter))
adapter = la.create("python", context)
suite = la.Suite(project, (test_file,))
result = adapter.tests.run_suite(SuiteRunRequest(suite, timeout_s=120, coverage_target=target))
print(result.outcome, result.counts)

A ProjectContext names the project directory and the interpreter (or JDK) that runs it. A relative interpreter path starts at the current directory, and a path with no file there is refused with InvalidRequest when the adapter is created. create returns the adapter registered for the language. Every operation takes a typed request and returns a typed result.

Read the result

Every result of an operation that runs the project's tools carries an outcome and a reason. Source operations report a status (SourceStatus) instead. completed means the operation ran and its observation is valid. Test failures, a collection error, a timeout, a broken tool, missing setup and an unsupported request each have their own outcome, described in Outcomes, which also says how a value the adapter did not observe reads.

To check before running: adapter.operation_capabilities() lists what the adapter implements for the context without starting a tool, and inspect_project reports what a project's setup is missing: adapter.inspect_project(request) with a ProjectSetupRequest, or la.inspect_project("python", request). The request names the context, the source files and the operations the program needs, by the names operation_capabilities() lists (langadapter.capabilities.OPERATION_NAMES):

request = la.ProjectSetupRequest(context, ("src/example.py",), ("tests.run_suite",))
report = la.inspect_project("python", request)

The report's outcome and reason say whether the check could run. Each item is one thing the project's setup needs, such as the runtime or pytest, with its readiness, the reason when it is not available and the next step (action). Each operation entry says whether the operation can run here:

print(report.outcome, report.reason)
for item in report.items:
    print(item.key, item.required, item.readiness, item.reason, item.action)
run_suite = next(entry for entry in report.operations if entry.name == "tests.run_suite")
print(run_suite.readiness, run_suite.reason, run_suite.action)

See Environment selection.

Keep the project unchanged

Tests can write, change or remove files. To run them against a disposable copy, prepare the project first. An environment inside the project, such as its .venv, is copied too, which takes time: leave it out with prepare_project(context, inputs=ProjectInputSelection(exclude=(".venv",))), ProjectInputSelection from langadapter.

from pathlib import Path
from langadapter import Interpreter, ProjectContext, Suite, create, prepare_project
from langadapter.runs import SuiteRunRequest

context = ProjectContext("/projects/example", Interpreter(path="/tools/python"))
prepared = prepare_project(context)

with prepared.use() as scoped:
    adapter = create("python", scoped)
    suite = Suite(str(Path(scoped.root) / "tests"), ("test_example.py",))
    result = adapter.tests.run_suite(SuiteRunRequest(suite, timeout_s=60))

prepared.close()

A suite's root matters. Its test files are named relative to it, and test IDs and the suite's identity use those names. pytest's root is set to it, and pytest loads no conftest.py above it. Every conftest.py in the root or in a directory between the root and a test file runs. List each of them in the suite's setup_files: a run refuses a suite that leaves one out or lists one that pytest would not load. Rooting the suite at the project, as the first example does, is the general case, and a conftest.py in the project directory then runs. Here the suite is rooted at the copy's tests directory, which holds its test file, so a conftest.py in the project directory does not run.

Prepared copies describes what a prepared copy guarantees.

Stop an operation from another thread

An operation runs on the caller's thread. To stop it from another thread, for example from a Ctrl+C handler, run it in a CancelScope:

import threading
from pathlib import Path
from langadapter import (CancelScope, Interpreter, OperationCancelled, ProjectContext, Suite,
                         create, prepare_project)
from langadapter.runs import SuiteRunRequest

context = ProjectContext("/projects/example", Interpreter(path="/tools/python"))
prepared = prepare_project(context)
scope = CancelScope()

def measure():
    try:
        with scope:
            with prepared.use() as scoped:
                suite = Suite(str(Path(scoped.root) / "tests"), ("test_example.py",))
                create("python", scoped).tests.run_suite(SuiteRunRequest(suite, timeout_s=60))
    except OperationCancelled:
        print("cancelled")

worker = threading.Thread(target=measure)
worker.start()
# On this thread, or in a signal handler:
scope.cancel()
worker.join()
# Once the scope has ended:
prepared.close()

cancel() returns at once. Every process started in the scope is asked to end now, and nothing new starts. The operation raises OperationCancelled, an interrupt and no result, after its own cleanup removed its temporary files. Leaving the scope after a cancel raises it too, whatever the block returned. Close the prepared copy afterwards as usual, inside the scope or after it: cleanup runs despite the cancel. One scope may be entered on several threads at once. A thread you start inside the block is outside the scope unless it runs in contextvars.copy_context().

Next steps