Core¶
langadapter ¶
Test execution, coverage, mutation and timing observations for a configured project.
A caller describes the project in a ProjectContext and creates an
adapter for its language with create. Results of executing operations
carry an Outcome, source operations a SourceStatus. A value the
adapter did not observe is absent, never zero.
EvidenceItem
module-attribute
¶
EvidenceItem = Union[LineItem, BranchItem, KillItem, CrashItem]
One item of a test's evidence: a covered line, a covered branch, a killed
mutant or a mutant that ended the test's process (parse_token reads one
from its token).
LangAdapter ¶
LangAdapter(context: ProjectContext)
A language adapter bound to context. Subclasses set language
and file_extensions and implement the operations they support.
language
class-attribute
instance-attribute
¶
language: str = ''
The language this adapter handles, as it is registered.
support
class-attribute
instance-attribute
¶
support: LanguageSupport = UNDECLARED
What the language supports (langadapter.versions): its runtime's
tested and expected versions and their wording. Without a
declaration every version is untested.
file_extensions
class-attribute
instance-attribute
¶
file_extensions: Tuple[str, ...] = ()
The file extensions of the language's sources, with the leading dot.
source_execution_kinds
class-attribute
instance-attribute
¶
source_execution_kinds: Tuple[str, ...] = ('local',)
The execution kinds the execution-backed source operations support.
test_execution_kinds
class-attribute
instance-attribute
¶
test_execution_kinds: Tuple[str, ...] = ('local',)
test_evidence_granularity
class-attribute
instance-attribute
¶
test_evidence_granularity: Tuple[Tuple[str, Tuple[str, ...]], ...] = ()
The granularity of the evidence each test operation reports, as pairs of
the operation's name and its granularities: aggregate for the suite
as a whole, per_test for each test, such as
(("run_suite", ("aggregate", "per_test")), ("per_test_evidence",
("per_test",))). The operation report gives each test operation's
granularity (OperationCapability.evidence_granularity), empty for an
operation the pairs leave out, beside mutation_evidence_granularity
for the mutation operations.
native_mutation_engines
class-attribute
instance-attribute
¶
native_mutation_engines: Tuple[str, ...] = ()
mutation_execution_kinds
class-attribute
instance-attribute
¶
mutation_execution_kinds: Tuple[str, ...] = ('local',)
mutation_evidence_granularity
class-attribute
instance-attribute
¶
mutation_evidence_granularity: Tuple[str, ...] = ()
scratch_stem
class-attribute
instance-attribute
¶
scratch_stem: str = 'candidate'
The stem of a test file written for no particular target, as the
test_file_name template names it.
collection_unit
class-attribute
instance-attribute
¶
collection_unit: str = ''
The unit prune_collection_breakers removes: the smallest part of a
test source that the language's loading of tests (collection,
compilation) names as the cause of its failure.
ignored_dirs
class-attribute
instance-attribute
¶
ignored_dirs: Tuple[str, ...] = ('.git', '.venv', 'venv', 'env', 'node_modules', 'build', 'target', 'dist', '__pycache__', '.tox', '.mypy_cache', '.pytest_cache', 'tests', 'test')
The directories a search for the project's source files skips: the usual version control, environment, build, cache and test directories.
non_source_files
class-attribute
instance-attribute
¶
non_source_files: Tuple[str, ...] = ()
File names a search for source files leaves out although they carry the language's file extension: build scripts, not sources to test.
reuses_bytecode
class-attribute
instance-attribute
¶
reuses_bytecode: bool = False
Whether a prepared copy may reuse the modules compiled by runs in earlier copies.
copies_external_suite
class-attribute
instance-attribute
¶
copies_external_suite: bool = False
Whether a prepared copy takes a suite directory outside the project along, beside it, so a run prints the paths a run in place prints.
provides_timeout_journal
class-attribute
instance-attribute
¶
provides_timeout_journal: bool = True
Whether run_suite takes SuiteRunRequest.timeout_journal, the
per-test journal that names the test running when a timeout or crash
ended the run. False tells callers not to ask for it. An adapter whose
answer depends on the project sets it on the instance, so a consumer
reads it on the adapter, not on its class.
seeds_delivered_runs
class-attribute
instance-attribute
¶
seeds_delivered_runs: bool = False
Whether run_as_delivered takes SuiteRunRequest.session_seed,
so a consumer may run a suite as delivered with a fixed seed as its
normal run. False for one that refuses the seed or states nothing. An
adapter whose answer depends on the project sets it on the instance,
so a consumer reads it on the adapter, not on its class.
source_merge_separator
class-attribute
instance-attribute
¶
source_merge_separator: Optional[str] = None
The comment prefix of the separator line merge_sources puts
between two folded test files (#, //), None for no line.
validity_policy_version
class-attribute
instance-attribute
¶
validity_policy_version: Optional[str] = None
The version of the generated-test validity policy whose findings
analyze_validity reports, which callers record with a run, or
None without a policy.
own_mutant_timeout_rule
class-attribute
instance-attribute
¶
own_mutant_timeout_rule: bool = False
Whether the mutation tool sets each mutant's timeout itself, so that a
request leaves per_mutant_timeout to it unless one is fixed.
coverage_matrix_in_place
class-attribute
instance-attribute
¶
coverage_matrix_in_place: bool = False
Whether a coverage matrix is measured in the project itself rather than in a prepared copy.
project_settings
class-attribute
instance-attribute
¶
project_settings: Tuple[ProjectSetting, ...] = ()
The settings a project of the language takes (ProjectSetting).
inspect_project
class-attribute
instance-attribute
¶
inspect_project = _setup_entry(inspect_project, 'inspection')
What the project's setup has and lacks for the operations of
request (a ProjectSetupRequest naming the project), as
langadapter.inspect_project(language, request) reports it,
provisioning and building nothing. An override receives the request
the assessment prepared. The default reports no setup check
(Outcome.UNSUPPORTED).
verify_project
class-attribute
instance-attribute
¶
verify_project = _setup_entry(verify_project, 'verification')
The bounded native verification of the request's selected inputs,
as langadapter.verify_project(language, request) reports it. An
adapter overrides it as it does inspect_project.
test_vocabulary ¶
test_vocabulary() -> Mapping[str, str]
The language's words for its tests (langadapter.vocabulary) in
this project. Empty by default: callers then build plain sentences.
file_rule ¶
file_rule(source: str) -> Optional[str]
The stem a file holding source must have, or None when the
language puts no rule on file names (the default).
rename_file_unit ¶
rename_file_unit(source: str, new_name: str) -> Any
source with what file_rule names in it renamed to
new_name to fit a file of that stem, as a SourceObservation:
completed with the new source, or with the reason it cannot be
renamed safely. Completed and unchanged by default.
test_id_prefix ¶
test_id_prefix(source: str, stem: str) -> str
The prefix of the ids of the tests that source, a file named
stem, holds: the stem by default.
project ¶
project() -> ProjectResolution
Whether this adapter's project resolves (by default it does).
test_id_text ¶
test_id_text(test) -> str
A test's id as the language writes it in kill matrices and
records, such as demo.CalcTest.adds: the TestId's own text
by default.
kill_tokens ¶
kill_tokens(kill_matrix, test_ids)
Each test's kill tokens (K:<mutant id>) from the tests a
stored kill matrix records as each kill's killers (killing_tests,
Mutant.killing_tests), for the tests test_ids as
test_id_text writes them, and the killers that match none: a
killing test none of them is, a killer its run placed in no test,
and the killers of a row stored without killing tests.
pool_kills ¶
pool_kills(kill_matrix, file_to_index, test_ids_by_index)
Kill tokens by suite and test from a stored kill matrix's killing
tests: each goes to the suite file_to_index names for its file's
name, matched there among test_ids_by_index as kill_tokens
matches, or by its qualified name when the pooled file has another
name.
request_options ¶
request_options(purpose: str, *, target: Optional[str] = None, traceback: str = 'short', stop_at_first_failure: bool = False, record_leaks: bool = False, session_seed: Optional[int] = None, reverse_order: bool = False, leave_out_timing_plugins: bool = False) -> Dict[str, Any]
The fields this language adds to a request for one kind of run
(RUN_PURPOSES, else InvalidRequest): target is the module
under test. traceback (None for the purpose's own),
stop_at_first_failure and record_leaks apply to a run of the
tests and to a run as delivered, session_seed and reverse_order
to a run as delivered. leave_out_timing_plugins asks any run to
leave out the plugins that change its time, where the language has
such plugins. Empty by default.
delivered_order_refusal ¶
delivered_order_refusal(reverse: bool) -> Optional[str]
Why a run as delivered in the asked order is not offered, or None.
collection_text ¶
collection_text(run) -> str
What a failed collection says, for the caller's record: the run's diagnostic, else its reason.
salvage_report ¶
salvage_report(source: str) -> SalvageReport
The language's syntax repair of source
(sources.salvage_syntax) as a SalvageReport. Without one the
source is judged as it stands: valid when the language's
validator, else its structure check, accepts it or it has neither,
validator_failed with its reason when that check could not run,
else beyond_repair. A repair that could not run, for want of
time, setup or an allowed runtime, is validator_failed too.
skip_removal ¶
skip_removal(source: str) -> Tuple[str, int]
source without its whole-file skips and how many there were
(sources.strip_module_level_skips).
collision_dependents ¶
collision_dependents(source: str, name: str, measured: Any) -> Optional[set]
The measured tests of source that depend on its binding
name, or None when every test counts (the default).
collision_resolution ¶
collision_resolution(sources: Sequence[str], *, localize_autouse: bool = False, preference: Any = None) -> Any
The test sources' name collisions resolved before they are merged
into one, as a SourceObservation of a CollisionResolution. By
default sources.resolve_collisions' observation.
mutation_catalogue ¶
mutation_catalogue(config: Mapping[str, Any]) -> Any
The mutation operators the caller's config selects, read by the
language's own keys: the full catalogue by default.
named_operators ¶
named_operators(name: str) -> List[str]
The operators of the named operator set name: the name itself
by default, which the tool expands.
mutation_session_attempts ¶
mutation_session_attempts(config: Mapping[str, Any], *, configured_workers: int, effective_workers: int, cache_base: str, log: Any = None) -> Tuple[MutationAttempt, ...]
The ways to open a mutation session for config, in order: the
caller tries the next when one does not open. cache_base is the
directory for the tool's caches. By default one session with the
configured workers. ValueError says why config cannot run.
mutation_dir_refusal ¶
mutation_dir_refusal(config: Mapping[str, Any], *, create: bool) -> Optional[str]
Why the mutation tool's configured session directory cannot hold a session, or None (the default).
mutation_workers_refusal ¶
mutation_workers_refusal(config: Mapping[str, Any]) -> Optional[str]
Why the configured workers cannot run, or None (the default).
setup_files ¶
setup_files(root: str, test_paths: Sequence[str]) -> Tuple[str, ...]
The files the language's runner loads beside test_paths of the
suite at root, such as pytest's conftest.py. Empty by default.
shared_setup_files ¶
shared_setup_files(directory: str) -> Tuple[str, ...]
The setup files a prepared suite in directory holds. Empty by
default.
test_files ¶
test_files(paths: Sequence[str]) -> List[str]
Each path, with a directory replaced by the test files the
language's runner collects in it, at any depth: by default the files
the test_file_name template names.
project_test_files ¶
project_test_files() -> List[str]
The test files the language's runner collects in the project as
the project configures them, in test_files' order: by default
test_files of the context's root.
coverage_texts ¶
coverage_texts(coverage) -> Dict[str, Tuple[Any, ...]]
A run's coverage as text for the caller's records: the covered
lines and each missed branch as "source->destination", both empty
without coverage.
delivered_case ¶
delivered_case(test, outcome: str) -> Tuple[str, str, str]
A test of a run as delivered, as text for the caller's records: its
id (test_id_text), its name with its parameters, and its outcome.
import_statement
classmethod
¶
import_statement(target: str, root: str) -> Optional[str]
The line a first test of the source file target in the project
at root uses to import it, or, where the language's tests need
none, a sentence that says so (none; ..., as Java's and the
tutorial Toy's), which a consumer shows in place of its own import
advice. It is "" when it cannot tell, or None when the adapter offers none
(the default, and then sources.import_statement is unsupported).
A class method, asked before a project context exists. It probes
nothing of the project.
code_smells ¶
code_smells(paths: Sequence[str]) -> Optional[int]
The count of the language's linter findings in paths, or None
when the linter could not run or the adapter offers none (the
default, and then sources.code_smells is unsupported).
metric_providers ¶
metric_providers() -> Sequence[Any]
The language's own metric providers (langadapter.MetricProvider),
which every adapter object registers when its source operations are
created, after the structural provider. No providers by default.
discover_runtime
classmethod
¶
discover_runtime(root: str, values: Mapping[str, Any], *, container: bool = False, environment: Optional[Mapping[str, str]] = None, resolve_path: Any = None) -> RuntimeChoice
The runtime of a project at root from its setting values
(project_settings), for a container (container) or this machine,
with the caller's captured environment and its resolve_path
for a host path. A class method, asked before a project context and
an adapter exist. By default the runtime fact found on PATH.
describe_runtime
classmethod
¶
describe_runtime(root: str, values: Mapping[str, Any], *, container: bool = False, environment: Optional[Mapping[str, str]] = None, resolve_path: Any = None) -> RuntimeChoice
The runtime the setting values select, described without
checking or discovering it: for a setup that states the selection
before anything runs. A language that has nothing to check
describes what discover_runtime finds (the default).
context_environment
classmethod
¶
context_environment(values: Mapping[str, Any], *, container: bool = False) -> Dict[str, str]
The variables the language's tools read, from the setting
values, for a project context's environment. A class method like
discover_runtime. Empty by default.
provision_runtime
classmethod
¶
provision_runtime(request: Any, *, store: Path, name: str, environment: Optional[Mapping[str, str]] = None, log: Any = None) -> str
The runtime of an environment made for request (a
ProvisioningRequest: a version and a requirements file), kept as
name under the caller's store and reused while its inputs
stay the same, made with the caller's environment. log
receives progress lines. A class method like discover_runtime.
By default the language provisions nothing: InvalidRequest.
setup_questions
classmethod
¶
setup_questions(project_dir: Path, probe: Any, *, consumer: str = 'the caller') -> Iterator[Union[SetupQuestion, SetupNote]]
The questions consumer's setup asks for a project of the
language in project_dir, in order, and the notes between them, the
texts naming consumer where they name the program. probe
looks at the machine the caller runs on, whose platform decides the
file names a language looks for, such as a build wrapper's or a
virtual environment's interpreter: is_dir(path),
is_file(path), which(name), environment(name),
read_text(path) (None when unreadable) and
run(argv, timeout) (its exit code and output, or None when it
does not start). By default each declared setting
(project_settings) is asked with its purpose, its default and
its choices, and checked by its kind.
setup_problem
classmethod
¶
setup_problem(setting: str, value: str, project_dir: Path, probe: Any) -> Optional[str]
Why a setup refuses value for the declared setting in the
project project_dir, as a sentence, or None. A language that provides
it checks its settings itself. By default the setting's kind decides:
a choice must be one of its values, a path an existing file or directory,
an executable a file or a program probe.which finds, a relative
path read from project_dir. An empty value leaves the setting to the
language's discovery.
run_counts ¶
run_counts(run) -> Dict[str, Optional[int]]
A run's collected, passed, failed and errored tests as the language's runner counts them: the run's counts by default, with collected the passed, failed, errored and skipped tests together. A count the run did not observe is None, never 0.
listed_failures ¶
listed_failures(run) -> Optional[Tuple[Any, ...]]
The failed and errored tests (TestId) a suite run's report
lists, possibly none, or None when the report cannot be trusted, by
default when the run neither completed nor had test failures.
profile ¶
profile() -> VersionProfile
This instance's version profile (langadapter.versions), chosen
once by choose_profile() from the versions it found, which every
version-dependent question asks.
choose_profile ¶
choose_profile() -> VersionProfile
The VersionProfile of the versions this adapter found, which
profile() asks for once. An adapter that refuses an older runtime
returns a subclass whose refusal() gives the reason, and every
execution-backed operation then answers unsupported with it. The
base profile reads no runtime and refuses no version.
operation_capabilities ¶
operation_capabilities() -> Tuple[OperationCapability, ...]
Every operation as the adapter implements it, found without
probing tools. An unimplemented one is unsupported, with its reason
and no next step. probe_capabilities() adds probed readiness.
probe_capabilities ¶
probe_capabilities() -> CapabilityReport
The capability report with each supported operation's probed
readiness added to operation_capabilities(). A context without a
runtime, or a container context whose runner is not on PATH, reports
missing setup without starting a process, an adapter without a
capabilities probe reports unsupported, and the readiness of a pure
read stays unknown until it is tried.
operation_context_issue ¶
operation_context_issue(name: str) -> str
Why the operation name, such as sources.stage_pool, cannot
run in this context, found without starting a tool, or "". With a
reason, the operation is unsupported, and the operation report and
the operation's own answer give that reason in place of "operation
is not implemented".
execution_setup_issue ¶
execution_setup_issue() -> str
A prerequisite of running the project's tools that is known to be missing without starting a tool, such as a setting a container run needs, or "". With a reason, every execution-backed operation reports missing setup with it.
operation_readiness ¶
operation_readiness(name: str, report: CapabilityReport)
The readiness of the supported operation name from the probe's
report, as a Capability named like the operation, or None when
the probe did not check its native prerequisites.
probe_capabilities() reports it and refuses a Capability named
otherwise.
Availability ¶
Capability
dataclass
¶
Capability(name: str, availability: Availability, reason: str = '')
One operation's availability, with the reason when it is not available.
CapabilityReport
dataclass
¶
CapabilityReport(language: str, interpreter: str, outcome: Outcome, reason: str, runtime: Optional[str], capabilities: Tuple[Capability, ...], tools: Tuple[ToolVersion, ...], branch_coverage: str, mutation_engines: Tuple[str, ...], operations: Tuple[OperationCapability, ...] = (), version_record: Optional[Version] = None)
The capabilities of an adapter instance for its context. outcome says
whether the environment probe ran: completed, else with reason
infrastructure_failure, missing_setup (for every operation needing
the environment) or unsupported (no readiness probe). branch_coverage
is the coverage tool's branch evidence: arcs, counters or none.
version_record
class-attribute
instance-attribute
¶
version_record: Optional[Version] = None
The version whose text runtime is (versions.capability_runtime), or None.
operation ¶
operation(name: str) -> OperationCapability
The entry of the operation name (OPERATION_NAMES), not
available with the reason "operation is not implemented" when the
report holds none. An unknown name raises InvalidRequest.
capability ¶
capability(name: str) -> Capability
The entry of the capability name (CAPABILITIES). A name the
report does not hold raises KeyError.
LanguageFacts
dataclass
¶
LanguageFacts(language: str, test_framework: str, code_fence: str, runtime_version: Optional[str], test_file_patterns: Tuple[str, ...], test_function_pattern: str, test_class_pattern: Optional[str], timeout_marker: Optional[str], version_record: Optional[Version] = None, language_level: Optional[str] = None, unmatched_evidence_kept: bool = False, unsound_test_patterns: Tuple[str, ...] = (), test_language: Optional[str] = None, target_code_fence: Optional[str] = None)
Descriptive facts about an adapter's language and context.
runtime_version is the interpreter's own version line, None when
unreadable. timeout_marker is the framework's per-test timeout syntax
with {seconds} for the value, None where it has none. language_level
is the language version the tests are written in, perhaps below the
runtime's (Java 8 for a release target, Python 3.12), None when
unreadable. test_function_pattern shows how a test is written,
such as test_* or @Test, and test_class_pattern how a test
class is named, such as *Test, None where tests have no class.
unmatched_evidence_kept says whether a reduction keeps the
items of unmatched contexts (PerTestEvidence.unmatched_items): for Python
a test run under its own context but missing from the report, for Java a
JUnit id naming no test. unsound_test_patterns are the regular
expressions of the warnings that mark a test leaving shared state behind:
a suite run reports each such warning as an unsound_test entry of its
diagnostics, naming the test where the run knows it. Empty by default,
and the language's runs report none.
version_record
class-attribute
instance-attribute
¶
version_record: Optional[Version] = None
The version whose text runtime_version is (versions.facts_runtime), or None.
unsound_test_patterns
class-attribute
instance-attribute
¶
unsound_test_patterns: Tuple[str, ...] = ()
test_language
class-attribute
instance-attribute
¶
test_language: Optional[str] = None
The registered name of the language the tests are written in, where it
is not the adapter's own (python for Python tests of a C file).
None: the adapter's own language.
target_code_fence
class-attribute
instance-attribute
¶
target_code_fence: Optional[str] = None
The code fence of the target's source, where it is not code_fence,
the fence of the tests (c for Python tests of a C file). None:
code_fence.
OperationCapability
dataclass
¶
OperationCapability(name: str, supported: bool, reason: str = '', readiness: Optional[Availability] = None, execution_backed: bool = False, execution_kinds: Tuple[str, ...] = (), evidence_granularity: Tuple[str, ...] = (), resolution_methods: Tuple[str, ...] = (), action: Optional[str] = None, tried_by: Optional[str] = None)
Static implementation support, separate from probed readiness (None:
unprobed, not unsupported). A probed setup can make an implemented operation
unsupported. Resolution says how evidence is obtained, granularity what it
holds. action is the next step for an operation not ready, None where
nobody can act or reason names the remedy. tried_by names what first
tries a supported, untried operation: "verification" or "run".
evidence_granularity
class-attribute
instance-attribute
¶
evidence_granularity: Tuple[str, ...] = ()
ToolVersion
dataclass
¶
ToolVersion(name: str, pinned: Optional[str], found: Optional[str])
ContainerExecution
dataclass
¶
ContainerExecution(container: str, work_dir: str, runner: str = 'docker', repository_dir: Optional[str] = None, image: Optional[str] = None, container_work_dir: Optional[str] = None, helper_python: Optional[str] = None, control_environment: Optional[Environment] = None)
Project commands run inside a running container through its runner.
work_dir is the shared work directory: the host directory shared with the
container, holding every file handed to a command there. The container sees
it at container_work_dir: by default the same path, or /workzone for
a Windows host path. One under the host path, or containing it, is refused.
repository_dir is the project's source root in the container, image
its image for observation identity. helper_python selects the helper
Python, the container's Python 3.10 or newer that supervises commands. The
project's interpreter can serve, since the helper runs isolated and without
site (no .pth files or sitecustomize). Its Linux facilities are checked
before it starts a command, and with it none runs unsupervised. LangAdapter
installs no helper. Without one, commands run unsupervised through the
runner's exec, which captured environment variables, Java operations
and prepare_project refuse.
control_environment
class-attribute
instance-attribute
¶
control_environment: Optional[Environment] = field(default=None, repr=False)
The host runner's captured environment, never the container's target values.
Environment ¶
Environment(values: Optional[Mapping[str, str]] = None, *, inherit: Optional[bool] = None)
Bases: Mapping[str, str]
Immutable command variables over the caller's environment, or alone with
inherit=False. An Environment copied without a mode keeps its mode.
inherit
property
¶
inherit: bool
Whether the variables extend the caller's environment, False when they stand alone.
Interpreter
dataclass
¶
Interpreter(path: Optional[str] = None, provisioning: Optional[ProvisioningRequest] = None)
LocalExecution
dataclass
¶
LocalExecution(work_dir: Optional[str] = None)
ProjectContext
dataclass
¶
ProjectContext(root: str, interpreter: Optional[Interpreter] = None, execution: Execution = LocalExecution(), test_framework: TestFrameworkConfig = TestFrameworkConfig(), interpreter_flags: Tuple[str, ...] = (), environment: Mapping[str, str] = Environment(), runtime_version: Optional[str] = None, _prepared_copy_binding: object = None, _derivation: object = None)
A project and its toolchain.
root is the project's root as the executing environment sees it (local,
or inside the container). interpreter_flags go to every interpreter the
adapter starts for the project, except the processes of Cosmic Ray itself,
Python's mutation tool. The test runs those processes start take them.
environment overlays every project command's environment, or replaces
it as Environment(..., inherit=False). runtime_version, when given,
is reported instead of asking the interpreter. interpreter=None is
source-only use. Executing operations need one. For local execution a
relative root, and a relative interpreter path with a directory, are
made absolute from the current directory when the context is made.
execution
class-attribute
instance-attribute
¶
execution: Execution = field(default_factory=LocalExecution)
test_framework
class-attribute
instance-attribute
¶
test_framework: TestFrameworkConfig = field(default_factory=TestFrameworkConfig)
environment
class-attribute
instance-attribute
¶
environment: Mapping[str, str] = field(default_factory=Environment)
validate ¶
validate() -> None
Refuse an invalid or incomplete field, naming it, without starting a process: local paths must exist, container paths must be absolute.
derived ¶
derived(*, environment: Mapping[str, str], work_dir: str, extend: Optional[Mapping[str, str]] = None) -> 'ProjectContext'
A context for an adapter that runs another adapter on this project
(a C adapter running Python tests through Python's): the same root,
interpreter, flags and framework, environment's variables added
to this context's, and work_dir, a directory the caller made, as the
local work directory. A variable this context sets itself is refused,
unless extend names it with its separator, such as
{"PYTHONPATH": os.pathsep}: the added value comes first and this
context's value follows. A variable of the caller's that this context
inherits is overlaid, as its own variables overlay the caller's. A
container context is refused. Made from the context a prepared use
yields, it belongs to that use, as that context does. The owned
directories made in its work directory keep their creation receipts there
(work_dir_tempdir). Its observations record the variables as
given (ExecutionIdentity.environment), without this context's
part of an extended one, and with the work directory's path written
<work_dir>.
within_work_dir ¶
within_work_dir(path: str) -> bool
Whether path lies in the shared work directory (always true locally).
ProvisioningRequest
dataclass
¶
ProvisioningRequest(version: str, requirements: Optional[str] = None)
An interpreter the adapter provisions only when asked to: a language
version and an optional requirements file for the environment.
version is that version, and requirements the path of that
file.
TestFrameworkConfig
dataclass
¶
TestFrameworkConfig(source: str = 'project', path: Optional[str] = None)
ProjectInputSelection
dataclass
¶
ProjectInputSelection(exclude: tuple[str, ...] = (), required: tuple[str, ...] = ())
Entries to leave out of the project's inputs, and paths that must be present.
Paths are exact, never globs. An excluded directory leaves out its
subtree. Python's bytecode caches (__pycache__, .pyc, .pyo)
are always left out, as the copy leaves them out. Excluding a required path,
or requiring such a cache, raises InvalidRequest. Call effective (case rule, inputs known to be required)
before walking or copying the tree, and keep the project unchanged
meanwhile. The object never reads the file system.
effective ¶
effective(required=(), *, case_sensitive=True)
This selection with the project-relative paths required added
to its required paths, checked with case_sensitive as the case
rule. An exclusion that removes a required path raises
InvalidRequest.
excludes ¶
excludes(relative, *, case_sensitive=True)
Whether the selection leaves out the project-relative path
relative: a bytecode cache, or a path an excluded entry covers,
compared with case_sensitive as the case rule.
ExtensionFoundOnlyByImport ¶
ExtensionFoundOnlyByImport(field: str, message: str)
Bases: InvalidRequest
A dotted module item resolve_extension cannot find without running
a parent package, such as a module in a package whose code changes its
own search path: the finders show its top-level package but not the
module itself. Only load_extension, which imports the package, finds
the module or finds that it does not exist.
InvalidRequest ¶
InvalidRequest(field: str, message: str)
Bases: LangAdapterError, ValueError
A request or project context the adapter cannot interpret. field
names the offending field, such as execution.helper_python.
LangAdapterError ¶
Bases: Exception
The base of the package's errors. Cancellations derive from KeyboardInterrupt.
OperationCancelled ¶
Bases: KeyboardInterrupt
An operation its CancelScope stopped, raised after its own cleanup.
Like an interrupt it is no failure and no result, never caught as an
ordinary error. An error the operation raised itself is its cause.
OwnershipRefused ¶
Bases: LangAdapterError, RuntimeError
A refusal to change or remove files whose ownership is unconfirmed: a
prepared copy or scratch directory no longer matching its ownership record,
or a cleanup of created files in a context prepare_project did not make.
RegistrationConflict ¶
RegistrationConflict(claimed: str, holder: str)
Bases: LangAdapterError, ValueError
A registration claims a language name or file extension that another
adapter holds. holder names that adapter.
SourceReadError ¶
SourceReadError(message: str, parser: Optional[str] = None)
Bases: LangAdapterError, ValueError
A source the parser of a source method (check_structure) cannot read,
raised instead of an empty result. The message says where, parser names
the parser. Typed source operations report SourceStatus.PARSE_ERROR.
UnsupportedLanguage ¶
UnsupportedLanguage(requested: str, registered: Iterable[str])
Bases: LangAdapterError, LookupError
No adapter is registered for the requested language or file.
registered lists the languages that are.
CancelScope ¶
CancelScope()
The operations a caller may stop from another thread: enter it with with
around them, call cancel to stop them. A cancelled scope stays cancelled:
every later operation in it starts nothing and raises OperationCancelled.
ExecutionIdentity
dataclass
¶
ExecutionIdentity(kind: str, image: Optional[str] = None, supervision: Optional[str] = None, environment: Optional[Tuple[Tuple[str, str], ...]] = None)
Where the observation ran: local, or container with its image if
known. supervision is pidfd, process_group (no process
descriptors), unreported (a container helper before its first command),
process_group_unconfirmed (a non-Linux POSIX host, which ends a group
without confirming every descendant ended), or None (unsupervised).
environment holds the variables a derived context added
(ProjectContext.derived), with its work directory written <work_dir>,
or None for a context that was not derived.
FileIdentity
dataclass
¶
FileIdentity(path: str, sha256: str)
A file by its relative POSIX path and the SHA-256 of its bytes.
InstrumentIdentity
dataclass
¶
InstrumentIdentity(name: str, version: str)
ObservationIdentity
dataclass
¶
ObservationIdentity(target: Optional[FileIdentity], suite: Tuple[FileIdentity, ...], toolchain: ToolchainIdentity, instrument: Optional[InstrumentIdentity], execution: ExecutionIdentity)
The identity every observation carries. target is None without a
target (a suite run without coverage), instrument when none ran.
to_dict ¶
to_dict() -> Dict[str, Any]
The identity as plain data, each part in its own to_dict
form, None for an absent target or instrument.
ToolchainIdentity
dataclass
¶
ToolchainIdentity(implementation: str, version: str, tools: Tuple[Tuple[str, str], ...] = (), structural_parser: Optional[str] = None)
The interpreter's implementation and version, each tool's version by distribution, and the source operations' parser and grammar, if one ran.
of
classmethod
¶
of(version: Any, tools: Tuple[Tuple[str, str], ...] = (), structural_parser: Optional[str] = None) -> 'ToolchainIdentity'
The identity of a toolchain whose runtime is version (a
langadapter.versions.Version), with its version text as found.
recorded
classmethod
¶
recorded(implementation: str, version: str, tools: Tuple[Tuple[str, str], ...] = (), structural_parser: Optional[str] = None) -> 'ToolchainIdentity'
An identity read back from a record, with its texts as recorded.
to_dict ¶
to_dict() -> Dict[str, Any]
The identity as plain data, with tools as a mapping from each
tool to its version.
Outcome ¶
Bases: str, Enum
What happened when an operation executed a tool.
COMPLETED
class-attribute
instance-attribute
¶
COMPLETED = 'completed'
The operation ran and its observation is valid. For a suite run, no collected test failed or raised an error (skips and expected failures aside).
TEST_FAILURES
class-attribute
instance-attribute
¶
TEST_FAILURES = 'test_failures'
The suite ran and at least one test failed or raised an error.
COLLECTION_FAILED
class-attribute
instance-attribute
¶
COLLECTION_FAILED = 'collection_failed'
An error during collection stopped the run before its tests ran.
TIMED_OUT
class-attribute
instance-attribute
¶
TIMED_OUT = 'timed_out'
The operation reached its timeout.
INFRASTRUCTURE_FAILURE
class-attribute
instance-attribute
¶
INFRASTRUCTURE_FAILURE = 'infrastructure_failure'
A tool crashed, its report is unreadable, a session or file it needs is gone, or an instrument check failed.
UNSUPPORTED
class-attribute
instance-attribute
¶
UNSUPPORTED = 'unsupported'
The language, engine or context does not provide the operation.
MISSING_SETUP
class-attribute
instance-attribute
¶
MISSING_SETUP = 'missing_setup'
A tool or file the operation needs is absent from the project environment.
ProjectSetupItem
dataclass
¶
ProjectSetupItem(key: str, required: bool, readiness: Optional[Availability], outcome: Optional[Outcome], selected: Optional[str] = None, version: Optional[str] = None, reason: str = '', action: Optional[str] = None, candidates: tuple[str, ...] = (), ambiguous: bool = False, tried_by: Optional[str] = None, version_record: Optional[Version] = None)
One prerequisite the check assessed. An outcome or readiness of None
means not tried or unknown. A selected path shows neither readiness nor
version. Reasons describe the assessment, never raw tool output.
tried_by names what first tries an untried item: "verification",
"run" (only a real run), or None.
candidates
class-attribute
instance-attribute
¶
candidates: tuple[str, ...] = ()
What the user selects from: when ambiguous is True, the choices found
(interpreters, build systems), else interpreter addresses the check did not
find or did not try.
ambiguous
class-attribute
instance-attribute
¶
ambiguous: bool = False
Whether the check found more than one choice, which candidates holds,
and the user selects one.
version_record
class-attribute
instance-attribute
¶
version_record: Optional[Version] = None
A runtime item's version, which version is the text of
(versions.setup_runtime), or None. A tool's version is its text.
ProjectSetupReport
dataclass
¶
ProjectSetupReport(scope: str, outcome: Outcome, reason: str, context: Optional[ProjectContext], inputs: Optional[ProjectInputSelection], items: tuple[ProjectSetupItem, ...], operations: tuple[OperationCapability, ...], diagnostic_paths: tuple[str, ...] = ())
Facts about an inspection or verification, not a promise of successful tests.
COMPLETED means the assessment ran. Required missing or unverified items stay
explicit. safe_export formats it, leaving out the selected context.
diagnostic_paths holds the paths of diagnostics as distinct texts,
which safe_export passes on with secrets redacted. The shipped
adapters leave it empty.
safe_export ¶
safe_export()
The statuses and paths as plain data, without environment or tool output.
ProjectSetupRequest
dataclass
¶
ProjectSetupRequest(context: ProjectContext, source_files: tuple[str, ...], required_operations: tuple[str, ...], inputs: Optional[ProjectInputSelection] = None, probe_timeout_s: float = 10.0, total_timeout_s: float = 120.0, allow_acquisition: bool = False, required_facets: tuple[tuple[str, tuple[str, ...]], ...] = ())
One explicit project selection and finite setup assessment scope.
Source paths are normalized project-relative inputs. Inspection never
acquires tools. Verification does only when allow_acquisition asks. The
deadline starts at each assessment, so a request is reusable.
required_facets names measurements required of a required operation
(OPERATION_FACETS), such as {"tests.run_suite": ("coverage",)}.
context: the project context to assess.source_files: the project-relative source files to assess, which the input selection requires.required_operations: the operations the project needs, by their names inOPERATION_NAMES. The prerequisites they depend on are required items, and the report states each of these operations.inputs: the entries to leave out of the project's inputs and the paths that must be present, None for none.probe_timeout_s: the time limit of one probe in seconds, capped by what is left oftotal_timeout_s.total_timeout_s: the time limit of one assessment in seconds, counted from its start.
required_facets
class-attribute
instance-attribute
¶
required_facets: tuple[tuple[str, tuple[str, ...]], ...] = ()
facets ¶
facets(operation: str) -> tuple[str, ...]
The measurements required of operation, empty when none are.
remaining_timeout ¶
remaining_timeout(stage='project setup', *, probe=True) -> float
The per-probe bound capped by this assessment's remaining deadline,
asked before each scan, probe, build or copy. probe=False gives all
the remaining time (for a build or copy) instead of a version probe's
short limit. Cleanup after a confirmed stop has its own allowance.
MetricDefinition
dataclass
¶
MetricDefinition(name: str, unit: str, definition_version: str, scopes: Tuple[str, ...])
What one metric measures: its name, its unit, the
definition_version that changes whenever its meaning does, and the
scopes it is reported for, file, function or both.
MetricProvider ¶
Bases: Protocol
A metric provider: a name, its metric definitions(), and
measure(source, *, language, parser), which returns the scopes it
measured as a source observation, unavailable (with no number) for
source the provider cannot read.
definitions ¶
definitions() -> Tuple[MetricDefinition, ...]
The definitions of the metrics the provider measures.
measure ¶
measure(source: str, *, language: str, parser: Optional[str]) -> SourceObservation[Tuple[MetricScope, ...]]
The scopes the provider measured in source, unavailable with
no number when the provider cannot read it.
MetricScope
dataclass
¶
MetricScope(kind: str, name: str, start_line: Optional[int], end_line: Optional[int], signature: Optional[str], metrics: Tuple[MetricValue, ...])
The metrics of one scope of a source. A file scope has no name,
span or signature. A function scope has its name, first and last
line (1-based, inclusive) and signature, if the language has one.
metrics holds at most one value per metric defined for the scope.
start_line and end_line are that first and last line, None for
a file scope.
MetricValue
dataclass
¶
MetricValue(definition: MetricDefinition, provider: str, parser: Optional[str], status: SourceStatus, value: int | float | None, reason: str = '')
One metric's observation of one scope by provider with parser.
A completed status carries a finite number as value, any other
none, and reason says why.
SourceMetrics
dataclass
¶
SourceMetrics(language: str, file: MetricScope, functions: Tuple[MetricScope, ...], unavailable: Tuple[MetricValue, ...] = ())
Every metric of one source: the file scope, the functions
scopes, and in unavailable the metrics no scope could report and
why.
SourceObservation
dataclass
¶
SourceObservation(status: SourceStatus, value: Optional[_T] = None, parser: Optional[str] = None, reason: str = '')
Bases: Generic[_T]
A completed observation with its value (False and empty count), or an unavailable one without. A validator that failed to run gives no verdict, unlike one that judged the source invalid.
SourceStatus ¶
Bases: str, Enum
A source observation's completion or reason for unavailability.
INFRASTRUCTURE_FAILURE
class-attribute
instance-attribute
¶
INFRASTRUCTURE_FAILURE = 'infrastructure_failure'
Extension
dataclass
¶
Extension(item: str, kind: str, file: str, sha256: str, distribution: Optional[str] = None, version: Optional[str] = None, languages: Tuple[str, ...] = (), metric_providers: Tuple[str, ...] = (), module: Optional[str] = None)
An extension item as found: the item as named, its kind
(file or module), the resolved file (a package's
__init__.py) and the SHA-256 of its bytes, and for a module of an
installed distribution that distribution and its version. After a
load it also names the languages and metric_providers it
registered and the module it was imported as.
Registration
dataclass
¶
Registration(name: str, file_extensions: Tuple[str, ...], factory: AdapterFactory, extension: Optional[Extension] = None)
A registered language: its name, file extensions and instance factory,
and the extension it was loaded from, None for a built-in language or
one registered by register.
BranchItem
dataclass
¶
BranchItem(file: str, source: int, destination: int)
A covered branch from line source to destination. A negative one
leaves the code object starting at line -destination, as coverage records it.
leaves_code_object
property
¶
leaves_code_object: bool
Whether the branch leaves a code object: a negative destination.
CrashItem
dataclass
¶
CrashItem(mutant: str)
KillItem
dataclass
¶
KillItem(mutant: str)
LineItem
dataclass
¶
LineItem(file: str, line: int)
Suite
dataclass
¶
Suite(root: str, test_files: Tuple[str, ...], setup_files: Tuple[str, ...] = ())
Test files and setup files under root, a directory of this machine,
resolved against the current directory when relative. Paths may be absolute
or relative to root, and each must name a file under it.
TestId
dataclass
¶
TestId(file: str, qualname: str, params: Optional[str] = None, text: str = '')
One test: its POSIX file relative to the suite root, its qualified name
in the file (Class.test for a method) and its parameters if any.
text is the language's canonical form, set by the adapter (for Python
the node id path::Class::test[params]). Identifiers compare by parts.
SourceOperations ¶
SourceOperations(adapter)
The source operations of an adapter, adapter.sources. Each calls
the adapter's method of its name, such as adapter.check_structure,
and answers unsupported without one. An adapter implements those
methods or supplies a subclass. Registered metric providers merge into
immutable results. Collision validation that could not run is distinct
from a completed resolution that rejected a source.
execution_backed ¶
execution_backed(name: str) -> bool
Whether the source operation name runs the project's tools: the
validation, the edits that validate their result, and
target_identity in a container context.
supports ¶
supports(name: str) -> bool
Whether the source operation name is implemented for this
adapter and its context's kind of execution. Nothing runs to find
out.
execution_kinds ¶
execution_kinds(name: str) -> Tuple[str, ...]
The kinds of execution (local, container) the source
operation name runs in for this adapter, none for an operation
that runs no project tool.
check_structure ¶
check_structure(source: str) -> SourceObservation[StructureCheck]
Whether the parser reads source without an error, with the first
error's message, place and text (StructureCheck).
normalize_bindings ¶
normalize_bindings(source: str) -> SourceObservation[SuiteNormalization]
The static normalization of shadowed bindings with every removal. A source that does not parse is kept as it is, and the project's runtime does not validate the rewrite.
test_metrics ¶
test_metrics(source: str) -> SourceObservation[TestMetrics]
The counts of the test source source (TestMetrics):
parse_error when the parser rejects it.
register_metric_provider ¶
register_metric_provider(provider: MetricProvider, *, replace: bool = False) -> None
Register provider's metric definitions on this adapter object.
A name already registered raises InvalidRequest unless
replace is true, which rebinds only the provider's own names.
metric_definitions ¶
metric_definitions() -> Tuple[MetricDefinition, ...]
The definition of every metric metrics measures on this adapter
object, by name. langadapter.sources.STANDARD_METRICS names the
operation that reports each standard metric. A listed name it does
not hold is the language's own metric of the source text.
metrics ¶
metrics(source: str, *, names: Optional[Sequence[str]] = None) -> SourceObservation[SourceMetrics]
The metrics of source that metric_definitions lists, or
those names names (SourceMetrics). unsupported when no
metric is registered.
project_source_files ¶
project_source_files() -> SourceObservation[Tuple[str, ...]]
The project's non-test source files, as sorted POSIX paths relative
to the context's root: the files with the language's file_extensions
outside its ignored_dirs and the directories whose name starts with
a dot, leaving out its non_source_files, the files
is_test_file names and empty files. A file that does not parse
stays in the list, since no file's content is read. The files are
read on the host, so a root or directory the host cannot read, such as
a container context's root outside the host, is an infrastructure
failure.
call_statistics ¶
call_statistics(target: str) -> SourceObservation[SourceMetrics]
The call statistics of the source file target, a path relative
to the context's root as project_source_files lists it. For each
function the target declares, call_site_count_estimate counts the
call sites of its bare name in the files project_source_files
lists, the target left out, so test files are not searched, and
calling_file_count_estimate counts the files that hold one. The
file scope holds their sums over the functions, each calling file
counted once. Names are matched, not resolved: a call of another
function with the same bare name counts, and an import under another
name, a reference that is not a call and a dynamic call do not. A
project file that does not parse is left out and named in the values'
reason. A target outside the root is refused as an invalid request.
skeleton ¶
skeleton(source: str) -> SourceObservation[Tuple[str, ...]]
The names of the definitions source declares, such as its
functions and classes, as the language qualifies them.
same_skeleton ¶
same_skeleton(first: str, second: str) -> SourceObservation[Tuple[bool, str]]
Whether first and second declare the same definition names,
as a (verdict, reason) pair.
function_fingerprints ¶
function_fingerprints(source: str) -> SourceObservation[Tuple[Tuple[str, str], ...]]
A fingerprint of each definition source declares, as (name,
fingerprint) pairs sorted by name.
imports ¶
imports(source: str) -> SourceObservation[Tuple[str, ...]]
The names source imports, in source order.
import_path_candidates ¶
import_path_candidates(name: str) -> SourceObservation[Tuple[str, ...]]
The source paths an absolute import of name may refer to, found
without resolving the import.
import_statement ¶
import_statement(target: str) -> SourceObservation[str]
The line a first test of target uses to import it, or a
sentence (none; ...) where the language's tests need none, for
this context's project, or "" when the adapter cannot tell.
code_smells ¶
code_smells(paths: Sequence[str]) -> SourceObservation[int]
The count of the language's linter findings in paths.
missing_setup when the linter could not run.
duplicate_test_names ¶
duplicate_test_names(source: str) -> SourceObservation[Tuple[str, ...]]
The names of the tests source defines more than once in one
scope, where the later definition hides the earlier.
merge_collisions ¶
merge_collisions(first: str, second: str) -> SourceObservation[Tuple[str, ...]]
The top-level names both first and second define, on which a
merge of the two collides. Nothing is rewritten.
is_test_file ¶
is_test_file(path: str) -> SourceObservation[bool]
Whether the file path is a test file of the language, by its
name.
validate_source ¶
validate_source(source: str) -> SourceObservation[SourceValidation]
Whether the project's own compiler or interpreter accepts source
(SourceValidation).
target_identity ¶
target_identity(target: str) -> SourceObservation[TargetIdentity]
The identity of the file under test target that the validity
policy reads (TargetIdentity), such as the name its tests import
it by.
resolve_collisions ¶
resolve_collisions(sources: Sequence[str], *, preference: Optional[Callable[[str, Tuple[int, ...]], int]] = None, localize_autouse: bool = False) -> SourceObservation[CollisionResolution]
The top-level name collisions among sources resolved before
they are merged into one file (CollisionResolution). preference
takes a name that several sources bind in a way no rename can change
and the indexes of those sources, and returns the index whose
binding stays. Without it each such name is reported as a conflict
and nothing is dropped. localize_autouse keeps each Python
source's module-level autouse fixtures to that source's own tests.
stage_pool ¶
stage_pool(members: Sequence[PoolMember], *, setup: Sequence[Tuple[str, str]] = ()) -> SourceObservation[StagedPool]
Several suites' test files staged as one pooled suite, with the
way back to each suite's standalone identities (StagedPool).
setup holds the setup files every suite shares, as (relative
path, text) pairs. A pool the language cannot stage is an
observation with its reason.
analyze_validity ¶
analyze_validity(source: str, target: TargetIdentity) -> SourceObservation
The validity policy's findings on the test source source for the
file under test target (PolicyReport): the places where a
test reads the implementation's source or internals instead of
calling it.
screen_source ¶
screen_source(source: str, target: TargetIdentity) -> SourceObservation
The validity policy's review-only signals on the test source
source for the file under test target (PolicyReport).
prune_tests ¶
prune_tests(source: str, tests: Iterable[TestId | str]) -> SourceResult
source without the tests tests names, given as names or
TestId values (SourceResult).
merge_sources ¶
merge_sources(first: str, second: str, *, separator: Optional[str] = None) -> SourceResult
first and second joined as one test file, with the comment
line separator between them (SourceResult). Without
separator the adapter's source_merge_separator is used, and
an adapter without one needs it. A given separator must start with
the adapter's source_merge_separator where it has one.
normalize_shadowed ¶
normalize_shadowed(source: str) -> SourceResult
source without the definitions a later one of the same name
hides (SourceResult).
salvage_syntax ¶
salvage_syntax(source: str) -> SourceResult
A bounded repair of a source the project's compiler or interpreter
rejects (SourceResult).
prune_unresolvable_imports ¶
prune_unresolvable_imports(source: str) -> SourceResult
source without the imports that do not resolve in the project
(SourceResult).
prune_collection_breakers ¶
prune_collection_breakers(source: str, *, timeout_s: float) -> SourceResult
source without what keeps its tests from loading, so the rest
run. Each of at most five rounds loads the tests as the language
does (collection, compilation) and removes the smallest unit the
failure names (the adapter's collection_unit), and removed
names each. A pruned source is returned only once its tests load.
unchanged when they load as they are, not_repaired with the
input when they still do not.
strip_module_level_skips ¶
strip_module_level_skips(source: str) -> SourceResult
source without the statements or annotations that skip the whole
test file (SourceResult).
finalize_source ¶
finalize_source(source: str) -> SourceResult
source without detached comment blocks (SourceResult).
TestOperations ¶
TestOperations(adapter)
The test operations of an adapter, adapter.tests, each passing its
request unchanged to the adapter's own method. Making the group and
asking what it supports run no tools. An unsupported operation or a
missing setup gives unknown counts, never an empty success. Setup files
and instruments come from the adapter's own methods.
supports ¶
supports(name: str) -> bool
Whether the operation name is implemented for this adapter and
its context's kind of execution. Nothing runs to find out.
execution_backed ¶
execution_backed(name: str) -> bool
Whether name is a test operation, which runs project tools.
execution_kinds ¶
execution_kinds(name: str) -> tuple[str, ...]
The kinds of execution (local, container) the operation
name runs in for this adapter.
evidence_granularity ¶
evidence_granularity(name: str) -> tuple[str, ...]
The granularities of the evidence the operation name reports
for this adapter, empty when it reports none.
run_suite ¶
run_suite(request: SuiteRunRequest) -> SuiteRun
Run the request's suite once and report every test, the counts
and, with a coverage_target, that file's coverage. An operation
the adapter does not offer is unsupported and a missing runtime
or tool is missing_setup, both with unknown counts.
collect_suite ¶
collect_suite(request: CollectionRequest) -> CollectionRun
Collect the request's suite without running its tests. Collection still runs imports, setup files and collection hooks.
run_as_delivered ¶
run_as_delivered(request: SuiteRunRequest, *, sources: Optional[Mapping[str, str]] = None) -> SuiteRun
Run each test file of the suite on its own, as delivered: with its
import setup and without the adapter's measurement instrument.
sources maps a declared test file's relative POSIX path to the
text to run in its place, without changing the file.
request.timeout_s bounds the whole run: each file runs in what is
left of it, and a run whose time is spent ends timed out before the
next file.
benchmark ¶
benchmark(request: BenchmarkRequest) -> Benchmark
Time repeated runs of the request's suite, after discarded
warm-up runs. The result's includes states what each time
covers: the runner's process and, in a container, the container's
or the execution wrapper's overhead.
profile_suite ¶
profile_suite(request: SuiteProfileRequest) -> SuiteProfile
Run the request's suite once under the language's profiler and
attribute the run's time to the functions it entered. A language
without it answers unsupported and runs nothing.
benchmark_warmed ¶
benchmark_warmed(request: WarmedSuiteBenchmarkRequest) -> WarmedSuiteBenchmark
Time the suite in a warmed runtime. A language without it answers
unsupported and runs nothing.
per_test_evidence ¶
per_test_evidence(request: PerTestEvidenceRequest) -> PerTestEvidence
Each test's covered items and time from one instrumented run, and with a mutation run, the kills attributed to each test.
MutationOperations ¶
MutationOperations(adapter)
The mutation operations of an adapter, adapter.mutations. The
adapter names its engines and where each runs, provides their catalogue
and session methods, and checks each engine's options. Listing the
engines does not check that they are installed. All sessions share one
lifecycle.
engines ¶
engines() -> tuple[str, ...]
The names of the mutation engines this adapter implements, without checking that they are installed.
execution_backed ¶
execution_backed(name: str) -> bool
Whether the mutation operation name runs the project's tools.
Listing the engines does not.
execution_kinds ¶
execution_kinds(name: str) -> tuple[str, ...]
The kinds of execution (local, container) the mutation
operation name runs in for this adapter.
supports ¶
supports(name: str) -> bool
Whether the mutation operation name is implemented for this
adapter and its context's kind of execution. Nothing runs to find
out.
evidence_granularity ¶
evidence_granularity(name: str) -> tuple[str, ...]
The granularities of the evidence a session's run or
per_suite reports for this adapter, empty for the other
operations.
resolution_methods ¶
resolution_methods(name: str) -> tuple[str, ...]
How the verdicts of run and per_suite are resolved:
measured, by running the tests.
operators ¶
operators(engine: str) -> OperatorCatalogue
The mutation operators engine offers in the project's
environment. An engine the adapter does not offer here is
unsupported, and a missing runtime or engine is
missing_setup.
open_session ¶
open_session(engine: str, *, workers: int = 1, distributor: Optional[str] = None, allow_fewer_workers: bool = False, settings: object = None) -> MutationSession
A mutation session of engine for the project, to run
measurements in and close (with closes it). The options are
those the settings guide lists: workers, and for Cosmic Ray
distributor, allow_fewer_workers and settings.
PreparedProject ¶
PreparedProject(context: ProjectContext, prepared_copy: PreparedCopy, *, incomplete=False)
A prepared copy of a project, made by prepare_project, that one
use at a time may run commands in. Each use starts from the copy's
original state. A context or adapter kept from a use cannot run commands
after that use ends, nor in a later use. A successful close()
removes the copy and its snapshot, so results that must last do not
belong under root.
root
property
¶
root: str
The copy's root as its commands see it: a path of this machine, or its path in the container when the project runs in one.
inputs
property
¶
inputs: Optional[ProjectInputSelection]
The effective immutable input selection, or None for a whole-tree copy.
companion_path ¶
companion_path(path: str) -> Optional[str]
Where a path in a companion directory lies in the copy, as a host path, or None for a path in no companion.
use ¶
use() -> Iterator[ProjectContext]
Begin a use of the copy and yield its ProjectContext, rooted
at the copy, for the adapters of this use. The copy starts from its
original state. A use of a closed prepared project, a use while
another is active and a nested use of the same prepared project are
refused.
worker ¶
worker(execution_context: ProjectContext) -> Iterator[ProjectContext]
Lend the current use to one worker thread, without a new lease or
restoring the snapshot. Pass the context use() yielded. A second
worker on the same use is refused, not queued. Close the worker's
sessions before the use ends, and keep an owner whose cleanup failed
until its cleanup succeeds.
stage_suite ¶
stage_suite(execution_context: ProjectContext, suite: Suite, *, protected_paths=(), absent_files=()) -> Suite
Copy the current files of suite into the copy without
restarting this use, and return the staged suite. Pass the exact
issued context and the original host-side suite. Only files in the
original project or a companion directory are copied, each into the copy
of its directory. Other external suites stay external.
Suite.setup_files names the setup files the runner loads by itself
(such as conftest.py), absent_files setup files to remove from
the copy, and protected_paths project inputs in the copy the
commands run in. Call only while native sessions are idle.
close ¶
close() -> None
Remove the copy and its snapshot, only once verified idle. A failed disposal can be retried, and a cancel of the caller's operation does not stop it.
recovery_owners ¶
recovery_owners(error: BaseException) -> Tuple[object, ...]
The owners error carries whose cleanup is not confirmed, empty
when it carries none: a mutation session whose cleanup failed (its
cleanup_handle), a prepared copy's owner, or a failed
preparation's, whose close() removes the directories it allocated, the
copy itself, an execution scope's failed cleanup, and a command with
its execution scope, each once. A launch that failed and left nothing
running carries no command or scope owner: each command it carries is
disposed, or is a Linux command whose control closed. An owner is
recovery evidence, never proof that nothing runs.
inspect_project ¶
inspect_project(language: str, request: ProjectSetupRequest) -> ProjectSetupReport
Inspect one selected project without acquiring tools or running a build. Metadata and bounded native version probes are adapter-specific. Static operation support does not prove that a requested operation will run.
verify_project ¶
verify_project(language: str, request: ProjectSetupRequest) -> ProjectSetupReport
Assess selected prerequisites with the registered adapter, acquiring tools only when the request allows it. A completed assessment does not guarantee that later tests or mutations succeed.
create ¶
create(name: str, context: ProjectContext) -> LangAdapter
An adapter of name for context, validated first: InvalidRequest names a bad field.
load_extension ¶
load_extension(item: Union[str, Extension], *, relative_to: Optional[str] = None) -> Extension
Load the extension item item, or an Extension that
resolve_extension returned, and register what LangAdapter reads in
its module: langadapter_language, an adapter class registered under
its language and file_extensions, and langadapter_metric_provider,
a metric provider that joins the source operations of every adapter
created afterwards. A module without either registers nothing. A
module's langadapter_toolchain, the part that installs its
language's tools, is no registration: langadapter toolchains reads it
from the loaded module. Each
resolved file is imported once per process: an item naming a loaded
file returns its first Extension. The returned Extension holds
the SHA-256 of the file's bytes at the load, the bytes a file item runs.
A module item runs as the import system loads it, which may be from
cached bytecode, and one imported before from another file is refused.
An item that cannot be resolved or imported raises InvalidRequest naming
it and the cause, as does a metric name a provider loaded before holds,
and a language another registration holds raises RegistrationConflict.
Either way nothing of the item stays registered.
register ¶
register(name: str, file_extensions: Iterable[str], factory: AdapterFactory, *, replace: bool = False) -> None
Register factory for the language name and its file_extensions. One
held by another registration raises RegistrationConflict unless replace.
registration ¶
registration(name: str) -> Registration
The registration of name. UnsupportedLanguage names the registered
languages.
resolve_extension ¶
resolve_extension(item: str, *, relative_to: Optional[str] = None) -> Extension
The Extension of the extension item item, found without running
any module. An item that ends in .py or holds a path separator is a
file, relative to the directory relative_to or else to the current
directory. Any other item is a module name, whose file the import system's
finders show, a dotted name's parent packages unrun. A dotted module
whose top-level package the finders show, but not the module itself,
raises ExtensionFoundOnlyByImport, a subclass of InvalidRequest:
load_extension imports the package, then finds the module or names
it missing. Any other item that resolves to no Python source file raises
a plain InvalidRequest.
unregister ¶
unregister(name: str) -> None
Remove the registration of name, else raise UnsupportedLanguage.
parse_token ¶
parse_token(token: str) -> EvidenceItem
The typed item of a canonical token, or InvalidRequest for an unknown kind.
prepare_project ¶
prepare_project(context: ProjectContext, *, parent: Optional[str] = None, receipt_parent: Optional[str] = None, inputs: Optional[ProjectInputSelection] = None, timeout_s: Optional[float] = None, companions: Sequence[str] = (), bytecode=None) -> PreparedProject
Copy a project that has no links and does not change while it is copied. The tools the context selects stay where they are, and a borrowed container is never removed.
companions names directories outside the project that its runs use, a
suite beside the project above all. Each is copied too, and all keep
their places below the directory they share, so the paths a run prints
relative to its working directory are those an in-place run prints. Files a
companion's tests write beside themselves land in the copy.
original_text writes a run's text with the original paths. In a
container, companions are host directories in the work directory.
bytecode is a langadapter.python.project.BytecodeStore: each use
writes its stored compiled modules into the copy before its runs and,
after a use that ended normally, stores what they compiled, so a later
copy's runs reuse them as a run in place reuses its own.
Framework paths in the project are mapped to the copy, while external
configuration and tools stay as selected. An environment value naming
the original root is refused, never rewritten: the caller adjusts it. A
container needs an explicit control helper and a verified mount. An
inputs plan excludes only its named entries and protects the
declared required inputs and the in-project framework file. The caller
or adapter declares other known requirements. Without a plan the whole
tree is copied except Python bytecode caches. timeout_s bounds
preparation, not use or cleanup, through cooperative checks of local
file system calls and the remaining bound of container command timeouts. A
partial copy stays attached to the failure.
langadapter.preparation.original_text ¶
original_text(context: ProjectContext, text: str) -> str
text from a run in a prepared copy, with the copy's paths written
as the original project's and its companions' (in either slash style, as
created and as resolved, and in a container as the container sees them).
Unchanged for a context that is no prepared copy.
langadapter.context.Execution
module-attribute
¶
Execution = Union[LocalExecution, ContainerExecution]
Where a context's commands run: on this machine (LocalExecution) or in
a container (ContainerExecution), the type of ProjectContext.execution.
langadapter.errors ¶
The exceptions the package raises on purpose.
A failed execution is never raised: it is the result's outcome. The
LangAdapterError subclasses mark an uninterpretable request, an unregistered
language, a registration conflict, a file the library may not change or
remove, and an unreadable source. OperationCancelled derives from
KeyboardInterrupt instead. A failure that leaves something to clean up
carries its owners, which recovery_owners returns.
MeasurementCancelled ¶
Bases: OperationCancelled
A measurement its session's cancel() stopped, raised after cleanup.
If cleanup could not finish, cleanup_handle is the session, whose close()
retries.
langadapter.execution ¶
Where a project's commands run, and the environment they run with.
A LocalExecutor runs a command as a process of this machine. A
ContainerExecutor runs it in a running container through its runner:
docker or podman (exec -w -e -i), apptainer or
singularity (exec --cleanenv --pwd --env). The caller builds the
command the same way for both, argv[0] (the project's interpreter or a tool)
named as the executing environment sees it, and the executor adds only the
exec wrapper and the variables it forwards.
The work directory is the one directory host and container share, and the files the adapter writes for a command lie under it. A host path that is valid in the container (Linux, WSL) is used unchanged on both sides. A Windows drive path is not, so the work directory is mounted at a fixed container path and every argument, working directory and forwarded value under it is rewritten.
A container starts from its own environment, so an executor forwards only
FORWARDED_KEYS, the names the caller adds and the variables with the
caller's prefixes, by default ENV_PREFIX.
ENV_PREFIX
module-attribute
¶
ENV_PREFIX = 'LANGADAPTER_'
The prefix of the variables through which LangAdapter talks to its own child processes.
FORWARDED_KEYS
module-attribute
¶
FORWARDED_KEYS = ('PYTHONPATH', 'PYTHONHASHSEED')
Variables forwarded into a container whatever the caller adds. LangAdapter sets the hash seed of its suite, coverage, evidence and benchmark runs (0, or a run as delivered's session seed), so set order, and coverage that depends on it, repeats between runs. Cosmic Ray's mutant runs keep the caller's.
Executor ¶
Bases: ABC
Runs a project command and returns its completed process. A command
run without env gets the context's environment.
run
abstractmethod
¶
run(argv: Sequence[str], *, cwd: Optional[str] = None, env: Optional[Dict[str, str]] = None, timeout: Optional[float] = None, input: Optional[str] = None) -> subprocess.CompletedProcess
Run argv and return its completed process, with its output
and error output as text. cwd is its working directory, env
its environment, input its standard input and timeout its
limit in seconds.
LocalExecutor ¶
LocalExecutor(context: Optional[ProjectContext] = None)
Bases: Executor
Runs the command directly on this machine.
run ¶
run(argv: Sequence[str], *, cwd: Optional[str] = None, env: Optional[Dict[str, str]] = None, timeout: Optional[float] = None, input: Optional[str] = None) -> subprocess.CompletedProcess
Run argv on this machine and return its completed process, as
Executor.run describes.
ContainerExecutor ¶
ContainerExecutor(container: str, *, runner: str = 'docker', work_dir: Optional[str] = None, container_work_dir: Optional[str] = None, forward_prefixes: Iterable[str] = (ENV_PREFIX,), forward_keys: Iterable[str] = (), helper_python: Optional[str] = None, control_environment: Optional[Environment] = None, context: Optional[ProjectContext] = None)
Bases: Executor
Runs the command inside a running container. forward_prefixes (by
default LangAdapter's) and forward_keys name the variables of env
that cross into it besides FORWARDED_KEYS. A helper_python
selects the container control helper: every command then waits for its
ready signal, ends only when every process it started has ended, and has
its host runner process ended too. Without it, the runner's exec runs
each command.
control_environment
instance-attribute
¶
control_environment = None if control_environment is None else Environment(control_environment, inherit=False)
forward_keys
instance-attribute
¶
forward_keys: Tuple[str, ...] = tuple(dict.fromkeys((*FORWARDED_KEYS, *forward_keys)))
run ¶
run(argv: Sequence[str], *, cwd: Optional[str] = None, env: Optional[Dict[str, str]] = None, timeout: Optional[float] = None, input: Optional[str] = None) -> subprocess.CompletedProcess
Run argv inside the container and return its completed
process, as Executor.run describes.
run_parallel ¶
run_parallel(function: Callable[[Any], Any], items: Iterable[Any], *, workers: int) -> List[Any]
function applied to each of items on at most workers threads,
for an engine that runs its units of work side by side (the C adapter's
mutants): the results in the order of items. Each call runs in a copy
of the caller's context. Inside a prepared use (PreparedProject.use)
the use lends itself to the threads for their calls, and every command
they run is checked as the use's own. The call returns once every
thread has ended, and then raises the first error a call raised. With one
worker or one item the calls run on the caller's thread. Each call must
write only in a directory of its own, never in the project's tree.
supervision_of ¶
supervision_of(context: ProjectContext) -> Optional[str]
How the context's commands are supervised: "pidfd", "process_group", "unreported" (a container whose control helper has started no command yet), "process_group_unconfirmed" (a POSIX host other than Linux), or None (Windows, a container without a control helper).
execution_identity ¶
execution_identity(context: ProjectContext)
The ExecutionIdentity of an observation made in context.
run_capture ¶
run_capture(argv: Sequence[str], *, input: Optional[str] = None, timeout: Optional[float] = None, env: Optional[Dict[str, str]] = None, cwd: Optional[str] = None) -> subprocess.CompletedProcess
Run argv capturing its output, with input sent as UTF-8 bytes
and stdout and stderr decoded with universal newlines. Text-mode stdin
would turn \n into \r\n on Windows, so a patch piped to
git apply would fail to apply.
container_work_dir_for ¶
container_work_dir_for(host_work_dir: str) -> str
Where the host work directory is mounted in the container: the host path itself when it is a valid POSIX container path (Linux, WSL), else (a Windows drive path) a fixed POSIX path the executor translates to.
rebase_path ¶
rebase_path(path: str, host_work_dir: Optional[str], container_work_dir: Optional[str]) -> str
path, a host path in the work directory, as POSIX in the container's
work directory. A path outside the work directory, and every path when there
is no work directory or both sides share it, is only written POSIX. A
drive-letter path parses as a Windows path on any platform.
container_path ¶
container_path(execution: ContainerExecution, path: str) -> str
path as a command in the container of execution sees it,
rebased as the executor rewrites arguments (onto container_work_dir,
else container_work_dir_for's), for a path written into a file a
command in the container reads, where the executor does not reach.
executor_for ¶
executor_for(context: ProjectContext, *, forward_prefixes: Iterable[str] = (ENV_PREFIX,), forward_keys: Iterable[str] = ()) -> Executor
The executor of context. A container executor forwards the
variables with forward_prefixes (LangAdapter's by default),
FORWARDED_KEYS, the names of the context's additional environment
and forward_keys.
path_separator ¶
path_separator(context: ProjectContext) -> str
The separator of a path list (PYTHONPATH and the like) in the
executing environment: : inside a container whatever the host is,
the host's separator for local execution.
prepend_path ¶
prepend_path(env: Dict[str, str], name: str, entry: str, separator: str) -> None
Put entry at the front of the path list env[name], in place.
utf8_environment ¶
utf8_environment(env: Dict[str, str]) -> Dict[str, str]
Make the Python processes started with env read and write UTF-8,
and return env, changed in place. On Windows a child writes
redirected streams in the ANSI code page, so it cannot write a character
that page lacks, or writes bytes a UTF-8 reader cannot decode. Values
the caller set are kept.
work_dir_tempdir ¶
work_dir_tempdir(context: ProjectContext, executor: Executor, prefix: str = 'langadapter_') -> Iterator[str]
An owned temporary directory, removed only while its receipt matches.
Exclusive scratch space, not a shared tree, in the context's work directory,
else the system's default location. A command in a container may write
into it as another user (some images run as root), so removal runs first
inside the container, then on the host. A directory that was replaced,
whose container path does not match or whose removal fails stays, with
its receipt outside it, and a RuntimeWarning is issued. An exit by an
exception keeps it too, since a process started in it may still run, and
so does a failure the operation recorded for retention. The caller stops
its commands before a normal exit. The exception to both is an operation
a cancel stopped (CancelScope.cancel or a mutation session's
cancel) with nothing left unresolved: its scratch is removed. The
receipt lies in the system's temporary directory, but a derived context's
(ProjectContext.derived) lies in its work directory, outside the
directory, so the caller's removal of that directory removes it too.