Sources¶
langadapter.sources ¶
Source operations on test files and what they report, pooled staging, source metrics, and the text helpers the languages share.
Every edit of adapter.sources (langadapter.capabilities.SOURCE_EDITS)
returns a SourceResult whose status (APPLIED and the constants after
it) says whether it applied and, if not, why. Its parser is empty when no
parser ran. adapter.sources.normalize_bindings removes shadowed
definitions without running the project, returns
SourceObservation[SuiteNormalization] and keeps the original bytes of a
source that does not parse. The project's runtime may still reject a
normalized source.
A pool (StagedPool) stages the test files of several suites as one suite
under names no other suite uses, and maps each staged file, test and killer
back to its suite's slot and standalone identity. MetricRegistry merges
the values of registered metric providers per scope. STANDARD_METRICS
names the metrics any language may report, each with the operation of
adapter.sources that reports it. The merge repairs share
one naming policy: the first source keeps a name, a later differing
declaration takes fresh_name, a binding that cannot be renamed is a
conflict the caller's Choose decides, and every rewrite goes through
apply_edits. rows and lines split lines where tree-sitter and the
language parsers do, unlike str.splitlines. whole and bounded
bound a tool's output for a result.
UNCHANGED
module-attribute
¶
UNCHANGED = 'unchanged'
The operation ran and found nothing to change. The source is the input.
PARSE_ERROR
module-attribute
¶
PARSE_ERROR = 'parse_error'
The source does not parse, so the operation did not run.
REFUSED
module-attribute
¶
REFUSED = 'refused'
The project interpreter rejects the source. No rewrite uses another grammar.
UNSUPPORTED
module-attribute
¶
UNSUPPORTED = 'unsupported'
The language does not provide the operation.
INFRASTRUCTURE_FAILURE
module-attribute
¶
INFRASTRUCTURE_FAILURE = 'infrastructure_failure'
A tool the operation needs could not run.
MISSING_SETUP
module-attribute
¶
MISSING_SETUP = 'missing_setup'
The implementation needs a runtime or parser that is not available.
TIMEOUT
module-attribute
¶
TIMEOUT = 'timeout'
An execution-backed operation exceeded its limit without a source verdict.
NOT_REPAIRED
module-attribute
¶
NOT_REPAIRED = 'not_repaired'
A repair ran and the problem remains. The source is the input.
SOURCE_STATUSES
module-attribute
¶
SOURCE_STATUSES = (APPLIED, UNCHANGED, PARSE_ERROR, REFUSED, UNSUPPORTED, INFRASTRUCTURE_FAILURE, NOT_REPAIRED, MISSING_SETUP, TIMEOUT)
Every status a source operation reports.
STANDARD_METRICS
module-attribute
¶
STANDARD_METRICS: Mapping[str, StandardMetric] = MappingProxyType({metric.name: metric for metric in (_standard('nloc', "the lines that hold code, not counting blank lines, comments or Python's docstrings", unit='lines'), _standard('function_count', 'the functions and methods declared'), _standard('class_count', 'the classes declared'), _standard('loop_count', 'the loops'), _standard('loop_max_nesting', 'the deepest nesting of loops'), _standard('cyclomatic_sum', 'the sum of the McCabe complexity of each function'), _standard('recursive_function_count', 'the functions that call their own name'), _standard('io_call_count', "the calls to the language's file, socket and process functions"), _standard('cyclomatic_complexity', "a function's McCabe complexity, its nested function bodies left out", scopes=('function',)), _standard('call_site_count_estimate', "the call sites of a function's bare name in the project's other source files, test files left out, matched by name without resolving it, and for the file their sum over its functions", operation='call_statistics'), _standard('calling_file_count_estimate', "the project's other source files that call a function's bare name, test files left out, and for the file those that call any of its functions, each counted once", operation='call_statistics'))})
Each standard metric's name mapped to its StandardMetric. A consumer
may request any of them from any language and sends each to the
operation it names. A language that does not measure one reports it
unavailable, never as a number. A function's values cover its whole text,
nested functions included, except cyclomatic_complexity.
Choose
module-attribute
¶
Choose = Callable[[str, Tuple[int, ...]], Optional[int]]
(name, owners) -> the owner that keeps name, or None to leave the
conflict unresolved. owners are the indices of the sources that bind
name un-renamably.
Edit
module-attribute
¶
Edit = Tuple[int, int, str]
A byte range of a source's UTF-8 text and its replacement.
SAFETY_CHARS
module-attribute
¶
SAFETY_CHARS = 4000000
The most characters of one captured text a result carries: a bound against memory exhaustion, not a display limit (the caller cuts what it shows).
RemovedBinding
dataclass
¶
RemovedBinding(scope: Tuple[str, ...], name: str, kind: str, removed_start_line: int, removed_end_line: int, retained_start_line: int, retained_end_line: int)
A shadowed definition removed in favor of the binding kept. Lines
are 1-based and inclusive, and scope names the enclosing classes.
removed_start_line and removed_end_line are the first and last
line of the removed definition, retained_start_line and
retained_end_line those of the binding kept.
SuiteNormalization
dataclass
¶
SuiteNormalization(source: str, changed: bool, parse_ok: bool, parse_error: Optional[str], removed_bindings: Tuple[RemovedBinding, ...], before_sha256: str, after_sha256: str, rule_version: str)
A static normalization with every removal. A source that does not
parse is returned whole, never partly rewritten. before_sha256 and
after_sha256 are the SHA-256 of the input and of the result.
parse_ok says whether the source parses, removed_bindings holds
every removal, and rule_version names the normalization rule and its
version, such as python-last-binding-v2.
CollisionResolution
dataclass
¶
CollisionResolution(sources: Tuple[str, ...], ejected: Tuple[int, ...] = (), eject_reasons: Tuple[str, ...] = (), renames: Tuple[Dict[str, str], ...] = (), deduped: Tuple[Tuple[str, ...], ...] = (), dropped_defs: Tuple[Tuple[str, ...], ...] = (), localized_autouse: Tuple[Tuple[str, ...], ...] = (), choices: Tuple[Tuple[str, int, Tuple[int, ...]], ...] = (), conflicts: Tuple[Tuple[str, Tuple[int, ...]], ...] = ())
Static collision rewrites of the input sources. sources,
renames, deduped and dropped_defs hold one entry for each
input source, in its order, and so does localized_autouse in the
answer of adapter.sources.resolve_collisions. Elsewhere it is empty
when nothing was localized. ejected names each rejected source's
input index once, with its reason at the same place in
eject_reasons. dropped_defs names the dropped bindings, not the
dependents removed with them. A conflict without a caller preference is
reported, never decided.
localized_autouse
class-attribute
instance-attribute
¶
localized_autouse: Tuple[Tuple[str, ...], ...] = ()
choices
class-attribute
instance-attribute
¶
choices: Tuple[Tuple[str, int, Tuple[int, ...]], ...] = ()
conflicts
class-attribute
instance-attribute
¶
conflicts: Tuple[Tuple[str, Tuple[int, ...]], ...] = ()
TargetIdentity
dataclass
¶
TargetIdentity(source_path: str, project_root: str, module_name: str, language: str, policy_version: str, target_source_sha256: str, target_source: str = '')
The target source a generated-test validity policy checks against.
Its text, which the policy compares with literals in the tests, is left
out of equality, repr and record().
source_path: the target's path, absolute or relative toproject_rootas the language gives it.project_root: the project's root.module_name: the target's module: its importable name for Python,package.Classfor Java, and its path without.cfor C.policy_version: the version of the validity policy.target_source_sha256: the SHA-256 of the target's text.target_source: the target's text.
target_source
class-attribute
instance-attribute
¶
target_source: str = field(repr=False, compare=False, default='')
SuiteValidityFinding
dataclass
¶
SuiteValidityFinding(rule_id: str, policy_version: str, operation: str, diagnostic: str, start_line: int, start_col: int, end_line: int, end_col: int, enclosing_test: Optional[str])
One violation of the generated-test validity policy. rule_id
names the rule, and policy_version the version of the policy.
start_line, start_col, end_line and end_col give its
span in the test source, lines counted from 1 and columns as the
language's parser counts them: from 0 for the Python adapter's tests, as
Python's ast counts them, and from 1 for Java's and C's tests,
Python tests of a C extension included. enclosing_test names the
test that holds it, None outside any test.
SuiteAuditFlag
dataclass
¶
SuiteAuditFlag(rule_id: str, diagnostic: str, start_line: int, start_col: int, end_line: int, end_col: int)
A review-only source-access signal, not automatic rejection.
rule_id names the rule. start_line, start_col, end_line
and end_col give its span in the test source, lines counted from 1
and columns as the language's parser counts them: from 0 for the Python
adapter's tests, as Python's ast counts them, except an
unparseable_source flag's, which count from 1 as a SyntaxError
does, and from 1 for Java's and C's tests.
SourceResult
dataclass
¶
SourceResult(status: str, source: str, parser: str, unmatched: Tuple[str, ...] = (), removed: Tuple[str, ...] = (), detail: str = '', repairs: int = 0)
The result of a source operation: its status, the resulting source
(the input when it did not apply), the parser, the requested names that
matched nothing, the items removed, a detail such as the parse error,
and for an applied syntax repair (salvage_syntax) how many repairs
it made.
StructureCheck
dataclass
¶
StructureCheck(ok: bool, message: str, line: int, column: int, parser: str, snippet: Optional[str] = None)
Whether a source parses, with the first error's position counted from
1 (0 when it parses). message names the error and its position
("syntax error at line 3:5"). snippet is the whole source text the
error covers, stripped, possibly several lines, for the caller to cut.
Both are empty when it parses. snippet is None for a language that
gives none, whose message is then its whole text. ok says
whether the source parses.
TestUnit
dataclass
¶
TestUnit(name: str, start: int, end: int)
One test of a test source from a language's test_units hook: its
qualified name (Class.test or test) and the span removing it cuts,
source[start:end]. Overloads are separate units of one name.
TestMetrics
dataclass
¶
TestMetrics(parsed: bool, parser: str, code_lines: Optional[int] = None, tests: Optional[int] = None, assertions: Optional[int] = None, fixtures: Optional[int] = None, parametrized_cases: Optional[int] = None, detail: str = '')
Counts of a test source, absent when it does not parse: code lines,
tests, assertions, fixtures and parametrized cases. code_lines
counts lizard's lines of code (blank lines, comments and docstrings left
out), and parametrized_cases the cases the parametrized tests run.
SourceValidation
dataclass
¶
SourceValidation(status: str, message: str, line: int, column: int, interpreter: str, source_sha256: str = '')
Whether the project interpreter accepts a source: valid,
syntax_error (with position and message), or timeout or
infrastructure_failure, which are no verdict on the source.
interpreter identifies the interpreter, as a diagnostic.
source_sha256 is the SHA-256 of the checked source.
PolicyReport
dataclass
¶
PolicyReport(status: str, items: Tuple[object, ...], policy_version: str, parser: str, detail: str = '', error_line: int = 0, error_column: int = 0)
The generated-test validity policy's reading of a source for a
target: completed with its items, possibly none, or parse_error
with the parser's message and no items. The items are its findings
(analyze_validity) or review-only signals (screen_source).
policy_version is the version of the policy. error_line and
error_column give the parse error's position, 0 when the source
parses.
PoolMember
dataclass
¶
PoolMember(slot: str, name: str, source: str)
One test file of a suite to pool. slot names the suite (letters,
digits and underscores), name is the file's path in the suite's
standalone run, relative to its root, and source is its text.
validate ¶
validate() -> None
Refuse a slot that is not letters, digits and underscores, a name
that is no relative POSIX path inside the suite, or a source that is
no text, with InvalidRequest.
StagedFile
dataclass
¶
StagedFile(slot: str, original: str, path: str, source: str, renamed: Tuple[Tuple[str, str], ...] = (), self_references: Tuple[str, ...] = ())
One pooled test file: its suite's slot, its original path in
the suite's standalone run, its path in the pool, the text that runs
there and the top-level types renamed in it as (original, staged) pairs
(none for Python). self_references lists where the file refers to its
own module or class name in a way the staged name changes at run time (a
name in a string or read at run time, an import of its own class): its
verdicts in the pool may then differ from its standalone run.
StagedPool
dataclass
¶
StagedPool(files: Tuple[StagedFile, ...], setup: Tuple[Tuple[str, str], ...] = ())
The pooled suite's test files, in member order, and the setup files every suite shares, as (relative path, text) pairs staged unchanged.
write ¶
write(root: str) -> Suite
Write the pooled files as given, line endings included, below the
existing directory root and return the pooled suite.
slot_of ¶
slot_of(path: str) -> Optional[str]
The slot of the pooled file path (relative POSIX), or None.
original_test ¶
original_test(test: TestId) -> Tuple[str, TestId]
The slot of the pooled run's test and the test as the slot's
standalone run identifies it.
slot_of_killer ¶
slot_of_killer(killer: str) -> Optional[str]
The slot whose file or type a killer as the engine printed it names, or None when it names none or more than one.
original_text ¶
original_text(text: str, slot: str) -> str
text (a test id or a killer as the engine printed it) with the
pooled names of slot written as its standalone run names them.
killed_by ¶
killed_by(kill_matrix: Mapping[str, Any], slot: str) -> Set[Any]
The ids of the mutants a named killer of slot killed in the
pooled run's saved kill_matrix.
derive ¶
derive(kill_matrix: Mapping[str, Any], slot: str, *, covered_lines: Iterable[int]) -> Dict[str, Any]
The kill matrix of slot derived from the pooled run's saved
kill_matrix by langadapter.mutation.restrict_kill_matrix,
keeping the mutants the slot's tests killed (killed_by) and
covered_lines, the target lines they cover. A row that stays
killed names only the slot's killers and killing tests, as its
standalone run names them. The result is marked derived, with
pool_slot. It equals the standalone run only when the slot's
tests behave in the pool as alone and the pooled run recorded every
killing test (attribution all).
StandardMetric
dataclass
¶
StandardMetric(name: str, meaning: str, unit: str, scopes: Tuple[str, ...], operation: str)
A metric any language may report under one name: its meaning,
its unit, the scopes a provider may report it for, and the
operation of adapter.sources that reports it, metrics for a
metric of a file's source text and call_statistics for a metric of
the project. A provider's definition of the name keeps the unit and
reports at most these scopes.
MetricRegistry ¶
MetricRegistry()
A registry local to one adapter object. Each collection of metrics uses the registrations as they are when it starts.
register ¶
register(provider: MetricProvider, *, replace: bool = False)
Register provider for each metric it defines. A metric name
already registered raises InvalidRequest unless replace is
true, and so do a provider without a name, definitions or
measure and a definition of a standard metric that differs from
STANDARD_METRICS.
definitions ¶
definitions() -> Tuple[MetricDefinition, ...]
The definition of every registered metric, by name.
measure ¶
measure(source: str, *, language: str, parser: Optional[str], names: Optional[Sequence[str]], parser_context=None) -> SourceObservation[SourceMetrics]
The SourceMetrics of source for the metrics names,
every registered one for None, each from its provider. A standard
metric without a provider, and one whose provider fails, is listed
in unavailable with its reason, and an unknown name raises
InvalidRequest. parser_context, when given, is called for
the parser the providers receive.
check_members ¶
check_members(members: Sequence[PoolMember]) -> Tuple[PoolMember, ...]
Validate the members of a pool: at least one, each valid, no file named twice within one slot.
check_setup ¶
check_setup(setup: Sequence[Tuple[str, str]]) -> Tuple[Tuple[str, str], ...]
Validate the shared setup files: (relative path, text) pairs.
unique_paths ¶
unique_paths(pool: StagedPool, *, by_name: bool) -> Optional[str]
None when the pool's files and setup files have distinct paths (and,
with by_name, distinct file names), else the reason.
fresh_name ¶
fresh_name(name: str, taken: Collection[str]) -> str
The first <name>_<k>, k from 2, not in taken. The caller
takes it when its rename succeeds.
apply_edits ¶
apply_edits(text: str, edits: Iterable[Edit]) -> str
text with the byte-range edits applied. Of overlapping edits,
the one that starts first is kept, the longest at an equal start. A
repeated edit counts once.
rows ¶
rows(text: AnyStr) -> List[AnyStr]
text's lines with their ends, as tree-sitter counts its rows:
each line ends after a newline. Bytes split at the same points, so a
tree-sitter column, a byte offset, indexes a row of text.encode().
lines ¶
lines(text: str) -> List[str]
text's lines with their ends, as Python and javac number them:
each line ends after a newline, a carriage return, or both.
line_number ¶
line_number(text: str, index: int) -> int
The line, numbered as lines numbers it, holding text[index].
whole ¶
whole(text: str, what: str, *, limit: int = SAFETY_CHARS) -> str
text whole up to limit characters, else its first
limit // 3 and last limit - limit // 3 characters around a
marker that says how many characters of what were omitted.
bounded ¶
bounded(output: str, *, limit: int = 5000, head: int = 1800, tail: int = 3200) -> str
output whole up to limit characters, else its first head
and last tail characters around a truncation marker, for
diagnostics, per-mutant output and exception messages. The head keeps
what was printed before a long failure traceback.
langadapter.sources.code_lines_and_complexities ¶
code_lines_and_complexities(text: str, suffix: str) -> Tuple[int, Tuple[int, ...]]
The lines of code of text, read as a file whose name ends in
suffix (blank lines, comments and Python's docstrings left out), and
the cyclomatic complexity of each function in it, in the order the
functions end: a nested function before the one around it, so the text
of one function gives its own complexity last. A function's complexity
is one plus its own decisions (branches, loops, exception handlers,
cases and short-circuit operators), and a nested function's decisions
count only toward it. The structural metrics count both this way, with
lizard, which needs no parse tree, so incomplete source is counted too.