Skip to content

Source analysis

Source operations read and edit source text. Some need only the calling process, others validate with the project's runtime.

Source operations

adapter.sources gives access to the language's own analysis and edits without reimplementing them. prune_tests and finalize_source use parsers in the calling process and work in contexts without a runtime. Merging, normalize_shadowed, repair and pruning of imports or of collection breakers (what keeps a test file's tests from loading) use the project's validator and need its runtime. normalize_bindings is static and works without one. A failed validation does not prove the source invalid, and an edit that is unavailable returns the input with an explicit status. The adapter's own edit methods, which these operations call, can also be used directly.

check_structure(source) says whether the parser reads the source without an error. For the first error, message names it and its position ("syntax error at line 3:5"), and snippet holds the source text the error covers, stripped and whole, which can span several lines. The caller decides how much of it to show.

A source that the library's own parser cannot read is reported, never read as empty. The typed source operations report PARSE_ERROR, test_metrics reports the source as not parsed, and the adapter methods duplicate_test_names and order_dependent_mock_warnings raise langadapter.SourceReadError, each naming the parser. stage_pool refuses a member whose imports cannot be read. For Python, that parser is the one of the Python the library runs on. When the project's Python is newer, the setup check adds a note, and a reading error names both versions.

For Python, a test is what pytest's default collection finds: a module-level function whose name starts with test, and such a method of a module-level class whose name starts with Test and that has no __init__ or __new__, or of a class based on unittest's TestCase or IsolatedAsyncioTestCase, whatever its name. A pytest fixture, a nested function, a method of another class and a class that sets __test__ = False are no tests. The rule reads the module's own text, so the methods a class inherits, tests imported into the module, Test classes nested in another and a TestCase reached through another base are not counted. The test count of test_metrics, duplicate_test_names, order_dependent_mock_warnings, a validity finding's enclosing test and the fixtures a merge requests for each test follow this rule.

merge_sources(first, second) joins two sources with a separator comment line, the adapter's source_merge_separator (# for Python, // for Java and C). An adapter from another package may set its own separator or require the caller to pass one, and its merge checks it. The library never guesses the separator from the language name or by running a tool. The Python merge puts the imports of both sources first, each once. Of any other top-level statement both sources hold identically, it keeps one copy when the statement only binds names to objects that exist or cannot change and nothing between the two copies can change what it binds. A statement that calls something, such as sys.path.insert, or builds a new list or dict stays twice.

resolve_collisions keeps native renames, sharing, conflicts and the caller's preferences. A completed resolution is a CollisionResolution, which may leave out a source the validator judged invalid, naming it in ejected with its reason in eject_reasons. If validation did not run, the observation has no resolution value: its status says why, and its reason names each source whose validation failed. The adapter's own resolve_collisions method returns that SourceObservation: completed with the CollisionResolution and the parser that read the sources, or with the status infrastructure_failure, timeout or missing_setup and a reason. Any other answer is refused.

Pooled suites

stage_pool(members, setup=...) stages the test files of several suites as one pooled suite, so one run measures all of them. Each PoolMember names its suite's slot (letters, digits and underscores), the file's path as the suite's own standalone run stages it, and its text. setup lists the setup files every suite shares, as (path, text) pairs, staged once and unchanged. The observation's value is a StagedPool:

from langadapter.sources import PoolMember

pool = adapter.sources.stage_pool((
    PoolMember("suite_000", "test_candidate.py", first_source),
    PoolMember("suite_001", "test_candidate.py", second_source),
), setup=(("conftest.py", conftest_text),)).value
suite = pool.write(empty_dir)

A Python file keeps its text and gets the slot in its name (test_candidate_suite_000.py), since pytest imports two files of one name as one module. A Java file's top-level types get the slot in their names, with every use of them in that suite's files, and the file takes its public class's new name (CalcTest_suite_000.java), since two suites usually declare the same test class. Names in string literals and in package and import declarations stay as they are. The rename is the one the Java merge repair applies, and langadapter.java.source_analysis.rename_class applies it to one source: a type's uses include the qualifier of a method or constructor reference (Helper::new), while method names, annotation element names and labels keep their text. A source that also declares a method, a variable, a parameter, a field or an annotation element named like the type cannot be renamed safely, and the observation says which.

pool.original_test(test) returns the slot of a pooled run's test and the test as the suite's standalone run identifies it. pool.slot_of_killer and pool.original_text do the same for a killer as the engine printed it. Each StagedFile keeps its staged text, its hash and the renamed types. Its self_references lists where the file refers to its own module or class name in a way the staged name can change at run time: the name in a string, the name or file read at run time, or an import of its own class. A Python file also lists the name of another of its suite's modules in a string, as for importlib. Such a suite's verdicts in the pool may differ for that reason alone. pool.derive(kill_matrix, slot, covered_lines=...) derives a suite's kill matrix from the pooled run's saved matrix, by the rule Mutation sessions describes, and pool.killed_by lists the mutants the suite's tests killed there.

A pool that cannot be staged is an observation with a reason: a Java file that does not parse (parse_error), one not named for its public class or whose types cannot be renamed safely, a Python file that imports another file of its own suite, whose module the pool renames, or names that would still collide (unsupported). A pooled run is not a standalone run of each suite: a test that depends on another suite's tests, on their order, on the process they share or on its own file or class name can give another verdict in the pool.

Static analysis

langadapter.python.source_analysis provides normalization, collision analysis and merging without starting project tools. Normalization keeps the rule, the removed and retained spans and the parse errors. When two sources define the same name, collision rewriting needs the caller to say whose definition wins. Without that, conflicts stay unresolved. Priorities based on measured coverage, kills or other goals belong to the caller.

These operations use the calling process's grammar. Rewritten sources need validation with the project's interpreter before delivery, and the context-bound source operations on PythonAdapter include that validation. Read-only skeletons, fingerprints, imports and structural metrics start no tools. Source that does not parse has unknown structural metrics. count_code_lines counts code lines as nloc does, with lizard, which also accepts incomplete source. It is not a count of coverage statements.

Metrics

adapter.sources.metrics(source, names=["nloc", "cyclomatic_complexity"]) returns immutable file and function scopes, with the provider, parser, definition version and units on each value. Python and Java report eight structural counters for the whole file and for each function, and C for the whole file only: nloc (lines that hold code, not counting blank lines, comments or Python's docstrings), function_count, class_count, loop_count, loop_max_nesting, cyclomatic_sum, recursive_function_count (functions that call their own name) and io_call_count (calls to the language's file, socket and process functions). For a function, the counters cover its whole text, nested functions included. A ninth value, cyclomatic_complexity, is reported only for functions and leaves nested function bodies out. nloc, cyclomatic_sum and cyclomatic_complexity are lizard's count of code lines and its McCabe complexity, and their definition version names lizard's version: lizard-<version>-1, and lizard-<version>-own-body-1 for cyclomatic_complexity. Read alone, a Java method declared without a body, such as an abstract or interface method, or a Python function whose body follows its signature on the same line, such as the stub def f(): ..., holds no function for lizard. Its cyclomatic_complexity is a value with status unsupported and no number, and where it is not nested in a function, it adds nothing to cyclomatic_sum. A function that lizard does not report when it reads the function's own text, such as a Python function holding a one-line def with a body or a Java method holding an anonymous class with a field, also gets an unsupported cyclomatic_complexity. The other six counters follow the language's tree-sitter grammar (tree-sitter-structural-1). These are static hints, not execution measurements or checks of semantic equivalence.

adapter.sources.metric_definitions() lists the definition of every metric the adapter's metrics operation measures. langadapter.sources.STANDARD_METRICS names the metrics a tool may request from any language, each a StandardMetric with its meaning, unit, the scopes a provider may report and the operation of adapter.sources that reports it: metrics for a metric of a file's source text, call_statistics for the two call statistics. A tool that requests metrics in several languages sends each name to the operation the table names. A standard metric a language does not measure is unavailable, with the status unsupported, the provider langadapter.standard and a reason that names the language and the metric's operation, never a number. A name that is neither standard nor measured is refused as an invalid request. A listed name the table does not hold is the language's own metric of the source text.

Qualified names and the original inclusive line spans identify occurrences. Java also keeps the parameter types as written. Declarations with the same name on different lines stay separate. Declarations that only their column could tell apart are refused rather than merged. Parse or setup failures appear as unavailable values, never zero, and valid source without functions has an empty function tuple. Read the unavailable entries as well as the successful scopes. More metrics can be registered on one adapter with adapter.sources.register_metric_provider, which A metric provider describes.

Project source files

adapter.sources.project_source_files() lists the project's source files. The paths are relative to the context's root, in POSIX form and sorted. They are the files with the language's file extensions, outside the directories the adapter ignores (ignored_dirs, by default the usual version control, environment, build, cache and test directories) and the directories whose name starts with a dot. The list leaves out the file names the adapter declares as no sources (non_source_files, such as Python's setup.py), the test files (is_test_file) and empty files. No file's content is read, so a file that does not parse stays in the list.

Call statistics

adapter.sources.call_statistics(target) estimates how widely the functions of the source file target are used. target is a path relative to the context's root, as project_source_files lists it. The result is SourceMetrics with the function scopes metrics reports: for each function, call_site_count_estimate counts the call sites of its bare name, name(...) or anything.name(...), in the project's other source files, and calling_file_count_estimate counts the files that hold one. The file scope holds their sums over the functions, each calling file counted once.

The search covers the files project_source_files lists, the target left out, so calls in test files are not counted. Names are matched, not resolved, so the values are estimates and their definition version is name-match-1. A call of another function with the same bare name counts. An import under another name, a reference that is not a call (a function passed as a value, a decorator without arguments) and a call through getattr or reflection do not. A project file that does not parse is left out, and the values' reason names it. A call site of a bare name that two of the target's functions share counts once for each of them in the file's sum. A target outside the context's root is refused as an invalid request, and another spelling of the target's path still leaves the target out of the search. A target that does not parse gives the status parse_error, and a language without the call search unsupported, never zero. Python and Java search with their tree-sitter grammars. C has no call search, so its result is unsupported.

Collection breakers

prune_collection_breakers(source, timeout_s=...) removes what keeps a test source's tests from loading, so that the rest of them run. Each round loads the tests as the language does, by collection or compilation, and removes the smallest unit the failure names. Python loads the source at most five times, so it removes at most four units. Java and C remove at most five units and load once more to confirm. The result is adopted only once the tests load, and removed names each unit removed. The unit is the language's own, and each adapter declares it in collection_unit:

  • Python: the smallest module-level or class-level statement around the line that pytest's collection traceback names, replaced by pass. A test that fails when it runs is not a collection breaker.
  • Java: the smallest declaration around a line javac reports an error on, or around the line of a static initialization that throws: an import, or a member of a type (a method, constructor, field, initializer or nested type). A test method that does not compile is removed with it, as a helper or field whose removal breaks the methods that use it is in the next round. A class that compiles runs once only when its static initialization could throw (a static block, or a static field whose value is not a literal).
  • C: the utest.h test, fixture block or top-level declaration around a line GCC reports an error on, as the entries of CAdapter.collection_unit and CAdapter.prune_collection_breakers state.

The result is not_repaired, with the input, when the failure names no unit that can be removed (for Java, an error in the package or a class's header), when no test is left, or when the tests still do not load after the last round.

Syntax repair

langadapter.python.source_analysis.salvage_syntax takes a compilation validator and applies a bounded repair at the locations it reports. Its result keeps the last validation and tells invalid source from a validator that did not run. It starts no process beyond those the validator starts. PythonAdapter.salvage_syntax supplies the project's validator.