Extension points¶
Testuary has eight extension points. Each is one file, or a few lines in a
file, that the model catalogue or a setting names, and each has its own
guide. testuary tutorial walks the model alias, the prompt profile and an
extension's hints on a small project without a model. testuary init
--examples copies working files of a client factory, a prompt profile and
hints, with a README that checks a project with them.
| Extension point | What it adds | Where it is named | Guide |
|---|---|---|---|
| Model alias | a model service, with no code | a section of models.ini, which testuary setup model writes. A run selects it with --model or [api] model |
models.ini fields |
| Client factory | a model service that is not OpenAI-compatible | a Python file that an alias of models.ini names with chat_model, which selects the protocol added |
Your own model client |
| Prompt profile | your own instructions to the model | a JSON file that [main] prompt_profile names |
A prompt profile |
| Hints | advice for the model that fits your project | a JSON file that [main] hints_file names, by default hints.json in the project directory |
Hints |
| Hints from an extension | advice that comes with a language or a tool | the top-level name testuary_hints in a Python file that [main] extensions names |
Hints |
| Language | test generation for a language Testuary has no code for | the top-level name langadapter_language in a Python file that [main] extensions names |
Make a language ready |
| A language's project settings | a language's runtime and tools, set for each project | the language adapter's project_settings (LangAdapter's ProjectSetting), set in [project] |
Project settings |
| Web interface extension | another product's pages in testuary-web |
the top-level name testuary_web in a Python file that [main] extensions names |
Extend the web interface |
Three kinds of file¶
- A Python file that an alias of models.ini names with
chat_model: the client factory, a Python function that returns the chat model Testuary sends its requests through. - A JSON file that
[main] prompt_profileor[main] hints_filenames: a prompt profile or hints. - A Python file that
[main] extensionsnames. Its top-level names say what it adds:langadapter_languagea language,testuary_hintshints andtestuary_weba web interface extension.
Each Python file can also be an installed module, named by its module name
in place of the file, such as my_package.chat:factory for chat_model or
my_tool.extension in [main] extensions. Demonstration data are no
extension point: every recording --demo plays comes with Testuary, as
Offline demonstrations describes.
The table Relative paths lists where a relative path in each of these settings starts.