Test Lab Plugin for Orangescrum Self-Hosted

Screenshots

Case editor with steps, Gherkin, and shared step insertion

Run execution with per-item results and evidence

Defect board with configurable statuses and resolution targets

Key Features

Three Ways to Author a Case

Write ordered steps with expected results, a Gherkin scenario, or free text. The template is a property of the case, so one suite can hold all three without anyone having to convert anything.

Suites and Scenarios

Suites group cases by feature. Scenarios sit above cases as a behaviour that owns many of them, and can carry feature-level Gherkin of their own.

Reusable Shared Steps

Save a named group of steps once and insert it into any case. When the login sequence changes you fix it in one place instead of in eighty cases.

Test Configurations

Define the operating system, browser, and device combinations you actually support. They are first-class records, not free text typed into a field each time.

Test Plans

A plan is a named case selection crossed with the configurations you care about. Launching it generates one run per configuration, already populated with the selection.

Run Execution

Record passed, failed, blocked, skipped, or retest per item, mark a whole set at once, and read the full result history for any single item.

Rerun the Failures

Turn the failed and blocked items of a finished run into a fresh run in one action, so a retest cycle does not mean rebuilding the selection by hand.

Exploratory Sessions

Start a run with no predefined cases and record what you find as notes. Unscripted testing leaves the same evidence trail as scripted testing.

Automation Ingest

Your build pipeline posts JUnit XML or a JSON payload with an ingest token. Coverage, match rate, ingest cadence, and results that matched no case are all reported back.

Built-In Defect Tracker

Defects live here, not in a second tool. Raise one from a failed result or on its own, with severity, priority, environment, assignee, due date, and tags.

A Defect Workflow You Configure

Statuses, transition rules, resolutions, and resolution targets per severity are company settings. Closing requires a resolution, and reopening clears it and counts the reopen.

Board, Relations, and Merge

Drag defects across a board, relate them as duplicate of, blocks, blocked by, or relates, merge duplicates into one, watch what matters, and save personal views.

GitHub and GitLab Sync

Push a defect out as an issue on GitHub or GitLab and keep the link, then pull the issue's open or closed state back onto the defect.

Traceability and Coverage

Link cases and defects to tasks, epics, stories, and features. The coverage report names the gaps and the traceability matrix exports as CSV.

Flaky and Comparison Reports

Find cases that both passed and failed recently, compare the last several runs side by side, and read a per-suite breakdown of the latest result.

Spreadsheet Drafts and CSV Import

Draft cases in an online sheet with either the fixed case columns or your own, or import a CSV with header detection and a mapping step before anything is written.

Permissions That Fail Closed

Seventeen keys across cases, execution, defects, data, and administration. An endpoint with no mapping requires the strictest key, so a missed entry locks down rather than opens up.

About this plugin

Why Test Lab for Orangescrum?

QA work usually ends up split across three tools: one for the cases, one for the bugs, and a spreadsheet for whatever the first two cannot hold. Test Lab keeps all of it in the same install as the delivery work. A case links to the story it verifies, a failed result becomes a defect with the failing step attached, and the defect links back to both.

Author cases the way the test is actually written

Some tests are a numbered list of steps. Some are better as Given, When, Then. Some are a paragraph. Test Lab supports all three as templates on the case itself, so you do not force a Gherkin team into a step grid or a manual team into Gherkin. Reusable shared steps cover the sequences that appear in dozens of cases.

Plans crossed with configurations

Define the operating system, browser, and device combinations you support once. A test plan is then a case selection crossed with the configurations that matter for this release. Launch it and you get one run per configuration, populated and ready, instead of copying the same selection five times.

Manual, automated, and exploratory in one view

Manual runs are executed in the interface. Automated results arrive from your build pipeline as JUnit XML or JSON with an ingest token, and the automation report shows coverage, match rate, ingest cadence, and any result that matched no case. Exploratory sessions are runs with no predefined cases, where findings are recorded as notes. All three show up in the same run history.

A defect workflow you can actually enforce

The status list, the transition matrix, the resolution list, and the resolution target per severity are all company settings. Closing a defect requires a resolution. Reopening clears the resolution, keeps the first resolution time for reporting, and counts the reopen. Every write path goes through the same rule check, including the board drag and bulk edits, so no route is a way round it.

Test Lab or Test Case Management?

Both add-ons cover cases, suites, plans, runs, and defects, so they do genuinely overlap. Test Lab is the newer implementation and the one to choose for a new rollout. What it adds is Gherkin authoring, reusable shared steps, configurations crossed into plans, exploratory sessions, ingest with an automation coverage report, its own configurable defect workflow with resolution targets, and two-way GitHub and GitLab sync. Test Case Management models the same ground differently, with named test environments, parameterised test data sets, and step-level conditions and dependencies. Run one or the other for a given team. Running both means two places to look for the same failure. If you want the marketing view of either side of this, see /test-case-management and /bug-and-issue-tracking.

What's included

Test cases with three authoring templates: ordered steps, Gherkin, and plain text
Preconditions, expected results, priority, type, state, owner, estimate, and tags on every case
A version snapshot written on every case save, with history and duplicate
Reusable shared step groups that can be inserted into any case
Test suites and test scenarios, with feature-level Gherkin on a scenario
Test configurations for operating system, browser, and device combinations
Test plans that cross a case selection with configurations and launch one run each
Test runs created from suites, filters, or a manual case selection
Result states: untested, passed, failed, blocked, skipped, and retest
Bulk result marking and per-item result history
Rerun of the failed and blocked items of a previous run
Exploratory sessions: runs with no predefined cases where findings are notes
Automation ingest of JUnit XML and JSON, token authenticated for your build pipeline
In-app JUnit upload for teams with no pipeline integration
Automation report covering coverage, match rate, ingest cadence, and unmatched results
Defect tracker with a board, list, saved views, watchers, and bulk edit
Company-configurable defect statuses, transition matrix, resolutions, and severities
Resolution targets per severity, measured in calendar hours and stopped at close
Defect relations: duplicate of, relates, blocks, and blocked by, plus merge
Two-way defect sync with GitHub and GitLab issues
Traceability links from cases and defects to tasks, epics, stories, and features
Requirements coverage report and a traceability matrix with CSV export
Flaky test report, cross-run comparison, and per-suite summary
Custom fields for cases and defects, with show, hide, required, and options
Online spreadsheet drafts with fixed or free-form columns
CSV import for cases and defects with header detection and a mapping step
Attachments, comments, and in-app notifications with an unread badge
A read-only token-authenticated API over cases, runs, and defects
Seventeen permission keys, with unmapped endpoints failing closed
A retention command that reports and then purges append-only history
Multi-tenant company-scoped data isolation

Compatibility

Requires the Orangescrum Self-Hosted edition running PHP 8.2+, CakePHP 4.6+, and PostgreSQL 16. The plugin folder is TestLab and it is gated by the TEST_LAB_ENABLED flag in config/constants.php. The interface is a Vue 3 single page application whose built bundle ships with the plugin, so there is no build step at install time. Automation ingest and the read-only API endpoints are token authenticated and need no session. Two-way defect sync needs outbound HTTPS to github.com, gitlab.com, or your own hosted instance of either. Multi-tenant aware, so every case, run, result, defect, and setting is company-scoped.

Installation

A self-hosted install takes a few minutes. Buy the add-on, drop the plugin into your plugins/ directory, run the migrations, and you're live.

1
Buy the add-on
Purchase the Test Lab add-on from /self-hosted/pricing. Pricing for this add-on is being finalised, so confirm the current figure with sales before you budget.
2
Drop the plugin into your install
Copy the TestLab/ folder into plugins/ on your self-hosted Orangescrum server.
3
Enable the feature flag and register the plugin
Add `define('TEST_LAB_ENABLED', true);` to config/constants.php and register `TestLab` in src/Application.php with `routes => true, bootstrap => true`.
4
Run the database migrations
Run `bin/cake migrations migrate --plugin TestLab` to create the case, suite, scenario, plan, run, result, defect, sheet, custom field, and permission tables.
5
Seed the permission matrix
Run `bin/cake migrations seed --plugin TestLab --seed TestLabRbacSeeder` to register the module, menu entry, and role defaults. It is safe to run more than once.
6
Configure the defect workflow
Open Settings and set your statuses and their open or closed category, the transition matrix if you want moves restricted, the resolution list, the severities and priorities, and the resolution target per severity.
7
Define configurations and author your first cases
Add the operating system, browser, and device combinations you support, then author cases directly, draft them in a spreadsheet, or import a CSV with the mapping step.
8
Wire up automation ingest and schedule the purge
Generate the ingest token from Settings, point your build pipeline at the results endpoint, and put `bin/cake test_lab_purge` on a schedule so append-only history stays bounded.

Frequently Asked Questions

How is this different from the Test Case Management plugin?

▾

They overlap on the basics, and that is worth saying plainly: both cover cases, suites, plans, runs, and defects. Test Lab is the newer implementation. It adds Gherkin authoring, reusable shared steps, operating system and browser and device configurations crossed into a plan, exploratory sessions, ingest with an automation coverage report, a defect tracker whose statuses, transitions, resolutions, and resolution targets you configure, and two-way GitHub and GitLab issue sync. Test Case Management models the ground differently, with named test environments, parameterised test data sets, and step-level conditions and dependencies. Choose one per team. Running both leaves you with two places to look for the same failure.

Can I write test cases in Gherkin?

▾

Yes. A case can be authored as ordered steps, as a Gherkin scenario, or as plain text, and the choice is per case. A test scenario, which is the grouping above cases, can also carry its own feature-level Gherkin.

Does it run my automated tests?

▾

No. Test Lab does not execute test scripts and has no runner of its own. Your build pipeline runs the tests and posts the results, either as JUnit XML or as a JSON payload, authenticated with an ingest token. There is also an in-app upload for JUnit files if you would rather not wire up the pipeline yet.

What happens when a test fails?

▾

The result is recorded against the run item with its evidence, and you can raise a defect from it. The defect carries the case and run context with it, then follows your configured status workflow. Defects can also be raised on their own, with no test case behind them, which is what you want for something a customer reported.

How do test plans and configurations work together?

▾

A configuration is a combination you support, for example Windows plus Chrome, or Android plus the mobile app. A plan is a case selection crossed with the configurations you want covered. Launching the plan generates a separate run for each configuration, so a browser-specific failure is visible as a browser-specific failure rather than as one noisy run.

Can we keep using GitHub or GitLab issues?

▾

Yes. Push a defect to either as an issue and the link is stored on the defect. Pulling brings the issue's open or closed state back. Note the limits of that sync: it moves the issue link and the open or closed state, not comments, labels, or attachments in either direction.

Does the history grow without limit?

▾

Case versions, activity, field changes, and notifications are append-only, and a version snapshot is written on every case save, so yes, they grow. A console command reports what sits past a retention window you choose and then deletes it when you pass the confirmation flag. Run it on a schedule and the tables stay a sensible size.