Testing Strategy

merql uses an oracle-driven testing model. Each test scenario defines a known set of inputs (three database states), a deterministic expected result (the oracle), and compares the actual merge output against that expectation. 32 scenarios cover clean merges, conflicts, identity resolution, data types, scale, and edge cases.

# Run the full verification suite.
./bin/verify-all

Oracle-driven testing

There is no existing reference tool that performs three-way database merge, so the oracle is computed from known inputs rather than generated by an external tool. Each scenario has a deterministic expected result that can be verified by logic:

1. Setup -- create three database states (base, ours, theirs) with known content.
2. Oracle -- compute the expected merge result from the known inputs.
3. Actual -- run merql's merge on the three states.
4. Compare -- the actual result must match the expected result exactly.

Known inputs (base, ours, theirs)
         |                    |
    Deterministic          merql merge
    computation
         |                    |
    Expected result       Actual result
         |                    |
         -----> Compare <-----
                  |
            Pass or Fail

Running tests

Full verification

The verify-all script is the final gate. It runs static analysis, coding standards, PHPUnit tests, and the oracle regression suite:

./bin/verify-all

This must pass before any work is considered complete.

Oracle regression suite

Run all 32 scenarios:

./bin/test-regression

Run scenarios in parallel:

./bin/test-regression --jobs 4

Run a single category:

./bin/test-regression --category clean
./bin/test-regression --category conflict
./bin/test-regression --category identity

Run with minimal output (pass/fail only):

./bin/test-regression --fast

Single scenario

Test one scenario through the full pipeline:

./bin/test-scenario column-level-clean

Individual oracle steps

./bin/oracle column-level-clean    # Compute expected result
./bin/actual column-level-clean    # Run merql, capture output
./bin/compare column-level-clean   # Diff oracle vs actual

Unit tests

composer test:unit     # Isolated component tests
composer test          # Full PHPUnit + oracle matrix

Code quality

composer cs            # PSR-12 coding standards check
composer cs:fix        # Auto-fix coding standards
composer analyse       # PHPStan level 8 static analysis

Scenario categories

clean (8 scenarios)

Clean merges with no conflicts. Both sides may change data, but never the same column of the same row.

conflict (6 scenarios)

Scenarios that must produce conflicts.

identity (4 scenarios)

Row identity edge cases.

types (6 scenarios)

Data type handling across all column types.

scale (4 scenarios)

Performance under load.

edge (4 scenarios)

Edge cases and boundary conditions.

Scenario structure

Each scenario is a directory under scenarios/<category>/<name>/ with a scenario.json configuration:

{
    "name": "column-level-clean",
    "category": "clean",
    "description": "Both sides modify same row but different columns",
    "tables": ["test_posts"],
    "expectations": {
        "conflicts": 0,
        "operations": 3,
        "changeset_match": "exact",
        "merge_match": "exact",
        "sql_match": "semantic"
    }
}

For conflict scenarios, the expected conflicts are specified in detail:

{
    "expectations": {
        "conflicts": 1,
        "conflict_details": [
            {
                "table": "test_posts",
                "primary_key": {"id": 42},
                "column": "title",
                "ours_value": "Welcome",
                "theirs_value": "Greetings"
            }
        ]
    }
}

The ScenarioRunner pipeline

ScenarioRunner::run() orchestrates a single scenario:

1. OracleCapture::compute() builds three snapshots from the scenario's JSON data and runs the merge.
2. ScenarioComparator::compare() checks the merge result against the expected values in scenario.json.
3. Returns pass/fail with a list of failure messages.

use Merql\Tests\Oracle\ScenarioRunner;

$result = ScenarioRunner::run($scenario);
// ['name' => 'column-level-clean', 'pass' => true, 'failures' => []]

ScenarioRunner::runAll() runs every scenario in sequence:

$results = ScenarioRunner::runAll($scenarios);

Adding a new scenario

1. Create a directory: scenarios/<category>/<name>/.
2. Create scenario.json with the scenario metadata and expectations.
3. Define the three database states in the JSON configuration: base data, ours mutations, theirs mutations.
4. Run the scenario: ./bin/test-scenario <name>.
5. Verify the full suite: ./bin/verify-all.

Code quality standards

merql enforces:

• PHPStan level 8 -- strict static analysis with no baseline exceptions.
• PSR-12 -- coding standards checked by phpcs.
• Oracle regression -- all 32 scenarios must pass.
• Unit tests -- isolated tests for each component.

The verify-all script runs all four checks in sequence. No partial sign-off is accepted.