Skip to main content

Testing Framework

Testing Framework Internals for Automation (TestNG / JUnit)​

Context (Why this topic exists NOW)​

At this stage, your automation code:

  • is clean (Java 8+)
  • is reusable (OOP + Generics)
  • handles data safely

But a new problem appears:

  • You don’t fully control execution order
  • Setup/teardown behavior feels magical
  • Retries, parameters, and listeners behave unexpectedly

This happens because Selenium does NOT control test execution. Java testing frameworks (TestNG / JUnit) do.

What Selenium Cannot Do (The Gap)​

Selenium can:

  • automate browser actions

Selenium cannot:

  • define test lifecycle
  • decide execution order
  • retry failed tests
  • parameterize test runs
  • listen to test events

If Selenium alone ran tests, everything would be manual and chaotic.

What Testing Frameworks Add (The Fix)​

Testing frameworks provide:

  • structured execution lifecycle
  • annotations for setup/teardown
  • assertions and verifications
  • parameterization
  • retry and listener mechanisms

In simple words:

Selenium drives the browser.
TestNG/JUnit drive the test execution.

Test Lifecycle (MOST IMPORTANT)​

Why Lifecycle Matters​

Automation tests need:

  • browser setup before tests
  • cleanup after tests
  • isolation between test cases

Typical Lifecycle (TestNG)​

@BeforeSuite
@BeforeClass
@BeforeMethod
@Test
@AfterMethod
@AfterClass
@AfterSuite

Understanding this prevents:

  • driver reuse bugs
  • data leakage between tests

Annotations (Execution Control)​

Automation Problem​

Manual setup and teardown logic everywhere.

Java Fix​

@BeforeMethod
public void setup() {
    // initialize driver
}

@AfterMethod
public void tearDown() {
    driver.quit();
}

Annotations tell the framework when to execute code.

Assertions vs Verifications​

Assertions (Hard Stop)​

Assert.assertTrue(isLoggedIn);
  • Stops test immediately on failure
  • Use for critical validations

Verifications (Soft Assertions)​

SoftAssert softAssert = new SoftAssert();
softAssert.assertTrue(isBannerVisible);
softAssert.assertAll();
  • Collects failures
  • Useful for UI-heavy validations

Parameterization (Data-Driven Tests)​

Automation Problem​

Hardcoded values inside tests.

Java Fix (TestNG DataProvider)​

@DataProvider
public Object[][] loginData() {
    return new Object[][] {
        {"user1", "pass1"},
        {"user2", "pass2"}
    };
}
@Test(dataProvider = "loginData")
public void loginTest(String user, String pass) {
    // test logic
}

Enables:

  • multiple scenarios
  • clean data-driven testing

Listeners (Observing Test Events)​

Automation Problem​

No visibility into test execution flow.

Java Fix​

Listeners observe:

  • test start
  • success
  • failure
  • skip

Used for:

  • logging
  • screenshots
  • reporting
public class TestListener implements ITestListener {
    public void onTestFailure(ITestResult result) {
        // capture screenshot
    }
}

Retry Logic (Framework-Level)​

Automation Problem​

Transient failures due to environment issues.

Java Fix​

Retry analyzers allow:

  • controlled re-execution
  • no retry logic inside tests

Rule:

Retry belongs to framework, not test logic.

Execution Order (Control When Needed)​

@Test(priority = 1)
public void testA() {}

@Test(priority = 2)
public void testB() {}

Use ordering:

  • sparingly
  • only when dependency is unavoidable

Real Automation Example (Clean Flow)​

public class LoginTest extends BaseTest {

    @Test
    public void validLogin() {
        LoginPage page = new LoginPage(driver);
        Assert.assertTrue(page.login("user", "pass"));
    }
}

Framework handles:

  • setup
  • execution
  • teardown
  • reporting

Test focuses only on validation.

Common Mistakes (Very Common)​

  • Writing setup logic inside test methods
  • Mixing assertions and verifications randomly
  • Hardcoding test data
  • Adding retry logic inside tests
  • Misunderstanding lifecycle order

Best Practices (Automation-Approved)​

  • Keep tests stateless
  • Use annotations for setup/teardown
  • Separate test logic from framework logic
  • Use DataProviders for test data
  • Use listeners for cross-cutting concerns

Interview Notes​

  • Selenium vs TestNG/JUnit responsibilities
  • Test lifecycle explanation
  • Assertions vs Soft Assertions
  • DataProvider usage
  • Role of listeners and retries

Summary (Human Understanding)​

Selenium automates browsers. Testing frameworks automate test execution.

Understanding framework internals:

  • prevents flaky tests
  • improves stability
  • makes you a real automation engineer