Why Testing Matters โ and the Trap Most Test Suites Fall Into
Automated tests exist to answer one question with confidence: "does the code actually do what it's supposed to do?" The trap โ and it's an extremely common one, even among developers fluent in JUnit and Mockito syntax โ is writing a test that makes the code pass through without ever actually checking that it did the right thing. A test suite full of syntactically correct tests that don't verify real behavior is worse than no tests at all, because it gives everyone false confidence to refactor and deploy.
The Testing Pyramid
/\
/ \ E2E Tests (few) โ full system, slowest
/----\
/ \ Integration Tests (some) โ real collaborators at the boundary
/--------\
/ \ Unit Tests (many) โ fast, isolated, the bulk of your suite
/------------\
Tests That Pass But Don't Test Anything
This is the single most common way an experienced team ends up with "100% coverage" and production bugs anyway. Three concrete failure patterns, and how to actually catch them.
1. Tautological assertions
// LOOKS like a test. Verifies almost nothing.
@Test
void createOrder() {
Order result = orderService.createOrder(request);
assertNotNull(result); // passes even if total, status, and items are all wrong
}
// ACTUALLY tests the behavior a caller depends on
@Test
void createOrder_calculatesCorrectTotal() {
Order result = orderService.createOrder(request);
assertEquals(OrderStatus.CREATED, result.getStatus());
assertEquals(new BigDecimal("59.98"), result.getTotal());
assertEquals(2, result.getItems().size());
}
2. Over-mocking โ the test verifies the implementation, not the outcome
// This test mocks every collaborator so thoroughly that it just re-describes
// the method body, line by line, instead of checking what the caller
// actually cares about: did the order get created correctly?
@Test
void createOrder_overSpecified() {
orderService.createOrder(request);
verify(inventoryService).checkAvailability("item-1", 2);
verify(pricingService).calculateSubtotal(any());
verify(taxService).calculateTax(any());
verify(paymentService).process(any());
verify(orderRepository).save(any());
verify(auditLog).record(any());
// Reorder these calls harmlessly inside createOrder(), or inline one of
// these collaborators โ this test breaks, even though the actual output
// (the returned Order) never changed. It's coupled to HOW, not WHAT.
}
// Verify only the interactions that represent a real side effect the
// caller depends on โ not every internal step
@Test
void createOrder_persistsAndCharges() {
Order result = orderService.createOrder(request);
assertEquals(OrderStatus.CREATED, result.getStatus());
verify(orderRepository).save(any(Order.class)); // a real side effect callers rely on
verify(paymentService).process(any()); // same
// No assertions about pricing/tax internals โ those get their own
// focused unit tests on PricingService and TaxService directly.
}
3. The mutation testing gut check
Here's the honest question coverage tools can't answer: if you deleted a business rule from the code, would any test actually fail? Coverage only tells you a line executed โ not that a test would notice if that line's logic were wrong or missing entirely.
public void validateOrder(Order order) {
if (order.getTotal().compareTo(BigDecimal.ZERO) <= 0) {
throw new InvalidOrderException("Total must be positive");
}
}
// If your test suite only ever exercises this method with a valid,
// positive-total order, this line has 100% line coverage โ and a test
// suite that would pass identically if you deleted the entire if-block.
PIT (pitest) automatically mutates your
compiled code in small ways โ flips a
<= to <, changes a
return true to return false,
removes a method call โ and reruns your test suite against
each mutant. If your tests still pass with the mutation in
place, that mutant survived, meaning no
test actually verifies that specific piece of logic. A
surviving mutant on a business rule like the one above is a
precise, automated way to find exactly the "tests that pass
but don't test anything" problem โ far more reliable than
eyeballing coverage percentages.
<!-- pom.xml -->
<plugin>
<groupId>org.pitest</groupId>
<artifactId>pitest-maven</artifactId>
<version>1.15.8</version>
<configuration>
<targetClasses>
<param>com.shop.order.*</param>
</targetClasses>
</configuration>
</plugin>
mvn org.pitest:pitest-maven:mutationCoverage
# Report shows KILLED vs SURVIVED mutants per class โ a surviving mutant
# on a conditional you thought was tested is a direct signal to add a
# test for the missing branch.
Running PIT on the full codebase on every commit is usually too slow โ most teams run it selectively on critical modules (payment, pricing, order validation) or as a periodic CI job rather than on every push.
Real Network Calls Leaking Into Your Test Suite
The other classic way "tests" quietly become something worse
than useless: a test that hardcodes a real URL โ
localhost:8080, a real payment gateway sandbox, a
real third-party API โ and makes an actual network call every
single time the suite runs. This makes tests slow, flaky
(they fail when the network or the third party is down, not
when your code is wrong), and occasionally dangerous (a "test"
that accidentally hits a production endpoint).
// BAD โ a "unit" test that makes a real HTTP call on every run
@Test
void chargePayment() {
RestTemplate restTemplate = new RestTemplate();
PaymentResponse response = restTemplate.postForObject(
"http://localhost:8080/payment-gateway/charge", // real call, real dependency, every run
request, PaymentResponse.class);
assertTrue(response.isSuccessful());
// Fails if the gateway isn't running locally. Fails differently in CI.
// Says nothing reliable about YOUR code's correctness.
}
For pure unit tests โ mock the collaborator, never the network
If PaymentService is a Spring bean wrapping the
HTTP call, mock PaymentService itself with
Mockito, exactly as covered in Section 6 โ there's no reason a
unit test for OrderService should know an HTTP
call is involved at all.
For integration tests that must exercise real HTTP wiring โ use WireMock
<!-- pom.xml -->
<dependency>
<groupId>org.wiremock</groupId>
<artifactId>wiremock-standalone</artifactId>
<version>3.9.1</version>
<scope>test</scope>
</dependency>
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class PaymentServiceIntegrationTest {
static WireMockServer wireMockServer = new WireMockServer(0); // random free port โ never hardcoded
@BeforeAll
static void startWireMock() {
wireMockServer.start();
configureFor("localhost", wireMockServer.port());
}
@AfterAll
static void stopWireMock() { wireMockServer.stop(); }
@Test
void chargePayment_gatewaySucceeds() {
stubFor(post(urlEqualTo("/charge"))
.willReturn(aResponse()
.withStatus(200)
.withHeader("Content-Type", "application/json")
.withBody("{\"successful\": true, \"transactionId\": \"tx-123\"}")));
PaymentResponse response = paymentService.charge(request);
assertTrue(response.isSuccessful());
verify(postRequestedFor(urlEqualTo("/charge"))); // confirms the real HTTP call shape, no real network
}
}
For database integration tests โ Testcontainers, never a real localhost service
Pointing an integration test's datasource URL at
jdbc:postgresql://localhost:5432/mydb means the
test only passes on machines that happen to have a Postgres
instance running locally with matching credentials โ it's not
reproducible, and it silently passes against whatever schema
state that local database happens to be in.
@Testcontainers
@DataJpaTest
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
class CustomerRepositoryTest {
@Container
static PostgreSQLContainer<?> postgres =
new PostgreSQLContainer<>("postgres:16-alpine"); // a real, disposable Postgres โ same engine as production
@DynamicPropertySource
static void configure(DynamicPropertyRegistry registry) {
registry.add("spring.datasource.url", postgres::getJdbcUrl);
registry.add("spring.datasource.username", postgres::getUsername);
registry.add("spring.datasource.password", postgres::getPassword);
}
@Autowired
private CustomerRepository customerRepository;
@Test
void findByEmail_customerExists_returnsCustomer() {
customerRepository.save(new Customer("jane@shop.com", "Jane Doe"));
Optional<Customer> found = customerRepository.findByEmail("jane@shop.com");
assertTrue(found.isPresent());
assertEquals("Jane Doe", found.get().getFullName());
}
}
Testcontainers starts an actual Postgres in Docker
specifically for this test class and tears it down
afterward โ every developer's machine and every CI run gets
an identical, empty database on the exact same engine
version as production. This closes the gap plain
@DataJpaTest with its default embedded H2
leaves open too: H2 doesn't enforce every constraint or SQL
dialect quirk the same way Postgres does, so a query that
passes against H2 can still fail against the real engine in
production.
JUnit 5 Fundamentals
Maven Dependencies
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>5.11.0</version>
<scope>test</scope>
</dependency>
Basic Test Structure
import org.junit.jupiter.api.*;
import static org.junit.jupiter.api.Assertions.*;
class DiscountCalculatorTest {
private DiscountCalculator calculator;
@BeforeEach
void setUp() {
calculator = new DiscountCalculator();
}
@Test
@DisplayName("Applies a 10% discount to orders over $100")
void appliesDiscountAboveThreshold() {
BigDecimal result = calculator.apply(new BigDecimal("200.00"));
assertEquals(new BigDecimal("180.00"), result);
}
@Test
@DisplayName("Rejects a negative order total")
void rejectsNegativeTotal() {
assertThrows(IllegalArgumentException.class,
() -> calculator.apply(new BigDecimal("-1")));
}
}
Common Assertions
assertEquals(expected, actual);
assertThrows(IllegalArgumentException.class, () -> service.process(null));
assertAll("Customer properties",
() -> assertEquals("Jane", customer.getFullName()),
() -> assertNotNull(customer.getEmail())
);
Lifecycle Annotations
@BeforeAll static void initAll() { } // once, before all tests in the class
@BeforeEach void init() { } // before every test โ fresh state, no leakage between tests
@AfterEach void tearDown() { } // after every test
@AfterAll static void tearDownAll() { } // once, after all tests
@Disabled("Blocked on PAY-1421") // skip with a reason, not silently
Parameterized Tests
@ParameterizedTest
@CsvSource({
"100.00, 90.00", // above threshold โ discounted
"50.00, 50.00", // below threshold โ unchanged
"100.01, 90.01" // boundary โ exactly above
})
void applyDiscount_variousTotals(BigDecimal input, BigDecimal expected) {
assertEquals(expected, calculator.apply(input));
}
@ParameterizedTest
@MethodSource("provideCustomers")
void isValid_variousCustomers(Customer customer, boolean expected) {
assertEquals(expected, validator.isValid(customer));
}
static Stream<Arguments> provideCustomers() {
return Stream.of(
Arguments.of(new Customer("jane@shop.com", "Jane"), true),
Arguments.of(new Customer("invalid-email", "Jane"), false)
);
}
Boundary cases (like the 100.01 row above) matter
more than adding a fourth arbitrary "normal" value โ this is
exactly the kind of case a surviving mutant from Section 1 tends
to expose.
Mockito
Maven Dependencies
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-junit-jupiter</artifactId>
<version>5.14.0</version>
<scope>test</scope>
</dependency>
Creating Mocks
@ExtendWith(MockitoExtension.class)
class CustomerServiceTest {
@Mock private CustomerRepository customerRepository;
@InjectMocks private CustomerService customerService;
@Test
void findById_customerExists_returnsCustomer() {
Customer mockCustomer = new Customer(1L, "Jane");
when(customerRepository.findById(1L)).thenReturn(Optional.of(mockCustomer));
Customer result = customerService.findById(1L);
assertEquals("Jane", result.getFullName());
}
}
Stubbing and Verification
when(repository.findById(1L)).thenReturn(Optional.of(customer));
when(service.process(null)).thenThrow(new IllegalArgumentException());
doThrow(new RuntimeException()).when(service).dangerousMethod();
verify(repository, times(1)).save(any());
verify(repository, never()).delete(any());
// ArgumentCaptor โ inspect what was actually passed, not just that it was called
ArgumentCaptor<Customer> captor = ArgumentCaptor.forClass(Customer.class);
verify(repository).save(captor.capture());
assertEquals("Jane", captor.getValue().getFullName());
verifyNoMoreInteractions is brittle by default โ use it sparinglyAsserting there were absolutely no other interactions with
a mock ties the test to the exact set of calls the current
implementation happens to make. Adding an unrelated,
harmless logging call inside the method under test breaks
every such assertion across the suite. Reserve it for cases
where "nothing else must happen" is itself the actual
business rule you're testing (Section 1's
createOrderItemNotAvailable example below is a
legitimate use โ payment must never be attempted if
inventory checks fail).
A Complete, Well-Scoped Test Class
@ExtendWith(MockitoExtension.class)
class OrderServiceTest {
@Mock private OrderRepository orderRepository;
@Mock private PaymentService paymentService;
@Mock private InventoryService inventoryService;
@InjectMocks private OrderService orderService;
@Nested
@DisplayName("When creating an order")
class CreateOrder {
@Test
@DisplayName("succeeds and persists the order with a CREATED status")
void createOrderSuccess() {
OrderRequest request = new OrderRequest("item-1", 2);
when(inventoryService.checkAvailability("item-1", 2)).thenReturn(true);
when(paymentService.process(any())).thenReturn(PaymentResult.success());
Order result = orderService.createOrder(request);
assertEquals(OrderStatus.CREATED, result.getStatus());
verify(orderRepository).save(any(Order.class));
}
@Test
@DisplayName("throws and never attempts payment when inventory is insufficient")
void createOrderItemNotAvailable() {
OrderRequest request = new OrderRequest("item-1", 100);
when(inventoryService.checkAvailability("item-1", 100)).thenReturn(false);
assertThrows(InsufficientInventoryException.class,
() -> orderService.createOrder(request));
verify(paymentService, never()).process(any()); // a real business rule, legitimately verified
}
}
}
Integration Testing with Spring Boot
@SpringBootTest
@AutoConfigureMockMvc
class CustomerControllerIntegrationTest {
@Autowired private MockMvc mockMvc;
@Autowired private ObjectMapper objectMapper;
@Test
void createCustomer_validRequest_returnsCreated() throws Exception {
CustomerDTO customer = new CustomerDTO("Jane", "jane@shop.com");
mockMvc.perform(post("/api/customers")
.contentType(MediaType.APPLICATION_JSON)
.content(objectMapper.writeValueAsString(customer)))
.andExpect(status().isCreated())
.andExpect(jsonPath("$.email").value("jane@shop.com"));
}
}
See Section 2 for how to keep the database and any downstream HTTP calls behind this kind of test real but not machine-dependent, via Testcontainers and WireMock.
Test Coverage โ and Its Real Limits
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.12</version>
<executions>
<execution><goals><goal>prepare-agent</goal></goals></execution>
<execution><id>report</id><phase>test</phase><goals><goal>report</goal></goals></execution>
</executions>
</plugin>
JaCoCo tells you a line executed at least once during the suite. It cannot tell you whether any assertion actually depended on that line's specific logic being correct โ that gap is exactly what Section 1's PIT setup measures. Treat a high JaCoCo percentage as necessary, not sufficient.
Best Practices and Common Pitfalls
โ Do
- Assert on the actual outcome a caller depends on (status, total, thrown exception) โ not just "it's not null"
- Verify only the mock interactions that represent a real side effect โ not every internal collaborator call
- Run mutation testing (PIT) on critical modules to find assertions that don't actually verify anything
- Use WireMock/MockWebServer for HTTP boundaries and Testcontainers for database boundaries โ never a hardcoded real URL or a "just make sure Postgres is running locally" instruction
- Test boundary and error conditions deliberately, not just one happy-path example
โ Don't
- Don't write a test whose only assertion is
assertNotNull()on a complex result - Don't mock every single collaborator so precisely that the test just restates the method body โ that couples the test to implementation, not behavior
- Don't hardcode
localhostor any real endpoint in a test โ it makes the suite flaky, slow, and non-reproducible across machines - Don't chase a coverage percentage as the goal โ a suite can hit 100% line coverage and still not catch a deleted business rule
- Don't use
verifyNoMoreInteractionsreflexively โ reserve it for cases where "nothing else happens" is itself the rule under test
Interview Questions
Q: What's wrong with a test that only calls assertNotNull(result) on the return value of a method that computes a discount?
It confirms the method returned something, but not that the
something is correct. A method with a completely wrong discount
calculation would still pass this test as long as it returns a
non-null value. The test should assert the actual expected
value.
Q: Why should a unit test for a service that calls an external payment gateway mock that gateway instead of calling it for real?
A real call makes the test depend on network availability, the
third party's uptime, and the state of a system outside your
control โ turning a fast, deterministic unit test into a slow,
flaky one that can fail for reasons that have nothing to do
with your code. Mocking the collaborator keeps the test focused
on your own logic.
Q: What's the AAA pattern in test structure?
Arrange (set up the inputs and mock behavior), Act (call the
method under test), Assert (verify the outcome). Keeping these
three phases visually separate makes a test's intent clear at
a glance.
Q: A codebase has 95% JaCoCo line coverage, yet a critical business rule was silently broken for two releases before anyone noticed. How is this possible, and what tool would have caught it?
Line coverage only confirms that a line of code executed
during the test run โ it says nothing about whether any
assertion actually depended on that line's specific behavior
being correct. A test can execute a validation check and still
pass regardless of whether that check's condition is right,
wrong, or entirely removed, as long as no assertion in the test
happens to depend on the distinction. Mutation testing (PIT)
is built exactly for this gap: it automatically introduces
small, deliberate faults into the compiled code โ inverting a
boolean condition, changing a comparison operator, deleting a
method call โ and reruns the suite against each mutant. A
mutant that survives (the suite still passes despite the
injected fault) is a precise, automated signal that no test
actually depends on that logic being correct, which a coverage
percentage alone can never reveal.
Q: Explain concretely why a test that calls verify() on every single mock interaction inside a method is a maintainability liability, even though it looks thorough.
Verifying every internal collaborator call couples the test to
the current implementation's exact sequence of steps rather
than to the method's observable contract. Any behavior-neutral
refactor โ reordering two independent calls, inlining a helper,
extracting a new private method that wraps an existing call โ
breaks the test even though the actual output for every caller
is identical. This inverts the entire purpose of a test suite:
instead of giving developers confidence to refactor freely, an
over-specified test actively punishes safe refactoring,
training the team to either avoid refactoring or to treat
broken tests as noise to silence rather than signal to
investigate โ which is how a team ends up ignoring a test
suite altogether. The fix is verifying only the interactions
that represent an actual side effect a caller or the business
depends on (a payment being charged, a row being persisted),
and testing internal collaborators' own logic through their
own dedicated unit tests instead.
Q: Your integration test suite passes locally but fails intermittently in CI with connection-refused errors against a database. What's the likely root cause, and what's the fix that removes the flakiness structurally rather than just retrying?
The likely cause is that the test's datasource configuration
points at a fixed host and port โ typically
localhost with a well-known port โ assuming a
compatible database is already running there. This works by
accident on a developer's machine that happens to have one
running locally, and fails unpredictably in CI depending on
container startup ordering, port availability, or whether the
CI environment provisions that service at all. Adding retries
treats the symptom, not the cause, and doesn't fix the
underlying non-reproducibility. The structural fix is
Testcontainers: the test itself launches a real, disposable
instance of the actual database engine used in production, gets
a dynamically allocated port and connection URL back from the
container at runtime, and tears the container down when the
test class finishes. This removes any assumption about what's
already running on the host entirely โ the test is
self-contained and produces the same result on any machine with
Docker available, developer laptop or CI runner alike.