Black box testing for e-commerce apps

With testing techniques examples

What is black box testing?

Black box testing is a type of software testing where you test an application without knowing its internal code structure. You focus only on the inputs and the outputs, validating the functionality of the software based on its specifications and requirements.

For example, if you’re testing a login form on an e-commerce app, you enter a username and password and check if it logs you in, without worrying about how the code behind it processes your inputs. The goal is just to make sure the software behaves as expected.

What is a testing technique?

A testing technique is simply a method used to check whether a piece of software is working properly.

These techniques exist because they help make sure that software works for everyone, no matter how they use it. They are like different strategies to ensure that nothing gets missed.

Think of this like shopping for the perfect outfit online: you might explore different styles, colors, and sizes to find the right fit. Just like that, testers use different techniques to make sure everything in an app works smoothly, covering all the ways users might interact with it. This helps:

1. Catch bugs that might cause issues for users.

2. Make sure the software’s key features work properly.

3. Avoid missing important tests by covering all the main parts of the software.

What is test coverage?

Test coverage is a term used in software testing to describe how much of the software has been tested.

Imagine shopping online—if you only check the homepage, you miss out on great deals and products. Exploring all the categories and reading reviews gives you a better sense of what's available and leads to a better shopping experience.

When it comes to software, test coverage means making sure you’ve tested it in as many situations as possible. High coverage means you’ve checked the software in different ways, so you’re more likely to find problems before real users do.

For example, if you’re testing an e-commerce app, you’d want to check things like:

• Does the “Add to Cart” button work when adding just one item?

• What happens when you try to add 50 items (testing the limits)?

• What if the cart is empty and you try to checkout?

• What if you forget to enter your shipping address?

Each of these tests would increase your coverage, making sure the system works for all kinds of situations.

When is test coverage enough?

Knowing when test coverage is "enough" can be tricky, but there are some practical ways to figure it out without getting too technical. In short:

1. Make sure your tests cover the critical features

For example, in an e-commerce app, you should make sure:

• Users can add, remove, and update items in their cart.

• The checkout process works smoothly.

• Payments go through correctly.

• Orders are correctly placed and confirmed.

• Orders can be tracked later on.

2. Test the most common user scenarios

Think about the most common ways people will use the software and make sure you’ve covered those scenarios. If it’s a banking app, have you tested:

• Browsing and searching for products?

• Adding items to the cart and updating quantities?

• Proceeding to checkout, perhaps with discounts or coupons?

• Completing a payment with different payment methods (credit card, PayPal)?

• Order confirmations once a purchase is completed?

3. Test for edge cases

Ideally you would also test some weird or rare situations. For example:

• What if a user tries to buy 1000 items at once?

• What happens if the internet connection drops during a payment?

• What if the user tries to apply multiple discount codes or combine incompatible promotions?

• What if a user tries to order more items than are in stock?

• What happens if a user takes a long time to complete the checkout, and their session expires?

You don’t need to test every possible edge case, but handling the most likely or risky ones adds confidence that the software won’t break in strange situations.

If you’ve already covered the critical features, common user scenarios, and edge cases, adding more tests might not reveal many new bugs (this is called “diminishing returns”). At this point, you might decide that further testing isn't worth the time or resources. Because in real life, time and resources are limited—you can’t always test everything.

In conclusion, test coverage is enough when:

• You’ve covered the important parts.

• You’ve handled common and risky scenarios.

• You’ve handled a few weird or rare cases that may occur.

• You’ve reached a point where more testing won’t add much value.

• You feel confident that the software is reliable.

Test coverage in black box testing

As we established earlier, when you’re doing black box testing, your main concern is what the software does, not how it works internally. You’re trying to make sure the software behaves as expected in different real-world situations, using different inputs and scenarios. Here are some techniques you might use for this:

• Equivalence Partitioning (EP)

  Instead of testing every possible input, you divide inputs into groups and test one value from each group.

  For example, if an e-commerce app allows users to apply discount codes, you can group the codes into three categories: valid discount codes (like AUTUMN2024), expired discount codes (like SUMMER2023), and invalid discount codes (incorrect ones or invalid format, like DISCOUNT). This saves time but still ensures you are covering different discount code scenarios.

• Boundary Value Analysis (BVA)

  This involves testing the software at its limits.

  For example, if the e-commerce app allows users to add between 1 and 50 items to the cart, you would test it with 1 item (the lower boundary), 50 items (the upper boundary), and values just outside that range (like 0 or 51 items) to see how the system handles these edge cases.

• Decision Tables

  This method helps you see how different combinations of inputs affect the system’s outcome.

  For example, when testing shipping costs, you can set up a decision table with combinations of order amount and shipping type (standard vs. express). The decision table would show how the system calculates shipping based on these variables, ensuring it handles each combination correctly (i.e. free shipping for orders over $50 with standard shipping, but a fee for express).

• State Transition Testing

  Some software has different "states", and this technique tests how the system handles moving between them.

  In e-commerce, an example could be testing the order status flow—when an order moves from "pending" to "processing", then "shipped", and finally "delivered". State transition testing ensures the system updates the status correctly as the order progresses and handles unexpected changes (i.e. cancelling an order after it's been shipped).

These techniques help you cover a wide range of scenarios, making sure the software works smoothly in different situations.

Applying testing techniques

Let’s detail the earlier examples on how to apply the testing techniques for an e-commerce app.

Equivalence Partitioning (EP)

As a reminder, equivalence partitioning divides input data into groups (partitions) to ensure that the system behaves consistently within each partition.

Scenario: Testing the flow of applying a discount code.

Equivalence partitions could be:

• Partition 1: Valid discount codes (i.e. 10%OFF, AUTUMN2024) — includes all correctly formatted, non-expired discount codes.

• Partition 2: Invalid discount codes (expired) — codes that have passed the expiration date (i.e. SUMMER2023).

• Partition 3: Invalid discount codes (incorrect format) — codes that do not follow the expected format (i.e. DISCOUNT, 1234).

Examples test cases:

• Entering a valid discount code at checkout: 10%OFF

• Entering an expired discount code at checkout: SUMMER2023

• Entering an incorrect discount code at checkout: DISCOUNT

Coverage—3 test cases:

• 1 test case for the valid equivalence partition

• 2 test cases for the invalid equivalence partitions (1 for expired, 1 for incorrect)

Boundary Value Analysis (BVA)

As we learned, this technique focuses on testing input values at the limits (boundaries) of valid ranges to uncover potential defects.

Scenario: Testing the ability to purchase a maximum of 50 items per order.

The boundary values will be:

• Valid Values: 1, 50 (within the limit)

• Invalid Values: 0 (below the limit), 51 (above the limit)

Examples test cases:

• Purchasing 1 item (minimum valid quantity—lower boundary)

• Attempting checkout with 0 items (below the lower boundary—invalid)

• Purchasing 50 items (maximum valid quantity—upper boundary)

• Purchasing 51 items (just above the upper boundary—invalid)

Coverage—4 test cases:

• 1 test case for lower boundary (valid)

• 1 test case for just below lower boundary (invalid)

• 1 test case for upper boundary (valid)

• 1 test case for just above upper boundary (invalid)

Decision Table

A decision table is a representation of conditions (inputs) and corresponding actions (outputs) used to specify the behavior of a system. 

Scenario: Testing shipping costs based on different combinations of order amount and shipping type.

Example conditions:

• Order amount:

  • C1: Less than $50

  • C2: $50 or more

• Shipping type:

  • C3: Standard shipping

  • C4: Express shipping

Example actions:

• A1: Shipping cost

Shipping cost rules:

• Standard shipping:

  • Orders under $50: $5 shipping fee

  • Orders $50 or more: free shipping

• Express shipping:

  • Orders under $50: $10 shipping fee

  • Orders $50 or more: $5 shipping fee

The decision table would look like this:

Coverage—4 test cases, 1 for each rule.

The table outlines how different combinations of conditions lead to specific shipping costs (actions) based on the defined rules. It can serve as guideline for testing.

State Transition Testing

A state transition diagram shows all valid transitions and includes pairs of transitions between two states. Any pair missing is considered invalid transition.

Scenario: Testing an e-commerce order status flow.

Example states:

1. Pending: The order has been placed but not yet processed.

2. Processing: The order is being prepared for shipment.

3. Shipped: The order has been dispatched from the warehouse.

4. Delivered: The order has reached the customer.

5. Cancelled: The order has been cancelled, which can occur at various stages.

The transitions would be:

A: Pending to Processing — when payment is confirmed.

B: Processing to Shipped — when order is packed and dispatched.

C: Shipped to Delivered — after delivery confirmation from the carrier.

D: Pending to Cancelled — if the user requests cancellation while the order is pending.

E: Processing to Cancelled — if the user requests cancellation while processing order.

F: Shipped to Cancelled — if the user requests cancellation after shipping.

A simple visual representation of the transitions would be this:

Notice that the normal transitions are represented at the top, but cancellation can also occur at different stages (unexpected transitions).

The state transition table could look like this:

Coverage—6 test cases, 1 for each possible transition.

You can also consider simulating invalid transitions, such as "Shipped to Processing", if the system allows it. A tester should ensure the system responds appropriately to undesired conditions, returning a helpful error message to the user.

Note that the state transition technique typically involves a visual representation of all the states and a table with the possible transitions based on different events. This visual representation helps testers identify all the potential cases and determine if any transitions (whether possible or not) have been overlooked and should be verified.

I hope you find these examples clear and useful. Understanding various testing techniques and making sure to cover all areas are important steps to deliver a product that meets user expectations and quality standards, even when the time and resources for testing are limited.