Postman Test Results Analysis: Interest Group Data Validation for Amazon Ad System API

I’ve recently run a Postman test suite on an API response related to our interest group data structure. This suite has previously worked without issue, but with the current response, several tests have failed. Here’s a breakdown of the results and insights into what might be causing these failures.

Current Response:

{
    "interestGroups": [
        {
            "adSizes": {},
            "ads": [],
            "biddingLogicUrl": "https://at-us-east.amazon-adsystem.com/biddingLogic/bidder_v2.js",
            "name": "293ceb3f-105f-4465-9abd-09ac915162c6",
            "owner": "https://at-us-east.amazon-adsystem.com/",
            "sizeGroups": {},
            "trustedBiddingSignalsKeys": [
                "FAKE"
            ],
            "trustedBiddingSignalsUrl": "https://at-us-east.amazon-adsystem.com/biddingSignals/cs",
            "updateUrl": "https://at-us-east.amazon-adsystem.com/interestGroup/update?ig_name=293ceb3f-105f-4465-9abd-09ac915162c6&label=C-A1JSFF43SRWGQ6&m=1",
            "userBiddingSignals": {}
        }
    ]
}

Test Script

// Parse the JSON response
const jsonData = pm.response.json();

// Test: Response status is 200
pm.test("Response status is 200", function () {
    pm.response.to.have.status(200);
});

// Test: Response time is less than 300ms
pm.test("Response time is acceptable", function () {
    pm.expect(pm.response.responseTime).to.be.below(300);
});

// Test: Response has the correct structure
pm.test("Response structure is correct", function () {
    pm.expect(jsonData).to.have.property('interestGroups').that.is.an('array').with.lengthOf.above(0);
    pm.expect(jsonData.interestGroups[0]).to.include.keys('adSizes', 'ads', 'biddingLogicUrl', 'name', 'owner', 'sizeGroups', 'trustedBiddingSignalsKeys', 'trustedBiddingSignalsUrl', 'updateUrl', 'userBiddingSignals');
});

// Test: adSizes is an object with specific dimensions
pm.test("adSizes structure is valid", function () {
    const adSizes = jsonData.interestGroups[0].adSizes;
    pm.expect(adSizes).to.be.an('object');
    // Validate specific ad sizes exist
    pm.expect(adSizes).to.have.property('350x420');
    pm.expect(adSizes['350x420']).to.include.keys('height', 'width');
    pm.expect(adSizes['350x420'].height).to.be.a('number').and.to.be.above(0);
    pm.expect(adSizes['350x420'].width).to.be.a('number').and.to.be.above(0);
});

// Test: ads is an array with expected properties
pm.test("ads structure is valid", function () {
    const ads = jsonData.interestGroups[0].ads;
    pm.expect(ads).to.be.an('array').with.lengthOf.above(0);
    pm.expect(ads[0]).to.include.keys('metadata', 'renderUrl');
    pm.expect(ads[0].metadata).to.include.keys('adId', 'campaignId', 'creativeId', 'creativeSizes', 'flightId', 'geoMatch', 'googleBillingId', 'igAudienceCategory', 'language', 'realTimeMatch', 'sizeGroup');
});

// Test: Check specific metadata values
pm.test("Metadata values are valid", function () {
    const metadata = jsonData.interestGroups[0].ads[0].metadata;
    pm.expect(metadata.adId).to.be.a('string');
    pm.expect(metadata.campaignId).to.be.a('string');
    pm.expect(metadata.creativeId).to.be.a('string');
    pm.expect(metadata.creativeSizes).to.be.an('array').that.is.not.empty;
    pm.expect(metadata.flightId).to.be.a('string');
});

// Test: Check URLs are valid
pm.test("URLs are valid", function () {
    const biddingLogicUrl = jsonData.interestGroups[0].biddingLogicUrl;
    const updateUrl = jsonData.interestGroups[0].updateUrl;
    pm.expect(biddingLogicUrl).to.match(/^https?:\/\//);
    pm.expect(updateUrl).to.match(/^https?:\/\//);
});

// Test: Check that trustedBiddingSignalsKeys is an array
pm.test("trustedBiddingSignalsKeys is an array", function () {
    pm.expect(jsonData.interestGroups[0].trustedBiddingSignalsKeys).to.be.an('array');
});

// Test: Check sizeGroups contain expected size groups
pm.test("sizeGroups structure is valid", function () {
    const sizeGroups = jsonData.interestGroups[0].sizeGroups;
    pm.expect(sizeGroups).to.be.an('object');
    pm.expect(sizeGroups).to.have.property('g1').that.is.an('array').with.lengthOf.above(0);
});

// Test: Check for presence of trustedBiddingSignalsUrl
pm.test("trustedBiddingSignalsUrl is present", function () {
    pm.expect(jsonData.interestGroups[0]).to.have.property('trustedBiddingSignalsUrl').that.is.a('string');
});

Overview of the Test Results

The tests aimed to validate the response structure, specific properties, and data integrity of certain fields. Here’s how each test fared, along with notes on any issues encountered:

1. Response Status and Performance

• Test: Response status is 200 – PASS
  • The server returned the expected 200 status code.
• Test: Response time is acceptable – PASS
  • Response time was within the acceptable threshold of 300ms.

2. Response Structure

• Test: Response structure is correct – PASS
  • The overall structure of the response contained the interestGroups array, which is essential for our validation.

3. adSizes Structure Validation

• Test: adSizes structure is valid – FAIL
  • Issue: The adSizes object was present but empty ({}), missing the expected 350x420 key with height and width properties.
  • Implication: This may indicate that the response structure has changed, or the 350x420 size group is no longer part of this interest group’s data.

4. ads Array Validation

• Test: ads structure is valid – FAIL
  • Issue: The ads array was empty, contrary to expectations that it should contain at least one ad with properties like metadata and renderUrl.
  • Implication: Either this interest group currently has no associated ads, or there’s been a structural change in how ads are handled in the response.

5. Metadata Values within ads

• Test: Metadata values are valid – FAIL
  • Issue: The test couldn’t find metadata within the ads array since the array itself was empty.
  • Implication: This is directly related to the failure in the ads array validation. With no ads present, metadata validation is not possible.

6. URL Validation

• Test: URLs are valid – PASS
  • The URLs for biddingLogicUrl and updateUrl were present and correctly formatted.

7. trustedBiddingSignalsKeys Array Validation

• Test: trustedBiddingSignalsKeys is an array – PASS
  • This array was correctly formatted and present as expected.

8. sizeGroups Structure Validation

• Test: sizeGroups structure is valid – FAIL
  • Issue: The sizeGroups object was present but empty, missing the expected g1 key with associated array values.
  • Implication: This suggests that either no size groups are currently defined, or there’s been a structural change impacting the sizeGroups content.

9. trustedBiddingSignalsUrl Presence

• Test: trustedBiddingSignalsUrl is present – PASS
  • This URL was correctly formatted and present in the response.

Summary of Failed Tests and Potential Causes

The failures in this test suite seem to stem from either missing data in the response or possible changes in the API’s response structure. Specifically:

• Empty Objects or Arrays: Fields like adSizes, ads, and sizeGroups are expected to contain specific values or structures but are empty. This might indicate that the interest group data has been trimmed or is no longer structured the way it was in previous responses.
• Dependent Failures: Several tests failed because they rely on the presence of specific structures or nested values that no longer appear in the response.

Next Steps

1. Investigate API Changes: Confirm if there have been any recent updates to the API that may have altered the response format or made optional data conditional.
2. Adapt Tests for Optional Fields: If some fields are now optional, update the test suite to accommodate these variations. This could involve adding conditional checks or fallbacks when data is missing.
3. Validate with Updated Data: Run the tests with different interest group data to determine if these issues are specific to this response or if they apply across other interest groups as well.