Getting Started with Contract Tests

[jbrains]

First things first, I need to fix the notification mechanism for comments here, because I didn’t notice one for your comment. I’m sorry that that failed and that I didn’t reply sooner. I would have wanted to.

I agree that Contract Testing doesn’t replace Customer Testing (which I believe to be Functional Testing with a name I prefer: testing designed to give Customers the warm, fuzzy feeling that we’ve enabled the capabilities they need). I consider Contract Testing as a kind of Programmer Testing (give the programmers confidence that their code behaves as they intend it to). I claim that relying on Customer Testing to find coding errors is overpaying for a guarantee compared to using Collaboration and Contract Testing for that purpose. Moreover, Collaboration and Contract Testing helps expose weaknesses in the design in a way that Customer Testing rarely does.

I use Contract Tests to change the role/purpose of Customer Tests, not to replace them. Even so, when we do it well, we need fewer Customer Tests for the same level of confidence both in the code and in the capabilities of the system.

@Cartius

I recall you saying that we can use contracts to test the scenario where an item gets put into a service with a request and then it appears in some kind of list after another request. However, it seems like the Pact maintainers recommend that you don’t keep track of the provider’s state with Pact and that the Pact contract is only for testing communication (syntax), not functional testing (semantic?).

I can’t comment on Pact, because I don’t use it. I can only comment on Contract Testing in general. I don’t know whether that invalidates whatever I write next in your eyes. You need to decide that.

As for the example you cite, I think I remember it. The example revolves around the contract of two Controllers in a Point of Sale system, collaborating to implement the feature of a Shopper (a person buy items from our store) purchasing multiple items in a single Purchase (a transaction unit with multiple items). One Controller handles scanning a product’s barcode and the other Controller handles the Cashier (the person scanning items and collecting money from the Shopper) signaling “these are all the items that this Shopper wants to buy”, presumably buy pressing some button probably marked “TOTAL”. In this case, we can imagine an event sequence like this:

barcode_scanned "12345"
barcode_scanned "23456"
barcode_scanned "34567"
total_pressed

Each Controller handles a different event and they have to work together to compute the total price of those three items (assuming that we know the prices of all three items) in order to display to both the Cashier and Shopper, “That’ll be $45.81”.

What is the responsibility of each Controller? Well, the Barcode Scanned Controller needs to signal to something that an item has been found and reserved for purchase (but, of course, only for barcodes that match products that we know about). The Total Pressed Controller needs to signal to something to summarize the current Shopper’s Purchase so that the Cashier knows how many money to demand from the current Shopper.

Which something? I don’t know. Let’s invent something. “Current Purchase”? No. What matters is something like pending Purchase. It’s the one accumulating at the moment. So we make an abstraction representing this idea and it needs two methods (I’ll use OOP language for now, just to make it easier to describe): addItem() and completePurchase(). What are the contract semantics of Pending Purchase?

We could try to get too detailed and say that completePurchase() must return a Purchase value whose totalCost property is equal to the sum of the prices of the items added by addItem(), but that’s already making assumptions about the implementation. What happens when we add taxes? or allow for discounts? Oy. No. Let’s let that be an implementation detail.

And maybe this is where the confusion arises. Certainly, we don’t want clients of Pending Purchase to know details about how Pending Purchase tracks its state. This means that we wouldn’t put in the contract semantics something such as “addItem() adds items to an internal list”. I mean… we could do that, but eventually we’d likely regret it. And maybe I made the mistake of suggesting that some time ago. If I did, then I was young and foolish.

If we choose not to do that, then what are the contract semantics of Pending Purchase? Something like this:

• addItem() never fails
• completePurchase() returns a Purchase that summarizes the pending purchase
• completePurchase() prepares Pending Purchase for the next Purchase, presumably buy doing something such as clearing its internal list of items reserved for purchase

…uh, no. Again, too many implementation details. What could we say instead?

• invoking completePurchase() a second time in a row returns an empty purchase

which is equivalent to

• invoking completePurchase() without any items previously added returns an empty purchase

(And, of course, if you prefer to avoid an empty Purchase, it could raise a “No Purchase in Progress” error or something like that.)

What Happened Here?

I tried to describe a Contract Test (or an aspect of the semantics of a contract), I noticed that the resuting test would overspecify the contract, so I used that as the trigger for raising the level of abstraction of the interaction between Client and Supplier. I used that as an opportunity to reduce the number of details that the Client needs to know to use the Supplier successfully.

At the same time, I illustrated the point of guidelines such as “Contract Tests aren’t designed to replace Customer Tests”, because I’ve moved the responsibility for “purchase cost = sum of item prices” somewhere else. I’ve made it a free choice in our design! This makes life easier for the programmers (less to get wrong), but riskier for the customers (more behavior to worry about). What’s truly happened is merely that the Controllers no longer need to care about this behavior, because either the implementation of Pending Purchase will handle it or the Purchase value object itself will handle it. Both options work fairly well, although I suspect that eventually the programmers will prefer one choice over the other. The OOP folks will push that behavior into Purchase and the FP folks will leave it as some pluggable policy that returns a Purchase Summary value that looks like { items = […], cost = Cents 4581 }.

This is how I use Contract Tests to guide my design.

What If I Don’t Have the Same Insight?

No worries. You can always write the more-complicated Contract Tests, which would result in more-complicated Collaboration Tests. At some point, you’d notice the duplication of irrelevant details in the Collaboration Tests (why do I have to do item price and sales taxes arithmetic in this damn test?! I’m not checking any of that arithmetic!!) and you could use that as the signal to raise the level of abstraction. And if you weren’t sure how, you’d ask for help. That’s how I learned it.

@Cartius

Also, I’m assuming that semantic contracts, similar to Pact contracts, aren’t used directly in top-to-bottom testing due to the combinatorial nature of those tests but instead alongside something like Mountebank, where the stubs are slightly permuted. Am I right in this assumption?

I use contract semantics in top-to-bottom design/implementation. I call it Client-First Design with Test Doubles. When I implement layer 6, my Collaboration Tests for that layer make assumptions about how layer 7 behaves. Those assumptions become the beginning of the Contract Test list for the abstractions in layer 7. I can often identify design regrets in those abstractions before I even try to implement layer 7. This way I notice the impending combinatorial explosion and stop it before it happens. Most of the time. Sometimes there are little explosions and I stop them before the factorial curve truly hurts me.

How much does all that help?