Hi, so I’ve been really interested in using Contract testing for replacing e2e tests. However, one thing that’s been really tripping me up is the idea of testing semantic vs. syntactic using contract tests. Advocates of syntactic testing seem to be against the idea of using contract testing to replace e2e tests.
I recall you saying that we can use contracts to test the scenario where an item gets put into a service with a request and then it appears in some kind of list after another request. However, it seems like the Pact maintainers recommend that you don’t keep track of the provider’s state with Pact and that the Pact contract is only for testing communication (syntax), not functional testing (semantic?).
Also, in the example I linked above, their issue doesn’t seem as much as an issue with functional testing, so much as robustness, because the detail that given the username is longer than 20 characters, when saving the user object, then the server returns an error, seems to be testing a property of the server, that in their case the client doesn’t depend on and therefore does not need to test. Although that doesn’t seem to imply anything about a limitation of doing functional testing using a contract such as testing given a request to put the object into the service, when making a request to list objects in the service, then it should appear in the list.
What can I do differently than Pact that would allow me to effectively test semantics without running into the same problems? Is it just allowing for sequences of requests in a single contract? If not would you happen to have an example of what I can do?
Also, I’m assuming that semantic contracts, similar to Pact contracts, aren’t used directly in top-to-bottom testing due to the combinatorial nature of those tests but instead alongside something like Mountebank, where the stubs are slightly permuted. Am I right in this assumption?
Thanks!