One of the people who watched the 2009 version of Integrated Tests Are A Scam recently asked me: I wonder how you deal with updates of third-party libraries. How do you detect subtle API or behaviour changes? At the moment, I write state-based integration tests for these cases and I wonder whether this isn’t a sensible use of integration tests.
Nice one.
Isn't there a bloat of tests if you have several 3rd party libraries? What is your experience on detecting changes in 3rd party code, did you catch a lot of bugs, would those bugs be hard to find or easy to spot?? if you can, pls share your experience, it is very interesting
A "bloat of tests" compared to what? Typically I'll write these tests either when I'm learning a new library, using a new part of a library, or when a library becomes difficult to trust. I wouldn't just write them completely blindly.
aha, that was my point, because I assumed you were writing tests for all your 3rd party software (or as much as possible - that's why I thought you'll end up with a lot of tests). I think running contract test against adapters is a good idea and I want to try it out on a small project I am currently working on. But because it is a small project I was wondering what was your experience with this approach - if the errors introduced by changes in 3rd party code were subtle or easily detectable or of both types.