I find this a matter of personal preference. Some people like to put everything in one place, then look for signals about it growing too large. Others, like me, prefer to build many little things, then look for signals about ideas being scattered too widely. I find that knowing one's preference and which signals to watch out for matters than choosing either extreme.
As you say, we can detect a SRP violation by a large test suite, but we can also detect that by duplication in the names of test case groupings.