Luckily I've not run into this problem myself, but would be interested to hear how it works out.
If it was me I'd do the same as you and speak to the Estate Agents first. In my experience they are without exception a useless shower of C**ts, but you might as well try to make them earn their service charge.
Unfortunately, when it comes down to eviction and if the tenants refuse to vacate, it nearly always comes down to getting a Court Order. Invariably estate agents will manage to side step that responsibility leaving it on you.
However...
If you have a full managed service contract with them it's their responsibility to attempt to sort out the issues in the first instance.
If you only have an introduction service they may try to say it's nothing to do with them. But, it would have been their responsibility to perform reference checks. They nearly all outsource that job to reference checking companies which guarantee the tenants for the first 6-12 months. Those 3rd party reference checking companies often also provide inclusive legal advise in the event the references turn out to be dodgy.
So in either case the Estate Agents should atleast be able to advise you on what steps need to be taken to proceed with an eviction. Including at what stage the Police should be involved. Police evidence may be needed to get an eviction court order on grounds of illegal or immoral activity.
Definitely get the Estate Agents to agree everything with you in writing before proceeding with any action.