I have this image

I can use the -trim option to produce this result

I would like to achieve a more aggressive trim to produce a result like this

Remove all of the border color while leaving the maximum possible remaining pixels. Is there a way to do that? I realize that there would be many cases that would produce a 0x0 image, but that won't be an issue in my case.
Debian Linux
Version: ImageMagick 6.8.9-9 Q16 i586 2015-01-05
Thanks