At your convenience, you can remove the bank’s app from your phone and shred any remaining paper checks from the old account. Some institutions offer sign-up bonuses when you open a new account.