If this is something that you’ve just recently paid, that it’s more likely that it will remain on your report. I would actually advise against it – sending false disputes to the credit bureaus will get you tagged and you might have problems getting legitimate errors fixed.
First, the credit bureaus can see for themselves that you have recently paid-off the account because it will show the recent status change.
Second, when the credit bureaus verify the account with the collection agency, most likely that the collection agency will verify an account that has been recently paid-off. You can have more luck with 3-4 years old debt though.
It goes without saying that it’s only worth a try for single item debts like unpaid utility bills or medical bills. It’s would defiantly won’t work with defaulted credit card or installment loan because the original default item (place on your credit report by the ORIGINAL creditor) will remain on your report, and you won’t be able to dispute it.