Most of the people have the habit of using “Safe Remove Drive” thinking that it is useful for long life of the USB drive. Which is (partially) incorrect. It helps you in case when the USB drive in use i.e. if there is a file transfer going on, which means if you are moving files between USB drive and computer, it warns you that the drive is in use.

But if there is no file transfer happening, you can pluck out the USB drive straight away.