If you only need to move them down a very small amount, then you may be able to get away with not having to remove the bar tape. Otherwise, yes you will have to take off the bar tape. Depending on the type of tape you have fitted, you should hopefully be able to just re-wrap it back on.