If's that's a stock fender and not aftermarket and the same with the hard bags, you might want to go the route TOMA advised. Loosen the bolts to the bracket on the exhaust and slide it over.If there is no play you can try to looking at the saddlebag brackets. If neither work, then it could be an issue with the slip on muffler itself and how the bracket was welded to it, could be out of spec. Just some ideas.