Neighbor’s Basketball Hoop On Our Property

We had a survey done on our house before we bought it (North Carolina) and discovered our neighbor’s permanently installed basketball hoop is on our property. We didn’t to be difficult new neighbors as it’s literally right over the line but is this going to cause issues down the line for us knowing it’s on our property and not doing or saying anything?