This article presents an observer-based adaptive sliding mode controller for a fully-actuated hexacopter unmanned aerial vehicle, performing trajectory tracking in a perturbed environment while carrying a cable-suspended payload. Based on the unavailability of a payload swing sensor, an extended high-gain observer is designed, providing full-state and disturbance estimation including payload motion. Such disturbances are compensated into the control loop to dampen the oscillations, thus improving the flight performance of the hexacopter driven by the adaptive control, providing robustness against bounded perturbations and chattering reduction. The stability of the observer and the control method on this system is guaranteed through Lyapunov theory. Simulations using a multibody emulator demonstrate time reduction in payload dampening while controlling the aircraft trajectory, compared to a feedback regulation-based adaptive controller.