Ghosts are presumably the souls, mind or other similar concepts of deceased ponies or other races.
It seems that ghost are formed when the individual dies with unfinished business at hand. Ghosts seem to retain much of their former lives, such as their memories, personality, voice, etc, but their mindset appears to be more one-note as they're driven to finish what they started, even hundreds of years after their demise. This can be seen with Squire, who died during his quest to tame the dragon Bazzle and forever seeks out to tame said dragon.
Powers and abilities
Ghosts are intangible beings who can fly and hover.