Uh, on one ledge right to the cave, there IS a whirlwind you can use. As far as I can remember, you have to land on the edge of the cave though, not trying to get over the hole...
Might be wrong though. I'm checking it now. D:
Edit: Made a shot with my dad's phone and drew the lines in in Photoshop.
At the flight (the transparent red line), close to the edge, Link will go up a little for some reason and you'll be on the ledge. It's somewhat tricky, but possible. Just believe that it works and you'll be fine.