Many things, including your fish's excrement, increase phosphate levels... I doubt your rocks alone would be the primary contributor to your high phosphate.
@ashirley ... Your high phosphate is due to one of 3 actions : 1) tremendous amount of organic decomposition in your pond; 2) your well water is high in phosphate; 3) not enough plant growth to consume those phosphates.
I think it is more of #2 and #3 because many of us water gardeners here have a tremendous amount of organic decomposition in our ponds, but rarely experience quite high phosphate levels.