EPDM 45 mil is the standard and you need to purchase the underlayment along with it. EPDM is like a tire inner tube, a thick very flexible rubber.
Some have used HDRPE which is strong, but stiffer to deal with and usually doesn't need an underlayment, depending on your soil.
My pond is EPDM with underlayment and my 14'x5' bog is HDRPE with no underlayment. My soil here in Pennsylvania is clay based and I picked out the rocks the best I could.
The HDRPE is stiff, but if you fold the corners like your wrapping a gift, it's manageable. If you get the HDRPE, make sure it has the R for reinforced. HDPE doesn't have the reinforcement strands.
Stay WAY FAR away from a PVC liner. Believe me, the lower price is tempting, but in the long run you will be saving money by not having to replace you liner in a few months. I can tell you that from experience when I was a novice and thought I was saving money.
Your liner is the very basic component of your pond. Don't skimp on it. You will end up taking everything apart and stressing out your fish. Probably what you are going through now.
There are a few good online liner sellers. I have bought from "American Talapia" which had good prices and free shipping. I have read here that some people like "Just Liners". Research online and see for yourself.
Good luck. Any questions, just ask and let us know how you make out.