I am neither a graphic designer nor do I consider myself one. I have little grasp of colour theory, and basic knowledge of design principles. I do possess a sense of what looks good, though, and I am good at recreating designs.
I chose a beautiful website and set about creating my own design using it as a template. I asked the site owner for permission to use his backgrounds, and he agreed upon the condition that I show him the finished product.
Having finished I am not comfortable doing so. The designs do look similar. I am using the same font, the same backgrounds, and the same heading style. The layout is different, amongst other things, but there are many similarities.
Is it ethical to copy the work of other designers? Where is the tipping point from being inspired by (ethical) to copying (unethical) something?
I think you've copied too much, and so no, it's not ethical. Using one or two elements of someone else's idea is one thing, but when your site is essentially Same Song, Different Verse of the other guy's, then no.
To fix this, change things up. Use a different font of similar style — if it's Helvetica, use Franklin Gothic or Stone Sans; if it's Times, use Garamond or Caslon. If it's blue and yellow, use gray and bright red. And what the hell, ask him if he has any suggestions for how to improve yours.
This may also help you to understand why the other guy's site works.