Yes I agree, your charge is fair (poss a little to low) but, as has already been said, you also will get bitten once in a while.....if the customer wont pay for the work because they think its steep your options become limited to accepting what they will pay, off to small claims court or your power of persuasion.....Sadly, these situation arrise once in a while......
i try to cover my back with a written quote each time, which also includes a variable for extra time on the job (if required) which means extra monies should the un-foreseen arrise (as in your case). The customer has to accept it, in writing (usually an email response), before I start. If time is too limited ie you are there and they need it doing right then, then I carefully explain, in complete lay-mans terms what the price is/hourly rate is, what it covers and make sure they say "OK". getting them to accept your price, even verbally, before-hand does seem to make a difference.
Hope this helps,
FFM