Generally speaking modals in the past have the following form:
Except for modals that express obligation,ability and lack of necessity:
|Modals in the Present||Modals in the Past|
|Obligation||You must / have to stop when the traffic lights are red.||You had to stop.|
|Advice||You should see a doctor.||You should have seen a doctor|
|Prohibition||You mustn't smoke here.||You mustn't have smoked there.|
|Ability||I can run fast.||I could run fast. now I am old.|
|Certainty||He has a Rolls Royce. He must be very rich.
He can't be American. His English is terrible.
|He must have been rich. He had a big house and an expensive car.
He can't have written that poem. He was illiterate.
|Permission||Can I go out?||She could drive her father's car when she was only 15.|
|Possibility||It may / can / could / might rain. It's cloudy.||I guess it may / can / could / might have been Lacy on the phone.|
|Lack of necessity||You don't have to / needn't buy any tomatoes. There are plenty in the fridge.||You didn't have to / didn't need to buy tomatoes.|