Muwatta Malik #445

Yahya related to me that Malik said, "The sunna concerning the time of prayer on the ids of Fitr and Adha - and there is no disagreement amongst us about it - is that the imam leaves his house and as soon as he has reached the place of prayer the prayer falls due." Yahya said that Malik was asked whether a man who prayed with the imam could leave before the khutba, and he said, "He should not leave until the imam leaves."