A construction worker accused of acting as the co-ordinator of an arson plot targeting the prime minister has told a court that he was drinking at a pub during the attacks.