The reversal comes just hours after they said they would take part in a pilot scheme involving daily testing instead of self-isolation.
Opposition parties had said it suggested there was one rule for them and another for the rest of the nation.
Downing Street said Johnson will conduct meetings remotely at Chequers.