Add remat as an optional dependency (#4163)