Release all high-precision init value in CPU host memory (#6507)