Restrict QuantizedCache to only full attention (#47160)