Only apply grpc defaults when grpc plugin is configured