deepspeed.init_inference() 返回一个 推理引擎,其类型为 InferenceEngine。
deepspeed.init_inference()
InferenceEngine
for step, batch in enumerate(data_loader): #forward() method loss = engine(batch)