RowParallelLinearWithoutBias¶
- class RowParallelLinearWithoutBias(input_size, output_size, *, bias=True, input_is_parallel=False, init_method=<function xavier_normal_>, stride=1, keep_master_weight_for_test=False, skip_bias_add=False, params_dtype=torch.float32, use_cpu_initialization=False, perform_initialization=True, gradient_accumulation_fusion=False, sequence_parallel_enabled=False)[源代码]¶
重写
megatron提供的行并行全连接层以去掉结果中的bias。在tp_size为 1 时返回普通的全连接层(支持peft中的lora方法替换全连接层)-
training:
bool¶
-
training: