New Insight of Variance reduce in Zero-Order Hard-Thresholding: Mitigating Gradient Error and Expansivity Contradictions
Researchers have identified a fundamental tension in zeroth-order optimization for sparse learning: the noise inherent in gradient-free methods conflicts with the hard-thresholding operator's behavior, limiting scalability. This work reframes variance reduction as a tool for resolving that contradiction, potentially unlocking zeroth-order methods for large-scale sparsity problems where true gradients are unavailable. The insight matters for federated learning, black-box optimization, and privacy-preserving training scenarios where gradient access is restricted.52
























