AN EXTENDED SCAFFOLD TECHNIQUE BASED ON COSINE SIMILARITY FOR CLIENT DRIFT MITIGATION IN CROSS-SILO FEDERATED LEARNING

Abstract

Federated learning enables multiple clients to collaboratively train a shared model without exchanging raw data, yet its effectiveness under non-independent and nonidentically distributed (non-IID) data is undermined by client drift, where repeated local updates pull each client away from the global objective. Existing mitigations, notably Scaffold, correct for drift using control variates but apply a fixed, uniform local learning rate to every client regardless of whether its update direction agrees with or opposes the prevailing global trajectory -a directional signal that a PRISMA 2020-compliant review of 33 studies (screened from 1,847 records) confirmed remains markedly underexplored in the literature. This gap motivated the development of CosScaffold (Cosine Similarity-extended Scaffold), a novel algorithm that, unlike prior approaches, modulates each client's effective learning rate by the cosine similarity between its gradient-direction probe and the previous round's aggregated server update, amplifying well-aligned updates and attenuating misaligned ones through a clipped affine modulator that gracefully recovers vanilla Scaffold when directional information is absent. Implemented in PyTorch and benchmarked against six baselines on EMNIST Balanced and CIFAR-10 under a pre-registered protocol, CosScaffold reached target accuracy in a median of 16.0 communication rounds at the most heterogeneous setting -17.9% fewer rounds than Scaffold (19.5) and 5.9% fewer than FedDyn (17.0) -while achieving the highest final test accuracy of 0.498 against Scaffold's 0.480, a 3.75% relative improvement. A construct-validity analysis of 1,960 paired observations further confirmed cosine and Euclidean alignment to be statistically distinct (R² < 0.01), substantiating the novelty of the directional signal exploited. CosScaffold thus contributes a lightweight, communication-free, theoretically grounded mechanism for exploiting gradient alignment in federated optimisation, offering both a new algorithmic paradigm for non-IID learning and a practical, drop-in enhancement that accelerates convergence precisely where data heterogeneity is most severe.