Federated learning (FL) has enabled global model training on decentralized
data in a privacy-preserving way by aggregating model updates. However, for
many natural language processing (NLP) tasks that utilize pre-trained language
models (PLMs) with large numbers of parameters, there are considerable
communication costs associated with FL. Recently, prompt tuning, which tunes
some soft prompts without modifying PLMs, has achieved excellent performance as
a new learning paradigm. Therefore we want to combine the two methods and
explore the effect of prompt tuning under FL. In this paper, we propose
“FedPrompt” to study prompt tuning in a model split aggregation way using FL,
and prove that split aggregation greatly reduces the communication cost, only
0.01% of the PLMs’ parameters, with little decrease on accuracy both on IID and
Non-IID data distribution. This improves the efficiency of FL method while also
protecting the data privacy in prompt tuning. In addition, like PLMs, prompts
are uploaded and downloaded between public platforms and personal users, so we
try to figure out whether there is still a backdoor threat using only soft
prompts in FL scenarios. We further conduct backdoor attacks by data poisoning
on FedPrompt. Our experiments show that normal backdoor attack can not achieve
a high attack success rate, proving the robustness of FedPrompt. We hope this
work can promote the application of prompt in FL and raise the awareness of the
possible security threats.

360 Mobile Vision - 360mobilevision.com North & South Carolina Security products and Systems Installations for Commercial and Residential - $55 Hourly Rate. ACCESS CONTROL, INTRUSION ALARM, ACCESS CONTROLLED GATES, INTERCOMS AND CCTV INSTALL OR REPAIR 360 Mobile Vision - 360mobilevision.com is committed to excellence in every aspect of our business. We uphold a standard of integrity bound by fairness, honesty and personal responsibility. Our distinction is the quality of service we bring to our customers. Accurate knowledge of our trade combined with ability is what makes us true professionals. Above all, we are watchful of our customers interests, and make their concerns the basis of our business.

By admin