Advances in private training for production on-device language models
… Run DP-FTRL training with limits on the magnitude of per-device updates chosen either via adaptive clipping , or fixed based on experience. …
Search
Croissant: a metadata format for ML-ready datasets
… However, these formats were designed for data discovery rather than for the specific needs of ML data, such as the ability to extract and combine data from structured and unstructured sources, to include metadata that would enable responsible use of the data, or to describe ML usage characteristics… …
Intervening on early readouts for mitigating spurious features and simplicity bi
… This suggests that confidence on errors from early readouts is a fairly strong, automated indicator of the model’s dependence on potentially spurious features. Illustrating the usage of early readouts i.e., output from the auxiliary layer in debiasing distillation. …