Home / AI Research / Training Vision Models for LLM-Driven Weight-Space Programming
TRAI Research
Training Vision Models for LLM-Driven Weight-Space Programming
· Timo Ropinski, Sebastian Kranz
It has been demonstrated that, in certain settings, large language models can perform weight-space programming to directly modify the weights of neural networks. However, it is unknown how dedicated training of these networks can contribute to the successes of weight-space programming. We present a benchmark of LLM-driven weight-space programming on vision models, in which five LLMs edit convolutional networks and vision transformers on relabeling and error-correction tasks on Waterbirds, CIFAR-100, PACS and Entity-30. We report more than 2,600 such edit processes, whereby each edit is scored on held-out images for success, side effects and accuracy loss. We compare standard training with…
Upvotes15
Comments2
Published18 Nov 2026