Answering queries about visual inputs is a complex task that requires both visual processing and reasoning. In this talk, Sachit will demonstrate how large language models can be instrumental in reasoning within such settings, which extend beyond traditional language tasks. ViperGPT utilizes a provided API to access computer vision modules and composes them by generating Python code that is subsequently executed. This simple approach requires no additional training and achieves state-of-the-art results across various complex visual tasks. Sachit will also discuss how ViperGPT inspired the development of code-based agents and share insights on the future potential of such agents.