[Pl-seminar] Federico Cassano - Making Code LLMs Work for Low-Resource Programming Languages

[Luna Phipps-Costin]

Hi PRL Seminar People!

On this spoooky Friday the 13th we'll be having PRL Seminar! (as usual).   Here are the details:

Speaker: Federico Cassano
Title: Making Code LLMs Work for Low-Resource Programming Languages with MultiPL-T
When: 12 to 1:30, Friday the 13th 👻
Where: Forsyth 237
[Abstract:

Large Language Models of Code (Code LLMs) seem to be very useful for high-resource programming languages such as Python and Java. However, they are far less effective for low-resource languages such as OCaml and Racket. In this talk, we will examine how Code LLMs work, how we evaluate their performance, and how we can make them better at low-resource programming languages.


First, we give a high-level introduction to Code LLMs. Subsequently, we ask how Code LLMs are evaluated. We focus on MultiPL-E, which is our framework that enables evaluation across multiple programming languages. Since we introduced MultiPL-E, it has become the de facto standard for evaluation of Code LLMs on multiple programming languages. Using MultiPL-E, we will quantify what programmers already know intuitively – Code LLMs work for Python, Java, etc. but are much less effective at Racket, OCaml, and other low-resource languages.


We address this problem with our latest work: MultiPL-T. With our approach, we have achieved substantial advancements in the capabilities of Code LLMs for low-resource languages. Our results demonstrate improvement in the performance of Code LLMs for languages like Racket, OCaml, and Lua, drawing them closer to the proficiency levels seen in high-resource languages.]


We provide lunch from 12 to 12:30.   Then Federico's talk is 12:30 to 1:30.

I'd also like to thank Cameron Moy for volunteering to help with seminar things!
Please majick irresponsibly! 🧙‍♀️
-------------- next part --------------
HTML attachment scrubbed and removed