Large language models (LLMs) are advanced AI systems that transform input data into outputs based on their content: inputs are sorted and processed through multiple layers of neural networks.
The goals of the early LMM developers were modest (for example, to complete a single sentence when writing an email), but as these models became larger and more sophisticated, they acquired capabilities that even a year ago seemed the stuff of science fiction: maintaining a long conversation or writing a poem on a given topic.
Today, most scientists agree that the best of these models have sparks of artificial general intelligence (AGI)—namely, some degree of understanding and the ability to reason about the real world. This is what distinguishes them from the legacy natural language processing models, such as sentiment analysis based on tone rather than meaning.
Following our latest AI initiatives, in November our inaugural Hackathon—a partnership between CompatibL and QuantMinds—will take place at the QuantMinds International event at the InterContinental O2 in London, where you can test your coding and prompt engineering skills using LLMs and win valuable prizes, including complimentary access to the main conference.
CompatibL’s Hackathon at QuantMinds International 2023
In addition to coding, we are excited that our Hackathon also embraces prompt engineering as a stepping stone to creating more intuitive AI systems. This event is your chance to engage with real-world quantitative finance challenges, regardless of whether you are an industry expert, academic, or student.
What are you hoping to see from the winners?
The Hackathon offers opportunities to enter as an individual or as a team. We hope to unleash your creativity and brilliance, and for all Hackathon participants to create something wonderful and unexpected, perhaps leading to a presentation at a future QuantMinds conference for the competition winners. We hope that all participants come away from the Hackathon with a sense of having accomplished something amazing, perhaps creating code they could use to do their job better and faster.
We believe it is very important for the Hackathon to include any conference participants who would like to participate, not just those who can code in Python. There are four different Hackathon categories to choose from, some of which do not require any programming at all and will be based on prompt engineering alone. Prompt engineering is the art of programming a large language model in a natural language, providing plain English instructions that must be stated in a precise manner to achieve the desired results.
We will also provide a small tutorial to begin, so even those who have no prior experience with LLMs can participate. Of course, other categories will require programming and will be a way for quants to showcase their coding skills. Our goal is for everyone who is interested in participating to find a category that matches their interests and their preferred style of participating.
Follow this link to register for the Hackathon.
What are you most excited to learn about/discuss at QuantMinds International?
The conference offers a chance to discuss the latest in AI and LLMs!
At first glance, LLMs may not seem to be readily applicable to quant finance, and in fact there are many examples of the GPT-4 model getting math hilariously wrong. And yet, when used in combination with carefully designed code and proper prompt engineering, it can be used to approach many problems in quant finance from a different direction.
Some of these problems, and their potential solutions, will be discussed in Alexander Sokol’s presentation on Customizing Large Language Models for Quant Finance Applications on November 14, 10:30–11:00 (QuantMinds Plenary Stage), and his workshop on Large Language Models in Financial Markets on November 14, 11:35–13:30. You can register by following the links above.
We know quants and risk managers are working apply LLMs to quant finance in many other new and unexpected ways, and we cannot wait to hear about those at QuantMinds this year.
The Applications of LLMs in Quantitative Finance
While quant finance is by nature mathematical, at CompatibL we have identified many areas that require language comprehension—some related to analytics, and some related to the process around developing those analytics. One very exciting example is model governance.
A bank’s risk and evaluation models are used in financial reporting and risk reporting for regulators, so it is extremely important that the models’ documentation is correct. But often things fall through the cracks, and the documentation is not always up to date.
When a bank’s models are modified by its quants, all changes must be documented for the regulators. Usually this results in a 200–500-page document for each model, and a large bank often has hundreds of models they must document. Doing this well is critical to the reliable functioning of the financial industry.
The CompatibL team has developed a product using OpenAI GPT models hosted in Azure to analyze code changes along with version control commit logs, model’s release notes, and model documents. We found that GPT can generate or update model documentation and/or release notes for a final review and approval by humans, or alternatively it can verify that the humans who prepared the model release notes did not miss anything and flag any omissions. The accuracy of this combined process where AI acts as a co-pilot to model governance analysts has been tremendous and has surpassed that of human checking alone.
AI can perform the volume of work that would be unfeasible for a purely human team—looking at every source code change, at every version control commit, etc. So, it is not replacing humans, but it helps humans produce higher quality work (which was exactly our goal).
The Opportunities for LLMs in Quantitative Finance
We carried out a case study with one of CompatibL’s early clients who was an early adopter of our LLMs, and somewhere among thousands and thousands of commit messages, they were able to find descriptions such as “removed a currency from the calibration” or “changed the numerical method”, changes that regulators absolutely had to know that but that had previously gone went unnoticed. Such comments are very easy to miss within a very, very large body of these code changes and commits.
We were absolutely amazed how well our LLM worked. The OpenAI GPT model was able to understand the nuances of quant speak on a totally unexpected level, even in situations where a human would take a moment to understand exactly what a certain description referred to. The model was able to understand such nuanced descriptions and add them to the release notes and model documentation correctly.
To be clear, we are not proposing to use large language models to completely automate the model governance process. Humans must be part of this process and sign off on the final version—regulators will not accept anything less. But we have found their use led to enormous quality enhancement with no change to the size of model governance team—and it also made the team’s job much easier and much more fun.
The Challenges of Using LLMs in Quantitative Finance
There has recently been a lot of concern that financial institutions’ data may be used by Open AI and other LLM vendors to fine-tune their models, causing this data to leak into chat sessions with the general user populations. Some of the early messaging and imprecise disclaimers around the release of ChatGPT added fuel to this fire. Recently, OpenAI addressed this issue and provided precise guarantees about data confidentiality, which is extremely important for the enterprise applications of this exciting new technology.
More importantly, the partnership between OpenAI and Microsoft Azure is making GPT models available within a bank’s Azure cloud. Microsoft will guarantee that OpenAI models will run in a bank’s Azure account and be completely isolated from any other Azure cloud or from OpenAI. The model data thus becomes subject to the same protections as any other data in a bank’s Azure cloud. And it is subject to the same privacy guarantees, so it is never used for training and never accessed by anyone outside the bank. As CompatibL is a trusted partner of many financial institutions with an existing Azure cloud contract, we expect Azure deployments to be the preferred choice of most banks and asset managers.
The use of AI in the EU will be regulated by the Artificial Intelligence Act (AI Act), the world’s first comprehensive AI law. We at CompatibL see this as a positive for AI adoption, because clear legislative guidelines will lead to a greater compliance effort from AI technology vendors, which in turn will give banks confidence to use this exciting new technology. Because the client will know that the data is secure, they will be able to use this natural language capability without any concern that, for example, any query they ask GPT about a particular counterparty will somehow become publicly available data. The AI Act is a step toward building confidence that will precipitate AI adoption in the financial industry.
