In this Q&A interview, CompatibL’s Executive Chairman and Head of Quant Research, Alexander Sokol, discusses the positive reception of the recent QuantMinds—CompatibL hackathon at QuantMinds International 2023. He also tells us about his conference presentation on CompatibL’s AI Co-Pilot and the rationale behind addressing model governance and document comprehension with AI. Alexander emphasizes the transformative impact of AI in quantitative finance on accelerating project completion and improving job satisfaction in the finance sector, and he offers some success stories highlighting real-world applications of AI.
What prompted you to organize the hackathon on quantitative finance? And what were your impressions of it?
We wanted people to see for themselves what this amazing technology can do for them in a practical setting. And this really succeeded, I think, beyond our wildest expectations, because so many people came to participate. So many people worked throughout the day. And so many people succeeded. I think a part of it is that we spent a lot of time preparing the hackathon and we made it accessible not just for people who code but for everybody.
There were two streams at the hackathon: one where you coded, and one where you developed with prompts using a natural language–based approach via the interface we had created and put online for that purpose. We are glad we did that—so many people were able to participate who normally don’t program but still could use this tool successfully.
Normally in a hackathon you want to make sure that the problem is challenging but can still be solved. Almost every participant was able to more than double the success rate of the initial basic prompt that we had put in. And by the end, it was working twice as well for everybody. Some people achieved a 90–95% success rate, which normally would be considered something that could take a week or so for a software development team to accomplish. But these people managed to do it during just the one-day hackathon.
What were some of the most innovative or promising projects or solutions that emerged from the hackathon? Could they be applied in quantitative finance?
Yes, absolutely. Both of the streams at the hackathon related to quant finance. These included tasks that our product does and what my presentation was about, but a simplified version. So, the two streams were model governance and term sheet comprehension.
The first, model governance stream, was about understanding and reading pricing functions. This is something that model governance experts do at a bank: they analyze model documentation, look at the models themselves, make sure those are in sync and properly documented.
The task was to understand the pricing function and have the AI detect what the instrument was, detect its parameters, and verify the pricing code was correct.
The second stream was about term sheet comprehension. There are many instruments in the financial industry, e.g., offering memoranda, structured notes, confirmations that involve a very complex, unstructured way to describe something very precise. So, for example, there is an instrument which is precise, such as a sequence of cash flows that go between two parties. But every bank has a different template of how this is formalized. Some banks describe it in one way; other banks describe it in a totally different way.
We had a selection of publicly available term sheets from EDGAR, which is the security commission database for public filings. And the participants instructed large language models to analyze and extract parameters from these documents. And everybody was so successful at that—it was amazing.
You mentioned your QuantMinds presentation, which was about CompatibL’s new model governance and document comprehension AI product. What reception did it receive from the quantitative finance community?
It was a very, very positive reception. In my presentation I shared the concept and technology behind the AI Co-Pilot, and how we built it, and what it took to build it. We would like the community to benefit from this product—for everybody to use these tools, for even our competitors to adopt these tools, because that is where the industry should be heading.
And what prompted the development of this AI Co-Pilot? What was the underlying rationale for addressing model governance and document comprehension with AI?
We wanted to understand where exactly we could apply these tools to benefit our clients and the industry. There is so much hype around AI, but sometimes either people just try to do something these tools are not ready to do, or it is just a bad idea.
For example, as financial professionals, we do not want to rely on Chat GPT to give us investment advice, right? This is not what our clients would expect us to do. We try to find a sweet spot at the intersection of specialized knowledge in our industry and natural language.
And we identified two areas where AI would be useful. One of them was model governance, where regulators require (and rightly so) that all the models at a bank are documented. On the one hand, there is a model’s source code, there are model’s commit messages, how engineers and clients describe what they have done every time they change the code. On the other hand, there are model release notes, which is more documentation, everything that regulators require. And it is tremendously difficult to keep the two things in sync. This is something that engineers and clients are often reluctant to do. You would rather be writing models than documenting them and composing long documents. Documenting models is also very labor intensive. It requires very high levels of expertise. Everybody in model governance must understand the models. Otherwise, they will not be able to do their job.
We felt there is so much effort going on and these kinds of projects take so long that this is where the industry could benefit from AI-based automation.
And the same applies for the other area: reading complex contract documents, anywhere from offering memoranda for structured securitization, to structured instruments, tranches, and so forth. For example, when one party pays another party a coupon based on equity appreciation with various kinds of instrument parameters.
So, these were the two areas where we felt language intersects with finance. We are normally used to precision in describing trading risk, something that must also be interpreted with precision, but we could describe this based on natural language. And this is exactly what these AI tools do to help.
We found that the adoption of these tools can accelerate project completion tremendously. Instead of, let’s say, six months, the same thing can be done in a month. We also make the job our clients do more enjoyable, because instead of reading hundreds and hundreds of pages of documents and marking them up, they are now working with a co-pilot; they are instructing and inspecting the work of the co-pilot and making some adjustments. And all the work that people really hate to do is now done by the machine.
Would you share some of your success stories or client testimonials that highlight the positive impact of your AI Co-Pilot on their goals?
Absolutely. One of the projects that I think really demonstrated the potential of this technology is when there was a transition to a new trading system at a bank. This new system had a different format for trades. There was a set of term sheets—natural language documents describing the trades—which were keyed in and captured in the old system.
Now they had to capture trades in the new system. This may seem simple, but actually it is not, because every system has a different way of describing identical features of the trade. Sometimes you enter a swap with two separate legs. Sometimes it is one leg, fixed or floating fields, etc. So, in so many cases you have doubt: the trade was keyed in in the old system in this way, but how can it be keyed in here in this new system? And also, what was the original document saying?
We built a solution very rapidly. It what one of our first projects we went into just shortly after Chat GPT had been announced. This project was about using AI to read the old trade capture, the original documents, and to produce a new trade capture for validation by humans. And it turned out to be much easier, in some ways, than we thought. We thought it would take months and months and months doing it. But it turned out these models were so good that they really required very little adjustment. So, this is one example.
My favorite example is model governance, as I am a quant. I do a lot of other work now, but at heart I am a quant; I started my career as a quant. I just love writing models. However, documenting models is not something quants enjoy doing. Engineers like writing code, but they do not enjoy writing documents describing what the code does.
So, I felt it was so empowering to basically be able to write model code and then have documentation updates in real time. You just tell the co-pilot what to do and adjust your instructions a little bit. But all the boring work of a quant—writing these model documents—is now automated by the co-pilot. This is really something that makes a quant’s job a lot more enjoyable.
Another thing is when the documentation has not been kept up to date, it takes a lot of time to update. With the co-pilot, now I feel like the moment that the model is finished, the documentation is also finished. And that is exactly what risk control and bank management want to see. AI allows you to play to your strengths.
Interested in trying out the CompatibL AI Co-Pilot?
Follow this link to learn more or drop us a line and schedule a free demo.
