Atticus Project: Developing Artificial Intelligence (AI) in the legal profession

by Nina Margariti, member of the Interviews Team

Wei Chen is the founder of the Atticus Project, a California based non-profit organization which aspires to accelerate AI development in the legal profession, starting with contract review. In her 20-year career as an M&A attorney, she has practiced at big firms like Skadden Arps, Cooley and Sun Microsystems, and she is currently an Associate General Counsel of the strategic transaction group at Salesforce. Today, she introduces us to the Atticus Project, the impacts of AI on the legal profession and ways to get involved in the Project.

When did you start learning about AI?

I started learning about AI more than 10 years ago because my husband is an AI researcher, focusing on machine learning but as you can imagine, knowing someone who does AI research is completely different from knowing AI yourself.

On a scale of 1 to 10, with 1 being the lowest and 10 the highest, how evolved do you think that AI currently is in the legal profession? What could we expect more and how soon?

AI development in the legal profession is probably 2, if not 1. I say that because AI is not widely adopted, if adopted at all, in today attorneys’ work, unlike in most other professions. The answer to what and how soon we could expect more, really depends on how fast the legal community can gather our resources and start doing projects, like Atticus, together. The problem is lying in our hands.

When did you come up with the idea of the Atticus Project?

I remember very vividly that date. It was in August 2019. After I tried out a couple of new AI tools, hoping that AI could solve the due diligence timeline in sense of a speedier, faster and more accurate review of legal contracts for our M&A transactions, I realized that my assumption that AI researchers or developers is my problem, was wrong. In other words, I realized that my problem for faster and more accurate contract review cannot be solved without the attorneys’ help and collaboration. That was when the Atticus Project idea popped into my head.

What exactly is the Atticus Project and how does it work?

In a nutshell, Atticus is a collaborative of legal professionals and law students to curate and label the first in history open-sourced label dataset of contracts for AI research. Now let me break it down. If you open your phone, you go to your photos, you type in the word cat, then magically all the pictures that have cats show up. How can a smartphone tell there is a cat in that picture? Technology is so mature therein that it gives people the impression that this is something that AI learned on its own. However, this is far away from the truth. The reason why your smartphone can recognize a cat from your cat pictures is because its system has been trained on this so-called label dataset. A label dataset is essentially a whole bunch of pictures which in that case were humanly tagged on the basis of whether a picture does or does not have cats. The same logic applies to legal contracts. In order for the AI to recognize clauses such as exclusivity or anti-assignment, someone must create an image database of these clauses similar to the dataset of cats. This is what we do in the Atticus Project. 

I have to say that the first time I tried an AI tool marketed to me as something that was potentially going to find the exclusivity clauses in my contracts, it was not very accurate. Then I asked why it is not accurate and they told me that I have to train it, to label it, essentially tag my own contracts, feed it into the system, train the system, and then the system would be able to recognize those types of clauses. I asked how many and they said maybe 10, maybe 15, maybe a million. That was a real shock to me. Looking back now, I was very naïve thinking that the AI researchers would magically solve the problem attorneys have. But once I recovered from that shock, I decided to take the problem into my own hands. I thought that I cannot code but I could create a training dataset of exclusivity and anti-assignment clauses. Through the Atticus Project we are going to leverage the power of the entire legal community to put together this training dataset, so that the AI can be better at reading legal text.

What are the current and future purposes of the Atticus Project? 

We are aiming to accelerate the AI development in the legal profession because currently there is an inability to review the massive number of legal contracts, legal documents and case law. Legal professionals have to deal with legal research but there are so many cases out there. Under this light, it would be really helpful if an AI system could identify, go into this large number of legal documents and find the needle in the haystack for you and present it in front of you, so that you can spend your valuable time digesting the data that has been searched for you and then trying to do the legal analysis and advise your clients accordingly on a timelier basis. Given the technology development, every other department, every other function, company and industry is heading toward data-driven executive decision-making in the future. When an attorney is currently presented with a question, she intuitively says yes or no. In the future, the follow up question of yes or no is going to be what the data is behind that answer. How do you reach that conclusion? Is that backed up by data? Show me the data. So, transparency and speed in future decision-making relies very much on what we do today, which is to help AI development in coming up with the data collection process faster.

Why did you name your project Atticus?

A participant in the project named it and it is coming from Atticus of Harper Lee’s novel “To Kill a Mockingbird.” Atticus is a symbol of integrity and justice. Besides, that was very suitable because we wanted to accelerate AI development like we wanted to make a bird fly.

Are there any other similar projects at this moment?

There is a lot of AI. Almost all the AI legal providers are doing a similar project to ours. They are labeling their data accordingly, in order to train their AI systems. However, all of them are doing it behind their closed doors. Thus, they are duplicating efforts, everyone is training their own datasets on the same provisions (contracting parties, renewal date, anti-assignment clauses etc.). More importantly, the broader AI community has not been mobilized to solve the legal problem and the reason is that the broader AI community is not used to working with open-source data, i.e. data made available to the public for free. This is what Atticus is creating and there is not anything like that right now. Atticus is essentially gathering legal experts from a variety of industries, different areas and organizations around the world trying to solve this problem together, and not within the four corners of a company. In Atticus Project everything that we do is going to be open sourced.

How easy/difficult was to make the idea of the Atticus Project come true in the United States? Have you ever thought that this project may have been realized more easily anywhere else in the world? And if yes, where would that be?

The Atticus Project idea is not something I am going to patent or copyright. It is such a simple idea. Everyone in the AI legal industry knows this is something that needs to be done. The real challenge is who is willing to do this. Trying to find an open dataset, where we can work on is very different from your day-to-day work because given the legal nature of our practice, most of our data is proprietary and it cannot be released to the public. So, currently seeding the growth of AI development needs to be a parallel project in addition to your day-to-day work and that is just an investment. It is almost like basic research that someone is willing to take on, to motivate people to do it together, to gather a whole community around this vision and make them feel passionate about this vision. If we can find someone like me, in any other country, this idea is going to flourish in that place.

What are some of the foreseeable effects of the project? How do lawyers react to the Atticus Project? Are they positive or hesitant?

Lawyers are risk averse by training, since our main job is risk management. Any change is faced with questions and skepticism. However, having worked with hundreds of volunteers for the Atticus Project during the past one and a half years, I have seen that there is a real desire and passion around trying to accelerate the development of AI to solve our day-to-day problem. No one is interested in doing the menial, time-consuming, low value work that does not fully utilize her brainpower. However, that is how the current legal system is set up, law firms filling their time and getting paid. When I first started practicing, every law firm had a word processing department, which typed up overnight the notes you had written down. That job clearly has disappeared, because of word processing by Word, Google Doc and a whole bunch of tighter productivity tools. It is going to be a similar movement. Basic legal document review will go through the same change, not to say that the lawyers are not going to look at legal contracts. They are just not going to spend hundreds of hours looking for the needles in the haystack. As a result, they will be able to focus on more high-quality work.

Who can participate in the labeling process of the Atticus Project? What is the profile of lawyers and law firms that are currently collaborating with the Atticus Project? 

Atticus Project currently engages law students from Berkeley, Santa Clara Law, UC Hastings and Southern University Law Center. We are also launching a student fellow program that is open to anyone around the world interested in participating. Our summer program intern application will be up in March. Participating students will also have the opportunity to train and get certified in a variety of areas demonstrating their proficiency in legal document review.

There is already a variety of attorneys with experience of 6-20 years, who supervise the law students working on the project. These attorneys come from leading companies and law firms across the country.

How much trust can we put in the AI contract review? How accurate can it really be?

As lawyers typically answer, it depends. The F1 score, which is the standard for measuring accuracy in the AI industry, currently is various depending on the type of clauses. So, for example, for governing law, dates and parties, the accuracy is 80% or even higher, but for others, like MFN, exclusivity and unlimited/uncapped liability it is very low, definitely lower than human performance. This is why we are doing what we are doing. We are releasing the next version of Atticus dataset in a few weeks and we will establish a benchmark for these different types of clauses that we label. The score will only get better once we engage the entire AI community.

Do you think that AI could be used in legal areas other than contract review?

Absolutely. Legal research is an example. The project can go as far as the datasets go. We are launching an executive fellow program, recruiting experienced attorneys like me who feel passionate about the Atticus Project to lead a data project on something they think is going to be valuable to solve attorneys’ day-to-day problems. That application will soon be available online.

What are your thoughts on the development of AI in Europe and Asia? When could we expect that the Atticus Project reviews contracts in languages other than English?

Artificial intelligence in general, in Europe and Asia are very promising. Very bullish in the short sense. As for the AI development in the legal profession more specifically, it is in your hands. The faster you can put together a label dataset in your language, the faster AI legal development will be. 

How do you think AI will affect our everyday lives in the near future and how would you advise us to prepare ourselves towards this change?

I am so jealous of the new generation of legal professionals because they live in the era of change. This is the transformative time. In the next 2-5 years, I expect a fundamental transition, similar to the one we lived when we converted working on physical documents to email. 

You need to develop skill sets like analytical skills and your ability to not only go deep, but also wide. You need to have working knowledge of AI, data science, entrepreneurship and more importantly, change management, meaning that you need to understand how to influence and motivate people to change. The new legal professionals need to be aware of all the above and try to develop proficiency thereinto. The future workforce is going to be AI-enabled. You will have to put different hats on your head and constantly question yourself if it is a job for AI or for a human, if it is a legal problem, or a medical problem. Then you should try to make connections and solve all of them, take responsibility for all those problems, instead of having this very siloed thinking of “that is a medical problem, the doctors will take care of it.” We really need to take all these problems into our hands and start learning the skills of collaboration and speaking each other’s language.


Απάντηση