Key Metric
15 errors detected
The Context
A 35-attorney firm in Bogotá, Colombia, with a strong commercial litigation and arbitration practice, handling complex disputes before the ordinary civil courts and the arbitration tribunals of the Cámara de Comercio de Bogotá.
Practice Area: Complex commercial litigation and domestic and international arbitration
Jurisdiction: Colombia (Bogotá Superior Courts, Corte Suprema de Justicia, Arbitration Tribunals)
Team Size: 35 attorneys (10 in the litigation group), 7 legal assistants
The Challenge
Problem: Following several cases of attorney sanctions in the U.S. and Europe for AI-fabricated citations, the managing partner ordered a review of source-verification practices. An internal audit revealed that 10% of citations in recently filed briefs contained errors — incorrect references to judgments, wrong case file numbers, or inaccurate transcriptions of rationes decidendi.
Previous Approach: Junior attorneys manually verified citations using databases from the Corte Constitucional, the Rama Judicial, and LexBase, typically under time pressure before procedural deadlines.
Stakes: Beyond the reputational risk of filing briefs with deficient citations, the firm faced potential disciplinary sanctions from the Consejo Superior de la Judicatura, client liability, and professional liability insurance implications.
The Approach
Tools Used: vLex Vincent AI for Colombian and Latin American jurisprudence verification, integrated with the Corte Constitucional's case law database. Claude for cross-checking verbatim transcriptions.
Implementation Strategy: Implemented a mandatory 'AI citation audit' as the final step before any brief is filed. Every brief passes through three layers: (1) drafting by the attorney using standard research tools, (2) AI citation verification checking judgment currency, file-number accuracy, and textual fidelity, (3) senior attorney review of the AI's findings.
Investment: $35,000 USD per year in additional AI tool licensing. The mandatory audit adds approximately 2–3 hours to each brief's timeline, offset by the reduction in time spent on manual verification.
The Results
Quantified Outcomes
- In the first six months, the AI audit flagged issues in 28 of 67 reviewed briefs (42%)
- Of those, 15 contained citation errors serious enough to trigger judicial inquiries
- 4 briefs cited judgments that had been unified or whose doctrine had changed — the most dangerous type of error
- Citation accuracy improved from 90% to 99.2% after implementation
Qualitative Outcomes
- Attorneys reported feeling more confident about the accuracy of their court filings
- A magistrate informally commented on the improvement in source quality during an oral argument hearing
- The process surfaced broader research quality issues that led to improved training for junior attorneys
The Lessons
What Worked
- Making the AI audit mandatory rather than optional ensured consistent adoption across the firm
- Presenting the tool as a safety net — not a replacement for research skills — reduced attorney resistance
- Sharing anonymized examples of detected errors in firm meetings demonstrated concrete value
What Didn't
- The AI occasionally flagged false positives, particularly with tutela rulings and interlocutory orders not published in major databases
- Initial resistance from senior partners who perceived it as a challenge to their competence required careful change management
Advice
After the sanctions cases for AI-fabricated citations, every litigation firm should have an AI verification step. The cost is minimal compared to the risk. But don't just install the tool — build a workflow around it and make it non-negotiable.
Our Takes
After Mata v. Avianca, citation verification is not optional — it is a professional obligation. This firm's three-layer approach (draft, AI audit, senior review) is a model for how AI should be integrated into quality assurance workflows. The key is making the audit mandatory, not optional.Lawra (The Moderate)
Let's be precise: the AI flagged 42% of briefs as problematic, but how many of those alerts were false positives? The firm mentions this as an issue with state court citations and unpublished opinions. In a high-pressure litigation environment, false positives create alert fatigue — attorneys start ignoring warnings, undermining the entire system.Lawrena (The Skeptic)
Fifteen citation errors caught before filing — any one of which could have led to sanctions. And four briefs citing reversed or superseded cases! In a post-Mata world, this isn't a luxury, it's malpractice prevention. The $35,000/year investment is negligible compared to a single sanctions motion.Lawrelai (The Enthusiast)
The most revealing detail here isn't the technology — it's the sociology. Senior partners initially resisted because they saw the AI review as a challenge to their competence. That is the innovator's dilemma in miniature: the people most invested in the current system are often the last to adopt what improves it.Carlos Miranda Levy (The Curator)
Sources & References
Have a Success Story to Share?
We're always looking for well-documented examples of AI adoption in legal practice. If your organization has a story worth telling, we'd love to hear from you.
Ready for structured learning? Explore the Learning Program →
Comments
Loading comments...