FOLIO Data Generator

A versatile Python library for generating synthetic legal data using the FOLIO (formerly SOLI) knowledge graph, providing both template-based and AI-powered generation methods for creating realistic legal documents and datasets.

Overview

The FOLIO Data Generator addresses the critical need for synthetic legal data in AI development, testing, and research. By leveraging the standardized FOLIO ontology, it generates contextually accurate legal text while avoiding privacy and confidentiality concerns.

Generation Methods

1. Procedural Template Generation

Uses templates with FOLIO and Faker tags for structured generation:

from soli import SOLI
from soli_data_generator.procedural.template import TemplateFormatter

soli_graph = SOLI()
formatter = TemplateFormatter()

template = """
LEGAL NOTICE

Company: <|company|>
Industry: <|industry|>
Legal Issue: <|area_of_law|>
Jurisdiction: <|jurisdiction|>
Date: <|date|>

The party of the first part, <|company|>, hereby notifies
all concerned parties regarding matters pertaining to <|area_of_law|>.
"""

formatted_text = formatter(template)

2. LLM-Based Generation

Leverages AI models for more dynamic content creation:

from alea_llm_client import VLLMModel
from soli_data_generator.llm.text import TextGenerator

model = VLLMModel()
generator = TextGenerator(model)

# Generate with constraints
generated_text = generator(
    document_type="contract",
    jurisdiction="Delaware",
    parties=2,
    complexity="medium"
)

Key Features

Template System

• FOLIO Tags: Access to 18,000+ legal concepts
• Faker Integration: Realistic names, dates, addresses
• Custom Tags: Extensible tag system
• Nested Templates: Complex document structures

Document Types

• Contracts and agreements
• Legal notices and disclaimers
• Court filings and motions
• Regulatory documents
• Corporate documents

Data Quality

• Ontology-driven accuracy
• Consistent legal terminology
• Jurisdiction-aware content
• Realistic formatting

Installation

pip install soli-data-generator

Use Cases

AI Training Data

Generate diverse, labeled datasets for:

• Contract analysis models
• Legal NER systems
• Document classification
• Compliance checking

Testing and Development

Create test data for:

• Legal tech applications
• Document management systems
• E-discovery platforms
• Contract lifecycle tools

Research

Support academic research with:

• Controlled experimental data
• Privacy-preserving datasets
• Benchmark generation
• Ablation studies

Architecture

Core Components

• Template Engine: Parses and processes templates
• Tag Resolver: Maps tags to FOLIO concepts
• Generation Pipeline: Orchestrates generation flow
• Output Formatters: Multiple format support

Integration Points

• FOLIO/SOLI knowledge graph
• Faker for realistic data
• LLM APIs for AI generation
• Custom tag providers

Advanced Features

Batch Generation

# Generate multiple documents
documents = generator.batch_generate(
    template="contract_template.txt",
    count=1000,
    variations=True
)

Constraint-Based Generation

# Generate with specific constraints
constraints = {
    "jurisdiction": ["New York", "California"],
    "document_length": (1000, 5000),
    "complexity": "high",
    "include_concepts": ["indemnification", "warranty"]
}

document = generator.generate_constrained(constraints)

Format Support

• Plain text
• Markdown
• JSON structured data
• XML legal formats
• Custom formats via plugins

Quality Assurance

Built-in validation ensures:

• Legal terminology accuracy
• Structural consistency
• Format compliance
• Concept coherence

Performance

Optimized for scale:

• Efficient template parsing
• Caching for repeated concepts
• Parallel batch processing
• Memory-efficient streaming

Future Development

Planned enhancements:

• Additional document templates
• Multi-language support
• Domain-specific generators
• Interactive generation UI

The FOLIO Data Generator empowers developers and researchers to create high-quality synthetic legal data, accelerating innovation while maintaining privacy and legal compliance.