LexIssue: Benchmarking Legal Issue Identification in Chinese Civil Litigation

The work introduces a legally grounded hierarchy that combines free-form descriptions of disputed matters with structured legal categories.

Source: [3], [10]

It frames the task as complementary generation of issue descriptions and classification of their legally grounded attributes.

Source: [4], [10]

The study examines computational identification of disputed matters in litigation together with their legal attributes.

Source: [14]

The evaluation resource comprises real-world civil litigation cases with expert annotation of disputed matters.

Source: [13]

Each annotated matter includes both a descriptive summary and structured category labels, and the cases cover diverse civil dispute types.

Source: [13]

The benchmark uses synthetic claim and defence materials reconstructed from judicial accounts of party pleadings.

Source: [8], [21]

This reconstruction approach was used because original pleadings are difficult to obtain at scale from public sources.

Source: [21]

Synthetic inputs cannot fully reproduce the form, style, strategic framing, or evidential detail of materials used in actual litigation.

Source: [21]

Assessment is challenging because equivalent disputed matters can differ substantially in wording and abstraction.

Source: [1]

Reliable prediction-reference alignment is therefore needed before computing evaluation metrics.

Source: [1]

An issue-centred legal knowledge resource was curated from doctrinal and practical materials alongside litigated cases.

Source: [15], [22]

Expert integration consolidated overlapping entries and normalized their granularity and style, producing reusable abstractions rather than case-specific annotations.

Source: [15]

Models are tested in direct instruction, legally guided prompting, and retrieval-augmented settings.

Source: [17], [23]

For retrieval augmentation, candidate issue entries are reranked for relevance and the selected entries are placed into the generation prompt.

Source: [6]

The comparison includes proprietary general-purpose systems, openly available general-purpose systems, and legal-domain systems.

Source: [11], [23]

Issue generation is scored through average case-level weighted F-score over the test set.

Source: [24]

Issue classification uses average case-level weighted F-score at progressively stricter hierarchy granularities, with a true positive requiring aligned issues and matching category labels.

Source: [7], [19], [20], [24]

Prediction-reference matching proceeds through coarse semantic candidate mapping followed by stricter rubric-based verification of issue equivalence.

Source: [16], [18]

Only candidate pairs passing the predefined acceptance criterion are retained as aligned.

Source: [16]

Retrieval augmentation generally improves performance relative to corresponding settings without retrieval, although reported exceptions remain.

Source: [5], [9]

Under direct instruction, a proprietary general-purpose system achieved the strongest overall reported performance across generation and hierarchical classification.

Source: [5]