LexIssue: Benchmarking Legal Issue Identification in Chinese Civil Litigation

The work introduces a legally grounded hierarchy that combines free-form descriptions of disputed matters with structured legal categories.

Source: E003, E010

It frames the task as complementary generation of issue descriptions and classification of their legally grounded attributes.

Source: E004, E010

Retrieval augmentation generally improves performance relative to corresponding settings without retrieval, although reported exceptions remain.

Source: E005, E009

Under direct instruction, a proprietary general-purpose system achieved the strongest overall reported performance across generation and hierarchical classification.

Source: E005

Legally informed prompting does not reliably improve performance and frequently harms it relative to the corresponding setting without that prompting.

Source: E012

Among predictions aligned to reference issues, classification performance declines markedly as hierarchical specificity increases, including for the strongest reported systems.

Source: E002

The work introduces a legally grounded hierarchy that combines free-form descriptions of disputed matters with structured legal categories.

Source: E003, E010

It frames the task as complementary generation of issue descriptions and classification of their legally grounded attributes.

Source: E004, E010

The study examines computational identification of disputed matters in litigation together with their legal attributes.

Source: E014

The work introduces a legally grounded hierarchy that combines free-form descriptions of disputed matters with structured legal categories.

Source: E003, E010

It frames the task as complementary generation of issue descriptions and classification of their legally grounded attributes.

Source: E004, E010

The evaluation resource comprises real-world civil litigation cases with expert annotation of disputed matters.

Source: E013

Each annotated matter includes both a descriptive summary and structured category labels, and the cases cover diverse civil dispute types.

Source: E013

The benchmark uses synthetic claim and defence materials reconstructed from judicial accounts of party pleadings.

Source: E008, E021

This reconstruction approach was used because original pleadings are difficult to obtain at scale from public sources.

Source: E021

Synthetic inputs cannot fully reproduce the form, style, strategic framing, or evidential detail of materials used in actual litigation.

Source: E021

Assessment is challenging because equivalent disputed matters can differ substantially in wording and abstraction.

Source: E001

Reliable prediction-reference alignment is therefore needed before computing evaluation metrics.

Source: E001

An issue-centred legal knowledge resource was curated from doctrinal and practical materials alongside litigated cases.

Source: E015, E022

Expert integration consolidated overlapping entries and normalized their granularity and style, producing reusable abstractions rather than case-specific annotations.

Source: E015

Models are tested in direct instruction, legally guided prompting, and retrieval-augmented settings.

Source: E017, E023

For retrieval augmentation, candidate issue entries are reranked for relevance and the selected entries are placed into the generation prompt.

Source: E006

The comparison includes proprietary general-purpose systems, openly available general-purpose systems, and legal-domain systems.

Source: E011, E023

Issue generation is scored through average case-level weighted F-score over the test set.

Source: E024

Issue classification uses average case-level weighted F-score at progressively stricter hierarchy granularities, with a true positive requiring aligned issues and matching category labels.

Source: E007, E019, E020, E024

Prediction-reference matching proceeds through coarse semantic candidate mapping followed by stricter rubric-based verification of issue equivalence.

Source: E016, E018

Only candidate pairs passing the predefined acceptance criterion are retained as aligned.

Source: E016

Retrieval augmentation generally improves performance relative to corresponding settings without retrieval, although reported exceptions remain.

Source: E005, E009

Under direct instruction, a proprietary general-purpose system achieved the strongest overall reported performance across generation and hierarchical classification.

Source: E005

Legally informed prompting does not reliably improve performance and frequently harms it relative to the corresponding setting without that prompting.

Source: E012

Among predictions aligned to reference issues, classification performance declines markedly as hierarchical specificity increases, including for the strongest reported systems.

Source: E002