Evaluating Large Language Models for Legal Q&A
Evaluated BERT-Double, Legal-BERT, and Custom Legal-BERT on CaseHOLD, a 53,000+ question legal reasoning dataset. Fine-tuned models and assessed performance using accuracy, Macro-F1, and evaluation loss. Designed and implemented a novel Weighted Correctness Score incorporating question difficulty, model confidence, and correctness to identify model weaknesses missed by traditional accuracy metrics.