Harrison.ai Foundation Model Excels in US Healthcare AI Challenge

Harrison.ai, a leading healthtech company, announced that its radiology-specific foundation model, Harrison.rad.1, delivered exceptional preliminary results in an independent, large-scale evaluation by Mass General Brigham (MGB) and the American College of Radiology.

Harrison.rad.1 was released last year and outperformed foundation models from OpenAI, Anthropic, Google and others on VQA-Rad, one of the most widely used benchmarks to evaluate and compare the performance of multimodal foundational models on medical tasks. Harrison.rad.1 achieved 82% accuracy and precision on closed questions filtered for plain radiographs. The results of the Healthcare AI Challenge are just now reaffirming these impressive results.

The challenge compared AI-generated chest radiograph reports with those written by radiologists in a Turing-style test, where participants couldn’t reliably tell the difference between AI-generated and human-written reports. Harrison.rad.1 achieved a 65.4% acceptability rate among practicing radiologists, versus 79.6% for clinicians’ reports.

Mass General Brigham and the American College of Radiology Data Science Institute organized this large-scale, independent evaluation, which involved 113 board-certified radiologists conducting 2,840 blinded evaluations across 117 reports during the 2025 ACR Annual Meeting. The interactive challenge invited participants to assess chest radiograph images and their corresponding radiology reports for clinical accuracy before revealing whether they were written by AI or human radiologists.

Harrison.rad.1 was the foundation model powering AI-generated reports that achieved a 65.4% acceptability rate compared to 79.6% for radiologist-written reports—a performance that Mass General Brigham noted demonstrates draft reporting AI solutions in radiology are “improving at breakneck speed” and are “closer than ever to meaningfully enhancing radiologist efficiency through draft report generation.”

These results come at a pivotal moment, as healthcare faces rising imaging volumes, radiologist shortages, and growing backlogs—highlighting AI’s potential to meet these demands while upholding clinical standards.

“We’re honored that Harrison.rad.1 was the foundation model behind these groundbreaking results in such a rigorous, independent clinical evaluation,” said Dr. Aengus Tran, CEO and Co-Founder of Harrison.ai. “Mass General Brigham’s challenge represents exactly the kind of real-world validation needed to advance AI in radiology responsibly.

“While this is a significant milestone in advancing meaningful AI for radiology, it also underscores that our work is not yet done. We will continue to enhance our model, expand testing, and engage with regulators to meet clinical evidence requirements for deployment.”

The Science Behind the Success

Harrison.rad.1’s exceptional performance stems from its specialized training on millions of DICOM images and radiology reports across all X-ray modalities. Unlike general-purpose AI models, Harrison.rad.1 has been specifically designed for factual correctness and clinical precision in radiology tasks.

The model’s capabilities are backed by rigorous benchmarking results:

• 85.67% score on the challenging FRCR 2B Rapids exam, outperforming other AI models by approximately 2x and exceeding the performance of human radiologists retaking the exam within a year of passing
• 82% accuracy on the VQA-Rad benchmark for plain radiographs
• 73% accuracy on RadBench, Harrison.ai‘s comprehensive radiology evaluation dataset

Powered by Comprehensive AI Infrastructure

Harrison.rad.1 leverages the high-quality radiology data used to train Harrison’s comprehensive clinical decision-support medical device for chest X-rays (Annalise Enterprise CXR¹). Annalise Enterprise CXR detects up to 124 chest X-ray findings and has been shown to increase diagnostic accuracy by 45% when used as an assistive tool for radiologists².

The breakthrough results reflect the natural synergy between Harrison.rad.1’s foundation model capabilities and the comprehensive AI algorithms that power Harrison’s clinical decision-support tools.

“It makes complete sense that the industry’s most promising report generation results would be powered by the most comprehensive AI algorithms on the market,” noted Dr. Jarrel Seah, Chief AI Officer at Harrison.ai. “Our Annalise.ai CXR and CTB models have been trained on some of the world’s largest and diverse radiologist hand-annotated datasets, providing the robust foundation needed for accurate report generation.”

Regulatory Recognition and Clinical Adoption

Harrison.ai’s comprehensive approach to medical AI has earned significant regulatory recognition globally, with clearance in 40+ countries and deployed in 15 countries. The company’s solutions are accessible to 50% of radiologists in Australia and are used to process more than 35% of chest X-rays in England. In the US, Annalise.ai has received Medicare New Technology Add-on Payment (NTAP) status and has achieved FDA 510(k) clearance for multiple CXR and CTB findings.

Health Technology Insights: Dried Plasma Innovation Addresses Critical Global Need

To participate in our interviews, please write to our HealthTech Media Room at sudipto@intentamplify.com