The anatomy of a personal health agent.
Researchers developed an open-source multi-agent framework for a personal health agent and evaluated three specialist sub-agents and the full system across 11 benchmark tasks. Based on four user journeys—general health knowledge, personal data interpretation, wellness advice and symptom assessment—the framework deploys three specialist agents: data science, health domain expert and health coach, processing multimodal data from consumer wellness devices and medical records. Automated and human evaluation drew on more than 7,000 annotations and 1,350 h of effort from health experts and end users.
Large language models are reshaping the landscape of AI applications in health. Previous studies have shown that such models can encode a considerable degree of clinical knowledge, achieving near-expert performance on medical question-answering benchmarks [2], and can provide clinically valuable recommendations in decision-support scenarios for precision oncology [3]. At the same time, the rapid accumulation of multimodal data from wearable devices, electronic health records, biosensors, and other sources has laid a data foundation for AI solutions that capture the complexity of individual health, and virtual health assistants are regarded as one of the most promising application directions among them [1]. But where the bottleneck lies is also quite clear: transforming data generated by wearables into personalized insights requires complex numerical reasoning that standard large models struggle with, often relying on tool-based approaches such as code generation [4]; complex tabular reasoning over electronic health records likewise requires specialized agentic architectures [5]; and even in relatively mature consumer applications such as conversational mental health support, systems face challenges like filtering harmful content, maintaining consistent communication, and remembering new information, and whether users are suited to rely on such tools remains unsettled [6]. In other words, a complete agent oriented toward ordinary people's everyday health needs—one that can integrate multimodal data and conduct personalized interactions—had previously lacked systematic construction and evaluation.
This paper is precisely the work undertaken against this backdrop. The research team first analyzed web search and health forum queries, combined with user-centered studies involving end users and health experts, to distill four core user journeys: general health knowledge acquisition, personal data interpretation, healthy living advice, and symptom assessment. Around these needs, they built a multi-agent framework composed of three expert sub-agents: a data science sub-agent responsible for analyzing longitudinal wearable and medical data and incorporating population-level statistics to provide context for numerical insights; a health domain expert sub-agent that combines personal data and contextual information with health knowledge to generate personalized insights; and a health coach sub-agent responsible for synthesizing insights, supporting multi-turn interaction and goal setting, applying specific psychological strategies, and tracking progress. The three sub-agents were integrated into an open-source personal health agent framework enabling dynamic, personalized health interactions. For evaluation, the researchers conducted both automated and human evaluations of the sub-agents and the full system across 11 benchmark tasks, accumulating more than 7000 annotations, with health experts and end users contributing over 1350 hours of effort. The paper describes this as the most comprehensive evaluation of a personal health agent to date, and it provides a concrete foundation for broader access to high-quality personalized health support.
Compared with existing knowledge, the advance of this work lies in integrating previously scattered capabilities into a complete system oriented toward everyday scenarios. Prior agents based on code generation and retrieval have already demonstrated high accuracy on wearable data analysis [4], and natural-language interaction agents for electronic health records have also demonstrated feasibility [5], but these two lines of work each focused on a single data type or task modality; this study instead incorporates data analysis, domain knowledge, and behavioral intervention coaching into a single user-centered architecture, systematically validated through large-scale human evaluation, and its evaluation scale (more than 7000 annotations, over 1350 hours of expert effort) is also the largest explicitly emphasized in the materials among comparable work. Notably, the team released the system as open source, which facilitates subsequent research and reproducibility.
Of course, limitations and unresolved questions remain. The paper's materials themselves do not report the system's deployment performance or safety data in real clinical environments, and its evaluation is still limited to benchmark tasks and controlled human assessment; whether the risks previously exposed in research on conversational mental health agents, such as harmful content filtering and user over-reliance [6], have been adequately addressed in this system is not directly answered in the materials. Personal health support involves deep challenges of data privacy and multimodal modeling [1], and how to move from benchmark performance to trustworthy everyday use remains an open question.
- 1.Multimodal biomedical AI · Nat Med 2022
- 2.Large language models encode clinical knowledge · Nature 2023
- 3.Leveraging Large Language Models for Decision Support in Personalized Oncology · JAMA Netw Open 2023
- 4.Transforming wearable data into personal health insights using large language model agents · Nat Commun 2026
- 5.EHRAgent: Code Empowers Large Language Models for Few-shot Complex Tabular Reasoning on Electronic Health Records · Proc Conf Empir Methods Nat Lang Process 2024
- 6.Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Support · AMIA Annu Symp Proc 2023
Maps four user journeys to three specialist agents and evaluates them on 11 tasks with over 7,000 annotations, offering a transferable design and evaluation framework.
Source: Nature · doi.org
