What the study found
The study found that human-centric evaluation of semantic resources, such as ontologies and knowledge graphs, has been studied in a way that can be organized into a theoretical framework. It also identified approaches, trends, and best practices from 15 years of published work.
Why the authors say this matters
The authors conclude that this work can help enable new research in this area and offer practical guidelines for performing human-centric evaluation of semantic resources to researchers and practitioners alike.
What the researchers tested
The researchers first integrated existing literature into a theoretical framework for defining and characterizing human-centric evaluation of semantic resources (evaluation that depends on human participation rather than automatic checking alone). They then conducted a systematic mapping study of 144 papers published over 15 years.
What worked and what didn't
The mapping study appears to have been useful for grounding and extending the theoretical framework. It also identified current trends and practices, although the abstract does not list specific methods that worked better or worse than others.
What to keep in mind
The abstract does not describe detailed limitations of the study. The summary also does not provide specific findings about particular semantic resources, domains, or evaluation methods beyond the general mapping of the literature.
Key points
- The article focuses on human-centric evaluation of semantic resources such as ontologies and knowledge graphs.
- The authors say this area lacked a systematic theoretical understanding and overview of approaches and trends.
- The study mapped 15 years of research and covered 144 papers.
- The work aimed to ground and extend a theoretical framework for the area.
- The authors say the resulting foundations, trends, and practices may support future research and practical guidance.
Disclosure
- Research title:
- Study maps human-centric evaluation of semantic resources
- Authors:
- Marta ; id_orcid 0000-0001-9301-8418 Sabou, Stefani Tsaneva, Miriam Fernández, María Poveda-Villalón, Mari Carmen Suárez-Figueroa
- Institutions:
- The Open University, Universidad Politécnica de Madrid, Universidad Politécnica de Madrid, Vienna University of Economics and Business, Vienna University of Economics and Business
- Publication date:
- 2026-03-05
- DOI:
- W7133796410
- OpenAlex record:
- View
Get the weekly research newsletter
Stay current with scholarly research without reading academic papers — one filtered digest, every Friday.