Evaluating the Value of User Experience in Humanitarian and Development Contexts: A Logic Model Approach
Wahid bin Ahsan
Department of Human-Centered Design
Userhub
Abstract
User experience (UX) plays an increasingly vital role in humanitarian and development initiatives, yet its contributions are often underrepresented in formal evaluations. Traditional frameworks such as the OECD DAC criteria, Theory of Change (ToC), and Social Return on Investment (SROI) typically overlook the role of design practices in shaping user-centered and systemic outcomes. This paper proposes a logic-informed conceptual model for evaluating UX value, developed through a cross-domain synthesis of literature from commercial UX evaluation, ICT4D, development theory, SROI, and systems thinking. The model identifies five dimensions of UX value—usability and interaction quality; adoption and engagement; empowerment and social value; programmatic and operational value; and strategic alignment and system impact—which collectively link UX design practices to user, organizational, and system outcomes. Designed as a hypothesis-generating tool, the model offers a structured vocabulary for evaluation planning and invites further empirical testing across diverse development sectors.
Keywords: user experience, UX evaluation, development programs, logic model, ICT4D, SROI, DAC criteria, systems thinking
Introduction
User experience (UX) is widely recognized in commercial sectors as a driver of product success, customer satisfaction, and return on investment. In humanitarian and development contexts, however, UX remains underdefined, undervalued, and inconsistently evaluated. While digital tools are now central to programs in health, education, identity, and humanitarian response, the design of those tools—and the experiences they create—are rarely examined in formal evaluations. UX may influence adoption, trust, efficiency, or sustainability, yet these effects often remain invisible in program logic, reporting systems, and funding decisions.
Evaluation models such as the OECD DAC criteria—covering relevance, effectiveness, efficiency, impact, and sustainability—provide structure for assessing outcomes (OECD, 2021). Theory of Change (ToC) models articulate causal pathways, while Social Return on Investment (SROI) frameworks attempt to value intangible outcomes such as empowerment or well-being. However, these approaches seldom account for how design quality or interaction experience shapes results (Chianca, 2008; Cooney & Lynch-Cerullo, 2014; LaFond & Cherney, 2021).
In commercial settings, tools for measuring UX value exist (Bias & Mayhew, 2005; Turner, 2011), but their assumptions often don’t translate to development programs, where value is collective, contextual, and not always financial. Even in ICT4D and HCI4D research, UX evaluation remains fragmented—focused on project-specific adoption studies rather than broader models of value (Sambasivan et al., 2009; Soehnchen et al., 2023).
This disconnect limits the ability of practitioners to demonstrate impact and of evaluators to recognize UX as a strategic asset. UX is often treated as an aesthetic layer or assumed input rather than a measurable contributor to development outcomes.
This paper responds by proposing a conceptual framework and corresponding logic model for evaluating the value of UX in development contexts. Drawing on a cross-domain synthesis of literature, the model maps UX design practices to five conceptual dimensions of value—usability and interaction quality, adoption and engagement, empowerment and social value, programmatic and operational value, and strategic alignment and system impact—which in turn link to user, organizational, and system outcomes. Rather than prescribe fixed metrics, the model is designed as a hypothesis-generating tool to support evaluation planning, critical reflection, and future empirical testing across diverse development settings.
Methodology
This paper adopts a conceptual framework development approach, grounded in structured literature synthesis, interdisciplinary integration, and design science principles. The aim is to construct a logic-informed, hypothesis-generating model that identifies potential areas where UX may contribute value in humanitarian and development contexts. The model maps UX design practices to five conceptual dimensions of value, which link to user, organizational, and system outcomes. Drawing on traditions such as Theory of Change and logic models, it is designed to support structured reflection, evaluation planning, and future hypothesis testing across diverse development settings.
Framework Orientation and Logic Model Structure
The framework addresses a key gap: although UX is increasingly integrated into development and humanitarian programs, its contributions are rarely evaluated using established models such as the OECD DAC Criteria (OECD, 2021), Theory of Change (Weiss, 1995), or Social Return on Investment (Nicholls et al., 2012). The framework seeks to surface where UX creates value, where it remains unrecognized, and how it might be represented more systematically through a logic-based evaluation model.
Literature Scope and Domain Selection
A targeted review of 20 conceptually rigorous sources was conducted across five relevant domains:
- Commercial UX ROI – cost justification, usability metrics, persuasive design
- ICT4D / HCI4D – usability in low-resource settings, sociotechnical context
- Development Evaluation – DAC Criteria, Theory of Change, Value for Money
- SROI – stakeholder-defined, non-financial impact
- Systems Thinking and Participatory Design – adaptive systems, empowerment
Sources were selected for conceptual clarity, methodological rigor, and practical relevance. Both peer-reviewed and credible gray literature were included.
Thematic Synthesis and Dimension Clustering
Following Jabareen’s (2009) conceptual framework methodology, a qualitative synthesis identified recurring value-related concepts across the domains. Terms such as usability, trust, adoption, empowerment, and systemic alignment were compared and clustered into five core dimensions. These dimensions form the foundation of the proposed framework and are accompanied by illustrative indicators from the literature.
Bias Mitigation and Reflexivity
The framework is explicitly positioned as hypothesis-generating, not normative. It does not assume that UX is inherently beneficial or universally measurable. Dimensions reflect potential value that should be tested contextually. Boundary conditions are acknowledged, particularly where UX influence may be minimal due to implementation quality or contextual misalignment.
Limitations
The framework has not been empirically validated. It is limited to theory-building and conceptual synthesis. Future research is needed to assess its utility in specific domains—such as health, education, or digital identity—and its adaptability across geographic and organizational settings.
Literature Review
User experience (UX) is well established in commercial settings but remains inconsistently recognized and rarely evaluated in humanitarian and development contexts. This section synthesizes insights across five relevant domains: commercial UX evaluation, ICT4D and HCI4D, development evaluation frameworks, social return on investment (SROI), and systems thinking and participatory design. Together, they form the structure and intent of the proposed framework.
UX ROI in Commercial Contexts
In commercial environments, UX is often justified through return-on-investment (ROI) models. Foundational sources such as Cost-Justifying Usability (Bias & Mayhew, 2005) emphasize efficiency metrics including task time, error rates, and support reduction. Turner (2011) adds strategic dimensions such as user satisfaction and loyalty. However, ROI logic often oversimplifies UX value, overlooking trust, inclusion, and context (Chawana & Adebesin, 2021; Dray et al., 2005). These assumptions rarely translate to non-commercial domains, where value is systemic, intangible, or collective.
UX in ICT4D and HCI4D
UX is increasingly acknowledged as critical in ICT4D and HCI4D, especially for adoption and contextual fit in digital health, education, and identity systems. Sambasivan et al. (2009) highlight how usability is shaped by social and infrastructural factors in low-resource environments. Rico-Olarte et al. (2018) and Soehnchen et al. (2023) propose field-based UX frameworks that include task success and user trust. Ng et al. (2024) link UX quality to sustained tool uptake. Yet these contributions often remain project-specific and disconnected from broader evaluation logic.
Development Evaluation Frameworks
Development evaluations typically use frameworks like the OECD DAC criteria or Theory of Change, which assess relevance, effectiveness, and impact. However, these models rarely incorporate UX explicitly. Chianca (2008) critiques DAC for its top-down orientation, while Peterson & Skolits (2020) note that Value for Money logic often neglects enabling factors like design. LaFond & Cherney (2021) argue for integrating human-centered design into ToC models, where design can influence stakeholder engagement and intermediate outcomes.
Social Return on Investment (SROI)
SROI attempts to value outcomes like empowerment, confidence, and well-being. Yet efforts to link these to UX are limited. Cooney & Lynch-Cerullo (2014) and Cordes (2017) highlight methodological challenges in monetizing qualitative change. While Gambhir et al. (2017) and Burford et al. (2013) propose values-based indicators that align with UX outcomes, these are rarely traced back to specific design elements or practices.
Systems Thinking and Participatory Design
Systems thinking and participatory design offer valuable insights into UX contributions to adaptive, resilient systems. Gates (2016) and Steele Gray & Shaw (2019) emphasize feedback loops and emergent evaluation, while Mahajan et al. (2019) show how user-centered design supports long-term sustainability. Drain et al. (2021) link participatory approaches to user empowerment and ownership. However, even in these models, interface quality and interaction design remain implicit rather than explicitly assessed.
Synthesis and Gaps
Across these domains, UX is positioned as a valuable—but inconsistently evaluated—contributor to development outcomes. Commercial models offer robust metrics but limited contextual transferability. ICT4D and participatory design point to contextual importance but lack generalizable evaluation tools. Development and SROI frameworks recognize outcomes aligned with UX but rarely connect them to design practices. This synthesis affirms the need for a structured, interdisciplinary model that articulates UX value in terms evaluators and funders can recognize.
The UX Value Logic Model: Dimensions and Outcome Pathways
This section introduces a logic-informed conceptual model for evaluating the value of user experience (UX) in humanitarian and development programs. Drawing from a structured synthesis of literature across five domains—commercial UX evaluation (Bias & Mayhew, 2005; Turner, 2011), ICT4D and HCI4D (Sambasivan et al., 2009; Soehnchen et al., 2023), development evaluation theory (Chianca, 2008; Peterson & Skolits, 2020), social return on investment (Cooney & Lynch-Cerullo, 2014; Gambhir et al., 2017), and systems thinking (Drain et al., 2021; Gates, 2016)—the model identifies key pathways through which UX design practices contribute to measurable value.
The framework links UX design practices (e.g., usability testing, co-design, interface simplification) to five conceptual value dimensions: usability, adoption, empowerment, operational value, and system impact. These dimensions reflect hypothesized UX contributions that influence outcomes at user, organizational, and system levels.
Figure 1 presents the UX Value Logic Model, which illustrates how design practices shape value across interrelated domains, leading to distinct but overlapping outcomes. The model supports reflection, evaluation planning, and hypothesis generation—encouraging practitioners to ask: what value does UX create, and how can it be surfaced and acted upon?
Figure 1. UX Value Logic Model for Humanitarian and Development Contexts
1. Usability and Interaction Quality
This dimension focuses on how effectively users engage with a system to complete tasks. It includes core usability metrics such as task success, error prevention, and learnability (Bias & Mayhew, 2005), as well as context-sensitive concerns like accessibility and cultural relevance (Sambasivan et al., 2009). In low-resource environments, usability is not just about efficiency—it often determines whether users can access services at all.
2. Adoption and Engagement
This dimension considers whether users trust and continue using a system over time. Sustained use is crucial for achieving outcomes in areas like health or education. While adoption is commonly measured in ICT4D, it is rarely linked directly to UX quality (Ng et al., 2024; Soehnchen et al., 2023). (Dray et al., 2005) show that persuasive, credible design builds early trust, which strongly influences long-term engagement.
3. Empowerment and Social Value
This dimension reflects intangible outcomes such as user confidence, inclusion, and perceived agency. SROI frameworks seek to measure such impacts (Cooney & Lynch-Cerullo, 2014; Gambhir et al., 2017), but often overlook how they are influenced by design. Research in participatory design (Burford et al., 2013; Drain et al., 2021) shows that clarity, co-creation, and respectful interaction shape user perceptions—especially for marginalized groups.
4. Programmatic and Operational Value
This dimension considers how UX affects organizational efficiency. It includes reduced training demands, fewer user errors, and lower support costs (LaFond & Cherney, 2021; Turner, 2011). Poor design often increases burdens on frontline workers and implementers, yet this operational impact is rarely captured in formal evaluations.
5. Strategic Alignment and System Impact
The final dimension links UX to broader goals such as policy coherence, sustainability, and adaptability. Drawing on systems thinking (Gates, 2016; Mahajan et al., 2019), it highlights how design choices—such as enabling feedback loops or modular workflows—support resilient, scalable systems. As Chianca (2008) notes, macro-level evaluations often miss these micro-level contributions.
These dimensions are not isolated. In practice, they reinforce or constrain one another. For example, high usability may drive adoption, which can in turn enhance operational efficiency or strategic alignment. Rather than prescribe metrics or hierarchies, the framework encourages context-specific reflection: which dimensions are relevant, and how can UX value be surfaced and measured?
By offering a shared vocabulary for articulating UX contributions, this framework supports more intentional integration of design into development evaluation. It invites practitioners, evaluators, and funders to see UX not as peripheral, but as a meaningful, multi-dimensional component of program value. Table 1 provides illustrative indicators for each dimension, offering practical entry points for evaluation design and future field validation.
Table 1. Illustrative Indicators for Evaluating UX Value in Development Contexts
| Key Indicators by UX Dimension | ||
| Dimension | Sample UX Contributions | Illustrative Indicators |
| Usability and Interaction Quality | Supports task success, reduces cognitive load, ensures accessibility | Task completion rate, time-on-task, error rate, user satisfaction score, accessibility compliance (e.g., WCAG) |
| Adoption and Engagement | Builds trust, encourages sustained use, reduces drop-off | Retention rate, repeat usage frequency, bounce rate, Net Promoter Score (NPS), first-week engagement rate |
| Empowerment and Social Value | Increases user confidence, inclusion, agency, and dignity | User-reported empowerment, inclusion metrics, qualitative feedback on agency, number of co-created features |
| Programmatic and Operational Value | Reduces support burden, improves implementation efficiency | Training time required, helpdesk inquiries, implementation delay reduction, staff workload impact |
| Strategic Alignment and System Impact | Enhances long-term fit, adaptability, policy coherence | Reusability of components, alignment with sector standards, integration into system architecture, perceived scalability |
Discussion and Conclusion
This paper has proposed a conceptual framework for evaluating the value of user experience (UX) in humanitarian and development contexts. Drawing from literature in UX evaluation, ICT4D, development theory, SROI, and systems thinking, the framework identifies five dimensions of potential UX value: usability and interaction quality, adoption and engagement, empowerment and social value, programmatic and operational value, and strategic alignment and system impact.
Organizing UX value into these dimensions helps address a common limitation in current practice: UX contributions are often excluded from evaluation logic. As digital tools become more central to development delivery, design quality directly influences trust, adoption, and program performance. Yet these effects often go unmeasured. This framework, supported by the UX Value Logic Model, shifts how UX is positioned—not as a peripheral design feature but as a contributor to outcomes that matter to users, funders, and implementing partners.
Rather than offer standardized indicators or fixed metrics, the framework invites contextual evaluation. It aligns UX contributions with sectoral goals, helping teams surface design impacts within established models like the DAC criteria or Theory of Change.
Implications for Stakeholders
For UX practitioners, the framework offers language to demonstrate value in terms aligned with program logic. It supports communication with funders and implementers by translating design choices into recognized forms of contribution.
For NGOs and implementers, it identifies where UX may improve operational outcomes—from lowering training costs to increasing tool adoption. It also helps justify design investment within resource-constrained environments.
Limitations and Future Directions
This framework is conceptual and has not yet been empirically validated. It should not be generalized without further research. Its value depends on the quality and contextual relevance of UX practice. Some dimensions—especially empowerment and strategic alignment—may be harder to isolate or quantify, requiring participatory and adaptive evaluation methods.
Future work should test the framework across domains such as mHealth, education, or digital ID; co-develop tools with NGOs and users; and explore how UX indicators align with program logic. These steps will help refine the logic model and assess its practical relevance across diverse settings.
Conclusion
As digital systems reshape development practice, UX must be evaluated with the same care as programmatic or policy outcomes. This paper offers a starting point: a conceptual framework for articulating the value of UX where it has long been assumed but rarely measured. Anchored by a logic model, it supports a more evidence-informed, strategic approach to human-centered development.
References
OECD. (2021). Applying Evaluation Criteria Thoughtfully. OECD. https://doi.org/10.1787/543e84ed-en
Bias, R. G., & Mayhew, D. J. (Eds.). (2005). Cost-Justifying Usability: An Update for the Internet Age. Morgan Kaufmann. https://www.sciencedirect.com/book/9780120958115/cost-justifying-usability
Burford, G., Hoover, E., Velasco, I., Janoušková, S., Jimenez, A., Piggot, G., Podger, D., & Harder, M. (2013). Bringing the “Missing Pillar” into Sustainable Development Goals: Towards Intersubjective Values-Based Indicators. Sustainability, 5(7), 3035–3059. https://doi.org/10.3390/su5073035
Chawana, T., & Adebesin, F. (2021). The current state of measuring return on investment in user experience design. South African Computer Journal, 33(1). https://doi.org/10.18489/sacj.v33i1.950
Chianca, T. (2008). The OECD/DAC Criteria for International Development Evaluations: An Assessment and Ideas for Improvement. Journal of MultiDisciplinary Evaluation, 5(9), 41–51. https://doi.org/10.56645/jmde.v5i9.167
Cooney, K., & Lynch-Cerullo, K. (2014). Measuring the Social Returns of Nonprofits and Social Enterprises: The Promise and Perils of the SROI. Nonprofit Policy Forum, 5(2), 367–393. https://doi.org/10.1515/npf-2014-0017
Cordes, J. J. (2017). Using cost-benefit analysis and social return on investment to evaluate the impact of social enterprise: Promises, implementation, and limitations. Evaluation and Program Planning, 64, 98–104. https://doi.org/10.1016/j.evalprogplan.2016.11.008
Drain, A., Shekar, A., & Grigg, N. (2021). Insights, Solutions and Empowerment: a framework for evaluating participatory design. CoDesign, 17(1), 1–21. https://doi.org/10.1080/15710882.2018.1540641
Dray, S., Karat, C.-M., Rosenberg, D., Siegel, D., & Wixon, D. (2005). Is ROI an effective approach for persuading decision-makers of the value of user-centered design? CHI ’05 Extended Abstracts on Human Factors in Computing Systems, 1168–1169. https://doi.org/10.1145/1056808.1056865
Gambhir, V. K., Majmudar, N., Sodhani, S., & Gupta, N. (2017). Social Return on Investment (SROI) for Hindustan Unilever’s (HUL) CSR initiative on livelihoods (Prabhat). Procedia Computer Science, 122, 556–563. https://doi.org/10.1016/j.procs.2017.11.406
Gates, E. F. (2016). Making sense of the emerging conversation in evaluation about systems thinking and complexity science. Evaluation and Program Planning, 59, 62–73. https://doi.org/10.1016/j.evalprogplan.2016.08.004
Jabareen, Y. (2009). Building a Conceptual Framework: Philosophy, Definitions, and Procedure. International Journal of Qualitative Methods, 8(4), 49–62. https://doi.org/10.1177/160940690900800406
LaFond, A., & Cherney, M. (2021). A Theory of Change for Guiding the Integration of Human-Centered Design Into Global Health Programming. Global Health: Science and Practice, 9(Supplement 2), S209–S216. https://doi.org/10.9745/GHSP-D-21-00334
Mahajan, S. L., Glew, L., Rieder, E., Ahmadia, G., Darling, E., Fox, H. E., Mascia, M. B., & McKinnon, M. (2019). Systems thinking for planning and evaluating conservation interventions. Conservation Science and Practice, 1(7). https://doi.org/10.1111/csp2.44
Ng, W. Y., Lau, N. Y., Lee, V. V., Vijayakumar, S., Leong, Q. Y., Ooi, S. Q. D., Su, L. L., Lee, Y. S., Chan, S.-Y., Blasiak, A., & Ho, D. (2024). Shaping Adoption and Sustained Use Across the Maternal Journey: Qualitative Study on Perceived Usability and Credibility in Digital Health Tools. JMIR Human Factors, 11, e59269. https://doi.org/10.2196/59269
Nicholls, J., Lawlor, E., Neitzert, E., & Goodspeed, T. (2012). A Guide to Social Return on Investment. The SROI Network. http://www.socialvalueuk.org/resource/a-guide-to-social-return-on-investment-2012/
Peterson, C., & Skolits, G. (2020). Value for money: A utilization-focused approach to extending the foundation and contribution of economic evaluation. Evaluation and Program Planning, 80, 101799. https://doi.org/10.1016/j.evalprogplan.2020.101799
Rico-Olarte, C., López, D. M., & Kepplinger, S. (2018). Towards a Conceptual Framework for the Objective Evaluation of User Experience (pp. 546–559). https://doi.org/10.1007/978-3-319-91797-9_39
Sambasivan, N., Rangaswamy, N., Toyama, K., & Nardi, B. (2009). UNDER DEVELOPMENT Encountering development ethnographically. Interactions, 16(6), 20–23. https://doi.org/10.1145/1620693.1620698
Soehnchen, C., Weirauch, V., Schmook, R., Henningsen, M., & Meister, S. (2023). An acceptance analysis of a sexual health education digital tool in resource-poor regions of Kenya: an UTAUT based survey study. BMC Women’s Health, 23(1), 676. https://doi.org/10.1186/s12905-023-02839-6
Steele Gray, C., & Shaw, J. (2019). From summative to developmental: Incorporating design-thinking into evaluations of complex interventions. Journal of Integrated Care, 27(3), 241–248. https://doi.org/10.1108/JICA-07-2018-0053/FULL/PDF
Turner, C. W. (2011). A Strategic Approach to Metrics for User Experience Designers. J. Usability Studies, 6(2), 52–59.
Weiss, C. H. (1995). Nothing as Practical as Good Theory: Exploring Theory-Based Evaluation for Comprehensive Community Initiatives for Children and Families. In J. P. Connell, A. C. Kubisch, L. B. Schorr, & C. H. Weiss (Eds.), New Approaches to Evaluating Community Initiatives. Aspen Institute.
Appendix: Glossary of Terms and Concepts
Adoption and Engagement
The extent to which users accept, trust, and continue using a digital tool or service over time. High engagement often reflects perceived usefulness, credibility, and satisfaction.
Co-created Features
Design elements or functionalities developed collaboratively with users, often through participatory or co-design processes. These features are intended to reflect user needs, preferences, and values more directly. See: Drain et al., 2018 – https://doi.org/10.1177/1356389018768371
Empowerment and Social Value
Refers to intangible benefits such as user confidence, dignity, inclusion, and perceived agency resulting from an interface, service, or participatory design process.
Net Promoter Score (NPS)
A widely used loyalty metric based on users’ likelihood to recommend a product or service to others.
Source: Reichheld, F. F. (2003). The One Number You Need to Grow. Harvard Business Review. https://hbr.org/2003/12/the-one-number-you-need-to-grow
Programmatic and Operational Value
UX contributions to organizational efficiency, such as reduced training needs, fewer support requests, or smoother implementation.
Scalability
The ability of a product, service, or system to adapt or expand effectively across multiple contexts, populations, or institutional environments. See: Mahajan et al., 2019
Strategic Alignment and System Impact
How UX decisions contribute to broader systemic outcomes, such as sustainability, interoperability, policy alignment, or long-term adaptability.
System Outcomes
Macro-level effects of UX design, including resilience, scalability, integration with public systems, or cross-sector alignment.
User Outcomes
Effects of UX on individual users, such as increased task success, trust, satisfaction, or confidence.
Usability and Interaction Quality
The degree to which a system allows users to perform tasks effectively and efficiently, including measures like task completion, error rates, and learnability.
User-Reported Empowerment
Self-reported increases in confidence, agency, and inclusion as a result of interacting with a system or service. See: Gambhir et al., 2017
WCAG (Web Content Accessibility Guidelines)
A set of technical standards developed by the W3C to ensure digital content is accessible to people with diverse abilities. WCAG focuses on content being perceivable, operable, understandable, and robust.