At TNO (Netherlands organisation for applied scientific research) and Delft University of Technology, researchers have been working on a research & development method for complex, intelligent, and interactive technology called "Socio-Cognitive Engineering" (SCE). The SCE method has been developed over the years in an iterative way, and has been applied in multiple projects and a wide variety of application domains such a task support system in naval ships (Neerincx & Lindenberg, 2008); ePartners for astronauts during long-term missions (Neerincx et al., 2011); a support system for elderly living at home (Spiekman et al., 2011); a human-agent/robot teamwork system for disaster response (Mioch et al., 2012); a virtual coach for low-literates to practice societal participation (Schouten et al., 2017); a virtual assistant for workload harmonization in train-traffic control teams (Harbers et al., 2017); and a hybrid agent-based system for child's diabetes management (Neerincx et al., 2019). An overview of the literature can be found at https://scetool.ewi.tudelft.nl.
The SCE method structures and guides the systematic creation, evaluation and documentation of technological designs that aim to provide social, cognitive and affective support to help humans in realizing their goals and values. The method combines different design approaches and techniques in order to establish a sound, theoretically and empirically grounded design solution. In contrast to a “waterfall procedure”, SCE allows for quick, incremental, and iterative generate-and-test cycles (e.g. agile development). In this process, researchers and developers are required to continuously specify, refine, and integrate different parts of the system (i.e. components). SCE supports the sharing and reusing of the evolving design knowledge by the heterogeneous, multi-disciplinary development community. Key is the generation, refinement, validation, maintenance, and reuse of coherent and concise design specifications. Such design specifications describe both what the technology should do and the underlying design rationale (the why and when). Three main segments are distinguished: (A) Foundation, (B) Specification, and (C) Evaluation (Below, we will explain the SCE method; for some short examples, see SCE Example).
The foundation describes the design rationale in terms of (a) operational demands, (b) relevant human factors knowledge, and (c) envisioned technologies. Together, these three constituents describe (1) the problem to be solved, (2) the existing knowledge on ways to solve the problem, and (3) the technology needed to implement that solution.
The operational demands describe the current practice as it is, i.e. without the envisioned technology. For the operational demands, SCE prescribes as main components the stakeholders and their characteristics (including their values), and the problem description and analysis thereof. A problem scenario provides a short storyline to illustrate the way in which stakeholders experience the problem. To obtain a clear overview of the problem, it needs to be analysed: what are the causes of the problem? Can the problem be broken down into smaller problems? And what type of problems are related to this problem yet beyond the scope of the current problem description?
When designing technology, there are two driving questions that need to be well-thought out: (1) What tasks and/or values is the human trying to accomplish and how can the technology support the human in doing so?, and (2) How can the technology be designed such that the human is able to work with the technology? The Human Factors component of the SCE method describes the available relevant knowledge about, for instance, human cognition, performance, task support, learning, human-machine interaction, ergonomics, etc. Note that we emphasize that this knowledge should be relevant for the problem and its design solution: the knowledge described here should lead to a better understanding of either (1) or (2). The three constituents important to the human factors analysis are: (a) the human factors knowledge, (b) measures, and (c) interaction design patterns. Human factors knowledge describes available knowledge coming from previous research about how to solve the problems that have been identified in the problem analysis. Measures describe how to operationalise the quality of the intended behaviour or performance, i.e. how well is a user working with the design able to reach his/her objectives and what is the quality of the collaboration between the human worker and the technology? Interaction design patterns (IDPs) focus on the human-computer interaction, such as usable interface design and control options. IDPs offer generic solutions to recurring HCI design problems that have been proven to be effective. IDPs are often made available in repositories.
The envisioned technology describes the available options of using existing technology and/or the need to develop novel technology in order to come to a system solution. The SCE method asks designers to specify what devices (hardware) and software could be used in the system design. In addition, for each type of technology, an argument should be provided as to why this technology might be of use and what the possible downsides might be of that particular type of technology.
The system design specification describes the solution to the problem in the form of a system design that makes use of the identified relevant human factors knowledge and the envisioned technology. The system specification consists of (a) design scenarios, (b) use cases, (c) requirements, and (d) claims.
The design evaluation aims to test and validate the system’s design, or to discriminate between multiple design options, such that the current design can be improved upon in incremental development cycles. The SCE method describes two parts that are relevant with respect to the system evaluation: (1) the prototype and/or simulation, and (2) the evaluation that describes the evaluation method and results.
The SCE method prescribes the construction of an ontology, i.e. a vocabulary describing a common language to be used throughout the system specification to avoid miscommunication, misunderstanding, and inconsistencies. Furthermore, the ontology can serve as the basis for the technology’s data structure. By specifying important concepts in the ontology and also choosing to use only one word instead of various ambiguous synonyms, communication becomes clearer and misunderstandings can be reduced to a minimum. The terms specified in the ontology are consistently used throughout the entire project.