# Using Work System Theory, Facets of Work, and Dimensions of Smartness to Characterize Applications and Impacts of Artificial Intelligence

Steven Alter

► To cite this version:

Steven Alter. Using Work System Theory, Facets of Work, and Dimensions of Smartness to Characterize Applications and Impacts of Artificial Intelligence. International Working Conference on Transfer and Diffusion of IT (TDIT), Dec 2020, Tiruchirappalli, India. pp.29-42, 10.1007/978-3-030-64849-7_4. hal-03701802

HAL Id: hal-03701802

https://inria.hal.science/hal-03701802

Submitted on 22 Jun 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Using Work System Theory, Facets of Work, and Dimensions of Smartness to Characterize Applications and Impacts of Artificial Intelligence

Steven Alter

University of San Francisco, San Francisco, CA 94117, USA

alter@usfca.edu

**Abstract.** This paper presents an approach for describing and characterizing algorithms that are discussed as though they embody artificial intelligence. After identifying key assumptions related to algorithms and summarizing work system theory (WST), this paper uses a hypothetical example to introduce aspects of WST and two additional ideas, facets of work and dimensions of smartness in devices and systems. Next, it applies those ideas to aspects of five AI-related examples presented by entrepreneurs and researchers at an MIT AI conference in July 2020. Those examples were selected because they illustrated many AI-related issues. This paper's contribution is a new approach for characterizing real world applications and impacts of almost any system that uses algorithms or is associated with artificial intelligence.

**Keywords:** Artificial Intelligence, Algorithm, Work System Theory, Facets of Work, Dimensions of Smartness

## 1 Moving Beyond the Multiple Meanings of AI

Many discussions of AI in academia and in public venues revolve around vague definitions, cherry-picked examples, and a mélange of diverse opinions and observations from pundits and researchers whose comments are often taken out of context. Examples often fall into categories that are only tangentially related to each other: intelligent machines, neural networks, machine learning, expert systems, smart systems, cognitive computing, natural language processing, pattern recognition, image recognition, statistical algorithms, automated decision-making, and so on. Beyond various historical and technical commonalities, those topics seem less like instances of a coherent and well-defined phenomenon and more like assorted topics huddled under an umbrella called AI. The lack of clarity about what AI means makes it difficult to discuss whether the benefits, risks, and ethics of using AI differ in any significant way from the benefits, risks, and ethics of automation or computerization in general.

**Goal and Assumptions.** This paper explains how to use work system theory, facets of work, and dimensions of smartness for characterizing applications and impacts of AI. Those ideas are introduced through an illustrative hypothetical example. This paper assumes that understanding affordances, benefits, and risks of AI applications
in sociotechnical contexts can be viewed as a special case of concerns about algorithms, which was the central topic of the IFIP 8.2 working conference "Living with Monsters? Social Implications of Algorithmic Phenomena, Hybrid Agency, and the Performativity of Technology" [1]. Also, instead of speaking about algorithms in general, we look at algorithms (and hence AI) in the context of sociotechnical or totally automated work systems in which they are used.

**Organization.** After identifying key assumptions related to algorithms and summarizing work system theory (WST), this paper uses a hypothetical example to introduce aspects of WST and two additional ideas, facets of work and dimensions of smartness in devices and systems. Next, it applies those ideas to aspects of five AI-related examples presented by entrepreneurs and researchers at an MIT AI conference in July 2020. Those examples were selected because they illustrate many AI-related issues. This paper's contribution is a new approach for characterizing applications and impacts of almost any system that uses algorithms, including systems whose algorithms are associated with AI, big data, block chain, Internet of things, social media, and other current areas of interest associated with emerging technologies.

## 2 Assumptions Related to Algorithms

The examples in Table 1 use algorithms that may or may not be associated with AI. Some of those algorithms might be simple decision rules such as allowing no more than 30% of applicants to be classified in category X. Even a simple algorithm like that one can have other important and far reaching effects such as favoring one group of people over other groups, as when category X is acceptance into college. Examples in Table 1 illustrate the difficulty of generalizing about benefits, risks, and ethics of AI without specifying the area of application and the specific problem addressed.

<table><caption>**Table 1.** Potential application situations for algorithms that might or might not use AI</caption><tr><td><ul><li>using facial images to identify people</li><li>converting spoken words into equivalent text</li><li>deciding which applicants should be hired or accepted by a university</li><li>deciding whether to alert medical staff about a change in a patient's condition</li><li>deciding which is the best target for a missile</li><li>deciding a person's salary or bonus</li><li>deciding whether an autonomous (self-driving) vehicle needs to stop or swerve</li><li>controlling the aerodynamics of a rocket</li></ul></td><td><ul><li>deciding whether to turn off a machine likely to have a mechanical failure soon</li><li>deciding where police should be deployed over the next eight hours</li><li>selecting defective items that are being moved on a conveyor belt</li><li>combining multiple items in a customer order to minimize mailing cost</li><li>translating a text between languages</li><li>finding the laws that are most relevant to a specific lawsuit</li></ul></td></tr></table>

## 3 Assumptions Related to Algorithms

A series of assumptions are a starting point for visualizing and evaluating algorithms.

**Algorithms as Specifications for Transforming Inputs into Outputs.** An algorithm specifies exactly how human and/or nonhuman actors can convert specific in-
puts into specifics outputs. Algorithms are abstractions that cannot do anything on their own. Human and/or nonhuman actors perform the transformations.

**Goals, Constraints, Other Parameters.** Algorithms pursue goals, operate within constraints, and may be guided by other situation-specific parameters or inputs.

**Omissions.** Most algorithms that are not derived directly from mathematics have omissions, i.e., potentially important topics or issues that the algorithm ignores.

**Biases.** Most algorithms not derived directly from mathematics or theory bring purposeful or accidental biases. Those biases may be based on viewpoints of the algorithm's creators or on unintended results of omissions, biased training data, other shortcomings of the algorithm, or unanticipated interactions with the environment.

**Areas of Greater and Lesser Acuity:** Algorithms apply to specific domains, i.e., defined sets of things or conditions. Often they have areas of maximum relevance and acuity and other areas of limited relevance and acuity. Applying an algorithm near or just beyond the boundaries of its domain of maximum acuity may generate answers that seem sensible, but that often need to be examined and questioned carefully.

**Stakeholders:** Algorithms affect stakeholders directly or indirectly. Often different stakeholders have different or even conflicting interests.

**Embedding.** Algorithms may be embedded within other algorithms. For example, a decades-old optimization algorithm might be embedded within a situation-specific algorithm for assigning shipment orders to available trucks in a specific setting.

**Fitness for purpose.** An algorithm's fitness for purpose is determined by 1) the form, operation, and goals of work systems within which it operates, 2) its impact on human and nonhuman actors within the work system, and 3) its impact on other stakeholders such as recipients or users of whatever the work system produces.

# 4 The Work System Perspective and Work System Theory

The work system perspective (WSP) is a general approach to understanding systems in organizations based on viewing those systems as work systems. The core of WSP is work system theory (WST), which consists of three components: the definition of WS plus two frameworks (Fig. 1) for understanding a work system: the work system framework (a static view for summarizing how a work system operates) and the work system life cycle model (WSLC - how a work system evolves through planned and unplanned change). [2]. The discussion of WST had to be abbreviated due to changes in the paper's max length just before publication. The originally cited papers include [3, 4, 5, 6], which can be found on Google Scholar. [3, 4] explain how the work system method (WSM) based on WST was used in various courses, mostly were directed at employed MBA and EMBA students. Individual students or teams of students used WSM templates to produce over 700 management briefings recommending improvements to IT-reliant WS during 2003-2017, mostly in their own organizations.

**Definition of WS.** A work system is a system in which human participants and/or machines perform work (processes and activities) using information, technology, and other resources to produce specific product/services for internal and/or external customers. [2]. The first and/or addresses trends toward service-orientation and
automation of work by saying that work systems may be sociotechnical (where hu-
man participants do some or all of the work) or totally automated (where all of the
work is done by machines). A WS usually is identified based on what it is designed to
accomplish and not based on software that it uses.

ISs and projects as special cases of WS. [5, 6]. An IS may be sociotechnical (e.g., financial analysts creating economic projections with the help of modeling software) or totally automated (e.g., computers generating economic projections automatically). A project is a WS designed to produce specific product/services and then go out of existence, e.g., software development, which can be executed in many ways.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Work system framework and work system life cycle model</figcaption></figure>

5 Example Illustrating the Work System Perspective on AI

To illustrate a WSP for a situation that might involve AI, Table 2 provides a WS snapshot (a tool from WSM) of a hypothetical hiring system that PQR Corp implemented two years ago. The goal was to improve a previous hiring WS that absorbed too much effort inside PQR Corp and was so slow that candidates sometimes went elsewhere before receiving offers. Also, it hired too many unsuitable candidates.

The new hiring work system used AlgoComm and AlgoRank from a suite of software tools provided by AlgoCorp. AlgoComm provided capabilities for posting job ads, receiving applications, setting up appointments, and performing other communication with candidates. AlgoRank ranked candidates based on job criteria and a neural network application driven by AlgoCorp's database of job qualifications, salaries, and other information. Both AlgoComm and AlgoRank are algorithms that perform specified processing. AlgoRank can be seen as an AI application, whereas AlgoComm is more like typical information processing even though certain parts of it apply AI technologies such as natural language processing (NLP). After two years of use, management is once again dissatisfied. Excessive effort and delays have been reduced, but interviewers and applicants find the interface mechanical and lacking a human feel. Also, three hires proved disastrous despite use of AlgoRank capabilities. Management wants to launch a new project to upgrade the hiring work system once again.

**Interpretation Based on WST.** The WS snapshot in Table 2 summarizes the WS, which involves much more than AlgoComm and AlgoRank. The hiring WS uses AI, but should not be viewed as an AI system. The transition from the previous WS to the
current WS started with an *initiation phase* in the WSLC (Fig. 1) in which management decided to develop a new hiring system using vendor software. The *development phase* acquired resources needed for implementation. Developers selected AlgoCorp as a vendor, installed AlgoCorp's software, set a group of parameters to fit it to PQR Corp's needs, and adapted AlgoCorp's training material for PQR Corp's users. The *implementation phase* was quick because of the nature of the hiring process. The subsequent *operation and maintenance phase* continued for two years during which AlgoCorp updated the neural network component of AlgoRank automatically to reflect job market changes. Several incidents occurred where managers worked around the standard process (enacting what the WSLC calls *unanticipated adaptations*) when talented individuals became available and might have been hired by a competitor. A division VP was consulted in one case but learned about the other workaround months later. Management is looking now at a new *initiation phase* to launch a project aimed at improving the hiring system further.

<table><caption>**Table 2.** Work System Snapshot of the Current Hiring System</caption><thead><tr><th>Customers</th><th>Product/services</th></tr></thead><tbody><tr><td><ul><li>Hiring manager</li><li>Larger organization (which will have the applicant as a colleague</li><li>HR manager (who will use the applications to analyze the nature of applicants)</li></ul></td><td><ul><li>Applications (which may be used for subsequent analysis)</li><li>Job offers</li><li>Rejection letters</li><li>Hiring of the applicant</li></ul></td></tr><tr><th colspan="2">Major activities and processes</th></tr><tr><td><ul><li>AlgoComm publicizes the position.</li><li>Applicants submit resumes to AlgoComm.</li><li>AlgoRank selects shortlisted applicants and sends the list to the hiring manager.</li><li>Hiring manager decides who to interview.</li><li>AlgoComm sets up interviews.</li></ul></td><td><ul><li>Interviewers perform interviews and provide comments about applicants.</li><li>AlgoRank evaluates candidates.</li><li>Hiring manager makes hiring decision.</li><li>AlgoComm notifies applicants.</li><li>Applicant accepts or rejects job offer.</li></ul></td></tr><tr><th>Participants</th><th>Information</th><th>Technology</th></tr><tr><td><ul><li>Hiring manager</li><li>Applicants</li><li>Other employees who perform interviews</li></ul></td><td><ul><li>Job requisition</li><li>Job description</li><li>Advertisements</li><li>Job applications</li><li>Cover letters</li><li>Applicant resumes</li></ul></td><td><ul><li>Applicant short list</li><li>Information and impressions from the interviews</li><li>Job offers</li><li>Rejection letters</li></ul></td></tr></tbody></table>

**Using Facets of Work to Look More Deeply.** The situation can be observed more deeply by using an extension of WST called *facets of work*. Two recent conference papers (blinded) explain that *facets of work* grew out of research attempting to bring richer and more evocative concepts to systems analysis and design (SA&D). The notion of facet is analogous to how a cut diamond is a single thing that has multiple facets. Table 3 identifies 18 facets of work, all of which describe a unique aspect of
the activities that occur. Table 3 briefly mentions issues that the first 7 facets of work highlight related to how algorithms associated with AI might help in generating better results. Similar issues for the other 11 are not included due to length limitations but are easy to imagine. All 18 facets satisfy a series of criteria related to usefulness related to most systems in organizations: They apply to both sociotechnical and totally automated systems; they bring many concepts for analyzing system-related situations; they are associated with evaluation criteria and typical trade-offs; they have sub-facets that can be discussed; they bring open-ended questions that can help in starting conversations. Other researchers might have used a different set of facets that satisfy those criteria. Also, facets do not have to be independent, e.g. how decision-making often involves communication. The main point for current purposes is that each facet provides a lens for thinking about a work system that uses algorithms (or AI).

<table><caption>**Table 3.** Issues related to potential use of algorithms (specifically AI) in the hiring system</caption><thead><tr><td>Facet</td><td>Issues related to potential use of AI in the hiring system</td></tr></thead><tbody><tr><td>Making decisions</td><td>How could AI support decisions more fully in this system? Should AI suggest decisions or make decisions?</td></tr><tr><td>Communicating</td><td>How can AI explain how it makes or suggests decisions? How can AI help work system participants communicate more effectively?</td></tr><tr><td>Processing information</td><td>Can AI play any special role in capturing, transmitting, storing, retrieving, deleting, manipulating, or displaying information?</td></tr><tr><td>Thinking</td><td>Are there any areas in which it would be beneficial for AI to replace or augment thinking done by work system participants?</td></tr><tr><td>Representing reality</td><td>Does AI represent reality in a biased way? For example, what about possible bias or omissions in the dataset used to train the neural network?</td></tr><tr><td>Providing information</td><td>Could AI provide more meaningful information to work system participants than would otherwise be available?</td></tr><tr><td>Applying knowledge</td><td>Could AI identify and provide specific knowledge that would help in evaluating applicants?</td></tr><tr><td>Similar questions for 11 other facets of work</td><td>Length limitations prevent listing similar questions for Planning, Control,ling execution, Improvising, Coordinating, Performing physical work, Performing support work, Interacting socially, Providing service, Creating value, Co-creating value, and Maintaining security.</td></tr></tbody></table>

**Using dimensions of smartness to look more deeply at AI applications.** Existing AI applications can be viewed as “weak AI” because they address highly constrained problems such as those in Table 1 and in the hiring example. That approach to AI has led to important breakthroughs and efficiencies in many situations, but is nothing like science fiction dreams of “strong AI” exhibited by humanoid robots that can reason and interact at a human or superhuman level [7]. This paper describes current AI capabilities using a set of dimensions of smartness in devices and systems that diverges from views of smartness in most current papers related to smartness involving things, devices, systems, cities, and so on (e.g., [8, 9, 10, 11, 12, 13, 14]).

This paper’s classification matrix for smart capabilities is organized around four categories: information processing, internal regulation, action in the world, and knowledge acquisition [8]. Each category identifies separate capabilities, in essence
separate dimensions on a continuum from not smart to somewhat smart to extremely smart based on a complex definition of smart: “Purposefully designed entity X is smart to the extent to which it performs and controls functions that attempt to produce useful results by applying automated capabilities and other physical, informational, technical, and intellectual resources for processing information, interpreting information, and/or learning from information that may or may not be specified by its designers.” Table 4 identifies 23 dimensions of smartness, each associated with one of four categories. Every dimension in Table 4 is a continuous variable extending from not smart at all to increasing levels of smartness including scripted execution, formulaic adaptation, creative adaptation, and unscripted or partially scripted invention. [8]. Very few existing systems are even close to the higher levels of smartness.

<table><caption>**Table 4.** Dimensions of smartness related to four categories of smartness [8]</caption><thead><tr><td>Category of smartness</td><td>Dimensions of smartness</td></tr></thead><tbody><tr><td>Information processing</td><td>capture, transmit, store, retrieve, delete, manipulate, display information</td></tr><tr><td>Internal regulation</td><td>self-detection, self-monitoring, self-diagnosis, self-correction, self-organization</td></tr><tr><td>Action in the world</td><td>sensing, actuation, coordination, communication, control, physical action</td></tr><tr><td>Knowledge acquisition</td><td>sensing or discovering, classifying, compiling, inferring or extrapolating from example, inferring or extrapolating from abstractions, testing and evaluating</td></tr></tbody></table>

Categories and dimensions in Table 4 can be used to see that the hypothetical hiring example is not very smart even though it uses AI. AlgoComm processes information by using mechanical, pre-specified capabilities when it captures, transmits, stores, retrieves, deletes, manipulates, and displays information. The neural network that provides AlgoRank's parameters for ranking candidates performs a type of knowledge acquisition (classifying and compiling) using techniques that are best described as scripted execution. Neither AlgoComm nor AlgoRank demonstrate internal regulation or action in the world.

The hypothetical hiring system example was designed to illustrate the relevance of WST, facets of work, and dimensions of smartness in describing AI applications. The next section applies those ideas to five real world AI applications mentioned at the “MIT AI Conference 2020: AI for a Better World” presented as a series of webinars by the MIT Club of Northern California during July 14-18, 2020.

## 6 Application of WST, Facets of Work and Dimensions of Smartness to Real Examples

The five real world AI applications discussed here were chosen because they illustrate current or potential integration into operational work systems. They are identified as A1, A2, etc., and will be named based on their purpose: (A1) performing fetal
screening for heart defects, (A2) finding defects in electronics manufacturing, (A3) supporting personalized learning in coursework, (A4) receiving and responding to IT help desk requests, (A5) producing useful notes from meetings. Three were discussed in 25-minute presentations that covered a variety of business, personal, and AI topics; one was presented in a research slam; one was discussed in an interview covering many topics. All involved digitalization applications including at least one algorithm associated with AI. Since the presenters were not available for in-depth interviews, this paper's descriptions are interpretations of webinars viewed on youtube.com. That suffices for the current goal of demonstrating the relevance of WST, facets of work, and dimensions of smartness for characterizing real world AI applications.

Each of the following discussions touches on the nature of the WS (Fig. 1, Table 2) that is being supported, the goal of the AI application within that WS, an overview of how the AI application was developed (WSLC in Figure 1) how it was or can be implemented as part of a WS, the main facets of work (Table 3) that are supported or automated, and aspects of relevant dimensions of smartness (Table 4).

**(A1) Performing fetal screening for heart defects.** [15] Congenital heart disease (CHD) occurs in 1% of live births. Fetal ultrasound screening at 20 weeks of gestation should detect over 90% of CHD but is often less than 50% accurate because it is difficult to build and maintain skills for a rare condition and because of difficulties of ultrasound imaging. Researchers created A1 as an ensemble of neural networks to detect CHD. They used clinical guidelines (medical knowledge) to identify five key screening views of the heart instead of looking at thousands of images for each ultrasound. This allowed them to perform an analysis based on a training dataset of 100,000 images from 1,300 ultrasounds. Later, they achieved 95% sensitivity and 96% specificity, far better than levels in current practice, in a test on a much large dataset. Their first step was training a convolutional neural network to distinguish the five screening views. Diagnostic classifiers determined whether a heart was normal for each view. Combining those classifiers gave a composite indicator of whether a heart was normal. Thus, deep learning combined with clinical knowledge and expert annotation of cases resulted in a possibly important way to improve practice.

A1 might be used in future practice during exams as a physician tries to decide whether a 20-week fetus has CHD. In terms of facets of work, A1 will perform extensive processing of information to represent reality with a diagnostic score that provides information to physicians that will help them in making decisions about leading to treatments. In terms of dimensions of smartness, A1 processes information in a prescribed manner to create useful diagnostic scores. A1 does not exercise internal regulation or take action in the world. A1 does not create knowledge, which was created previously by the researchers using their neural network approach.

**(A2) Finding defects in electronics manufacturing.** [16] Inefficiencies in electronics manufacturing waste 20%-30% of expenses through scrap and rework, product returns, mistakes and experiments, and underutilized human resources. Methods for monitoring problems include automated optical inspection, functional testing, daily build reports, failure analysis reports, and analysis of customer returns. Underlying
issues often are dark yield problems, i.e., defects that cannot be found through a test of function but that may cause a unit to fail later – incorrect cable routing, cold solder failures, glue overflow, connectors not fully mated, and misassembled parts such as a screw that is not fully inserted. With the COVID pandemic, engineers are prevented from going to remote factories. Merely taking pictures of work in progress or completed units in the factory is insufficient because solving problems requires tracking specific units back to specific production steps where their problems occurred. Trying to record a complete history of production units including photos during production generates a great deal of data that might have to be transmitted to the cloud from factories in remote locations where data transmission capabilities are limited. A2 compares production units to other production units at a specific point in assembly. It can start with as few as 30 initial units before production stabilizes. It identifies anomalies such as tape and label defects, missing foams that keep components in place, missing functional parts, incorrect cable routing, and glue issues. A2 reduces the amount of labeling of defects that people need to do by sorting the images in order from totally conforming to highly nonconforming, thereby helping with decisions on cutoffs for labeling defects as consequential or not. Users can identify areas where problems are likely to occur, but that is not necessary. Thus, A2's algorithms can be trained without examples that are labeled in advance as defects. All of the training is done using software in the cloud, not on premises.

A2's work system is manufacturing of electronic items such as phones. Its algorithms are used for quick identification of defective units, even with dark yield problems, before additional defective units are manufactured. The training uses images of important parts (e.g., the front of a phone) and identifying anomalies that differentiate one unit from others. A person decides whether a unit's anomaly is serious enough to declare the unit defective. After training, algorithms can be used to monitor production to find defective units. The relevant facets of work touched directly by A2 are *processing information* (capturing images and identifying anomalies between units), *providing information* about anomalies, and *making decisions* by identifying defective units after A2 has been trained. In terms of smartness, A2 *processes information* using pre-defined scripts. It does not *perform self-regulation* or *take action in the world*. It acquires knowledge through the training process. The presentation implied that training on specific problem areas can be repeated if the produce design changes.

**(A3) Supporting personalized learning in coursework.** [17, 18] This research involved working with students using coursework available through the Khan Academy, which provided an anonymized dataset of 50K elementary through high school students solving 1.4 million math problems. The dataset included a history of every problem that each student tried to solve, whether the answer was correct, and how long the student worked on the problem. The researchers trained a neural network to take as input the complete history of a student's (correct and incorrect) answers and to try to predict their answer to the next question. This created a complex vector for any student that predicts whether that student would solve any math problem that might appear next. In aggregate across all students, the neural network learned which skills are needed to answer any question. It represented the pedagogical structure of the
mathematics students were trying to learn. That knowledge provides hints about what other problems students will be able to solve after they acquire a specific skill. In effect this knowledge graph describes in a data-driven way the prerequisite structure in learning mathematics and therefore provides a data-driven window into the learning process. A3 was developed in research reported in [18]. Real world applications are easy to imagine although the closest the webinar discussion [17] came to discussing actual applications was a few comments about a MOOC that taught coding. The very large set of student exercises and related comments by teaching assistants might be a step toward automation of some aspects of grading of coding exercises.

In a real-world application of something like A3, the work system would be stu-
dents trying to learn specific coursework. A3 or something like it would hasten learn-
ing by predicting immediate difficulties students might experience and by looking
ahead to provide an optimal learning sequence. In terms of facets of work, the learn-
ing management system would *process information* by storing and retrieving the stu-
dent's history. It would use a student's history and a course-related neural network to
*decide* what problems the student should see next. It would *represent reality* as the
student's progress to date. A learning management system would *communicate* with
the student through online interactions. A3 would *control* the learning process to max-
imize learning. In terms of smartness, all of the *information processing* would be done
based on scripts that use the current state of the recurrent neural network. A3 would
perform *internal regulation* and *action in the world* in the sense of using each stu-
dent's history and the structure of the subject matter to decide what the student should
see next. It would *acquire knowledge* by applying pre-specified neural network tech-
niques to deepen its own knowledge as students answers problems.

(A4) **Receiving and responding to IT help desk requests.** [19] The firm Move-
works provides a chatbot for handling IT help requests for firms. The average time
before an agent looks at an IT help request averages 5 hours and a response often
takes 3 days. This problem cannot be solved with big data approaches that work in the
consumer space (e.g. billions of items from webpages, documents, etc) because the IT
help desk of a firm with 1000 employees might have 100 laptop requests and 10 VPN
connection requests every year, not enough to serve as a training dataset for deep
learning related to IT requests in that firm. As a result, chatbots often rely on hard
coded logic that leads to frustrating endless loops for users. One of the problems with
IT requests is that the requests are often ambiguous, e.g., "how do I get my laptop
running?" The A4 approach was to build a conversational AI system that uses ma-
chine learning with "small data." The trick in teaching the neural network was to ab-
stract from sentence data by labeling recurrent elements, i.e., using labels that de-
scribe categories rather than instances (e.g., PC rather than Dell vs. HP vs. Lenovo).
Converting sentences such as "I need Trello capability" or "Joe needs a Windows
password" into a more general form like "$PERSON wants $SOFTWARE access"
was a starting point for generating a large number of possible sentences that can be
linked to actual help desk requests. A4 uses "collective learning" by applying the
same learning approach across many firms that have IT help desks. It also used
"transfer learning" by extracting universal language patterns (e.g., that good answers
often have instructions in the form of lists) that can be applied across domains. Initially they used stackoverflow.com, a website for software developers that contains millions of help requests and related answers. The ultimate result is a chatbot that can completely answer around 40% of help requests and can escalate the others to human operators. As a result, the human operators handle many fewer IT tickets, a great saving in the use of a scarce resource.

A4 is part of a WS that answers IT help requests. Facets of work include *making decisions, communicating, processing information, providing information, and representing reality*. In terms of smartness, every type of *information processing* is present in a scripted form. A4 performs *internal regulation* by recognizing the current state of its dialogue with a user and trying to respond appropriately. It *takes action in the world* by engaging in a dialogue with users. It *acquires knowledge* from its usage.

**(A5) Producing useful notes from meetings.** [20] Fireflies.ai is a commercial product that records meetings and generates transcripts automatically. A5 is the basis of “Fred,” an automated voice assistant that records meetings, produces transcripts, and performs other tasks to make meetings and their aftermath more efficient. A user invites fred@fireflies.ai to an online meeting on Zoom, WebEx, or other platforms. Fred captures and transcribes voice conversations, indexes the notes to make them useful, and routes the notes to anyone who should receive them. Action items can be transferred automatically to project management systems such as Trello or to customer relationship management systems such as Salesforce without doing a lot of manual work. Maintaining a complete history of meetings makes it possible to find details of meetings that may have happened months ago.

In effect, A5 is the technical basis of an automated work system that is created through three main steps, two of which are in the WSLC *development phase*. The first step is collecting relevant language data (sentences, keywords, etc.) from users and public sources, storing the data, and labeling the data to make it useful. For example, the founders of Fireflies.ai developed some of their ideas by labeling 20K data points (basically sentences) from their own meeting notes and recordings by using a yes/no binary classification model (important or unimportant). The initial trial use of the resulting model led to more language data that could be incorporated. The second main step was to get the model running using available software that fit the model and making sure that errors and duplicates in the data (terms and sample sentences) were eliminated. The third step was benchmarking and performance improvement (e.g., minimizing false positives and false negatives) as part of an *operation and maintenance phase*. Subsequent use in different industries is accommodated by having users pick a domain such as health care or sales when they sign up. Each industry has keywords that appear on user dashboards. Users add other keywords. Meeting agendas and user edits to transcripts provide more industry-specific jargon.

The relevant facets of work *processing information* (recording and transcribing meetings), *providing information* in the form of transcripts. In terms of smartness, A2 *processes information* using pre-defined scripts. It does not *perform self-regulation*. It *takes action in the world* by producing and distributing transcripts. It acquires knowledge by improving its language models every time it is used.
# 7 Conclusion

This paper's contribution is way to describe and characterize AI applications and their impacts instead of just talking about AI in general, often based on cherry-picked examples. This paper used a hypothetical hiring example to illustrate the ideas and then applied those ideas to five real examples. The five examples illustrate a number of points that are not evident from many attempts to talk about AI in general.

**Integration with work systems.** (Figure 1, Table 2). In all five cases, the AI algorithm was part of an actual work system (A2, A4, A5) or was developed as research with a high potential for application in work systems (A1, A3).

**Facets of work.** (Table 3) All of the examples *process information, provide information, and represent reality*. All *create value*. Algorithms in A1, A2, and A3 contribute to *making decisions* that matter. A4 and A5 *perform support work*. A2 helps in *controlling execution* by identifying anomalies. The chatbot in A4 *communicates* with customers of the help desk. A1 and A2 *communicate* in a more structured way.

**Smartness** (Table 4) All five examples *process information* in a scripted way. A4 exhibits a form of *internal regulation*. A1 and A2 identify problems but do not *take action in the world*. A4, and A5 *take action in the world*, and A3 has a potential to do so. All use knowledge built into neural networks. A3, A4, and A5 *acquire knowledge*. Relevant dimensions of smartness in all cases are handled through scripted execution of algorithms rather than by autonomous modification of algorithms.

**Importance of domain knowledge.** The neural networks in A1, A2, A4, and A5 all depend on domain knowledge built into their design. A3 is more like unsupervised learning based on histories of students answering problems. Unsupervised learning seems most relevant where situational knowledge is not necessary, such in Open AI's GPT-3 system, whose 1.5 billion “parameters” describe the likelihood that specific words occur next to other words in a vast training dataset of texts. [21]

**Big data or little data.** Some AI algorithms such as GPT-3 are built on huge data bases, but A2 could start being useful after training with only 30 examples. A1, A4, and A5 also used knowledge as a way to reduce the size of training datasets.

**Visibility to users.** General discussions of AI often mention the lack of visibility to users. A1 is based on ultrasound data that is understood by highly skilled users. A2 finds anomalies that are visible. A3 is hidden within a learning management system. A4 and A5 perform support work where most errors are easily identified.

**Is AI inherently ethically suspect?** General discussions of AI frequently focus on harm that may occur when algorithms are used to identify people or suggest important decisions related to specific individuals. Biased decisions may result from biased training datasets and/or biased logic of the work system. On the other hand, it is also obvious that almost any technology can contribute to work systems that harm people. The issue is not with AI as a category, but rather with work systems and/or algorithms that fail to provide equitable treatment for all customers and other stakeholders.
References

1. Schultze, U., et al.: Living with Monsters? Social Implications of Algorithmic Phenomena, Hybrid Agency, and the Performativity of Technology. Proceedings of IFIP WG 8.2 Working Conference on the Interaction of Information Systems and the Organization. December 11–12. San Francisco, CA, USA. (2018).

2. Alter, S.: Work system theory: overview of core concepts, extensions, and challenges for the future. Journal of the Association for Information Systems, 14(2), pp. 72–121. (2013).

3. Truex, D., Alter, S., Long, C.: Systems analysis for everyone else: Empowering business professionals through a systems analysis method that fits their needs. ECIS (2010).

4. Truex, D., et al.: Extending a systems analysis method for business professionals. In European Design Science Symposium (pp. 15-26). Springer, Berlin, Heidelberg. (2011).

5. Alter, S.: The work system method: Connecting people, processes, and IT for business results. Work System Press, Larkspur, USA (2006).

6. Alter, S.: Defining information systems as work systems: implications for the IS field, European Journal of Information Systems, (17) pp. 448-469, (2008).

7. Iansiti, M., Lakhani, K.R.: Competing in the Age of AI: Strategy and Leadership When Algorithms and Networks Run the World. HBS Press, Cambridge, USA, (2020).

8. Alter, S.: Making Sense of Smartness in the Context of Smart Devices and Smart Systems, Inf. Syst. Frontiers, 22(2), pp.275-280

9. Strozzi, F., et al.: Literature review on the ‘Smart Factory’ concept using bibliometric tools. International Journal of Production Research, 55(22), pp. 6572-6591 (2017).

10. Romero, M., et al.: Towards a characterisation of smart systems: A systematic literature review. Computers in Industry, 120, in press, (2020).

11. Elbanna, A.R., et al.: The Search for Smartness in Working, Living and Organising: Beyond the 'Technomagic'. Inf. Syst. Frontiers, 22(2), pp.275-280. (2020)

12. Langley, D.J., et al.: The Internet of Everything: Smart things and their impact on business models. Journal of Business Research, in press. (2020).

13. Wolf, V., et al.: Establishing Smart Service Systems is a Challenge: A Case Study on Pitfalls and Implications. Wirtschaftsinformatik 2020 Zentrale Tracks. (2020).

14. Anke, J., et al.: Joining Forces: Understanding Organizational Roles in Inter-organizational Smart Service Systems Engineering. Wirtschaftsinformatik 2020 Zentrale Tracks. (2020).

15. Arnaout, R.: Improving fetal heart screening with deep learning, https://www.youtube.com/watch?v=76WvsHhCyko, (2020).

16. Shedietsky, A-K.: How COVID-19 is driving adoption of AI in electronics manufacturing. https://www.youtube.com/watch?v=DIuuVTJKoLs, (2020).

17. Ganguli, S.: Learning in Humans and Machines: From Optimizing Our Biology to Colluding With AI's, https://www.youtube.com/watch?v=OL89f0mnJ00, (2020).

18. Piech, C., et al.: Deep knowledge tracing. In Advances in neural information processing systems (pp. 505-513). (2015).

19. Shah, B.: How Conversational AI Will Empower One Billion Knowledge Workers, https://www.youtube.com/watch?v=bzBZA_2kkNE, (2020).

20. Udotong, S.: Practical Deep Learning: How It Is Used in Production and Already Revolutionizing Enterprise Work. https://www.youtube.com/watch?v=ywhY2DCIZqU (2020).

21. Manjoo, F.: How Do You Know a Human Wrote This? New York Times, Editorial, July 29, 2020. https://www.nytimes.com/2020/07/29/opinion/gpt-3-ai-automation.html